Package com.banuba.sdk.effect_player

Class EffectPlayer.CppProxy

    • Method Detail

      • _djinni_private_destroy

        public void _djinni_private_destroy()

      • finalize

        protected void finalize()
                 throws java.lang.Throwable
        Overrides:
        finalize in class java.lang.Object
        Throws:
        java.lang.Throwable

      • addFrameDataListener

        public void addFrameDataListener​(FrameDataListener frameDataListener)
        Description copied from interface: EffectPlayer
        Add callback to receive frame data right after processing in recognizer. Thread-safe. May be called from any thread
        Specified by:
        addFrameDataListener in interface EffectPlayer

      • removeFrameDataListener

        public void removeFrameDataListener​(FrameDataListener frameDataListener)
        Description copied from interface: EffectPlayer
        Remove callback to receive frame data right after processing in recognizer. Thread-safe. May be called from any thread
        Specified by:
        removeFrameDataListener in interface EffectPlayer

      • addCameraPoiListener

        public void addCameraPoiListener​(CameraPoiListener cameraPoiListener)
        Description copied from interface: EffectPlayer
        Add callback to receive center of the face in frame relative to top-left corner in [0, 1] space. Thread-safe. May be called from any thread
        Specified by:
        addCameraPoiListener in interface EffectPlayer

      • removeCameraPoiListener

        public void removeCameraPoiListener​(CameraPoiListener cameraPoiListener)
        Description copied from interface: EffectPlayer
        Remove callback to receive center of the face in frame relative to top-left Thread-safe. May be called from any thread
        Specified by:
        removeCameraPoiListener in interface EffectPlayer

      • addEffectInfoListener

        public void addEffectInfoListener​(EffectInfoListener effectInfoListener)
        Description copied from interface: EffectPlayer
        Add callback to receive current effect info from Effect Player. Thread-safe. May be called from any thread
        Specified by:
        addEffectInfoListener in interface EffectPlayer

      • removeEffectInfoListener

        public void removeEffectInfoListener​(EffectInfoListener effectInfoListener)
        Description copied from interface: EffectPlayer
        Remove callback to receive current effect info from Effect Player. Thread-safe. May be called from any thread
        Specified by:
        removeEffectInfoListener in interface EffectPlayer

      • setMaxFaces

        public void setMaxFaces​(int maxFaces)
        Description copied from interface: EffectPlayer
        Sets maximum allowed face results, if face tracking feature is present. Thread-safe. May be called from any thread This option is relevant only for GLFX effects.
        Specified by:
        setMaxFaces in interface EffectPlayer

      • surfaceCreated

        public void surfaceCreated​(int width,
                           int height)
        Description copied from interface: EffectPlayer
        Use to notify the EffectPlayer that the surface exists and effect can be played. 0, 0 for width and height are valid values. MUST be called from the render thread
        Specified by:
        surfaceCreated in interface EffectPlayer

      • surfaceChanged

        public void surfaceChanged​(int width,
                           int height)
        Description copied from interface: EffectPlayer
        Notify about rendering surface being resized. MUST be called from the render thread
        Specified by:
        surfaceChanged in interface EffectPlayer

      • surfaceDestroyed

        public void surfaceDestroyed()
        Description copied from interface: EffectPlayer
        This method should be called right before an active context will become invalid. Switches playback state to inactive state. If it's not done an application will be crashed on next draw iteration. After losing the surface effect playback can't be resumed from last position. MUST be called from the render thread
        Specified by:
        surfaceDestroyed in interface EffectPlayer

      • draw

        public long draw()
        Description copied from interface: EffectPlayer
        Draw the current effect into the current OpenGL framebuffer. Uses internal frame_data object obtained from latest push_frame recognition result. Return current frame number if drawing was performed and caller should swap buffers otherwise `DRAW_SKIPPED`(-1) MUST be called from the render thread
        Specified by:
        draw in interface EffectPlayer

      • drawWithExternalFrameData

        public long drawWithExternalFrameData​(FrameData frameData)
        Description copied from interface: EffectPlayer
        Draw the current effect into the current OpenGL framebuffer. Uses externally provided frameData object instead of internal one obtained from latest push_frame recognition result. Return frame number from provided frameData if drawing was performed and caller should swap buffers otherwise `DRAW_SKIPPED`(-1) MUST be called from the render thread
        Specified by:
        drawWithExternalFrameData in interface EffectPlayer

      • captureBlit

        public void captureBlit​(int captureWidth,
                        int captureHeight)
        Description copied from interface: EffectPlayer
        Record last rendering ("draw_()") result into current OpenGL framebuffer. Content is cropped to maintain effect_size() aspect ratio MUST be called from the render thread
        Specified by:
        captureBlit in interface EffectPlayer

      • readPixels

        public PixelBuffer readPixels()
        Description copied from interface: EffectPlayer
        Receive the last rendered frame in binary form.
        Specified by:
        readPixels in interface EffectPlayer
        Returns:
        4-byte per pixel data, size is `fx_width * fx_height * 4` MUST be called from the render thread

      • processImage

        public Data processImage​(FullImageData inputImage,
                         PixelFormat outputPixelFormat)
        Description copied from interface: EffectPlayer
        Process an image with current effect. Must be called from the render thread.
        Specified by:
        processImage in interface EffectPlayer
        Parameters:
        inputImage - to avoid conversion recommended to use YUV image
        outputPixelFormat - to avoid conversion recommended to use RGBA

      • processImageFrameData

        public Data processImageFrameData​(FrameData inputFrameData,
                                  PixelFormat outputPixelFormat)
        Description copied from interface: EffectPlayer
        Process an image with current effect. Prefer this method over `processImage` when you have extra input data besides just an input image. Must be called from render thread.
        Specified by:
        processImageFrameData in interface EffectPlayer
        Parameters:
        inputFrameData - `FrameData` with an image
        outputPixelFormat - to avoid conversion recommended to use RGBA

      • processImageData

        public byte[] processImageData​(byte[] inputImage,
                               int width,
                               int height,
                               CameraOrientation orientation,
                               boolean isMirrored,
                               PixelFormat inputPixelFormat,
                               PixelFormat outputPixelFormat)
        Description copied from interface: EffectPlayer
        Process image with current effect. Must be called from render thread. NOTE: inputImage have copy overhead, preferable to use process_image method MUST be called from the render thread
        Specified by:
        processImageData in interface EffectPlayer
        Parameters:
        inputImage - input image with `width * height * 4` size
        outputPixelFormat - to avoid conversion recommended to use RGBA

      • pushFrame

        public void pushFrame​(FullImageData fullImage)
        Description copied from interface: EffectPlayer
        Provides image to process and to play effect. Thread-safe. May be called from any thread
        Specified by:
        pushFrame in interface EffectPlayer

      • pushFrameWithNumber

        public void pushFrameWithNumber​(FullImageData fullImage,
                                long frameNumber)
        Description copied from interface: EffectPlayer
        Provides image to process and to play effect. Thread-safe. May be called from any thread
        Specified by:
        pushFrameWithNumber in interface EffectPlayer

      • pushFrameData

        public void pushFrameData​(FrameData frameData)
        Description copied from interface: EffectPlayer
        Provides FrameData to process and to play effect. Must contain full image. Thread-safe. May be called from any thread
        Specified by:
        pushFrameData in interface EffectPlayer

      • pushFrameDataWithNumber

        public void pushFrameDataWithNumber​(FrameData frameData,
                                    long frameNumber)
        Description copied from interface: EffectPlayer
        Provides FrameData with frame number to process and to play effect. Must contain full image. Thread-safe. May be called from any thread
        Specified by:
        pushFrameDataWithNumber in interface EffectPlayer

      • playbackPlay

        public void playbackPlay()
        Description copied from interface: EffectPlayer
        MUST be called from the main(render) thread
        Specified by:
        playbackPlay in interface EffectPlayer

      • getInputManager

        public InputManager getInputManager()
        Description copied from interface: EffectPlayer
        Get interface to control user iterations. This events will be passed to effect. Thread-safe. May be called from any thread
        Specified by:
        getInputManager in interface EffectPlayer

      • startVideoProcessing

        public void startVideoProcessing​(long screenWidth,
                                 long screenHeight,
                                 CameraOrientation orientation,
                                 boolean resetEffect,
                                 boolean offlineMode)
        Description copied from interface: EffectPlayer
        Initialize video processing. To provide frames use *process_video_frame* methods. Effect audio is recorded as well and can be accessed using *process_recorded_audio* and *write_recorded_audio*. *push_frame* should not be called during processing. MUST be called from the render thread
        Specified by:
        startVideoProcessing in interface EffectPlayer

      • stopVideoProcessing

        public void stopVideoProcessing​(boolean resetEffect)
        Description copied from interface: EffectPlayer
        Finish processing and return renderer to normal state. MUST be called from the render thread
        Specified by:
        stopVideoProcessing in interface EffectPlayer

      • processVideoFrameData

        public FrameData processVideoFrameData​(FrameData inputFrameData,
                                       java.lang.Integer recognizerIterations)
        Description copied from interface: EffectPlayer
        Provide frame data to evaluate video processing. MUST be called from the render thread
        Specified by:
        processVideoFrameData in interface EffectPlayer
        Parameters:
        inputFrameData - Frame Data to process.
        recognizerIterations - Number of processing iterations. Higher number means higher processing quality, but lower speed. Must be greater than 1. Pass null value for default number.
        Returns:
        Frame data with frame processing results.

      • processVideoFrame

        public FrameData processVideoFrame​(FullImageData inputImage,
                                   java.lang.Integer recognizerIterations)
        Description copied from interface: EffectPlayer
        Provide frame to evaluate video processing. MUST be called from the render thread
        Specified by:
        processVideoFrame in interface EffectPlayer
        Parameters:
        inputImage - Image to process.
        recognizerIterations - Number of processing iterations. Higher number means higher processing quality, but lower speed. Must be greater than 1. Pass null value for default number.
        Returns:
        Frame data with frame processing results.

      • drawVideoFrame

        public Data drawVideoFrame​(FrameData inputFrameData,
                           long timeNs,
                           PixelFormat outputPixelFormat)
        Description copied from interface: EffectPlayer
        Draw processed frame to image buffer. MUST be called from the render thread
        Specified by:
        drawVideoFrame in interface EffectPlayer
        Parameters:
        inputFrameData - FrameData to draw.
        timeNs - Frame position on timeline.
        outputPixelFormat - Output image format.
        Returns:
        Buffer with processed image in selected format.

      • drawVideoFrameAllocated

        public Data drawVideoFrameAllocated​(FrameData inputFrameData,
                                    long timeNs,
                                    PixelFormat outputPixelFormat,
                                    Data allocatedBuffer)
        Description copied from interface: EffectPlayer
        The same as *draw_video_frame* but accepts preallocated buffer to avoid recreation for every frame. MUST be called from the render thread
        Specified by:
        drawVideoFrameAllocated in interface EffectPlayer

      • writeRecordedAudio

        public void writeRecordedAudio​(java.lang.String filename,
                               long lengthMs)
        Description copied from interface: EffectPlayer
        Save recorded audio. Not thread-safe but can be called from any thread.
        Specified by:
        writeRecordedAudio in interface EffectPlayer
        Parameters:
        filename - ".wav" extension must be used.
        lengthMs - Track length. Set to '0' to use full duration.

      • onVideoRecordStart

        public void onVideoRecordStart​(boolean playAudioWhileRecording)
        Description copied from interface: EffectPlayer
        Trigger record start events in EP and Effect. Records all sounds during recording to be replayed by `processRecordedAudio`. If @param playAudioWhileRecording is true, the audio will continue to play. Otherwise, the audio will be muted. Thread-safe. May be called from any thread
        Specified by:
        onVideoRecordStart in interface EffectPlayer

      • onVideoRecordEnd

        public void onVideoRecordEnd()
        Description copied from interface: EffectPlayer
        Triggers record stop events in EP and Effect. Thread-safe. May be called from any thread
        Specified by:
        onVideoRecordEnd in interface EffectPlayer

      • setRecognizerOfflineMode

        public void setRecognizerOfflineMode​(boolean on)
        Description copied from interface: EffectPlayer
        Force recognizer offline mode Thread-safe. May be called from any thread
        Specified by:
        setRecognizerOfflineMode in interface EffectPlayer

      • setRecognizerUseFutureFilter

        public void setRecognizerUseFutureFilter​(boolean on)
        Description copied from interface: EffectPlayer
        Set future frame filtrarion mode. Produce smoother recognition result (anti jitter), however adds inconsistency in push'ed/draw'ed frames (one frame lag) Example: push frame 1 - draw frame 1, push frame 2 - draw frame 1, push frame 3 - draw frame 2, ...
        Specified by:
        setRecognizerUseFutureFilter in interface EffectPlayer

      • setRecognizerUseFutureInterpolate

        public void setRecognizerUseFutureInterpolate​(boolean on)
        Description copied from interface: EffectPlayer
        Set future frame interpolation mode. Produce faster recognition result (skip even frames), however adds inconsistency in push'ed/pop'ed frames (one frame lag) Example: push frame 1 - pop frame 1, push frame 2 - pop frame 1, push frame 3 - pop frame 2, ...
        Specified by:
        setRecognizerUseFutureInterpolate in interface EffectPlayer

      • setRenderBackend

        public static void setRenderBackend​(@NonNull
                                    RenderBackendType backendType)

      • getCurrentRenderBackendType

        @NonNull
public static RenderBackendType getCurrentRenderBackendType()