@virtualscenery/greenscreenstream
    Preparing search index...

    Class GreenScreenStream

    The GreenScreenStream class provides a virtual background solution for video streams, supporting both chroma key (green screen) and machine learning-based background segmentation. It manages video sources, background images or videos, and rendering via WebGL, and can output the processed stream as a MediaStream suitable for use in web applications.

    Features:

    • Supports chroma key (green screen) and ML-based segmentation (BodyPix).
    • Allows dynamic background replacement with images or videos.
    • Provides methods to start/stop rendering, add video tracks, and capture the output stream.
    • Offers color analysis utilities (dominant color, palette extraction).
    • Configurable mask, chroma key, and rendering settings.
    • Integrates with WebGL for efficient real-time compositing.

    Usage:

    1. Instantiate with the desired green screen method and resolution.
    2. Call initialize() with a background source and optional configuration.
    3. Add a video track (e.g., from a webcam) using addVideoTrack().
    4. Start rendering with start().
    5. Capture the processed stream via captureStream().
    6. Stop rendering with stop().
    const gss = new GreenScreenStream(GreenScreenMethod.VirtualBackground, { x: 1280, y: 720 });
    await gss.initialize('background.jpg');
    await gss.addVideoTrack(webcamTrack);
    gss.start();
    const outputStream = gss.captureStream();
    Index

    Constructors

    constructor

    Properties

    backgroundColor backgroundSource bodyPix bufferFrag bufferVert canvas canvasEl? ctx demolished flipHorizontal foregroundColor frame greenScreenMethod isRendering mainFrag mainVert maskBlurAmount maxFps mediaStream modelLoaded opacity rafId resolution startTime

    Methods

    addVideoTrack captureStream dominant flipStreamHorizontal getColorsFromStream initialize pallette scaleImageToCanvas setBackground setBodyPixModel setChromaKey setMaskRange start stop

    Constructors

    • Constructs a new instance of the green screen stream handler.

      Parameters

      • greenScreenMethod: GreenScreenMethod

        The method to use for green screen processing.

      • resolution: VideoResolution | Vector2

        The desired video resolution or a vector specifying width and height.

      • OptionalcanvasEl: HTMLCanvasElement

        (Optional) An existing HTMLCanvasElement to use for rendering. If not provided, a new canvas will be created.

        Initializes the media stream and canvas, sets the canvas resolution, and determines whether to use machine learning-based background removal based on the selected green screen method.

      Returns GreenScreenStream

    Properties

    backgroundColor: RGBA
    backgroundSource: HTMLImageElement | HTMLVideoElement
    bodyPix: BodyPix
    bufferFrag: string = BUFFER_FRAG
    bufferVert: string = BUFFER_VERT
    canvas: HTMLCanvasElement
    canvasEl?: HTMLCanvasElement

    (Optional) An existing HTMLCanvasElement to use for rendering. If not provided, a new canvas will be created.

    Initializes the media stream and canvas, sets the canvas resolution, and determines whether to use machine learning-based background removal based on the selected green screen method.

    ctx: WebGLRenderingContext | WebGL2RenderingContext
    demolished: DemolishedRenderer
    flipHorizontal: boolean
    foregroundColor: RGBA
    frame: number = -1
    greenScreenMethod: GreenScreenMethod

    The method to use for green screen processing.

    isRendering: boolean
    mainFrag: string = MAIN_FRAG
    mainVert: string = MAIN_VERT
    maskBlurAmount: number
    maxFps: number
    mediaStream: MediaStream
    modelLoaded: boolean
    opacity: number
    rafId: number
    resolution: Vector2
    startTime: number = null

    Methods

    • Adds a video track to the media stream and sets up the source video element.

      Parameters

      • track: MediaStreamTrack

        The video track to add.

      Returns Promise<any>

      A promise that resolves when the track is added and the source video is ready.

      GreenScreenStream

    • Captures the current state of the canvas as a MediaStream. Optionally, you can specify the frames per second (fps) for the captured stream.

      Parameters

      • Optionalfps: number

        The frames per second for the captured stream (default is 25).

      Returns MediaStream

      The captured MediaStream from the canvas.

      GreenScreenStream

    • Get the most dominant color in the imageData provided

      Parameters

      • imageData: ImageData
      • pixelCount: number

      Returns [number, number, number]

      The dominant color as an RGB array.

      GreenScreenStream

    • Flips the video stream horizontally. Toggles the flipHorizontal property in the segment configuration.

      Returns void

      GreenScreenStream

    • Retrieves the color palette and dominant color from the current video stream. This method captures the current frame from the video source, processes it, and returns an object containing the color palette and dominant color.

      Returns { dominant: [number, number, number]; palette: [number, number, number][] }

      An object containing the color palette and dominant color.

      GreenScreenStream

    • Initializes the green screen stream with a background source and optional configuration.

      Parameters

      • backgroundUrl: string

        The URL of the background image or video to use.

      • Optionalconfig: IGreenScreenConfig

        Optional configuration settings for the green screen.

      Returns Promise<void>

      A promise that resolves when the initialization is complete.

      If no renderer is created or if the background source is invalid.

      GreenScreenStream

    • Get a palette of the most common colors in the imageData provided

      Parameters

      • imageData: ImageData
      • pixelCount: number

      Returns [number, number, number][]

      An array of RGB color arrays representing the palette.

      GreenScreenStream

    • Scales the provided image to fit the canvas dimensions. Returns a promise that resolves with the scaled HTMLImageElement.

      Parameters

      • image: HTMLImageElement

        The image to scale.

      • OptionalimageOptions: ImageBitmapOptions

        Optional options for creating the image bitmap.

      Returns Promise<HTMLImageElement>

      A promise that resolves with the scaled image.

      GreenScreenStream

    • Sets the background image or video for the green screen. Returns a promise that resolves with the loaded background element (HTMLImageElement or HTMLVideoElement).

      Parameters

      • src: string

        The source URL of the background image or video.

      Returns Promise<HTMLImageElement | HTMLVideoElement>

      A promise that resolves with the loaded background element.

      GreenScreenStream

    • Sets the BodyPix model for machine learning-based background removal. Loads the model based on the provided configuration or uses a default configuration.

      Parameters

      Returns Promise<void>

      A promise that resolves when the model is loaded and ready.

      GreenScreenStream

    • Sets the chroma key color used for green screen processing. The chroma key color is specified as RGB values.

      Parameters

      • r: number

        The red component of the chroma key color (0-255).

      • g: number

        The green component of the chroma key color (0-255).

      • b: number

        The blue component of the chroma key color (0-255).

      Returns void

      GreenScreenStream

    • Sets the mask range for the green screen processing. The mask range is specified as a vector with x and y components.

      Parameters

      • x: number

        The x component of the mask range.

      • y: number

        The y component of the mask range.

      Returns void

      GreenScreenStream

    • Start the rendering process with an optional maximum frames per second (maxFps). If maxFps is not provided, defaults to 25.

      Parameters

      • OptionalmaxFps: number

        The maximum frames per second for rendering.

      Returns void

      GreenScreenStream

    • Stops the rendering process and optionally stops the media streams.

      Parameters

      • OptionalstopMediaStreams: boolean

        Whether to stop the media streams (default is true).

      Returns void

      GreenScreenStream

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Methods