react-video-compositor

A flexible React video component for virtual backgrounds, foreground effects, and bokeh background blur.

Resources

• Repository: https://github.com/benrosen/react-video-compositor
• Documentation: https://benrosen.github.io/react-video-compositor
• Package: https://www.npmjs.com/package/react-video-compositor

Getting Started

Install

npm i react-video-compositor

Import

Add the following line to the top of your index.js file.

import * as tfjs from "@tensorflow/tfjs";

Implement

The following example shows a BodyPixCompositor component that has been configured to render 60 times per second.

The rendered video will have a blurred background and will be overlaid with an image from a file.

The onCaptureStream callback will be used to log a MediaStream to the console.

import React from "react";
import { BodyPixCompositor } from "react-video-compositor";

export default function App() {
  return (
    <BodyPixCompositor
      background="blur"
      foreground="images/foreground.png"
      fps={60}
      onCaptureStream={(stream) => {
        console.log(stream);
      }}
      style={{
        border: "1px dotted magenta",
        maxHeight: "50%",
        minHeight: "10%",
      }}
    />
  );
}

TIP: You can use the onCaptureStream callback to send the MediaStream argument to a service like the AWS Chime SDK.

Configuring Virtual Backgrounds

The background prop can be used to configure virtual backgrounds for the video stream. The background prop accepts a variety of input types, including:

• Bypass
• Background Blur
• CSS Color Name
• Hex Code
• RGBA
• Image Element
• Video Element
• Image Path
• Video Path

Bypass

import React from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  return <VideoCompositor />;
}

Background Blur

import React from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  return <VideoCompositor background="blur" />;
}

CSS Color Name

import React from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  return <VideoCompositor background="yellow" />;
}

Hex Code

import React from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  return <VideoCompositor background="#ffff00" />;
}

RGBA

import React from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  return <VideoCompositor background="rgba(255, 255, 0, 1)" />;
}

Image Element

import React, { Fragement, useRef } from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  const imageElement = useRef();
  return (
    <Fragment>
      <img ref={imageElement} />
      <VideoCompositor background={imageElement} />
    </Fragment>
  );
}

Video Element

import React, { Fragement, useRef } from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  const videoElement = useRef();
  return (
    <Fragment>
      <video ref={videoElement} />
      <VideoCompositor background={videoElement} />
    </Fragment>
  );
}

Image Path

import React from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  return <VideoCompositor background="images/background.png" />;
}

Video Path

import React from "react";
import { VideoCompositor } from "react-video-compositor";

export default function App() {
  return <VideoCompositor background="videos/background.mp4" />;
}

Bibliography

• https://dannadori.medium.com/virtual-background-with-amazon-chime-sdk-bodypix-23fb59ac8c64
• https://github.com/FLECT-DEV-TEAM/LocalVideoEffector
• https://blog.tensorflow.org/2019/11/updated-bodypix-2.html
• https://blog.logrocket.com/responsive-camera-component-react-hooks/
• https://github.com/vinooniv/video-bg-blur/blob/master/index.html
• https://github.com/vinooniv/video-bg-blur/blob/master/js/video.js
• https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API
• https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video