Skip to content

Playing around with Augmented Reality on the web using OpenCV and WebAssembly.

Notifications You must be signed in to change notification settings

webarkit/wasm-ar

 
 

Repository files navigation

Planar Homography Estimation and Image Tracking Running in WASM

Demo on laptop Demo on iPhone
PC demo phone demo

Try it out

Try it out here (tap the screen to enable and disable tracking):

https://edwardlu2018.github.io/wasm-ar/

Reference image here.

Pipeline

  1. Pass image from video stream as an array from JavaScript to WASM program. Turn image to grayscale.
  2. Find homography matrix by creating and matching ORB descriptor keypoints from reference image to video frame.
  3. Take those descriptor points from the video frame and track them using Lucas-Kanade tracking algorithm.
  4. Find 2d affine transform T of descriptor points from one frame to the next and update homography matrix. H = T * H
  5. Pass homography matrix and warped corner points back to JavaScript and warp iframe element with homography matrix.
  6. If homography matrix becomes invalid or at most 1/3 of tracked descriptor points are lost, repeat step 2.

Building

You will need git, cmake, and python installed.

The first step is to download and install emsdk (version 1.39.16 is recommended):

git clone https://github.com/emscripten-core/emsdk
cd emsdk
./emsdk update
./emsdk install latest
./emsdk activate latest

Next, you need opencv with WebAssembly support. You should build opencv_js manually (make sure emsdk is installed!): https://docs.opencv.org/4.x/d4/da1/tutorial_js_setup.html

git clone https://github.com/opencv/opencv.git
cd opencv
python3 ./platforms/js/build_js.py build_wasm --build_wasm

The python script will build the static and the WASM lib in the build_wasm folder.

Then, run:

npm run build

About

Playing around with Augmented Reality on the web using OpenCV and WebAssembly.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 94.3%
  • C++ 3.5%
  • HTML 0.6%
  • CSS 0.6%
  • CMake 0.6%
  • GLSL 0.3%
  • Shell 0.1%