Chad Wallace Hart

webrtcHacks - Computer Vision on the Web with WebRTC and TensorFlow

TensorFlow is one of the most popular Machine Learning frameworks out there – probably THE most popular one. One of the great things about TensorFlow is that many libraries are actively maintained and updated. One of my favorites is the TensorFlow Object Detection API. The Tensorflow Object Detection API classifies and provides the location of multiple objects in an image. It comes pre-trained on nearly 1000 object classes with a wide variety of pre-trained models that let you trade off speed vs. accuracy.

The guides are great, but all of them rely on using images you need to add to a folder yourself. I really wanted to hook this into a live WebRTC stream to do real-time computer vision with the web. I could not find any examples or guides on how to do this, so I am writing this. For RTC people, this will be a quick guide on how to use TensorFlow to process WebRTC streams. For TensorFlow people, this will be a quick intro on how to add WebRTC to your project. WebRTC people will need to get used to Python. TensorFlow people will need to get used to web interaction and some JavaScript.

Check out my full blog here:
https://webrtchacks.com/webrtc-cv-tensorflow/

Author image
About Chad W Hart
Cambridge, MA USA Website
Chad is a communications market analyst and product consultant at cwh.consulting, a product management, marketing, and strategy advisory. See more at his bio page.