This article is the continuation of the previous article on MediaPipe Face Mesh model in TensorFlow.js, where we looked at creating the triangle mesh of the face using the model’s output. Here, we will look at detecting and tracking iris within the eyes using the MediaPipe Iris model. 1. Iris Detection MediaPipe Iris model accurately estimates the iris landmarks within the eye and tracks it. Iris detection & tracking can be used in augmented reality to create avatars and also to determine the distance of the camera to the user. ...

This article is the continuation of the previous article on MediaPipe Face Mesh model in TensorFlow.js, where we looked at the basic usage of this model. Here, we will look at how to connect the 468 3D facial landmark predictions as multiple triangles to create a triangle mesh of the face. 1. Triangle Mesh To create a triangle mesh from the 468 [x,y,z] coordinates, we will need to identify which are the set of points that need to be connected to form triangles. Luckily Google has provided those indexes for us to draw the triangle mesh. It is present in the triangulation.js file located in the Github repo of the pre-trained model. ...

In this article, we will walk through an example to identify facial landmarks using the state of the art MediaPipe Face Mesh model . This model is present in the Face Landmarks Detection package in TensorFlow .js library’s pre-trained model suite. This is part 1 of a series of articles in which we dive into the details of the Face Mesh model and its usage. 1. Overview The MediaPipe Face Mesh model estimates 468 3D facial landmarks in real time covering the overall surface geometry of a human face. It also supports Iris detection that accurately tracks the iris within the eye. Iris detection when enabled provides an additional set of 10 landmarks - 5 points for each eye. ...

In this article, we will look at how to use Pose Detection in TensorFlow.js to classify cricket shots. In the game of cricket, there are many different shots played, such as cover drive, square cut, flick, sweep, straight drive, etc. In this example, we will attempt to classify three cricket shots - cover drive, flick and square cut. We will utilize BlazePose TensorFlow.js model to obtain the poses from images of players playing the above shots. We will then feed these poses data to a KNN classifier to classify between these three shots. ...

Human Pose Detection is fascinating as there are endless use cases and opens up the possibility for various kinds of applications which were earlier either difficult to build or required specialized hardware and software. Photo by Alex Shaw on Unsplash In this article, I will walk you through the steps required to build a simple JavaScript app that detects human poses using Pose Detection API in TensorFlow.js utilizing the Mediapipe BlazePose model. ...

I have often wondered if I can build a tool that can accept custom queries to filter photos and images based on the items present in the image. There are some popular apps such as Google Photos, OneDrive, etc. which provide this sort of functionality to various degrees but they don’t necessarily let the user enter custom queries to filter images. In this article, we will look at utilizing TensorFlow.js pre-trained COCO-SSD object detection model to categorize and filter images. ...

As most of us are working from home now, it has become a normal thing to set a custom background for our video calls through popular apps such as Zoom or Microsoft Teams. In this article, we will look at how to change the background for live webcam videos using TensorFlow.js . Person Segmentation in action We will be making use of the BodyPix model which is a pre-trained TensorFlow.js model. This model performs image segmentation - specifically person and human body parts segmentation. ...

Most of us interested in TensorFlow.js would be familiar with the popular demo of disappearing people in real time by Jason Mayes. This is a fantastic example of what is possible using TensorFlow.js in the browser. Jason is gracious to share the code of this application for others to learn and implement their own versions of such applications. In this article, I will attempt to document and walkthrough this code, so that it helps anyone who is trying to understand the code for this app. ...

In this article, we will be building a simple app to detect objects on an image using TensorFlow.js library. To do this we will make use of the TensorFlow.js pre-trained “coco-ssd ” model. In this example, user will browse and select an image file to identify objects present in that image. This selected image file is only handled on the client side within the browser and not uploaded to any server. ...