Tensorflow-Js on Selvam Subbiah

TensorFlow.js - Remove or restore segments of an image in the browser using DeepLab model

Sat, 11 Dec 2021 00:00:00 +0000

This article is a continuation of the previous article on Semantic Image Segmentation in the browser using TensorFlow.js pre-trained DeepLab model. Here, we will use the DeepLab model’s output to manipulate specific parts of the image based on user selection. Essentially, we will allow the user to remove or restore a specific segment of an image that corresponds to an object class identified by the model.

TensorFlow.js - Semantic Image Segmentation in browser using DeepLab model

Sun, 05 Dec 2021 00:00:00 +0000

In Computer Vision, one of the main challenges is to understand an image at a higher level like the way we humans do, i.e., to identify the objects, their location and their surroundings to make sense of the scene captured in the image. Semantic Image Segmentation is the process of identifying the objects present in the image and labelling each pixel by associating it to an identified object class. Semantic Image Segmentation can be applied to a wide variety of use cases such as human body segmentation, separating the background from foreground in an image, and understanding the surroundings for self-driving cars, just to name a few.

Face Landmarks Detection using MediaPipe Face Mesh in TensorFlow.js - Part 3

Sun, 28 Nov 2021 00:00:00 +0000

This article is the continuation of the previous article on MediaPipe Face Mesh model in TensorFlow.js, where we looked at creating the triangle mesh of the face using the model’s output. Here, we will look at detecting and tracking iris within the eyes using the MediaPipe Iris model.

1. Iris Detection

MediaPipe Iris model accurately estimates the iris landmarks within the eye and tracks it. Iris detection & tracking can be used in augmented reality to create avatars and also to determine the distance of the camera to the user.

Facial Landmarks Detection using MediaPipe Face Mesh in TensorFlow.js - Part 2

Sun, 21 Nov 2021 00:00:00 +0000

This article is the continuation of the previous article on MediaPipe Face Mesh model in TensorFlow.js, where we looked at the basic usage of this model. Here, we will look at how to connect the 468 3D facial landmark predictions as multiple triangles to create a triangle mesh of the face.

1. Triangle Mesh

To create a triangle mesh from the 468 [x,y,z] coordinates, we will need to identify which are the set of points that need to be connected to form triangles. Luckily Google has provided those indexes for us to draw the triangle mesh. It is present in the triangulation.js file located in the Github repo of the pre-trained model.

Facial Landmarks Detection using MediaPipe Face Mesh in TensorFlow.js

Wed, 17 Nov 2021 00:00:00 +0000

In this article, we will walk through an example to identify facial landmarks using the state of the art MediaPipe Face Mesh model . This model is present in the Face Landmarks Detection package in TensorFlow .js library’s pre-trained model suite.

This is part 1 of a series of articles in which we dive into the details of the Face Mesh model and its usage.

1. Overview

The MediaPipe Face Mesh model estimates 468 3D facial landmarks in real time covering the overall surface geometry of a human face. It also supports Iris detection that accurately tracks the iris within the eye. Iris detection when enabled provides an additional set of 10 landmarks - 5 points for each eye.

TensorFlow.js - Cricket Shot classification using Pose Detection

Sat, 13 Nov 2021 00:00:00 +0000

In this article, we will look at how to use Pose Detection in TensorFlow.js to classify cricket shots. In the game of cricket, there are many different shots played, such as cover drive, square cut, flick, sweep, straight drive, etc.

In this example, we will attempt to classify three cricket shots - cover drive, flick and square cut. We will utilize BlazePose TensorFlow.js model to obtain the poses from images of players playing the above shots. We will then feed these poses data to a KNN classifier to classify between these three shots.

Pose Detection using BlazePose in TensorFlow.js

Mon, 11 Oct 2021 00:00:00 +0000

Human Pose Detection is fascinating as there are endless use cases and opens up the possibility for various kinds of applications which were earlier either difficult to build or required specialized hardware and software.

Photo by Alex Shaw on Unsplash

In this article, I will walk you through the steps required to build a simple JavaScript app that detects human poses using Pose Detection API in TensorFlow.js utilizing the Mediapipe BlazePose model.

Image Categorization using Object Detection in TensorFlow.js

Wed, 06 Oct 2021 00:00:00 +0000

I have often wondered if I can build a tool that can accept custom queries to filter photos and images based on the items present in the image. There are some popular apps such as Google Photos, OneDrive, etc. which provide this sort of functionality to various degrees but they don’t necessarily let the user enter custom queries to filter images.

In this article, we will look at utilizing TensorFlow.js pre-trained COCO-SSD object detection model to categorize and filter images.

Change background for videos using TensorFlow.js

Mon, 04 Oct 2021 00:00:00 +0000

As most of us are working from home now, it has become a normal thing to set a custom background for our video calls through popular apps such as Zoom or Microsoft Teams. In this article, we will look at how to change the background for live webcam videos using TensorFlow.js .

Person Segmentation in action

We will be making use of the BodyPix model which is a pre-trained TensorFlow.js model. This model performs image segmentation - specifically person and human body parts segmentation.

Understanding the Real Time Person Removal project - TensorFlow.js

Sat, 02 Oct 2021 00:00:00 +0000

Most of us interested in TensorFlow.js would be familiar with the popular demo of disappearing people in real time by Jason Mayes. This is a fantastic example of what is possible using TensorFlow.js in the browser. Jason is gracious to share the code of this application for others to learn and implement their own versions of such applications.

In this article, I will attempt to document and walkthrough this code, so that it helps anyone who is trying to understand the code for this app.

Object Detection using TensorFlow.js

Fri, 01 Oct 2021 00:00:00 +0000

In this article, we will be building a simple app to detect objects on an image using TensorFlow.js library. To do this we will make use of the TensorFlow.js pre-trained “coco-ssd ” model.

In this example, user will browse and select an image file to identify objects present in that image. This selected image file is only handled on the client side within the browser and not uploaded to any server.