How to Use TensorFlow.js for Image Classification

Posted on by Panther

Image classification is a common task in machine learning where we train a model to classify images into different categories or labels. TensorFlow.js is a powerful library that allows us to run machine learning models directly in the browser using JavaScript. In this tutorial, we will learn how to use TensorFlow.js for image classification.

Prerequisites

To follow along with this tutorial, you need to have the following prerequisites:

  • Basic understanding of JavaScript
  • Familiarity with machine learning concepts
  • Node.js and npm installed on your machine

Setting Up the Project

To begin, let’s set up a new project with TensorFlow.js and create a basic HTML file.

Step 1: Create a New Project Directory

Open your terminal and create a new directory for your project.

mkdir tfjs-image-classification

Navigate into the project directory.

cd tfjs-image-classification

Step 2: Initialize a New Node.js Project

Next, let’s initialize a new Node.js project in the project directory.

npm init -y

This will create a package.json file with default values.

Step 3: Install TensorFlow.js

Install TensorFlow.js as a project dependency.

npm install @tensorflow/tfjs @tensorflow-models/mobilenet

This will install TensorFlow.js and the MobileNet image classification model.

Step 4: Create the HTML File

Create a new file called index.html in the project directory and add the following code:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <title>Image Classification with TensorFlow.js</title>
  </head>
  <body>
    <input type="file" id="image-selector" accept="image/*" />
    <button id="predict-button">Predict</button>
    <div id="prediction-container"></div>

    <script src="index.js"></script>
  </body>
</html>

This HTML file includes an input element for selecting an image, a predict button, and a container for displaying the predictions. We also included a JavaScript file called index.js that we will create in the next step.

Step 5: Create the JavaScript File

Create a new file called index.js in the project directory and add the following code:

// Code goes here

We will now write the JavaScript code to load the MobileNet model, make predictions, and display the results.

Writing the JavaScript Code

Let’s start writing the JavaScript code in index.js.

Loading the MobileNet Model

To use the MobileNet model for image classification, we need to load it first. Add the following code inside the index.js file:

let model;

async function loadModel() {
  model = await tf.loadGraphModel('https://tfhub.dev/google/tfjs-model/imagenet/mobilenet_v2_100_224/feature_vector/4/default/1', {
    fromTFHub: true
  });

  console.log('Model loaded');

  // Enable predict button
  document.querySelector('#predict-button').disabled = false;
}

loadModel();

This code defines the loadModel function, which loads the MobileNet model using tf.loadGraphModel function from TensorFlow.js. We pass the URL of the model and set fromTFHub option to true to load the model from TensorFlow Hub. Once the model is loaded, we log a message to the console and enable the predict button.

Preprocessing the Image

Before we can make predictions with the loaded model, we need to preprocess the selected image. Add the following code after the loadModel function:

let inputImage;

function preprocessImage(image) {
  // Resize the image to 224x224
  const resizedImage = tf.image.resizeBilinear(image, [224, 224]);

  // Normalize the pixels
  const mean = tf.tensor1d([123.68, 116.779, 103.939]);
  const normalizedImage = resizedImage.sub(mean);

  // Expand the dimensions for the model input
  inputImage = normalizedImage.expandDims(0);

  return inputImage;
}

This code defines the preprocessImage function, which takes an image tensor as input and applies the necessary preprocessing steps. We resize the image to 224x224 pixels, normalize the pixel values using the mean values [123.68, 116.779, 103.939], and expand the dimensions to match the input shape of the model.

Making Predictions

Next, let’s write the code to make predictions using the loaded model. Add the following code after the preprocessImage function:

async function predict() {
  const imageSelector = document.getElementById('image-selector');
  const image = imageSelector.files[0];

  const reader = new FileReader();

  reader.onload = async function(event) {
    const imageElement = document.createElement('img');
    imageElement.src = event.target.result;
    document.body.appendChild(imageElement);

    await tf.nextFrame();

    const imageData = tf.browser.fromPixels(imageElement);
    const preprocessedImage = preprocessImage(imageData);

    const predictions = await model.predict(preprocessedImage);

    // Display top 5 predictions
    const topPredictions = Array.from(predictions.dataSync())
      .map((prediction, index) => ({
        probability: prediction,
        className: IMAGENET_CLASSES[index]
      }))
      .sort((a, b) => b.probability - a.probability)
      .slice(0, 5);

    const predictionContainer = document.getElementById('prediction-container');
    predictionContainer.innerHTML = '';

    topPredictions.forEach(({ className, probability }) => {
      const row = document.createElement('div');
      row.innerHTML = `${className}: ${(probability * 100).toFixed(2)}%`;
      predictionContainer.appendChild(row);
    });
  };

  reader.readAsDataURL(image);
}

const predictButton = document.getElementById('predict-button');
predictButton.addEventListener('click', predict);

This code defines the predict function, which is called when the predict button is clicked. Inside the predict function, we get the selected image from the input element, read the image using FileReader, and create an img element to display the selected image on the page.

After that, we preprocess the image using the preprocessImage function we defined earlier. We then make predictions using the model’s predict function on the preprocessed image. The predictions are an array of probabilities for each class in the ImageNet dataset.

Finally, we display the top 5 predictions with the highest probabilities in the prediction-container div on the webpage.

Adding the ImageNet Classes

To display the class names alongside the predictions, we need to define the class names for the ImageNet dataset. Add the following code above the loadModel function:

const IMAGENET_CLASSES = [
  'tench',
  'goldfish',
  'great_white_shark',
  // ...
  // List of 1000 class names
  // ...
];

Replace the comment with the complete list of 1000 class names from the ImageNet dataset. You can find the complete list of class names in the TensorFlow.js repository.

Testing the Classification Model

Now that our project is set up and the JavaScript code is written, it’s time to test the image classification model.

To run the project, open your terminal and navigate to the project directory. Then, run the following command:

npm start

This will start a local server and display a URL. Open the URL in your web browser to see the webpage.

  1. Click on the “Choose File” button to select an image from your local machine.
  2. Click the “Predict” button to make predictions based on the selected image.
  3. The top 5 predictions with their respective probabilities will be displayed on the webpage.

Congratulations! You have successfully built an image classification model using TensorFlow.js.

Conclusion

In this tutorial, we learned how to use TensorFlow.js for image classification. We set up a project with TensorFlow.js, loaded the MobileNet model, preprocessed the image, and made predictions using the model. TensorFlow.js makes it easy to run machine learning models in the browser using JavaScript, opening up new possibilities for creating interactive machine learning applications.

Related Post