Sooty MusCAT Detector with ML5.js

October 25, 2020

When Life Gives You Lockdown… make things!

The Program

How it Works in a Nutshell…

index.html - import p5.js and ml5.js libraries

1
2
3
4
5
6

    <head>
        <meta charset="utf-8" />
        <!-- Load p5.js and ml5.js libraries from UNPKG content delivery network -->
        <script src="https://unpkg.com/p5@1.1.9/lib/p5.min.js"></script>
        <script src="https://unpkg.com/ml5@0.5.0/dist/ml5.min.js"></script>
    </head>

sketch.js - preload images, initiate ml5 object detector with COCO-SSD dataset

1
2
3
4
5
6
7
8
9

    function preload() {
        // Load all sooty#.jpg images from 1 to NUM
        for (let i = 0; i < NUM; i++){
            img[i] = loadImage('sooty'  + (i + 1) + '.jpg');
        }

        // Initiate COCO-SSD object detector through ml5.js framework
        detector = ml5.objectDetector('cocossd');
    }

sketch.js - if the object detected is a ‘cat’ draw a rectangle and label as Sooty!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

    for (let i = 0; i < results.length; i++){
        let object = results[i];
        
        if (object.label == "cat"){
            stroke(0, 255, 0);
            strokeWeight(2);
            noFill();
            rect(object.x, object.y, object.width, object.height);
            
            noStroke();
            fill(0,255,0);
            textSize(14);
            text("Sooty! " + round(object.confidence*100, 0) + "%", object.x + 10, object.y + 24);
            foundSooty = true;
        }
    }

What could be next?

  • Update program with a video capture feed (webcam) for real-time image detection.
  • Detecting human postures with COCO DensePose.
  • Implement a physics demo with matter.js.
  • Implement a multiplayer demo with socket.io.

Mike