See more from Coding and the Arts
This project is featured in chapter 5 of Coding and the Arts, where it serves as an introduction to Teachable Machine and training a machine learning model. I find the latest version of Teachable Machine to be really intuitive, particularly when capturing training images from your webcam, but even with its ease of use I think the following introductory video is a super helpful bit of context setting for students.
You can definitely run this activity without any programming by focusing students on training and testing their models in Teachable Machine, but it’s expected that many students will want to do something with the model they’ve trained. The exemplars below all use this model that I trained with a few pieces of fruit and serve as a demonstration for students of how to wire up their trained models in programming environments that you may already be using in the classroom.
As with all things ML, this example model can only ever be as good as the data it was trained with, which happened to be whatever fruit I had on hand with my desk as a background. This inevitably means that it won’t be terribly accurate for a wide range of actual fruit in a wide range of environments. You can use this as a teachable moment for your students as well - the models that they train may work well for them, but will inevitably perform differently for other users in other environments.
Using p5.js
The ml5.js library makes is relatively quick and easy to integrate Teachable Machine models with p5. In fact, when you go to export a model they show some boilerplate p5 code that can be opened directly in the p5 web editor.
In this sketch I’ve pulled in my fruit categorization model to draw the detected fruit on the canvas. You can play with the threshold variable on line 25 to determine how confident the detection needs to be in order to draw the detected fruit.
Using Scratch
You can’t use the standard Scratch environment with Teachable Machine (yet), but there’s an experimental extension that works pretty dang well. There’s no easy way for to create a sharable exemplar, but if you head to the github repo you can follow the instructions there. All you need to do is copy and paste the shareable URL from Teachable Machine (eg. teachablemachine.withgoogle.com/models/FYiXCGNVc/) into a special block and then use the additional provided blocks to write programs that use the classifications produced by the model. See the brief example code below.
