Skip to main content

Demo Project: Film Slate Detector

·2 mins

🎬πŸŽ₯πŸ’» A computer vision (object detection) pet project πŸ‘¨β€πŸ’»πŸΏπŸŽ€


Demo

A live demo should still be working if you’re reading this. It’s served by HuggingFace πŸ€— Spaces. If it doesn’t work embedded below here, try clicking through the link.

Why

Back in ~2016-2018 I used to spend many, many hours in a dark room in SoHo scrubbing through video files to see where the slate claps were, so that I could fix the audio sync. It got very boring, to the point where I quit to go learn how to code.

What

Why not have AI automate the boring stuff? Given enough labeled examples, a pre-trained image model should be able to spot slates, after some fine-tuning. I achieved a COCOmetric of like 88% with a few hundred images. It’s just a fine-tuned faster-rcnn model.

The natural next step would be to train it on two classes: “Open” and “closed” slates, so that you could find the inflection point where the slate goes from open to closed. That might save some dailies technicians a lot of time.

Tools

My data science mentor pointed me to IceVision, FastAI, and various TIMM models. I got set up with LabelStudio and Weights and Biases. I wrote a long LinkedIn post about how I accomplished the fine-tuning.

In case you didn’t already know, it can be incredibly frustrating to do data-intensive machine learning stuff on an Apple Silicone laptop. Rather than swapping out my local hardware, I took this opportunity to get really dirty with Ec2 spot instances and used Terraform to make it dead simple to spin up and tear down spot GPU instances.

Robert Boscacci
Author
Robert Boscacci
Film production guy who learned to code