An algorithm watching a movie trailer
We ran the trailer for The Wolf of Wall Street through an object-detection algorithm that identifies and labels everything on screen. In three separate videos, we essentially see how algorithms watch movies: They label the essentials—a tie, a wine glass, a chair—but leave the specifics out. It’s like visual ad-libs.
An algorithm watching a movie trailer is a project that tries to explore how an object detection algorithm experiences a movie trailer. Object detection is the process of identifying specific objects such as persons, cars and chairs in digital images or video. For most humans this task requires little effort regardless of how the objects may vary in different sizes, scales and rotations or whether they are partially obstructed from view. For long these tasks have been difficult for computers to solve, but recent developments have shown impressive improvements in accuracy and speed, even while detecting multiple objects in the same image.
We wondered what a fast paced movie trailer would look like seen through the lens of an object detection algorithm. Therefore we sent all individual frames from the movie trailer for The Wolf of Wall Street through Yolo-2 with a threshold of 0.15, meaning that it only reacts to objects detected with a confidence of 15% or higher.