Netflix uses computer vision to find the best thumbnail for shows.
As imagery plays such a key role in hooking in a consumer, it is undoubtedly important for the likes of Netflix to consider the artwork used to present the titles.
The process is as follows: If the artwork representing a show is of interest to you, then it acts as a gateway, giving you some visual “evidence” as to why the title might be worth watching. The artwork may highlight an actor that you have seen before, show a thrilling moment like a plane crash, or contain an enticing frame that summarises the essence of a movie or TV program.
Netflix discovered that if they layout a set of perfectly curated images on your homepage, then you will more likely give it a view. Makes sense, right?
The folks at Netflix have recognised this, innovated and clearly went through some in-depth discovery to then develop their own collection of tools and algorithms. These are designed to retrieve high quality imagery from the videos on their service.
The solution outlined was labelled "AVA".
AVA encompasses the key intersections of computer vision while putting to use the fundamentals of filmmaking and photo editing. It takes a movie or episode of a show and processes, then annotates every frame to then surface the 'best' images from through an automated pipeline. As the article states, "AVA looks for images that meet criterias of colours, shot typles, saliency and visual similarity".
It seems like a lot of work occurring behind the scenes seeing as we'd rather watch the actual videos than sift through program artwork. However, it is an exciting new innovation that’s responsible for much of the artwork you see on the Netflix service today!
AVA is a collection of tools and algorithms designed to surface high quality imagery from the videos on our service. A single season of average TV show (about 10 episodes) contains nearly 9 million total frames. Asking creative editors to efficiently sift through that many frames of video to identify one frame that will capture an audience’s attention is tedious and ineffective. Netflix set out to build a tool that quickly and effectively identifies which frames are the best moments to represent a title on the Netflix service.