Humans are visual…very visual. at any one time something close to 65% of our brain run time activity is taken up with image processing. We also inherently look for patterns…it’s why we see castles in the clouds and Jesus on toast. Recognizing and responding to images is so second nature it’s easy to assume that huge powerful computers should be able to do the same thing…not so.
Although images have been a huge part of the web since day one the ability to recognize and label an image has been minimal at best. There are billions of images on line and many (most?) aren’t identified well or at all. I remember well back in the early days of image search at FAST we would get all kinds of complaints from users for delivering adult content in search results which wasn’t flagged as “adult.” What was happening in many cases was that adult sites would inadequately protect their content and our indexer would just breeze past and index all their images then make them available for free. The images were typically names something like TS1234, TS1235, TS1236 etc. Not only were there no captions to indicate their content our algorithms were rarely able to detect that they were adult programmaticly which meant we had to fall back on spot checks by humans. If you think you have a horrible job it’s nothing compared to that task.
That was over a decade ago and only now are we seeing real progress in this area. Google has announced an artificial intelligence program which can read an image and figure out what it’s of pretty much as well as a person could. It’s not perfect, but it is pretty impressive and can scan and categorize content at a massively faster rate than human. They achieve this pretty impressive feat through neural net technology which mimics the ways that brains think. This kind of ability is particularity interesting for content areas like Flickr or Instagram where useful captions are the exception rather than the rule. Once it works on images it’s only a short hop to doing the same for video.
Of course the incentive here is if Google can index content and figure out what it is on its own it can make that content readily available to searchers, improving search results and increasing the volume of target-able data it can stack ads against…very cool.