The Future of Image Search Belongs to Social Networks

BlahBlah Hosted on Zooomr

The current state of image search Niall Kennedy is out with a post titled “The current state of image search” where he profiles the basic ways that machines read pages with images on them today to come up with how to categorize an image.

Niall talks about search engines reading the img element of an image name to try and figure out what it is. He also mentions other information about an image including tags, EXIF data, geolocation data, copyright data etc. He also touches on some of the technologies out there today by people like Riya who are trying to do things to get at what’s inside of a picture from a text or facial recognition standpoint.

And all of this is good and accurate in terms of where we are at today with some of the basic tools that are being used by various image hosting properties.

But the question that I’m interested in is not what is the state of image search today, but where are we headed in the future.

At present most search engines do a pretty poor job of image search. Flickr by far has the best image search on the internet and I’ve written a lot on this in the past. Most recently, in March, when Microsoft’s live image search came out I blogged about how when comparing the various image search engines that Flickr came up with better results than anyone else (remember it was the Search group at Yahoo that bought Flickr).

The basic problem is two fold. 1. How do you know what is actually in an image and then 2. how do you rank relevancy since it’s only really the first few pages of search results that matter.

In terms of getting at what is actually inside of a photo, I would argue that using a social network with tag based functionality will produce far superior and accurate results over what relying on image file names and surrounding text alone will do.

One of the problems with relying on image file names is that frequently these names do not accurately identify the contents of a photo. Sometimes they work sometimes they don’t. Take this first photo of mine for example. Yes, this title works quite well. This is a 1995 Chateau Margaux. The tags tell you even more, that it’s a Bordeaux, that it’s wine, etc. But other times file names don’t work. This photo of mine titled “An’ it Ain’t No Use in Turnin’ on Your Light, Babe, I’m on the Dark Side of the Road,” is less descriptive. Of course it’s more Bob Dylanish, but that’s another story.

So the question becomes how should a search engine best get at what’s in a photo if title alone isn’t always realiable. And this is where tags come in. On the previously mentioned image “An’ it Ain’t No Use in Turnin’ on Your Light, Babe, I’m on the Dark Side of the Road,” you can still figure out what it is by looking at the tags. Posey Tube, Oakland, cars, tunnel, etc. All pretty good indicators of what is inside of this particular photo.

So the two biggest search engines are going about tagging in different ways. Yahoo! is using their social network Flickr to tag photos — and in time I suspect you begin to see more and more Flickr images heavily rotated into Yahoo! image search. Flickr is being careful with this so as not to upset the Flickr community. They have created an opt out of search function and they have only just recently started flirting with this idea at all releasing just 5 image terms into general Yahoo Search. Expect more to come though.

Google on the other hand is trying something else. They are trying to create a game, called Image Labler, where people look at an image and then have a race to see who can type the most tags. The problem withi Image Labeler is that it is generally boring and nowhere near as fun as tagging at Flickr and it has no utility outside of the game. At Flickr when you tag your own photos (which probably represents 98% of the tagging at flickr in my opinion) you do this not because it’s a game but because more than anything it helps you keep your own photos organized.

Of course getting images tagged is only one part of a two part problem. And while I’d give the advantage to Yahoo! so far (I will guarantee you that far more tags have been typed at Flickr than Google Image Labeler), Yahoo definitely has the upperhand also when it comes to relevancy. You see, in addition to collecting tags on photos, Flickr also measures activity through their interestingness algorithm. And searches by interestingness actually pull up pretty stunning images of most common search terms.

But there are ways to refine and make image search even better from a social networking perspective and these are some of the things that we are working on at Zooomr. Take peopletags for instance, a technology unique to Zooomr. Right now as people are tagged using these special tags they are signifying to us specifically who is in a photo and where in the photo they are. Here is a collection of photos people tagged “Thomas Hawk.” Compare this with an image search for the term Thomas Hawk on Google and you can see the superiority of people tags. Here’s another example, photos peopletagged Tantek Celik at Zooomr vs. a Google Image Search for Tantek Celik. Here by the way are the most peopletagged people on Zooomr today.

The other thing we are doing at Zooomr is giving the searcher far greater control over the criteria used to search. Advanced search tools through SmartSet technology make search even more powerful. Our image library doesn’t compare to Google or Flickr or Yahoo *yet* but you will begin to see some of the potential by looking at these two SmartSets (essentially advanced filtered search) that I’ve created. The first one is photos of dogs with over 50 views and the second is photos of cats with over 50 views.

We are adding favorites to Zooomr shortly and favorites will then also be able to be used as a criteria to sort. Cats viewed over 50 times outrank dogs viewed over 50 times as one might expect, but of course SmartSet technology is not limited to just cats and dogs.

Future SmartSets will be able to allow you to filter results by time stamp, day vs. night, geolocation, etc. It is social networks that exist at places like Flickr and Zooomr that make this technology possible. And this in my opinion is where the future of image search is headed. We think giving advanced controls to your users allowing them to search your library of images will create a better overall ex
perience for the user with powerful future image search technology as a byproduct.

Be Sociable, Share!
Loading Facebook Comments ...

One Comment

  1. Eric Rumsey says:

    Good words on image search. At Hardin MD we link to medical/disease images at external sites. Flickr works well for non-technical photography, but for technical subjects like we work with, Google Image Search is unsurpassed.

    Incidentally, the dog-cat picture at the top of your page brings to mind an interesting adage from medical informatics about the pattern-recognition skills of the human brain — Computer-based image-recognition systems aren’t able to distinguish a dog from a cat, but 1-year old human infant can do it.