|November 20, 1996||
Press Contact: Steve Koppes|
WebSeer surfs the Web for pictures
Have you ever wondered whether theres a picture of the Mona Lisa on the World Wide Web? Or wanted to find a good picture of an eagle? Or the aurora borealis, the Space Shuttle or Michael Jordan?
Now theres a new image-based search engine that lets you find images directly, rather than surfing blindly or using text-based searches to try to find pictures. Dubbed WebSeer, this innovative image-based search engine was designed and built at the University of Chicagos Intelligent Information Laboratory.
This is the first useful way for people to search the Web for images, both photographs and drawings, said Michael Swain, Assistant Professor in Computer Science, and the main architect of WebSeer. I think this is going to be the most widely used application of computer vision ever.
WebSeer stores contextual information about each imagesuch as the caption, the title of the Web page and the name of the image fileweighted as to how relevant it is to the content of the picture. But WebSeer also knows whether its a photograph or a drawing, whether its in color or black and white, whether or not there are peoples faces in the picture and if so, how many there are and how close up they are.
Just looking at the pixels of an image, its beyond the current computer vision technology to figure out whether or not it is a picture of a horse," said third-year graduate student Charles Frankel, who developed the architecture and the indexing system for WebSeer. Because its so difficult we have to use a combination of techniques that include the context that the image is in and the image-understanding algorithms that we do have. The architecture allows us to incorporate new image-understanding techniques as they are developed, either here or in collaboration with other universities.
WebSeer has a sophisticated automatic indexing system that can add tens of thousands of images a day to its data base. To date there are 500,000 images, growing at a rate of one every two seconds.
When youre using a text-based search engine like AltaVista, its usually easier to look for very specific information, like the Chicago Bulls schedule, for example, said Frankel. But with WebSeer, you can search for a variety of different things that range from the very specific, like O.J. Simpson or Bill Clinton, to more general like eagles or roses, or even types of images like skylines or aerial views. Its also fun to use WebSeer if you dont really know what youre looking for; you can see a picture and use it as a jumping-off place.
Searching for Michael Jordan, photograph retrieves images of Jordan kissing the championship trophy, swinging a baseball bat and driving for the basket. Searching for aurora (northern or southern lights), yields several photographsas well as a diagram of how the aurorae are generatedand links to pages such as the Space Plasma Physics Center, tourism in Scandinavia and Alaska, and NASA studies of the solar wind and auroral physics.
Clicking on the thumbnail of the image takes you directly to the image itself on the Web; to see it in context, click on the page icon next to the thumbnail.
How WebSeer analyzes images
When the crawler finds an image on a Web page, it first analyzes the text, like the caption, the name of the image file, the title of the Web page, etc. to gather clues about what is actually in the image. It weights the information according to how relevant it is likely to be (caption vs. page title, for example) and stores it. Next, WebSeer analyzes the image to figure out whether it is a photograph or a drawingusing information such as the file type, color, distinct boundaries, etc., in a decision tree developed by second-year University of Chicago graduate student Vassilis Athitsos and trained on 30,000 images.
Then WebSeer searches for skin hues to determine whether or not there are people in an image. If so, it uses a neural network, developed at Carnegie-Mellon University and trained on thousands of images, to locate each face in the image.
WebSeer stores in its data base a thumbnail version of the image, its URL and the URL of the page its on, along with information about the picture: whether its a photograph or a drawing, how many faces, the file type of the image, etc.
Plugging in new features
Collaborators at Berkeley are working on detecting animals and people in pictures; MIT vision researchers are developing a trainable object-recognition system, and Caltech is developing a face-finding algorithm that will compete with Carnegie-Mellons. University of Chicago statistician Yali Amit, Associate Professor in Statistics, is working on reading text in images using optical character recognition techniques.
What weve developed here at the University of Chicago is an incredibly valuable research tool for us as well as for researchers at dother universities, said Swain, and we hope to provide an interesting problem to motivate research in computer vision.
Swains research is funded by the Office of Naval Research and the National Science Foundation. Some key equipment for WebSeer was donated by IBM.
The University of Chicago Intelligent Information Laboratory is focused on issues of information access and management. With an eye toward problems that are real rather than academic, the Info Lab is dedicated to the development of information technology that will form a bridge between people and the machines that serve them.
Last modified at 03:50 PM CST on Wednesday, June 14, 2000.
5801 South Ellis Avenue - Room 200
Chicago, Illinois 60637-1473
Fax: (773) 702-8324