We describe a prototype visual information system for searching for images and v ideos on the World�Wide Web. New visual information in the form of images, graphics, animations and vide os is being published on the Web at an incredible rate. However, cataloging this visual data is beyond the ca pabilities of current text�based Web search engines. The key to cataloging it is the marriage of text�based proce ssing and content�based visual analysis of the images and videos. In this paper, we describe a complete system by which visual information on the Web is (1) collected by automated agents, (2) processed in both text and visual feature domains, (3) catalogued and (4) indexed for fast search and retrieval. We introduce an image and video search engine which utilizes both text�based navigation and content�based technology for searching v isually through the catalogued images and videos. Finally, we provide an initial evaluation based upon the cata loging of over one half million images and videos collected from the Web.