Daylife Select
A point & click tool to create dynamic content portals. Learn More »
6, 2007) Web site policy makers who use robots.txt files as gatekeepers to specify what is open and what is off limits to Web crawlers favor Google over other search engines, say Penn State researchers whose study of more than 7,500 Web sites revealed... Full Article at Science Daily