Study of Web Crawling Approaches

World Wide Web (WWW) has huge information and latest information is updated so the size of the Web is of order billions of pages. The most significant search engine component is Web crawler. The pages are downloaded continuously and database are stored with pages after indexing. For a crawler, it is not so easy to crawl the complete web and to keep fresh index. To achieve better efficiency, various approaches for Web crawling exist.