Websnatcher: Customized WWW Prefetching
Gullickson, Maria L.
Chervenak, Ann L.
MetadataShow full item record
This document describes WebSnatcher, an application that prefetches Web pages and stores them on a local file system based on a user profile. Users can specify explicit URLs, the location of a bookmarks file, and areas of interest for searching. Along with areas for searching, users may select one or more of the supported search engines to use in making queries. Users can also specify how deeply it follows hyperlinks within the page. A perl script is included to allow users to run WebSnatcher automatically at periodic intervals. WebSnatcher automatically cleans up old data when retrieving new pages and creates an index to help users navigate new data. Users may optionally archive old data for future use. Files are retrieved by requests sent to a server's HTTP port. For searches, a request is sent to the search engine's server, WebSnatcher parses the results to find matching pages, and then it retrieves these pages. Along with the main HTML file, WebSnatcher retrieves any images and frames contained within the file. We have planned several improvements for WebSnatcher, including a more robust and user-friendly interface to the profile. We also discussed incorporating an Anycasting scheme to allow users to specify a set of sites or servers, any of which could be used to satisfy a query; by monitoring past performance of different servers, WebSnatcher could guess which server will provide the best performance.