URL Spider Pro

URL Spider Pro is a free utility designed to crawl a website of your choice and index the results. You can export to a variety of different databases to creat your own code to have a search on your site.