Site Search - A Minefield

Posted on Wed, 24 Jan 2007
 at 16:13:56

     Site search is a pretty standard thing on most of today's web sites - it allows quick navigation to specific pages and can make disabled users visit to your site far less strenuous. This is especially true on sites with more than a few pages - navigating these pages can be impossible.

     From a personal stand point I have only had limited experience with internal site searches, however I have tried many different solutions to the problem. These range from pre-made scripts to an attempt at a bespoke solution, that was about as ineffective as it could have been.

Google Site Search (AKA Google Co-Op)

     Google's site search feature is an incredibly powerful web site search facility, but it fails in any kind of practical sense. In essence, all this software does is look into it's own indexes and just return results from your web site, however because it is only updated as often as Google crawls your web site the results could be months out of date, depending on the popularity of your web site. This is fine if your site is static and is rarely updated, but for a site like this blog, which can be updated several times in one day, it's pretty useless in all fairness.

Free Find

     Free Find is far more practical than Google's software. Like Google, the service is free although there is an option to pay a varying fee to increase the size of the index that Free Find creates of your site(s) and also remove adverts. Indexes can be scheduled of your site for varying intervals, including daily, and can also be performed manually through the control panel. Unlike Google's Site Search, the template in which the results are displayed is 100% customizable, so it need not break the flow from page to page. The adverts on the free version are not too distracting and the software works well.

Bespoke v1.0

     This leads me on to another experience I have had with creating a search facility on a web site. I attempted to create a bespoke solution for searching my web sites with very poor success - building an index of a site is incredibly difficult, as you have to remove common words like 'the', 'or' and 'and' etc., as well as HTML tags, which is easy to do in PHP (strip_tags function) and also allow for stemming (recognizing 'bounce' as similar to 'bouncy' and 'bouncing' for instance). This is very time consuming and difficult to get right. My solution was far less elegant - using PHP I created a copy of the web site's pages in a MySQL database and used a full text search to retrieve relevant pages. As an estimate, I would say that it worked only 50% of the time.

What About Dan Blog?

     For Dan Blog I once again face the problem of how best to implement a search facility. In the long run I intend to take another stab at developing my own, but until then Free Find is my search software of choice.