|
ECOLN is an experimental search engine. It runs on cheap rented web space. The server components are written in Perl. It also has administrator components written in C++ and Visual Basic.
One day the web will be served by many millions of interconnected search engines which share information freely in background processes. Specialist search engines covering a single topic will connect to other search engines that cover similar or related topics. A relatively simple non-proprietary protocol will allow this to happen. Because the protocol will be easy to understand and exist without usage restrictions in the public domain, anybody will be able to write participating software. Any software which affects the system adversely will be rejected by other participants and conversely, any software which benefits the system will thrive and provide functional evolution.
In one sense, the web already works like this. People write search engines. If they are good - people put links to them in their web pages. If they are not - people don't. However, to my knowledge, no search engine yet in existence has a software interface that would allow information to be shared with other unknown search engines in an automatic, controllable and standardised fashion. I will explain why I think this would be useful by describing two types of searches I typically do on different search engines...
1) I use Google for many of my searches to find information about Cambridgeshire. It is very, very large and very, very powerful. Indeed the paper "The Anatomy of a Large-Scale Hypertextual Web Search Engine" written by Google inventors Sergey Brin and Lawrence Page was an inspiration to me (or at least it was when interesting parts of it were explained to me by a friend). I search for all sorts of information about Cambridgeshire on Google and tend to get very good results.
2) I use the search facility on the Cambridge Evening News web site because there is huge amounts of information there which isn't available on Google. However, this search engine retrieves only information that exists on the Cambridge Evening News's web site.
|
Now if these two search engines had been sharing information, then I would conceivably get a better set of results by using either of them. I would also have a small search engine running on my PC that the search engines at Google and at the Cambridge Evening News may or may not be aware of (information about available search engines could also be propagated through the system as a background process). The Cambridge Evening News's search engine would be able to poll my search engine to see if I had indexed anything recently which would be of interest to other people using their search engine - likewise the Google engine would be able to do the same.
One particularly attractive aspect of this system is that it would allow information which causes pages to become findable on a variety of search engines to propagate through the Internet faster than it currently does. At the moment it takes the better engines typically days or weeks to get around to indexing most pages - if they get around to indexing them at all! The system I describe would mean that anyone could add indexes any pages they wanted to at any time. The speed at which that information would get propagated would depend on how interesting it is to the other participating search engines. It would, as I see it, propagate more quickly to search engines which index material of a similar nature and more slowly, if at all, to other search engines.
ECOLN's specialist topic is Cambridgeshire. There are many pages from the Cambridge Evening News indexed on it and many others from completely different sources. The sorts of pages you might find on ECOLN include personal, academic, business, fun, community, genealogical and all sorts of other types of Cambridgeshire related web sites written by lots of different people. It doesn't work in the distributed way described above yet - but it is a precursor to one that will.
If you are interested in knowing more about how ECOLN works - then see this page.
Click here to contact me. Please let me if you would like me to index your web site or any other web site for that matter. One day you will be doing this for yourself - instantly.
|
.com