HotWired have just launched a new service which acts as a layer over their
Hotbot search engine called Newbot. It is similar in approach to standard
engines except that it also indexes constantly updated news sources.
It's an interesting, though very limited, approach to the problem of
indexing real-time information. Obviously the various metadata projects
have this as one of their aims, but this solution works to some extent
right now. Obvious problems are the limited scope of the sites
continuously indexed, the load issue which would occur if multiple engines
started doing this, and of course the limitations of current indexing
techniques.
The informational post from Hotwired appears at the end of this message.
And so now a little about me, since this is my first post to this list.
For the last year or so I have been very interested in the whole issue of
indexing information. We all know the problems with text-retrieval search
engines and I spent many months trying to independently come up with an
indexing solution for Online World (www.onlineworld.net.au) my previous
employer.
The approach was novel, but hastily and shoddily implemented. Basically we
had a series of bit fields within each record which corresponded to each
category. In this way records could fit within multiple categories, unlike
Dewey Decimal for example. We developed an indexer which was able to very
rapidly whip through all the bitmasks comparing it to a query bitmask in a
boolean fashion to achieve a ranked result.
More recently my thoughts have moved to the problems of indexing
information Internet-wide. Search engines are only useful for archival
material and currently do not index database or dynamic information--which
unfortunately rules out many of the best information sources.
An unfinished (yet) discussion paper on this issue appears at this URL:
http://www.geocities.com/Paris/LeftBank/5803/infoarch.html
Looking forward to some interesting discussion!
---
___ [log in to unmask]
|___ http://www.geocities.com/Paris/LeftBank/5803/
___| H E R M
omphaloskepsis n: contemplation of one's navel
|