On Tue, 16 Jun 1998, Ian Winship, Univ. Northumbria wrote:
> For the HyLiFe project we are looking at what information we will need
to
> assess use and haven't found any transaction log analysis software
that does
> what we want. We need something - for Digital Unix - that will analyse
who the
> users are, particularly if there is authenticated access.
>
> Any ideas I can refer to our Unix server manager?
>
We also found, originally in the Decomate project, that more detailed
logs were required for proper analysis of user behaviour, and we made
our CGI server (which can now either sit behind a standard Web(i.e.
HTTPD)server, or run as it's own HTTPDaemon (which is technically what a
Webserver is) log date-time, IP, username (some were individual, some
group names) and details of actions performed, such as search terms
used. The log records of a (very brief) session look like (view this in
a WIDE window!):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~
19980623%115822%158.143.193.20%9950230668%login%username=john%usertype=%
department=%result=SUCCESS%
19980623%115822%158.143.193.20%9950230668%authenticate%username=john%
19980623%115822%158.143.193.20%9950230668%selectdatabase%newdbname=d_art
%
19980623%115822%158.143.193.20%9950230668%init_search%
19980623%115838%158.143.193.20%9950230668%brief_record%numrec=10%startre
cno=1%query=politics and hunting%dbname=d_art%
19980623%115838%158.143.193.20%9950230668%search%query=politics and
hunting%dbname=d_art%
19980623%115844%158.143.193.20%9950230668%document_link%authority=lse.ac
.uk%identifier=ESP0016718595000275%docserver=http://decomate.lse.ac.uk:7
080/docserver/DOC.cgi%scheme=set-reference%
19980623%115845%158.143.193.20%9950230668%full_record%numrec=1%startrecn
o=1%query=politics and hunting%dbname=d_art%
19980623%115858%158.143.193.20%9950230668%exit%
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~
We kept a session number ("9950230668" in the example above), which is
essential to allow later records from the 'stateless' HTTP server to be
related to earlier ones (such as the "login" which gives the username).
We did have problems with (most) users not quitting from sessions
properly, and us deciding what the correct timeout should be when that
happened. We process logs by loading them into relational database
tables.
All the Decomate server s/w is available in a 'free' (GNU Public
License) package, via http://www.lse.ac.uk/decomate/ - from which not
*all* the other links now work, I've just noticed :-( - but I don't
strongly recommend downloading it just to read the sourcecode (Perl) to
see how the logging is done.
There are alternative techniques, such as using 'cookies', but these can
be quite annoying to cautious users who set their browsers to notify
receipt of every cookie (...and your server sends one for *every* user
action).
I hope your Unix manager is adventurous ;->
John Paschoud
Project Manager, HeadLine project
(http://www.headline.ac.uk)
& Workpackage Leader (Document Delivery, Access & Accounting),
Decomate-2 project
(http://www.bib.uab.es/decomate2)
British Library of Political & Economic Science
at the London School of Economics
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|