Re: Log Analysis

On Fri, 2 Jul 1999 kayleigh@morgoth.celcom.com.my wrote:
]On Thu, 1 Jul 1999, Dax Kelson wrote:
]> Davi Klock said once upon a time (Thu, 1 Jul 1999):
]>> Does anyone know a log analysis script or software that make a report
]>> sorted by source host.
] [... lots of 'me toos' deleted ...]

I don't know, if the latest calamaris-2.27 http://calamaris.cord.de/ will
do that - usually it sorts by either request count or byte count. But it
is a Perl script and easy to extend to your needs - it just looks worse
than it actually is.

Suppose you have hash %clients with your client hosts as keys, and the
client host keys are

A) in numeric, dotted quad notation, compare big endian network
byte order addresses (32 bit number) as string (char by char), so you
shouldn't suffer from endianess problems, use s.t. like:

        use Socket;
        ...
        for $client ( sort { inet_aton($a) cmp inet_aton($b) } \
                keys %clients )

B) already in network by order (preferred, fastest) structure, use s.t.
like:

        for $client ( sort keys %clients ) \
                ... inet_ntoa($client) ...

C) in symbolic dotted notation, sorting from toplevel domain backward
while maintaining the original fqdn, use s.t. like the following. The
Schwartzian Transform uses only O(N) join-reverse-split operation, compare
item 14, "Effective Perl Programming" by Joseph N. Hall. The downside is
that is needs more memory than the direct sort-join-reverse-split
approach (which needs O(N*log_2 N) .. O(N^2) processing time):

        foreach $client
            ( map { $_->[0] }
              sort { $a->[1] cmp $b->[1] }
              map { [ $_, join('.', reverse split/\./) ] }
              keys %clients ) {

---
Additionally, there might be something about log file processing (and
a "starting hint" for doing it yourself) for your gusto at:
	http://www.terena.nl/tech/d2-workshop/d2cache99/
Le deagh dh�rachd,
Dipl.-Ing. Jens-S. V�ckler (voeckler@rvs.uni-hannover.de)
Institute for Computer Networks and Distributed Systems
University of Hanover, Germany; +49 511 762 4726