Re: Faster analyzing tools needed

David J N Begley wrote:
>
> On Tue, 18 Feb 1997, Hans Verbrugge wrote:
>
> > We experienced the same thing using the tools from Duane
> > Wessels. Parsing a 40Mb+ logfile took 6 hours. However, most of
> > the time was spended on hostnamelookups and creating assoc
> [...]
> > You miss all hostnames in the logging analysis, but it will run
> > much faster. 15 minutes actualy.
>
> What are the chances of those processing scripts piggy-backing on the
> running dnsservers and/or Squid's IP/FQDN caches to help speed things up?
I was thinking of creating a named pipe for the access log and doing it
on the fly like that...

The top script could use hashed arrays, this way it would only swap out the
relevant bits into ram, so you could get away with a large increase in
"total virtual size" of squid and associated programs, but without much
change in the "total real memory used"... I hope :)

        Oskar

=============================================
'experience made art, but inexperience luck.'
=============================================
Received on Tue Feb 18 1997 - 03:58:49 MST