Re: [squid-users] log server

Hi, Hendrik...

On Sun, Aug 22, 2004 at 02:06:03AM +0200, Henrik Nordstrom wrote:
> >Well, built-in syslog only works for the cache.log so this is not an
> >option. I wrote a Perl script called "tail2syslog" (which I provide as a
> >Debian package but which is useful on other systems as well) which
> >follows the changes in the access.log (like a "tail -f") and forwards
> >them to a syslog server.
>
> I could not resist to beat you on this one. Perl is quite nice for this
> kind of hacks.

I probably deserve it. :)

> #!/usr/bin/perl -w
> use File::Tail;
> use Sys::Syslog;
>
> openlog "squid", "ndelay", "local0";
> my $ref=tie *FH,"File::Tail",(name=>"/usr/local/squid/var/logs/access.log");
>
> while (<FH>) {
> chomp;
> syslog "info", $_;
> }

This is nice and short. Just that what I did is a little more universal
(watches different files in different threads and is more customisable
via a config file). But generally these lines are all it takes.

But I'd like to bite back: why doesn't Squid offer to forward access.log
entries via syslog? Struggling about whose script is shorter is one
thing. But the reasons we need those crazy hacks anyway is something I
don't understand.

Lemma: Every decent network daemon can log to syslog! ;)

> >- the script sucks a lot of CPU on busy systems (we have 60 requests
> > per second and the script produces much higher load than Squid)
>
> This I have a hard time buying.. perl usually isn't that slow. But on the
> other hand the syslog protocol is very chatty (one message per line) so
> maybe...

You can buy it. Perl's speed is probably somewhere between C and shell.
But using File::Tail and sending 120 log lines per second to another
server is quite time consuming.

> >- the output will not be plain access.log output as syslog adds some
> > data before each line (so automated statistics tools won't work
> > without some conversion)
> >- long log lines may get cut (if you log mime headers or have very
> > long URLs)
>
> And log lines may get lost if logging over network.

The charme of UDP. :) Until the Squid dev team will offer syslog I think
NFS is the best solution.

 Christoph

-- 
~
~
".signature" [Modified] 3 lines --100%--                3,41         All