Re: Plese help with Fatal error

On Thu, 15 Jul 1999, Henrik Nordstrom wrote:

> Dirk Moerenhout wrote:
> >
> > To make sure everybody knows what I normally do:
> > - I compile linux + ac-patch
> > - I compile glibc 2.1.1 against the new kernel
> > - I change /usr/include/bits/types.h
>
> There is no point in rebuilding glibc unless you also tune it's view of
> __FD_SETSIZE. The reason you need to rebuild glibc is limitations of
> FD_SETSIZE, not the kernel. You may perfectly well build a glibc
> supporting a trillion of filedescriptors using a machine with a kernel
> only supporting a few (100 should be enought) as a build machine.

There is another reason to rebuild it. Most people run a distribution
based on linux 2.0 and as a result the poll() call in their glibc is not
based on calling poll() in the kernel (which exists as of 2.2) but is
emulated by using select().

But if you rebuild glibc after you changed everything to 4096 in the
kernel source and rebuild your kernel why doesn't it pick up the 4096
itself? Should glibc be recompiled in a way where you alter it before you
compiled it then? So that the includes immediately give 4096 instead of
1024.

What I did now is: recompile linux with 4096 as FD_SETSIZE and 4096 as
NR_OPEN in the one place where it is still 1024 (Somewhere else it is
defined as 1024*1024). I also recompiled glibc 2.1.1 against this and
rebooted.

What is weird is that SEGV's happen at all times. So even when there are
less then 1024 FD's in use and don't necessarily happen when more than
1024 FD's are in use. Also when I start squid on a glibc 2.1.1 + kernel
2.2 box that has no cache and so it will have 500KB of accounted memory
and has somewhat more than 4MB in use. So it starts out with a 4.5MB leak?

The now just restarted squid on my production box gives:
Resource usage for squid:
        UP Time: 188.304 seconds
        CPU Time: 13.010 seconds
        CPU Usage: 6.91%
        CPU Usage, 5 minute avg: 7.02%
        CPU Usage, 60 minute avg: 7.02%
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 347
Memory usage for squid via mallinfo():
        Total space in arena: 9667 KB
        Ordinary blocks: 9607 KB 36 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 420 KB 1 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 59 KB
        Total in use: 10027 KB 104%
        Total free: 59 KB 1%
Memory accounted for:
        Total accounted: 5318 KB

--- and somewhat later ---

Resource usage for squid:
        UP Time: 510.250 seconds
        CPU Time: 30.150 seconds
        CPU Usage: 5.91%
        CPU Usage, 5 minute avg: 5.00%
        CPU Usage, 60 minute avg: 5.70%
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 352
Memory usage for squid via mallinfo():
        Total space in arena: 14079 KB
        Ordinary blocks: 13838 KB 103 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 420 KB 1 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 240 KB
        Total in use: 14258 KB 101%
        Total free: 240 KB 2%
Memory accounted for:
        Total accounted: 8695 KB

How do I use the leakfinding code? I've taken a quick look at it but it
has no points whatsoever on how you can track something and I am not
familiar enough witch squid-code to just plug it in. Can somebody give me
a simple example of how I can do tracking?

I now have a new Dual Xeon 550 to set up as a proxy so I do have a machine
to work on but the new machine should go into production ASAP and I'd love
to get everything solved first.

Dirk Moerenhout ///// System Administrator ///// Planet Internet NV
Received on Thu Jul 15 1999 - 02:21:56 MDT