Re: [squid-users] Squid eating too much memory

Tek Bahadur Limbu wrote:
> Hi Rihad,
>
> rihad wrote:
>> CacheMgr output:
>> Memory accounted for:
>> Total accounted: 1323944 KB
>>
>> Memory usage using top(1):
>> PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU
>> COMMAND
>> 29601 squid 29 20 0 2533M 2465M kserel 92:12 0.00% squid
>>
>>
>> Almost twice as much memory! Any hints?
>
> What is your settings for the following parameters?
>
> cache_mem
> maximum_object_size
> maximum_object_size_in_memory
> cache_replacement_policy
> memory_replacement_policy
> ipcache_size
> fqdncache_size
>
cache_mem 1000 MB
maximum_object_size 100 MB
# maximum_object_size_in_memory 8 KB
cache_replacement_policy heap LFUDA
memory_replacement_policy heap LFUDA
# ipcache_size 1024
# fqdncache_size 1024

> Which storage scheme are you using?
>
aufs

>>
>> Number of clients accessing cache: 517
>> Normally load is within 500-1500 clients.
>
> How long is your Squid process been running?
>
3 to 4 days, but it doesn't matter: I've had weeks or months of uptime,
too. Eventually the process hits its memory size limit (kern.maxdsiz)
set at 2.5 gigs at the moment and croaks (and restarts) unless I
decrease cache_mem. There's little point in using less RAM as the box
has ~3.3 gigs of it, and is dedicated to Squid (and its dedicated dnscache).

> Your server's memory usage seems unusually high even for 1500 clients.
> My proxy server which serves 2000-3000 clients is currently using 650 MB
> of memory despite running for more than 55 days.
>
How big is your cache store?

>
>>
>> Squid 2.6.16
>> FreeBSD 6.2-RELEASE-p8
>
> What is your machines specs?
>
# grep -e ^CPU: -e memory -e ^ad /var/run/dmesg.boot
CPU: Intel(R) Pentium(R) D CPU 2.80GHz (2813.85-MHz 686-class CPU)
real memory = 3489595392 (3327 MB)
avail memory = 3414970368 (3256 MB)
ad4: 238475MB <Seagate ST3250824AS 3.AAE> at ata2-master SATA150
ad6: 238475MB <Seagate ST3250824AS 3.AAE> at ata3-master SATA150

Two cache_dir's lie on both disks (25 gigs each at the moment, but I'm
planning for more).

>>
>> Port compiled with all options unchecked and both AUFS & KQUEUE checked
>> (as per "make config" and /var/db/ports/squid/options)
>
> Can you post the full output of "squidclient mgr:info"?
>
Squid Object Cache: Version 2.6.STABLE16
Start Time: Thu, 18 Oct 2007 17:10:09 GMT
Current Time: Sun, 21 Oct 2007 09:39:51 GMT
Connection information for squid:
         Number of clients accessing cache: 517
         Number of HTTP requests received: 9410288
         Number of ICP messages received: 0
         Number of ICP messages sent: 0
         Number of queued ICP replies: 0
         Request failure ratio: 0.00
         Average HTTP requests per minute since start: 2431.8
         Average ICP messages per minute since start: 0.0
         Select loop called: 74113837 times, 3.133 ms avg
Cache information for squid:
         Request Hit Ratios: 5min: 35.6%, 60min: 37.1%
         Byte Hit Ratios: 5min: 21.4%, 60min: 18.1%
         Request Memory Hit Ratios: 5min: 12.0%, 60min: 15.0%
         Request Disk Hit Ratios: 5min: 22.9%, 60min: 26.2%
         Storage Swap size: 22427274 KB
         Storage Mem size: 1023900 KB
         Mean Object Size: 13.62 KB
         Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
         HTTP Requests (All): 0.08265 0.08265
         Cache Misses: 0.17711 0.18699
         Cache Hits: 0.00463 0.00562
         Near Hits: 0.12106 0.14252
         Not-Modified Replies: 0.00286 0.00379
         DNS Lookups: 0.11405 0.10906
         ICP Queries: 0.00000 0.00000
Resource usage for squid:
         UP Time: 232181.468 seconds
         CPU Time: 5971.253 seconds
         CPU Usage: 2.57%
         CPU Usage, 5 minute avg: 3.98%
         CPU Usage, 60 minute avg: 3.55%
         Process Data Segment Size via sbrk(): -1583756 KB
         Maximum Resident Size: 2549632 KB
         Page faults with physical i/o: 3799
Memory accounted for:
         Total accounted: 1333488 KB
         memPoolAlloc calls: 1090270339
         memPoolFree calls: 1082572188
File descriptor usage for squid:
         Maximum number of file descriptors: 11072
         Largest file desc currently in use: 1715
         Number of file desc currently in use: 1622
         Files queued for open: 0
         Available number of file descriptors: 9450
         Reserved number of file descriptors: 100
         Store Disk files open: 41
         IO loop method: kqueue
Internal Data Structures:
         1652462 StoreEntries
         200285 StoreEntries with MemObjects
         200125 Hot Object Cache Items
         1646504 on-disk objects

>
> Thanking you...
>
Thank you too!
Received on Sun Oct 21 2007 - 03:42:01 MDT