[squid-users] cache removal procedure

From: Pat Lendon <[email protected]>
Date: Mon, 20 Oct 2003 15:08:50 -0400

Hi All,

Is it possible to set specific times to run cache removal (lru)?

Reason being, at random times the squid process cpu is a high 40% and
greater. Sometimes the squid process uses 99% cpu, but this is more rare.
Normally the squid process uses 10% or less of cpu.

Is it possible that the cache is causing the high cpu and if so, could
the users use cache without lru running? And then later in
evening/night activate cache removal lru?

Should I switch the cache replacement policy from "lru" to GDSF or LFUDA ?

The high cpu has occured for quite some time - with no difference on
which squid version or system machines. I've always checked cache log,
but it has no errors. Just a lot of "sslReadServer: FD 149: read
failure: (104) Connection reset by peer". The random high cpu for
squid process is occurring on our main parent squid cache system. We've
got 20 remote systems running squid caches that then proxy thru to our
main parent system.

I just placed squid in debug mode (while the squid process was using a
lot of cpu). Lots of log info to look at.

Squid Cache: Version 2.5.STABLE2-20030402
configure options: --prefix=/s1/squid2.5.2 --enable-delay-pools
--enable-snmp --enable-basic-auth-helpers=LDAP
--enable-basic-auth-helpers=NCSA --enable-storeio=aufs

System OS is RedHat 7.1.2 with kernel 2.4.9-31smp. cpu is AMD Athlon
with cpu MHz 1533.431. Mem: 1027936K av, 1022528K used, 5408K free.
system has raid.

Any suggestions or ideas would be great. I've never been able to
figure out why squid process sometimes uses a lot of cpu. Listed is
the Store Dir Stats and cache mgr info.

Thank you !

Store Directory Statistics:
Store Entries : 428328
Maximum Swap Size : 5242880 KB
Current Store Swap Size: 4718588 KB
Current Capacity : 90% used, 10% free

Store Directory #0 (aufs): /d1/squid_cache
FS Block Size 4096 Bytes
First level subdirectories: 16
Second level subdirectories: 256
Maximum Size: 5242880 KB
Current Size: 4718588 KB
Percent Used: 90.00%
Filemap bits in use: 428190 of 524288 (82%)
Filesystem Space in use: 11910012/17488652 KB (68%)
Filesystem Inodes in use: 453971/2223872 (20%)
Flags: SELECTED
Removal policy: lru
LRU reference age: 4.91 days

Squid Object Cache: Version 2.5.STABLE2-20030402
Start Time: Mon, 08 Sep 2003 17:14:09 GMT
Current Time: Mon, 20 Oct 2003 17:44:54 GMT
Connection information for squid:
        Number of clients accessing cache: 440
        Number of HTTP requests received: 25319108
        Number of ICP messages received: 18278
        Number of ICP messages sent: 18278
        Number of queued ICP replies: 0
        Request failure ratio: 0.00
        Average HTTP requests per minute since start: 418.4
        Average ICP messages per minute since start: 0.6
        Select loop called: 366800967 times, 9.898 ms avg
Cache information for squid:
        Request Hit Ratios: 5min: 32.9%, 60min: 32.0%
        Byte Hit Ratios: 5min: 6.8%, 60min: 7.2%
        Request Memory Hit Ratios: 5min: 2.0%, 60min: 1.0%
        Request Disk Hit Ratios: 5min: 15.1%, 60min: 19.1%
        Storage Swap size: 4718588 KB
        Storage Mem size: 8152 KB
        Mean Object Size: 11.08 KB
        Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
        HTTP Requests (All): 0.16775 0.15888
        Cache Misses: 0.20843 0.22004
        Cache Hits: 0.01387 0.01387
        Near Hits: 0.12783 0.14252
        Not-Modified Replies: 0.00865 0.00865
        DNS Lookups: 0.00779 0.00669
        ICP Queries: 0.00000 0.00000
Resource usage for squid:
        UP Time: 3630645.350 seconds
        CPU Time: 200231.640 seconds
        CPU Usage: 5.52%
        CPU Usage, 5 minute avg: 14.85%
        CPU Usage, 60 minute avg: 15.72%
        Process Data Segment Size via sbrk(): 90135 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 9365
Memory usage for squid via mallinfo():
        Total space in arena: 90135 KB
        Ordinary blocks: 67305 KB 2854 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 324 KB 2 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 22829 KB
        Total in use: 67629 KB 75%
        Total free: 22829 KB 25%
        Total size: 90459 KB
Memory accounted for:
        Total accounted: 53066 KB
        memPoolAlloc calls: 3782602115
        memPoolFree calls: 3781280094
File descriptor usage for squid:
        Maximum number of file descriptors: 1024
        Largest file desc currently in use: 409
        Number of file desc currently in use: 335
        Files queued for open: 0
        Available number of file descriptors: 689
        Reserved number of file descriptors: 100
        Store Disk files open: 0
Internal Data Structures:
        426041 StoreEntries
          1770 StoreEntries with MemObjects
          1759 Hot Object Cache Items
        425934 on-disk objects
Received on Mon Oct 20 2003 - 12:54:18 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 17:20:33 MST