Hello Ow at Neuromancer :-) and all Squid guys,
I promised to post here my Squid results.
I am trying to find out what a single Squid box can do - I have 3200
unique IPs and 240 HTTP requests per second during peek time.
I hande this load with one box, using WCCP to get the traffic to Squid
(so it is a transparet proxy - the evil thing :-).
My goal is to achive maximum BYTE hit ratio.
USED HARDWARE:
server: IBM x305
Processor: P4 2.8GHz
Memory: 4 GB (you can switch the swap off)
Hardisk: 2 x 36GB 10 RPM, 2 x 73 15 RPM scsi disks
Controler: IBM 71P8594 Kendall Card Option
<Adaptec 29320LP Ultra320 SCSI adapter>
Hardisk for logs: IDE 7200, DMA
Ethernet card: Broadcom TG3
OS: Linux Debian/woody, kernel 2.4.25, tproxy, wccp
Unfortunately, I do not have enough clients to reach the box limits.
But I can see that my median HIT time triples during peak time, which
probably means a disk bottleneck. But it will be the log disk, because I
use an IDE disk for log (with log_mime_header on, so I get an 500 MB
acccess.log per hour). And also the DNS server (see below) - DNS
responses are longer during peak time.
I know it is not good to combine different disk (15 rpm and 10 rpm), but
I had no choice.
Maximum performance I have achived (MRTG stats gaind by SNMP from squid):
Client HTTP Requests per second: 220
Server Requests per second: 210
Server In 1600 kBytes/s
(amount of traffic read from origin WWW servers)
System load: 3, occasionally rises to 10 and I don't know why
HTTP Out: 2000 kBytes/s
(amount of traffic written to clients)
HTTP Hits per second: 110
Squid generated Error Pages: 2 errors per second
Total accounted memory in GRI: 830 MB
Storage disk size: 164 GB
Storage Mem Size: 200 MB
HTTP I/O Number of Reads: 300 reads/sec
Number of Clients Accessing Cache: 3200
(unique IPs since Squid started)
HTTP all service time: 150 ms
HTTP miss service time: 490 ms
HTTP Near Miss Service Time: 8 ms
HTTP hit service time: 30 ms (average is 10 ms)
Byte Hit Ratio: 20% (heap LFUDA)
Request Hit Ratio: 64% (49% average)
Disks: 800 writes per second
Log disk (IDE - the bottleneck): 500 writes per sec
DNS service time: 70 ms (bottleneck)
Configuration (squid.conf)
maximum_object_size 200 MB
cache_mem 200 MB
cache_replacement_policy heap LFUDA
log_mime_hdrs on
store_avg_object_size 22 KB
store_dir_select_algorithm round-robin
(if least-load, it does not distribute the load properly)
cache_dir aufs /squidcache/cache1 28000 60 256
cache_dir aufs /squidcache/cache2 56000 60 256
cache_dir aufs /squidcache/cache3 28000 60 256
cache_dir aufs /squidcache/cache4 56000 60 256
Please notice that I encoutered a DNS issue.
Squid generated more requests per second (500?) than the DNS server was
able to accept (200?). My coleague administrator had to recompile it and
icrease the number. From this point, the squid performance has been
better in peak time. The HIT service time does not tripple, but double
only during peak time.
Another important think:
I had to renice all standard linux maintenance programs, e.g. in
/etc/crontab:
/usr/bin/nice -n 19 run-parts --report /etc/cron.daily
these tasks had negative impact to Squid.
Especially: /etc/cron.daily/standard
and /etc/cron.daily/find
(see /etc/checksecurity.conf
and /etc/updatedb.conf
- don't let find crawl through cache dirs)
Some stats from my cachemgr page follow.
I did not take them in the real peak time, unfortunately, but it was
close to :-)
I am happy to post more details if someone is interested in.
Best regards,
have a nice weekend,
Marji
Connection information for squid:
Number of clients accessing cache: 2925
Number of HTTP requests received: 13830996
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 5128.4
Average ICP messages per minute since start: 0.0
Select loop called: 38791775 times, 4.171 ms avg
Cache information for squid:
Request Hit Ratios: 5min: 54.0%, 60min: 52.2%
Byte Hit Ratios: 5min: 23.5%, 60min: 24.7%
Request Memory Hit Ratios: 5min: 4.9%, 60min: 6.5%
Request Disk Hit Ratios: 5min: 25.2%, 60min: 26.2%
Storage Swap size: 161338652 KB
Storage Mem size: 204820 KB
Mean Object Size: 22.54 KB
Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.08265 0.10281
Cache Misses: 0.46965 0.46965
Cache Hits: 0.01309 0.01164
Near Hits: 0.42149 0.42149
Not-Modified Replies: 0.00678 0.00562
DNS Lookups: 0.01609 0.01464
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 161816.436 seconds
CPU Time: 121702.940 seconds
CPU Usage: 75.21%
CPU Usage, 5 minute avg: 94.01%
CPU Usage, 60 minute avg: 93.22%
Process Data Segment Size via sbrk(): 915378 KB
Maximum Resident Size: 0 KB
Page faults with physical i/o: 935
Memory usage for squid via mallinfo():
Total space in arena: 185544 KB
Ordinary blocks: 183916 KB 9731 blks
Small blocks: 0 KB 0 blks
Holding blocks: 6012 KB 10 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 1627 KB
Total in use: 189928 KB 99%
Total free: 1627 KB 1%
Total size: 191556 KB
Memory accounted for:
Total accounted: 789021 KB
memPoolAlloc calls: 1795219551
memPoolFree calls: 1772798798
File descriptor usage for squid:
Maximum number of file descriptors: 8192
Largest file desc currently in use: 3489
Number of file desc currently in use: 3222
Files queued for open: 1
Available number of file descriptors: 4969
Reserved number of file descriptors: 100
Store Disk files open: 21
Internal Data Structures:
7160517 StoreEntries
35656 StoreEntries with MemObjects
35457 Hot Object Cache Items
7158519 on-disk objects
Received on Fri Jan 14 2005 - 02:44:28 MST
This archive was generated by hypermail pre-2.1.9 : Mon Mar 07 2005 - 12:59:35 MST