Re: me too (was Re: [SQU] Tune up Squid Cache Hit)

From: M. Yu <[email protected]>
Date: Mon, 26 Feb 2001 19:42:37 +0800

Hi all,

Marc, thanks for the sample config. Something weird just came up. As I've
mentioned I upgraded the hardware but still used the Squid that came with
RH6.2 (2.3.STABLE1-5) and using the original squid.conf with the added
sample you gave. Suddenly, instead of seeing a hit rate of 50% or so, my
5min request hit ratio is now 1.1% and 60min is now 0.9%. My byte hit ratio
is -5.2% for 5min and -2.5% for 60min. These are all under General
Runtime's cache information for squid in cachemgr.cgi.

What's also weird is that my /cache1 (formatted partition size 17Gb) which
is configured to only be 60% used by Squid (10200 in cache_dir), is now 0%
used (16Mb is used as reported by df -h) and doing an ls -lahR1 in /cache1
reveals all cache subdirs are empty.

I guess it's safe to assume my squid is NOT caching anything. Please help
anyone. What do I need to look at? My bandwidth usage is hitting the roof
and I can already almost see my ISP admin frowning upon our link bursting to
over twice what we're paying for (256K burstable to 512K) at all hours of
the day.

I have included my squid.conf below without any changes (well except for the
password). As you can see pretty much everything is default. The lines I
added as indicated below was from a sample config Marc van Selm gave me to
try and increase my hit rate to go close to 50%. My df -h output is also
included below as well as cachemgr's output from General Runtime Info

Thanks for any help guys!

M. Yu

--- df -h output ---

[sa@cache1 19:46:52 ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/hda5 2.9G 343M 2.4G 12% /
/dev/hda1 7.6M 647k 6.5M 9% /boot
/dev/sda1 17G 16M 16G 0% /cache1
/dev/hda6 6.2G 710M 5.1G 12% /var

--- output from cachemgr.cgi ---

Squid Object Cache: Version 2.3.STABLE1

Start Time: Sat, 24 Feb 2001 16:11:33 GMT
Current Time: Mon, 26 Feb 2001 11:23:04 GMT

Connection information for squid:
 Number of clients accessing cache: 79
 Number of HTTP requests received: 611966
 Number of ICP messages received: 19
 Number of ICP messages sent: 19
 Number of queued ICP replies: 0
 Request failure ratio: 0.00%
 HTTP requests per minute: 236.1
 ICP messages per minute: 0.0
 Select loop called: 9051723 times, 17.178 ms avg
Cache information for squid:
 Request Hit Ratios: 5min: 1.1%, 60min: 0.9%
 Byte Hit Ratios: 5min: -5.2%, 60min: -2.5%
 Storage Swap size: 0 KB
 Storage Mem size: 11748 KB
 Storage Replacement Threshold: 0.000000
 Mean Object Size: 0.00 KB
 Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
 HTTP Requests (All): 7.79421 7.38455
 Cache Misses: 7.79421 7.38455
 Cache Hits: 93.38594 5.63745
 Near Hits: 0.00000 3.86308
 Not-Modified Replies: 0.00000 0.00919
 DNS Lookups: 0.00278 0.00190
 ICP Queries: 0.00000 0.00000
Resource usage for squid:
 UP Time: 155490.518 seconds
 CPU Time: 6469.270 seconds
 CPU Usage: 4.16%
 CPU Usage, 5 minute avg: 19.84%
 CPU Usage, 60 minute avg: 16.33%
 Maximum Resident Size: 0 KB
 Page faults with physical i/o: 845
Memory usage for squid via mallinfo():
 Total space in arena: 30945 KB
 Ordinary blocks: 25020 KB 2711 blks
 Small blocks: 0 KB 0 blks
 Holding blocks: 176 KB 1 blks
 Free Small blocks: 0 KB
 Free Ordinary blocks: 5925 KB
 Total in use: 25196 KB 81%
 Total free: 5925 KB 19%
Memory accounted for:
 Total accounted: 19337 KB
File descriptor usage for squid:
 Maximum number of file descriptors: 1024
 Largest file desc currently in use: 569
 Number of file desc currently in use: 512
 Files queued for open: 0
 Available number of file descriptors: 512
 Reserved number of file descriptors: 100
 Store Disk files open: 0
Internal Data Structures:
   2270 StoreEntries
   2270 StoreEntries with MemObjects
   2045 Hot Object Cache Items
      0 Filemap bits set
      0 on-disk objects

--- start of SQUID.CONF ---

# NETWORK OPTIONS
# --------------------------------------------------------------------------

---
http_port 3128
icp_port 3130
# OPTIONS WHICH AFFECT THE NEIGHBOR SELECTION ALGORITHM
# --------------------------------------------------------------------------
---
# OPTIONS WHICH AFFECT THE CACHE SIZE
# --------------------------------------------------------------------------
---
cache_mem  12 MB
cache_swap_low  94
cache_swap_high 98
maximum_object_size 3000000 KB
ipcache_size 16384
ipcache_low  90
ipcache_high 95
fqdncache_size 16384
# LOGFILE PATHNAMES AND CACHE DIRECTORIES
# --------------------------------------------------------------------------
---
cache_dir ufs /cache1 10200 16 256
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log none
emulate_httpd_log off
mime_table /etc/squid/mime.conf
log_mime_hdrs on
useragent_log /var/log/squid/useragent.log
pid_filename /var/run/squid.pid
debug_options ALL,1
log_fqdn on
client_netmask 255.255.255.255
# OPTIONS FOR EXTERNAL SUPPORT PROGRAMS
# --------------------------------------------------------------------------
---
ftp_user Squid@
ftp_list_width 64
cache_dns_program /usr/lib/squid/dnsserver
dns_children 10
dns_defnames off
unlinkd_program /usr/lib/squid/unlinkd
# OPTIONS FOR TUNING THE CACHE
# --------------------------------------------------------------------------
---
request_header_max_size 10 KB
request_body_max_size 7 MB
reply_body_max_size 0
replacement_policy LFUDA
negative_ttl 5 minutes
positive_dns_ttl 6 hours
negative_dns_ttl 5 minutes
range_offset_limit 0 KB
# TIMEOUTS
# --------------------------------------------------------------------------
---
connect_timeout 120 seconds
peer_connect_timeout 30 seconds
siteselect_timeout 4 seconds
read_timeout 15 minutes
request_timeout 30 seconds
client_lifetime 1 day
half_closed_clients off
pconn_timeout 120 seconds
ident_timeout 10 seconds
shutdown_lifetime 30 seconds
# ACCESS CONTROLS
# --------------------------------------------------------------------------
---
acl cable_subscribers src 64.110.88.0/255.255.255.128
acl servers src 64.110.82.80/255.255.255.240
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl SSL_ports port 443 563
acl Safe_ports port 80 21 443 563 70 210 1025-65535
acl Safe_ports port 280  # http-mgmt
acl Safe_ports port 488  # gss-http
acl Safe_ports port 591  # filemaker
acl Safe_ports port 777  # multiling http
acl CONNECT method CONNECT
http_access allow cable_subscribers
http_access allow manager servers
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT
http_access allow localhost
http_access deny all
icp_access deny all
miss_access allow all
ident_lookup_access deny all
# ADMINISTRATIVE PARAMETERS
# --------------------------------------------------------------------------
---
cache_mgr cachemgr@websprinter.net
cache_effective_user squid
cache_effective_group squid
visible_hostname cache.websprinter.net
unique_hostname cache1.websprinter.net
# MISCELLANEOUS
# --------------------------------------------------------------------------
---
logfile_rotate 10
append_domain .websprinter.net
tcp_recv_bufsize 0 bytes
memory_pools on
forwarded_for off
log_icp_queries off
icp_hit_stale off
minimum_direct_hops 1
cachemgr_passwd whatever all
store_avg_object_size 13 KB
store_objects_per_bucket 50
client_db on
netdb_low 900
netdb_high 1000
query_icmp off
test_reachability off
buffered_logs off
acl local-servers dstdomain websprinter.net
always_direct allow local-servers
minimum_retry_timeout 5 seconds
maximum_single_addr_tries 3
snmp_port 0
uri_whitespace strip
ignore_unknown_nameservers on
# Added by MYu on 2000-02-21
# Don't cache these.
acl QUERY url_regex cgi-bin \?
no_cache deny QUERY
# Handle dynamic html (don't cache and always go direct)
acl DHTML urlpath_regex \.(jhtml|phtml|shtml) \.asp \? cgi-bin \.cgi \.pl
always_direct allow DHTML
no_cache deny DHTML
# Special cases to improve mega-sites
# News sites
refresh_pattern ^http://.*\.cnn\.com  360 50% 4320 override-lastmod
override-expire ignore-reload
refresh_pattern ^http://news\.bbc\.co\.uk 360 50% 4320 override-lastmod
override-expire ignore-reload
# Inefficient sites
refresh_pattern microsoft   1080 150% 10080 override-expire override-lastmod
override-expire
refresh_pattern msn\.com   4320 150% 10080 override-expire override-lastmod
override-expire
# Adds from doublclick
refresh_pattern ^http://.*\.doubleclick\.net   10080 300% 40320
override-expire override-lastmod override-expire ignore-reload
# Pointcast
refresh_pattern ^http://.*FIDO   360 1000% 480
refresh_pattern \.phtml$   240 200% 10080
#
# Audio:
#     these min 7 days, 150% of age, max 28 days (not likely to change)
refresh_pattern \.avi$   10080 150% 40320
refresh_pattern \.mov$   10080 150% 40320
refresh_pattern \.wav$   10080 150% 40320
refresh_pattern \.mp3$   10080 150% 40320
refresh_pattern \.qtm$   10080 150% 40320
refresh_pattern \.mid$   10080 150% 40320
refresh_pattern \.viv$   10080 150% 40320
refresh_pattern \.mpg$   10080 150% 40320
refresh_pattern \.jpg$   10080 150% 40320
refresh_pattern \.rar$   10080 150% 40320
refresh_pattern \.ram$   10080 150% 40320
refresh_pattern \.r[0-9][0-0]$  10080 150% 40320
#
# gif are most likely in html lets do them a bit less (more likely)
# asis are most likely in html lets do them a bit less (more likely)
#     min 1 day, 50% of age and max 14 days
refresh_pattern ^http://.*\.gif$ 1440 50% 20160
refresh_pattern ^http://.*\.asis$ 1440 50% 20160
# other gifs a bit more. (ftp downloaded images?)
refresh_pattern \.gif$   10080 300% 40320
#
# Txt
refresh_pattern \.txt$   1440 100% 20160 override-lastmod
#
# Downloads: these PC-things
#     min 2 days, 200% of age, max 28 days (not so likely to change)
refresh_pattern \.zip$   2880 200% 40320
refresh_pattern \.arj$   2880 200% 40320
refresh_pattern \.exe$   2880 200% 40320
#
# Unix distributions: not likely at all to change:
#     min 7 days, 200% of age, max 28 days
refresh_pattern \.tgz$   10080 200% 40320
refresh_pattern \.gz$   10080 200% 40320
refresh_pattern \.tgz$   10080 200% 40320
refresh_pattern \.tar$   10080 200% 40320
refresh_pattern \.Z$   10080 200% 40320
#
# Other stuff
# PDF min 6 hours, 100% of last modified max 28 days and enforce min
(override-lastmod)
refresh_pattern \.pdf$   720 100% 40320 override-lastmod
#
# keep ftp/gopher (mainly directories) fresh for 1 day, 50%, max 10 days
refresh_pattern ^ftp://   1440 50% 14400 override-lastmod
refresh_pattern ^gopher://  1440 50% 14400 override-lastmod
#
# all the rest 6 hours, 50% of age, max 5 days
refresh_pattern .   360 50% 7200 override-lastmod
#
quick_abort_min 100 KB
quick_abort_max 750 KB
quick_abort_pct 60
#
reload_into_ims on
--
To unsubscribe, see http://www.squid-cache.org/mailing-lists.html
Received on Mon Feb 26 2001 - 04:44:50 MST

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:58:11 MST