Re: [squid-users] strange reports after dl-malloc

Thanks Mark.

 Just while writing to you tried an old setting Henrick recommended.

client_persistent_connections off

from on to off, it gave me a very great difference, alas after 3,4 months of
messing around.
Load dropped by 50%.
tuning server_persistent_connections off
gave me another 25%, but

HTTP Requests (All): 0.61549 went bad to 1 + seconds so went back on
that. But have enabled pipeline prefetch.

http_port 0.0.0.0:3128
icp_port 0
udp_incoming_address 0.0.0.0
udp_outgoing_address 255.255.255.255
icp_query_timeout 0
maximum_icp_query_timeout 2000
mcast_icp_query_timeout 2000
dead_peer_timeout 10 seconds
cache_mem 188743680 bytes
cache_swap_low 90
cache_swap_high 95
maximum_object_size 6242304 bytes
minimum_object_size 0 bytes
maximum_object_size_in_memory 8192 bytes
ipcache_size 6500
ipcache_low 90
ipcache_high 95
fqdncache_size 1024
cache_replacement_policy lru
memory_replacement_policy lru
cache_dir diskd /var/squid/disk1 41000 73 256 Q1=64 Q2=72
cache_dir diskd /var/squid/disk2 41000 73 256 Q1=64 Q2=72
cache_dir diskd /var/squid/disk3 41000 73 256 Q1=64 Q2=72
cache_dir diskd /var/squid/disk4 41000 73 256 Q1=64 Q2=72
cache_access_log /dev/null
cache_log /usr/local/squid/var/logs/cache.log
cache_store_log none
emulate_httpd_log off
log_ip_on_direct on
mime_table /etc/squid/mime.conf
log_mime_hdrs off
pid_filename /usr/local/squid/var/logs/squid.pid
debug_options ALL,1
log_fqdn off
client_netmask 255.255.255.255
ftp_user proxy@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on
ftp_telnet_protocol on
dns_retransmit_interval 5 seconds
dns_timeout 45 seconds
hosts_file /etc/hosts
diskd_program /usr/local/squid/libexec/diskd
unlinkd_program /usr/local/squid/libexec/unlinkd
redirect_children 5
redirect_rewrites_host_header off
authenticate_cache_garbage_interval 3600 seconds
authenticate_ttl 3600 seconds
authenticate_ip_ttl 0 seconds
wais_relay_port 0
request_header_max_size 51200 bytes
request_body_max_size 0 bytes
quick_abort_min 16 KB
quick_abort_max 16 KB
quick_abort_pct 95
negative_ttl 300 seconds
positive_dns_ttl 21600 seconds
negative_dns_ttl 60 seconds
range_offset_limit 0 bytes
forward_timeout 150 seconds
connect_timeout 60 seconds
peer_connect_timeout 30 seconds
read_timeout 180 seconds
request_timeout 20 seconds
persistent_request_timeout 60 seconds
client_lifetime 3600 seconds
half_closed_clients off
pconn_timeout 60 seconds
shutdown_lifetime 90 seconds
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl PURGE method PURGE
acl localhost src 127.0.0.1
acl SSL_ports port 443
acl SSL_ports port 563
acl Safe_ports port 80
acl Safe_ports port 443
acl Safe_ports port 21
acl Safe_ports port 563
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 3130
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
reply_header_max_size 20480 bytes
reply_body_max_size 0 Allow all
cache_mgr hostmaster@
cache_effective_user squid
cache_effective_group squid
visible_hostname proxy
announce_period 31536000 seconds
announce_host tracker.ircache.net
announce_port 3131
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_single_host off
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
dns_testnames netscape.com
dns_testnames internic.net
dns_testnames nlanr.net
dns_testnames microsoft.com
logfile_rotate 10
tcp_recv_bufsize 0 bytes
err_html_text
memory_pools on
memory_pools_limit 0 bytes
forwarded_for on
log_icp_queries off
icp_hit_stale off
minimum_direct_hops 4
minimum_direct_rtt 400
cachemgr_passwd XXXXXXXXXX all
store_avg_object_size 13 KB
store_objects_per_bucket 20
client_db on
netdb_low 900
netdb_high 1000
netdb_ping_period 300 seconds
query_icmp off
test_reachability off
buffered_logs off
reload_into_ims off
icon_directory /usr/local/squid/share/icons
short_icon_urls off
error_directory /usr/local/squid/share/errors/English
maximum_single_addr_tries 1
snmp_port 3401
snmp_access Allow snmpmonitor
snmp_access Allow localhost
snmp_access Allow admin
snmp_access Deny all
snmp_incoming_address 0.0.0.0
snmp_outgoing_address 255.255.255.255
as_whois_server whois.ra.net
wccp_router ip address
wccp_version 4
wccp_incoming_address 0.0.0.0
wccp_outgoing_address 255.255.255.255
incoming_icp_average 6
incoming_http_average 4
incoming_dns_average 4
min_icp_poll_cnt 8
min_dns_poll_cnt 8
min_http_poll_cnt 8
max_open_disk_fds 0
offline_mode off
uri_whitespace strip
nonhierarchical_direct on
prefer_direct off
strip_query_terms on
coredump_dir none
redirector_bypass off
ignore_unknown_nameservers on
client_persistent_connections off
server_persistent_connections on
detect_broken_pconn on
balance_on_multiple_ip on
pipeline_prefetch on
request_entities off
high_response_time_warning 0
high_page_fault_warning 0
high_memory_warning 0 bytes
store_dir_select_algorithm round-robin
ie_refresh off
vary_ignore_expire off
sleep_after_fork 0

Still a long way to go......

Can someone figure out any goof ups in this config.

----- Original Message -----
From: "Elsen Marc" <elsen@imec.be>
To: "sk" <suresh.kumar@online.com.kh>
Cc: "Elsen Marc" <elsen@imec.be>
Sent: Thursday, December 09, 2004 7:33 PM
Subject: RE: [squid-users] strange reports after dl-malloc

>
> Hi Mark,
>
> Read in many places taht Dl-malloc may do better
> than shipped
> malloc. So compiled with one from the site.
> The one shipped with squid is 2.6.X and i've put in 2.7.x
>
> My worries are manyfold.
>
> 3 months of prodding along with this box, still no noticeable
> improvements.
>
> IBM X235 with dual Xeon 3.06, 2.5 G Ram,
> 4 * 73G HDD( 10K) set to fill till 55%
> Using wccp,. latest from site.
>
> The squid behaves like 1% cpu per request.
> for around 50 rqst/sec - cpu around 50 -55
> and above 80/sec its near 96 -99.
> Ive seen 125 hits maximum on this box.
>
> Have read all these 3 months and tried so many options, still no
> improvement. Have read Henrick written somewhere taht such a
> box should give
> 400rqst/sec without problems.
>
>

 I can't be of much help. I am for instance not familiar with
wccp and I don't use it. Perhaps the problem area is in there,
you never know.
I can only offer general guidelines as in , make sure that squid
is not swapping and that there is enough phys. mem w.r.t. cache size
(see FAQ).
If CPU situation is 99% then, perhaps (very perhaps) :

   % squid -k debug ; sleep 2; squid -k debug

will 'offer' a 2 second full debug session (output in cache.log).
Its not all guaranteed that this will offer you immediate clues
as what squid is doing , and what keeps it busy.
It may give an indication, yet perhaps not either.

Marc.
Received on Thu Dec 09 2004 - 05:48:18 MST