Re: [squid-users] SQUID makes few slow connections (see video)

On Wed, Jun 2, 2010 at 06:11, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On Tue, 1 Jun 2010 23:50:35 +0200, Tytus Rogalewski <tytanick_at_gmail.com>
> wrote:
>> I thik that this is something to do it with many requests, because
>> half of images are loaded imiedietly (with hits MISS) , and rest are
>> loaded after few seconds (also MISSED) - missed because i have erased
>> cache. and is page is cached - it works fine. Only NEW requests are
>> working slowly
>
> Sounds like normal network delays. Horribly long though one there being
> 12s.
>
> As asked earlier are you forward-proxying or reverse-proxying with this
> Squid?
> ie, are you providing access to a local web server where these pages are
> hosted?
> �or just trying to get internal clients good access to the external site?
>
> <snip earlier provided traces>
>
>>> ===========================
>>> SQUID SETTINGS
>>>
>>> http_port 8123 transparent
>>> cache_mgr tytan..._at_gmail.com
>>> visible_hostname sky-link.net
>>> error_directory /usr/share/squid/errors/Polish_clean
>>>
>>> cache_mem 1500 MB # ilosc ramu
>>> memory_pools on
>>> memory_pools_limit 100 MB
>>> cache_swap_low �94
>>> cache_swap_high 98
>>> fqdncache_size 51200
>>> ipcache_size 51200
>>> ipcache_low 90
>>> ipcache_high 95
>>> pipeline_prefetch on
>
> mumble. This _might_ be affecting lag time since pipelining accepts a
> second request and starts it's timers before the first request has finished
> and the second can actually start being handled.
> Squid will still be able to handle a small number of client-pipelined
> requests with this turned off.
>
>>> minimum_object_size 0 KB
>>> maximum_object_size 50000 KB
>>> maximum_object_size_in_memory 512 KB
>>> half_closed_clients on
>>> shutdown_lifetime 2 seconds
>>> #ident_timeout 1 seconds
>>> cache_replacement_policy heap LFUDA
>>> memory_replacement_policy heap lru
>>> #cache_dir aufs /mnt/dysk/squid 100000 256 256
>>> cache_dir aufs /mnt/dysk/squid 20000 32 256
>>> #quick_abort_min 200 KB
>>> #quick_abort_max 200 KB
>>> quick_abort_min 0 KB
>>> quick_abort_max 0 KB
>>>
>>> ### OPCJE ###
>>> icp_port 0
>>> htcp_port 0
>>> icp_access deny all
>>> htcp_access deny all
>>> snmp_port 0
>>> snmp_access deny all
>>>
>>> ### LOGI ###
>>> access_log /var/log/squid/access.log
>>> cache_log /var/log/squid/cache.log
>>> cache_store_log /var/log/squid/store.log
>
> store.log is not very useful for most and wastes disk IO resources.
> You can probably set this to:
> �cache_store_log none
>
>>> log_fqdn on # pelne nazwy domen w logach
>
> Forces rDNS lookups to log things. This can add lag.
>
>>> #debug_options ALL,1
>>>
>>> ie_refresh on
>>> cache_effective_user squid
>>> cache_effective_group squid
>>> negative_ttl 1 minutes
>>> positive_dns_ttl 24 hours
>>> negative_dns_ttl 1 minutes
>
> Storing DNS results for longer than the authoritive website allows can
> lead you to the lag problem, if the web server IP has changed and Squid
> struggles to find a bad route to the old IP.
>
>>>
>>> ## ACL
>>> acl manager proto cache_object
>>> acl localhost src 127.0.0.1/255.255.255.255
>>> acl sky-link src 192.168.0.0/255.255.255.0 #Nasza LAN
>>> acl SSL_ports port 443 563
>>> acl Safe_ports port 21 80 442 563 70 210 280 488 591 777 1025-65535
>>> acl CONNECT method CONNECT
>>> #acl MCONN maxconn 300
>>> # Youtube ACL
>>> acl youtube_domains
>>> dstdomain .youtube.com .googlevideo.com .ytimg.com .wrzuta.pl
>>>
>>> ### ADL Blokada Reklam ###
>>> acl allowlist_urlregex url_regex "/etc/squid/acl/
>>> allowlist_urlregex.acl"
>>> acl allowlist_dstdomain dstdomain "/etc/squid/acl/
>>> allowlist_dstdomain.acl"
>>> acl denylist_urlregex url_regex "/etc/squid/acl/denylist_urlregex.acl"
>>> acl denylist_dstdomain dstdomain "/etc/squid/acl/
>>> denylist_dstdomain.acl"
>>>
>>> ## HTTP ACCESS
>>> http_access allow allowlist_dstdomain
>>> http_access allow allowlist_urlregex
>>> http_access deny denylist_urlregex
>>> http_access deny denylist_dstdomain
>
> �Optimization Tip #1:
> � �place regex rules as late as possible.
>
> �Meaning:
> �http_access deny denylist_dstdomain
> �http_access deny denylist_urlregex
>
>>> http_access allow sky-link
>>> http_access allow manager localhost
>>> http_access deny manager
>>> http_access deny CONNECT !SSL_ports
>>> http_access deny !Safe_ports
>>> http_access deny youtube_domains
>
> You access lists can be re-ordered to gain a fair bit of speed on some
> requests. Not sure if its related to the ones showing you problems
> though...
>
>
> �# These rules are the security boundaries preventing nasty abusive
> behaviour
> �# by your clients intentionally or when infected.
> �# They also form the second level of DDoS protection for your Squid.
> �http_access allow manager localhost
> �http_access deny manager
> �http_access deny CONNECT !SSL_ports
> �http_access deny !Safe_ports
>
> I have to assume that since you are using "transparent" option, your proxy
> is forward-only. (It's a giant security breach of your network to do
> reverse-proxy that way.)
>
> �http_access deny !sky-link
> �http_access allow allowlist_dstdomain
> �http_access allow allowlist_urlregex
> �http_access deny denylist_dstdomain
> �http_access deny denylist_urlregex
> �http_access allow sky-link
> �http_access deny all
>
> Note how the youtube rule is now missing, it was not having any affect on
> the LAN as defined by sky-link ACL.
>
>>>
>>> #tcp_outgoing_address 79.190.121.50
>>> #zph_tos_local 8
>>> #zph_tos_peer 0
>>> #zph_tos_parent off
>>>
>>> #vary_ignore_expire on
>>> #relaxed_header_parser on
>>> #server_persistent_connections off
>>>
>>> request_header_max_size 100 KB
>>> refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp)$ 10080 �90% � �43200
>>> refresh-ims
>>> refresh_pattern -i \.(iso|avi|wav|ogg|mp3|mp4|mpg|mpeg|swf|flv|x-flv)$
>>> 10080 �90% � �43200 refresh-ims
>>> refresh_pattern -i \.(deb|rpm|exe|zip|tar|gz|bz2|tgz|ram|rar|bin|ppt|
>>> doc|tiff)$ 10080 �90% � �43200 refresh-ims
>>> refresh_pattern windowsupdate.com/.*\.(cab|exe|dll) � � � � � � 10080
>>> 90% � �43200 refresh-ims
>>> refresh_pattern download.microsoft.com/.*\.(cab|exe|dll) � � � �10080
>>> 90% � �43200 refresh-ims
>>> refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll) 10080
>>> 90% � �43200 refresh-ims
>>> refresh_pattern symantecliveupdate.com/.*\.(zip|exe) � � � � � �10080
>>> 90% � �43200 refresh-ims
>>> refresh_pattern windowsupdate.com/.*\.(cab|exe) � � � � � � � � 10080
>>> 90% � �43200 refresh-ims
>>> refresh_pattern download.microsoft.com/.*\.(cab|exe) � � � � � �10080
>>> 90% � �43200 refresh-ims
>>> refresh_pattern avast.com/.*\.(vpu|vpaa) � � � � � � � � � � � �10080
>>> 90% � �43200 refresh-ims
>
> You are missing the refresh_pattern that ensures your Squid behaves
> correctly when handling broken dynamic pages:
>
> �refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>
>>> refresh_pattern . � � � � � � � � � � � � � � � � � � � � � � � 0
>>> 90% � �43200 refresh-ims
>>>
>
> Amos
>

Hi,

You said "turned off redirect". �Can you elaborate?

Also, you did not past the info from "General Runtime Info" on
CacheMgr. That would be also be helpful in troubleshooting.

Amos did a wonderful line by line analysis of your config. That
should help a lot.

The main argument in favor of 2.7x is the COSS storage, which is very
efficient in handling small (less then 256kb in my case) files. Also.
2.7.x is not an "older" Squid. At some point of the 2.5 releases, the
developers felt it necessary to rewrite a lot of the code in the
2.HEAD that was unnecessary or poorly done. Since that was a lot of
rewriting, they branched out the development to 3.HEAD.

Some developers did not participate in this 3.HEAD and devoted all
their efforts into 2.HEAD, which made it more efficient for forward
proxy setups. There was quite an interesting argument about this in
the mailing list a couple of years ago.

Regards
HASSAN

Note: I'm bottom posting. If this is not desired, please let me know.
Received on Wed Jun 02 2010 - 07:12:59 MDT