Re: [squid-users] Re: Multiple NICs

On 10/11/10 01:39, Nick Cairncross wrote:
>>
>> Hi!
>>
>> I wouldn't think you need multiple network cards to use squid, unless
>> your internet connection is on or above 1GB/s. If your ISP provides you
>> less, I would think a regular gigabit Nic would do the job.
>> Your Hard Drives probably wont be fast enough to cache data on multiple
>> Nics anyways.
>>
>> We have over 1000 Clients, and the previous setup we used, we had only 1
>> GB network interface of our squid. It was sitting in the DMZ, and the
>> connections went trough it.
>> It was fine. Had no connection problems.
>>
>> Tibby
>> ________________________________________
>> Felad�: Nick Cairncross
>>
>> Hi list,
>>
>> I'm looking at building a couple more 3.1.8 servers on RHEL 5.5 x86. The
>> servers are nicely high-powered have multiple Gb NICs (4 in total). My
>> previous proxy server (bluecoat) had two NICs. I understand that one was
>> used to listen to requests and send to our upstream accelerator and one
>> was used if the equivalent 'send direct' was used i.e bypass the
>> accelerator. Can the list make any thoughts or recommendations about the
>> best way to utilise the NICs for best performance? Can I achieve the same
>> outbound as above? Should I even bother trying to do this? User base
>> would be about 700 users; I'm not caching. Simple ACLs but with two
>> authentication helpers (depending on browser).
>>
>> Cheers
>> Nick
>
> Thanks Tibby for your input - sounds sensible. Net connection is fast and
> wide, so gb should be ok.
>
> In that case another question for the list: Seeing as I'm not doing ANY
> caching at all and just proxying traffic are there any recommendations for
> squid.conf settings that might optimise my users' experiences (other than
> caching..).

Why such a thing against caching?
That is the #1 speed gain (about 3-4 orders of magnitude faster to fetch
something from RAM cache than the network).

> I have fast ACLs where possible in place and my squid.conf is
> as below. I'm looking for any tips on maximising memory, processes etc

memory maximization is almost all about caching.

Besides the cache_mem for in-transit objects (your "cache deny all"
causes them to drop when completed even if they need fetching again
immediately).

You could also possibly check and tune the DNS ipcache/fqdncache sizes
for more entries, and bump the auth cache size up enough to hold all
your user credentials.

> from within the squid.conf so that the end user has as quick an experience
> as possible - Are there any other tags I should look at using? Server spec
> is a single cpu Xeon X5660 @ 2.8, 6gb 1333 ram, 250 gb R1
>

To start with TMF (the measurement factory) are looking into a few
things right now regards to the speed of 3.1. There are likely to be
some extra speed patches in 3.1.10 next month.

The first one being the default buffer size in src/MemBuf.h (currently
2*1024, could be upped to 64*1024 for bigger network reads).

> ===
> http_port 8080
>
> auth_param negotiate program /usr/lib/squid/squid_kerb_auth -r
> auth_param negotiate children 80
> auth_param negotiate keep_alive on
>
> auth_param ntlm program /usr/bin/ntlm_auth
> --helper-protocol=squid-2.5-ntlmssp
> auth_param ntlm children 60
> auth_param ntlm keep_alive on
>
> auth_param basic program /usr/bin/ntlm_auth
> --helper-protocol=squid-2.5-basic
> auth_param basic children 20
> auth_param basic realm Squid proxy-caching web server
> auth_param basic credentialsttl 2 hours
>
> cache_peer [upstream] parent 8080 0 no-query proxy-only no-digest default
>
> cache_mgr [blanked]
> cachemgr_passwd [blanked] all
> client_persistent_connections on
> #server_persistent_connections on
> persistent_connection_after_error on
>
> access_log /var/log/squid/access.log squid
> cache_store_log none squid
> cache_log /var/log/squid/cache.log squid
>
> ###### Delay Pool Definitions #######
>
> # Total number of delay pools
> delay_pools 1
>
> #### ACCESS CONTROL LISTS #####
>
> ## USER-AGENT (Browser-type) ACLs
> acl Java_jvm browser "/etc/squid/ACL/USERAGENTS/USER-AGENTS_JAVA.txt"
> acl iTunes browser "/etc/squid/ACL/USERAGENTS/USER-AGENTS_APPLE.txt"
> acl MSNMessenger browser "/etc/squid/ACL/USERAGENTS/USER-AGENTS_MSN.txt"
>
>
> ## USER AUTHENTICATION ACLs
> acl AuthenticatedUsers proxy_auth REQUIRED
>
> ## URL DESTINATION ACLs
> acl URL_ALLOWDstDomains dstdomain
> "/etc/squid/ACL/URL/URL_ALLOWDstDomains.txt"
>
> ## IP ACLS ##
> acl CNP_SERVERIP src 172.16.10.176
> acl CNP_SERVERIP src 172.16.100.50
> acl CNP_CLIENTIP src "/etc/squid/ACL/IPADDRESSES/IP_CLIENTIP.txt"
>
> ## Windows Update ACLS
> acl WSUS_IP src 172.16.10.127
>
> # LAN IP ACLs
> acl CNP_172SUBNETS src 172.16.0.0/16
> acl CNP_SERVERSUBNETS src 172.16.10.0/24
> acl CNP_SERVERSUBNETS src 172.16.100.0/24
>
> # Blocks CONNECT method to IP addresses (Blocks Skype amongst other things)
> acl StopDirectIP url_regex ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+

dstdom_regex for a often much shorter size of bytes duplicated and sent
out for regex scanning.

>
> # MSN Messenger Allow IP ACL
> acl IP_MSNMessenger src "/etc/squid/ACL/IPADDRESSES/IP_MSNMESSENGER.txt"
>
> # SEND DIRECT ACLs
> acl SENDDIRECT_DstDomains dstdomain
> "/etc/squid/ACL/SENDDIRECT/SENDDIRECT_DSTDOMAINS.txt"
> acl SENDDIRECT_IPAddresses src
> "/etc/squid/ACL/SENDDIRECT/SENDDIRECT_IPADDRESSES.txt"
>
> # CONNECT Method Direct IP ACLs
> acl IP_CONNECTALLOW src "/etc/squid/ACL/IPADDRESSES/IP_CONNECTALLOW.txt"
>
> ## LOCALHOST ACLs
> acl localhost src 127.0.0.1
> acl to_localhost dst 127.0.0.0/8
>
> ## CACHEMGR ACL
> acl manager proto cache_object
>
> ## PORTS ACLs
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 8080 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl RTMP_ports port 1935 # RTMP
>
> # CONNECTION METHOD ACL
> acl CONNECT method CONNECT
> acl POST method POST
>
> ######### ICAP SERVER #########
> acl ICAP_BYPASS dstdomain "/etc/squid/ACL/ICAP/ICAP_BYPASS_URL.txt"
>
> ## ICAP Settings
> icap_enable on
> icap_preview_enable on
> icap_persistent_connections on
> icap_send_client_ip on
> icap_send_client_username on
> icap_client_username_header X-Authenticated-User
> icap_client_username_encode on
> icap_service ss reqmod_precache 0 icap://localhost:1344/ssreqmod
> adaptation_service_set c1 ss
> adaptation_access c1 deny ICAP_BYPASS
> adaptation_access c1 deny POST
> adaptation_access c1 deny CONNECT
> adaptation_access c1 allow all
>
> ###### ACCESS RULES #####
>
> ## CACHEMGR ALLOW
> http_access allow manager CNP_172SUBNETS
>
> ## GLOBAL DENY RULES - Do NOT include User Authentication ACLs here
> http_access deny !Safe_ports
> http_access deny manager !CNP_172SUBNETS

You already allowed "manager CNP_172SUBNETS" the !CNP_172SUBNETS has no
effect and can be dropped.

This bunch of denies could be sped up somewhat:

  * moving the fastest to the top (CONNECT, POST)

> http_access deny CONNECT !SSL_Ports !CNP_172SUBNETS
> http_access deny POST !SSL_Ports !RTMP_ports !CNP_172SUBNETS

  * re-ordering the placement of ACL tests in each of the rest.

> http_access deny MSNMessenger CNP_172SUBNETS !IP_MSNMESSENGER
> http_access deny StopDirectIP !IP_CONNECTALLOW
> http_access deny iTunes !CNP_172SUBNETS
> http_access deny Java_jvm !CNP_172SUBNETS

For example:
   http_access deny CNP_172SUBNETS !IP_MSNMESSENGER MSNMessenger
instead of:
   http_access deny MSNMessenger CNP_172SUBNETS !IP_MSNMESSENGER

Why? They all start with regex right now ("browser" ACL uses regex too).
That is markedly slower than IP testing.

>
> # CACHEMGR ALLOW
> http_access allow manager CNP_172SUBNETS
>
> # POST/CONNECT Method ALLOW #
> http_access allow CONNECT CNP_172SUBNETS
> http_access allow POST CNP_172SUBNETS
>
> # USER AGENTS ALLOW #
> http_access allow iTunes CNP_172SUBNETS
> http_access allow Java_jvm CNP_172SUBNETS

Same speed-up ordering can be done for the above agent lines as fro teh
denials earlier.

>
> # URL and IP DESTINATIONS NOT REQUIRING AUTHENTICATION
> http_access allow URL_ALLOWDstDomains
> http_access allow CNP_SERVERIP
> http_access allow CNP_CLIENTIP
> http_access allow WSUS_IP
>
> ## USERS AUTHENTICATION ACL##
> http_access allow AuthenticatedUsers
>
> ## SEND DIRECT ALLOW
> always_direct allow SENDDIRECT_DstDomains
> always_direct allow SENDDIRECT_IPAddresses
>
> ## LOCAL HOST ALLOW ##
> http_access allow localhost
>
> ###### DELAY ACCESS RULES ######
> # First delay class pool is User allocated
> delay_class 1 4
>
> # First delay class pool is limited to 4mb i.e. all users not in
> exceptions: CNP_SERVERSUBNETS
> delay_parameters 1 -1/-1 -1/-1 -1/-1 4000000/4000000
>
> #First delay pool applied to Authenticated users except those in Server
> Subnet
> delay_access 1 allow AuthenticatedUsers !CNP_SERVERSUBNETS

Reverse the order of these too...
delay_access 1 allow !CNP_SERVERSUBNETS AuthenticatedUsers all

(tacking "all" on the end to keep Squid quiet about the auth. Delay
pools will ignore non-authenticated users, but cache.log would get a
complaint).

> delay_access 1 deny all
>
> http_access deny to_localhost

Very much too late for this one. The above rule is a security barrier to
prevent infected or broken clients looping requests back at Squid etc.
If you need it it should be up with the "deny !Safe_ports" one.

> never_direct allow all
> snmp_access deny CNP_172SUBNETS

Hmm, implicit allow SNMP from *everywhere* except 172.* places.
Somehow I don't think this is what you want. But I dont have enough
knowledge of your network to say what should be there instead.
Something(s) allowed the rest denied.

> cache deny all
> http_access deny all
>

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.9
   Beta testers wanted for 3.2.0.3