Re: [squid-users] Configuring Squid to behave as fast as possible

Dererk wrote:
> Hi there!
>
> I'm doing some experiments on caching HTTP requests in the middle of the
> way between my webservers to the SOLR load balancer (lvs).
>
> What I need to achieve is an extreme hot cache in the fastest possible
> way. The trick is that objects don't live longer than 15 minutes :-(
> This is because the "objects" (in Solr language, searches) are likely to
> change between that 15 min period. Unfortunately, that's a requirement,
> unless so far until I learn more how to write Squid plugins to play with
> timeouts and stuff like that, which hope it's soon :-)
>
> The interesting part is that the incoming traffic handled by my squid is
> averaged 20Mbytes with peaks of 45Mbytes/s, with a hit ratio of 25 to
> 30%! It's awesome to see how great it behaves!
>
> The reason why I send this message here, above from thank you about this
> amazing software, is to see if I could get some tips I miss to suck the
> most I could from Squid in the way to store/get objects as fast as
> possible.
>
> I've compiled Squid version 2.7.STABLE6 with this parameters:
> '--prefix=/usr/local/squid' '--bindir=/usr/local/bin'
> '--sbindir=/usr/local/sbin' '--sysconfdir=/etc/squid'
> '--localstatedir=/var' '--mandir=/usr/local/man'
> '--infodir=/usr/local/info'

> '--disable-internal-dns'

Step 1: re-enable the parallel-threaded DNS client inside Squid.

> '--enable-async-io'
> '--enable-storeio=diskd,aufs,ufs,coss' '--with-aio' '--with-large-files'
> '--enable-snmp' '--with-maxfd=8192'
>
> And I attach my conf below.
>
> Thanks in advance!
>
> ---
> http_port serverPort:8983 accel vhost
> http_port 3128
> hierarchy_stoplist cgi-bin
> acl QUERY urlpath_regex cgi-bin
> no_cache deny QUERY
> cache_mem 7168 MB
> cache_swap_low 90
> cache_swap_high 95
> maximum_object_size 4096 KB
> minimum_object_size 0 KB
> cache_replacement_policy lru
> memory_replacement_policy lru

You want one of the "heap "algorithms. Search for and Read about them
carefully then decide which is fastest for your traffic.

> cache_dir diskd /var/cache/squid/vol1 10000 100 10
> cache_dir diskd /var/cache/squid/vol2 10000 100 10

What OS are you running on? You enable the Linux-centric async-io / aio
  but don't use AUFS to benefit from them...

NP: making the L1/L2 values "100 10" is probably a bad idea. AFAIK they
should be exponentials of 2 for fastest file path location.

For 10GB caches you probably want something like "32 256" or "256 32"

Step 3: (Assuming from some other config option hints that this is a
Squid-2.7)
    add a COSS cache_dir for "small" objects (under about 256 KB). For
small stuff this is _much_ faster than the other disk types.

> access_log /var/log/squid/access.log
> cache_log /var/log/squid/cache.log
> cache_store_log none
> emulate_httpd_log on
> log_ip_on_direct off
> redirect_rewrites_host_header off
> refresh_pattern . 0 20% 4320

With a 15 minute maximum age of objects you probably want the above to be:

   refresh_pattern . 0 20% 900

> cache_peer loadbalancer parent 8983 0 no-query originserver login=bleh
> cache_mgr me_at_me.com
> cache_effective_user squid
> cache_effective_group squid
> coredump_dir /var/cache/squid
>
>
> Greetings,
>
> Dererk
>

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
   Current Beta Squid 3.1.0.13