Re: [squid-users] CPU utilization performance issue

I know that ext3 uses a lot more CPU on a lot of disk I/O. This is a
known issue.
Btw, did you compile the Squid with the gnu-regex option? Because that
one is faster on Linux systems. (if you're using a lot of acl's)

rgrds,
          BArt
Tay Teck Wee wrote:

>Hi Bart,
>
>I'm using reiserfs. aufs coz its more suitable for
>linux.
>
>--
>Wolf
>
> --- Schelstraete Bart <bart@schelstraete.org> wrote:
>
>
>>Hzllo,
>>
>>Why not using Reiser instead of ext3 with diskd?
>>I read a lot of articles that are saying that reiser
>>is much fast for
>>Squid. (a lot of 'small files')
>>
>>
>>
>>
>> Bart
>>
>>Zand, Nooshin wrote:
>>
>>
>>
>>>Hi,
>>>I am just wonder why you are not using diskd.
>>>Based on Benchmarking that I read, diskd provides
>>>
>>>
>>faster I/O performance.
>>
>>
>>>I am planning to run squid on Linux Redhat 9.0 and
>>>
>>>
>>thinking to use ext3 and diskd.
>>
>>
>>>Thanks,
>>>Nooshin
>>>
>>>-----Original Message-----
>>>From: Tay Teck Wee
>>>
>>>
>>[mailto:wolfpacks01@yahoo.com.sg]
>>
>>
>>>Sent: Friday, August 08, 2003 2:22 AM
>>>To: squid-users
>>>Subject: Re: [squid-users] CPU utilization
>>>
>>>
>>performance issue
>>
>>
>>>Hi everyone,
>>>
>>>thanks for the input. The ACL list have since been
>>>slightly altered, using only src(22 entries),
>>>dstdomain(114 entries) and url_regex(20 entries). I
>>>
>>>
>>am
>>
>>
>>>currently on kernel 2.4.20-19.9 so the
>>>
>>>
>>Hyperthreading
>>
>>
>>>might hv been optimized.
>>>
>>>Now the machine is handling about 110 req/s but
>>>
>>>
>>again
>>
>>
>>>the CPU will climb to abt 90-95%. Is it possible
>>>
>>>
>>for
>>
>>
>>>my squid box to go beyond 180 req/s, which is the
>>>
>>>
>>peak
>>
>>
>>>for each proxies in the existing pool(ISP env)? I
>>>
>>>
>>am
>>
>>
>>>trying to replace my existing NetCaches with
>>>squids...one box for one box.
>>>
>>>I am wondering if its because reiserfs will consume
>>>more CPU than other fs like ext3? Will changing my
>>>cache partitions to reiserf lower down the CPU
>>>
>>>
>>usage?
>>
>>
>>>Or can anyone suggest other possible improvements?
>>>Thanks.
>>>
>>>my 3 caching partitions are on 3 separate disks:-
>>>/dev/sdb1 /cdata1 reiserfs notail,noatime 1 2
>>>/dev/sdc1 /cdata2 reiserfs notail,noatime 1 2
>>>/dev/sdd1 /cdata3 reiserfs notail,noatime 1 2
>>>
>>>--
>>>Wolf
>>>
>>>--- Tay Teck Wee <wolfpacks01@yahoo.com.sg> wrote:
>>>
>>>Hi,
>>>
>>>
>>>
>>>
>>>>when I'm getting about 90 req/s or 800 concurrent
>>>>connection(according to my foundry L4) to my
>>>>squid(RedHat 8.0/2.5 Stable3 w deny_info patch),
>>>>
>>>>
>>the
>>
>>
>>>>CPU utilization avg abt 80%. How do I lower the
>>>>
>>>>
>>CPU
>>
>>
>>>>utilization of my squid? Thanks.
>>>>
>>>>Below is my machine specs:-
>>>>
>>>>Intel Xeon single-processor 2.4GHz(DELL 2650)
>>>>2G physical RAM(w 2G swap under linux)
>>>>2X 33G for everything except caching (mirror)
>>>>3X 33G for caching (volume)
>>>>
>>>>/dev/sda7 505605 68437 411064
>>>>
>>>>
>>>>15% /
>>>>/dev/sda1 124427 9454 108549
>>>>
>>>>
>>
>>
>>
>>>>9% /boot
>>>>/dev/sdb1 35542688 201248 35341440
>>>>
>>>>
>>
>>
>>
>>>>1% /cdata1
>>>>/dev/sdc1 35542688 200888 35341800
>>>>
>>>>
>>
>>
>>
>>>>1% /cdata2
>>>>/dev/sdd1 35542688 200940 35341748
>>>>
>>>>
>>
>>
>>
>>>>1% /cdata3
>>>>/dev/sda3 1035692 49796 933284
>>>>
>>>>
>>
>>
>>
>>>>6% /home
>>>>none 1032588 0 1032588
>>>>
>>>>
>>
>>
>>
>>>>0% /dev/shm
>>>>/dev/sda5 1035660 691648 291404
>>>>
>>>>
>>>>71% /usr
>>>>/dev/sda6 505605 76236 403265
>>>>
>>>>
>>>>16% /usr/local
>>>>/dev/sda8 29695892 83456 28103936
>>>>
>>>>
>>
>>
>>
>>>>1% /var
>>>>
>>>>Below is my squid.conf(only the essential).
>>>>
>>>>For ACL, basically I hv 3 acl list(in 3 separate
>>>>files), one containing allowable IPs while the
>>>>
>>>>
>>other
>>
>>
>>>>contains deny IPs. I also hv 3 list of banned
>>>>
>>>>
>>sites
>>
>>
>>>>list(in 3 separate files).:-
>>>>
>>>>http_port 8080
>>>>acl QUERY urlpath_regex cgi-bin \?
>>>>no_cache deny QUERY
>>>>cache_mem 400 MB
>>>>cache_swap_low 92
>>>>cache_swap_high 95
>>>>maximum_object_size 2 MB
>>>>maximum_object_size_in_memory 100 KB
>>>>cache_replacement_policy heap GDSF
>>>>memory_replacement_policy heap GDSF
>>>>cache_dir aufs /cdata1 16000 36 256
>>>>cache_dir aufs /cdata2 16000 36 256
>>>>cache_dir aufs /cdata3 16000 36 256
>>>>cache_access_log
>>>>
>>>>
>>/var/log/cachelog/cache.access.log
>>
>>
>>>>cache_log /var/log/cachelog/cache.log
>>>>cache_store_log none
>>>>quick_abort_min -1 KB
>>>>acl all src 0.0.0.0/0.0.0.0
>>>>acl manager proto cache_object
>>>>acl localhost src 127.0.0.1/255.255.255.255
>>>>#3 banned list files
>>>>acl SBA dstdomain "/usr/local/squid/etc/SBA.txt"
>>>>acl CNB dstdomain "/usr/local/squid/etc/CNB.txt"
>>>>acl CNB2 url_regex "/usr/local/squid/etc/CNB2.txt"
>>>>#3 access list files
>>>>acl NetTP src "/usr/local/squid/etc/NetTPsrc.acl"
>>>>acl NetDeny src "/usr/local/squid/etc/deny.acl"
>>>>acl NetAllow src "/usr/local/squid/etc/allow.acl"
>>>>http_access deny SBA
>>>>http_access deny CNB
>>>>http_access deny CNB2
>>>>http_access deny NetDeny
>>>>http_access allow NetAllow
>>>>http_access allow NetTP
>>>>http_access deny all
>>>>http_reply_access allow all
>>>>cache_effective_user squid
>>>>cache_effective_group squid
>>>>logfile_rotate 10
>>>>deny_info ERR_SBA_DENIED SBA
>>>>deny_info ERR_CNB_DENIED CNB CNB2
>>>>memory_pools off
>>>>coredump_dir /var/log/cachelog
>>>>
>>>>Thanks again!
>>>>
>>>>Regards,
>>>>Wolf
>>>>
>>>>
>>>>__________________________________________________
>>>>Do You Yahoo!?
>>>>Send free SMS from your PC!
>>>>http://sg.sms.yahoo.com
>>>>
>>>>
>>>>
>>>>
>>>__________________________________________________
>>>Do You Yahoo!?
>>>Send free SMS from your PC!
>>>http://sg.sms.yahoo.com
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>__________________________________________________
>Do You Yahoo!?
>Send free SMS from your PC!
>http://sg.sms.yahoo.com
>
>
>
Received on Fri Aug 08 2003 - 11:43:55 MDT