[squid-users] Re: Vedr.: [squid-users] squid failed stresstest.. what did I configure wrong?

hi klavs,

look for a string FD_SETSIZE in /usr/include/bits/types.h (linux)
or try <grep -r __FD_SETSIZE *> in /usr/include

further more, you should run a <ulimit -HSn 8192> before configuring squid.

you should also insert a <ulimit -HSn 8192> into the RunCache script.
(...at least that makes quite sure that you really have your 8192
filedescriptors...)

torsten

+++++++++++++++

Hi Torsten,

Thanks for your input.. I checked the FAQ and found the notes..
unfortunately, even when I changed the file-max to 65535 (on a 2.4 kernel)
and
set the /usr/include/linux/limits.h

OPEN_MAX to 65535 and NR_OPEN to 65535 - no change.. the configure script
still says..

max 1024 open file descriptors..
why is that ??

/Klavs

hi,

<<"WARNING! Your cache is running out of filedescriptors">>

you should increase the number of maximum filedescriptors to at least 8192.
there is a word on this in the squid faqs.

Best regards

Torsten Lange

---------------------- Forwarded by Torsten Lange/Kerpen/GECITS-EU on
12.11.2001 16:31 ---------------------------

ktk@metropol.dk on 12.11.2001 16:28:40

To: squid-users@squid-cache.org
cc:
Subject: [squid-users] squid failed stresstest.. what did I configure
        wrong?

 --------------------------------------------------------------------------

Hi guys,

I tried to give squid a lot of users, and it served fine..for a few
seconds.. now it doesn't answer me.. not even with objects in cache..

what would you recommend to keep squid from - freezing..
- how do I set more Filedescriptors ?

And why can't i set how many async-io processes it can run?

I have configured it with

configure --enable-snmp --prefix=/usr/local/squid --disable-ident-lookup
--disable-wccp --enable-removal-policies --enable-storeio=aufs
--enable-async-io=50

in the log file I see the following - from the log the few min. I
stresstested it..:

2001/11/12 16:23:22| clientKeepaliveNextRequest: FD 406 Sending next
2001/11/12 16:23:22| Failed to select source for
'http://ad.maximedia.dk/hotel/xban.nsf/logStat?openagent&action=subsiteexpotime&subsite=2&host=aok&bid=9BE&expotime=1000&rnd=0.4222200343366116'

2001/11/12 16:23:22| always_direct = 0
2001/11/12 16:23:22| never_direct = 1
2001/11/12 16:23:22| timedout = 0
2001/11/12 16:23:23| WARNING! Your cache is running out of filedescriptors
2001/11/12 16:23:23| this be aioCancel
2001/11/12 16:23:27| this be aioCancel
2001/11/12 16:23:28| this be aioCancel
2001/11/12 16:23:28| this be aioCancel
2001/11/12 16:23:30| clientSendMoreHeaderData: Deferring
http://www3.cnn.dk/img/partnerknapper/gulgratis.gif
2001/11/12 16:23:30| clientKeepaliveNextRequest: FD 768 Sending next
2001/11/12 16:23:35| Starting Squid Cache version 2.5-DEVEL for
i686-pc-linux-gnu...
2001/11/12 16:23:35| Process ID 3065
2001/11/12 16:23:35| With 1024 file descriptors available
2001/11/12 16:23:35| DNS Socket created at 0.0.0.0, port 32800, FD 5
2001/11/12 16:23:35| Adding nameserver 194.192.81.80 from /etc/resolv.conf
2001/11/12 16:23:35| Unlinkd pipe opened on FD 10
2001/11/12 16:23:35| Swap maxSize 1024000 KB, estimated 78769 objects
2001/11/12 16:23:35| Target number of buckets: 3938
2001/11/12 16:23:35| Using 8192 Store buckets
2001/11/12 16:23:35| Max Mem size: 819200 KB
2001/11/12 16:23:35| Max Swap size: 1024000 KB
2001/11/12 16:23:35| Store logging disabled
2001/11/12 16:23:35| Rebuilding storage in /www/squidcache (DIRTY)
2001/11/12 16:23:35| Using Least Load store dir selection
2001/11/12 16:23:35| chdir: /usr/local/squid/cache: (2) No such file or
directory
2001/11/12 16:23:35| Current Directory is /usr/local/squid/etc
2001/11/12 16:23:35| Loaded Icons.
2001/11/12 16:23:35| Accepting HTTP connections at 0.0.0.0, port 80, FD 34.
2001/11/12 16:23:35| Accepting SNMP messages on port 3401, FD 35.
2001/11/12 16:23:35| Ready to serve requests.
2001/11/12 16:23:35| Configuring Parent www1.cnn.dk/80/0
2001/11/12 16:23:35| Configuring Parent www2.aok.dk/80/0
2001/11/12 16:23:35| Store rebuilding is 0.7% complete
2001/11/12 16:23:39| CACHEMGR: manager@127.0.0.1 requesting 'config'
2001/11/12 16:23:42| CACHEMGR: manager@127.0.0.1 requesting 'menu'
2001/11/12 16:23:45| CACHEMGR: manager@127.0.0.1 requesting 'info'
2001/11/12 16:23:50| Store rebuilding is 74.3% complete
2001/11/12 16:23:55| Done reading /www/squidcache swaplog (611272 entries)
2001/11/12 16:23:55| Finished rebuilding storage from disk.
2001/11/12 16:23:55| 326987 Entries scanned
2001/11/12 16:23:55| 0 Invalid entries.
2001/11/12 16:23:55| 0 With invalid flags.
2001/11/12 16:23:55| 42856 Objects loaded.
2001/11/12 16:23:55| 0 Objects expired.
2001/11/12 16:23:55| 283207 Objects cancelled.
2001/11/12 16:23:55| 6 Duplicate URLs purged.
2001/11/12 16:23:55| 918 Swapfile clashes avoided.
2001/11/12 16:23:55| Took 20.3 seconds (2112.7 objects/sec).
2001/11/12 16:23:55| Beginning Validation Procedure
2001/11/12 16:23:55| Completed Validation Procedure
2001/11/12 16:23:55| Validated 42879 Entries
2001/11/12 16:23:55| store_swap_size = 936172k
2001/11/12 16:23:56| squidaio_queue_request: WARNING - Queue congestion
2001/11/12 16:23:56| squidaio_queue_request: WARNING - Queue congestion
2001/11/12 16:23:56| squidaio_queue_request: WARNING - Queue congestion
2001/11/12 16:23:56| squidaio_queue_request: WARNING - Queue congestion

The load on the machine, was 50-70 percent constantly idle..

output from: vmstat 1 100
 0 0 0 4344 262480 71104 563940 0 0 0 0 1192 952 12 27
60
 0 0 0 4344 262432 71104 563956 0 0 0 0 1191 891 11 30
59
 0 0 0 4344 262420 71104 563968 0 0 0 0 1491 794 13 28
59
 1 0 0 4344 262444 71104 563968 0 0 0 0 1099 596 15 35
50
 1 0 0 4344 262404 71104 563984 0 0 0 308 1382 1015 15 24
62
 0 0 0 4344 262416 71104 564000 0 0 0 0 1508 1058 15 25
59
 1 0 0 4344 262412 71104 564008 0 0 0 0 1950 1344 14 33
52
 0 0 0 4344 262392 71104 564028 0 0 0 0 1691 1225 17 27
56
 3 0 1 4344 262360 71104 564044 0 0 0 0 1214 875 13 30
57
 2 0 0 4344 262356 71104 564060 0 0 0 476 1195 1001 13 24
64
 0 0 0 4344 262340 71104 564076 0 0 0 0 1188 858 19 29
52
 0 0 0 4344 262300 71104 564084 0 0 0 0 1125 635 10 24
66
 1 0 0 4344 262304 71104 564096 0 0 0 0 1298 816 10 25
65
 0 0 0 4344 262292 71104 564112 0 0 0 0 1340 976 15 29
56
 0 0 0 4344 262256 71104 564124 0 0 0 228 960 642 17 32
50
 1 0 0 4344 262244 71104 564136 0 0 0 0 1230 940 13 24
63
 0 0 0 4344 262260 71104 564152 0 0 0 0 1174 765 10 22
68
 1 0 0 4344 262240 71104 564160 0 0 0 0 1059 635 10 25
66
 1 0 1 4344 262224 71104 564176 0 0 0 0 1589 944 15 27
58
 0 0 0 4344 262188 71104 564188 0 0 0 404 1209 759 14 29
57
 1 0 0 4344 262204 71104 564204 0 0 0 0 1484 942 11 30
59
 1 0 0 4344 262172 71104 564220 0 0 0 0 1192 952 16 19
65
 1 0 0 4344 262132 71104 564236 0 0 0 0 1287 1125 15 25
60
 0 0 0 4344 262140 71104 564248 0 0 0 0 1119 757 12 22
66

-------------| This mail has been sent to you by: |------------
Klavs Klavsen, IT-coordinator and Systems Administrator at
Metropol Online - http://www.metropol.dk
Tlf. 33752700, Fax 33752720, Email ktk@metropol.dk

Private- Email [email protected] - http://www.vsen.dk

--------------------[ I believe that... ]-----------------------
It is a myth that people resist change. People resist what other
people make them do, not what they themselves choose to do...
That's why companies that innovate successfully year after year
seek their peopl's ideas, let them initiate new projects and
encourage more experiments. -- Rosabeth Moss Kanter
Received on Mon Nov 12 2001 - 09:01:31 MST