Re: [SQU] HELP: cache too big!

On Tue, Dec 12, 2000 at 02:14:52PM -0600, Joe Cooper wrote:
> I've found that if Squid 2.2.STABLE5 (I don't know about later versions)
> is being shut down improperly or crashes (due to load, for example) on a
> regular basis the cache object store will grow, because Squid will
> 'lose' objects in the store. The objects will still be present, as
> files, but the store index will not know about them. Deleting your
> swap.state file(s), starting squid, and giving it a few minutes/hours to
> rebuild it's swap.state fixes this problem.

No, it doesn't :( At least it doesn't for me.
I have shut down squid the first time it happend (in the meantime I did
this 3 or 4 times) and gone through the "nightmare" of a 12 hours full
rebuild of 50 GB cache data.
I do monitor my squid with Duanes scripts (vitals graphs). These show
that squid knows about 60 GB data on the disks, clains to reduce it
(the graphs and the output of mgr:storedir tell me so) but the DISK
space remains the same (and grows further).
Squid uses a LRU max-age of about 20 days.

The strange thing is - I have modified some of the logging values,
level 2 was too less and level 3 was much to much ;-) - that squid IS
deleting files.
I continuing lines in the logfile now like:

2000/12/06 01:59:49| Cleaned 10 unused files from /share/zuse2/disk06/squid/cache/0F/C3
2000/12/06 01:59:50| storeDirClean: Cleaning directory /share/zuse2/disk02/squid/cache/10/C3
2000/12/06 01:59:50| storeDirClean: Cleaning file 0010C321
2000/12/06 01:59:50| storeDirClean: Cleaning file 0010C320
2000/12/06 01:59:50| storeDirClean: Cleaning file 0010C31F
[ ... ]
2000/12/06 01:59:50| Cleaned 10 unused files from /share/zuse2/disk02/squid/cache/10/C3

and if I check back, these files are gone.

Could it be (haven't looked deeper in the code for some time, sorry)
that the "cleanup manager" isn't fast enough to keep up with the "store
new objects" ??

Can the "cleanup manager" be configured to be more aggressive?
I "solved" the problem running a
    find disk0?/squid/cache -type f -mtime +30 -print | xargs rm
The produces a lot of SWAPFILE_MISS errors, but at least it helps.

Just to make it clear: this happens without any crash in between!

Here's some more info:
------------------------------------------------------------------------
cache_dir ufs /share/zuse2/disk02/squid/cache 12000 32 256
cache_dir ufs /share/zuse2/disk03/squid/cache 12000 32 256
cache_dir ufs /share/zuse2/disk05/squid/cache 12000 32 256
cache_dir ufs /share/zuse2/disk06/squid/cache 12000 32 256
------------------------------------------------------------------------
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/da1s1e 17370434 15264803 715997 96% /share/zuse2/disk02
/dev/da2s1e 17370434 15328626 652174 96% /share/zuse2/disk03
/dev/da4s1e 17370434 15314194 666606 96% /share/zuse2/disk05
/dev/da5s1e 17370434 15338550 642250 96% /share/zuse2/disk06

/dev/da3s1e 8173747 5457192 2062656 73% /share/zuse2/disk04
(The databases files are on /share/zuse2/disk04, not in the cache dirs).
------------------------------------------------------------------------
Store Directory Statistics:
Store Entries : 3724151
Maximum Swap Size : 49152000 KB
Current Store Swap Size: 45094958 KB
Current Capacity : 92% used, 8% free

Store Directory #0: /share/zuse2/disk02/squid/cache
First level subdirectories: 32
Second level subdirectories: 256
Maximum Size: 12288000 KB
Current Size: 11273387 KB
Percent Used: 91.74%
Filemap bits in use: 929837 of 2097152 (44%)
Flags: SELECTED

Store Directory #1: /share/zuse2/disk03/squid/cache
First level subdirectories: 32
Second level subdirectories: 256
Maximum Size: 12288000 KB
Current Size: 11270803 KB
Percent Used: 91.72%
Filemap bits in use: 908958 of 2097152 (43%)
Flags: SELECTED

Store Directory #2: /share/zuse2/disk05/squid/cache
First level subdirectories: 32
Second level subdirectories: 256
Maximum Size: 12288000 KB
Current Size: 11274650 KB
Percent Used: 91.75%
Filemap bits in use: 938818 of 2097152 (45%)
Flags: SELECTED

Store Directory #3: /share/zuse2/disk06/squid/cache
First level subdirectories: 32
Second level subdirectories: 256
Maximum Size: 12288000 KB
Current Size: 11276118 KB
Percent Used: 91.77%
Filemap bits in use: 946335 of 2097152 (45%)
Flags:
------------------------------------------------------------------------

Thanks for any help.

        \Maex

--
To unsubscribe, see http://www.squid-cache.org/mailing-lists.html