Re: [squid-users] Squid Under High Load

On Sun, Feb 04, 2007, Manoj Rajkarnikar wrote:
> On Fri, 2 Feb 2007, Adrian Chadd wrote:
>
> > Part of the work I did quite a while ago was to try and allow people to
> > store very large objects on another spool. I guessed that the large objects
> > were accessed less frequently and so could happily be stored in a UNIX
> > filesystem. The file open rate for a "normal" UNIX filesystem is what, 50 ish
> > requests a second for a single-spindle disk filesystem? Maybe slightly higher
> > if all your directory entries are cached?
> >
> > Research has mostly shown that to be true; ie the overhead of UNIX filesystems
> > becomes less of a concern after the object size grows past a couple hundred
> > kilobytes. I'd quote the references but I don't have them handy - I'll make
> > sure they appear in the document library once the new Squid website is released.
> >
> > So as long as you're able to store small objects seperately from large objects
> > and make sure one doesn't starve IO from the other then you'll be able to both
> > enjoy your cake and eat it too. :P
>
> That would be a great feature to have, to be able to put larger objects in
> a separate space. Thanks. :)

You already can, to some extent. See the cache_dir configuration information;
there's a "max object" size parameter. cache_dirs are checked in order so just
have your small object stores have a max objsize parameter of something other
than -1.

Its not 100% correct as it requires (or required in the past) for the object
size to be known at swapout selection time; which happens after the headers
are parsed but before any data arrives. this means objects without an explicit
content-length header will always fall through to the cache_dir without a max
object size. I plan on fixing this at some point with the new store manager
work thats going on in squid-2-HEAD.

But please, play with it in squid-2.6 and let me know how you go. It works fine
enough for those of you using COSS - stick small object (under a few hundred
kbytes) into COSS and the rest into AUFS.

Adrian
Received on Sat Feb 03 2007 - 22:51:19 MST