Re: Actual sizes (Was: CCD striping on FreeBSD & Squid)

Alex Rousskov wrote:
> On Thu, 13 Nov 1997, Yar Tikhiy wrote:
>
> > IMHO, Squid does not differ too much from the newsserver,
> > because the average object size is larger than the average
> > article size only by one order of magnitude (15-20K vs 2-4K,
> > not considering alt.binaries.* :-)
>
> IMHO, an average (median) swap request is probably about 2-4K for Squid.
> The "magic" 15-20K is for _mean_ network _transfer_ size measured by
> counting every entry in the access log. The actual median transfer size is
> smaller than 4K in reality. Median [cached] _file_ size is usually less
> than 4K too:
>
> http://www.cs.ndsu.nodak.edu/~rousskov/research/cache/squid/profiling/stats/gindex.all.wd/
> Look under "Transfer Sizes" and "File Sizes".
>
>
> You might want to consider that when setting your "store_avg_object_size"
> as well as the interleaving factor for CCD.

Anyway, the request size is small enough to consider setting
the interleave factor to 65536 if you don't want to get two spindles
activated for a single object.

And "store_avg_object_size" must be an estimation of
the average object size (i.e. swap_size/"number of objects")
as it is used to initialize Squid's object hash table
where _every_ cached object has an entry (see src/store.c),
and that is the only place where the parameter is used:

  static void
  storeInitHashValues(void)
  {
      int i;
      /* Calculate size of hash table (maximum currently 64k buckets). */
      i = Config.Swap.maxSize / Config.Store.avgObjectSize;
      debug(20, 1, "Swap maxSize %d, estimated %d objects\n",
          Config.Swap.maxSize, i);
      i /= Config.Store.objectsPerBucket;
      debug(20, 1, "Target number of buckets: %d\n", i);
      /* ideally the full scan period should be configurable, for the
       * moment it remains at approximately 24 hours. */
      if (i < 8192)
          store_buckets = 7951, store_maintain_rate = 10;
      else if (i < 12288)
          store_buckets = 12149, store_maintain_rate = 7;
      else if (i < 16384)
          store_buckets = 16231, store_maintain_rate = 5;
      else if (i < 32768)
          store_buckets = 33493, store_maintain_rate = 2;
      else
          store_buckets = 65357, store_maintain_rate = 1;
  ...

Of course, if you run The Really Big Cache and it contains 1M+
objects the estimations are almost meaningless due to the
64kbuckets limitation :-)

SY, Yar
Received on Thu Nov 13 1997 - 09:34:57 MST