Re: Transparent-ly obtained objects in a name-based hierarchy: problems?

On Tue, 17 Feb 1998, John Todd wrote:

> How does one get around the discrepancy caused by IP-based versus name-based
> cache object storage when working in a hierarchy of mixed transparent and
> proxy-based caches?

The "ultimate" solution is to have some "is the content of this data the
same as the data I already have" comparison - something like comparing MD5
hashes of the data for two objects.

Issues might include:

- if an identifying code for the requested object is available (such as
  the ETag in HTTP/1.1 responses), compare those directly without having
  to download the data

- if no identifying code is available, you have to generate one yourself,
  which of course involves downloading the whole object .. but that
  doesn't mean that upon finding out that you've downloaded an object you
  already have that you have to *keep* the object in the cache (ie., it
  could be removed from the cache)

- if you're constantly downloading an object over and over (as might be
  the case for the previous para), you may need to setup some sort of
  "matching" system so that URLs can sometimes be said to be equivalent
  with some other URLs (which of course then requires a cache and expiry
  just as with DNS queries) .. then as you get requests you first have
  to check the equivalence table and.. and.. . . .

Getting rather complicated, ain't it? :-/

You could of course use ETags when available and assume different objects
when they're not available .. then there's always a hard-coded RE-based
matching table (in squid.conf) that could force equivalence (much in the
same way as we can already force aging/expiry of objects).

So many options, so little time to code anything. :-/

dave
Received on Tue Feb 17 1998 - 16:25:52 MST