Re: [squid-users] Optimizing mirror use through squid

Its not easily possible at the moment as you need to use a URL redirector
to redirect to another proxy server which then accesses the file and
then store it locally as the "cleaned up" name.

Solomon's Youtube stuff is an example of that.

I'm trying to find time to implement this inside Squid by rewriting
URLs for storage but fetching from the requested origin server.
Its a frequently requested feature and I'd like to do it but
university and paid work is getting in the way.

Someone can beat me to it if they'd like :) You need to:

* create a new uri field in the request_t structure;
* strdup the canonical uri into that to start with;
* modify all the store lookup/store key set code to use that
  uri instead of request->uri;
* make sure the forwarding code uses the unmodified, original
  uri;
* then when this is working, insert some code (that looks like
  the redirector code!) into the client http request path that
  passes the uri to an external redirect helper if it matches an ACL
  (so you don't run the redirector over every request, only the
  ones you care about.)

Would anyone like this as a pet project? Or, would people view
this as worthy for contribution and like to talk to me about contributing
some small amount of money to its completion in Squid-2 and Squid-3?

Adrian

On Thu, Oct 11, 2007, Christian Ullrich wrote:
> Hello all,
>
> I'm looking for a way to use squid to optimize use of mirror servers.
> What I'd like is to tell squid, "these servers have the same content, so
> if you have, for instance, "http://mirror1.org/path/to/file" cached, if
> someone asks for "http://mirror2.org/path/to/file", you can just return
> the cached object.
>
> Is something like that possible? I could use a redirector to send all
> requests for any number of host names to a single server, but that would
> defeat the purpose of having mirrors, i.e. spreading the load.
>
> My intention is to work around what I'd call "client-based load
> balancing", that is, applications that discover a list of mirrors for
> the content they want and then select one of them. When several of these
> run on the same network, the more mirrors there are, the more often the
> data will be downloaded.
>
> For example, consider the various SourceForge download mirrors or
> FreeBSD's portsnap servers.
Received on Thu Oct 11 2007 - 03:14:04 MDT