Re: Managing large http_access lists: alternative methods

Try this for an idea.

Put a primary DNS zone for a fake domain name on the squid server (or on
it's DNS source):

allourcustomers.fake

(or more correctly, whip up a script to generate some DNS zone files for
each of the customers to map their IP addresses into the fake-domain)

Then allow the fake domain name access. The penalties are the storage of
the zone files, an inability to look up their _real_ names for the logs
(but you can log their numeric IP address instead), and a reverse DNS
lookup for them all. Best to increase the DNS cache sizes in squid.conf
if you do this.

acl legit_customers srcdomain allourcustomers.fake
http_access allow legit_customers
miss_access allow legit_customers

Scott Lystig Fritchie wrote:
>
> This will be my last question/observation/suggestion message for the
> evening, I promise. I also searched my archive of previous messages,
> the FAQ, etc. as best as I could looking for possible answers.
>
> MRNet is operating some Squid caches which, at the moment, have no
> access restrictions on them. The goal was to make it as easy as
> possible for our customers to see the beauty and wisdom of
> participating in the cache hierarchy. That policy has worked
> moderately well for our customers ... and too well for non-customers
> who attempt to launder their connections when breaking in to Web-based
> chat systems, etc. So, it's (beyond reasonable) time to clamp down.
>
> However, I'd like to keep open-to-our-customers policy. However, the
> list of *aggregated* networks that Squid would have to check is a list
> of at least 122 networks. Lots of CIDR blocks, holes punched in CIDR
> blocks, Class B networks, multiply-homed customers, you name it. And
> since we were recently purchased by another ISP, I'd have to add
> another two to four dozen networks to encompass our entire customer
> network base.
>
> My first question is: is there a significant performance penalty for
> checking so many http_access statements, or throwing lots of networks
> statements in a fewer number of acl statements? When our busiest
> cache is busiest, we see 75K TCP connections and 30K ICP queries per
> hour. (Dunno exactly how many of those would disappear once the ACLs
> go into place.)
>
> I don't really want to have to manage that list of networks by hand.
> Am I too idealistic (or too lazy) in hoping for a better way to manage
> these things?
>
> If there isn't a better way, how harebrained would these ideas be
> considered?
>
> * Since our routers have a lot of info of who's close and who
> isn't, it would be cool to ask a router that info.
> Unfortunately, that's difficult to do. Perhaps a hacked
> version of gated, listening in on what its neighbor routers
> are saying, which could answer queries from Squid about who's
> close (i.e. a customer) and who isn't. (I've been incubating
> this idea for a while ... would also be cool for making an
> intelligent, dynamic autoconfiguration script writer pretty
> easy. "Oh, you're located near the Duluth POP. You want to use
> Duluth's cache first, not the one in Mankato.")
>
> * A pool of external processes, like the DNS resolver and
> redirector processes, which use traceroute back to a client.
> If the client is more than N hops away, or if certain router
> interfaces are reached within N hops, they're too far away to
> be a customer.
>
> * Cache this info (or the first 3 octets of the client's
> address) in the data structures Squid already keeps. It would
> make subsequent postitive/negative lookups quick, there
> would be an automatic expiration mechanism (for those
> customers silly enough to leave us for another ISP), and
> stateful memory when squid is killed/restarted.
>
> OK, call me lazy and crazy. :-)
>
> -Scott

-- 
Did you read the documentation AND the FAQ?
If not, I'll probably still answer your question, but my patience will
be limited, and you take the risk of sarcasm and ridicule.