Re: Webbrowser cache implementation.

On Wed, 28 May 1997, Graham Toal wrote:

> Personally I think it's entirely on topic, especially if you feed back
> any pre-fetching code to the squid project :-)
 :) Then I guess I'm on topic..

> > setup.. But, what are that chances that most home users would set that up?
>
> Very high *if* you write it so it involves no setup *and* it is installed
> at the same time as the browser by the same installation package. In fact
> it could even be part of the browser, offering an http port on 127.0.0.1,
> so if you wanted to use it with another browser, you'd just start Mnemonic
> and minimise it. If my understanding of Mnemonic's structure is correct, it
> could be a Mnemonic module (are you making them dynamically loadable?) which
> was invoked only when the first request arrived on the appropriate port,
> thus keeping the code size down for people not using it; likewise if
> using it only as a proxy for another browser, all the other mnemonic
> modules except the cache would be unloaded thuis making it equivalent
> to a simple cache only.
 I think that it would be an execlent to create a "Squid for Mnemonic"
that would function just as you describe.. (BTW- Parts of mnemonic run in
seperate threads where approiate).. With it being a mnemonic module it
would be able to accomplish the tightness described a bit below but still
be able to function as a cache for those who insist on using mosaic.. :)
Also as the cheif cache for Mnemonic it would get ported along with it..

>
> > users to use the squid proxy.. If we want to get a decent amount using it
> > we have to use transparent proxying (not an ideal solution)...
>
> I've heard about this but haven't seen it in practice yet. What are the
> drawbacks? - as i understand it it sounds ideal. Only reason I haven't
> tried this here yet (actually I want to use the technique on port 25 to
> filter all mail for outgoing spams rather than on the web cache) is
> that we have a Cisco as a router (not surprisingly) rather than
> a linux, and I believe it's just Linux that can do this at the moment
> with some mods to its IP Firewall code.

Well you can put the linux box in as a secondary router/cache.. The
drawback is that the cache sees all traffic as IP addresses and not full
URLs... (like socks).. You just run squid in http accel mode..

> > A web browser NEEDS to have some caching: Otherwise a <back> would take
> > as long as the orignal load... The only place where I would use a
> > webbrowser cacheless is on a lan with a cache on the lan.. But why would
>
> Yes, but if the browser and the cache are on the same machine with the
> same disk (which is what I was proposing) the loading speed should be
> damn near identical, barring the tiny small IPC overhead. You can listen
> on the loopback interface like Alta Vista personal search engine does.

 Then why not make it a mnemonic object so it could be tighter bound to
Mnemonic (for flexibility).. Rember mnemonic could be set to operate as a
cache for other browsers..

> > cache functionality in Mnemonic to be flexible enough that a cache could
> > tell the browser: "Look, we're on a lan, there is no point in you doing
> > much caching at all.. Give it up" And with the object oriented modular
>
> Which with the squid protocols would work because you know the cost to
> the nearest neighbour, I think?
 
 Preety much.. It would need some tweaking.. Running on a user computer is
a bit differnt then running on a well configured server..

> > module from memory.. Also makeing the browser more cache concious it could
> > have seperate reload, redisplay, refresh functions.. Reload would force a
>
> good point; this is a shortcoming of current browsers, but I don't think
> the fix to this is tied to whether or not the browser implements the
> cache. Or at least to the degree to which the cache implemented in
> the browser suite is tightly coupled to the browsers internals, which
> is the point I'm trying to make. By all means have them in the same
> program, but make the logical interface just the same as the external one
> to a real cache, communicating over IP on the lopback interface (optionally
> openable to the real IP) and have the browser be totally dumb about
> remembering pages, i.e. make it so it *doesn't* remember pages and
> let the cache do *all* the remembering. It can be the same actual
> code you have now, just moved to another module. See - your cache
> is almost written! :-)

 I would say that I agree completely.. Execpt that I would think having
the cache inside mnemonic (as another thread of course) with an object
that would allow outside programs to use it.. The reason for this is that
1) tighter interface
2) Parts of the cache itself could be unloaded when not in use..
3) The socket cache interface need not be loaded unless in use..

> > Furthermore it would be possible to create a Mnemonic module that would
> > listen on a socket and allow other browsers to use Mnemonic's cache..
> > While in this mode the rest could be unloaded.. The only overhead would be
> > the object broker and network manager... Although decidated caches will
> > always be more effiecnt: On the desktop such ability might be desirable..
>
> Yeah, i think that;s what I said, except I *strongly* advice you use
> the same protocols as squid and make it functionally compatible, even if
> the guts are a whole rewrite.
>

I agree entirely!.... Though I think we might want to cooridnate with the
squid developers about some possible enhancements to the squid protocols..

> Hmmm. I wonmder if we need an extension to ICP to tell a squid cache
> to go fetch a page, but don't bother delivering it to me. That way a
> client can get something prefetched ready for when it has the bandwidth
> to fetch it or the user asks for it?
 
 I think that might be a good idea.. As well as a few others.. Like a
gimme what you got, I dont care if it's been modified.. and probably a few
others..

>
> > When you are on a 14.4k or 28.8k link in south africa everything is slow!
>
> That's OK, when I started playing this network game back in scotland
> in the 70's, we had a single 56K link to the USA for the whole country,
> and echo times to the machine at MIT I was using were on the order of
> a minute for single character remote echo, which we tried to avoid using!

:)

> PS Also think about a squid cache detection mechanism so that your
> program automatically finds the nearest registered squid cache to
> your site, if it hasn't been programmed in by the distributor or
> typed in by the user or extracted from an autoconfig URL. Also
> consider a broadcast request to find one, if that would work down
> a PPP link?

Yes, I would very much like to figure that out... That was one of the
points of my orignal message.. I think that it would be an execlent idea..

Anyone know if you could do this:
a webclient's ip is:
 123.123.123.30
could it send a udp icp quary to:
 123.123.123.255
to see if there is a cache listening to that brodcast address?
Any ideas on how to do some form of automatic reg?

And comments on what kind of changes you would make to squid were it to be
a cache for a single machine..

[Btw- I am forwarding this to the mnemonic-dev list too..]
Received on Wed May 28 1997 - 22:06:54 MDT