[squid-users] RE: Record all traffic option?

> From: Robert Collins [mailto:robertc@squid-cache.org]
> Definately not.

---
	:-)
> 
> Well, first thoughts:
> 
> 1) if you want to redirect everything coming to from squid, 
> use network
> layer interception. It exists, and works. It has issues - but no more
> than 100% alteration via squid itself.
---
	One of the issues, is it makes it harder to session analysis or am I missing something?  It's also an easy of use issue.
> 2) iCAP is an evolving standard to do inline content alteration within
> squid or other such tools - there is a patch for squid to act 
> as an iCAP
> client.
===
	icap? -- maybe something like that...I never heard of it before
your note, but looking at it, It seems like it would do most of #4.  
Are there plans to mainline the patch?  Conceptually, it appears to provide much of what I was looking for...not sure how obvious it
is to setup yet.
> 3) I've no real idea of what you want to achieve, that 
> tcpdump won't do
> for you. 
---
	Ease of use? :-)
> Perhaps if you were do describe what you want to *achieve* we
> could more accurately critique the design.
	Good point.
	1) allow recording.  I have one or more devices, or programs that want to speak to a webserver.  I don't trust them.  I want
to audit everything
they say to each other.  Policy or design is that the programs aren't using encoded information (or https).  I don't want to see
"ACK"s and "SYN"s, 
Just "he said/she said" (well it and it, but nevermind).
	 2) I want to be able to monitor the recording in real time and potentially block any request to a site I don't recognize.
I may want to edit out selective HTML elements: embedded objects, scripts, images, site references.  
	Under '2', I'd like to see the ability to use something like junkbuster but the ultimate goal would be to gain efficiency
over using multiple TCP sockets.  I don't know if local pipes are more efficient than TCP sockets
or not, but the concept of a pipe could possibly be enhanced to provide
a high-speed message transfer using shared memory between processes to lower overhead.  
> From: Mike Cudmore [mailto:Mike.Cudmore@dft.gsi.gov.uk] 
> 1 2 and 3 looks a lot like snort functionality to me......
---
	A quick glance shows snort to be an IDS.  An IDS would likely support 1 2 and 3, but it certainly wouldn't be the first
place to look if I just
wanted to have complete content/data logging instead of just the requests  that are in the current logfile.  Somehow I thought it
would be more
efficient, on a computer, if squid could just 'logall' rather than using a separate package to analyze all network traffic and pick
out only http
content from/to a specific host to log.
	Eventually, latency and efficiency could be concerns.  
	Somehow I just like the idea of extending squid's logging (seems 
conceptually simple) and adding more "plugin" points.  Conceptually all
those seem simple (perhaps the icap patch does exactly the latter).  I know there are complex ways to do what I want, but my
preference would just
be to use a different squid log file or plugin reference.  Seems to be, conceptually, simple.
	Think of squid as a form of a high level TCP language.  With higher
order concepts it implements intelligent caching on top of basic IP packets.
Maybe like "C" vs. assembler.  Yes, "C" could have not included bitfields
and told people they could use assembler for such, but it's alot more convenient to use them in C.  "Etherfind" and such are
definitely more at the "assembler" (more exactly, disassembler) end of the scale.
	Using SNORT for logging (and coming from not having used it) seems
like using ADA for doing the equivalent of "printf" (or a missile to shoot down a mosquito).  Might do the job, but it seems like it
might be overkill?
	Thanks for the lead on icap -- I'll have too look at it some more.
But I hate patching.  Right patch for right version...rebuilding images, when all I wanted was a simple plugin to call a 2-line perl
script [sic].  The solution becomes significantly more work than just adding a few lines to
squid.conf.
	Yeah...I know, DIY, (it is on my list of things that might be fun to do if I ever have free time)  :-).
-linda