[squid-users] Download corruption on transparent proxy

We've been experiencing some file corruption on large downloads. As an example, we've been trying to download the following file from Novell's site:

http://ftp3.novell.com/pub/allupdates/nw6sp2.exe

which is ~250MB. Some information about our proxies:

3 x Squid 2.5s1 servers west (3 x 2.4r7 east)
configure options: --prefix=/opt/squid --with-pthreads --enable-storeio=diskd,aufs,ufs --disable-snmp --enable-cache-digests --enable-linux-netfilter --disable-ident-lookups
1gb RAM each on west coast (2gb RAM each on east coast farm)
5 18 gb partitions per server (cache_dir aufs /cache/disk1 15000 16 256)
Linux 2.4.19
ReiserFS partitions (noatime,notail)

These three servers are configured on a load balancer, and we are using transparent proxying at our Linux firewall to redirect outbound port 80 traffic to the virtual IP on the loadbalancer.

I've conducted a few testcases to try to pinpoint it:

case a = bypass squid
        result: file OK
case b = specifically set http_proxy variable on a 'wget' command
        result: file OK
case c = rely upon transparent proxying
        result: corrupt file

Here are the filesizes (all identical):

-rw-r--r-- 1 root root 257053201 Oct 3 14:21 nw6sp2_direct.exe
-rw-r--r-- 1 root root 257053201 Sep 11 14:20 nw6sp2_proxy.exe
-rw-r--r-- 1 root root 257053201 Sep 11 14:20 nw6sp2_xparent.exe

But - the transparent version shows interesting things inside, such as:

Date: Fri, 13 Sep 2002 20:19:15 GMT^M
Server: Apache/1.3.26 (Unix) FrontPage/4.0.4.3^M
Last-Modified: Tue, 11 Jun 2002 18:39:12 GMT^M
ETag: "4a64e3-1057-3d0643d0"^M
Accept-Ranges: bytes^M
Content-Length: 4183^M
Keep-Alive: timeout=10, max=92^M
Connection: Keep-Alive^M
Content-Type: image/gif^M

random headers from other requests.

It seems as though this was reported previously, in this thread:

http://list.cineca.it/cgi-bin/wa?A2=ind0205&L=squid&D=0&P=47955

but apparently nothing was ever figured out.

Anybody have any ideas on this???

- Phil Oester
Received on Thu Oct 03 2002 - 17:38:17 MDT