------------------------------------------------------------
revno: 13528 [merge]
revision-id: rousskov@measurement-factory.com-20140811160906-ddpvzkk9yoci20mc
parent: squid3@treenet.co.nz-20140808115258-xazeqbpcs71bw8hh
parent: rousskov@measurement-factory.com-20140810231833-ix9i0vaum2eycdqe
committer: Alex Rousskov <rousskov@measurement-factory.com>
branch nick: trunk
timestamp: Mon 2014-08-11 10:09:06 -0600
message:
  Initial native FTP Relay support.
  
  * Added ftp_port directive telling Squid to relay native FTP commands.
  * Active and passive FTP support on the user-facing side;
    require passive connections to come from the control connection src IP.
  * IPv6 support (EPSV and, on the user-facing side, EPRT).
  * Intelligent adaptation of relayed FTP FEAT responses.
  * Relaying of multi-line FTP control responses using various formats.
  * Support relaying of FTP MLSD and MLST commands (RFC 3659).
  * Several Microsoft FTP server compatibility features.
  * ICAP/eCAP support (at individual FTP command/response level).
  * Optional "current FTP directory" tracking (cannot be 100% reliable due to
    symbolic links and such, but is helpful in some common use cases).
  * FTP origin control connection is pinned to the FTP user connection.
  * No caching support -- no reliable Request URIs for that (see above).
  * Significant FTP code restructuring on the server-facing side.
  * Initial steps towards HTTP code restructuring on the client-facing side.
  
  See merged revisions commit log for details.
------------------------------------------------------------
Use --include-merges or -n0 to see merged revisions.
------------------------------------------------------------
# Bazaar merge directive format 2 (Bazaar 0.90)
# revision_id: rousskov@measurement-factory.com-20140811160906-\
#   ddpvzkk9yoci20mc
# target_branch: http://bzr.squid-cache.org/bzr/squid3/trunk/
# testament_sha1: 7d801438887d76d5a7895c6998454edb52194c19
# timestamp: 2014-08-11 16:54:28 +0000
# source_branch: http://bzr.squid-cache.org/bzr/squid3/trunk/
# base_revision_id: squid3@treenet.co.nz-20140808115258-\
#   xazeqbpcs71bw8hh
# 
# Begin patch
=== modified file 'configure.ac'
--- configure.ac	2014-07-13 08:49:42 +0000
+++ configure.ac	2014-08-04 21:44:31 +0000
@@ -3457,8 +3457,11 @@
 	scripts/Makefile
 	src/Makefile
 	src/anyp/Makefile
+	src/ftp/Makefile
 	src/base/Makefile
 	src/acl/Makefile
+	src/clients/Makefile
+	src/servers/Makefile
 	src/fs/Makefile
 	src/repl/Makefile
 	src/auth/Makefile

=== modified file 'doc/release-notes/release-3.5.sgml'
--- doc/release-notes/release-3.5.sgml	2014-08-02 14:03:21 +0000
+++ doc/release-notes/release-3.5.sgml	2014-08-11 16:09:06 +0000
@@ -43,6 +43,7 @@
 	<item>Support named services
 	<item>Upgraded squidclient tool
 	<item>Helper support for concurrency channels
+	<item>Native FTP Relay
 </itemize>
 
 Most user-facing changes are reflected in squid.conf (see below).
@@ -163,6 +164,48 @@
    With these helpers concurrency may now be set to 0 or any higher number as desired.
 
 
+<sect1>Native FTP Relay
+<p>Details at <url url="http://wiki.squid-cache.org/Features/FtpRelay">.
+
+<p>Squid is now capable of accepting native FTP commands and relaying native
+   FTP messages between FTP clients and FTP servers. Native FTP commands
+   accepted at ftp_port are internally converted or wrapped into HTTP-like
+   messages. The same happens to Native FTP responses received from FTP origin
+   servers. Those HTTP-like messages are shoveled through regular access
+   control and adaptation layers between the FTP client and the FTP origin
+   server. This allows Squid to examine, adapt, block, and log FTP exchanges.
+   Squid reuses most HTTP mechanisms when shoveling wrapped FTP messages. For
+   example, http_access and adaptation_access directives are used.
+
+<p>FTP Relay is a new, experimental, complex feature that has seen limited
+   production exposure. Some Squid modules (e.g., caching) do not currently
+   work with native FTP proxying, and many features have not even been tested
+   for compatibility. Test well before deploying!
+
+<p>Native FTP proxying differs substantially from proxying HTTP requests with
+   <em>ftp://</em> URIs because Squid works as an FTP server and receives
+   actual FTP commands (rather than HTTP requests with FTP URLs).
+
+<p>FTP Relay highlights:</p>
+
+<itemize>
+    <item>Added ftp_port directive telling Squid to relay native FTP commands.
+    <item>Active and passive FTP support on the user-facing side; require
+	  passive connections to come from the control connection source IP
+	  address.
+    <item>IPv6 support (EPSV and, on the user-facing side, EPRT).
+    <item>Intelligent adaptation of relayed FTP FEAT responses.
+    <item>Relaying of multi-line FTP control responses using various formats.
+    <item>Support relaying of FTP MLSD and MLST commands (RFC 3659).
+    <item>Several Microsoft FTP server compatibility features.
+    <item>ICAP/eCAP support (at individual FTP command/response level).
+    <item>Optional "current FTP directory" tracking with the assistance of
+	  injected (by Squid) PWD commands (cannot be 100% reliable due to
+	  symbolic links and such, but is helpful in some common use cases).
+    <item>No caching support -- no reliable Request URIs for that (see above).
+</itemize>
+
+
 <sect>Changes to squid.conf since Squid-3.4
 <p>
 There have been changes to Squid's configuration file since Squid-3.4.
@@ -228,6 +271,22 @@
 	</verb>
 	<p>The default value for extras is: "%>a/%>A %un %>rm myip=%la myport=%lp"
 
+	<tag>ftp_port</tag>
+
+	<p>New configuration directive to accept and relay native FTP
+	   commands. Typically used for port 21 traffic.  By default, native
+	   FTP commands are not accepted.
+
+	<tag>ftp_client_idle_timeout</tag>
+
+	<p>This new configuration directive controls how long Squid should
+	   wait for an FTP request on a connection to an ftp_port.  Many FTP
+	   clients do not deal with idle connection closures well,
+	   necessitating a longer default timeout (30 minutes) than
+	   client_idle_pconn_timeout used for incoming HTTP requests (2
+	   minutes). The current default may be changed as we get more
+	   experience with FTP relaying.
+
 </descrip>
 
 <sect1>Changes to existing tags<label id="modifiedtags">

=== modified file 'src/FwdState.cc'
--- src/FwdState.cc	2014-07-21 14:55:27 +0000
+++ src/FwdState.cc	2014-08-10 02:28:33 +0000
@@ -39,6 +39,7 @@
 #include "CacheManager.h"
 #include "CachePeer.h"
 #include "client_side.h"
+#include "clients/forward.h"
 #include "comm/Connection.h"
 #include "comm/ConnOpener.h"
 #include "comm/Loops.h"
@@ -47,7 +48,6 @@
 #include "event.h"
 #include "fd.h"
 #include "fde.h"
-#include "ftp.h"
 #include "FwdState.h"
 #include "globals.h"
 #include "gopher.h"
@@ -719,6 +719,10 @@
     }
 #endif
 
+    // should reach ConnStateData before the dispatched Client job starts
+    CallJobHere1(17, 4, request->clientConnectionManager, ConnStateData,
+                 ConnStateData::notePeerConnection, serverConnection());
+
     dispatch();
 }
 
@@ -813,7 +817,7 @@
         debugs(17,7, "pinned peer connection: " << pinned_connection);
         // pinned_connection may become nil after a pconn race
         if (pinned_connection)
-            serverConn = pinned_connection->validatePinnedConnection(request, serverDestinations[0]->getPeer());
+            serverConn = pinned_connection->borrowPinnedConnection(request, serverDestinations[0]->getPeer());
         else
             serverConn = NULL;
         if (Comm::IsConnOpen(serverConn)) {
@@ -995,7 +999,10 @@
             break;
 
         case AnyP::PROTO_FTP:
-            ftpStart(this);
+            if (request->flags.ftpNative)
+                Ftp::StartRelay(this);
+            else
+                Ftp::StartGateway(this);
             break;
 
         case AnyP::PROTO_CACHE_OBJECT:

=== modified file 'src/HttpHdrCc.h'
--- src/HttpHdrCc.h	2013-11-23 00:58:42 +0000
+++ src/HttpHdrCc.h	2014-08-07 22:33:48 +0000
@@ -75,8 +75,10 @@
     //manipulation for Cache-Control: private header
     bool hasPrivate() const {return isSet(CC_PRIVATE);}
     const String &Private() const {return private_;}
-    void Private(String &v) {
+    void Private(const String &v) {
         setMask(CC_PRIVATE,true);
+        if (!v.size())
+            return;
         // uses append for multi-line headers
         if (private_.size() > 0)
             private_.append(",");
@@ -87,10 +89,12 @@
     //manipulation for Cache-Control: no-cache header
     bool hasNoCache() const {return isSet(CC_NO_CACHE);}
     const String &noCache() const {return no_cache;}
-    void noCache(String &v) {
+    void noCache(const String &v) {
         setMask(CC_NO_CACHE,true);
+        if (!v.size())
+            return;
         // uses append for multi-line headers
-        if (no_cache.size() > 0)
+        if (no_cache.size() > 0 && v.size() > 0)
             no_cache.append(",");
         no_cache.append(v);
     }

=== modified file 'src/HttpHeader.cc'
--- src/HttpHeader.cc	2014-07-31 05:16:21 +0000
+++ src/HttpHeader.cc	2014-08-11 16:09:06 +0000
@@ -170,6 +170,11 @@
     {"Surrogate-Capability", HDR_SURROGATE_CAPABILITY, ftStr},
     {"Surrogate-Control", HDR_SURROGATE_CONTROL, ftPSc},
     {"Front-End-Https", HDR_FRONT_END_HTTPS, ftStr},
+    {"FTP-Command", HDR_FTP_COMMAND, ftStr},
+    {"FTP-Arguments", HDR_FTP_ARGUMENTS, ftStr},
+    {"FTP-Pre", HDR_FTP_PRE, ftStr},
+    {"FTP-Status", HDR_FTP_STATUS, ftInt},
+    {"FTP-Reason", HDR_FTP_REASON, ftStr},
     {"Other:", HDR_OTHER, ftStr}	/* ':' will not allow matches */
 };
 
@@ -772,23 +777,37 @@
     HttpHeaderPos pos = HttpHeaderInitPos;
     const HttpHeaderEntry *e;
     assert(p);
-    debugs(55, 7, "packing hdr: (" << this << ")");
+    debugs(55, 7, this << " into " << p <<
+           (mask_sensitive_info ? " while masking" : ""));
     /* pack all entries one by one */
     while ((e = getEntry(&pos))) {
         if (!mask_sensitive_info) {
             e->packInto(p);
             continue;
         }
+
+        bool maskThisEntry = false;
         switch (e->id) {
         case HDR_AUTHORIZATION:
         case HDR_PROXY_AUTHORIZATION:
+            maskThisEntry = true;
+            break;
+
+        case HDR_FTP_ARGUMENTS:
+            if (const HttpHeaderEntry *cmd = findEntry(HDR_FTP_COMMAND))
+                maskThisEntry = (cmd->value == "PASS");
+            break;
+
+        default:
+            break;
+        }
+        if (maskThisEntry) {
             packerAppend(p, e->name.rawBuf(), e->name.size());
             packerAppend(p, ": ** NOT DISPLAYED **\r\n", 23);
-            break;
-        default:
+        } else {
             e->packInto(p);
-            break;
         }
+
     }
     /* Pack in the "special" entries */
 

=== modified file 'src/HttpHeader.h'
--- src/HttpHeader.h	2014-07-31 05:12:06 +0000
+++ src/HttpHeader.h	2014-08-11 16:09:06 +0000
@@ -45,6 +45,7 @@
 class HttpHdrSc;
 class Packer;
 class StoreEntry;
+class SBuf;
 
 /* constant attributes of http header fields */
 
@@ -148,6 +149,11 @@
     HDR_SURROGATE_CAPABILITY,           /**< Edge Side Includes (ESI) header */
     HDR_SURROGATE_CONTROL,              /**< Edge Side Includes (ESI) header */
     HDR_FRONT_END_HTTPS,                /**< MS Exchange custom header we may have to add */
+    HDR_FTP_COMMAND,                    /**< Internal header for FTP command */
+    HDR_FTP_ARGUMENTS,                  /**< Internal header for FTP command arguments */
+    HDR_FTP_PRE,                        /**< Internal header containing leading FTP control response lines */
+    HDR_FTP_STATUS,                     /**< Internal header for FTP reply status */
+    HDR_FTP_REASON,                     /**< Internal header for FTP reply reason */
     HDR_OTHER,                          /**< internal tag value for "unknown" headers */
     HDR_ENUM_END
 } http_hdr_type;
@@ -298,6 +304,10 @@
 };
 
 int httpHeaderParseQuotedString(const char *start, const int len, String *val);
+
+/// quotes string using RFC 7230 quoted-string rules
+SBuf httpHeaderQuoteString(const char *raw);
+
 int httpHeaderHasByNameListMember(const HttpHeader * hdr, const char *name, const char *member, const char separator);
 void httpHeaderUpdate(HttpHeader * old, const HttpHeader * fresh, const HttpHeaderMask * denied_mask);
 void httpHeaderCalcMask(HttpHeaderMask * mask, http_hdr_type http_hdr_type_enums[], size_t count);

=== modified file 'src/HttpHeaderTools.cc'
--- src/HttpHeaderTools.cc	2014-06-02 07:19:35 +0000
+++ src/HttpHeaderTools.cc	2014-08-10 02:28:33 +0000
@@ -297,6 +297,37 @@
     return 1;
 }
 
+SBuf
+httpHeaderQuoteString(const char *raw)
+{
+    assert(raw);
+
+    // TODO: Optimize by appending a sequence of characters instead of a char.
+    // This optimization may be easier with Tokenizer after raw becomes SBuf.
+
+    // RFC 7230 says a "sender SHOULD NOT generate a quoted-pair in a
+    // quoted-string except where necessary" (i.e., DQUOTE and backslash)
+    bool needInnerQuote = false;
+    for (const char *s = raw; !needInnerQuote &&  *s; ++s)
+        needInnerQuote = *s == '"' || *s == '\\';
+
+    SBuf quotedStr;
+    quotedStr.append('"');
+
+    if (needInnerQuote) {
+        for (const char *s = raw; *s; ++s) {
+            if (*s == '"' || *s == '\\')
+                quotedStr.append('\\');
+            quotedStr.append(*s);
+        }
+    } else {
+        quotedStr.append(raw);
+    }
+
+    quotedStr.append('"');
+    return quotedStr;
+}
+
 /**
  * Checks the anonymizer (header_access) configuration.
  *

=== modified file 'src/HttpReply.h'
--- src/HttpReply.h	2013-10-25 00:13:46 +0000
+++ src/HttpReply.h	2014-05-23 06:11:56 +0000
@@ -131,6 +131,8 @@
     /// Remove Warnings with warn-date different from Date value
     void removeStaleWarnings();
 
+    virtual void hdrCacheInit();
+
 private:
     /** initialize */
     void init();
@@ -161,8 +163,6 @@
     virtual void packFirstLineInto(Packer * p, bool) const { sline.packInto(p); }
 
     virtual bool parseFirstLine(const char *start, const char *end);
-
-    virtual void hdrCacheInit();
 };
 
 MEMPROXY_CLASS_INLINE(HttpReply);

=== modified file 'src/Makefile.am'
--- src/Makefile.am	2014-07-26 13:26:27 +0000
+++ src/Makefile.am	2014-08-07 17:31:05 +0000
@@ -46,8 +46,8 @@
 	LoadableModules.h \
 	LoadableModules.cc
 
-SUBDIRS	= base anyp parser comm eui acl format fs repl
-DIST_SUBDIRS = base anyp parser comm eui acl format fs repl
+SUBDIRS	= base anyp ftp parser comm eui acl format clients servers fs repl
+DIST_SUBDIRS = base anyp ftp parser comm eui acl format clients servers fs repl
 
 if ENABLE_AUTH
 SUBDIRS += auth
@@ -312,6 +312,7 @@
 	ClientRequestContext.h \
 	clientStream.cc \
 	clientStream.h \
+	clientStreamForward.h \
 	CollapsedForwarding.cc \
 	CollapsedForwarding.h \
 	CompletionDispatcher.cc \
@@ -363,8 +364,6 @@
 	filemap.cc \
 	fqdncache.h \
 	fqdncache.cc \
-	ftp.h \
-	ftp.cc \
 	FwdState.cc \
 	FwdState.h \
 	Generic.h \
@@ -636,6 +635,9 @@
 	icmp/libicmp.la icmp/libicmp-core.la \
 	log/liblog.la \
 	format/libformat.la \
+	clients/libclients.la \
+	servers/libservers.la \
+	ftp/libftp.la \
 	$(XTRA_OBJS) \
 	$(DISK_LINKOBJS) \
 	$(REPL_OBJS) \
@@ -671,12 +673,15 @@
 	$(AUTH_LIBS) \
 	acl/libapi.la \
 	base/libbase.la \
+	clients/libclients.la \
+	ftp/libftp.la \
 	libsquid.la \
 	ip/libip.la \
 	fs/libfs.la \
 	format/libformat.la \
 	ipc/libipc.la \
-	mgr/libmgr.la
+	mgr/libmgr.la \
+	servers/libservers.la
 
 if ENABLE_LOADABLE_MODULES
 squid_SOURCES += $(LOADABLE_MODULES_SOURCES)
@@ -1465,8 +1470,6 @@
 	filemap.cc \
 	fqdncache.h \
 	fqdncache.cc \
-	ftp.h \
-	ftp.cc \
 	FwdState.cc \
 	FwdState.h \
 	gopher.h \
@@ -1604,11 +1607,15 @@
 	$(DISKIO_GEN_SOURCE)
 # comm.cc only requires comm/libcomm.la until fdc_table is dead.
 tests_testCacheManager_LDADD = \
+	clients/libclients.la \
+	servers/libservers.la \
 	http/libsquid-http.la \
+	ftp/libftp.la \
 	ident/libident.la \
 	acl/libacls.la \
 	acl/libstate.la \
 	acl/libapi.la \
+	parser/libsquid-parser.la \
 	base/libbase.la \
 	libsquid.la \
 	ip/libip.la \
@@ -1884,8 +1891,6 @@
 	filemap.cc \
 	fqdncache.h \
 	fqdncache.cc \
-	ftp.h \
-	ftp.cc \
 	FwdState.cc \
 	FwdState.h \
 	gopher.h \
@@ -2038,11 +2043,15 @@
 	$(BUILT_SOURCES) \
 	$(DISKIO_GEN_SOURCE)
 tests_testEvent_LDADD = \
+	clients/libclients.la \
+	servers/libservers.la \
 	http/libsquid-http.la \
+	ftp/libftp.la \
 	ident/libident.la \
 	acl/libacls.la \
 	acl/libstate.la \
 	acl/libapi.la \
+	parser/libsquid-parser.la \
 	base/libbase.la \
 	libsquid.la \
 	ip/libip.la \
@@ -2134,8 +2143,6 @@
 	filemap.cc \
 	fqdncache.h \
 	fqdncache.cc \
-	ftp.h \
-	ftp.cc \
 	FwdState.cc \
 	FwdState.h \
 	gopher.h \
@@ -2288,11 +2295,15 @@
 	$(BUILT_SOURCES) \
 	$(DISKIO_GEN_SOURCE)
 tests_testEventLoop_LDADD = \
+	clients/libclients.la \
+	servers/libservers.la \
 	http/libsquid-http.la \
+	ftp/libftp.la \
 	ident/libident.la \
 	acl/libacls.la \
 	acl/libstate.la \
 	acl/libapi.la \
+	parser/libsquid-parser.la \
 	base/libbase.la \
 	libsquid.la \
 	ip/libip.la \
@@ -2380,8 +2391,6 @@
 	filemap.cc \
 	fqdncache.h \
 	fqdncache.cc \
-	ftp.h \
-	ftp.cc \
 	FwdState.cc \
 	FwdState.h \
 	gopher.h \
@@ -2531,11 +2540,15 @@
 	$(BUILT_SOURCES) \
 	$(DISKIO_GEN_SOURCE)
 tests_test_http_range_LDADD = \
+	clients/libclients.la \
+	servers/libservers.la \
 	http/libsquid-http.la \
+	ftp/libftp.la \
 	ident/libident.la \
 	acl/libacls.la \
 	acl/libstate.la \
 	acl/libapi.la \
+	parser/libsquid-parser.la \
 	libsquid.la \
 	ip/libip.la \
 	fs/libfs.la \
@@ -2688,8 +2701,6 @@
 	fde.cc \
 	fqdncache.h \
 	fqdncache.cc \
-	ftp.h \
-	ftp.cc \
 	FwdState.cc \
 	FwdState.h \
 	gopher.h \
@@ -2823,10 +2834,14 @@
 nodist_tests_testHttpRequest_SOURCES = \
 	$(BUILT_SOURCES)
 tests_testHttpRequest_LDADD = \
+	clients/libclients.la \
+	servers/libservers.la \
+	ftp/libftp.la \
 	ident/libident.la \
 	acl/libacls.la \
 	acl/libstate.la \
 	acl/libapi.la \
+	parser/libsquid-parser.la \
 	libsquid.la \
 	ip/libip.la \
 	fs/libfs.la \
@@ -3508,8 +3523,6 @@
 	filemap.cc \
 	fqdncache.h \
 	fqdncache.cc \
-	ftp.h \
-	ftp.cc \
 	FwdState.cc \
 	FwdState.h \
 	gopher.h \
@@ -3664,13 +3677,17 @@
 nodist_tests_testURL_SOURCES = \
 	$(BUILT_SOURCES)
 tests_testURL_LDADD = \
+	clients/libclients.la \
+	servers/libservers.la \
 	http/libsquid-http.la \
+	ftp/libftp.la \
 	anyp/libanyp.la \
 	ident/libident.la \
 	acl/libacls.la \
 	eui/libeui.la \
 	acl/libstate.la \
 	acl/libapi.la \
+	parser/libsquid-parser.la \
 	base/libbase.la \
 	libsquid.la \
 	ip/libip.la \

=== modified file 'src/RequestFlags.h'
--- src/RequestFlags.h	2013-05-13 03:57:03 +0000
+++ src/RequestFlags.h	2014-08-04 21:44:31 +0000
@@ -127,6 +127,8 @@
     bool done_follow_x_forwarded_for :1;
     /** set for ssl-bumped requests */
     bool sslBumped :1;
+    /// carries a representation of an FTP command [received on ftp_port]
+    bool ftpNative :1;
     bool destinationIpLookedUp:1;
     /** request to reset the TCP stream */
     bool resetTcp:1;

=== modified file 'src/SBuf.cc'
--- src/SBuf.cc	2014-05-19 06:06:36 +0000
+++ src/SBuf.cc	2014-08-07 17:13:26 +0000
@@ -237,6 +237,12 @@
     return lowAppend(S, Ssize);
 }
 
+SBuf &
+SBuf::append(const char c)
+{
+    return lowAppend(&c, 1);
+}
+
 SBuf&
 SBuf::Printf(const char *fmt, ...)
 {
@@ -834,34 +840,30 @@
     return os;
 }
 
-SBuf
-SBuf::toLower() const
+void
+SBuf::toLower()
 {
     debugs(24, 8, "\"" << *this << "\"");
-    SBuf rv(*this);
     for (size_type j = 0; j < length(); ++j) {
         const int c = (*this)[j];
         if (isupper(c))
-            rv.setAt(j, tolower(c)); //will cow() if needed
+            setAt(j, tolower(c));
     }
-    debugs(24, 8, "result: \"" << rv << "\"");
+    debugs(24, 8, "result: \"" << *this << "\"");
     ++stats.caseChange;
-    return rv;
 }
 
-SBuf
-SBuf::toUpper() const
+void
+SBuf::toUpper()
 {
     debugs(24, 8, "\"" << *this << "\"");
-    SBuf rv(*this);
     for (size_type j = 0; j < length(); ++j) {
         const int c = (*this)[j];
         if (islower(c))
-            rv.setAt(j, toupper(c)); //will cow() if needed
+            setAt(j, toupper(c));
     }
-    debugs(24, 8, "result: \"" << rv << "\"");
+    debugs(24, 8, "result: \"" << *this << "\"");
     ++stats.caseChange;
-    return rv;
 }
 
 /**

=== modified file 'src/SBuf.h'
--- src/SBuf.h	2014-04-21 17:14:44 +0000
+++ src/SBuf.h	2014-08-07 17:13:26 +0000
@@ -182,6 +182,9 @@
      */
     SBuf& append(const SBuf & S);
 
+    /// Append a single character. The character may be NUL (\0).
+    SBuf& append(const char c);
+
     /** Append operation for C-style strings.
      *
      * Append the supplied c-string to the SBuf; extend storage
@@ -543,19 +546,11 @@
      */
     int scanf(const char *format, ...);
 
-    /** Lower-case SBuf
-     *
-     * Returns a lower-cased COPY of the SBuf
-     * \see man tolower(3)
-     */
-    SBuf toLower() const;
+    /// converts all characters to lower case; \see man tolower(3)
+    void toLower();
 
-    /** Upper-case SBuf
-     *
-     * Returns an upper-cased COPY of the SBuf
-     * \see man toupper(3)
-     */
-    SBuf toUpper() const;
+    /// converts all characters to upper case; \see man toupper(3)
+    void toUpper();
 
     /** String export function
      * converts the SBuf to a legacy String, by copy.
@@ -628,4 +623,20 @@
     return S.print(os);
 }
 
+/// Returns a lower-cased copy of its parameter.
+inline SBuf
+ToUpper(SBuf buf)
+{
+    buf.toUpper();
+    return buf;
+}
+
+/// Returns an upper-cased copy of its parameter.
+inline SBuf
+ToLower(SBuf buf)
+{
+    buf.toLower();
+    return buf;
+}
+
 #endif /* SQUID_SBUF_H */

=== modified file 'src/Server.cc'
--- src/Server.cc	2014-06-05 14:57:58 +0000
+++ src/Server.cc	2014-07-30 17:33:14 +0000
@@ -831,10 +831,11 @@
     debugs(11,5, HERE << "handleAdaptationCompleted");
     cleanAdaptation();
 
-    // We stop reading origin response because we have no place to put it and
+    // We stop reading origin response because we have no place to put it(*) and
     // cannot use it. If some origin servers do not like that or if we want to
     // reuse more pconns, we can add code to discard unneeded origin responses.
-    if (!doneWithServer()) {
+    // (*) TODO: Is it possible that the adaptation xaction is still running?
+    if (mayReadVirginReplyBody()) {
         debugs(11,3, HERE << "closing origin conn due to ICAP completion");
         closeServer();
     }

=== modified file 'src/Server.h'
--- src/Server.h	2013-12-18 17:19:00 +0000
+++ src/Server.h	2014-08-10 23:18:33 +0000
@@ -127,6 +127,8 @@
 
     virtual void closeServer() = 0;            /**< end communication with the server */
     virtual bool doneWithServer() const = 0;   /**< did we end communication? */
+    /// whether we may receive more virgin response body bytes
+    virtual bool mayReadVirginReplyBody() const = 0;
 
     /// Entry-dependent callbacks use this check to quit if the entry went bad
     bool abortOnBadEntry(const char *abortReason);

=== modified file 'src/SquidConfig.h'
--- src/SquidConfig.h	2014-08-02 13:01:28 +0000
+++ src/SquidConfig.h	2014-08-11 16:09:06 +0000
@@ -108,6 +108,7 @@
         time_t request;
         time_t clientIdlePconn;
         time_t serverIdlePconn;
+        time_t ftpClientIdle;
         time_t siteSelect;
         time_t deadPeer;
         int icp_query;      /* msec */

=== modified file 'src/acl/forward.h'
--- src/acl/forward.h	2013-05-28 15:05:58 +0000
+++ src/acl/forward.h	2014-08-04 21:44:31 +0000
@@ -21,6 +21,9 @@
 
 } // namespace Acl
 
+class allow_t;
+typedef void ACLCB(allow_t, void *);
+
 #define ACL_NAME_SZ 64
 
 // TODO: Consider renaming all users and removing. Cons: hides the difference

=== modified file 'src/anyp/PortCfg.cc'
--- src/anyp/PortCfg.cc	2014-07-31 14:02:20 +0000
+++ src/anyp/PortCfg.cc	2014-08-11 16:09:06 +0000
@@ -13,6 +13,7 @@
 #if USE_OPENSSL
 AnyP::PortCfgPointer HttpsPortList;
 #endif
+AnyP::PortCfgPointer FtpPortList;
 
 int NHttpSockets = 0;
 int HttpSockets[MAXTCPLISTENPORTS];
@@ -29,6 +30,7 @@
         actAsOrigin(false),
         ignore_cc(false),
         connection_auth_disabled(false),
+        ftp_track_dirs(false),
         vport(0),
         disable_pmtu_discovery(0),
         listenConn()
@@ -105,6 +107,7 @@
     b->vhost = vhost;
     b->vport = vport;
     b->connection_auth_disabled = connection_auth_disabled;
+    b->ftp_track_dirs = ftp_track_dirs;
     b->disable_pmtu_discovery = disable_pmtu_discovery;
     b->tcp_keepalive = tcp_keepalive;
 
@@ -185,19 +188,3 @@
     }
 }
 #endif
-
-void
-AnyP::PortCfg::setTransport(const char *aProtocol)
-{
-    // HTTP/1.0 not supported because we are version 1.1 which contains a superset of 1.0
-    // and RFC 2616 requires us to upgrade 1.0 to 1.1
-
-    if (strcasecmp("http", aProtocol) == 0 || strcmp("HTTP/1.1", aProtocol) == 0)
-        transport = AnyP::ProtocolVersion(AnyP::PROTO_HTTP, 1,1);
-
-    else if (strcasecmp("https", aProtocol) == 0 || strcmp("HTTPS/1.1", aProtocol) == 0)
-        transport = AnyP::ProtocolVersion(AnyP::PROTO_HTTPS, 1,1);
-
-    else
-        fatalf("http(s)_port protocol=%s is not supported\n", aProtocol);
-}

=== modified file 'src/anyp/PortCfg.h'
--- src/anyp/PortCfg.h	2014-07-29 10:28:17 +0000
+++ src/anyp/PortCfg.h	2014-08-09 00:35:28 +0000
@@ -24,13 +24,6 @@
     void configureSslServerContext();
 #endif
 
-    /**
-     * Set this ports transport type from a string representation.
-     * Unknown transport type representations will halt Squid.
-     * Supports: HTTP, HTTP/1.1, HTTPS, HTTPS/1.1.
-     */
-    void setTransport(const char *aProtocol);
-
     PortCfgPointer next;
 
     Ip::Address s;
@@ -47,6 +40,8 @@
 
     bool connection_auth_disabled; ///< Don't support connection oriented auth
 
+    bool ftp_track_dirs; ///< whether transactions should track FTP directories
+
     int vport;               ///< virtual port support. -1 if dynamic, >0 static
     int disable_pmtu_discovery;
 
@@ -106,6 +101,9 @@
 extern AnyP::PortCfgPointer HttpsPortList;
 #endif
 
+/// list of Squid ftp_port configured
+extern AnyP::PortCfgPointer FtpPortList;
+
 #if !defined(MAXTCPLISTENPORTS)
 // Max number of TCP listening ports
 #define MAXTCPLISTENPORTS 128

=== modified file 'src/base/CharacterSet.cc'
--- src/base/CharacterSet.cc	2014-06-11 10:31:56 +0000
+++ src/base/CharacterSet.cc	2014-08-07 17:20:27 +0000
@@ -1,6 +1,9 @@
 #include "squid.h"
 #include "CharacterSet.h"
 
+#include <algorithm>
+#include <functional>
+
 CharacterSet &
 CharacterSet::operator +=(const CharacterSet &src)
 {
@@ -44,6 +47,16 @@
     return *this;
 }
 
+CharacterSet
+CharacterSet::complement(const char *label) const
+{
+    CharacterSet result((label ? label : "complement_of_some_other_set"), "");
+    // negate each of our elements and add them to the result storage
+    std::transform(chars_.begin(), chars_.end(), result.chars_.begin(),
+                   std::logical_not<Storage::value_type>());
+    return result;
+}
+
 CharacterSet::CharacterSet(const char *label, const char * const c) :
         name(label == NULL ? "anonymous" : label),
         chars_(Storage(256,0))

=== modified file 'src/base/CharacterSet.h'
--- src/base/CharacterSet.h	2014-06-07 10:23:47 +0000
+++ src/base/CharacterSet.h	2014-08-07 17:20:27 +0000
@@ -33,6 +33,12 @@
     /// return a new CharacterSet containing the union of two sets
     CharacterSet operator +(const CharacterSet &src) const;
 
+    /// return a new CharacterSet containing characters not in this set
+    CharacterSet complement(const char *complementLabel = NULL) const;
+
+    /// change name; handy in const declarations that use operators
+    CharacterSet &rename(const char *label) { name = label; return *this; }
+
     /// optional set label for debugging (default: "anonymous")
     const char * name;
 

=== modified file 'src/cache_cf.cc'
--- src/cache_cf.cc	2014-07-21 14:55:27 +0000
+++ src/cache_cf.cc	2014-08-10 23:18:33 +0000
@@ -52,6 +52,7 @@
 #include "eui/Config.h"
 #include "ExternalACL.h"
 #include "format/Format.h"
+#include "ftp/Elements.h"
 #include "globals.h"
 #include "HttpHeaderTools.h"
 #include "HttpRequestMethod.h"
@@ -3566,6 +3567,27 @@
     }
 }
 
+/// parses the protocol= option of the *_port directive, returning parsed value
+/// unsupported option values result in a fatal error message
+/// upper case values required; caller may convert for backward compatibility
+static AnyP::ProtocolVersion
+parsePortProtocol(const SBuf &value)
+{
+    // HTTP/1.0 not supported because we are version 1.1 which contains a superset of 1.0
+    // and RFC 2616 requires us to upgrade 1.0 to 1.1
+    if (value.cmp("HTTP") == 0 || value.cmp("HTTP/1.1") == 0)
+        return AnyP::ProtocolVersion(AnyP::PROTO_HTTP, 1,1);
+
+    if (value.cmp("HTTPS") == 0 || value.cmp("HTTPS/1.1") == 0)
+        return AnyP::ProtocolVersion(AnyP::PROTO_HTTPS, 1,1);
+
+    if (value.cmp("FTP") == 0)
+        return Ftp::ProtocolVersion();
+
+    fatalf("%s directive does not support protocol=" SQUIDSBUFPH "\n", cfg_directive, SQUIDSBUFPRINT(value));
+    return AnyP::ProtocolVersion(); // not reached
+}
+
 static void
 parse_port_option(AnyP::PortCfgPointer &s, char *token)
 {
@@ -3573,14 +3595,14 @@
 
     if (strcmp(token, "accel") == 0) {
         if (s->flags.isIntercepted()) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: Accelerator mode requires its own port. It cannot be shared with other modes.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": Accelerator mode requires its own port. It cannot be shared with other modes.");
             self_destruct();
         }
         s->flags.accelSurrogate = true;
         s->vhost = true;
     } else if (strcmp(token, "transparent") == 0 || strcmp(token, "intercept") == 0) {
         if (s->flags.accelSurrogate || s->flags.tproxyIntercept) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: Intercept mode requires its own interception port. It cannot be shared with other modes.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": Intercept mode requires its own interception port. It cannot be shared with other modes.");
             self_destruct();
         }
         s->flags.natIntercept = true;
@@ -3590,7 +3612,7 @@
         debugs(3, DBG_IMPORTANT, "Disabling Authentication on port " << s->s << " (interception enabled)");
     } else if (strcmp(token, "tproxy") == 0) {
         if (s->flags.natIntercept || s->flags.accelSurrogate) {
-            debugs(3,DBG_CRITICAL, "FATAL: http(s)_port: TPROXY option requires its own interception port. It cannot be shared with other modes.");
+            debugs(3,DBG_CRITICAL, "FATAL: " << cfg_directive << ": TPROXY option requires its own interception port. It cannot be shared with other modes.");
             self_destruct();
         }
         s->flags.tproxyIntercept = true;
@@ -3599,61 +3621,61 @@
         debugs(3, DBG_IMPORTANT, "Disabling Authentication on port " << s->s << " (TPROXY enabled)");
 
         if (!Ip::Interceptor.ProbeForTproxy(s->s)) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: TPROXY support in the system does not work.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": TPROXY support in the system does not work.");
             self_destruct();
         }
 
     } else if (strncmp(token, "defaultsite=", 12) == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: defaultsite option requires Acceleration mode flag.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": defaultsite option requires Acceleration mode flag.");
             self_destruct();
         }
         safe_free(s->defaultsite);
         s->defaultsite = xstrdup(token + 12);
     } else if (strcmp(token, "vhost") == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_CRITICAL, "WARNING: http(s)_port: vhost option is deprecated. Use 'accel' mode flag instead.");
+            debugs(3, DBG_CRITICAL, "WARNING: " << cfg_directive << ": vhost option is deprecated. Use 'accel' mode flag instead.");
         }
         s->flags.accelSurrogate = true;
         s->vhost = true;
     } else if (strcmp(token, "no-vhost") == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_IMPORTANT, "ERROR: http(s)_port: no-vhost option requires Acceleration mode flag.");
+            debugs(3, DBG_IMPORTANT, "ERROR: " << cfg_directive << ": no-vhost option requires Acceleration mode flag.");
         }
         s->vhost = false;
     } else if (strcmp(token, "vport") == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: vport option requires Acceleration mode flag.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": vport option requires Acceleration mode flag.");
             self_destruct();
         }
         s->vport = -1;
     } else if (strncmp(token, "vport=", 6) == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: vport option requires Acceleration mode flag.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": vport option requires Acceleration mode flag.");
             self_destruct();
         }
         s->vport = xatos(token + 6);
     } else if (strncmp(token, "protocol=", 9) == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: protocol option requires Acceleration mode flag.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": protocol option requires Acceleration mode flag.");
             self_destruct();
         }
-        s->setTransport(token + 9);
+        s->transport = parsePortProtocol(ToUpper(SBuf(token + 9)));
     } else if (strcmp(token, "allow-direct") == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: allow-direct option requires Acceleration mode flag.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": allow-direct option requires Acceleration mode flag.");
             self_destruct();
         }
         s->allow_direct = true;
     } else if (strcmp(token, "act-as-origin") == 0) {
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_IMPORTANT, "ERROR: http(s)_port: act-as-origin option requires Acceleration mode flag.");
+            debugs(3, DBG_IMPORTANT, "ERROR: " << cfg_directive << ": act-as-origin option requires Acceleration mode flag.");
         } else
             s->actAsOrigin = true;
     } else if (strcmp(token, "ignore-cc") == 0) {
 #if !USE_HTTP_VIOLATIONS
         if (!s->flags.accelSurrogate) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: ignore-cc option requires Acceleration mode flag.");
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": ignore-cc option requires Acceleration mode flag.");
             self_destruct();
         }
 #endif
@@ -3680,7 +3702,7 @@
             self_destruct();
     } else if (strcmp(token, "ipv4") == 0) {
         if ( !s->s.setIPv4() ) {
-            debugs(3, DBG_CRITICAL, "FATAL: http(s)_port: IPv6 addresses cannot be used as IPv4-Only. " << s->s );
+            debugs(3, DBG_CRITICAL, "FATAL: " << cfg_directive << ": IPv6 addresses cannot be used as IPv4-Only. " << s->s );
             self_destruct();
         }
     } else if (strcmp(token, "tcpkeepalive") == 0) {
@@ -3702,7 +3724,7 @@
 #if USE_OPENSSL
     } else if (strcmp(token, "sslBump") == 0) {
         debugs(3, DBG_CRITICAL, "WARNING: '" << token << "' is deprecated " <<
-               "in http_port. Use 'ssl-bump' instead.");
+               "in " << cfg_directive << ". Use 'ssl-bump' instead.");
         s->flags.tunnelSslBumping = true;
     } else if (strcmp(token, "ssl-bump") == 0) {
         s->flags.tunnelSslBumping = true;
@@ -3752,8 +3774,10 @@
     } else if (strncmp(token, "dynamic_cert_mem_cache_size=", 28) == 0) {
         parseBytesOptionValue(&s->dynamicCertMemCacheSize, B_BYTES_STR, token + 28);
 #endif
+    } else if (strcmp(token, "ftp-track-dirs") == 0) {
+        s->ftp_track_dirs = true;
     } else {
-        debugs(3, DBG_CRITICAL, "FATAL: Unknown http(s)_port option '" << token << "'.");
+        debugs(3, DBG_CRITICAL, "FATAL: Unknown " << cfg_directive << " option '" << token << "'.");
         self_destruct();
     }
 }
@@ -3762,7 +3786,7 @@
 add_http_port(char *portspec)
 {
     AnyP::PortCfgPointer s = new AnyP::PortCfg();
-    s->setTransport("HTTP");
+    s->transport = parsePortProtocol(SBuf("HTTP"));
     parsePortSpecification(s, portspec);
     // we may need to merge better if the above returns a list with clones
     assert(s->next == NULL);
@@ -3773,13 +3797,15 @@
 static void
 parsePortCfg(AnyP::PortCfgPointer *head, const char *optionName)
 {
-    const char *protocol = NULL;
+    SBuf protoName;
     if (strcmp(optionName, "http_port") == 0 ||
             strcmp(optionName, "ascii_port") == 0)
-        protocol = "http";
+        protoName = "HTTP";
     else if (strcmp(optionName, "https_port") == 0)
-        protocol = "https";
-    if (!protocol) {
+        protoName = "HTTPS";
+    else if (strcmp(optionName, "ftp_port") == 0)
+        protoName = "FTP";
+    if (protoName.isEmpty()) {
         self_destruct();
         return;
     }
@@ -3792,7 +3818,7 @@
     }
 
     AnyP::PortCfgPointer s = new AnyP::PortCfg();
-    s->setTransport(protocol);
+    s->transport = parsePortProtocol(protoName); // default; protocol=... overwrites
     parsePortSpecification(s, token);
 
     /* parse options ... */
@@ -3812,6 +3838,12 @@
             debugs(3, DBG_CRITICAL, "FATAL: tproxy/intercept on https_port requires ssl-bump which is missing.");
             self_destruct();
         }
+    } else if (protoName.cmp("FTP") == 0) {
+        /* ftp_port does not support ssl-bump */
+        if (s->flags.tunnelSslBumping) {
+            debugs(3, DBG_CRITICAL, "FATAL: ssl-bump is not supported for ftp_port.");
+            self_destruct();
+        }
     }
 #endif
 

=== modified file 'src/cf.data.pre'
--- src/cf.data.pre	2014-08-02 14:03:21 +0000
+++ src/cf.data.pre	2014-08-11 16:09:06 +0000
@@ -913,7 +913,7 @@
 	acl aclname localport 3128 ...	      # TCP port the client connected to [fast]
 	                                      # NP: for interception mode this is usually '80'
 
-	acl aclname myportname 3128 ...       # http(s)_port name [fast]
+	acl aclname myportname 3128 ...       # *_port name [fast]
 
 	acl aclname proto HTTP FTP ...        # request protocol [fast]
  
@@ -1290,7 +1290,7 @@
 DOC_START
 	Allowing or Denying access based on defined access lists
 
-	Access to the HTTP port:
+	To allow or deny a message received on an HTTP, HTTPS, or FTP port:
 	http_access allow|deny [!]aclname ...
 
 	NOTE on default values:
@@ -1921,6 +1921,70 @@
 	See http_port for a list of available options.
 DOC_END
 
+NAME: ftp_port
+TYPE: PortCfg
+DEFAULT: none
+LOC: FtpPortList
+DOC_START
+	Enables Native FTP proxy by specifying the socket address where Squid
+	listens for FTP client requests. See http_port directive for various
+	ways to specify the listening address and mode.
+
+	Usage: ftp_port address [mode] [options]
+
+	WARNING: This is a new, experimental, complex feature that has seen
+	limited production exposure. Some Squid modules (e.g., caching) do not
+	currently work with native FTP proxying, and many features have not
+	even been tested for compatibility. Test well before deploying!
+
+	Native FTP proxying differs substantially from proxying HTTP requests
+	with ftp:// URIs because Squid works as an FTP server and receives
+	actual FTP commands (rather than HTTP requests with FTP URLs).
+
+	Native FTP commands accepted at ftp_port are internally converted or
+	wrapped into HTTP-like messages. The same happens to Native FTP
+	responses received from FTP origin servers. Those HTTP-like messages
+	are shoveled through regular access control and adaptation layers
+	between the FTP client and the FTP origin server. This allows Squid to
+	examine, adapt, block, and log FTP exchanges. Squid reuses most HTTP
+	mechanisms when shoveling wrapped FTP messages. For example,
+	http_access and adaptation_access directives are used.
+
+	Modes:
+
+	   intercept	Same as http_port intercept. The FTP origin address is
+			determined based on the intended destination of the
+			intercepted connection.
+
+	   tproxy	Support Linux TPROXY for spoofing outgoing
+			connections using the client IP address.
+			NP: disables authentication and maybe IPv6 on the port.
+
+	By default (i.e., without an explicit mode option), Squid extracts the
+	FTP origin address from the login@origin parameter of the FTP USER
+	command. Many popular FTP clients support such native FTP proxying.
+
+	Options:
+
+	   name=token	Specifies an internal name for the port. Defaults to
+			the port address. Usable with myportname ACL.
+
+	   ftp-track-dirs
+			Enables tracking of FTP directories by injecting extra
+			PWD commands and adjusting Request-URI (in wrapping
+			HTTP requests) to reflect the current FTP server
+			directory. Tracking is disabled by default.
+
+	   protocol=FTP	Protocol to reconstruct accelerated and intercepted
+			requests with. Defaults to FTP. No other accepted
+			values have been tested with. An unsupported value
+			results in a FATAL error. Accepted values are FTP,
+			HTTP (or HTTP/1.1), and HTTPS (or HTTPS/1.1).
+
+	Other http_port modes and options that are not specific to HTTP and
+	HTTPS may also work.
+DOC_END
+
 NAME: tcp_outgoing_tos tcp_outgoing_ds tcp_outgoing_dscp
 TYPE: acl_tos
 DEFAULT: none
@@ -5857,6 +5921,17 @@
 	client connection after the previous request completes.
 DOC_END
 
+NAME: ftp_client_idle_timeout
+TYPE: time_t
+LOC: Config.Timeout.ftpClientIdle
+DEFAULT: 30 minutes
+DOC_START
+	How long to wait for an FTP request on a connection to Squid ftp_port.
+	Many FTP clients do not deal with idle connection closures well,
+	necessitating a longer default timeout than client_idle_pconn_timeout
+	used for incoming HTTP requests.
+DOC_END
+
 NAME: client_lifetime
 COMMENT: time-units
 TYPE: time_t

=== modified file 'src/clientStream.h'
--- src/clientStream.h	2012-10-29 04:59:58 +0000
+++ src/clientStream.h	2014-08-08 03:07:03 +0000
@@ -33,6 +33,7 @@
 #define SQUID_CLIENTSTREAM_H
 
 #include "base/RefCount.h"
+#include "clientStreamForward.h"
 #include "dlink.h"
 #include "StoreIOBuffer.h"
 
@@ -96,28 +97,6 @@
  */
 
 /// \ingroup ClientStreamAPI
-typedef RefCount<Lock> ClientStreamData;
-
-class clientStreamNode;
-class ClientHttpRequest;
-class HttpReply;
-
-/* client stream read callback */
-/// \ingroup ClientStreamAPI
-typedef void CSCB(clientStreamNode *, ClientHttpRequest *, HttpReply *, StoreIOBuffer);
-
-/* client stream read */
-/// \ingroup ClientStreamAPI
-typedef void CSR(clientStreamNode *, ClientHttpRequest *);
-
-/* client stream detach */
-/// \ingroup ClientStreamAPI
-typedef void CSD(clientStreamNode *, ClientHttpRequest *);
-
-/// \ingroup ClientStreamAPI
-typedef clientStream_status_t CSS(clientStreamNode *, ClientHttpRequest *);
-
-/// \ingroup ClientStreamAPI
 class clientStreamNode
 {
 

=== added file 'src/clientStreamForward.h'
--- src/clientStreamForward.h	1970-01-01 00:00:00 +0000
+++ src/clientStreamForward.h	2014-08-10 02:28:33 +0000
@@ -0,0 +1,29 @@
+#ifndef SQUID_CLIENTSTREAM_FORWARD_H
+#define SQUID_CLIENTSTREAM_FORWARD_H
+
+#include "enums.h" /* for clientStream_status_t */
+
+class Lock;
+template <class C> class RefCount;
+
+typedef RefCount<Lock> ClientStreamData;
+
+/* Callbacks for ClientStreams API */
+
+class clientStreamNode;
+class ClientHttpRequest;
+class HttpReply;
+class StoreIOBuffer;
+
+/// client stream read callback
+typedef void CSCB(clientStreamNode *, ClientHttpRequest *, HttpReply *, StoreIOBuffer);
+
+/// client stream read
+typedef void CSR(clientStreamNode *, ClientHttpRequest *);
+
+/// client stream detach
+typedef void CSD(clientStreamNode *, ClientHttpRequest *);
+
+typedef clientStream_status_t CSS(clientStreamNode *, ClientHttpRequest *);
+
+#endif /* SQUID_CLIENTSTREAM_FORWARD_H */

=== modified file 'src/client_side.cc'
--- src/client_side.cc	2014-07-16 12:10:11 +0000
+++ src/client_side.cc	2014-08-11 16:09:06 +0000
@@ -121,6 +121,7 @@
 #include "mime_header.h"
 #include "profiler/Profiler.h"
 #include "rfc1738.h"
+#include "servers/forward.h"
 #include "SquidConfig.h"
 #include "SquidTime.h"
 #include "StatCounters.h"
@@ -197,13 +198,9 @@
 #endif
 static CTCB clientLifetimeTimeout;
 static ClientSocketContext *parseHttpRequestAbort(ConnStateData * conn, const char *uri);
-static ClientSocketContext *parseHttpRequest(ConnStateData *, HttpParser *, HttpRequestMethod *, Http::ProtocolVersion *);
 #if USE_IDENT
 static IDCB clientIdentDone;
 #endif
-static CSCB clientSocketRecipient;
-static CSD clientSocketDetach;
-static void clientSetKeepaliveFlag(ClientHttpRequest *);
 static int clientIsContentLengthValid(HttpRequest * r);
 static int clientIsRequestBodyTooLargeForPolicy(int64_t bodyLength);
 
@@ -215,7 +212,6 @@
 #ifndef PURIFY
 static bool connIsUsable(ConnStateData * conn);
 #endif
-static int responseFinishedOrFailed(HttpReply * rep, StoreIOBuffer const &receivedData);
 static void ClientSocketContextPushDeferredIfNeeded(ClientSocketContext::Pointer deferredRequest, ConnStateData * conn);
 static void clientUpdateSocketStats(LogTags logType, size_t size);
 
@@ -237,6 +233,12 @@
     return (clientStreamNode *)http->client_stream.tail->prev->data;
 }
 
+ConnStateData *
+ClientSocketContext::getConn() const
+{
+    return http->getConn();
+}
+
 /**
  * This routine should be called to grow the in.buf and then
  * call Comm::Read().
@@ -349,28 +351,16 @@
 void
 ClientSocketContext::writeControlMsg(HttpControlMsg &msg)
 {
-    const HttpReply::Pointer rep(msg.reply);
+    HttpReply::Pointer rep(msg.reply);
     Must(rep != NULL);
 
-    // apply selected clientReplyContext::buildReplyHeader() mods
-    // it is not clear what headers are required for control messages
-    rep->header.removeHopByHopEntries();
-    rep->header.putStr(HDR_CONNECTION, "keep-alive");
-    httpHdrMangleList(&rep->header, http->request, ROR_REPLY);
-
     // remember the callback
     cbControlMsgSent = msg.cbSuccess;
 
-    MemBuf *mb = rep->pack();
-
-    debugs(11, 2, "HTTP Client " << clientConnection);
-    debugs(11, 2, "HTTP Client CONTROL MSG:\n---------\n" << mb->buf << "\n----------");
-
     AsyncCall::Pointer call = commCbCall(33, 5, "ClientSocketContext::wroteControlMsg",
                                          CommIoCbPtrFun(&WroteControlMsg, this));
-    Comm::Write(clientConnection, mb, call);
 
-    delete mb;
+    getConn()->writeControlMsgAndCall(this, rep.getRaw(), call);
 }
 
 /// called when we wrote the 1xx response
@@ -392,6 +382,9 @@
     // close on 1xx errors to be conservative and to simplify the code
     // (if we do not close, we must notify the source of a failure!)
     conn->close();
+
+    // XXX: writeControlMsgAndCall() should handle writer-specific writing
+    // results, including errors and then call us with success/failure outcome.
 }
 
 /// wroteControlMsg() wrapper: ClientSocketContext is not an AsyncJob
@@ -616,7 +609,7 @@
 
     debugs(33, 9, "clientLogRequest: http.code='" << al->http.code << "'");
 
-    if (loggingEntry() && loggingEntry()->mem_obj)
+    if (loggingEntry() && loggingEntry()->mem_obj && loggingEntry()->objectLen() >= 0)
         al->cache.objectSize = loggingEntry()->contentLen(); // payload duplicate ?? with or without TE ?
 
     al->http.clientRequestSz.header = req_sz;
@@ -845,7 +838,7 @@
     assert(areAllContextsForThisConnection());
     freeAllContexts();
 
-    unpinConnection();
+    unpinConnection(true);
 
     if (Comm::IsConnOpen(clientConnection))
         clientConnection->close();
@@ -892,7 +885,7 @@
  * to set this relatively early in the request processing
  * to handle hacks for broken servers and clients.
  */
-static void
+void
 clientSetKeepaliveFlag(ClientHttpRequest * http)
 {
     HttpRequest *request = http->request;
@@ -965,15 +958,6 @@
     return;
 }
 
-int
-responseFinishedOrFailed(HttpReply * rep, StoreIOBuffer const & receivedData)
-{
-    if (rep == NULL && receivedData.data == NULL && receivedData.length == 0)
-        return 1;
-
-    return 0;
-}
-
 bool
 ClientSocketContext::startOfOutput() const
 {
@@ -1470,7 +1454,7 @@
  *   data context is not NULL
  *   There are no more entries in the stream chain.
  */
-static void
+void
 clientSocketRecipient(clientStreamNode * node, ClientHttpRequest * http,
                       HttpReply * rep, StoreIOBuffer receivedData)
 {
@@ -1490,30 +1474,10 @@
 
     /* TODO: check offset is what we asked for */
 
-    if (context != http->getConn()->getCurrentContext()) {
+    if (context != http->getConn()->getCurrentContext())
         context->deferRecipientForLater(node, rep, receivedData);
-        PROF_stop(clientSocketRecipient);
-        return;
-    }
-
-    // After sending Transfer-Encoding: chunked (at least), always send
-    // the last-chunk if there was no error, ignoring responseFinishedOrFailed.
-    const bool mustSendLastChunk = http->request->flags.chunkedReply &&
-                                   !http->request->flags.streamError && !context->startOfOutput();
-    if (responseFinishedOrFailed(rep, receivedData) && !mustSendLastChunk) {
-        context->writeComplete(context->clientConnection, NULL, 0, Comm::OK);
-        PROF_stop(clientSocketRecipient);
-        return;
-    }
-
-    if (!context->startOfOutput())
-        context->sendBody(rep, receivedData);
-    else {
-        assert(rep);
-        http->al->reply = rep;
-        HTTPMSGLOCK(http->al->reply);
-        context->sendStartOfMessage(rep, receivedData);
-    }
+    else
+        http->getConn()->handleReply(rep, receivedData);
 
     PROF_stop(clientSocketRecipient);
 }
@@ -1563,7 +1527,7 @@
     typedef CommCbMemFunT<ConnStateData, CommTimeoutCbParams> TimeoutDialer;
     AsyncCall::Pointer timeoutCall = JobCallback(33, 5,
                                      TimeoutDialer, this, ConnStateData::requestTimeout);
-    commSetConnTimeout(clientConnection, Config.Timeout.clientIdlePconn, timeoutCall);
+    commSetConnTimeout(clientConnection, idleTimeout(), timeoutCall);
 
     readSomeData();
     /** Please don't do anything with the FD past here! */
@@ -1942,10 +1906,6 @@
     }
 }
 
-SQUIDCEXTERN CSR clientGetMoreData;
-SQUIDCEXTERN CSS clientReplyStatus;
-SQUIDCEXTERN CSD clientReplyDetach;
-
 static ClientSocketContext *
 parseHttpRequestAbort(ConnStateData * csd, const char *uri)
 {
@@ -2133,7 +2093,7 @@
                  AnyP::UriScheme(conn->port->transport.protocol).c_str(), conn->port->defaultsite, vportStr, url);
         debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: '" << http->uri <<"'");
     } else if (vport > 0 /* && (!vhost || no Host:) */) {
-        debugs(33, 5, "ACCEL VPORT REWRITE: http_port IP + vport=" << vport);
+        debugs(33, 5, "ACCEL VPORT REWRITE: *_port IP + vport=" << vport);
         /* Put the local socket IP address as the hostname, with whatever vport we found  */
         int url_sz = strlen(url) + 32 + Config.appendDomainLen;
         http->uri = (char *)xcalloc(url_sz, 1);
@@ -2187,7 +2147,7 @@
  *  \return NULL on incomplete requests,
  *          a ClientSocketContext structure on success or failure.
  */
-static ClientSocketContext *
+ClientSocketContext *
 parseHttpRequest(ConnStateData *csd, HttpParser *hp, HttpRequestMethod * method_p, Http::ProtocolVersion *http_ver)
 {
     char *req_hdr = NULL;
@@ -2441,11 +2401,18 @@
 }
 
 void
+ConnStateData::consumeInput(const size_t byteCount)
+{
+    assert(byteCount > 0 && byteCount <= in.buf.length());
+    in.buf.consume(byteCount);
+    debugs(33, 5, "in.buf has " << in.buf.length() << " unused bytes");
+}
+
+// TODO: Remove when renaming ConnStateData
+void
 connNoteUseOfBuffer(ConnStateData* conn, size_t byteCount)
 {
-    assert(byteCount > 0 && byteCount <= conn->in.buf.length());
-    conn->in.buf.consume(byteCount);
-    debugs(33, 5, "conn->in.buf has " << conn->in.buf.length() << " bytes unused.");
+    conn->consumeInput(byteCount);
 }
 
 /// respond with ERR_TOO_BIG if request header exceeds request_header_max_size
@@ -2581,7 +2548,7 @@
 }
 #endif // USE_OPENSSL
 
-static void
+void
 clientProcessRequest(ConnStateData *conn, HttpParser *hp, ClientSocketContext *context, const HttpRequestMethod& method, Http::ProtocolVersion http_ver)
 {
     ClientHttpRequest *http = context->http;
@@ -2592,11 +2559,19 @@
     bool unsupportedTe = false;
     bool expectBody = false;
 
-    /* We have an initial client stream in place should it be needed */
-    /* setup our private context */
-    context->registerWithConn();
+    // temporary hack to avoid splitting this huge function with sensitive code
+    const bool isFtp = !hp;
+    if (isFtp) {
+        // In FTP, case, we already have the request parsed and checked, so we
+        // only need to go through the final body/conn setup to doCallouts().
+        assert(http->request);
+        request = http->request;
+        notedUseOfBuffer = true;
+        goto doFtpAndHttp;
+    }
 
     if (context->flags.parsed_ok == 0) {
+        assert(hp);
         clientStreamNode *node = context->getClientReplyContext();
         debugs(33, 2, "clientProcessRequest: Invalid Request");
         conn->quitAfterError(NULL);
@@ -2637,6 +2612,7 @@
 
     /* RFC 2616 section 10.5.6 : handle unsupported HTTP major versions cleanly. */
     /* We currently only support 0.9, 1.0, 1.1 properly */
+    /* TODO: move HTTP-specific processing into servers/HttpServer and such */
     if ( (http_ver.major == 0 && http_ver.minor != 9) ||
             (http_ver.major > 1) ) {
 
@@ -2671,6 +2647,11 @@
         goto finish;
     }
 
+doFtpAndHttp:
+    // Some blobs below are still HTTP-specific, but we would have to rewrite
+    // this entire function to remove them from the FTP code path. Connection
+    // setup and body_pipe preparation blobs are needed for FTP.
+
     request->clientConnectionManager = conn;
 
     request->flags.accelerated = http->flags.accel;
@@ -2790,10 +2771,12 @@
         }
     }
 
-    http->request = request.getRaw();
-    HTTPMSGLOCK(http->request);
+    if (!isFtp) {
+        http->request = request.getRaw();
+        HTTPMSGLOCK(http->request);
+    }
+
     clientSetKeepaliveFlag(http);
-
     // Let tunneling code be fully responsible for CONNECT requests
     if (http->request->method == Http::METHOD_CONNECT) {
         context->mayUseConnection(true);
@@ -2815,9 +2798,11 @@
         request->body_pipe = conn->expectRequestBody(
                                  chunked ? -1 : request->content_length);
 
-        // consume header early so that body pipe gets just the body
-        connNoteUseOfBuffer(conn, http->req_sz);
-        notedUseOfBuffer = true;
+        if (!isFtp) {
+            // consume header early so that body pipe gets just the body
+            connNoteUseOfBuffer(conn, http->req_sz);
+            notedUseOfBuffer = true;
+        }
 
         /* Is it too large? */
         if (!chunked && // if chunked, we will check as we accumulate
@@ -2834,15 +2819,17 @@
             goto finish;
         }
 
-        // We may stop producing, comm_close, and/or call setReplyToError()
-        // below, so quit on errors to avoid http->doCallouts()
-        if (!conn->handleRequestBodyData())
-            goto finish;
+        if (!isFtp) {
+            // We may stop producing, comm_close, and/or call setReplyToError()
+            // below, so quit on errors to avoid http->doCallouts()
+            if (!conn->handleRequestBodyData())
+                goto finish;
 
-        if (!request->body_pipe->productionEnded()) {
-            debugs(33, 5, HERE << "need more request body");
-            context->mayUseConnection(true);
-            assert(conn->flags.readMore);
+            if (!request->body_pipe->productionEnded()) {
+                debugs(33, 5, "need more request body");
+                context->mayUseConnection(true);
+                assert(conn->flags.readMore);
+            }
         }
     }
 
@@ -2877,6 +2864,12 @@
     }
 }
 
+int
+ConnStateData::pipelinePrefetchMax() const
+{
+    return Config.pipeline_max_prefetch;
+}
+
 /**
  * Limit the number of concurrent requests.
  * \return true  when there are available position(s) in the pipeline queue for another request.
@@ -2889,7 +2882,7 @@
 
     // default to the configured pipeline size.
     // add 1 because the head of pipeline is counted in concurrent requests and not prefetch queue
-    const int concurrentRequestLimit = Config.pipeline_max_prefetch + 1;
+    const int concurrentRequestLimit = pipelinePrefetchMax() + 1;
 
     // when queue filled already we cant add more.
     if (existingRequestCount >= concurrentRequestLimit) {
@@ -2909,7 +2902,6 @@
 bool
 ConnStateData::clientParseRequests()
 {
-    HttpRequestMethod method;
     bool parsed_req = false;
 
     debugs(33, 5, HERE << clientConnection << ": attempting to parse");
@@ -2927,14 +2919,8 @@
         if (concurrentRequestQueueFilled())
             break;
 
-        /* Begin the parsing */
-        PROF_start(parseHttpRequest);
-        HttpParserInit(&parser_, in.buf.c_str(), in.buf.length());
-
-        /* Process request */
         Http::ProtocolVersion http_ver;
-        ClientSocketContext *context = parseHttpRequest(this, &parser_, &method, &http_ver);
-        PROF_stop(parseHttpRequest);
+        ClientSocketContext *context = parseOneRequest(http_ver);
 
         /* partial or incomplete request */
         if (!context) {
@@ -2951,7 +2937,7 @@
                                              CommTimeoutCbPtrFun(clientLifetimeTimeout, context->http));
             commSetConnTimeout(clientConnection, Config.Timeout.lifetime, timeoutCall);
 
-            clientProcessRequest(this, &parser_, context, method, http_ver);
+            processParsedRequest(context, http_ver);
 
             parsed_req = true; // XXX: do we really need to parse everything right NOW ?
 
@@ -3208,26 +3194,13 @@
 }
 
 void
-ConnStateData::noteMoreBodySpaceAvailable(BodyPipe::Pointer )
-{
-    if (!handleRequestBodyData())
-        return;
-
-    // too late to read more body
-    if (!isOpen() || stoppedReceiving())
-        return;
-
-    readSomeData();
-}
-
-void
 ConnStateData::noteBodyConsumerAborted(BodyPipe::Pointer )
 {
     // request reader may get stuck waiting for space if nobody consumes body
     if (bodyPipe != NULL)
         bodyPipe->enableAutoConsumption();
 
-    stopReceiving("virgin request body consumer aborted"); // closes ASAP
+    // kids extend
 }
 
 /** general lifetime handler for HTTP requests */
@@ -3258,7 +3231,7 @@
 }
 
 ConnStateData::ConnStateData(const MasterXaction::Pointer &xact) :
-        AsyncJob("ConnStateData"),
+        AsyncJob("ConnStateData"), // kids overwrite
 #if USE_OPENSSL
         sslBumpMode(Ssl::bumpEnd),
         switchedToHttps_(false),
@@ -3280,6 +3253,15 @@
     log_addr = xact->tcpClient->remote;
     log_addr.applyMask(Config.Addrs.client_netmask);
 
+    flags.readMore = true; // kids may overwrite
+}
+
+void
+ConnStateData::start()
+{
+    BodyProducer::start();
+    HttpControlMsgSink::start();
+
     // ensure a buffer is present for this connection
     in.maybeMakeSpaceAvailable();
 
@@ -3309,54 +3291,17 @@
 #if USE_IDENT
     if (Ident::TheConfig.identLookup) {
         ACLFilledChecklist identChecklist(Ident::TheConfig.identLookup, NULL, NULL);
-        identChecklist.src_addr = xact->tcpClient->remote;
-        identChecklist.my_addr = xact->tcpClient->local;
+        identChecklist.src_addr = clientConnection->remote;
+        identChecklist.my_addr = clientConnection->local;
         if (identChecklist.fastCheck() == ACCESS_ALLOWED)
-            Ident::Start(xact->tcpClient, clientIdentDone, this);
+            Ident::Start(clientConnection, clientIdentDone, this);
     }
 #endif
 
     clientdbEstablished(clientConnection->remote, 1);
 
-    flags.readMore = true;
-}
-
-/** Handle a new connection on HTTP socket. */
-void
-httpAccept(const CommAcceptCbParams &params)
-{
-    MasterXaction::Pointer xact = params.xaction;
-    AnyP::PortCfgPointer s = xact->squidPort;
-
-    // NP: it is possible the port was reconfigured when the call or accept() was queued.
-
-    if (params.flag != Comm::OK) {
-        // Its possible the call was still queued when the client disconnected
-        debugs(33, 2, "httpAccept: " << s->listenConn << ": accept failure: " << xstrerr(params.xerrno));
-        return;
-    }
-
-    debugs(33, 4, HERE << params.conn << ": accepted");
-    fd_note(params.conn->fd, "client http connect");
-
-    if (s->tcp_keepalive.enabled) {
-        commSetTcpKeepalive(params.conn->fd, s->tcp_keepalive.idle, s->tcp_keepalive.interval, s->tcp_keepalive.timeout);
-    }
-
-    ++ incoming_sockets_accepted;
-
-    // Socket is ready, setup the connection manager to start using it
-    ConnStateData *connState = new ConnStateData(xact);
-
-    typedef CommCbMemFunT<ConnStateData, CommTimeoutCbParams> TimeoutDialer;
-    AsyncCall::Pointer timeoutCall =  JobCallback(33, 5,
-                                      TimeoutDialer, connState, ConnStateData::requestTimeout);
-    commSetConnTimeout(params.conn, Config.Timeout.request, timeoutCall);
-
-    connState->readSomeData();
-
 #if USE_DELAY_POOLS
-    fd_table[params.conn->fd].clientInfo = NULL;
+    fd_table[clientConnection->fd].clientInfo = NULL;
 
     if (Config.onoff.client_db) {
         /* it was said several times that client write limiter does not work if client_db is disabled */
@@ -3367,8 +3312,8 @@
         // TODO: we check early to limit error response bandwith but we
         // should recheck when we can honor delay_pool_uses_indirect
         // TODO: we should also pass the port details for myportname here.
-        ch.src_addr = params.conn->remote;
-        ch.my_addr = params.conn->local;
+        ch.src_addr = clientConnection->remote;
+        ch.my_addr = clientConnection->local;
 
         for (unsigned int pool = 0; pool < pools.size(); ++pool) {
 
@@ -3380,11 +3325,11 @@
 
                     /*  request client information from db after we did all checks
                         this will save hash lookup if client failed checks */
-                    ClientInfo * cli = clientdbGetInfo(params.conn->remote);
+                    ClientInfo * cli = clientdbGetInfo(clientConnection->remote);
                     assert(cli);
 
                     /* put client info in FDE */
-                    fd_table[params.conn->fd].clientInfo = cli;
+                    fd_table[clientConnection->fd].clientInfo = cli;
 
                     /* setup write limiter for this request */
                     const double burst = floor(0.5 +
@@ -3398,6 +3343,36 @@
         }
     }
 #endif
+
+    // kids must extend to actually start doing something (e.g., reading)
+}
+
+/** Handle a new connection on an HTTP socket. */
+void
+httpAccept(const CommAcceptCbParams &params)
+{
+    MasterXaction::Pointer xact = params.xaction;
+    AnyP::PortCfgPointer s = xact->squidPort;
+
+    // NP: it is possible the port was reconfigured when the call or accept() was queued.
+
+    if (params.flag != Comm::OK) {
+        // Its possible the call was still queued when the client disconnected
+        debugs(33, 2, s->listenConn << ": accept failure: " << xstrerr(params.xerrno));
+        return;
+    }
+
+    debugs(33, 4, params.conn << ": accepted");
+    fd_note(params.conn->fd, "client http connect");
+
+    if (s->tcp_keepalive.enabled)
+        commSetTcpKeepalive(params.conn->fd, s->tcp_keepalive.idle, s->tcp_keepalive.interval, s->tcp_keepalive.timeout);
+
+    ++incoming_sockets_accepted;
+
+    // Socket is ready, setup the connection manager to start using it
+    ConnStateData *connState = Http::NewServer(xact);
+    AsyncJob::Start(connState); // usually async-calls readSomeData()
 }
 
 #if USE_OPENSSL
@@ -3660,13 +3635,18 @@
     ++incoming_sockets_accepted;
 
     // Socket is ready, setup the connection manager to start using it
-    ConnStateData *connState = new ConnStateData(xact);
+    ConnStateData *connState = Https::NewServer(xact);
+    AsyncJob::Start(connState); // usually async-calls postHttpsAccept()
+}
 
-    if (s->flags.tunnelSslBumping) {
-        debugs(33, 5, "httpsAccept: accept transparent connection: " << params.conn);
+void
+ConnStateData::postHttpsAccept()
+{
+    if (port->flags.tunnelSslBumping) {
+        debugs(33, 5, "accept transparent connection: " << clientConnection);
 
         if (!Config.accessList.ssl_bump) {
-            httpsSslBumpAccessCheckDone(ACCESS_DENIED, connState);
+            httpsSslBumpAccessCheckDone(ACCESS_DENIED, this);
             return;
         }
 
@@ -3674,19 +3654,19 @@
         // using tproxy/intercept provided destination IP and port.
         HttpRequest *request = new HttpRequest();
         static char ip[MAX_IPSTRLEN];
-        assert(params.conn->flags & (COMM_TRANSPARENT | COMM_INTERCEPTION));
-        request->SetHost(params.conn->local.toStr(ip, sizeof(ip)));
-        request->port = params.conn->local.port();
-        request->myportname = s->name;
+        assert(clientConnection->flags & (COMM_TRANSPARENT | COMM_INTERCEPTION));
+        request->SetHost(clientConnection->local.toStr(ip, sizeof(ip)));
+        request->port = clientConnection->local.port();
+        request->myportname = port->name;
 
         ACLFilledChecklist *acl_checklist = new ACLFilledChecklist(Config.accessList.ssl_bump, request, NULL);
-        acl_checklist->src_addr = params.conn->remote;
-        acl_checklist->my_addr = s->s;
-        acl_checklist->nonBlockingCheck(httpsSslBumpAccessCheckDone, connState);
+        acl_checklist->src_addr = clientConnection->remote;
+        acl_checklist->my_addr = port->s;
+        acl_checklist->nonBlockingCheck(httpsSslBumpAccessCheckDone, this);
         return;
     } else {
-        SSL_CTX *sslContext = s->staticSslContext.get();
-        httpsEstablish(connState, sslContext, Ssl::bumpNone);
+        SSL_CTX *sslContext = port->staticSslContext.get();
+        httpsEstablish(this, sslContext, Ssl::bumpNone);
     }
 }
 
@@ -4127,6 +4107,31 @@
 }
 #endif
 
+void
+clientStartListeningOn(AnyP::PortCfgPointer &port, const RefCount< CommCbFunPtrCallT<CommAcceptCbPtrFun> > &subCall, const Ipc::FdNoteId fdNote)
+{
+    // Fill out a Comm::Connection which IPC will open as a listener for us
+    port->listenConn = new Comm::Connection;
+    port->listenConn->local = port->s;
+    port->listenConn->flags =
+        COMM_NONBLOCKING |
+        (port->flags.tproxyIntercept ? COMM_TRANSPARENT : 0) |
+        (port->flags.natIntercept ? COMM_INTERCEPTION : 0);
+
+    // route new connections to subCall
+    typedef CommCbFunPtrCallT<CommAcceptCbPtrFun> AcceptCall;
+    Subscription::Pointer sub = new CallSubscription<AcceptCall>(subCall);
+    AsyncCall::Pointer listenCall =
+        asyncCall(33, 2, "clientListenerConnectionOpened",
+                  ListeningStartedDialer(&clientListenerConnectionOpened,
+                                         port, fdNote, sub));
+    Ipc::StartListening(SOCK_STREAM, IPPROTO_TCP, port->listenConn, fdNote, listenCall);
+
+    assert(NHttpSockets < MAXTCPLISTENPORTS);
+    HttpSockets[NHttpSockets] = -1;
+    ++NHttpSockets;
+}
+
 /// process clientHttpConnectionsOpen result
 static void
 clientListenerConnectionOpened(AnyP::PortCfgPointer &s, const Ipc::FdNoteId portTypeNote, const Subscription::Pointer &sub)
@@ -4159,13 +4164,14 @@
 #if USE_OPENSSL
     clientHttpsConnectionsOpen();
 #endif
+    Ftp::StartListening();
 
     if (NHttpSockets < 1)
-        fatal("No HTTP or HTTPS ports configured");
+        fatal("No HTTP, HTTPS, or FTP ports configured");
 }
 
 void
-clientHttpConnectionsClose(void)
+clientConnectionsClose()
 {
     for (AnyP::PortCfgPointer s = HttpPortList; s != NULL; s = s->next) {
         if (s->listenConn != NULL) {
@@ -4185,6 +4191,8 @@
     }
 #endif
 
+    Ftp::StopListening();
+
     // TODO see if we can drop HttpSockets array entirely */
     for (int i = 0; i < NHttpSockets; ++i) {
         HttpSockets[i] = -1;
@@ -4271,8 +4279,6 @@
     return ch;
 }
 
-CBDATA_CLASS_INIT(ConnStateData);
-
 bool
 ConnStateData::transparent() const
 {
@@ -4422,31 +4428,37 @@
     assert(pinning.serverConnection == io.conn);
     pinning.closeHandler = NULL; // Comm unregisters handlers before calling
     const bool sawZeroReply = pinning.zeroReply; // reset when unpinning
-    unpinConnection();
+    unpinConnection(false);
+
     if (sawZeroReply && clientConnection != NULL) {
         debugs(33, 3, "Closing client connection on pinned zero reply.");
         clientConnection->close();
     }
-}
-
-void
-ConnStateData::pinConnection(const Comm::ConnectionPointer &pinServer, HttpRequest *request, CachePeer *aPeer, bool auth)
-{
-    char desc[FD_DESC_SZ];
-
-    if (Comm::IsConnOpen(pinning.serverConnection)) {
-        if (pinning.serverConnection->fd == pinServer->fd) {
-            startPinnedConnectionMonitoring();
-            return;
-        }
-    }
-
-    unpinConnection(); // closes pinned connection, if any, and resets fields
+
+}
+
+void
+ConnStateData::pinConnection(const Comm::ConnectionPointer &pinServer, HttpRequest *request, CachePeer *aPeer, bool auth, bool monitor)
+{
+    if (!Comm::IsConnOpen(pinning.serverConnection) ||
+            pinning.serverConnection->fd != pinServer->fd)
+        pinNewConnection(pinServer, request, aPeer, auth);
+
+    if (monitor)
+        startPinnedConnectionMonitoring();
+}
+
+void
+ConnStateData::pinNewConnection(const Comm::ConnectionPointer &pinServer, HttpRequest *request, CachePeer *aPeer, bool auth)
+{
+    unpinConnection(true); // closes pinned connection, if any, and resets fields
 
     pinning.serverConnection = pinServer;
 
     debugs(33, 3, HERE << pinning.serverConnection);
 
+    Must(pinning.serverConnection != NULL);
+
     // when pinning an SSL bumped connection, the request may be NULL
     const char *pinnedHost = "[unknown]";
     if (request) {
@@ -4461,6 +4473,7 @@
         pinning.peer = cbdataReference(aPeer);
     pinning.auth = auth;
     char stmp[MAX_IPSTRLEN];
+    char desc[FD_DESC_SZ];
     snprintf(desc, FD_DESC_SZ, "%s pinned connection for %s (%d)",
              (auth || !aPeer) ? pinnedHost : aPeer->name,
              clientConnection->remote.toUrl(stmp,MAX_IPSTRLEN),
@@ -4475,11 +4488,10 @@
     Params &params = GetCommParams<Params>(pinning.closeHandler);
     params.conn = pinning.serverConnection;
     comm_add_close_handler(pinning.serverConnection->fd, pinning.closeHandler);
-
-    startPinnedConnectionMonitoring();
 }
 
-/// Assign a read handler to an idle pinned connection so that we can detect connection closures.
+/// [re]start monitoring pinned connection for peer closures so that we can
+/// propagate them to an _idle_ client pinned to that peer
 void
 ConnStateData::startPinnedConnectionMonitoring()
 {
@@ -4546,14 +4558,24 @@
 
     if (!valid) {
         /* The pinning info is not safe, remove any pinning info */
-        unpinConnection();
+        unpinConnection(true);
     }
 
     return pinning.serverConnection;
 }
 
+Comm::ConnectionPointer
+ConnStateData::borrowPinnedConnection(HttpRequest *request, const CachePeer *aPeer)
+{
+    debugs(33, 7, pinning.serverConnection);
+    if (validatePinnedConnection(request, aPeer) != NULL)
+        stopPinnedConnectionMonitoring();
+
+    return pinning.serverConnection; // closed if validation failed
+}
+
 void
-ConnStateData::unpinConnection()
+ConnStateData::unpinConnection(const bool andClose)
 {
     debugs(33, 3, HERE << pinning.serverConnection);
 
@@ -4565,9 +4587,13 @@
             comm_remove_close_handler(pinning.serverConnection->fd, pinning.closeHandler);
             pinning.closeHandler = NULL;
         }
-        /// also close the server side socket, we should not use it for any future requests...
-        // TODO: do not close if called from our close handler?
-        pinning.serverConnection->close();
+
+        stopPinnedConnectionMonitoring();
+
+        // close the server side socket if requested
+        if (andClose)
+            pinning.serverConnection->close();
+        pinning.serverConnection = NULL;
     }
 
     safe_free(pinning.host);

=== modified file 'src/client_side.h'
--- src/client_side.h	2014-07-14 09:48:47 +0000
+++ src/client_side.h	2014-08-10 02:28:33 +0000
@@ -33,9 +33,11 @@
 #ifndef SQUID_CLIENTSIDE_H
 #define SQUID_CLIENTSIDE_H
 
+#include "clientStreamForward.h"
 #include "comm.h"
 #include "HttpControlMsg.h"
 #include "HttpParser.h"
+#include "ipc/FdNotes.h"
 #include "SBuf.h"
 #if USE_AUTH
 #include "auth/UserRequest.h"
@@ -137,6 +139,7 @@
     void buildRangeHeader(HttpReply * rep);
     clientStreamNode * getTail() const;
     clientStreamNode * getClientReplyContext() const;
+    ConnStateData *getConn() const;
     void connIsFinished();
     void removeFromConnectionList(ConnStateData * conn);
     void deferRecipientForLater(clientStreamNode * node, HttpReply * rep, StoreIOBuffer receivedData);
@@ -193,7 +196,7 @@
 
 public:
     explicit ConnStateData(const MasterXaction::Pointer &xact);
-    ~ConnStateData();
+    virtual ~ConnStateData();
 
     void readSomeData();
     bool areAllContextsForThisConnection() const;
@@ -268,6 +271,7 @@
         int port;               /* port of pinned connection */
         bool pinned;             /* this connection was pinned */
         bool auth;               /* pinned for www authentication */
+        bool reading;   ///< we are monitoring for peer connection closure
         bool zeroReply; ///< server closed w/o response (ERR_ZERO_SIZE_OBJECT)
         CachePeer *peer;             /* CachePeer the connection goes via */
         AsyncCall::Pointer readHandler; ///< detects serverConnection closure
@@ -292,21 +296,21 @@
 
     void expectNoForwarding(); ///< cleans up virgin request [body] forwarding state
 
+    /* BodyPipe API */
     BodyPipe::Pointer expectRequestBody(int64_t size);
-    virtual void noteMoreBodySpaceAvailable(BodyPipe::Pointer);
-    virtual void noteBodyConsumerAborted(BodyPipe::Pointer);
+    virtual void noteMoreBodySpaceAvailable(BodyPipe::Pointer) = 0;
+    virtual void noteBodyConsumerAborted(BodyPipe::Pointer) = 0;
 
     bool handleReadData();
     bool handleRequestBodyData();
 
-    /**
-     * Correlate the current ConnStateData object with the pinning_fd socket descriptor.
-     */
-    void pinConnection(const Comm::ConnectionPointer &pinServerConn, HttpRequest *request, CachePeer *peer, bool auth);
-    /**
-     * Decorrelate the ConnStateData object from its pinned CachePeer
-     */
-    void unpinConnection();
+    /// Forward future client requests using the given server connection.
+    /// Optionally, monitor pinned server connection for server-side closures.
+    void pinConnection(const Comm::ConnectionPointer &pinServerConn, HttpRequest *request, CachePeer *peer, bool auth, bool monitor = true);
+    /// Undo pinConnection() and, optionally, close the pinned connection.
+    void unpinConnection(const bool andClose);
+    /// Returns validated pinnned server connection (and stops its monitoring).
+    Comm::ConnectionPointer borrowPinnedConnection(HttpRequest *request, const CachePeer *aPeer);
     /**
      * Checks if there is pinning info if it is valid. It can close the server side connection
      * if pinned info is not valid.
@@ -321,15 +325,20 @@
     CachePeer *pinnedPeer() const {return pinning.peer;}
     bool pinnedAuth() const {return pinning.auth;}
 
+    /// called just before a FwdState-dispatched job starts using connection
+    virtual void notePeerConnection(Comm::ConnectionPointer) {}
+
     // pining related comm callbacks
-    void clientPinnedConnectionClosed(const CommCloseCbParams &io);
+    virtual void clientPinnedConnectionClosed(const CommCloseCbParams &io);
 
     // comm callbacks
     void clientReadRequest(const CommIoCbParams &io);
+    void clientReadFtpData(const CommIoCbParams &io);
     void connStateClosed(const CommCloseCbParams &io);
     void requestTimeout(const CommTimeoutCbParams &params);
 
     // AsyncJob API
+    virtual void start();
     virtual bool doneAll() const { return BodyProducer::doneAll() && false;}
     virtual void swanSong();
 
@@ -341,6 +350,9 @@
     void stopPinnedConnectionMonitoring();
 
 #if USE_OPENSSL
+    /// the second part of old httpsAccept, waiting for future HttpsServer home
+    void postHttpsAccept();
+
     /// called by FwdState when it is done bumping the server
     void httpsPeeked(Comm::ConnectionPointer serverConnection);
 
@@ -385,6 +397,16 @@
     const SBuf &connectionTag() const { return connectionTag_; }
     void connectionTag(const char *aTag) { connectionTag_ = aTag; }
 
+    /// handle a control message received by context from a peer and call back
+    virtual void writeControlMsgAndCall(ClientSocketContext *context, HttpReply *rep, AsyncCall::Pointer &call) = 0;
+
+    /// ClientStream calls this to supply response header (once) and data
+    /// for the current ClientSocketContext.
+    virtual void handleReply(HttpReply *header, StoreIOBuffer receivedData) = 0;
+
+    /// remove no longer needed leading bytes from the input buffer
+    void consumeInput(const size_t byteCount);
+
 protected:
     void startDechunkingRequest();
     void finishDechunkingRequest(bool withSuccess);
@@ -394,20 +416,32 @@
     void startPinnedConnectionMonitoring();
     void clientPinnedConnectionRead(const CommIoCbParams &io);
 
+    /// parse input buffer prefix into a single transfer protocol request
+    virtual ClientSocketContext *parseOneRequest(Http::ProtocolVersion &ver) = 0;
+
+    /// start processing a freshly parsed request
+    virtual void processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver) = 0;
+
+    /// returning N allows a pipeline of 1+N requests (see pipeline_prefetch)
+    virtual int pipelinePrefetchMax() const;
+
+    /// timeout to use when waiting for the next request
+    virtual time_t idleTimeout() const = 0;
+
+    BodyPipe::Pointer bodyPipe; ///< set when we are reading request body
+
 private:
     int connFinishedWithConn(int size);
     void clientAfterReadingRequests();
     bool concurrentRequestQueueFilled() const;
 
+    void pinNewConnection(const Comm::ConnectionPointer &pinServer, HttpRequest *request, CachePeer *aPeer, bool auth);
+
 #if USE_AUTH
     /// some user details that can be used to perform authentication on this connection
     Auth::UserRequest::Pointer auth_;
 #endif
 
-    HttpParser parser_;
-
-    // XXX: CBDATA plays with public/private and leaves the following 'private' fields all public... :(
-
 #if USE_OPENSSL
     bool switchedToHttps_;
     /// The SSL server host name appears in CONNECT request or the server ip address for the intercepted requests
@@ -426,11 +460,8 @@
     const char *stoppedReceiving_;
 
     AsyncCall::Pointer reader; ///< set when we are reading
-    BodyPipe::Pointer bodyPipe; // set when we are reading request body
 
     SBuf connectionTag_; ///< clt_conn_tag=Tag annotation for client connection
-
-    CBDATA_CLASS2(ConnStateData);
 };
 
 void setLogUri(ClientHttpRequest * http, char const *uri, bool cleanUrl = false);
@@ -439,8 +470,26 @@
 
 int varyEvaluateMatch(StoreEntry * entry, HttpRequest * req);
 
+/// accept requests to a given port and inform subCall about them
+void clientStartListeningOn(AnyP::PortCfgPointer &port, const RefCount< CommCbFunPtrCallT<CommAcceptCbPtrFun> > &subCall, const Ipc::FdNoteId noteId);
+
 void clientOpenListenSockets(void);
-void clientHttpConnectionsClose(void);
+void clientConnectionsClose(void);
 void httpRequestFree(void *);
 
+/// decide whether to expect multiple requests on the corresponding connection
+void clientSetKeepaliveFlag(ClientHttpRequest *http);
+
+/* misplaced declaratrions of Stream callbacks provided/used by client side */
+SQUIDCEXTERN CSR clientGetMoreData;
+SQUIDCEXTERN CSS clientReplyStatus;
+SQUIDCEXTERN CSD clientReplyDetach;
+CSCB clientSocketRecipient;
+CSD clientSocketDetach;
+
+/* TODO: Move to HttpServer. Warning: Move requires large code nonchanges! */
+ClientSocketContext *parseHttpRequest(ConnStateData *, HttpParser *, HttpRequestMethod *, Http::ProtocolVersion *);
+void clientProcessRequest(ConnStateData *conn, HttpParser *hp, ClientSocketContext *context, const HttpRequestMethod& method, Http::ProtocolVersion http_ver);
+void clientPostHttpsAccept(ConnStateData *connState);
+
 #endif /* SQUID_CLIENTSIDE_H */

=== modified file 'src/client_side_reply.h'
--- src/client_side_reply.h	2014-03-15 02:50:12 +0000
+++ src/client_side_reply.h	2014-08-04 21:44:31 +0000
@@ -31,6 +31,7 @@
 #ifndef SQUID_CLIENTSIDEREPLY_H
 #define SQUID_CLIENTSIDEREPLY_H
 
+#include "acl/forward.h"
 #include "client_side_request.h"
 #include "ip/forward.h"
 #include "RequestFlags.h"

=== added directory 'src/clients'
=== added file 'src/clients/FtpClient.cc'
--- src/clients/FtpClient.cc	1970-01-01 00:00:00 +0000
+++ src/clients/FtpClient.cc	2014-08-10 23:18:33 +0000
@@ -0,0 +1,1126 @@
+/*
+ * DEBUG: section 09    File Transfer Protocol (FTP)
+ *
+ */
+
+#include "squid.h"
+#include "acl/FilledChecklist.h"
+#include "client_side.h"
+#include "clients/FtpClient.h"
+#include "comm/ConnOpener.h"
+#include "comm/Read.h"
+#include "comm/TcpAcceptor.h"
+#include "comm/Write.h"
+#include "errorpage.h"
+#include "fd.h"
+#include "ftp/Parsing.h"
+#include "ip/tools.h"
+#include "Mem.h"
+#include "SquidConfig.h"
+#include "SquidString.h"
+#include "StatCounters.h"
+#include "tools.h"
+#include "wordlist.h"
+
+#include <set>
+
+namespace Ftp
+{
+
+const char *const crlf = "\r\n";
+
+static char *
+escapeIAC(const char *buf)
+{
+    int n;
+    char *ret;
+    unsigned const char *p;
+    unsigned char *r;
+
+    for (p = (unsigned const char *)buf, n = 1; *p; ++n, ++p)
+        if (*p == 255)
+            ++n;
+
+    ret = (char *)xmalloc(n);
+
+    for (p = (unsigned const char *)buf, r=(unsigned char *)ret; *p; ++p) {
+        *r = *p;
+        ++r;
+
+        if (*p == 255) {
+            *r = 255;
+            ++r;
+        }
+    }
+
+    *r = '\0';
+    ++r;
+    assert((r - (unsigned char *)ret) == n );
+    return ret;
+}
+
+/* Ftp::Channel */
+
+/// configures the channel with a descriptor and registers a close handler
+void
+Ftp::Channel::opened(const Comm::ConnectionPointer &newConn,
+                     const AsyncCall::Pointer &aCloser)
+{
+    assert(!Comm::IsConnOpen(conn));
+    assert(closer == NULL);
+
+    assert(Comm::IsConnOpen(newConn));
+    assert(aCloser != NULL);
+
+    conn = newConn;
+    closer = aCloser;
+    comm_add_close_handler(conn->fd, closer);
+}
+
+/// planned close: removes the close handler and calls comm_close
+void
+Ftp::Channel::close()
+{
+    // channels with active listeners will be closed when the listener handler dies.
+    if (Comm::IsConnOpen(conn)) {
+        comm_remove_close_handler(conn->fd, closer);
+        conn->close(); // we do not expect to be called back
+    }
+    clear();
+}
+
+void
+Ftp::Channel::forget()
+{
+    if (Comm::IsConnOpen(conn)) {
+        commUnsetConnTimeout(conn);
+        comm_remove_close_handler(conn->fd, closer);
+    }
+    clear();
+}
+
+void
+Ftp::Channel::clear()
+{
+    conn = NULL;
+    closer = NULL;
+}
+
+/* Ftp::CtrlChannel */
+
+Ftp::CtrlChannel::CtrlChannel():
+        buf(NULL),
+        size(0),
+        offset(0),
+        message(NULL),
+        last_command(NULL),
+        last_reply(NULL),
+        replycode(0)
+{
+    buf = static_cast<char*>(memAllocBuf(4096, &size));
+}
+
+Ftp::CtrlChannel::~CtrlChannel()
+{
+    memFreeBuf(size, buf);
+    if (message)
+        wordlistDestroy(&message);
+    safe_free(last_command);
+    safe_free(last_reply);
+}
+
+/* Ftp::DataChannel */
+
+Ftp::DataChannel::DataChannel():
+        readBuf(NULL),
+        host(NULL),
+        port(0),
+        read_pending(false)
+{
+}
+
+Ftp::DataChannel::~DataChannel()
+{
+    delete readBuf;
+}
+
+void
+Ftp::DataChannel::addr(const Ip::Address &import)
+{
+    static char addrBuf[MAX_IPSTRLEN];
+    import.toStr(addrBuf, sizeof(addrBuf));
+    xfree(host);
+    host = xstrdup(addrBuf);
+    port = import.port();
+}
+
+/* Ftp::Client */
+
+Ftp::Client::Client(FwdState *fwdState):
+        AsyncJob("Ftp::Client"),
+        ::ServerStateData(fwdState),
+        ctrl(),
+        data(),
+        state(BEGIN),
+        old_request(NULL),
+        old_reply(NULL),
+        shortenReadTimeout(false)
+{
+    ++statCounter.server.all.requests;
+    ++statCounter.server.ftp.requests;
+
+    ctrl.last_command = xstrdup("Connect to server");
+
+    typedef CommCbMemFunT<Client, CommCloseCbParams> Dialer;
+    const AsyncCall::Pointer closer = JobCallback(9, 5, Dialer, this,
+                                      Ftp::Client::ctrlClosed);
+    ctrl.opened(fwdState->serverConnection(), closer);
+}
+
+Ftp::Client::~Client()
+{
+    if (data.opener != NULL) {
+        data.opener->cancel("Ftp::Client destructed");
+        data.opener = NULL;
+    }
+    data.close();
+
+    safe_free(old_request);
+    safe_free(old_reply);
+    fwd = NULL; // refcounted
+}
+
+void
+Ftp::Client::start()
+{
+    scheduleReadControlReply(0);
+}
+
+void
+Ftp::Client::initReadBuf()
+{
+    if (data.readBuf == NULL) {
+        data.readBuf = new MemBuf;
+        data.readBuf->init(4096, SQUID_TCP_SO_RCVBUF);
+    }
+}
+
+/**
+ * Close the FTP server connection(s). Used by serverComplete().
+ */
+void
+Ftp::Client::closeServer()
+{
+    if (Comm::IsConnOpen(ctrl.conn)) {
+        debugs(9, 3, "closing FTP server FD " << ctrl.conn->fd << ", this " << this);
+        fwd->unregister(ctrl.conn);
+        ctrl.close();
+    }
+
+    if (Comm::IsConnOpen(data.conn)) {
+        debugs(9, 3, "closing FTP data FD " << data.conn->fd << ", this " << this);
+        data.close();
+    }
+
+    debugs(9, 3, "FTP ctrl and data connections closed. this " << this);
+}
+
+/**
+ * Did we close all FTP server connection(s)?
+ *
+ \retval true  Both server control and data channels are closed. And not waiting for a new data connection to open.
+ \retval false Either control channel or data is still active.
+ */
+bool
+Ftp::Client::doneWithServer() const
+{
+    return !Comm::IsConnOpen(ctrl.conn) && !Comm::IsConnOpen(data.conn);
+}
+
+void
+Ftp::Client::failed(err_type error, int xerrno)
+{
+    debugs(9, 3, "entry-null=" << (entry?entry->isEmpty():0) << ", entry=" << entry);
+
+    const char *command, *reply;
+    const Http::StatusCode httpStatus = failedHttpStatus(error);
+    ErrorState *const ftperr = new ErrorState(error, httpStatus, fwd->request);
+    ftperr->xerrno = xerrno;
+
+    ftperr->ftp.server_msg = ctrl.message;
+    ctrl.message = NULL;
+
+    if (old_request)
+        command = old_request;
+    else
+        command = ctrl.last_command;
+
+    if (command && strncmp(command, "PASS", 4) == 0)
+        command = "PASS <yourpassword>";
+
+    if (old_reply)
+        reply = old_reply;
+    else
+        reply = ctrl.last_reply;
+
+    if (command)
+        ftperr->ftp.request = xstrdup(command);
+
+    if (reply)
+        ftperr->ftp.reply = xstrdup(reply);
+
+    fwd->request->detailError(error, xerrno);
+    fwd->fail(ftperr);
+
+    closeServer(); // we failed, so no serverComplete()
+}
+
+Http::StatusCode
+Ftp::Client::failedHttpStatus(err_type &error)
+{
+    if (error == ERR_NONE)
+        error = ERR_FTP_FAILURE;
+    return error == ERR_READ_TIMEOUT ? Http::scGatewayTimeout :
+           Http::scBadGateway;
+}
+
+/**
+ * DPW 2007-04-23
+ * Looks like there are no longer anymore callers that set
+ * buffered_ok=1.  Perhaps it can be removed at some point.
+ */
+void
+Ftp::Client::scheduleReadControlReply(int buffered_ok)
+{
+    debugs(9, 3, ctrl.conn);
+
+    if (buffered_ok && ctrl.offset > 0) {
+        /* We've already read some reply data */
+        handleControlReply();
+    } else {
+        /*
+         * Cancel the timeout on the Data socket (if any) and
+         * establish one on the control socket.
+         */
+        if (Comm::IsConnOpen(data.conn)) {
+            commUnsetConnTimeout(data.conn);
+        }
+
+        const time_t tout = shortenReadTimeout ?
+                            min(Config.Timeout.connect, Config.Timeout.read):
+                            Config.Timeout.read;
+        shortenReadTimeout = false; // we only need to do this once, after PASV
+
+        typedef CommCbMemFunT<Client, CommTimeoutCbParams> TimeoutDialer;
+        AsyncCall::Pointer timeoutCall = JobCallback(9, 5, TimeoutDialer, this, Ftp::Client::timeout);
+        commSetConnTimeout(ctrl.conn, tout, timeoutCall);
+
+        typedef CommCbMemFunT<Client, CommIoCbParams> Dialer;
+        AsyncCall::Pointer reader = JobCallback(9, 5, Dialer, this, Ftp::Client::readControlReply);
+        comm_read(ctrl.conn, ctrl.buf + ctrl.offset, ctrl.size - ctrl.offset, reader);
+    }
+}
+
+void
+Ftp::Client::readControlReply(const CommIoCbParams &io)
+{
+    debugs(9, 3, "FD " << io.fd << ", Read " << io.size << " bytes");
+
+    if (io.size > 0) {
+        kb_incr(&(statCounter.server.all.kbytes_in), io.size);
+        kb_incr(&(statCounter.server.ftp.kbytes_in), io.size);
+    }
+
+    if (io.flag == Comm::ERR_CLOSING)
+        return;
+
+    if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) {
+        abortTransaction("entry aborted during control reply read");
+        return;
+    }
+
+    assert(ctrl.offset < ctrl.size);
+
+    if (io.flag == Comm::OK && io.size > 0) {
+        fd_bytes(io.fd, io.size, FD_READ);
+    }
+
+    if (io.flag != Comm::OK) {
+        debugs(50, ignoreErrno(io.xerrno) ? 3 : DBG_IMPORTANT,
+               "FTP control reply read error: " << xstrerr(io.xerrno));
+
+        if (ignoreErrno(io.xerrno)) {
+            scheduleReadControlReply(0);
+        } else {
+            failed(ERR_READ_ERROR, io.xerrno);
+            /* failed closes ctrl.conn and frees ftpState */
+        }
+        return;
+    }
+
+    if (io.size == 0) {
+        if (entry->store_status == STORE_PENDING) {
+            failed(ERR_FTP_FAILURE, 0);
+            /* failed closes ctrl.conn and frees ftpState */
+            return;
+        }
+
+        /* XXX this may end up having to be serverComplete() .. */
+        abortTransaction("zero control reply read");
+        return;
+    }
+
+    unsigned int len =io.size + ctrl.offset;
+    ctrl.offset = len;
+    assert(len <= ctrl.size);
+    if (Comm::IsConnOpen(ctrl.conn))
+        commUnsetConnTimeout(ctrl.conn); // we are done waiting for ctrl reply
+    handleControlReply();
+}
+
+void
+Ftp::Client::handleControlReply()
+{
+    debugs(9, 3, status());
+
+    size_t bytes_used = 0;
+    wordlistDestroy(&ctrl.message);
+
+    if (!parseControlReply(bytes_used)) {
+        /* didn't get complete reply yet */
+
+        if (ctrl.offset == ctrl.size) {
+            ctrl.buf = static_cast<char*>(memReallocBuf(ctrl.buf, ctrl.size << 1, &ctrl.size));
+        }
+
+        scheduleReadControlReply(0);
+        return;
+    }
+
+    assert(ctrl.message); // the entire FTP server response, line by line
+    assert(ctrl.replycode >= 0); // FTP status code (from the last line)
+    assert(ctrl.last_reply); // FTP reason (from the last line)
+
+    if (ctrl.offset == bytes_used) {
+        /* used it all up */
+        ctrl.offset = 0;
+    } else {
+        /* Got some data past the complete reply */
+        assert(bytes_used < ctrl.offset);
+        ctrl.offset -= bytes_used;
+        memmove(ctrl.buf, ctrl.buf + bytes_used, ctrl.offset);
+    }
+
+    debugs(9, 3, "state=" << state << ", code=" << ctrl.replycode);
+}
+
+bool
+Ftp::Client::handlePasvReply(Ip::Address &srvAddr)
+{
+    int code = ctrl.replycode;
+    char *buf;
+    debugs(9, 3, status());
+
+    if (code != 227) {
+        debugs(9, 2, "PASV not supported by remote end");
+        return false;
+    }
+
+    /*  227 Entering Passive Mode (h1,h2,h3,h4,p1,p2).  */
+    /*  ANSI sez [^0-9] is undefined, it breaks on Watcom cc */
+    debugs(9, 5, "scanning: " << ctrl.last_reply);
+
+    buf = ctrl.last_reply + strcspn(ctrl.last_reply, "0123456789");
+
+    const char *forceIp = Config.Ftp.sanitycheck ?
+                          fd_table[ctrl.conn->fd].ipaddr : NULL;
+    if (!Ftp::ParseIpPort(buf, forceIp, srvAddr)) {
+        debugs(9, DBG_IMPORTANT, "Unsafe PASV reply from " <<
+               ctrl.conn->remote << ": " << ctrl.last_reply);
+        return false;
+    }
+
+    data.addr(srvAddr);
+
+    return true;
+}
+
+bool
+Ftp::Client::handleEpsvReply(Ip::Address &remoteAddr)
+{
+    int code = ctrl.replycode;
+    char *buf;
+    debugs(9, 3, status());
+
+    if (code != 229 && code != 522) {
+        if (code == 200) {
+            /* handle broken servers (RFC 2428 says OK code for EPSV MUST be 229 not 200) */
+            /* vsftpd for one send '200 EPSV ALL ok.' without even port info.
+             * Its okay to re-send EPSV 1/2 but nothing else. */
+            debugs(9, DBG_IMPORTANT, "Broken FTP Server at " << ctrl.conn->remote << ". Wrong accept code for EPSV");
+        } else {
+            debugs(9, 2, "EPSV not supported by remote end");
+        }
+        return sendPassive();
+    }
+
+    if (code == 522) {
+        /* Peer responded with a list of supported methods:
+         *   522 Network protocol not supported, use (1)
+         *   522 Network protocol not supported, use (1,2)
+         *   522 Network protocol not supported, use (2)
+         * TODO: Handle the (1,2) case which may happen after EPSV ALL. Close
+         * data + control without self-destructing and re-open from scratch.
+         */
+        debugs(9, 5, "scanning: " << ctrl.last_reply);
+        buf = ctrl.last_reply;
+        while (buf != NULL && *buf != '\0' && *buf != '\n' && *buf != '(')
+            ++buf;
+        if (buf != NULL && *buf == '\n')
+            ++buf;
+
+        if (buf == NULL || *buf == '\0') {
+            /* handle broken server (RFC 2428 says MUST specify supported protocols in 522) */
+            debugs(9, DBG_IMPORTANT, "Broken FTP Server at " << ctrl.conn->remote << ". 522 error missing protocol negotiation hints");
+            return sendPassive();
+        } else if (strcmp(buf, "(1)") == 0) {
+            state = SENT_EPSV_2; /* simulate having sent and failed EPSV 2 */
+            return sendPassive();
+        } else if (strcmp(buf, "(2)") == 0) {
+            if (Ip::EnableIpv6) {
+                /* If server only supports EPSV 2 and we have already tried that. Go straight to EPRT */
+                if (state == SENT_EPSV_2) {
+                    return sendEprt();
+                } else {
+                    /* or try the next Passive mode down the chain. */
+                    return sendPassive();
+                }
+            } else {
+                /* Server only accept EPSV in IPv6 traffic. */
+                state = SENT_EPSV_1; /* simulate having sent and failed EPSV 1 */
+                return sendPassive();
+            }
+        } else {
+            /* handle broken server (RFC 2428 says MUST specify supported protocols in 522) */
+            debugs(9, DBG_IMPORTANT, "WARNING: Server at " << ctrl.conn->remote << " sent unknown protocol negotiation hint: " << buf);
+            return sendPassive();
+        }
+        failed(ERR_FTP_FAILURE, 0);
+        return false;
+    }
+
+    /*  229 Entering Extended Passive Mode (|||port|) */
+    /*  ANSI sez [^0-9] is undefined, it breaks on Watcom cc */
+    debugs(9, 5, "scanning: " << ctrl.last_reply);
+
+    buf = ctrl.last_reply + strcspn(ctrl.last_reply, "(");
+
+    char h1, h2, h3, h4;
+    unsigned short port;
+    int n = sscanf(buf, "(%c%c%c%hu%c)", &h1, &h2, &h3, &port, &h4);
+
+    if (n < 4 || h1 != h2 || h1 != h3 || h1 != h4) {
+        debugs(9, DBG_IMPORTANT, "Invalid EPSV reply from " <<
+               ctrl.conn->remote << ": " <<
+               ctrl.last_reply);
+
+        return sendPassive();
+    }
+
+    if (0 == port) {
+        debugs(9, DBG_IMPORTANT, "Unsafe EPSV reply from " <<
+               ctrl.conn->remote << ": " <<
+               ctrl.last_reply);
+
+        return sendPassive();
+    }
+
+    if (Config.Ftp.sanitycheck) {
+        if (port < 1024) {
+            debugs(9, DBG_IMPORTANT, "Unsafe EPSV reply from " <<
+                   ctrl.conn->remote << ": " <<
+                   ctrl.last_reply);
+
+            return sendPassive();
+        }
+    }
+
+    remoteAddr = ctrl.conn->remote;
+    remoteAddr.port(port);
+    data.addr(remoteAddr);
+    return true;
+}
+
+// FTP clients do not support EPRT and PORT commands yet.
+// The Ftp::Client::sendEprt() will fail because of the unimplemented
+// openListenSocket() or sendPort() methods
+bool
+Ftp::Client::sendEprt()
+{
+    if (!Config.Ftp.eprt) {
+        /* Disabled. Switch immediately to attempting old PORT command. */
+        debugs(9, 3, "EPRT disabled by local administrator");
+        return sendPort();
+    }
+
+    debugs(9, 3, status());
+
+    if (!openListenSocket()) {
+        failed(ERR_FTP_FAILURE, 0);
+        return false;
+    }
+
+    debugs(9, 3, "Listening for FTP data connection with FD " << data.conn);
+    if (!Comm::IsConnOpen(data.conn)) {
+        // TODO: Set error message.
+        failed(ERR_FTP_FAILURE, 0);
+        return false;
+    }
+
+    static MemBuf mb;
+    mb.reset();
+    char buf[MAX_IPSTRLEN];
+    /* RFC 2428 defines EPRT as IPv6 equivalent to IPv4 PORT command. */
+    /* Which can be used by EITHER protocol. */
+    debugs(9, 3, "Listening for FTP data connection on port" << comm_local_port(data.conn->fd) << " or port?" << data.conn->local.port());
+    mb.Printf("EPRT |%d|%s|%d|%s",
+              ( data.conn->local.isIPv6() ? 2 : 1 ),
+              data.conn->local.toStr(buf,MAX_IPSTRLEN),
+              comm_local_port(data.conn->fd), Ftp::crlf );
+
+    state = SENT_EPRT;
+    writeCommand(mb.content());
+    return true;
+}
+
+bool
+Ftp::Client::sendPort()
+{
+    failed(ERR_FTP_FAILURE, 0);
+    return false;
+}
+
+bool
+Ftp::Client::sendPassive()
+{
+    debugs(9, 3, status());
+
+    /** \par
+      * Checks for EPSV ALL special conditions:
+      * If enabled to be sent, squid MUST NOT request any other connect methods.
+      * If 'ALL' is sent and fails the entire FTP Session fails.
+      * NP: By my reading exact EPSV protocols maybe attempted, but only EPSV method. */
+    if (Config.Ftp.epsv_all && state == SENT_EPSV_1 ) {
+        // We are here because the last "EPSV 1" failed, but because of epsv_all
+        // no other method allowed.
+        debugs(9, DBG_IMPORTANT, "FTP does not allow PASV method after 'EPSV ALL' has been sent.");
+        failed(ERR_FTP_FAILURE, 0);
+        return false;
+    }
+
+    /// Closes any old FTP-Data connection which may exist. */
+    data.close();
+
+    /** \par
+      * Checks for previous EPSV/PASV failures on this server/session.
+      * Diverts to EPRT immediately if they are not working. */
+    if (!Config.Ftp.passive || state == SENT_PASV) {
+        sendEprt();
+        return true;
+    }
+
+    static MemBuf mb;
+    mb.reset();
+    /** \par
+      * Send EPSV (ALL,2,1) or PASV on the control channel.
+      *
+      *  - EPSV ALL  is used if enabled.
+      *  - EPSV 2    is used if ALL is disabled and IPv6 is available and ctrl channel is IPv6.
+      *  - EPSV 1    is used if EPSV 2 (IPv6) fails or is not available or ctrl channel is IPv4.
+      *  - PASV      is used if EPSV 1 fails.
+      */
+    switch (state) {
+    case SENT_EPSV_ALL: /* EPSV ALL resulted in a bad response. Try ther EPSV methods. */
+        if (ctrl.conn->local.isIPv6()) {
+            debugs(9, 5, "FTP Channel is IPv6 (" << ctrl.conn->remote << ") attempting EPSV 2 after EPSV ALL has failed.");
+            mb.Printf("EPSV 2%s", Ftp::crlf);
+            state = SENT_EPSV_2;
+            break;
+        }
+        // else fall through to skip EPSV 2
+
+    case SENT_EPSV_2: /* EPSV IPv6 failed. Try EPSV IPv4 */
+        if (ctrl.conn->local.isIPv4()) {
+            debugs(9, 5, "FTP Channel is IPv4 (" << ctrl.conn->remote << ") attempting EPSV 1 after EPSV ALL has failed.");
+            mb.Printf("EPSV 1%s", Ftp::crlf);
+            state = SENT_EPSV_1;
+            break;
+        } else if (Config.Ftp.epsv_all) {
+            debugs(9, DBG_IMPORTANT, "FTP does not allow PASV method after 'EPSV ALL' has been sent.");
+            failed(ERR_FTP_FAILURE, 0);
+            return false;
+        }
+        // else fall through to skip EPSV 1
+
+    case SENT_EPSV_1: /* EPSV options exhausted. Try PASV now. */
+        debugs(9, 5, "FTP Channel (" << ctrl.conn->remote << ") rejects EPSV connection attempts. Trying PASV instead.");
+        mb.Printf("PASV%s", Ftp::crlf);
+        state = SENT_PASV;
+        break;
+
+    default: {
+        bool doEpsv = true;
+        if (Config.accessList.ftp_epsv) {
+            ACLFilledChecklist checklist(Config.accessList.ftp_epsv, fwd->request, NULL);
+            doEpsv = (checklist.fastCheck() == ACCESS_ALLOWED);
+        }
+        if (!doEpsv) {
+            debugs(9, 5, "EPSV support manually disabled. Sending PASV for FTP Channel (" << ctrl.conn->remote <<")");
+            mb.Printf("PASV%s", Ftp::crlf);
+            state = SENT_PASV;
+        } else if (Config.Ftp.epsv_all) {
+            debugs(9, 5, "EPSV ALL manually enabled. Attempting with FTP Channel (" << ctrl.conn->remote <<")");
+            mb.Printf("EPSV ALL%s", Ftp::crlf);
+            state = SENT_EPSV_ALL;
+        } else {
+            if (ctrl.conn->local.isIPv6()) {
+                debugs(9, 5, "FTP Channel (" << ctrl.conn->remote << "). Sending default EPSV 2");
+                mb.Printf("EPSV 2%s", Ftp::crlf);
+                state = SENT_EPSV_2;
+            }
+            if (ctrl.conn->local.isIPv4()) {
+                debugs(9, 5, "Channel (" << ctrl.conn->remote <<"). Sending default EPSV 1");
+                mb.Printf("EPSV 1%s", Ftp::crlf);
+                state = SENT_EPSV_1;
+            }
+        }
+        break;
+    }
+    }
+
+    if (ctrl.message)
+        wordlistDestroy(&ctrl.message);
+    ctrl.message = NULL; //No message to return to client.
+    ctrl.offset = 0; //reset readed response, to make room read the next response
+
+    writeCommand(mb.content());
+
+    shortenReadTimeout = true;
+    return true;
+}
+
+void
+Ftp::Client::connectDataChannel()
+{
+    safe_free(ctrl.last_command);
+
+    safe_free(ctrl.last_reply);
+
+    ctrl.last_command = xstrdup("Connect to server data port");
+
+    // Generate a new data channel descriptor to be opened.
+    Comm::ConnectionPointer conn = new Comm::Connection;
+    conn->setAddrs(ctrl.conn->local, data.host);
+    conn->local.port(0);
+    conn->remote.port(data.port);
+    conn->tos = ctrl.conn->tos;
+    conn->nfmark = ctrl.conn->nfmark;
+
+    debugs(9, 3, "connecting to " << conn->remote);
+
+    typedef CommCbMemFunT<Client, CommConnectCbParams> Dialer;
+    data.opener = JobCallback(9, 3, Dialer, this, Ftp::Client::dataChannelConnected);
+    Comm::ConnOpener *cs = new Comm::ConnOpener(conn, data.opener, Config.Timeout.connect);
+    cs->setHost(data.host);
+    AsyncJob::Start(cs);
+}
+
+bool
+Ftp::Client::openListenSocket()
+{
+    return false;
+}
+
+/// creates a data channel Comm close callback
+AsyncCall::Pointer
+Ftp::Client::dataCloser()
+{
+    typedef CommCbMemFunT<Client, CommCloseCbParams> Dialer;
+    return JobCallback(9, 5, Dialer, this, Ftp::Client::dataClosed);
+}
+
+/// handler called by Comm when FTP data channel is closed unexpectedly
+void
+Ftp::Client::dataClosed(const CommCloseCbParams &io)
+{
+    debugs(9, 4, status());
+    if (data.listenConn != NULL) {
+        data.listenConn->close();
+        data.listenConn = NULL;
+        // NP clear() does the: data.fd = -1;
+    }
+    data.clear();
+}
+
+void
+Ftp::Client::writeCommand(const char *buf)
+{
+    char *ebuf;
+    /* trace FTP protocol communications at level 2 */
+    debugs(9, 2, "ftp<< " << buf);
+
+    if (Config.Ftp.telnet)
+        ebuf = escapeIAC(buf);
+    else
+        ebuf = xstrdup(buf);
+
+    safe_free(ctrl.last_command);
+
+    safe_free(ctrl.last_reply);
+
+    ctrl.last_command = ebuf;
+
+    if (!Comm::IsConnOpen(ctrl.conn)) {
+        debugs(9, 2, "cannot send to closing ctrl " << ctrl.conn);
+        // TODO: assert(ctrl.closer != NULL);
+        return;
+    }
+
+    typedef CommCbMemFunT<Client, CommIoCbParams> Dialer;
+    AsyncCall::Pointer call = JobCallback(9, 5, Dialer, this,
+                                          Ftp::Client::writeCommandCallback);
+    Comm::Write(ctrl.conn, ctrl.last_command, strlen(ctrl.last_command), call, NULL);
+
+    scheduleReadControlReply(0);
+}
+
+void
+Ftp::Client::writeCommandCallback(const CommIoCbParams &io)
+{
+
+    debugs(9, 5, "wrote " << io.size << " bytes");
+
+    if (io.size > 0) {
+        fd_bytes(io.fd, io.size, FD_WRITE);
+        kb_incr(&(statCounter.server.all.kbytes_out), io.size);
+        kb_incr(&(statCounter.server.ftp.kbytes_out), io.size);
+    }
+
+    if (io.flag == Comm::ERR_CLOSING)
+        return;
+
+    if (io.flag) {
+        debugs(9, DBG_IMPORTANT, "FTP command write error: " << io.conn << ": " << xstrerr(io.xerrno));
+        failed(ERR_WRITE_ERROR, io.xerrno);
+        /* failed closes ctrl.conn and frees ftpState */
+        return;
+    }
+}
+
+/// handler called by Comm when FTP control channel is closed unexpectedly
+void
+Ftp::Client::ctrlClosed(const CommCloseCbParams &io)
+{
+    debugs(9, 4, status());
+    ctrl.clear();
+    mustStop("Ftp::Client::ctrlClosed");
+}
+
+void
+Ftp::Client::timeout(const CommTimeoutCbParams &io)
+{
+    debugs(9, 4, io.conn << ": '" << entry->url() << "'" );
+
+    if (abortOnBadEntry("entry went bad while waiting for a timeout"))
+        return;
+
+    failed(ERR_READ_TIMEOUT, 0);
+    /* failed() closes ctrl.conn and frees ftpState */
+}
+
+const Comm::ConnectionPointer &
+Ftp::Client::dataConnection() const
+{
+    return data.conn;
+}
+
+void
+Ftp::Client::maybeReadVirginBody()
+{
+    // too late to read
+    if (!Comm::IsConnOpen(data.conn) || fd_table[data.conn->fd].closing())
+        return;
+
+    if (data.read_pending)
+        return;
+
+    initReadBuf();
+
+    const int read_sz = replyBodySpace(*data.readBuf, 0);
+
+    debugs(9, 9, "FTP may read up to " << read_sz << " bytes");
+
+    if (read_sz < 2) // see http.cc
+        return;
+
+    data.read_pending = true;
+
+    typedef CommCbMemFunT<Client, CommTimeoutCbParams> TimeoutDialer;
+    AsyncCall::Pointer timeoutCall =  JobCallback(9, 5,
+                                      TimeoutDialer, this, Ftp::Client::timeout);
+    commSetConnTimeout(data.conn, Config.Timeout.read, timeoutCall);
+
+    debugs(9,5,"queueing read on FD " << data.conn->fd);
+
+    typedef CommCbMemFunT<Client, CommIoCbParams> Dialer;
+    entry->delayAwareRead(data.conn, data.readBuf->space(), read_sz,
+                          JobCallback(9, 5, Dialer, this, Ftp::Client::dataRead));
+}
+
+void
+Ftp::Client::dataRead(const CommIoCbParams &io)
+{
+    int j;
+    int bin;
+
+    data.read_pending = false;
+
+    debugs(9, 3, "FD " << io.fd << " Read " << io.size << " bytes");
+
+    if (io.size > 0) {
+        kb_incr(&(statCounter.server.all.kbytes_in), io.size);
+        kb_incr(&(statCounter.server.ftp.kbytes_in), io.size);
+    }
+
+    if (io.flag == Comm::ERR_CLOSING)
+        return;
+
+    assert(io.fd == data.conn->fd);
+
+    if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) {
+        abortTransaction("entry aborted during dataRead");
+        return;
+    }
+
+    if (io.flag == Comm::OK && io.size > 0) {
+        debugs(9, 5, "appended " << io.size << " bytes to readBuf");
+        data.readBuf->appended(io.size);
+#if USE_DELAY_POOLS
+        DelayId delayId = entry->mem_obj->mostBytesAllowed();
+        delayId.bytesIn(io.size);
+#endif
+        ++ IOStats.Ftp.reads;
+
+        for (j = io.size - 1, bin = 0; j; ++bin)
+            j >>= 1;
+
+        ++ IOStats.Ftp.read_hist[bin];
+    }
+
+    if (io.flag != Comm::OK) {
+        debugs(50, ignoreErrno(io.xerrno) ? 3 : DBG_IMPORTANT,
+               "FTP data read error: " << xstrerr(io.xerrno));
+
+        if (ignoreErrno(io.xerrno)) {
+            maybeReadVirginBody();
+        } else {
+            failed(ERR_READ_ERROR, 0);
+            /* failed closes ctrl.conn and frees ftpState */
+            return;
+        }
+    } else if (io.size == 0) {
+        debugs(9, 3, "Calling dataComplete() because io.size == 0");
+        /*
+         * DPW 2007-04-23
+         * Dangerous curves ahead.  This call to dataComplete was
+         * calling scheduleReadControlReply, handleControlReply,
+         * and then ftpReadTransferDone.  If ftpReadTransferDone
+         * gets unexpected status code, it closes down the control
+         * socket and our FtpStateData object gets destroyed.   As
+         * a workaround we no longer set the 'buffered_ok' flag in
+         * the scheduleReadControlReply call.
+         */
+        dataComplete();
+    }
+
+    processReplyBody();
+}
+
+void
+Ftp::Client::dataComplete()
+{
+    debugs(9, 3,status());
+
+    /* Connection closed; transfer done. */
+
+    /// Close data channel, if any, to conserve resources while we wait.
+    data.close();
+
+    /* expect the "transfer complete" message on the control socket */
+    /*
+     * DPW 2007-04-23
+     * Previously, this was the only place where we set the
+     * 'buffered_ok' flag when calling scheduleReadControlReply().
+     * It caused some problems if the FTP server returns an unexpected
+     * status code after the data command.  FtpStateData was being
+     * deleted in the middle of dataRead().
+     */
+    /* AYJ: 2011-01-13: Bug 2581.
+     * 226 status is possibly waiting in the ctrl buffer.
+     * The connection will hang if we DONT send buffered_ok.
+     * This happens on all transfers which can be completly sent by the
+     * server before the 150 started status message is read in by Squid.
+     * ie all transfers of about one packet hang.
+     */
+    scheduleReadControlReply(1);
+}
+
+/**
+ * Quickly abort the transaction
+ *
+ \todo destruction should be sufficient as the destructor should cleanup,
+ * including canceling close handlers
+ */
+void
+Ftp::Client::abortTransaction(const char *reason)
+{
+    debugs(9, 3, "aborting transaction for " << reason <<
+           "; FD " << (ctrl.conn!=NULL?ctrl.conn->fd:-1) << ", Data FD " << (data.conn!=NULL?data.conn->fd:-1) << ", this " << this);
+    if (Comm::IsConnOpen(ctrl.conn)) {
+        ctrl.conn->close();
+        return;
+    }
+
+    fwd->handleUnregisteredServerEnd();
+    mustStop("Ftp::Client::abortTransaction");
+}
+
+/**
+ * Cancel the timeout on the Control socket and establish one
+ * on the data socket
+ */
+void
+Ftp::Client::switchTimeoutToDataChannel()
+{
+    commUnsetConnTimeout(ctrl.conn);
+
+    typedef CommCbMemFunT<Client, CommTimeoutCbParams> TimeoutDialer;
+    AsyncCall::Pointer timeoutCall = JobCallback(9, 5, TimeoutDialer, this,
+                                     Ftp::Client::timeout);
+    commSetConnTimeout(data.conn, Config.Timeout.read, timeoutCall);
+}
+
+void
+Ftp::Client::sentRequestBody(const CommIoCbParams &io)
+{
+    if (io.size > 0)
+        kb_incr(&(statCounter.server.ftp.kbytes_out), io.size);
+    ::ServerStateData::sentRequestBody(io);
+}
+
+/**
+ * called after we wrote the last byte of the request body
+ */
+void
+Ftp::Client::doneSendingRequestBody()
+{
+    ::ServerStateData::doneSendingRequestBody();
+    debugs(9, 3, status());
+    dataComplete();
+    /* NP: RFC 959  3.3.  DATA CONNECTION MANAGEMENT
+     * if transfer type is 'stream' call dataComplete()
+     * otherwise leave open. (reschedule control channel read?)
+     */
+}
+
+/// Parses FTP server control response into ctrl structure fields,
+/// setting bytesUsed and returning true on success.
+bool
+Ftp::Client::parseControlReply(size_t &bytesUsed)
+{
+    char *s;
+    char *sbuf;
+    char *end;
+    int usable;
+    int complete = 0;
+    wordlist *head = NULL;
+    wordlist *list;
+    wordlist **tail = &head;
+    size_t linelen;
+    debugs(9, 3, status());
+    /*
+     * We need a NULL-terminated buffer for scanning, ick
+     */
+    const size_t len = ctrl.offset;
+    sbuf = (char *)xmalloc(len + 1);
+    xstrncpy(sbuf, ctrl.buf, len + 1);
+    end = sbuf + len - 1;
+
+    while (*end != '\r' && *end != '\n' && end > sbuf)
+        --end;
+
+    usable = end - sbuf;
+
+    debugs(9, 3, "usable = " << usable);
+
+    if (usable == 0) {
+        debugs(9, 3, "didn't find end of line");
+        safe_free(sbuf);
+        return false;
+    }
+
+    debugs(9, 3, len << " bytes to play with");
+    ++end;
+    s = sbuf;
+    s += strspn(s, crlf);
+
+    for (; s < end; s += strcspn(s, crlf), s += strspn(s, crlf)) {
+        if (complete)
+            break;
+
+        debugs(9, 5, "s = {" << s << "}");
+
+        linelen = strcspn(s, crlf) + 1;
+
+        if (linelen < 2)
+            break;
+
+        if (linelen > 3)
+            complete = (*s >= '0' && *s <= '9' && *(s + 3) == ' ');
+
+        list = new wordlist();
+
+        list->key = (char *)xmalloc(linelen);
+
+        xstrncpy(list->key, s, linelen);
+
+        /* trace the FTP communication chat at level 2 */
+        debugs(9, 2, "ftp>> " << list->key);
+
+        if (complete) {
+            // use list->key for last_reply because s contains the new line
+            ctrl.last_reply = xstrdup(list->key + 4);
+            ctrl.replycode = atoi(list->key);
+        }
+
+        *tail = list;
+
+        tail = &list->next;
+    }
+
+    bytesUsed = static_cast<size_t>(s - sbuf);
+    safe_free(sbuf);
+
+    if (!complete) {
+        wordlistDestroy(&head);
+        return false;
+    }
+
+    ctrl.message = head;
+    assert(ctrl.replycode >= 0);
+    assert(ctrl.last_reply);
+    assert(ctrl.message);
+    return true;
+}
+
+}; // namespace Ftp

=== added file 'src/clients/FtpClient.h'
--- src/clients/FtpClient.h	1970-01-01 00:00:00 +0000
+++ src/clients/FtpClient.h	2014-08-08 03:07:03 +0000
@@ -0,0 +1,193 @@
+/*
+ * DEBUG: section 09    File Transfer Protocol (FTP)
+ *
+ */
+
+#ifndef SQUID_FTP_CLIENT_H
+#define SQUID_FTP_CLIENT_H
+
+#include "Server.h"
+
+class String;
+namespace Ftp
+{
+
+extern const char *const crlf;
+
+/// Common code for FTP server control and data channels.
+/// Does not own the channel descriptor, which is managed by Ftp::Client.
+class Channel
+{
+public:
+    /// called after the socket is opened, sets up close handler
+    void opened(const Comm::ConnectionPointer &conn, const AsyncCall::Pointer &aCloser);
+
+    /** Handles all operations needed to properly close the active channel FD.
+     * clearing the close handler, clearing the listen socket properly, and calling comm_close
+     */
+    void close();
+
+    void forget(); /// remove the close handler, leave connection open
+
+    void clear(); ///< just drops conn and close handler. does not close active connections.
+
+    Comm::ConnectionPointer conn; ///< channel descriptor
+
+    /** A temporary handle to the connection being listened on.
+     * Closing this will also close the waiting Data channel acceptor.
+     * If a data connection has already been accepted but is still waiting in the event queue
+     * the callback will still happen and needs to be handled (usually dropped).
+     */
+    Comm::ConnectionPointer listenConn;
+
+    AsyncCall::Pointer opener; ///< Comm opener handler callback.
+private:
+    AsyncCall::Pointer closer; ///< Comm close handler callback
+};
+
+/// FTP channel for control commands.
+/// This channel is opened once per transaction.
+class CtrlChannel: public Ftp::Channel
+{
+public:
+    CtrlChannel();
+    ~CtrlChannel();
+
+    char *buf;
+    size_t size;
+    size_t offset;
+    wordlist *message;
+    char *last_command;
+    char *last_reply;
+    int replycode;
+
+private:
+    CtrlChannel(const CtrlChannel &); // not implemented
+    CtrlChannel &operator =(const CtrlChannel &); // not implemented
+};
+
+/// FTP channel for data exchanges.
+/// This channel may be opened/closed a few times.
+class DataChannel: public Ftp::Channel
+{
+public:
+    DataChannel();
+    ~DataChannel();
+
+    void addr(const Ip::Address &addr); ///< import host and port
+
+public:
+    MemBuf *readBuf;
+    char *host;
+    unsigned short port;
+    bool read_pending;
+};
+
+/// FTP client functionality shared among FTP Gateway and Relay clients.
+class Client: public ::ServerStateData
+{
+public:
+    explicit Client(FwdState *fwdState);
+    virtual ~Client();
+
+    /// handle a fatal transaction error, closing the control connection
+    virtual void failed(err_type error = ERR_NONE, int xerrno = 0);
+
+    /// read timeout handler
+    virtual void timeout(const CommTimeoutCbParams &io);
+
+    /* ServerStateData API */
+    virtual void maybeReadVirginBody();
+
+    void writeCommand(const char *buf);
+
+    /// extracts remoteAddr from PASV response, validates it,
+    /// sets data address details, and returns true on success
+    bool handlePasvReply(Ip::Address &remoteAddr);
+    bool handleEpsvReply(Ip::Address &remoteAddr);
+
+    bool sendEprt();
+    bool sendPort();
+    bool sendPassive();
+    void connectDataChannel();
+    bool openListenSocket();
+    void switchTimeoutToDataChannel();
+
+    CtrlChannel ctrl; ///< FTP control channel state
+    DataChannel data; ///< FTP data channel state
+
+    enum {
+        BEGIN,
+        SENT_USER,
+        SENT_PASS,
+        SENT_TYPE,
+        SENT_MDTM,
+        SENT_SIZE,
+        SENT_EPRT,
+        SENT_PORT,
+        SENT_EPSV_ALL,
+        SENT_EPSV_1,
+        SENT_EPSV_2,
+        SENT_PASV,
+        SENT_CWD,
+        SENT_LIST,
+        SENT_NLST,
+        SENT_REST,
+        SENT_RETR,
+        SENT_STOR,
+        SENT_QUIT,
+        READING_DATA,
+        WRITING_DATA,
+        SENT_MKDIR,
+        SENT_FEAT,
+        SENT_PWD,
+        SENT_CDUP,
+        SENT_DATA_REQUEST, // LIST, NLST or RETR requests..
+        SENT_COMMAND, // General command
+        END
+    } ftp_state_t;
+
+    int state;
+    char *old_request;
+    char *old_reply;
+
+protected:
+    /* AsyncJob API */
+    virtual void start();
+
+    /* ServerStateData API */
+    virtual void closeServer();
+    virtual bool doneWithServer() const;
+    virtual const Comm::ConnectionPointer & dataConnection() const;
+    virtual void abortTransaction(const char *reason);
+
+    virtual Http::StatusCode failedHttpStatus(err_type &error);
+    void ctrlClosed(const CommCloseCbParams &io);
+    void scheduleReadControlReply(int buffered_ok);
+    void readControlReply(const CommIoCbParams &io);
+    virtual void handleControlReply();
+    void writeCommandCallback(const CommIoCbParams &io);
+    virtual void dataChannelConnected(const CommConnectCbParams &io) = 0;
+    void dataRead(const CommIoCbParams &io);
+    void dataComplete();
+    AsyncCall::Pointer dataCloser();
+    virtual void dataClosed(const CommCloseCbParams &io);
+    void initReadBuf();
+
+    // sending of the request body to the server
+    virtual void sentRequestBody(const CommIoCbParams &io);
+    virtual void doneSendingRequestBody();
+
+private:
+    bool parseControlReply(size_t &bytesUsed);
+
+    /// XXX: An old hack for FTP servers like ftp.netscape.com that may not
+    /// respond to PASV. Use faster connect timeout instead of read timeout.
+    bool shortenReadTimeout;
+
+    CBDATA_CLASS2(Client);
+};
+
+} // namespace Ftp
+
+#endif /* SQUID_FTP_CLIENT_H */

=== renamed file 'src/ftp.cc' => 'src/clients/FtpGateway.cc'
--- src/ftp.cc	2014-07-30 16:04:19 +0000
+++ src/clients/FtpGateway.cc	2014-08-11 16:09:06 +0000
@@ -32,11 +32,12 @@
 
 #include "squid.h"
 #include "acl/FilledChecklist.h"
+#include "clients/forward.h"
+#include "clients/FtpClient.h"
 #include "comm.h"
 #include "comm/ConnOpener.h"
 #include "comm/Read.h"
 #include "comm/TcpAcceptor.h"
-#include "comm/Write.h"
 #include "CommCalls.h"
 #include "compat/strtoll.h"
 #include "errorpage.h"
@@ -71,46 +72,10 @@
 
 #include <cerrno>
 
-/**
- \defgroup ServerProtocolFTPInternal Server-Side FTP Internals
- \ingroup ServerProtocolFTPAPI
- */
-
-/// \ingroup ServerProtocolFTPInternal
-static const char *const crlf = "\r\n";
-
-#define CTRL_BUFLEN 1024
-/// \ingroup ServerProtocolFTPInternal
-static char cbuf[CTRL_BUFLEN];
-
-/// \ingroup ServerProtocolFTPInternal
-typedef enum {
-    BEGIN,
-    SENT_USER,
-    SENT_PASS,
-    SENT_TYPE,
-    SENT_MDTM,
-    SENT_SIZE,
-    SENT_EPRT,
-    SENT_PORT,
-    SENT_EPSV_ALL,
-    SENT_EPSV_1,
-    SENT_EPSV_2,
-    SENT_PASV,
-    SENT_CWD,
-    SENT_LIST,
-    SENT_NLST,
-    SENT_REST,
-    SENT_RETR,
-    SENT_STOR,
-    SENT_QUIT,
-    READING_DATA,
-    WRITING_DATA,
-    SENT_MKDIR
-} ftp_state_t;
-
-/// \ingroup ServerProtocolFTPInternal
-struct _ftp_flags {
+namespace Ftp
+{
+
+struct GatewayFlags {
 
     /* passive mode */
     bool pasv_supported;  ///< PASV command is allowed
@@ -142,48 +107,17 @@
     bool completed_forwarding;
 };
 
-class FtpStateData;
-
-/// \ingroup ServerProtocolFTPInternal
-typedef void (FTPSM) (FtpStateData *);
-
-/// common code for FTP control and data channels
-/// does not own the channel descriptor, which is managed by FtpStateData
-class FtpChannel
-{
-public:
-    FtpChannel() {};
-
-    /// called after the socket is opened, sets up close handler
-    void opened(const Comm::ConnectionPointer &conn, const AsyncCall::Pointer &aCloser);
-
-    /** Handles all operations needed to properly close the active channel FD.
-     * clearing the close handler, clearing the listen socket properly, and calling comm_close
-     */
-    void close();
-
-    void clear(); ///< just drops conn and close handler. does not close active connections.
-
-    Comm::ConnectionPointer conn; ///< channel descriptor
-
-    /** A temporary handle to the connection being listened on.
-     * Closing this will also close the waiting Data channel acceptor.
-     * If a data connection has already been accepted but is still waiting in the event queue
-     * the callback will still happen and needs to be handled (usually dropped).
-     */
-    Comm::ConnectionPointer listenConn;
-
-    AsyncCall::Pointer opener; ///< Comm opener handler callback.
-private:
-    AsyncCall::Pointer closer; ///< Comm close handler callback
-};
-
-/// \ingroup ServerProtocolFTPInternal
-class FtpStateData : public ServerStateData
-{
-public:
-    FtpStateData(FwdState *, const Comm::ConnectionPointer &conn);
-    ~FtpStateData();
+class Gateway;
+typedef void (StateMethod)(Ftp::Gateway *);
+
+/// FTP Gateway: An FTP client that takes an HTTP request with an ftp:// URI,
+/// converts it into one or more FTP commands, and then
+/// converts one or more FTP responses into the final HTTP response.
+class Gateway : public Ftp::Client
+{
+public:
+    Gateway(FwdState *);
+    virtual ~Gateway();
     char user[MAX_URL];
     char password[MAX_URL];
     int password_url;
@@ -194,7 +128,6 @@
     String base_href;
     int conn_att;
     int login_att;
-    ftp_state_t state;
     time_t mdtm;
     int64_t theSize;
     wordlist *pathcomps;
@@ -204,109 +137,71 @@
     char *proxy_host;
     size_t list_width;
     String cwd_message;
-    char *old_request;
-    char *old_reply;
     char *old_filepath;
     char typecode;
     MemBuf listing;		///< FTP directory listing in HTML format.
 
-    // \todo: optimize ctrl and data structs member order, to minimize size
-    /// FTP control channel info; the channel is opened once per transaction
-    struct CtrlChannel: public FtpChannel {
-        char *buf;
-        size_t size;
-        size_t offset;
-        wordlist *message;
-        char *last_command;
-        char *last_reply;
-        int replycode;
-    } ctrl;
-
-    /// FTP data channel info; the channel may be opened/closed a few times
-    struct DataChannel: public FtpChannel {
-        MemBuf *readBuf;
-        char *host;
-        unsigned short port;
-        bool read_pending;
-    } data;
-
-    struct _ftp_flags flags;
+    GatewayFlags flags;
 
 public:
     // these should all be private
     virtual void start();
+    virtual Http::StatusCode failedHttpStatus(err_type &error);
     void loginParser(const char *, int escaped);
     int restartable();
     void appendSuccessHeader();
-    void hackShortcut(FTPSM * nextState);
-    void failed(err_type, int xerrno);
-    void failedErrorMessage(err_type, int xerrno);
+    void hackShortcut(StateMethod *nextState);
     void unhack();
-    void scheduleReadControlReply(int);
-    void handleControlReply();
     void readStor();
     void parseListing();
     MemBuf *htmlifyListEntry(const char *line);
     void completedListing(void);
-    void dataComplete();
-    void dataRead(const CommIoCbParams &io);
 
-    /// ignore timeout on CTRL channel. set read timeout on DATA channel.
-    void switchTimeoutToDataChannel();
     /// create a data channel acceptor and start listening.
-    void listenForDataChannel(const Comm::ConnectionPointer &conn, const char *note);
+    void listenForDataChannel(const Comm::ConnectionPointer &conn);
 
     int checkAuth(const HttpHeader * req_hdr);
     void checkUrlpath();
     void buildTitleUrl();
     void writeReplyBody(const char *, size_t len);
     void printfReplyBody(const char *fmt, ...);
-    virtual const Comm::ConnectionPointer & dataConnection() const;
-    virtual void maybeReadVirginBody();
-    virtual void closeServer();
     virtual void completeForwarding();
-    virtual void abortTransaction(const char *reason);
     void processHeadResponse();
     void processReplyBody();
-    void writeCommand(const char *buf);
     void setCurrentOffset(int64_t offset) { currentOffset = offset; }
     int64_t getCurrentOffset() const { return currentOffset; }
 
-    static CNCB ftpPasvCallback;
+    virtual void dataChannelConnected(const CommConnectCbParams &io);
     static PF ftpDataWrite;
-    void ftpTimeout(const CommTimeoutCbParams &io);
-    void ctrlClosed(const CommCloseCbParams &io);
-    void dataClosed(const CommCloseCbParams &io);
-    void ftpReadControlReply(const CommIoCbParams &io);
-    void ftpWriteCommandCallback(const CommIoCbParams &io);
+    virtual void timeout(const CommTimeoutCbParams &io);
     void ftpAcceptDataConnection(const CommAcceptCbParams &io);
 
     static HttpReply *ftpAuthRequired(HttpRequest * request, const char *realm);
     const char *ftpRealm(void);
     void loginFailed(void);
-    static wordlist *ftpParseControlReply(char *, size_t, int *, size_t *);
-
-    // sending of the request body to the server
-    virtual void sentRequestBody(const CommIoCbParams&);
-    virtual void doneSendingRequestBody();
 
     virtual void haveParsedReplyHeaders();
 
-    virtual bool doneWithServer() const;
     virtual bool haveControlChannel(const char *caller_name) const;
-    AsyncCall::Pointer dataCloser(); /// creates a Comm close callback
-    AsyncCall::Pointer dataOpener(); /// creates a Comm connect callback
+
+protected:
+    virtual void handleControlReply();
+    virtual void dataClosed(const CommCloseCbParams &io);
 
 private:
+    virtual bool mayReadVirginReplyBody() const;
     // BodyConsumer for HTTP: consume request body.
     virtual void handleRequestBodyProducerAborted();
 
-    CBDATA_CLASS2(FtpStateData);
+    CBDATA_CLASS2(Gateway);
 };
 
-CBDATA_CLASS_INIT(FtpStateData);
-
-/// \ingroup ServerProtocolFTPInternal
+} // namespace Ftp
+
+typedef Ftp::StateMethod FTPSM; // to avoid lots of non-changes
+
+CBDATA_NAMESPACED_CLASS_INIT(Ftp, Gateway);
+
 typedef struct {
     char type;
     int64_t size;
@@ -316,12 +211,13 @@
     char *link;
 } ftpListParts;
 
-/// \ingroup ServerProtocolFTPInternal
 #define FTP_LOGIN_ESCAPED	1
 
-/// \ingroup ServerProtocolFTPInternal
 #define FTP_LOGIN_NOT_ESCAPED	0
 
+#define CTRL_BUFLEN 1024
+static char cbuf[CTRL_BUFLEN];
+
 /*
  * State machine functions
  * send == state transition
@@ -412,7 +308,6 @@
 Quit			-
 ************************************************/
 
-/// \ingroup ServerProtocolFTPInternal
 FTPSM *FTP_SM_FUNCS[] = {
     ftpReadWelcome,		/* BEGIN */
     ftpReadUser,		/* SENT_USER */
@@ -435,29 +330,19 @@
     ftpReadQuit,		/* SENT_QUIT */
     ftpReadTransferDone,	/* READING_DATA (RETR,LIST,NLST) */
     ftpWriteTransferDone,	/* WRITING_DATA (STOR) */
-    ftpReadMkdir		/* SENT_MKDIR */
+    ftpReadMkdir,		/* SENT_MKDIR */
+    NULL,			/* SENT_FEAT */
+    NULL,			/* SENT_PWD */
+    NULL,			/* SENT_CDUP*/
+    NULL,			/* SENT_DATA_REQUEST */
+    NULL			/* SENT_COMMAND */
 };
 
-/// handler called by Comm when FTP control channel is closed unexpectedly
-void
-FtpStateData::ctrlClosed(const CommCloseCbParams &io)
-{
-    debugs(9, 4, HERE);
-    ctrl.clear();
-    mustStop("FtpStateData::ctrlClosed");
-}
-
 /// handler called by Comm when FTP data channel is closed unexpectedly
 void
-FtpStateData::dataClosed(const CommCloseCbParams &io)
+Ftp::Gateway::dataClosed(const CommCloseCbParams &io)
 {
-    debugs(9, 4, HERE);
-    if (data.listenConn != NULL) {
-        data.listenConn->close();
-        data.listenConn = NULL;
-        // NP clear() does the: data.fd = -1;
-    }
-    data.clear();
+    Ftp::Client::dataClosed(io);
     failed(ERR_FTP_FAILURE, 0);
     /* failed closes ctrl.conn and frees ftpState */
 
@@ -467,12 +352,12 @@
      */
 }
 
-FtpStateData::FtpStateData(FwdState *theFwdState, const Comm::ConnectionPointer &conn) : AsyncJob("FtpStateData"), ServerStateData(theFwdState)
+Ftp::Gateway::Gateway(FwdState *fwdState):
+        AsyncJob("FtpStateData"),
+        Ftp::Client(fwdState)
 {
     const char *url = entry->url();
     debugs(9, 3, HERE << "'" << url << "'" );
-    ++ statCounter.server.all.requests;
-    ++ statCounter.server.ftp.requests;
     theSize = -1;
     mdtm = -1;
 
@@ -481,62 +366,31 @@
 
     flags.rest_supported = 1;
 
-    typedef CommCbMemFunT<FtpStateData, CommCloseCbParams> Dialer;
-    AsyncCall::Pointer closer = JobCallback(9, 5, Dialer, this, FtpStateData::ctrlClosed);
-    ctrl.opened(conn, closer);
-
     if (request->method == Http::METHOD_PUT)
         flags.put = 1;
+
+    initReadBuf();
 }
 
-FtpStateData::~FtpStateData()
+Ftp::Gateway::~Gateway()
 {
     debugs(9, 3, HERE << entry->url()  );
 
+    if (Comm::IsConnOpen(ctrl.conn)) {
+        debugs(9, DBG_IMPORTANT, "Internal bug: FTP Gateway left open " <<
+               "control channel " << ctrl.conn);
+    }
+
     if (reply_hdr) {
         memFree(reply_hdr, MEM_8K_BUF);
         reply_hdr = NULL;
     }
 
-    if (data.opener != NULL) {
-        data.opener->cancel("FtpStateData destructed");
-        data.opener = NULL;
-    }
-    data.close();
-
-    if (Comm::IsConnOpen(ctrl.conn)) {
-        debugs(9, DBG_IMPORTANT, HERE << "Internal bug: FtpStateData left " <<
-               "open control channel " << ctrl.conn);
-    }
-
-    if (ctrl.buf) {
-        memFreeBuf(ctrl.size, ctrl.buf);
-        ctrl.buf = NULL;
-    }
-
-    if (data.readBuf) {
-        if (!data.readBuf->isNull())
-            data.readBuf->clean();
-
-        delete data.readBuf;
-    }
-
     if (pathcomps)
         wordlistDestroy(&pathcomps);
 
-    if (ctrl.message)
-        wordlistDestroy(&ctrl.message);
-
     cwd_message.clean();
 
-    safe_free(ctrl.last_reply);
-
-    safe_free(ctrl.last_command);
-
-    safe_free(old_request);
-
-    safe_free(old_reply);
-
     safe_free(old_filepath);
 
     title_url.clean();
@@ -546,10 +400,6 @@
     safe_free(filepath);
 
     safe_free(dirpath);
-
-    safe_free(data.host);
-
-    fwd = NULL;	// refcounted
 }
 
 /**
@@ -557,7 +407,7 @@
  * Produces filled member variables user, password, password_url if anything found.
  */
 void
-FtpStateData::loginParser(const char *login, int escaped)
+Ftp::Gateway::loginParser(const char *login, int escaped)
 {
     const char *u = NULL; // end of the username sub-string
     int len;              // length of the current sub-string to handle.
@@ -609,29 +459,16 @@
     debugs(9, 9, HERE << ": OUT: login='" << login << "', escaped=" << escaped << ", user=" << user << ", password=" << password);
 }
 
-/**
- * Cancel the timeout on the Control socket and establish one
- * on the data socket
- */
-void
-FtpStateData::switchTimeoutToDataChannel()
-{
-    commUnsetConnTimeout(ctrl.conn);
-
-    typedef CommCbMemFunT<FtpStateData, CommTimeoutCbParams> TimeoutDialer;
-    AsyncCall::Pointer timeoutCall = JobCallback(9, 5, TimeoutDialer, this, FtpStateData::ftpTimeout);
-    commSetConnTimeout(data.conn, Config.Timeout.read, timeoutCall);
-}
-
-void
-FtpStateData::listenForDataChannel(const Comm::ConnectionPointer &conn, const char *note)
+void
+Ftp::Gateway::listenForDataChannel(const Comm::ConnectionPointer &conn)
 {
     assert(!Comm::IsConnOpen(data.conn));
 
-    typedef CommCbMemFunT<FtpStateData, CommAcceptCbParams> AcceptDialer;
+    typedef CommCbMemFunT<Gateway, CommAcceptCbParams> AcceptDialer;
     typedef AsyncCallT<AcceptDialer> AcceptCall;
-    RefCount<AcceptCall> call = static_cast<AcceptCall*>(JobCallback(11, 5, AcceptDialer, this, FtpStateData::ftpAcceptDataConnection));
+    RefCount<AcceptCall> call = static_cast<AcceptCall*>(JobCallback(11, 5, AcceptDialer, this, Ftp::Gateway::ftpAcceptDataConnection));
     Subscription::Pointer sub = new CallSubscription<AcceptCall>(call);
+    const char *note = entry->url();
 
     /* open the conn if its not already open */
     if (!Comm::IsConnOpen(conn)) {
@@ -655,17 +492,12 @@
 }
 
 void
-FtpStateData::ftpTimeout(const CommTimeoutCbParams &io)
+Ftp::Gateway::timeout(const CommTimeoutCbParams &io)
 {
-    debugs(9, 4, HERE << io.conn << ": '" << entry->url() << "'" );
-
-    if (abortOnBadEntry("entry went bad while waiting for a timeout"))
-        return;
-
     if (SENT_PASV == state) {
         /* stupid ftp.netscape.com, of FTP server behind stupid firewall rules */
         flags.pasv_supported = false;
-        debugs(9, DBG_IMPORTANT, "ftpTimeout: timeout in SENT_PASV state" );
+        debugs(9, DBG_IMPORTANT, "FTP Gateway timeout in SENT_PASV state");
 
         // cancel the data connection setup.
         if (data.opener != NULL) {
@@ -675,17 +507,14 @@
         data.close();
     }
 
-    failed(ERR_READ_TIMEOUT, 0);
-    /* failed() closes ctrl.conn and frees ftpState */
+    Ftp::Client::timeout(io);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static const char *Month[] = {
     "Jan", "Feb", "Mar", "Apr", "May", "Jun",
     "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"
 };
 
-/// \ingroup ServerProtocolFTPInternal
 static int
 is_month(const char *buf)
 {
@@ -698,7 +527,6 @@
     return 0;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
 ftpListPartsFree(ftpListParts ** parts)
 {
@@ -709,12 +537,10 @@
     safe_free(*parts);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 #define MAX_TOKENS 64
 
-/// \ingroup ServerProtocolFTPInternal
 static ftpListParts *
-ftpListParseParts(const char *buf, struct _ftp_flags flags)
+ftpListParseParts(const char *buf, struct Ftp::GatewayFlags flags)
 {
     ftpListParts *p = NULL;
     char *t = NULL;
@@ -944,7 +770,7 @@
 }
 
 MemBuf *
-FtpStateData::htmlifyListEntry(const char *line)
+Ftp::Gateway::htmlifyListEntry(const char *line)
 {
     char icon[2048];
     char href[2048 + 40];
@@ -1081,7 +907,7 @@
 }
 
 void
-FtpStateData::parseListing()
+Ftp::Gateway::parseListing()
 {
     char *buf = data.readBuf->content();
     char *sbuf;			/* NULL-terminated copy of termedBuf */
@@ -1163,150 +989,10 @@
     xfree(sbuf);
 }
 
-const Comm::ConnectionPointer &
-FtpStateData::dataConnection() const
-{
-    return data.conn;
-}
-
-void
-FtpStateData::dataComplete()
-{
-    debugs(9, 3,HERE);
-
-    /* Connection closed; transfer done. */
-
-    /// Close data channel, if any, to conserve resources while we wait.
-    data.close();
-
-    /* expect the "transfer complete" message on the control socket */
-    /*
-     * DPW 2007-04-23
-     * Previously, this was the only place where we set the
-     * 'buffered_ok' flag when calling scheduleReadControlReply().
-     * It caused some problems if the FTP server returns an unexpected
-     * status code after the data command.  FtpStateData was being
-     * deleted in the middle of dataRead().
-     */
-    /* AYJ: 2011-01-13: Bug 2581.
-     * 226 status is possibly waiting in the ctrl buffer.
-     * The connection will hang if we DONT send buffered_ok.
-     * This happens on all transfers which can be completly sent by the
-     * server before the 150 started status message is read in by Squid.
-     * ie all transfers of about one packet hang.
-     */
-    scheduleReadControlReply(1);
-}
-
-void
-FtpStateData::maybeReadVirginBody()
-{
-    // too late to read
-    if (!Comm::IsConnOpen(data.conn) || fd_table[data.conn->fd].closing())
-        return;
-
-    if (data.read_pending)
-        return;
-
-    const int read_sz = replyBodySpace(*data.readBuf, 0);
-
-    debugs(11,9, HERE << "FTP may read up to " << read_sz << " bytes");
-
-    if (read_sz < 2)	// see http.cc
-        return;
-
-    data.read_pending = true;
-
-    typedef CommCbMemFunT<FtpStateData, CommTimeoutCbParams> TimeoutDialer;
-    AsyncCall::Pointer timeoutCall =  JobCallback(9, 5,
-                                      TimeoutDialer, this, FtpStateData::ftpTimeout);
-    commSetConnTimeout(data.conn, Config.Timeout.read, timeoutCall);
-
-    debugs(9,5,HERE << "queueing read on FD " << data.conn->fd);
-
-    typedef CommCbMemFunT<FtpStateData, CommIoCbParams> Dialer;
-    entry->delayAwareRead(data.conn, data.readBuf->space(), read_sz,
-                          JobCallback(9, 5, Dialer, this, FtpStateData::dataRead));
-}
-
-void
-FtpStateData::dataRead(const CommIoCbParams &io)
-{
-    int j;
-    int bin;
-
-    data.read_pending = false;
-
-    debugs(9, 3, HERE << "ftpDataRead: FD " << io.fd << " Read " << io.size << " bytes");
-
-    if (io.size > 0) {
-        kb_incr(&(statCounter.server.all.kbytes_in), io.size);
-        kb_incr(&(statCounter.server.ftp.kbytes_in), io.size);
-    }
-
-    if (io.flag == Comm::ERR_CLOSING)
-        return;
-
-    assert(io.fd == data.conn->fd);
-
-    if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) {
-        abortTransaction("entry aborted during dataRead");
-        return;
-    }
-
-    if (io.flag == Comm::OK && io.size > 0) {
-        debugs(9,5,HERE << "appended " << io.size << " bytes to readBuf");
-        data.readBuf->appended(io.size);
-#if USE_DELAY_POOLS
-        DelayId delayId = entry->mem_obj->mostBytesAllowed();
-        delayId.bytesIn(io.size);
-#endif
-        ++ IOStats.Ftp.reads;
-
-        for (j = io.size - 1, bin = 0; j; ++bin)
-            j >>= 1;
-
-        ++ IOStats.Ftp.read_hist[bin];
-    }
-
-    if (io.flag != Comm::OK) {
-        debugs(50, ignoreErrno(io.xerrno) ? 3 : DBG_IMPORTANT,
-               "ftpDataRead: read error: " << xstrerr(io.xerrno));
-
-        if (ignoreErrno(io.xerrno)) {
-            typedef CommCbMemFunT<FtpStateData, CommTimeoutCbParams> TimeoutDialer;
-            AsyncCall::Pointer timeoutCall = JobCallback(9, 5,
-                                             TimeoutDialer, this, FtpStateData::ftpTimeout);
-            commSetConnTimeout(io.conn, Config.Timeout.read, timeoutCall);
-
-            maybeReadVirginBody();
-        } else {
-            failed(ERR_READ_ERROR, 0);
-            /* failed closes ctrl.conn and frees ftpState */
-            return;
-        }
-    } else if (io.size == 0) {
-        debugs(9,3, HERE << "Calling dataComplete() because io.size == 0");
-        /*
-         * DPW 2007-04-23
-         * Dangerous curves ahead.  This call to dataComplete was
-         * calling scheduleReadControlReply, handleControlReply,
-         * and then ftpReadTransferDone.  If ftpReadTransferDone
-         * gets unexpected status code, it closes down the control
-         * socket and our FtpStateData object gets destroyed.   As
-         * a workaround we no longer set the 'buffered_ok' flag in
-         * the scheduleReadControlReply call.
-         */
-        dataComplete();
-    }
-
-    processReplyBody();
-}
-
-void
-FtpStateData::processReplyBody()
-{
-    debugs(9, 3, HERE << "FtpStateData::processReplyBody starting.");
+void
+Ftp::Gateway::processReplyBody()
+{
+    debugs(9, 3, status());
 
     if (request->method == Http::METHOD_HEAD && (flags.isdir || theSize != -1)) {
         serverComplete();
@@ -1329,7 +1015,7 @@
 #if USE_ADAPTATION
 
     if (adaptationAccessCheckPending) {
-        debugs(9,3, HERE << "returning from FtpStateData::processReplyBody due to adaptationAccessCheckPending");
+        debugs(9, 3, "returning from Ftp::Gateway::processReplyBody due to adaptationAccessCheckPending");
         return;
     }
 
@@ -1371,7 +1057,7 @@
  \retval 0	if something is missing.
  */
 int
-FtpStateData::checkAuth(const HttpHeader * req_hdr)
+Ftp::Gateway::checkAuth(const HttpHeader * req_hdr)
 {
     /* default username */
     xstrncpy(user, "anonymous", MAX_URL);
@@ -1416,7 +1102,7 @@
 
 static String str_type_eq;
 void
-FtpStateData::checkUrlpath()
+Ftp::Gateway::checkUrlpath()
 {
     int l;
     size_t t;
@@ -1454,7 +1140,7 @@
 }
 
 void
-FtpStateData::buildTitleUrl()
+Ftp::Gateway::buildTitleUrl()
 {
     title_url = "ftp://";
 
@@ -1496,15 +1182,8 @@
     base_href.append("/");
 }
 
-/// \ingroup ServerProtocolFTPAPI
-void
-ftpStart(FwdState * fwd)
-{
-    AsyncJob::Start(new FtpStateData(fwd, fwd->serverConnection()));
-}
-
-void
-FtpStateData::start()
+void
+Ftp::Gateway::start()
 {
     if (!checkAuth(&request->header)) {
         /* create appropriate reply */
@@ -1518,334 +1197,34 @@
     buildTitleUrl();
     debugs(9, 5, HERE << "FD " << ctrl.conn->fd << " : host=" << request->GetHost() <<
            ", path=" << request->urlpath << ", user=" << user << ", passwd=" << password);
-
     state = BEGIN;
-    ctrl.last_command = xstrdup("Connect to server");
-    ctrl.buf = (char *)memAllocBuf(4096, &ctrl.size);
-    ctrl.offset = 0;
-    data.readBuf = new MemBuf;
-    data.readBuf->init(4096, SQUID_TCP_SO_RCVBUF);
-    scheduleReadControlReply(0);
+    Ftp::Client::start();
 }
 
 /* ====================================================================== */
 
-/// \ingroup ServerProtocolFTPInternal
-static char *
-escapeIAC(const char *buf)
-{
-    int n;
-    char *ret;
-    unsigned const char *p;
-    unsigned char *r;
-
-    for (p = (unsigned const char *)buf, n = 1; *p; ++n, ++p)
-        if (*p == 255)
-            ++n;
-
-    ret = (char *)xmalloc(n);
-
-    for (p = (unsigned const char *)buf, r=(unsigned char *)ret; *p; ++p) {
-        *r = *p;
-        ++r;
-
-        if (*p == 255) {
-            *r = 255;
-            ++r;
-        }
-    }
-
-    *r = '\0';
-    ++r;
-    assert((r - (unsigned char *)ret) == n );
-    return ret;
-}
-
-void
-FtpStateData::writeCommand(const char *buf)
-{
-    char *ebuf;
-    /* trace FTP protocol communications at level 2 */
-    debugs(9, 2, "ftp<< " << buf);
-
-    if (Config.Ftp.telnet)
-        ebuf = escapeIAC(buf);
-    else
-        ebuf = xstrdup(buf);
-
-    safe_free(ctrl.last_command);
-
-    safe_free(ctrl.last_reply);
-
-    ctrl.last_command = ebuf;
-
-    if (!Comm::IsConnOpen(ctrl.conn)) {
-        debugs(9, 2, HERE << "cannot send to closing ctrl " << ctrl.conn);
-        // TODO: assert(ctrl.closer != NULL);
-        return;
-    }
-
-    typedef CommCbMemFunT<FtpStateData, CommIoCbParams> Dialer;
-    AsyncCall::Pointer call = JobCallback(9, 5, Dialer, this, FtpStateData::ftpWriteCommandCallback);
-    Comm::Write(ctrl.conn, ctrl.last_command, strlen(ctrl.last_command), call, NULL);
-
-    scheduleReadControlReply(0);
-}
-
-void
-FtpStateData::ftpWriteCommandCallback(const CommIoCbParams &io)
-{
-
-    debugs(9, 5, "ftpWriteCommandCallback: wrote " << io.size << " bytes");
-
-    if (io.size > 0) {
-        fd_bytes(io.fd, io.size, FD_WRITE);
-        kb_incr(&(statCounter.server.all.kbytes_out), io.size);
-        kb_incr(&(statCounter.server.ftp.kbytes_out), io.size);
-    }
-
-    if (io.flag == Comm::ERR_CLOSING)
-        return;
-
-    if (io.flag) {
-        debugs(9, DBG_IMPORTANT, "ftpWriteCommandCallback: " << io.conn << ": " << xstrerr(io.xerrno));
-        failed(ERR_WRITE_ERROR, io.xerrno);
-        /* failed closes ctrl.conn and frees ftpState */
-        return;
-    }
-}
-
-wordlist *
-FtpStateData::ftpParseControlReply(char *buf, size_t len, int *codep, size_t *used)
-{
-    char *s;
-    char *sbuf;
-    char *end;
-    int usable;
-    int complete = 0;
-    wordlist *head = NULL;
-    wordlist *list;
-    wordlist **tail = &head;
-    size_t offset;
-    size_t linelen;
-    int code = -1;
-    debugs(9, 3, HERE);
-    /*
-     * We need a NULL-terminated buffer for scanning, ick
-     */
-    sbuf = (char *)xmalloc(len + 1);
-    xstrncpy(sbuf, buf, len + 1);
-    end = sbuf + len - 1;
-
-    while (*end != '\r' && *end != '\n' && end > sbuf)
-        --end;
-
-    usable = end - sbuf;
-
-    debugs(9, 3, HERE << "usable = " << usable);
-
-    if (usable == 0) {
-        debugs(9, 3, HERE << "didn't find end of line");
-        safe_free(sbuf);
-        return NULL;
-    }
-
-    debugs(9, 3, HERE << len << " bytes to play with");
-    ++end;
-    s = sbuf;
-    s += strspn(s, crlf);
-
-    for (; s < end; s += strcspn(s, crlf), s += strspn(s, crlf)) {
-        if (complete)
-            break;
-
-        debugs(9, 5, HERE << "s = {" << s << "}");
-
-        linelen = strcspn(s, crlf) + 1;
-
-        if (linelen < 2)
-            break;
-
-        if (linelen > 3)
-            complete = (*s >= '0' && *s <= '9' && *(s + 3) == ' ');
-
-        if (complete)
-            code = atoi(s);
-
-        offset = 0;
-
-        if (linelen > 3)
-            if (*s >= '0' && *s <= '9' && (*(s + 3) == '-' || *(s + 3) == ' '))
-                offset = 4;
-
-        list = new wordlist();
-
-        list->key = (char *)xmalloc(linelen - offset);
-
-        xstrncpy(list->key, s + offset, linelen - offset);
-
-        /* trace the FTP communication chat at level 2 */
-        debugs(9, 2, "ftp>> " << code << " " << list->key);
-
-        *tail = list;
-
-        tail = &list->next;
-    }
-
-    *used = (size_t) (s - sbuf);
-    safe_free(sbuf);
-
-    if (!complete)
-        wordlistDestroy(&head);
-
-    if (codep)
-        *codep = code;
-
-    return head;
-}
-
-/**
- * DPW 2007-04-23
- * Looks like there are no longer anymore callers that set
- * buffered_ok=1.  Perhaps it can be removed at some point.
- */
-void
-FtpStateData::scheduleReadControlReply(int buffered_ok)
-{
-    debugs(9, 3, HERE << ctrl.conn);
-
-    if (buffered_ok && ctrl.offset > 0) {
-        /* We've already read some reply data */
-        handleControlReply();
-    } else {
-        /*
-         * Cancel the timeout on the Data socket (if any) and
-         * establish one on the control socket.
-         */
-        if (Comm::IsConnOpen(data.conn)) {
-            commUnsetConnTimeout(data.conn);
-        }
-
-        typedef CommCbMemFunT<FtpStateData, CommTimeoutCbParams> TimeoutDialer;
-        AsyncCall::Pointer timeoutCall = JobCallback(9, 5, TimeoutDialer, this, FtpStateData::ftpTimeout);
-        commSetConnTimeout(ctrl.conn, Config.Timeout.read, timeoutCall);
-
-        typedef CommCbMemFunT<FtpStateData, CommIoCbParams> Dialer;
-        AsyncCall::Pointer reader = JobCallback(9, 5, Dialer, this, FtpStateData::ftpReadControlReply);
-        comm_read(ctrl.conn, ctrl.buf + ctrl.offset, ctrl.size - ctrl.offset, reader);
-    }
-}
-
-void FtpStateData::ftpReadControlReply(const CommIoCbParams &io)
-{
-    debugs(9, 3, "ftpReadControlReply: FD " << io.fd << ", Read " << io.size << " bytes");
-
-    if (io.size > 0) {
-        kb_incr(&(statCounter.server.all.kbytes_in), io.size);
-        kb_incr(&(statCounter.server.ftp.kbytes_in), io.size);
-    }
-
-    if (io.flag == Comm::ERR_CLOSING)
-        return;
-
-    if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) {
-        abortTransaction("entry aborted during control reply read");
-        return;
-    }
-
-    assert(ctrl.offset < ctrl.size);
-
-    if (io.flag == Comm::OK && io.size > 0) {
-        fd_bytes(io.fd, io.size, FD_READ);
-    }
-
-    if (io.flag != Comm::OK) {
-        debugs(50, ignoreErrno(io.xerrno) ? 3 : DBG_IMPORTANT,
-               "ftpReadControlReply: read error: " << xstrerr(io.xerrno));
-
-        if (ignoreErrno(io.xerrno)) {
-            scheduleReadControlReply(0);
-        } else {
-            failed(ERR_READ_ERROR, io.xerrno);
-            /* failed closes ctrl.conn and frees ftpState */
-        }
-        return;
-    }
-
-    if (io.size == 0) {
-        if (entry->store_status == STORE_PENDING) {
-            failed(ERR_FTP_FAILURE, 0);
-            /* failed closes ctrl.conn and frees ftpState */
-            return;
-        }
-
-        /* XXX this may end up having to be serverComplete() .. */
-        abortTransaction("zero control reply read");
-        return;
-    }
-
-    unsigned int len =io.size + ctrl.offset;
-    ctrl.offset = len;
-    assert(len <= ctrl.size);
-    handleControlReply();
-}
-
-void
-FtpStateData::handleControlReply()
-{
-    wordlist **W;
-    size_t bytes_used = 0;
-    wordlistDestroy(&ctrl.message);
-    ctrl.message = ftpParseControlReply(ctrl.buf,
-                                        ctrl.offset, &ctrl.replycode, &bytes_used);
-
-    if (ctrl.message == NULL) {
-        /* didn't get complete reply yet */
-
-        if (ctrl.offset == ctrl.size) {
-            ctrl.buf = (char *)memReallocBuf(ctrl.buf, ctrl.size << 1, &ctrl.size);
-        }
-
-        scheduleReadControlReply(0);
-        return;
-    } else if (ctrl.offset == bytes_used) {
-        /* used it all up */
-        ctrl.offset = 0;
-    } else {
-        /* Got some data past the complete reply */
-        assert(bytes_used < ctrl.offset);
-        ctrl.offset -= bytes_used;
-        memmove(ctrl.buf, ctrl.buf + bytes_used, ctrl.offset);
-    }
-
-    /* Move the last line of the reply message to ctrl.last_reply */
-    for (W = &ctrl.message; (*W)->next; W = &(*W)->next);
-    safe_free(ctrl.last_reply);
-
-    ctrl.last_reply = xstrdup((*W)->key);
-
-    wordlistDestroy(W);
-
-    /* Copy the rest of the message to cwd_message to be printed in
-     * error messages
-     */
-    if (ctrl.message) {
-        for (wordlist *w = ctrl.message; w; w = w->next) {
-            cwd_message.append('\n');
-            cwd_message.append(w->key);
-        }
-    }
-
-    debugs(9, 3, HERE << "state=" << state << ", code=" << ctrl.replycode);
+void
+Ftp::Gateway::handleControlReply()
+{
+    Ftp::Client::handleControlReply();
+    if (ctrl.message == NULL)
+        return; // didn't get complete reply yet
+
+    /* Copy the message except for the last line to cwd_message to be
+     * printed in error messages.
+     */
+    for (wordlist *w = ctrl.message; w && w->next; w = w->next) {
+        cwd_message.append('\n');
+        cwd_message.append(w->key);
+    }
 
     FTP_SM_FUNCS[state] (this);
 }
 
 /* ====================================================================== */
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadWelcome(FtpStateData * ftpState)
+ftpReadWelcome(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -1876,7 +1255,7 @@
  * its NOT a general failure. But a correct FTP response type.
  */
 void
-FtpStateData::loginFailed()
+Ftp::Gateway::loginFailed()
 {
     ErrorState *err = NULL;
     const char *command, *reply;
@@ -1940,7 +1319,7 @@
 }
 
 const char *
-FtpStateData::ftpRealm()
+Ftp::Gateway::ftpRealm()
 {
     static char realm[8192];
 
@@ -1955,9 +1334,8 @@
     return realm;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendUser(FtpStateData * ftpState)
+ftpSendUser(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendUser"))
@@ -1972,12 +1350,11 @@
 
     ftpState->writeCommand(cbuf);
 
-    ftpState->state = SENT_USER;
+    ftpState->state = Ftp::Client::SENT_USER;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadUser(FtpStateData * ftpState)
+ftpReadUser(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -1991,9 +1368,8 @@
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendPass(FtpStateData * ftpState)
+ftpSendPass(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendPass"))
@@ -2001,12 +1377,11 @@
 
     snprintf(cbuf, CTRL_BUFLEN, "PASS %s\r\n", ftpState->password);
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_PASS;
+    ftpState->state = Ftp::Client::SENT_PASS;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadPass(FtpStateData * ftpState)
+ftpReadPass(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE << "code=" << code);
@@ -2018,9 +1393,8 @@
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendType(FtpStateData * ftpState)
+ftpSendType(Ftp::Gateway * ftpState)
 {
     const char *t;
     const char *filename;
@@ -2068,12 +1442,11 @@
 
     ftpState->writeCommand(cbuf);
 
-    ftpState->state = SENT_TYPE;
+    ftpState->state = Ftp::Client::SENT_TYPE;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadType(FtpStateData * ftpState)
+ftpReadType(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     char *path;
@@ -2112,9 +1485,8 @@
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpTraverseDirectory(FtpStateData * ftpState)
+ftpTraverseDirectory(Ftp::Gateway * ftpState)
 {
     wordlist *w;
     debugs(9, 4, HERE << (ftpState->filepath ? ftpState->filepath : "<NULL>"));
@@ -2150,9 +1522,8 @@
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendCwd(FtpStateData * ftpState)
+ftpSendCwd(Ftp::Gateway * ftpState)
 {
     char *path = NULL;
 
@@ -2174,12 +1545,11 @@
 
     ftpState->writeCommand(cbuf);
 
-    ftpState->state = SENT_CWD;
+    ftpState->state = Ftp::Client::SENT_CWD;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadCwd(FtpStateData * ftpState)
+ftpReadCwd(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -2208,9 +1578,8 @@
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendMkdir(FtpStateData * ftpState)
+ftpSendMkdir(Ftp::Gateway * ftpState)
 {
     char *path = NULL;
 
@@ -2222,12 +1591,11 @@
     debugs(9, 3, HERE << "with path=" << path);
     snprintf(cbuf, CTRL_BUFLEN, "MKD %s\r\n", path);
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_MKDIR;
+    ftpState->state = Ftp::Client::SENT_MKDIR;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadMkdir(FtpStateData * ftpState)
+ftpReadMkdir(Ftp::Gateway * ftpState)
 {
     char *path = ftpState->filepath;
     int code = ftpState->ctrl.replycode;
@@ -2247,18 +1615,16 @@
         ftpSendReply(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpGetFile(FtpStateData * ftpState)
+ftpGetFile(Ftp::Gateway * ftpState)
 {
     assert(*ftpState->filepath != '\0');
     ftpState->flags.isdir = 0;
     ftpSendMdtm(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpListDir(FtpStateData * ftpState)
+ftpListDir(Ftp::Gateway * ftpState)
 {
     if (ftpState->flags.dir_slash) {
         debugs(9, 3, HERE << "Directory path did not end in /");
@@ -2269,9 +1635,8 @@
     ftpSendPassive(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendMdtm(FtpStateData * ftpState)
+ftpSendMdtm(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendMdtm"))
@@ -2280,12 +1645,11 @@
     assert(*ftpState->filepath != '\0');
     snprintf(cbuf, CTRL_BUFLEN, "MDTM %s\r\n", ftpState->filepath);
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_MDTM;
+    ftpState->state = Ftp::Client::SENT_MDTM;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadMdtm(FtpStateData * ftpState)
+ftpReadMdtm(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -2301,9 +1665,8 @@
     ftpSendSize(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendSize(FtpStateData * ftpState)
+ftpSendSize(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendSize"))
@@ -2317,15 +1680,14 @@
         assert(*ftpState->filepath != '\0');
         snprintf(cbuf, CTRL_BUFLEN, "SIZE %s\r\n", ftpState->filepath);
         ftpState->writeCommand(cbuf);
-        ftpState->state = SENT_SIZE;
+        ftpState->state = Ftp::Client::SENT_SIZE;
     } else
         /* Skip to next state no non-binary transfers */
         ftpSendPassive(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadSize(FtpStateData * ftpState)
+ftpReadSize(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -2348,148 +1710,24 @@
     ftpSendPassive(ftpState);
 }
 
-/**
- \ingroup ServerProtocolFTPInternal
- */
 static void
-ftpReadEPSV(FtpStateData* ftpState)
+ftpReadEPSV(Ftp::Gateway* ftpState)
 {
-    int code = ftpState->ctrl.replycode;
-    Ip::Address ipa_remote;
-    char *buf;
-    debugs(9, 3, HERE);
-
-    if (code != 229 && code != 522) {
-        if (code == 200) {
-            /* handle broken servers (RFC 2428 says OK code for EPSV MUST be 229 not 200) */
-            /* vsftpd for one send '200 EPSV ALL ok.' without even port info.
-             * Its okay to re-send EPSV 1/2 but nothing else. */
-            debugs(9, DBG_IMPORTANT, "Broken FTP Server at " << ftpState->ctrl.conn->remote << ". Wrong accept code for EPSV");
-        } else {
-            debugs(9, 2, "EPSV not supported by remote end");
-            ftpState->state = SENT_EPSV_1; /* simulate having failed EPSV 1 (last EPSV to try before shifting to PASV) */
-        }
-        ftpSendPassive(ftpState);
-        return;
-    }
-
-    if (code == 522) {
-        /* server response with list of supported methods   */
-        /*   522 Network protocol not supported, use (1)    */
-        /*   522 Network protocol not supported, use (1,2)  */
-        /*   522 Network protocol not supported, use (2)  */
-        /* TODO: handle the (1,2) case. We might get it back after EPSV ALL
-         * which means close data + control without self-destructing and re-open from scratch. */
-        debugs(9, 5, HERE << "scanning: " << ftpState->ctrl.last_reply);
-        buf = ftpState->ctrl.last_reply;
-        while (buf != NULL && *buf != '\0' && *buf != '\n' && *buf != '(')
-            ++buf;
-        if (buf != NULL && *buf == '\n')
-            ++buf;
-
-        if (buf == NULL || *buf == '\0') {
-            /* handle broken server (RFC 2428 says MUST specify supported protocols in 522) */
-            debugs(9, DBG_IMPORTANT, "Broken FTP Server at " << ftpState->ctrl.conn->remote << ". 522 error missing protocol negotiation hints");
-            ftpSendPassive(ftpState);
-        } else if (strcmp(buf, "(1)") == 0) {
-            ftpState->state = SENT_EPSV_2; /* simulate having sent and failed EPSV 2 */
-            ftpSendPassive(ftpState);
-        } else if (strcmp(buf, "(2)") == 0) {
-            if (Ip::EnableIpv6) {
-                /* If server only supports EPSV 2 and we have already tried that. Go straight to EPRT */
-                if (ftpState->state == SENT_EPSV_2) {
-                    ftpSendEPRT(ftpState);
-                } else {
-                    /* or try the next Passive mode down the chain. */
-                    ftpSendPassive(ftpState);
-                }
-            } else {
-                /* Server only accept EPSV in IPv6 traffic. */
-                ftpState->state = SENT_EPSV_1; /* simulate having sent and failed EPSV 1 */
-                ftpSendPassive(ftpState);
-            }
-        } else {
-            /* handle broken server (RFC 2428 says MUST specify supported protocols in 522) */
-            debugs(9, DBG_IMPORTANT, "WARNING: Server at " << ftpState->ctrl.conn->remote << " sent unknown protocol negotiation hint: " << buf);
-            ftpSendPassive(ftpState);
-        }
-        return;
-    }
-
-    /*  229 Entering Extended Passive Mode (|||port|) */
-    /*  ANSI sez [^0-9] is undefined, it breaks on Watcom cc */
-    debugs(9, 5, "scanning: " << ftpState->ctrl.last_reply);
-
-    buf = ftpState->ctrl.last_reply + strcspn(ftpState->ctrl.last_reply, "(");
-
-    char h1, h2, h3, h4;
-    unsigned short port;
-    int n = sscanf(buf, "(%c%c%c%hu%c)", &h1, &h2, &h3, &port, &h4);
-
-    if (n < 4 || h1 != h2 || h1 != h3 || h1 != h4) {
-        debugs(9, DBG_IMPORTANT, "Invalid EPSV reply from " <<
-               ftpState->ctrl.conn->remote << ": " <<
-               ftpState->ctrl.last_reply);
-
-        ftpSendPassive(ftpState);
-        return;
-    }
-
-    if (0 == port) {
-        debugs(9, DBG_IMPORTANT, "Unsafe EPSV reply from " <<
-               ftpState->ctrl.conn->remote << ": " <<
-               ftpState->ctrl.last_reply);
-
-        ftpSendPassive(ftpState);
-        return;
-    }
-
-    if (Config.Ftp.sanitycheck) {
-        if (port < 1024) {
-            debugs(9, DBG_IMPORTANT, "Unsafe EPSV reply from " <<
-                   ftpState->ctrl.conn->remote << ": " <<
-                   ftpState->ctrl.last_reply);
-
-            ftpSendPassive(ftpState);
-            return;
-        }
-    }
-
-    ftpState->data.port = port;
-
-    safe_free(ftpState->data.host);
-    ftpState->data.host = xstrdup(fd_table[ftpState->ctrl.conn->fd].ipaddr);
-
-    safe_free(ftpState->ctrl.last_command);
-
-    safe_free(ftpState->ctrl.last_reply);
-
-    ftpState->ctrl.last_command = xstrdup("Connect to server data port");
-
-    // Generate a new data channel descriptor to be opened.
-    Comm::ConnectionPointer conn = new Comm::Connection;
-    conn->setAddrs(ftpState->ctrl.conn->local, ftpState->ctrl.conn->remote);
-    conn->local.port(0);
-    conn->remote.port(port);
-    conn->tos = ftpState->ctrl.conn->tos;
-    conn->nfmark = ftpState->ctrl.conn->nfmark;
-
-    debugs(9, 3, HERE << "connecting to " << conn->remote);
-
-    ftpState->data.opener = commCbCall(9,3, "FtpStateData::ftpPasvCallback", CommConnectCbPtrFun(FtpStateData::ftpPasvCallback, ftpState));
-    Comm::ConnOpener *cs = new Comm::ConnOpener(conn, ftpState->data.opener, Config.Timeout.connect);
-    cs->setHost(ftpState->data.host);
-    AsyncJob::Start(cs);
+    Ip::Address srvAddr; // unused
+    if (ftpState->handleEpsvReply(srvAddr)) {
+        if (ftpState->ctrl.message == NULL)
+            return; // didn't get complete reply yet
+
+        ftpState->connectDataChannel();
+    }
 }
 
-/** \ingroup ServerProtocolFTPInternal
- *
- * Send Passive connection request.
+/** Send Passive connection request.
  * Default method is to use modern EPSV request.
  * The failover mechanism should check for previous state and re-call with alternates on failure.
  */
 static void
-ftpSendPassive(FtpStateData * ftpState)
+ftpSendPassive(Ftp::Gateway * ftpState)
 {
     /** Checks the server control channel is still available before running. */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendPassive"))
@@ -2498,118 +1736,21 @@
     debugs(9, 3, HERE);
 
     /** \par
-      * Checks for EPSV ALL special conditions:
-      * If enabled to be sent, squid MUST NOT request any other connect methods.
-      * If 'ALL' is sent and fails the entire FTP Session fails.
-      * NP: By my reading exact EPSV protocols maybe attempted, but only EPSV method. */
-    if (Config.Ftp.epsv_all && ftpState->flags.epsv_all_sent && ftpState->state == SENT_EPSV_1 ) {
-        debugs(9, DBG_IMPORTANT, "FTP does not allow PASV method after 'EPSV ALL' has been sent.");
-        ftpFail(ftpState);
-        return;
-    }
-
-    /** \par
-      * Checks for 'HEAD' method request and passes off for special handling by FtpStateData::processHeadResponse(). */
+      * Checks for 'HEAD' method request and passes off for special handling by Ftp::Gateway::processHeadResponse(). */
     if (ftpState->request->method == Http::METHOD_HEAD && (ftpState->flags.isdir || ftpState->theSize != -1)) {
         ftpState->processHeadResponse(); // may call serverComplete
         return;
     }
 
-    /// Closes any old FTP-Data connection which may exist. */
-    ftpState->data.close();
-
-    /** \par
-      * Checks for previous EPSV/PASV failures on this server/session.
-      * Diverts to EPRT immediately if they are not working. */
-    if (!ftpState->flags.pasv_supported) {
-        ftpSendEPRT(ftpState);
-        return;
-    }
-
-    /** \par
-      * Send EPSV (ALL,2,1) or PASV on the control channel.
-      *
-      *  - EPSV ALL  is used if enabled.
-      *  - EPSV 2    is used if ALL is disabled and IPv6 is available and ctrl channel is IPv6.
-      *  - EPSV 1    is used if EPSV 2 (IPv6) fails or is not available or ctrl channel is IPv4.
-      *  - PASV      is used if EPSV 1 fails.
-      */
-    switch (ftpState->state) {
-    case SENT_EPSV_ALL: /* EPSV ALL resulted in a bad response. Try ther EPSV methods. */
-        ftpState->flags.epsv_all_sent = true;
-        if (ftpState->ctrl.conn->local.isIPv6()) {
-            debugs(9, 5, HERE << "FTP Channel is IPv6 (" << ftpState->ctrl.conn->remote << ") attempting EPSV 2 after EPSV ALL has failed.");
-            snprintf(cbuf, CTRL_BUFLEN, "EPSV 2\r\n");
-            ftpState->state = SENT_EPSV_2;
-            break;
-        }
-        // else fall through to skip EPSV 2
-
-    case SENT_EPSV_2: /* EPSV IPv6 failed. Try EPSV IPv4 */
-        if (ftpState->ctrl.conn->local.isIPv4()) {
-            debugs(9, 5, HERE << "FTP Channel is IPv4 (" << ftpState->ctrl.conn->remote << ") attempting EPSV 1 after EPSV ALL has failed.");
-            snprintf(cbuf, CTRL_BUFLEN, "EPSV 1\r\n");
-            ftpState->state = SENT_EPSV_1;
-            break;
-        } else if (ftpState->flags.epsv_all_sent) {
-            debugs(9, DBG_IMPORTANT, "FTP does not allow PASV method after 'EPSV ALL' has been sent.");
-            ftpFail(ftpState);
-            return;
-        }
-        // else fall through to skip EPSV 1
-
-    case SENT_EPSV_1: /* EPSV options exhausted. Try PASV now. */
-        debugs(9, 5, HERE << "FTP Channel (" << ftpState->ctrl.conn->remote << ") rejects EPSV connection attempts. Trying PASV instead.");
-        snprintf(cbuf, CTRL_BUFLEN, "PASV\r\n");
-        ftpState->state = SENT_PASV;
-        break;
-
-    default: {
-        bool doEpsv = true;
-        if (Config.accessList.ftp_epsv) {
-            ACLFilledChecklist checklist(Config.accessList.ftp_epsv, ftpState->fwd->request, NULL);
-            doEpsv = (checklist.fastCheck() == ACCESS_ALLOWED);
-        }
-        if (!doEpsv) {
-            debugs(9, 5, HERE << "EPSV support manually disabled. Sending PASV for FTP Channel (" << ftpState->ctrl.conn->remote <<")");
-            snprintf(cbuf, CTRL_BUFLEN, "PASV\r\n");
-            ftpState->state = SENT_PASV;
-        } else if (Config.Ftp.epsv_all) {
-            debugs(9, 5, HERE << "EPSV ALL manually enabled. Attempting with FTP Channel (" << ftpState->ctrl.conn->remote <<")");
-            snprintf(cbuf, CTRL_BUFLEN, "EPSV ALL\r\n");
-            ftpState->state = SENT_EPSV_ALL;
-            /* block other non-EPSV connections being attempted */
+    if (ftpState->sendPassive()) {
+        // SENT_EPSV_ALL blocks other non-EPSV connections being attempted
+        if (ftpState->state == Ftp::Client::SENT_EPSV_ALL)
             ftpState->flags.epsv_all_sent = true;
-        } else {
-            if (ftpState->ctrl.conn->local.isIPv6()) {
-                debugs(9, 5, HERE << "FTP Channel (" << ftpState->ctrl.conn->remote << "). Sending default EPSV 2");
-                snprintf(cbuf, CTRL_BUFLEN, "EPSV 2\r\n");
-                ftpState->state = SENT_EPSV_2;
-            }
-            if (ftpState->ctrl.conn->local.isIPv4()) {
-                debugs(9, 5, HERE << "Channel (" << ftpState->ctrl.conn->remote <<"). Sending default EPSV 1");
-                snprintf(cbuf, CTRL_BUFLEN, "EPSV 1\r\n");
-                ftpState->state = SENT_EPSV_1;
-            }
-        }
-    }
-    break;
-    }
-
-    ftpState->writeCommand(cbuf);
-
-    /*
-     * ugly hack for ftp servers like ftp.netscape.com that sometimes
-     * dont acknowledge PASV commands. Use connect timeout to be faster then read timeout (minutes).
-     */
-    typedef CommCbMemFunT<FtpStateData, CommTimeoutCbParams> TimeoutDialer;
-    AsyncCall::Pointer timeoutCall =  JobCallback(9, 5,
-                                      TimeoutDialer, ftpState, FtpStateData::ftpTimeout);
-    commSetConnTimeout(ftpState->ctrl.conn, Config.Timeout.connect, timeoutCall);
+    }
 }
 
 void
-FtpStateData::processHeadResponse()
+Ftp::Gateway::processHeadResponse()
 {
     debugs(9, 5, HERE << "handling HEAD response");
     ftpSendQuit(this);
@@ -2617,7 +1758,7 @@
 
     /*
      * On rare occasions I'm seeing the entry get aborted after
-     * ftpReadControlReply() and before here, probably when
+     * readControlReply() and before here, probably when
      * trying to write to the client.
      */
     if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) {
@@ -2636,133 +1777,42 @@
     processReplyBody();
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadPasv(FtpStateData * ftpState)
+ftpReadPasv(Ftp::Gateway * ftpState)
 {
-    int code = ftpState->ctrl.replycode;
-    int h1, h2, h3, h4;
-    int p1, p2;
-    int n;
-    unsigned short port;
-    Ip::Address ipa_remote;
-    char *buf;
-    LOCAL_ARRAY(char, ipaddr, 1024);
-    debugs(9, 3, HERE);
-
-    if (code != 227) {
-        debugs(9, 2, "PASV not supported by remote end");
-        ftpSendEPRT(ftpState);
-        return;
-    }
-
-    /*  227 Entering Passive Mode (h1,h2,h3,h4,p1,p2).  */
-    /*  ANSI sez [^0-9] is undefined, it breaks on Watcom cc */
-    debugs(9, 5, HERE << "scanning: " << ftpState->ctrl.last_reply);
-
-    buf = ftpState->ctrl.last_reply + strcspn(ftpState->ctrl.last_reply, "0123456789");
-
-    n = sscanf(buf, "%d,%d,%d,%d,%d,%d", &h1, &h2, &h3, &h4, &p1, &p2);
-
-    if (n != 6 || p1 < 0 || p2 < 0 || p1 > 255 || p2 > 255) {
-        debugs(9, DBG_IMPORTANT, "Unsafe PASV reply from " <<
-               ftpState->ctrl.conn->remote << ": " <<
-               ftpState->ctrl.last_reply);
-
-        ftpSendEPRT(ftpState);
-        return;
-    }
-
-    snprintf(ipaddr, 1024, "%d.%d.%d.%d", h1, h2, h3, h4);
-
-    ipa_remote = ipaddr;
-
-    if ( ipa_remote.isAnyAddr() ) {
-        debugs(9, DBG_IMPORTANT, "Unsafe PASV reply from " <<
-               ftpState->ctrl.conn->remote << ": " <<
-               ftpState->ctrl.last_reply);
-
-        ftpSendEPRT(ftpState);
-        return;
-    }
-
-    port = ((p1 << 8) + p2);
-
-    if (0 == port) {
-        debugs(9, DBG_IMPORTANT, "Unsafe PASV reply from " <<
-               ftpState->ctrl.conn->remote << ": " <<
-               ftpState->ctrl.last_reply);
-
-        ftpSendEPRT(ftpState);
-        return;
-    }
-
-    if (Config.Ftp.sanitycheck) {
-        if (port < 1024) {
-            debugs(9, DBG_IMPORTANT, "Unsafe PASV reply from " <<
-                   ftpState->ctrl.conn->remote << ": " <<
-                   ftpState->ctrl.last_reply);
-
-            ftpSendEPRT(ftpState);
-            return;
-        }
-    }
-
-    ftpState->data.port = port;
-
-    safe_free(ftpState->data.host);
-    if (Config.Ftp.sanitycheck)
-        ftpState->data.host = xstrdup(fd_table[ftpState->ctrl.conn->fd].ipaddr);
-    else
-        ftpState->data.host = xstrdup(ipaddr);
-
-    safe_free(ftpState->ctrl.last_command);
-
-    safe_free(ftpState->ctrl.last_reply);
-
-    ftpState->ctrl.last_command = xstrdup("Connect to server data port");
-
-    Comm::ConnectionPointer conn = new Comm::Connection;
-    conn->setAddrs(ftpState->ctrl.conn->local, ipaddr);
-    conn->local.port(0);
-    conn->remote.port(port);
-    conn->tos = ftpState->ctrl.conn->tos;
-    conn->nfmark = ftpState->ctrl.conn->nfmark;
-
-    debugs(9, 3, HERE << "connecting to " << conn->remote);
-
-    ftpState->data.opener = commCbCall(9,3, "FtpStateData::ftpPasvCallback", CommConnectCbPtrFun(FtpStateData::ftpPasvCallback, ftpState));
-    Comm::ConnOpener *cs = new Comm::ConnOpener(conn, ftpState->data.opener, Config.Timeout.connect);
-    cs->setHost(ftpState->data.host);
-    AsyncJob::Start(cs);
+    Ip::Address srvAddr; // unused
+    if (ftpState->handlePasvReply(srvAddr))
+        ftpState->connectDataChannel();
+    else {
+        ftpSendEPRT(ftpState);
+        return;
+    }
 }
 
 void
-FtpStateData::ftpPasvCallback(const Comm::ConnectionPointer &conn, Comm::Flag status, int xerrno, void *data)
+Ftp::Gateway::dataChannelConnected(const CommConnectCbParams &io)
 {
-    FtpStateData *ftpState = (FtpStateData *)data;
     debugs(9, 3, HERE);
-    ftpState->data.opener = NULL;
+    data.opener = NULL;
 
-    if (status != Comm::OK) {
+    if (io.flag != Comm::OK) {
         debugs(9, 2, HERE << "Failed to connect. Retrying via another method.");
 
         // ABORT on timeouts. server may be waiting on a broken TCP link.
-        if (status == Comm::TIMEOUT)
-            ftpState->writeCommand("ABOR");
+        if (io.xerrno == Comm::TIMEOUT)
+            writeCommand("ABOR");
 
         // try another connection attempt with some other method
-        ftpSendPassive(ftpState);
+        ftpSendPassive(this);
         return;
     }
 
-    ftpState->data.opened(conn, ftpState->dataCloser());
-    ftpRestOrList(ftpState);
+    data.opened(io.conn, dataCloser());
+    ftpRestOrList(this);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpOpenListenSocket(FtpStateData * ftpState, int fallback)
+ftpOpenListenSocket(Ftp::Gateway * ftpState, int fallback)
 {
     /// Close old data channels, if any. We may open a new one below.
     if (ftpState->data.conn != NULL) {
@@ -2795,12 +1845,11 @@
         temp->local.port(0);
     }
 
-    ftpState->listenForDataChannel(temp, ftpState->entry->url());
+    ftpState->listenForDataChannel(temp);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendPORT(FtpStateData * ftpState)
+ftpSendPORT(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendPort"))
@@ -2840,14 +1889,13 @@
              addrptr[0], addrptr[1], addrptr[2], addrptr[3],
              portptr[0], portptr[1]);
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_PORT;
+    ftpState->state = Ftp::Client::SENT_PORT;
 
     Ip::Address::FreeAddrInfo(AI);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadPORT(FtpStateData * ftpState)
+ftpReadPORT(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -2861,9 +1909,8 @@
     ftpRestOrList(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendEPRT(FtpStateData * ftpState)
+ftpSendEPRT(Ftp::Gateway * ftpState)
 {
     if (Config.Ftp.epsv_all && ftpState->flags.epsv_all_sent) {
         debugs(9, DBG_IMPORTANT, "FTP does not allow EPRT method after 'EPSV ALL' has been sent.");
@@ -2898,11 +1945,11 @@
              ftpState->data.listenConn->local.port() );
 
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_EPRT;
+    ftpState->state = Ftp::Client::SENT_EPRT;
 }
 
 static void
-ftpReadEPRT(FtpStateData * ftpState)
+ftpReadEPRT(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -2917,15 +1964,12 @@
     ftpRestOrList(ftpState);
 }
 
-/**
- \ingroup ServerProtocolFTPInternal
- \par
- * "read" handler to accept FTP data connections.
+/** "read" handler to accept FTP data connections.
  *
  \param io    comm accept(2) callback parameters
  */
 void
-FtpStateData::ftpAcceptDataConnection(const CommAcceptCbParams &io)
+Ftp::Gateway::ftpAcceptDataConnection(const CommAcceptCbParams &io)
 {
     debugs(9, 3, HERE);
 
@@ -2982,9 +2026,7 @@
     /** On Comm::OK start using the accepted data socket and discard the temporary listen socket. */
     data.close();
     data.opened(io.conn, dataCloser());
-    static char ntoapeer[MAX_IPSTRLEN];
-    io.conn->remote.toStr(ntoapeer,sizeof(ntoapeer));
-    data.host = xstrdup(ntoapeer);
+    data.addr(io.conn->remote);
 
     debugs(9, 3, HERE << "Connected data socket on " <<
            io.conn << ". FD table says: " <<
@@ -2997,9 +2039,8 @@
     // Ctrl channel operations will determine what happens to this data connection
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpRestOrList(FtpStateData * ftpState)
+ftpRestOrList(Ftp::Gateway * ftpState)
 {
     debugs(9, 3, HERE);
 
@@ -3021,9 +2062,8 @@
         ftpSendRetr(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendStor(FtpStateData * ftpState)
+ftpSendStor(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendStor"))
@@ -3035,27 +2075,26 @@
         /* Plain file upload */
         snprintf(cbuf, CTRL_BUFLEN, "STOR %s\r\n", ftpState->filepath);
         ftpState->writeCommand(cbuf);
-        ftpState->state = SENT_STOR;
+        ftpState->state = Ftp::Client::SENT_STOR;
     } else if (ftpState->request->header.getInt64(HDR_CONTENT_LENGTH) > 0) {
         /* File upload without a filename. use STOU to generate one */
         snprintf(cbuf, CTRL_BUFLEN, "STOU\r\n");
         ftpState->writeCommand(cbuf);
-        ftpState->state = SENT_STOR;
+        ftpState->state = Ftp::Client::SENT_STOR;
     } else {
         /* No file to transfer. Only create directories if needed */
         ftpSendReply(ftpState);
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 /// \deprecated use ftpState->readStor() instead.
 static void
-ftpReadStor(FtpStateData * ftpState)
+ftpReadStor(Ftp::Gateway * ftpState)
 {
     ftpState->readStor();
 }
 
-void FtpStateData::readStor()
+void Ftp::Gateway::readStor()
 {
     int code = ctrl.replycode;
     debugs(9, 3, HERE);
@@ -3083,16 +2122,15 @@
     } else if (code == 150) {
         /* When client code is 150 with no data channel, Accept data channel. */
         debugs(9, 3, "ftpReadStor: accepting data channel");
-        listenForDataChannel(data.conn, data.host);
+        listenForDataChannel(data.conn);
     } else {
         debugs(9, DBG_IMPORTANT, HERE << "Unexpected reply code "<< std::setfill('0') << std::setw(3) << code);
         ftpFail(this);
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendRest(FtpStateData * ftpState)
+ftpSendRest(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendRest"))
@@ -3102,11 +2140,11 @@
 
     snprintf(cbuf, CTRL_BUFLEN, "REST %" PRId64 "\r\n", ftpState->restart_offset);
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_REST;
+    ftpState->state = Ftp::Client::SENT_REST;
 }
 
 int
-FtpStateData::restartable()
+Ftp::Gateway::restartable()
 {
     if (restart_offset > 0)
         return 1;
@@ -3132,9 +2170,8 @@
     return 1;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadRest(FtpStateData * ftpState)
+ftpReadRest(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -3152,9 +2189,8 @@
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendList(FtpStateData * ftpState)
+ftpSendList(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendList"))
@@ -3169,12 +2205,11 @@
     }
 
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_LIST;
+    ftpState->state = Ftp::Client::SENT_LIST;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendNlst(FtpStateData * ftpState)
+ftpSendNlst(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendNlst"))
@@ -3191,12 +2226,11 @@
     }
 
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_NLST;
+    ftpState->state = Ftp::Client::SENT_NLST;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadList(FtpStateData * ftpState)
+ftpReadList(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -3206,12 +2240,12 @@
         debugs(9, 3, HERE << "begin data transfer from " << ftpState->data.conn->remote << " (" << ftpState->data.conn->local << ")");
         ftpState->switchTimeoutToDataChannel();
         ftpState->maybeReadVirginBody();
-        ftpState->state = READING_DATA;
+        ftpState->state = Ftp::Client::READING_DATA;
         return;
     } else if (code == 150) {
         /* Accept data channel */
         debugs(9, 3, HERE << "accept data channel from " << ftpState->data.conn->remote << " (" << ftpState->data.conn->local << ")");
-        ftpState->listenForDataChannel(ftpState->data.conn, ftpState->data.host);
+        ftpState->listenForDataChannel(ftpState->data.conn);
         return;
     } else if (!ftpState->flags.tried_nlst && code > 300) {
         ftpSendNlst(ftpState);
@@ -3221,9 +2255,8 @@
     }
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendRetr(FtpStateData * ftpState)
+ftpSendRetr(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendRetr"))
@@ -3234,12 +2267,11 @@
     assert(ftpState->filepath != NULL);
     snprintf(cbuf, CTRL_BUFLEN, "RETR %s\r\n", ftpState->filepath);
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_RETR;
+    ftpState->state = Ftp::Client::SENT_RETR;
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadRetr(FtpStateData * ftpState)
+ftpReadRetr(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -3249,10 +2281,10 @@
         debugs(9, 3, HERE << "begin data transfer from " << ftpState->data.conn->remote << " (" << ftpState->data.conn->local << ")");
         ftpState->switchTimeoutToDataChannel();
         ftpState->maybeReadVirginBody();
-        ftpState->state = READING_DATA;
+        ftpState->state = Ftp::Client::READING_DATA;
     } else if (code == 150) {
         /* Accept data channel */
-        ftpState->listenForDataChannel(ftpState->data.conn, ftpState->data.host);
+        ftpState->listenForDataChannel(ftpState->data.conn);
     } else if (code >= 300) {
         if (!ftpState->flags.try_slash_hack) {
             /* Try this as a directory missing trailing slash... */
@@ -3270,10 +2302,10 @@
  * directory listing display.
  */
 void
-FtpStateData::completedListing()
+Ftp::Gateway::completedListing()
 {
     assert(entry);
-    entry->lock("FtpStateData");
+    entry->lock("Ftp::Gateway");
     ErrorState ferr(ERR_DIR_LISTING, Http::scOkay, request);
     ferr.ftp.listing = &listing;
     ferr.ftp.cwd_msg = xstrdup(cwd_message.size()? cwd_message.termedBuf() : "");
@@ -3282,12 +2314,11 @@
     entry->replaceHttpReply( ferr.BuildHttpReply() );
     EBIT_CLR(entry->flags, ENTRY_FWD_HDR_WAIT);
     entry->flush();
-    entry->unlock("FtpStateData");
+    entry->unlock("Ftp::Gateway");
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpReadTransferDone(FtpStateData * ftpState)
+ftpReadTransferDone(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -3309,27 +2340,15 @@
 
 // premature end of the request body
 void
-FtpStateData::handleRequestBodyProducerAborted()
+Ftp::Gateway::handleRequestBodyProducerAborted()
 {
     ServerStateData::handleRequestBodyProducerAborted();
     debugs(9, 3, HERE << "ftpState=" << this);
     failed(ERR_READ_ERROR, 0);
 }
 
-/**
- * This will be called when the put write is completed
- */
-void
-FtpStateData::sentRequestBody(const CommIoCbParams &io)
-{
-    if (io.size > 0)
-        kb_incr(&(statCounter.server.ftp.kbytes_out), io.size);
-    ServerStateData::sentRequestBody(io);
-}
-
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpWriteTransferDone(FtpStateData * ftpState)
+ftpWriteTransferDone(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     debugs(9, 3, HERE);
@@ -3344,9 +2363,8 @@
     ftpSendReply(ftpState);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendQuit(FtpStateData * ftpState)
+ftpSendQuit(Ftp::Gateway * ftpState)
 {
     /* check the server control channel is still available */
     if (!ftpState || !ftpState->haveControlChannel("ftpSendQuit"))
@@ -3354,24 +2372,20 @@
 
     snprintf(cbuf, CTRL_BUFLEN, "QUIT\r\n");
     ftpState->writeCommand(cbuf);
-    ftpState->state = SENT_QUIT;
+    ftpState->state = Ftp::Client::SENT_QUIT;
 }
 
-/**
- * \ingroup ServerProtocolFTPInternal
- *
- *  This completes a client FTP operation with success or other page
+/** Completes a client FTP operation with success or other page
  *  generated and stored in the entry field by the code issuing QUIT.
  */
 static void
-ftpReadQuit(FtpStateData * ftpState)
+ftpReadQuit(Ftp::Gateway * ftpState)
 {
     ftpState->serverComplete();
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpTrySlashHack(FtpStateData * ftpState)
+ftpTrySlashHack(Ftp::Gateway * ftpState)
 {
     char *path;
     ftpState->flags.try_slash_hack = 1;
@@ -3399,7 +2413,7 @@
  * Forget hack status. Next error is shown to the user
  */
 void
-FtpStateData::unhack()
+Ftp::Gateway::unhack()
 {
     debugs(9, 3, HERE);
 
@@ -3410,7 +2424,7 @@
 }
 
 void
-FtpStateData::hackShortcut(FTPSM * nextState)
+Ftp::Gateway::hackShortcut(FTPSM * nextState)
 {
     /* Clear some unwanted state */
     setCurrentOffset(0);
@@ -3433,9 +2447,8 @@
     nextState(this);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpFail(FtpStateData *ftpState)
+ftpFail(Ftp::Gateway *ftpState)
 {
     debugs(9, 6, HERE << "flags(" <<
            (ftpState->flags.isdir?"IS_DIR,":"") <<
@@ -3451,9 +2464,9 @@
 
         switch (ftpState->state) {
 
-        case SENT_CWD:
+        case Ftp::Client::SENT_CWD:
 
-        case SENT_RETR:
+        case Ftp::Client::SENT_RETR:
             /* Try the / hack */
             ftpState->hackShortcut(ftpTrySlashHack);
             return;
@@ -3467,101 +2480,43 @@
     /* failed() closes ctrl.conn and frees this */
 }
 
-void
-FtpStateData::failed(err_type error, int xerrno)
-{
-    debugs(9,3,HERE << "entry-null=" << (entry?entry->isEmpty():0) << ", entry=" << entry);
-    if (entry->isEmpty())
-        failedErrorMessage(error, xerrno);
-
-    serverComplete();
-}
-
-void
-FtpStateData::failedErrorMessage(err_type error, int xerrno)
-{
-    ErrorState *ftperr = NULL;
-    const char *command, *reply;
-
-    /* Translate FTP errors into HTTP errors */
-    switch (error) {
-
-    case ERR_NONE:
-
+Http::StatusCode
+Ftp::Gateway::failedHttpStatus(err_type &error)
+{
+    if (error == ERR_NONE) {
         switch (state) {
 
         case SENT_USER:
 
         case SENT_PASS:
 
-            if (ctrl.replycode > 500)
-                if (password_url)
-                    ftperr = new ErrorState(ERR_FTP_FORBIDDEN, Http::scForbidden, fwd->request);
-                else
-                    ftperr = new ErrorState(ERR_FTP_FORBIDDEN, Http::scUnauthorized, fwd->request);
-
-            else if (ctrl.replycode == 421)
-                ftperr = new ErrorState(ERR_FTP_UNAVAILABLE, Http::scServiceUnavailable, fwd->request);
-
+            if (ctrl.replycode > 500) {
+                error = ERR_FTP_FORBIDDEN;
+                return password_url ? Http::scForbidden : Http::scUnauthorized;
+            } else if (ctrl.replycode == 421) {
+                error = ERR_FTP_UNAVAILABLE;
+                return Http::scServiceUnavailable;
+            }
             break;
 
         case SENT_CWD:
 
         case SENT_RETR:
-            if (ctrl.replycode == 550)
-                ftperr = new ErrorState(ERR_FTP_NOT_FOUND, Http::scNotFound, fwd->request);
-
+            if (ctrl.replycode == 550) {
+                error = ERR_FTP_NOT_FOUND;
+                return Http::scNotFound;
+            }
             break;
 
         default:
             break;
         }
-
-        break;
-
-    case ERR_READ_TIMEOUT:
-        ftperr = new ErrorState(error, Http::scGatewayTimeout, fwd->request);
-        break;
-
-    default:
-        ftperr = new ErrorState(error, Http::scBadGateway, fwd->request);
-        break;
     }
-
-    if (ftperr == NULL)
-        ftperr = new ErrorState(ERR_FTP_FAILURE, Http::scBadGateway, fwd->request);
-
-    ftperr->xerrno = xerrno;
-
-    ftperr->ftp.server_msg = ctrl.message;
-    ctrl.message = NULL;
-
-    if (old_request)
-        command = old_request;
-    else
-        command = ctrl.last_command;
-
-    if (command && strncmp(command, "PASS", 4) == 0)
-        command = "PASS <yourpassword>";
-
-    if (old_reply)
-        reply = old_reply;
-    else
-        reply = ctrl.last_reply;
-
-    if (command)
-        ftperr->ftp.request = xstrdup(command);
-
-    if (reply)
-        ftperr->ftp.reply = xstrdup(reply);
-
-    entry->replaceHttpReply( ftperr->BuildHttpReply() );
-    delete ftperr;
+    return Ftp::Client::failedHttpStatus(error);
 }
 
-/// \ingroup ServerProtocolFTPInternal
 static void
-ftpSendReply(FtpStateData * ftpState)
+ftpSendReply(Ftp::Gateway * ftpState)
 {
     int code = ftpState->ctrl.replycode;
     Http::StatusCode http_code;
@@ -3606,7 +2561,7 @@
 }
 
 void
-FtpStateData::appendSuccessHeader()
+Ftp::Gateway::appendSuccessHeader()
 {
     const char *mime_type = NULL;
     const char *mime_enc = NULL;
@@ -3687,7 +2642,7 @@
 }
 
 void
-FtpStateData::haveParsedReplyHeaders()
+Ftp::Gateway::haveParsedReplyHeaders()
 {
     ServerStateData::haveParsedReplyHeaders();
 
@@ -3708,7 +2663,7 @@
 }
 
 HttpReply *
-FtpStateData::ftpAuthRequired(HttpRequest * request, const char *realm)
+Ftp::Gateway::ftpAuthRequired(HttpRequest * request, const char *realm)
 {
     ErrorState err(ERR_CACHE_ACCESS_DENIED, Http::scUnauthorized, request);
     HttpReply *newrep = err.BuildHttpReply();
@@ -3719,20 +2674,8 @@
     return newrep;
 }
 
-/**
- \ingroup ServerProtocolFTPAPI
- \todo Should be a URL class API call.
- *
- *  Construct an URI with leading / in PATH portion for use by CWD command
- *  possibly others. FTP encodes absolute paths as beginning with '/'
- *  after the initial URI path delimiter, which happens to be / itself.
- *  This makes FTP absolute URI appear as:  ftp:host:port//root/path
- *  To encompass older software which compacts multiple // to / in transit
- *  We use standard URI-encoding on the second / making it
- *  ftp:host:port/%2froot/path  AKA 'the FTP %2f hack'.
- */
 const char *
-ftpUrlWith2f(HttpRequest * request)
+Ftp::UrlWith2f(HttpRequest * request)
 {
     String newbuf = "%2f";
 
@@ -3753,7 +2696,7 @@
 }
 
 void
-FtpStateData::printfReplyBody(const char *fmt, ...)
+Ftp::Gateway::printfReplyBody(const char *fmt, ...)
 {
     va_list args;
     va_start (args, fmt);
@@ -3769,35 +2712,20 @@
  * which should be sent to either StoreEntry, or to ICAP...
  */
 void
-FtpStateData::writeReplyBody(const char *dataToWrite, size_t dataLength)
+Ftp::Gateway::writeReplyBody(const char *dataToWrite, size_t dataLength)
 {
     debugs(9, 5, HERE << "writing " << dataLength << " bytes to the reply");
     addVirginReplyBody(dataToWrite, dataLength);
 }
 
 /**
- * called after we wrote the last byte of the request body
- */
-void
-FtpStateData::doneSendingRequestBody()
-{
-    ServerStateData::doneSendingRequestBody();
-    debugs(9,3, HERE);
-    dataComplete();
-    /* NP: RFC 959  3.3.  DATA CONNECTION MANAGEMENT
-     * if transfer type is 'stream' call dataComplete()
-     * otherwise leave open. (reschedule control channel read?)
-     */
-}
-
-/**
  * A hack to ensure we do not double-complete on the forward entry.
  *
- \todo FtpStateData logic should probably be rewritten to avoid
+ \todo Ftp::Gateway logic should probably be rewritten to avoid
  *	double-completion or FwdState should be rewritten to allow it.
  */
 void
-FtpStateData::completeForwarding()
+Ftp::Gateway::completeForwarding()
 {
     if (fwd == NULL || flags.completed_forwarding) {
         debugs(9, 3, HERE << "completeForwarding avoids " <<
@@ -3811,45 +2739,13 @@
 }
 
 /**
- * Close the FTP server connection(s). Used by serverComplete().
- */
-void
-FtpStateData::closeServer()
-{
-    if (Comm::IsConnOpen(ctrl.conn)) {
-        debugs(9,3, HERE << "closing FTP server FD " << ctrl.conn->fd << ", this " << this);
-        fwd->unregister(ctrl.conn);
-        ctrl.close();
-    }
-
-    if (Comm::IsConnOpen(data.conn)) {
-        debugs(9,3, HERE << "closing FTP data FD " << data.conn->fd << ", this " << this);
-        data.close();
-    }
-
-    debugs(9,3, HERE << "FTP ctrl and data connections closed. this " << this);
-}
-
-/**
- * Did we close all FTP server connection(s)?
- *
- \retval true	Both server control and data channels are closed. And not waiting for a new data connection to open.
- \retval false	Either control channel or data is still active.
- */
-bool
-FtpStateData::doneWithServer() const
-{
-    return !Comm::IsConnOpen(ctrl.conn) && !Comm::IsConnOpen(data.conn);
-}
-
-/**
  * Have we lost the FTP server control channel?
  *
  \retval true	The server control channel is available.
  \retval false	The server control channel is not available.
  */
 bool
-FtpStateData::haveControlChannel(const char *caller_name) const
+Ftp::Gateway::haveControlChannel(const char *caller_name) const
 {
     if (doneWithServer())
         return false;
@@ -3864,64 +2760,15 @@
     return true;
 }
 
-/**
- * Quickly abort the transaction
- *
- \todo destruction should be sufficient as the destructor should cleanup,
- *	including canceling close handlers
- */
-void
-FtpStateData::abortTransaction(const char *reason)
-{
-    debugs(9, 3, HERE << "aborting transaction for " << reason <<
-           "; FD " << (ctrl.conn!=NULL?ctrl.conn->fd:-1) << ", Data FD " << (data.conn!=NULL?data.conn->fd:-1) << ", this " << this);
-    if (Comm::IsConnOpen(ctrl.conn)) {
-        ctrl.conn->close();
-        return;
-    }
-
-    fwd->handleUnregisteredServerEnd();
-    mustStop("FtpStateData::abortTransaction");
-}
-
-/// creates a data channel Comm close callback
-AsyncCall::Pointer
-FtpStateData::dataCloser()
-{
-    typedef CommCbMemFunT<FtpStateData, CommCloseCbParams> Dialer;
-    return JobCallback(9, 5, Dialer, this, FtpStateData::dataClosed);
-}
-
-/// configures the channel with a descriptor and registers a close handler
-void
-FtpChannel::opened(const Comm::ConnectionPointer &newConn, const AsyncCall::Pointer &aCloser)
-{
-    assert(!Comm::IsConnOpen(conn));
-    assert(closer == NULL);
-
-    assert(Comm::IsConnOpen(newConn));
-    assert(aCloser != NULL);
-
-    conn = newConn;
-    closer = aCloser;
-    comm_add_close_handler(conn->fd, closer);
-}
-
-/// planned close: removes the close handler and calls comm_close
-void
-FtpChannel::close()
-{
-    // channels with active listeners will be closed when the listener handler dies.
-    if (Comm::IsConnOpen(conn)) {
-        comm_remove_close_handler(conn->fd, closer);
-        conn->close(); // we do not expect to be called back
-    }
-    clear();
-}
-
-void
-FtpChannel::clear()
-{
-    conn = NULL;
-    closer = NULL;
+bool
+Ftp::Gateway::mayReadVirginReplyBody() const
+{
+    // TODO: Can we do what Ftp::Relay::mayReadVirginReplyBody() does instead?
+    return !doneWithServer();
+}
+
+AsyncJob::Pointer
+Ftp::StartGateway(FwdState *const fwdState)
+{
+    return AsyncJob::Start(new Ftp::Gateway(fwdState));
 }

=== added file 'src/clients/FtpRelay.cc'
--- src/clients/FtpRelay.cc	1970-01-01 00:00:00 +0000
+++ src/clients/FtpRelay.cc	2014-08-10 23:18:33 +0000
@@ -0,0 +1,700 @@
+/*
+ * DEBUG: section 09    File Transfer Protocol (FTP)
+ *
+ */
+
+#include "squid.h"
+#include "anyp/PortCfg.h"
+#include "client_side.h"
+#include "clients/forward.h"
+#include "clients/FtpClient.h"
+#include "ftp/Elements.h"
+#include "ftp/Parsing.h"
+#include "HttpHdrCc.h"
+#include "HttpRequest.h"
+#include "SBuf.h"
+#include "Server.h"
+#include "servers/FtpServer.h"
+#include "SquidTime.h"
+#include "Store.h"
+#include "wordlist.h"
+
+namespace Ftp
+{
+
+/// An FTP client receiving native FTP commands from our FTP server
+/// (Ftp::Server), forwarding them to the next FTP hop,
+/// and then relaying FTP replies back to our FTP server.
+class Relay: public Ftp::Client
+{
+public:
+    explicit Relay(FwdState *const fwdState);
+    virtual ~Relay();
+
+protected:
+    const Ftp::MasterState &master() const;
+    Ftp::MasterState &updateMaster();
+    Ftp::ServerState serverState() const { return master().serverState; }
+    void serverState(const Ftp::ServerState newState);
+
+    /* Ftp::Client API */
+    virtual void failed(err_type error = ERR_NONE, int xerrno = 0);
+    virtual void dataChannelConnected(const CommConnectCbParams &io);
+
+    /* ServerStateData API */
+    virtual void serverComplete();
+    virtual void handleControlReply();
+    virtual void processReplyBody();
+    virtual void handleRequestBodyProducerAborted();
+    virtual bool mayReadVirginReplyBody() const;
+    virtual void completeForwarding();
+
+    /* AsyncJob API */
+    virtual void start();
+
+    void forwardReply();
+    void forwardError(err_type error = ERR_NONE, int xerrno = 0);
+    void failedErrorMessage(err_type error, int xerrno);
+    HttpReply *createHttpReply(const Http::StatusCode httpStatus, const int64_t clen = 0);
+    void handleDataRequest();
+    void startDataDownload();
+    void startDataUpload();
+    bool startDirTracking();
+    void stopDirTracking();
+    bool weAreTrackingDir() const {return savedReply.message != NULL;}
+
+    typedef void (Relay::*PreliminaryCb)();
+    void forwardPreliminaryReply(const PreliminaryCb cb);
+    void proceedAfterPreliminaryReply();
+    PreliminaryCb thePreliminaryCb;
+
+    typedef void (Relay::*SM_FUNC)();
+    static const SM_FUNC SM_FUNCS[];
+    void readGreeting();
+    void sendCommand();
+    void readReply();
+    void readFeatReply();
+    void readPasvReply();
+    void readDataReply();
+    void readTransferDoneReply();
+    void readEpsvReply();
+    void readCwdOrCdupReply();
+    void readUserOrPassReply();
+
+    void scheduleReadControlReply();
+
+    bool forwardingCompleted; ///< completeForwarding() has been called
+
+    struct {
+        wordlist *message; ///< reply message, one  wordlist entry per message line
+        char *lastCommand; ///< the command caused the reply
+        char *lastReply; ///< last line of reply: reply status plus message
+        int replyCode; ///< the reply status
+    } savedReply; ///< set and delayed while we are tracking using PWD
+
+    CBDATA_CLASS2(Relay);
+};
+
+} // namespace Ftp
+
+CBDATA_NAMESPACED_CLASS_INIT(Ftp, Relay);
+
+const Ftp::Relay::SM_FUNC Ftp::Relay::SM_FUNCS[] = {
+    &Ftp::Relay::readGreeting, // BEGIN
+    &Ftp::Relay::readUserOrPassReply, // SENT_USER
+    &Ftp::Relay::readUserOrPassReply, // SENT_PASS
+    NULL,/* &Ftp::Relay::readReply */ // SENT_TYPE
+    NULL,/* &Ftp::Relay::readReply */ // SENT_MDTM
+    NULL,/* &Ftp::Relay::readReply */ // SENT_SIZE
+    NULL, // SENT_EPRT
+    NULL, // SENT_PORT
+    &Ftp::Relay::readEpsvReply, // SENT_EPSV_ALL
+    &Ftp::Relay::readEpsvReply, // SENT_EPSV_1
+    &Ftp::Relay::readEpsvReply, // SENT_EPSV_2
+    &Ftp::Relay::readPasvReply, // SENT_PASV
+    &Ftp::Relay::readCwdOrCdupReply,  // SENT_CWD
+    NULL,/* &Ftp::Relay::readDataReply, */ // SENT_LIST
+    NULL,/* &Ftp::Relay::readDataReply, */ // SENT_NLST
+    NULL,/* &Ftp::Relay::readReply */ // SENT_REST
+    NULL,/* &Ftp::Relay::readDataReply */ // SENT_RETR
+    NULL,/* &Ftp::Relay::readReply */ // SENT_STOR
+    NULL,/* &Ftp::Relay::readReply */ // SENT_QUIT
+    &Ftp::Relay::readTransferDoneReply, // READING_DATA
+    &Ftp::Relay::readReply, // WRITING_DATA
+    NULL,/* &Ftp::Relay::readReply */ // SENT_MKDIR
+    &Ftp::Relay::readFeatReply, // SENT_FEAT
+    NULL,/* &Ftp::Relay::readPwdReply */ // SENT_PWD
+    &Ftp::Relay::readCwdOrCdupReply, // SENT_CDUP
+    &Ftp::Relay::readDataReply,// SENT_DATA_REQUEST
+    &Ftp::Relay::readReply, // SENT_COMMAND
+    NULL
+};
+
+Ftp::Relay::Relay(FwdState *const fwdState):
+        AsyncJob("Ftp::Relay"),
+        Ftp::Client(fwdState),
+        forwardingCompleted(false)
+{
+    savedReply.message = NULL;
+    savedReply.lastCommand = NULL;
+    savedReply.lastReply = NULL;
+    savedReply.replyCode = 0;
+
+    // Nothing we can do at request creation time can mark the response as
+    // uncachable, unfortunately. This prevents "found KEY_PRIVATE" WARNINGs.
+    entry->releaseRequest();
+}
+
+Ftp::Relay::~Relay()
+{
+    closeServer(); // TODO: move to Server.cc?
+    if (savedReply.message)
+        wordlistDestroy(&savedReply.message);
+
+    xfree(savedReply.lastCommand);
+    xfree(savedReply.lastReply);
+}
+
+void
+Ftp::Relay::start()
+{
+    if (!master().clientReadGreeting)
+        Ftp::Client::start();
+    else if (serverState() == fssHandleDataRequest ||
+             serverState() == fssHandleUploadRequest)
+        handleDataRequest();
+    else
+        sendCommand();
+}
+
+/// Keep control connection for future requests, after we are done with it.
+/// Similar to COMPLETE_PERSISTENT_MSG handling in http.cc.
+void
+Ftp::Relay::serverComplete()
+{
+    CbcPointer<ConnStateData> &mgr = fwd->request->clientConnectionManager;
+    if (mgr.valid()) {
+        if (Comm::IsConnOpen(ctrl.conn)) {
+            debugs(9, 7, "completing FTP server " << ctrl.conn <<
+                   " after " << ctrl.replycode);
+            fwd->unregister(ctrl.conn);
+            if (ctrl.replycode == 221) { // Server sends FTP 221 before closing
+                mgr->unpinConnection(false);
+                ctrl.close();
+            } else {
+                mgr->pinConnection(ctrl.conn, fwd->request,
+                                   ctrl.conn->getPeer(),
+                                   fwd->request->flags.connectionAuth);
+                ctrl.forget();
+            }
+        }
+    }
+    Ftp::Client::serverComplete();
+}
+
+/// Safely returns the master state,
+/// with safety checks in case the Ftp::Server side of the master xact is gone.
+Ftp::MasterState &
+Ftp::Relay::updateMaster()
+{
+    CbcPointer<ConnStateData> &mgr = fwd->request->clientConnectionManager;
+    if (mgr.valid()) {
+        if (Ftp::Server *srv = dynamic_cast<Ftp::Server*>(mgr.get()))
+            return *srv->master;
+    }
+    // this code will not be necessary once the master is inside MasterXaction
+    debugs(9, 3, "our server side is gone: " << mgr);
+    static Ftp::MasterState Master;
+    Master = Ftp::MasterState();
+    return Master;
+}
+
+/// A const variant of updateMaster().
+const Ftp::MasterState &
+Ftp::Relay::master() const
+{
+    return const_cast<Ftp::Relay*>(this)->updateMaster(); // avoid code dupe
+}
+
+/// Changes server state and debugs about that important event.
+void
+Ftp::Relay::serverState(const Ftp::ServerState newState)
+{
+    Ftp::ServerState &cltState = updateMaster().serverState;
+    debugs(9, 3, "client state was " << cltState << " now: " << newState);
+    cltState = newState;
+}
+
+/**
+ * Ensure we do not double-complete on the forward entry.
+ * We complete forwarding when the response adaptation is over
+ * (but we may still be waiting for 226 from the FTP server) and
+ * also when we get that 226 from the server (and adaptation is done).
+ *
+ \todo Rewrite FwdState to ignore double completion?
+ */
+void
+Ftp::Relay::completeForwarding()
+{
+    debugs(9, 5, forwardingCompleted);
+    if (forwardingCompleted)
+        return;
+    forwardingCompleted = true;
+    Ftp::Client::completeForwarding();
+}
+
+void
+Ftp::Relay::failed(err_type error, int xerrno)
+{
+    if (!doneWithServer())
+        serverState(fssError);
+
+    // TODO: we need to customize ErrorState instead
+    if (entry->isEmpty())
+        failedErrorMessage(error, xerrno); // as a reply
+
+    Ftp::Client::failed(error, xerrno);
+}
+
+void
+Ftp::Relay::failedErrorMessage(err_type error, int xerrno)
+{
+    const Http::StatusCode httpStatus = failedHttpStatus(error);
+    HttpReply *const reply = createHttpReply(httpStatus);
+    entry->replaceHttpReply(reply);
+    EBIT_CLR(entry->flags, ENTRY_FWD_HDR_WAIT);
+    fwd->request->detailError(error, xerrno);
+}
+
+void
+Ftp::Relay::processReplyBody()
+{
+    debugs(9, 3, status());
+
+    if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) {
+        /*
+         * probably was aborted because content length exceeds one
+         * of the maximum size limits.
+         */
+        abortTransaction("entry aborted after calling appendSuccessHeader()");
+        return;
+    }
+
+#if USE_ADAPTATION
+
+    if (adaptationAccessCheckPending) {
+        debugs(9, 3, "returning due to adaptationAccessCheckPending");
+        return;
+    }
+
+#endif
+
+    if (data.readBuf != NULL && data.readBuf->hasContent()) {
+        const mb_size_t csize = data.readBuf->contentSize();
+        debugs(9, 5, "writing " << csize << " bytes to the reply");
+        addVirginReplyBody(data.readBuf->content(), csize);
+        data.readBuf->consume(csize);
+    }
+
+    entry->flush();
+
+    maybeReadVirginBody();
+}
+
+void
+Ftp::Relay::handleControlReply()
+{
+    if (!request->clientConnectionManager.valid()) {
+        debugs(9, 5, "client connection gone");
+        closeServer();
+        return;
+    }
+
+    Ftp::Client::handleControlReply();
+    if (ctrl.message == NULL)
+        return; // didn't get complete reply yet
+
+    assert(state < END);
+    assert(this->SM_FUNCS[state] != NULL);
+    (this->*SM_FUNCS[state])();
+}
+
+void
+Ftp::Relay::handleRequestBodyProducerAborted()
+{
+    ::ServerStateData::handleRequestBodyProducerAborted();
+
+    failed(ERR_READ_ERROR);
+}
+
+bool
+Ftp::Relay::mayReadVirginReplyBody() const
+{
+    // TODO: move this method to the regular FTP server?
+    return Comm::IsConnOpen(data.conn);
+}
+
+void
+Ftp::Relay::forwardReply()
+{
+    assert(entry->isEmpty());
+    EBIT_CLR(entry->flags, ENTRY_FWD_HDR_WAIT);
+
+    HttpReply *const reply = createHttpReply(Http::scNoContent);
+
+    setVirginReply(reply);
+    adaptOrFinalizeReply();
+
+    serverComplete();
+}
+
+void
+Ftp::Relay::forwardPreliminaryReply(const PreliminaryCb cb)
+{
+    debugs(9, 5, "forwarding preliminary reply to client");
+
+    // we must prevent concurrent ConnStateData::sendControlMsg() calls
+    Must(thePreliminaryCb == NULL);
+    thePreliminaryCb = cb;
+
+    const HttpReply::Pointer reply = createHttpReply(Http::scContinue);
+
+    // the Sink will use this to call us back after writing 1xx to the client
+    typedef NullaryMemFunT<Relay> CbDialer;
+    const AsyncCall::Pointer call = JobCallback(11, 3, CbDialer, this,
+                                    Ftp::Relay::proceedAfterPreliminaryReply);
+
+    CallJobHere1(9, 4, request->clientConnectionManager, ConnStateData,
+                 ConnStateData::sendControlMsg, HttpControlMsg(reply, call));
+}
+
+void
+Ftp::Relay::proceedAfterPreliminaryReply()
+{
+    debugs(9, 5, "proceeding after preliminary reply to client");
+
+    Must(thePreliminaryCb != NULL);
+    const PreliminaryCb cb = thePreliminaryCb;
+    thePreliminaryCb = NULL;
+    (this->*cb)();
+}
+
+void
+Ftp::Relay::forwardError(err_type error, int xerrno)
+{
+    failed(error, xerrno);
+}
+
+HttpReply *
+Ftp::Relay::createHttpReply(const Http::StatusCode httpStatus, const int64_t clen)
+{
+    HttpReply *const reply = Ftp::HttpReplyWrapper(ctrl.replycode, ctrl.last_reply, httpStatus, clen);
+    if (ctrl.message) {
+        for (wordlist *W = ctrl.message; W && W->next; W = W->next)
+            reply->header.putStr(HDR_FTP_PRE, httpHeaderQuoteString(W->key).c_str());
+        // no hdrCacheInit() is needed for after HDR_FTP_PRE addition
+    }
+    return reply;
+}
+
+void
+Ftp::Relay::handleDataRequest()
+{
+    data.addr(master().clientDataAddr);
+    connectDataChannel();
+}
+
+void
+Ftp::Relay::startDataDownload()
+{
+    assert(Comm::IsConnOpen(data.conn));
+
+    debugs(9, 3, "begin data transfer from " << data.conn->remote <<
+           " (" << data.conn->local << ")");
+
+    HttpReply *const reply = createHttpReply(Http::scOkay, -1);
+    EBIT_CLR(entry->flags, ENTRY_FWD_HDR_WAIT);
+    setVirginReply(reply);
+    adaptOrFinalizeReply();
+
+    maybeReadVirginBody();
+    state = READING_DATA;
+}
+
+void
+Ftp::Relay::startDataUpload()
+{
+    assert(Comm::IsConnOpen(data.conn));
+
+    debugs(9, 3, "begin data transfer to " << data.conn->remote <<
+           " (" << data.conn->local << ")");
+
+    if (!startRequestBodyFlow()) { // register to receive body data
+        failed();
+        return;
+    }
+
+    state = WRITING_DATA;
+}
+
+void
+Ftp::Relay::readGreeting()
+{
+    assert(!master().clientReadGreeting);
+
+    switch (ctrl.replycode) {
+    case 220:
+        updateMaster().clientReadGreeting = true;
+        if (serverState() == fssBegin)
+            serverState(fssConnected);
+
+        // Do not forward server greeting to the user because our FTP Server
+        // has greeted the user already. Also, an original origin greeting may
+        // confuse a user that has changed the origin mid-air.
+
+        start();
+        break;
+    case 120:
+        if (NULL != ctrl.message)
+            debugs(9, DBG_IMPORTANT, "FTP server is busy: " << ctrl.message->key);
+        forwardPreliminaryReply(&Ftp::Relay::scheduleReadControlReply);
+        break;
+    default:
+        failed();
+        break;
+    }
+}
+
+void
+Ftp::Relay::sendCommand()
+{
+    if (!fwd->request->header.has(HDR_FTP_COMMAND)) {
+        abortTransaction("Internal error: FTP relay request with no command");
+        return;
+    }
+
+    HttpHeader &header = fwd->request->header;
+    assert(header.has(HDR_FTP_COMMAND));
+    const String &cmd = header.findEntry(HDR_FTP_COMMAND)->value;
+    assert(header.has(HDR_FTP_ARGUMENTS));
+    const String &params = header.findEntry(HDR_FTP_ARGUMENTS)->value;
+
+    if (params.size() > 0)
+        debugs(9, 5, "command: " << cmd << ", parameters: " << params);
+    else
+        debugs(9, 5, "command: " << cmd << ", no parameters");
+
+    if (serverState() == fssHandlePasv ||
+            serverState() == fssHandleEpsv ||
+            serverState() == fssHandleEprt ||
+            serverState() == fssHandlePort) {
+        sendPassive();
+        return;
+    }
+
+    SBuf buf;
+    if (params.size() > 0)
+        buf.Printf("%s %s%s", cmd.termedBuf(), params.termedBuf(), Ftp::crlf);
+    else
+        buf.Printf("%s%s", cmd.termedBuf(), Ftp::crlf);
+
+    writeCommand(buf.c_str());
+
+    state =
+        serverState() == fssHandleCdup ? SENT_CDUP :
+        serverState() == fssHandleCwd ? SENT_CWD :
+        serverState() == fssHandleFeat ? SENT_FEAT :
+        serverState() == fssHandleDataRequest ? SENT_DATA_REQUEST :
+        serverState() == fssHandleUploadRequest ? SENT_DATA_REQUEST :
+        serverState() == fssConnected ? SENT_USER :
+        serverState() == fssHandlePass ? SENT_PASS :
+        SENT_COMMAND;
+}
+
+void
+Ftp::Relay::readReply()
+{
+    assert(serverState() == fssConnected ||
+           serverState() == fssHandleUploadRequest);
+
+    if (100 <= ctrl.replycode && ctrl.replycode < 200)
+        forwardPreliminaryReply(&Ftp::Relay::scheduleReadControlReply);
+    else
+        forwardReply();
+}
+
+void
+Ftp::Relay::readFeatReply()
+{
+    assert(serverState() == fssHandleFeat);
+
+    if (100 <= ctrl.replycode && ctrl.replycode < 200)
+        return; // ignore preliminary replies
+
+    forwardReply();
+}
+
+void
+Ftp::Relay::readPasvReply()
+{
+    assert(serverState() == fssHandlePasv || serverState() == fssHandleEpsv || serverState() == fssHandlePort || serverState() == fssHandleEprt);
+
+    if (100 <= ctrl.replycode && ctrl.replycode < 200)
+        return; // ignore preliminary replies
+
+    if (handlePasvReply(updateMaster().clientDataAddr))
+        forwardReply();
+    else
+        forwardError();
+}
+
+void
+Ftp::Relay::readEpsvReply()
+{
+    if (100 <= ctrl.replycode && ctrl.replycode < 200)
+        return; // ignore preliminary replies
+
+    if (handleEpsvReply(updateMaster().clientDataAddr)) {
+        if (ctrl.message == NULL)
+            return; // didn't get complete reply yet
+
+        forwardReply();
+    } else
+        forwardError();
+}
+
+void
+Ftp::Relay::readDataReply()
+{
+    assert(serverState() == fssHandleDataRequest ||
+           serverState() == fssHandleUploadRequest);
+
+    if (ctrl.replycode == 125 || ctrl.replycode == 150) {
+        if (serverState() == fssHandleDataRequest)
+            forwardPreliminaryReply(&Ftp::Relay::startDataDownload);
+        else // serverState() == fssHandleUploadRequest
+            forwardPreliminaryReply(&Ftp::Relay::startDataUpload);
+    } else
+        forwardReply();
+}
+
+bool
+Ftp::Relay::startDirTracking()
+{
+    if (!fwd->request->clientConnectionManager->port->ftp_track_dirs)
+        return false;
+
+    debugs(9, 5, "start directory tracking");
+    savedReply.message = ctrl.message;
+    savedReply.lastCommand = ctrl.last_command;
+    savedReply.lastReply = ctrl.last_reply;
+    savedReply.replyCode = ctrl.replycode;
+
+    ctrl.last_command = NULL;
+    ctrl.last_reply = NULL;
+    ctrl.message = NULL;
+    ctrl.offset = 0;
+    writeCommand("PWD\r\n");
+    return true;
+}
+
+void
+Ftp::Relay::stopDirTracking()
+{
+    debugs(9, 5, "got code from pwd: " << ctrl.replycode << ", msg: " << ctrl.last_reply);
+
+    if (ctrl.replycode == 257)
+        updateMaster().workingDir = Ftp::UnescapeDoubleQuoted(ctrl.last_reply);
+
+    wordlistDestroy(&ctrl.message);
+    safe_free(ctrl.last_command);
+    safe_free(ctrl.last_reply);
+
+    ctrl.message = savedReply.message;
+    ctrl.last_command = savedReply.lastCommand;
+    ctrl.last_reply = savedReply.lastReply;
+    ctrl.replycode = savedReply.replyCode;
+
+    savedReply.message = NULL;
+    savedReply.lastReply = NULL;
+    savedReply.lastCommand = NULL;
+}
+
+void
+Ftp::Relay::readCwdOrCdupReply()
+{
+    assert(serverState() == fssHandleCwd ||
+           serverState() == fssHandleCdup);
+
+    debugs(9, 5, "got code " << ctrl.replycode << ", msg: " << ctrl.last_reply);
+
+    if (100 <= ctrl.replycode && ctrl.replycode < 200)
+        return;
+
+    if (weAreTrackingDir()) { // we are tracking
+        stopDirTracking(); // and forward the delayed response below
+    } else if (startDirTracking())
+        return;
+
+    forwardReply();
+}
+
+void
+Ftp::Relay::readUserOrPassReply()
+{
+    if (100 <= ctrl.replycode && ctrl.replycode < 200)
+        return; //Just ignore
+
+    if (weAreTrackingDir()) { // we are tracking
+        stopDirTracking(); // and forward the delayed response below
+    } else if (ctrl.replycode == 230) { // successful login
+        if (startDirTracking())
+            return;
+    }
+
+    forwardReply();
+}
+
+void
+Ftp::Relay::readTransferDoneReply()
+{
+    debugs(9, 3, status());
+
+    if (ctrl.replycode != 226 && ctrl.replycode != 250) {
+        debugs(9, DBG_IMPORTANT, "got FTP code " << ctrl.replycode <<
+               " after reading response data");
+    }
+
+    serverComplete();
+}
+
+void
+Ftp::Relay::dataChannelConnected(const CommConnectCbParams &io)
+{
+    debugs(9, 3, status());
+    data.opener = NULL;
+
+    if (io.flag != Comm::OK) {
+        debugs(9, 2, "failed to connect FTP server data channel");
+        forwardError(ERR_CONNECT_FAIL, io.xerrno);
+        return;
+    }
+
+    debugs(9, 2, "connected FTP server data channel: " << io.conn);
+
+    data.opened(io.conn, dataCloser());
+
+    sendCommand();
+}
+
+void
+Ftp::Relay::scheduleReadControlReply()
+{
+    Ftp::Client::scheduleReadControlReply(0);
+}
+
+AsyncJob::Pointer
+Ftp::StartRelay(FwdState *const fwdState)
+{
+    return AsyncJob::Start(new Ftp::Relay(fwdState));
+}

=== added file 'src/clients/Makefile.am'
--- src/clients/Makefile.am	1970-01-01 00:00:00 +0000
+++ src/clients/Makefile.am	2014-08-10 23:18:33 +0000
@@ -0,0 +1,11 @@
+include $(top_srcdir)/src/Common.am
+include $(top_srcdir)/src/TestHeaders.am
+
+noinst_LTLIBRARIES = libclients.la
+
+libclients_la_SOURCES = \
+	FtpClient.cc \
+	FtpClient.h \
+	FtpGateway.cc \
+	FtpRelay.cc \
+	forward.h

=== renamed file 'src/ftp.h' => 'src/clients/forward.h'
--- src/ftp.h	2012-09-21 14:57:30 +0000
+++ src/clients/forward.h	2014-08-10 02:28:33 +0000
@@ -1,48 +1,34 @@
-/*
- * DEBUG: section 09    File Transfer Protocol (FTP)
- * AUTHOR: Harvest Derived
- *
- * SQUID Web Proxy Cache          http://www.squid-cache.org/
- * ----------------------------------------------------------
- *
- *  Squid is the result of efforts by numerous individuals from
- *  the Internet community; see the CONTRIBUTORS file for full
- *  details.   Many organizations have provided support for Squid's
- *  development; see the SPONSORS file for full details.  Squid is
- *  Copyrighted (C) 2001 by the Regents of the University of
- *  California; see the COPYRIGHT file for full details.  Squid
- *  incorporates software developed and/or copyrighted by other
- *  sources; see the CREDITS file for full details.
- *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License as published by
- *  the Free Software Foundation; either version 2 of the License, or
- *  (at your option) any later version.
- *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
- *
- *  You should have received a copy of the GNU General Public License
- *  along with this program; if not, write to the Free Software
- *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
- *
- */
-
-#ifndef SQUID_FTP_H_
-#define SQUID_FTP_H_
+#ifndef SQUID_CLIENTS_FORWARD_H
+#define SQUID_CLIENTS_FORWARD_H
 
 class FwdState;
-
-/**
- * \defgroup ServerProtocolFTPAPI Server-Side FTP API
- * \ingroup ServerProtocol
+class HttpRequest;
+
+class AsyncJob;
+template <class Cbc> class CbcPointer;
+typedef CbcPointer<AsyncJob> AsyncJobPointer;
+
+namespace Ftp
+{
+
+/// A new FTP Gateway job
+AsyncJobPointer StartGateway(FwdState *const fwdState);
+
+/// A new FTP Relay job
+AsyncJobPointer StartRelay(FwdState *const fwdState);
+
+/** Construct an URI with leading / in PATH portion for use by CWD command
+ *  possibly others. FTP encodes absolute paths as beginning with '/'
+ *  after the initial URI path delimiter, which happens to be / itself.
+ *  This makes FTP absolute URI appear as:  ftp:host:port//root/path
+ *  To encompass older software which compacts multiple // to / in transit
+ *  We use standard URI-encoding on the second / making it
+ *  ftp:host:port/%2froot/path  AKA 'the FTP %2f hack'.
+ *
+ * \todo Should be a URL class API call.
  */
-
-/// \ingroup ServerProtocolFTPAPI
-void ftpStart(FwdState *);
-/// \ingroup ServerProtocolFTPAPI
-const char *ftpUrlWith2f(HttpRequest *);
-
-#endif /* SQUID_FTP_H_ */
+const char *UrlWith2f(HttpRequest *);
+
+} // namespace Ftp
+
+#endif /* SQUID_CLIENTS_FORWARD_H */

=== modified file 'src/errorpage.cc'
--- src/errorpage.cc	2014-06-24 22:52:53 +0000
+++ src/errorpage.cc	2014-08-05 00:17:16 +0000
@@ -31,13 +31,13 @@
  */
 #include "squid.h"
 #include "cache_cf.h"
+#include "clients/forward.h"
 #include "comm/Connection.h"
 #include "comm/Write.h"
 #include "disk.h"
 #include "err_detail_type.h"
 #include "errorpage.h"
 #include "fde.h"
-#include "ftp.h"
 #include "html_quote.h"
 #include "HttpHeaderTools.h"
 #include "HttpReply.h"
@@ -831,7 +831,7 @@
 
     case 'B':
         if (building_deny_info_url) break;
-        p = request ? ftpUrlWith2f(request) : "[no URL]";
+        p = request ? Ftp::UrlWith2f(request) : "[no URL]";
         break;
 
     case 'c':

=== added directory 'src/ftp'
=== added file 'src/ftp/Elements.cc'
--- src/ftp/Elements.cc	1970-01-01 00:00:00 +0000
+++ src/ftp/Elements.cc	2014-08-10 02:28:33 +0000
@@ -0,0 +1,187 @@
+/*
+ * DEBUG: section 09    File Transfer Protocol (FTP)
+ */
+
+#include "squid.h"
+#include "ftp/Elements.h"
+#include "HttpHdrCc.h"
+#include "HttpReply.h"
+#include "SBuf.h"
+
+// FTP does not have a notion of a "protocol version" but we need something for
+// compatibility with the current HttpMsg wrapping layer. We use version 1.1:
+// * some ICAP services probably expect /1.0 or /1.1 when parsing HTTP headers;
+// * FTP commands are sent on a "persistent by default" connection, just like
+//   HTTP/1.1. Using 1.1 leads to fewer exceptions in current code shared by
+//   HTTP and FTP.
+AnyP::ProtocolVersion
+Ftp::ProtocolVersion()
+{
+    return AnyP::ProtocolVersion(AnyP::PROTO_FTP, 1, 1);
+}
+
+HttpReply *
+Ftp::HttpReplyWrapper(const int ftpStatus, const char *ftpReason, const Http::StatusCode httpStatus, const int64_t clen)
+{
+    HttpReply *const reply = new HttpReply;
+
+    Http::ProtocolVersion httpVersion = Http::ProtocolVersion(
+                                            Ftp::ProtocolVersion().major, Ftp::ProtocolVersion().minor);
+    reply->sline.set(httpVersion, httpStatus);
+
+    HttpHeader &header = reply->header;
+    header.putTime(HDR_DATE, squid_curtime);
+    {
+        HttpHdrCc cc;
+        cc.Private(String());
+        header.putCc(&cc);
+    }
+    if (ftpStatus > 0)
+        header.putInt(HDR_FTP_STATUS, ftpStatus);
+    if (ftpReason)
+        header.putStr(HDR_FTP_REASON, ftpReason);
+    if (clen >= 0)
+        header.putInt64(HDR_CONTENT_LENGTH, clen);
+    reply->hdrCacheInit();
+    return reply;
+}
+
+const SBuf &
+Ftp::cmdAppe()
+{
+    static const SBuf cmd("APPE");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdAuth()
+{
+    static const SBuf cmd("AUTH");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdCwd()
+{
+    static const SBuf cmd("CWD");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdDele()
+{
+    static const SBuf cmd("DELE");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdEprt()
+{
+    static const SBuf cmd("EPRT");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdEpsv()
+{
+    static const SBuf cmd("EPSV");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdList()
+{
+    static const SBuf cmd("LIST");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdMkd()
+{
+    static const SBuf cmd("MKD");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdMlsd()
+{
+    static const SBuf cmd("MLSD");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdMlst()
+{
+    static const SBuf cmd("MLST");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdNlst()
+{
+    static const SBuf cmd("NLST");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdRetr()
+{
+    static const SBuf cmd("RETR");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdRmd()
+{
+    static const SBuf cmd("RMD");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdRnfr()
+{
+    static const SBuf cmd("RNFR");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdRnto()
+{
+    static const SBuf cmd("RNTO");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdSmnt()
+{
+    static const SBuf cmd("SMNT");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdStat()
+{
+    static const SBuf cmd("STAT");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdStor()
+{
+    static const SBuf cmd("STOR");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdStou()
+{
+    static const SBuf cmd("STOU");
+    return cmd;
+}
+
+const SBuf &
+Ftp::cmdUser()
+{
+    static const SBuf cmd("USER");
+    return cmd;
+}

=== added file 'src/ftp/Elements.h'
--- src/ftp/Elements.h	1970-01-01 00:00:00 +0000
+++ src/ftp/Elements.h	2014-08-09 00:35:28 +0000
@@ -0,0 +1,47 @@
+#ifndef SQUID_FTP_ELEMENTS_H
+#define SQUID_FTP_ELEMENTS_H
+
+#include "http/StatusCode.h"
+
+class SBuf;
+class HttpReply;
+
+namespace AnyP
+{
+class ProtocolVersion;
+}
+
+namespace Ftp
+{
+
+/// Protocol version to use in HttpMsg structures wrapping FTP messages.
+AnyP::ProtocolVersion ProtocolVersion();
+
+/// Create an internal HttpReply structure to house FTP control response info.
+HttpReply *HttpReplyWrapper(const int ftpStatus, const char *ftpReason, const Http::StatusCode httpStatus, const int64_t clen);
+
+/* FTP Commands used by Squid. ALLCAPS case. Safe for static initializaton. */
+const SBuf &cmdAppe();
+const SBuf &cmdAuth();
+const SBuf &cmdCwd();
+const SBuf &cmdDele();
+const SBuf &cmdEprt();
+const SBuf &cmdEpsv();
+const SBuf &cmdList();
+const SBuf &cmdMkd();
+const SBuf &cmdMlsd();
+const SBuf &cmdMlst();
+const SBuf &cmdNlst();
+const SBuf &cmdRetr();
+const SBuf &cmdRmd();
+const SBuf &cmdRnfr();
+const SBuf &cmdRnto();
+const SBuf &cmdSmnt();
+const SBuf &cmdStat();
+const SBuf &cmdStor();
+const SBuf &cmdStou();
+const SBuf &cmdUser();
+
+} // namespace Ftp
+
+#endif /* SQUID_FTP_ELEMENTS_H */

=== added file 'src/ftp/Makefile.am'
--- src/ftp/Makefile.am	1970-01-01 00:00:00 +0000
+++ src/ftp/Makefile.am	2014-08-10 23:18:33 +0000
@@ -0,0 +1,10 @@
+include $(top_srcdir)/src/Common.am
+include $(top_srcdir)/src/TestHeaders.am
+
+noinst_LTLIBRARIES = libftp.la
+
+libftp_la_SOURCES = \
+	Elements.cc \
+	Elements.h \
+	Parsing.cc \
+	Parsing.h

=== added file 'src/ftp/Parsing.cc'
--- src/ftp/Parsing.cc	1970-01-01 00:00:00 +0000
+++ src/ftp/Parsing.cc	2014-08-04 21:44:31 +0000
@@ -0,0 +1,108 @@
+/*
+ * DEBUG: section 09    File Transfer Protocol (FTP)
+ */
+
+#include "squid.h"
+#include "ftp/Parsing.h"
+#include "ip/Address.h"
+#include "MemBuf.h"
+#include "SquidConfig.h"
+
+bool
+Ftp::ParseIpPort(const char *buf, const char *forceIp, Ip::Address &addr)
+{
+    int h1, h2, h3, h4;
+    int p1, p2;
+    const int n = sscanf(buf, "%d,%d,%d,%d,%d,%d",
+                         &h1, &h2, &h3, &h4, &p1, &p2);
+
+    if (n != 6 || p1 < 0 || p2 < 0 || p1 > 255 || p2 > 255)
+        return false;
+
+    if (forceIp) {
+        addr = forceIp; // but the above code still validates the IP we got
+    } else {
+        static char ipBuf[1024];
+        snprintf(ipBuf, sizeof(ipBuf), "%d.%d.%d.%d", h1, h2, h3, h4);
+        addr = ipBuf;
+
+        if (addr.isAnyAddr())
+            return false;
+    }
+
+    const int port = ((p1 << 8) + p2);
+
+    if (port <= 0)
+        return false;
+
+    if (Config.Ftp.sanitycheck && port < 1024)
+        return false;
+
+    addr.port(port);
+    return true;
+}
+
+bool
+Ftp::ParseProtoIpPort(const char *buf, Ip::Address &addr)
+{
+
+    const char delim = *buf;
+    const char *s = buf + 1;
+    const char *e = s;
+    const int proto = strtol(s, const_cast<char**>(&e), 10);
+    if ((proto != 1 && proto != 2) || *e != delim)
+        return false;
+
+    s = e + 1;
+    e = strchr(s, delim);
+    char ip[MAX_IPSTRLEN];
+    if (static_cast<size_t>(e - s) >= sizeof(ip))
+        return false;
+    strncpy(ip, s, e - s);
+    ip[e - s] = '\0';
+    addr = ip;
+
+    if (addr.isAnyAddr())
+        return false;
+
+    if ((proto == 2) != addr.isIPv6()) // proto ID mismatches address version
+        return false;
+
+    s = e + 1; // skip port delimiter
+    const int port = strtol(s, const_cast<char**>(&e), 10);
+    if (port < 0 || *e != '|')
+        return false;
+
+    if (Config.Ftp.sanitycheck && port < 1024)
+        return false;
+
+    addr.port(port);
+    return true;
+}
+
+const char *
+Ftp::UnescapeDoubleQuoted(const char *quotedPath)
+{
+    static MemBuf path;
+    path.reset();
+    const char *s = quotedPath;
+    if (*s == '"') {
+        ++s;
+        bool parseDone = false;
+        while (!parseDone) {
+            if (const char *e = strchr(s, '"')) {
+                path.append(s, e - s);
+                s = e + 1;
+                if (*s == '"') {
+                    path.append(s, 1);
+                    ++s;
+                } else
+                    parseDone = true;
+            } else { //parse error
+                parseDone = true;
+                path.reset();
+            }
+        }
+    }
+    return path.content();
+}

=== added file 'src/ftp/Parsing.h'
--- src/ftp/Parsing.h	1970-01-01 00:00:00 +0000
+++ src/ftp/Parsing.h	2014-08-08 03:07:03 +0000
@@ -0,0 +1,21 @@
+#ifndef SQUID_FTP_PARSING_H
+#define SQUID_FTP_PARSING_H
+
+#include "ip/forward.h"
+
+namespace Ftp
+{
+
+/// parses and validates "A1,A2,A3,A4,P1,P2" IP,port sequence
+bool ParseIpPort(const char *buf, const char *forceIp, Ip::Address &addr);
+
+/// parses and validates EPRT "<d><net-prt><d><net-addr><d><tcp-port><d>"
+/// proto,IP,port sequence
+bool ParseProtoIpPort(const char *buf, Ip::Address &addr);
+
+/// parses an FTP-quoted quote-escaped path
+const char *UnescapeDoubleQuoted(const char *quotedPath);
+
+} // namespace Ftp
+
+#endif /* SQUID_FTP_PARSING_H */

=== modified file 'src/http.cc'
--- src/http.cc	2014-06-05 14:57:58 +0000
+++ src/http.cc	2014-08-10 23:18:33 +0000
@@ -1494,6 +1494,15 @@
     maybeReadVirginBody();
 }
 
+bool
+HttpStateData::mayReadVirginReplyBody() const
+{
+    // TODO: Be more precise here. For example, if/when reading trailer, we may
+    // not be doneWithServer() yet, but we should return false. Similarly, we
+    // could still be writing the request body after receiving the whole reply.
+    return !doneWithServer();
+}
+
 void
 HttpStateData::maybeReadVirginBody()
 {

=== modified file 'src/http.h'
--- src/http.h	2012-10-19 09:17:33 +0000
+++ src/http.h	2014-08-10 23:18:33 +0000
@@ -110,6 +110,7 @@
     virtual void closeServer(); // end communication with the server
     virtual bool doneWithServer() const; // did we end communication?
     virtual void abortTransaction(const char *reason); // abnormal termination
+    virtual bool mayReadVirginReplyBody() const;
 
     // consuming request body
     virtual void handleMoreRequestBodyAvailable();

=== modified file 'src/ip/Intercept.h'
--- src/ip/Intercept.h	2013-04-16 02:21:47 +0000
+++ src/ip/Intercept.h	2014-08-10 02:28:33 +0000
@@ -37,7 +37,7 @@
      * Detects IPv6 and IPv4 level of support matches the address being listened on
      * and if the compiled v2/v4 is usable as far down as a bind()ing.
      *
-     * \param test    Address set on the http(s)_port being checked.
+     * \param test    Address set on the squid.conf *_port being checked.
      * \retval true   TPROXY is available.
      * \retval false  TPROXY is not available.
      */

=== modified file 'src/ipc/FdNotes.cc'
--- src/ipc/FdNotes.cc	2012-09-01 14:38:36 +0000
+++ src/ipc/FdNotes.cc	2013-04-16 13:55:06 +0000
@@ -14,6 +14,7 @@
         "None", // fdnNone
         "HTTP Socket", // fdnHttpSocket
         "HTTPS Socket", // fdnHttpsSocket
+        "FTP Socket", // fdnFtpSocket
 #if SQUID_SNMP
         "Incoming SNMP Socket", // fdnInSnmpSocket
         "Outgoing SNMP Socket", // fdnOutSnmpSocket

=== modified file 'src/ipc/FdNotes.h'
--- src/ipc/FdNotes.h	2012-10-04 09:14:06 +0000
+++ src/ipc/FdNotes.h	2013-04-16 13:55:06 +0000
@@ -12,7 +12,7 @@
 /// We cannot send char* FD notes to other processes. Pass int IDs and convert.
 
 /// fd_note() label ID
-typedef enum { fdnNone, fdnHttpSocket, fdnHttpsSocket,
+typedef enum { fdnNone, fdnHttpSocket, fdnHttpsSocket, fdnFtpSocket,
 #if SQUID_SNMP
                fdnInSnmpSocket, fdnOutSnmpSocket,
 #endif

=== modified file 'src/main.cc'
--- src/main.cc	2014-07-18 11:51:17 +0000
+++ src/main.cc	2014-07-30 17:33:14 +0000
@@ -730,7 +730,7 @@
 #endif
     }
     if (IamWorkerProcess()) {
-        clientHttpConnectionsClose();
+        clientConnectionsClose();
         icpConnectionShutdown();
 #if USE_HTCP
         htcpSocketShutdown();

=== modified file 'src/parser/Tokenizer.cc'
--- src/parser/Tokenizer.cc	2014-06-16 22:45:23 +0000
+++ src/parser/Tokenizer.cc	2014-08-08 03:07:03 +0000
@@ -28,20 +28,35 @@
 #endif
 #endif
 
+/// convenience method: consumes up to n bytes, counts, and returns them
+SBuf
+Parser::Tokenizer::consume(const SBuf::size_type n)
+{
+    // careful: n may be npos!
+    const SBuf result = buf_.consume(n);
+    parsed_ += result.length();
+    return result;
+}
+
+/// convenience method: consume()s up to n bytes and returns their count
+SBuf::size_type
+Parser::Tokenizer::success(const SBuf::size_type n)
+{
+    return consume(n).length();
+}
+
 bool
 Parser::Tokenizer::token(SBuf &returnedToken, const CharacterSet &delimiters)
 {
-    const SBuf savebuf(buf_);
-    skip(delimiters);
+    const Tokenizer saved(*this);
+    skipAll(delimiters);
     const SBuf::size_type tokenLen = buf_.findFirstOf(delimiters); // not found = npos => consume to end
-    if (tokenLen == SBuf::npos && !delimiters['\0']) {
-        // no delimiter found, nor is NUL/EOS/npos acceptible as one
-        buf_ = savebuf;
+    if (tokenLen == SBuf::npos) {
+        *this = saved;
         return false;
     }
-    const SBuf retval = buf_.consume(tokenLen);
-    skip(delimiters);
-    returnedToken = retval;
+    returnedToken = consume(tokenLen); // cannot be empty
+    skipAll(delimiters);
     return true;
 }
 
@@ -51,37 +66,40 @@
     const SBuf::size_type prefixLen = buf_.substr(0,limit).findFirstNotOf(tokenChars);
     if (prefixLen == 0)
         return false;
-    returnedToken = buf_.consume(prefixLen);
+    returnedToken = consume(prefixLen);
     return true;
 }
 
-bool
-Parser::Tokenizer::skip(const CharacterSet &tokenChars)
+SBuf::size_type
+Parser::Tokenizer::skipAll(const CharacterSet &tokenChars)
 {
     const SBuf::size_type prefixLen = buf_.findFirstNotOf(tokenChars);
     if (prefixLen == 0)
-        return false;
-    buf_.consume(prefixLen);
-    return true;
+        return 0;
+    return success(prefixLen);
+}
+
+bool
+Parser::Tokenizer::skipOne(const CharacterSet &chars)
+{
+    if (!buf_.isEmpty() && chars[buf_[0]])
+        return success(1);
+    return false;
 }
 
 bool
 Parser::Tokenizer::skip(const SBuf &tokenToSkip)
 {
-    if (buf_.startsWith(tokenToSkip)) {
-        buf_.consume(tokenToSkip.length());
-        return true;
-    }
+    if (buf_.startsWith(tokenToSkip))
+        return success(tokenToSkip.length());
     return false;
 }
 
 bool
 Parser::Tokenizer::skip(const char tokenChar)
 {
-    if (buf_[0] == tokenChar) {
-        buf_.consume(1);
-        return true;
-    }
+    if (!buf_.isEmpty() && buf_[0] == tokenChar)
+        return success(1);
     return false;
 }
 
@@ -156,6 +174,5 @@
         acc = -acc;
 
     result = acc;
-    buf_.consume(s - buf_.rawContent() -1);
-    return true;
+    return success(s - buf_.rawContent() - 1);
 }

=== modified file 'src/parser/Tokenizer.h'
--- src/parser/Tokenizer.h	2014-06-02 00:15:10 +0000
+++ src/parser/Tokenizer.h	2014-08-07 17:26:03 +0000
@@ -21,10 +21,13 @@
 class Tokenizer
 {
 public:
-    explicit Tokenizer(const SBuf &inBuf) : buf_(inBuf) {}
-
-    // return a copy the current contents of the parse buffer
-    const SBuf buf() const { return buf_; }
+    explicit Tokenizer(const SBuf &inBuf) : buf_(inBuf), parsed_(0) {}
+
+    /// yet unparsed data
+    SBuf buf() const { return buf_; }
+
+    /// number of parsed bytes, including skipped ones
+    SBuf::size_type parsedSize() const { return parsed_; }
 
     /// whether the end of the buffer has been reached
     bool atEnd() const { return buf_.isEmpty(); }
@@ -33,35 +36,32 @@
     const SBuf& remaining() const { return buf_; }
 
     /// reinitialize processing for a new buffer
-    void reset(const SBuf &newBuf) { buf_ = newBuf; }
+    void reset(const SBuf &newBuf) { buf_ = newBuf; parsed_ = 0; }
 
     /** Basic strtok(3):
      *  Skips all leading delimiters (if any),
-     *  accumulates all characters up to the next delimiter (a token), and
-     *  skips all trailing delimiters.
+     *  extracts all characters up to the next delimiter (a token), and
+     *  skips all trailing delimiters (at least one must be present).
      *
      *  Want to extract delimiters? Use prefix() instead.
      *
-     * At least one terminating delimiter is required. \0 may be passed
-     * as a delimiter to treat end of buffer content as the end of token.
+     *  Note that Tokenizer cannot tell whether the trailing delimiters will
+     *  continue when/if more input data becomes available later.
      *
-     * \return false if no terminal delimiter is found.
+     * \return true if found a non-empty token followed by a delimiter
      */
     bool token(SBuf &returnedToken, const CharacterSet &delimiters);
 
-    /** Accumulates all sequential permitted characters up to an optional length limit.
+    /** Extracts all sequential permitted characters up to an optional length limit.
+     *
+     *  Note that Tokenizer cannot tell whether the prefix will
+     *  continue when/if more input data becomes available later.
      *
      * \retval true one or more characters were found, the sequence (string) is placed in returnedToken
      * \retval false no characters from the permitted set were found
      */
     bool prefix(SBuf &returnedToken, const CharacterSet &tokenChars, SBuf::size_type limit = SBuf::npos);
 
-    /** skips all sequential characters from the set, in any order
-     *
-     * \return whether one or more characters in the set were found
-     */
-    bool skip(const CharacterSet &tokenChars);
-
     /** skips a given character sequence (string)
      *
      * \return whether the exact character sequence was found and skipped
@@ -70,11 +70,23 @@
 
     /** skips a given single character
      *
-     * \return whether the character was found and skipped
+     * \return whether the character was skipped
      */
     bool skip(const char tokenChar);
 
-    /** parse an unsigned int64_t at the beginning of the buffer
+    /** Skips a single character from the set.
+     *
+     * \return whether a character was skipped
+     */
+    bool skipOne(const CharacterSet &discardables);
+
+    /** Skips all sequential characters from the set, in any order.
+     *
+     * \returns the number of skipped characters
+     */
+    SBuf::size_type skipAll(const CharacterSet &discardables);
+
+    /** Extracts an unsigned int64_t at the beginning of the buffer.
      *
      * strtoll(3)-alike function: tries to parse unsigned 64-bit integer
      * at the beginning of the parse buffer, in the base specified by the user
@@ -88,8 +100,13 @@
      */
     bool int64(int64_t &result, int base = 0);
 
+protected:
+    SBuf consume(const SBuf::size_type n);
+    SBuf::size_type success(const SBuf::size_type n);
+
 private:
     SBuf buf_; ///< yet unparsed input
+    SBuf::size_type parsed_; ///< bytes successfully parsed, including skipped
 };
 
 } /* namespace Parser */

=== modified file 'src/parser/testTokenizer.cc'
--- src/parser/testTokenizer.cc	2014-06-02 00:15:10 +0000
+++ src/parser/testTokenizer.cc	2014-08-07 17:31:05 +0000
@@ -60,8 +60,8 @@
     CPPUNIT_ASSERT(t.prefix(s,alpha));
     CPPUNIT_ASSERT_EQUAL(SBuf("GET"),s);
 
-    // test skip testing character set
-    CPPUNIT_ASSERT(t.skip(whitespace));
+    // test skipping one character from a character set
+    CPPUNIT_ASSERT(t.skipOne(whitespace));
     // check that skip was right
     CPPUNIT_ASSERT(t.prefix(s,alpha));
     CPPUNIT_ASSERT_EQUAL(SBuf("http"),s);
@@ -73,10 +73,14 @@
     CPPUNIT_ASSERT_EQUAL(SBuf("resource"),s);
 
     // no skip
-    CPPUNIT_ASSERT(!t.skip(alpha));
+    CPPUNIT_ASSERT(!t.skipOne(alpha));
     CPPUNIT_ASSERT(!t.skip(SBuf("://")));
     CPPUNIT_ASSERT(!t.skip('a'));
 
+    // test skipping all characters from a character set while looking at .com
+    CPPUNIT_ASSERT(t.skip('.'));
+    CPPUNIT_ASSERT_EQUAL(static_cast<SBuf::size_type>(3), t.skipAll(alpha));
+    CPPUNIT_ASSERT(t.remaining().startsWith(SBuf("/path")));
 }
 
 void

=== added directory 'src/servers'
=== added file 'src/servers/FtpServer.cc'
--- src/servers/FtpServer.cc	1970-01-01 00:00:00 +0000
+++ src/servers/FtpServer.cc	2014-08-10 23:18:33 +0000
@@ -0,0 +1,1614 @@
+/*
+ * DEBUG: section 33    Transfer protocol servers
+ */
+
+#include "squid.h"
+#include "base/CharacterSet.h"
+#include "base/RefCount.h"
+#include "base/Subscription.h"
+#include "client_side_reply.h"
+#include "client_side_request.h"
+#include "clientStream.h"
+#include "comm/ConnOpener.h"
+#include "comm/Read.h"
+#include "comm/TcpAcceptor.h"
+#include "comm/Write.h"
+#include "errorpage.h"
+#include "fd.h"
+#include "ftp/Elements.h"
+#include "ftp/Parsing.h"
+#include "globals.h"
+#include "HttpHdrCc.h"
+#include "ip/tools.h"
+#include "ipc/FdNotes.h"
+#include "parser/Tokenizer.h"
+#include "servers/forward.h"
+#include "servers/FtpServer.h"
+#include "SquidConfig.h"
+#include "StatCounters.h"
+#include "tools.h"
+
+#include <set>
+#include <map>
+
+CBDATA_NAMESPACED_CLASS_INIT(Ftp, Server);
+
+namespace Ftp
+{
+static void PrintReply(MemBuf &mb, const HttpReply *reply, const char *const prefix = "");
+static bool SupportedCommand(const SBuf &name);
+static bool CommandHasPathParameter(const SBuf &cmd);
+};
+
+Ftp::Server::Server(const MasterXaction::Pointer &xact):
+        AsyncJob("Ftp::Server"),
+        ConnStateData(xact),
+        master(new MasterState),
+        uri(),
+        host(),
+        gotEpsvAll(false),
+        onDataAcceptCall(),
+        dataListenConn(),
+        dataConn(),
+        uploadAvailSize(0),
+        listener(),
+        connector(),
+        reader()
+{
+    flags.readMore = false; // we need to announce ourselves first
+}
+
+Ftp::Server::~Server()
+{
+    closeDataConnection();
+}
+
+int
+Ftp::Server::pipelinePrefetchMax() const
+{
+    return 0; // no support for concurrent FTP requests
+}
+
+time_t
+Ftp::Server::idleTimeout() const
+{
+    return Config.Timeout.ftpClientIdle;
+}
+
+void
+Ftp::Server::start()
+{
+    ConnStateData::start();
+
+    if (transparent()) {
+        char buf[MAX_IPSTRLEN];
+        clientConnection->local.toUrl(buf, MAX_IPSTRLEN);
+        host = buf;
+        calcUri(NULL);
+        debugs(33, 5, "FTP transparent URL: " << uri);
+    }
+
+    writeEarlyReply(220, "Service ready");
+}
+
+/// schedules another data connection read if needed
+void
+Ftp::Server::maybeReadUploadData()
+{
+    if (reader != NULL)
+        return;
+
+    const size_t availSpace = sizeof(uploadBuf) - uploadAvailSize;
+    if (availSpace <= 0)
+        return;
+
+    debugs(33, 4, dataConn << ": reading FTP data...");
+
+    typedef CommCbMemFunT<Server, CommIoCbParams> Dialer;
+    reader = JobCallback(33, 5, Dialer, this, Ftp::Server::readUploadData);
+    comm_read(dataConn, uploadBuf + uploadAvailSize, availSpace,
+              reader);
+}
+
+/// react to the freshly parsed request
+void
+Ftp::Server::doProcessRequest()
+{
+    // zero pipelinePrefetchMax() ensures that there is only parsed request
+    ClientSocketContext::Pointer context = getCurrentContext();
+    Must(context != NULL);
+    Must(getConcurrentRequestCount() == 1);
+
+    ClientHttpRequest *const http = context->http;
+    assert(http != NULL);
+    HttpRequest *const request = http->request;
+    assert(request != NULL);
+    debugs(33, 9, request);
+
+    HttpHeader &header = request->header;
+    assert(header.has(HDR_FTP_COMMAND));
+    String &cmd = header.findEntry(HDR_FTP_COMMAND)->value;
+    assert(header.has(HDR_FTP_ARGUMENTS));
+    String &params = header.findEntry(HDR_FTP_ARGUMENTS)->value;
+
+    const bool fwd = !http->storeEntry() && handleRequest(cmd, params);
+
+    if (http->storeEntry() != NULL) {
+        debugs(33, 4, "got an immediate response");
+        assert(http->storeEntry() != NULL);
+        clientSetKeepaliveFlag(http);
+        context->pullData();
+    } else if (fwd) {
+        debugs(33, 4, "forwarding request to server side");
+        assert(http->storeEntry() == NULL);
+        clientProcessRequest(this, NULL /*parser*/, context.getRaw(),
+                             request->method, request->http_ver);
+    } else {
+        debugs(33, 4, "will resume processing later");
+    }
+}
+
+void
+Ftp::Server::processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &)
+{
+    // Process FTP request asynchronously to make sure FTP
+    // data connection accept callback is fired first.
+    CallJobHere(33, 4, CbcPointer<Server>(this),
+                Ftp::Server, doProcessRequest);
+}
+
+/// imports more upload data from the data connection
+void
+Ftp::Server::readUploadData(const CommIoCbParams &io)
+{
+    debugs(33, 5, io.conn << " size " << io.size);
+    Must(reader != NULL);
+    reader = NULL;
+
+    assert(Comm::IsConnOpen(dataConn));
+    assert(io.conn->fd == dataConn->fd);
+
+    if (io.flag == Comm::OK && bodyPipe != NULL) {
+        if (io.size > 0) {
+            kb_incr(&(statCounter.client_http.kbytes_in), io.size);
+
+            char *const current_buf = uploadBuf + uploadAvailSize;
+            if (io.buf != current_buf)
+                memmove(current_buf, io.buf, io.size);
+            uploadAvailSize += io.size;
+            shovelUploadData();
+        } else if (io.size == 0) {
+            debugs(33, 5, io.conn << " closed");
+            closeDataConnection();
+            if (uploadAvailSize <= 0)
+                finishDechunkingRequest(true);
+        }
+    } else { // not Comm::Flags::OK or unexpected read
+        debugs(33, 5, io.conn << " closed");
+        closeDataConnection();
+        finishDechunkingRequest(false);
+    }
+
+}
+
+/// shovel upload data from the internal buffer to the body pipe if possible
+void
+Ftp::Server::shovelUploadData()
+{
+    assert(bodyPipe != NULL);
+
+    debugs(33, 5, "handling FTP request data for " << clientConnection);
+    const size_t putSize = bodyPipe->putMoreData(uploadBuf,
+                           uploadAvailSize);
+    if (putSize > 0) {
+        uploadAvailSize -= putSize;
+        if (uploadAvailSize > 0)
+            memmove(uploadBuf, uploadBuf + putSize, uploadAvailSize);
+    }
+
+    if (Comm::IsConnOpen(dataConn))
+        maybeReadUploadData();
+    else if (uploadAvailSize <= 0)
+        finishDechunkingRequest(true);
+}
+
+void
+Ftp::Server::noteMoreBodySpaceAvailable(BodyPipe::Pointer)
+{
+    shovelUploadData();
+}
+
+void
+Ftp::Server::noteBodyConsumerAborted(BodyPipe::Pointer ptr)
+{
+    ConnStateData::noteBodyConsumerAborted(ptr);
+    closeDataConnection();
+}
+
+/// accept a new FTP control connection and hand it to a dedicated Server
+void
+Ftp::Server::AcceptCtrlConnection(const CommAcceptCbParams &params)
+{
+    MasterXaction::Pointer xact = params.xaction;
+    AnyP::PortCfgPointer s = xact->squidPort;
+
+    // NP: it is possible the port was reconfigured when the call or accept() was queued.
+
+    if (params.flag != Comm::OK) {
+        // Its possible the call was still queued when the client disconnected
+        debugs(33, 2, s->listenConn << ": FTP accept failure: " << xstrerr(params.xerrno));
+        return;
+    }
+
+    debugs(33, 4, params.conn << ": accepted");
+    fd_note(params.conn->fd, "client ftp connect");
+
+    if (s->tcp_keepalive.enabled)
+        commSetTcpKeepalive(params.conn->fd, s->tcp_keepalive.idle, s->tcp_keepalive.interval, s->tcp_keepalive.timeout);
+
+    ++incoming_sockets_accepted;
+
+    AsyncJob::Start(new Server(xact));
+}
+
+void
+Ftp::StartListening()
+{
+    for (AnyP::PortCfgPointer s = FtpPortList; s != NULL; s = s->next) {
+        if (MAXTCPLISTENPORTS == NHttpSockets) {
+            debugs(1, DBG_IMPORTANT, "Ignoring ftp_port lines exceeding the" <<
+                   " limit of " << MAXTCPLISTENPORTS << " ports.");
+            break;
+        }
+
+        // direct new connections accepted by listenConn to Accept()
+        typedef CommCbFunPtrCallT<CommAcceptCbPtrFun> AcceptCall;
+        RefCount<AcceptCall> subCall = commCbCall(5, 5, "Ftp::Server::AcceptCtrlConnection",
+                                       CommAcceptCbPtrFun(Ftp::Server::AcceptCtrlConnection,
+                                                          CommAcceptCbParams(NULL)));
+        clientStartListeningOn(s, subCall, Ipc::fdnFtpSocket);
+    }
+}
+
+void
+Ftp::StopListening()
+{
+    for (AnyP::PortCfgPointer s = HttpPortList; s != NULL; s = s->next) {
+        if (s->listenConn != NULL) {
+            debugs(1, DBG_IMPORTANT, "Closing FTP port " << s->listenConn->local);
+            s->listenConn->close();
+            s->listenConn = NULL;
+        }
+    }
+}
+
+void
+Ftp::Server::notePeerConnection(Comm::ConnectionPointer conn)
+{
+    // find request
+    ClientSocketContext::Pointer context = getCurrentContext();
+    Must(context != NULL);
+    ClientHttpRequest *const http = context->http;
+    Must(http != NULL);
+    HttpRequest *const request = http->request;
+    Must(request != NULL);
+
+    // this is not an idle connection, so we do not want I/O monitoring
+    const bool monitor = false;
+
+    // make FTP peer connection exclusive to our request
+    pinConnection(conn, request, conn->getPeer(), false, monitor);
+}
+
+void
+Ftp::Server::clientPinnedConnectionClosed(const CommCloseCbParams &io)
+{
+    ConnStateData::clientPinnedConnectionClosed(io);
+
+    // if the server control connection is gone, reset state to login again
+    resetLogin("control connection closure");
+
+    // XXX: Reseting is not enough. FtpRelay::sendCommand() will not re-login
+    // because FtpRelay::serverState() is not going to be fssConnected.
+}
+
+/// clear client and server login-related state after the old login is gone
+void
+Ftp::Server::resetLogin(const char *reason)
+{
+    debugs(33, 5, "will need to re-login due to " << reason);
+    master->clientReadGreeting = false;
+    changeState(fssBegin, reason);
+}
+
+/// computes uri member from host and, if tracked, working dir with file name
+void
+Ftp::Server::calcUri(const SBuf *file)
+{
+    uri = "ftp://";
+    uri.append(host);
+    if (port->ftp_track_dirs && master->workingDir.length()) {
+        if (master->workingDir[0] != '/')
+            uri.append("/");
+        uri.append(master->workingDir);
+    }
+
+    if (uri[uri.length() - 1] != '/')
+        uri.append("/");
+
+    if (port->ftp_track_dirs && file) {
+        static const CharacterSet Slash("/", "/");
+        Parser::Tokenizer tok(*file);
+        tok.skipAll(Slash);
+        uri.append(tok.remaining());
+    }
+}
+
+/// Starts waiting for a data connection. Returns listening port.
+/// On errors, responds with an error and returns zero.
+unsigned int
+Ftp::Server::listenForDataConnection()
+{
+    closeDataConnection();
+
+    Comm::ConnectionPointer conn = new Comm::Connection;
+    conn->flags = COMM_NONBLOCKING;
+    conn->local = transparent() ? port->s : clientConnection->local;
+    conn->local.port(0);
+    const char *const note = uri.c_str();
+    comm_open_listener(SOCK_STREAM, IPPROTO_TCP, conn, note);
+    if (!Comm::IsConnOpen(conn)) {
+        debugs(5, DBG_CRITICAL, "comm_open_listener failed for FTP data: " <<
+               conn->local << " error: " << errno);
+        writeCustomReply(451, "Internal error");
+        return 0;
+    }
+
+    typedef CommCbMemFunT<Server, CommAcceptCbParams> AcceptDialer;
+    typedef AsyncCallT<AcceptDialer> AcceptCall;
+    RefCount<AcceptCall> call = static_cast<AcceptCall*>(JobCallback(5, 5, AcceptDialer, this, Ftp::Server::acceptDataConnection));
+    Subscription::Pointer sub = new CallSubscription<AcceptCall>(call);
+    listener = call.getRaw();
+    dataListenConn = conn;
+    AsyncJob::Start(new Comm::TcpAcceptor(conn, note, sub));
+
+    const unsigned int listeningPort = comm_local_port(conn->fd);
+    conn->local.port(listeningPort);
+    return listeningPort;
+}
+
+void
+Ftp::Server::acceptDataConnection(const CommAcceptCbParams &params)
+{
+    if (params.flag != Comm::OK) {
+        // Its possible the call was still queued when the client disconnected
+        debugs(33, 2, dataListenConn << ": accept "
+               "failure: " << xstrerr(params.xerrno));
+        return;
+    }
+
+    debugs(33, 4, "accepted " << params.conn);
+    fd_note(params.conn->fd, "passive client ftp data");
+    ++incoming_sockets_accepted;
+
+    if (!clientConnection) {
+        debugs(33, 5, "late data connection?");
+        closeDataConnection(); // in case we are still listening
+        params.conn->close();
+    } else if (params.conn->remote != clientConnection->remote) {
+        debugs(33, 2, "rogue data conn? ctrl: " << clientConnection->remote);
+        params.conn->close();
+        // Some FTP servers close control connection here, but it may make
+        // things worse from DoS p.o.v. and no better from data stealing p.o.v.
+    } else {
+        closeDataConnection();
+        dataConn = params.conn;
+        uploadAvailSize = 0;
+        debugs(33, 7, "ready for data");
+        if (onDataAcceptCall != NULL) {
+            AsyncCall::Pointer call = onDataAcceptCall;
+            onDataAcceptCall = NULL;
+            // If we got an upload request, start reading data from the client.
+            if (master->serverState == fssHandleUploadRequest)
+                maybeReadUploadData();
+            else
+                Must(master->serverState == fssHandleDataRequest);
+            MemBuf mb;
+            mb.init();
+            mb.Printf("150 Data connection opened.\r\n");
+            Comm::Write(clientConnection, &mb, call);
+        }
+    }
+}
+
+void
+Ftp::Server::closeDataConnection()
+{
+    if (listener != NULL) {
+        listener->cancel("no longer needed");
+        listener = NULL;
+    }
+
+    if (Comm::IsConnOpen(dataListenConn)) {
+        debugs(33, 5, "FTP closing client data listen socket: " <<
+               *dataListenConn);
+        dataListenConn->close();
+    }
+    dataListenConn = NULL;
+
+    if (reader != NULL) {
+        // Comm::ReadCancel can deal with negative FDs
+        Comm::ReadCancel(dataConn->fd, reader);
+        reader = NULL;
+    }
+
+    if (Comm::IsConnOpen(dataConn)) {
+        debugs(33, 5, "FTP closing client data connection: " <<
+               *dataConn);
+        dataConn->close();
+    }
+    dataConn = NULL;
+}
+
+/// Writes FTP [error] response before we fully parsed the FTP request and
+/// created the corresponding HTTP request wrapper for that FTP request.
+void
+Ftp::Server::writeEarlyReply(const int code, const char *msg)
+{
+    debugs(33, 7, code << ' ' << msg);
+    assert(99 < code && code < 1000);
+
+    MemBuf mb;
+    mb.init();
+    mb.Printf("%i %s\r\n", code, msg);
+
+    typedef CommCbMemFunT<Server, CommIoCbParams> Dialer;
+    AsyncCall::Pointer call = JobCallback(33, 5, Dialer, this, Ftp::Server::wroteEarlyReply);
+    Comm::Write(clientConnection, &mb, call);
+
+    flags.readMore = false;
+
+    // TODO: Create master transaction. Log it in wroteEarlyReply().
+}
+
+void
+Ftp::Server::writeReply(MemBuf &mb)
+{
+    debugs(9, 2, "FTP Client " << clientConnection);
+    debugs(9, 2, "FTP Client REPLY:\n---------\n" << mb.buf <<
+           "\n----------");
+
+    typedef CommCbMemFunT<Server, CommIoCbParams> Dialer;
+    AsyncCall::Pointer call = JobCallback(33, 5, Dialer, this, Ftp::Server::wroteReply);
+    Comm::Write(clientConnection, &mb, call);
+}
+
+void
+Ftp::Server::writeCustomReply(const int code, const char *msg, const HttpReply *reply)
+{
+    debugs(33, 7, code << ' ' << msg);
+    assert(99 < code && code < 1000);
+
+    const bool sendDetails = reply != NULL &&
+                             reply->header.has(HDR_FTP_STATUS) && reply->header.has(HDR_FTP_REASON);
+
+    MemBuf mb;
+    mb.init();
+    if (sendDetails) {
+        mb.Printf("%i-%s\r\n", code, msg);
+        mb.Printf(" Server reply:\r\n");
+        Ftp::PrintReply(mb, reply, " ");
+        mb.Printf("%i \r\n", code);
+    } else
+        mb.Printf("%i %s\r\n", code, msg);
+
+    writeReply(mb);
+}
+
+void
+Ftp::Server::changeState(const ServerState newState, const char *reason)
+{
+    if (master->serverState == newState) {
+        debugs(33, 3, "client state unchanged at " << master->serverState <<
+               " because " << reason);
+        master->serverState = newState;
+    } else {
+        debugs(33, 3, "client state was " << master->serverState <<
+               ", now " << newState << " because " << reason);
+        master->serverState = newState;
+    }
+}
+
+/// whether the given FTP command has a pathname parameter
+static bool
+Ftp::CommandHasPathParameter(const SBuf &cmd)
+{
+    static std::set<SBuf> PathedCommands;
+    if (!PathedCommands.size()) {
+        PathedCommands.insert(cmdMlst());
+        PathedCommands.insert(cmdMlsd());
+        PathedCommands.insert(cmdStat());
+        PathedCommands.insert(cmdNlst());
+        PathedCommands.insert(cmdList());
+        PathedCommands.insert(cmdMkd());
+        PathedCommands.insert(cmdRmd());
+        PathedCommands.insert(cmdDele());
+        PathedCommands.insert(cmdRnto());
+        PathedCommands.insert(cmdRnfr());
+        PathedCommands.insert(cmdAppe());
+        PathedCommands.insert(cmdStor());
+        PathedCommands.insert(cmdRetr());
+        PathedCommands.insert(cmdSmnt());
+        PathedCommands.insert(cmdCwd());
+    }
+
+    return PathedCommands.find(cmd) != PathedCommands.end();
+}
+
+/// Parses a single FTP request on the control connection.
+/// Returns NULL on errors and incomplete requests.
+ClientSocketContext *
+Ftp::Server::parseOneRequest(Http::ProtocolVersion &ver)
+{
+    // OWS <command> [ RWS <parameter> ] OWS LF
+
+    // InlineSpaceChars are isspace(3) or RFC 959 Section 3.1.1.5.2, except
+    // for the LF character that we must exclude here (but see FullWhiteSpace).
+    static const char * const InlineSpaceChars = " \f\r\t\v";
+    static const CharacterSet InlineSpace = CharacterSet("Ftp::Inline", InlineSpaceChars);
+    static const CharacterSet FullWhiteSpace = (InlineSpace + CharacterSet::LF).rename("Ftp::FWS");
+    static const CharacterSet CommandChars = FullWhiteSpace.complement("Ftp::Command");
+    static const CharacterSet TailChars = CharacterSet::LF.complement("Ftp::Tail");
+
+    // This set is used to ignore empty commands without allowing an attacker
+    // to keep us endlessly busy by feeding us whitespace or empty commands.
+    static const CharacterSet &LeadingSpace = FullWhiteSpace;
+
+    SBuf cmd;
+    SBuf params;
+
+    Parser::Tokenizer tok(in.buf);
+
+    (void)tok.skipAll(LeadingSpace); // leading OWS and empty commands
+    const bool parsed = tok.prefix(cmd, CommandChars); // required command
+
+    // note that the condition below will eat either RWS or trailing OWS
+    if (parsed && tok.skipAll(InlineSpace) && tok.prefix(params, TailChars)) {
+        // now params may include trailing OWS
+        // TODO: Support right-trimming using CharacterSet in Tokenizer instead
+        static const SBuf bufWhiteSpace(InlineSpaceChars);
+        params.trim(bufWhiteSpace, false, true);
+    }
+
+    // technically, we may skip multiple NLs below, but that is OK
+    if (!parsed || !tok.skipAll(CharacterSet::LF)) { // did not find terminating LF yet
+        // we need more data, but can we buffer more?
+        if (in.buf.length() >= Config.maxRequestHeaderSize) {
+            changeState(fssError, "huge req");
+            writeEarlyReply(421, "Huge request");
+            return NULL;
+        } else {
+            debugs(33, 5, "Waiting for more, up to " <<
+                   (Config.maxRequestHeaderSize - in.buf.length()));
+            return NULL;
+        }
+    }
+
+    Must(parsed && cmd.length());
+    consumeInput(tok.parsedSize()); // TODO: Would delaying optimize copying?
+
+    debugs(33, 2, ">>ftp " << cmd << (params.isEmpty() ? "" : " ") << params);
+
+    cmd.toUpper(); // this should speed up and simplify future comparisons
+
+    // interception cases do not need USER to calculate the uri
+    if (!transparent()) {
+        if (!master->clientReadGreeting) {
+            // the first command must be USER
+            if (!pinning.pinned && cmd != cmdUser()) {
+                writeEarlyReply(530, "Must login first");
+                return NULL;
+            }
+        }
+
+        // process USER request now because it sets FTP peer host name
+        if (cmd == cmdUser() && !handleUserRequest(cmd, params))
+            return NULL;
+    }
+
+    if (!Ftp::SupportedCommand(cmd)) {
+        writeEarlyReply(502, "Unknown or unsupported command");
+        return NULL;
+    }
+
+    const HttpRequestMethod method =
+        cmd == cmdAppe() || cmd == cmdStor() || cmd == cmdStou() ?
+        Http::METHOD_PUT : Http::METHOD_GET;
+
+    const SBuf *path = (params.length() && CommandHasPathParameter(cmd)) ?
+                       &params : NULL;
+    calcUri(path);
+    char *newUri = xstrdup(uri.c_str());
+    HttpRequest *const request = HttpRequest::CreateFromUrlAndMethod(newUri, method);
+    if (!request) {
+        debugs(33, 5, "Invalid FTP URL: " << uri);
+        writeEarlyReply(501, "Invalid host");
+        uri.clear();
+        safe_free(newUri);
+        return NULL;
+    }
+
+    ver = Http::ProtocolVersion(Ftp::ProtocolVersion().major, Ftp::ProtocolVersion().minor);
+    request->flags.ftpNative = true;
+    request->http_ver = ver;
+
+    // Our fake Request-URIs are not distinctive enough for caching to work
+    request->flags.cachable = false; // XXX: reset later by maybeCacheable()
+    request->flags.noCache = true;
+
+    request->header.putStr(HDR_FTP_COMMAND, cmd.c_str());
+    request->header.putStr(HDR_FTP_ARGUMENTS, params.c_str()); // may be ""
+    if (method == Http::METHOD_PUT) {
+        request->header.putStr(HDR_EXPECT, "100-continue");
+        request->header.putStr(HDR_TRANSFER_ENCODING, "chunked");
+    }
+
+    ClientHttpRequest *const http = new ClientHttpRequest(this);
+    http->request = request;
+    HTTPMSGLOCK(http->request);
+    http->req_sz = tok.parsedSize();
+    http->uri = newUri;
+
+    ClientSocketContext *const result =
+        new ClientSocketContext(clientConnection, http);
+
+    StoreIOBuffer tempBuffer;
+    tempBuffer.data = result->reqbuf;
+    tempBuffer.length = HTTP_REQBUF_SZ;
+
+    ClientStreamData newServer = new clientReplyContext(http);
+    ClientStreamData newClient = result;
+    clientStreamInit(&http->client_stream, clientGetMoreData, clientReplyDetach,
+                     clientReplyStatus, newServer, clientSocketRecipient,
+                     clientSocketDetach, newClient, tempBuffer);
+
+    Must(!getConcurrentRequestCount());
+    result->registerWithConn();
+    result->flags.parsed_ok = 1;
+    flags.readMore = false;
+    return result;
+}
+
+void
+Ftp::Server::handleReply(HttpReply *reply, StoreIOBuffer data)
+{
+    // the caller guarantees that we are dealing with the current context only
+    ClientSocketContext::Pointer context = getCurrentContext();
+    assert(context != NULL);
+
+    if (context->http && context->http->al != NULL &&
+            !context->http->al->reply && reply) {
+        context->http->al->reply = reply;
+        HTTPMSGLOCK(context->http->al->reply);
+    }
+
+    static ReplyHandler handlers[] = {
+        NULL, // fssBegin
+        NULL, // fssConnected
+        &Ftp::Server::handleFeatReply, // fssHandleFeat
+        &Ftp::Server::handlePasvReply, // fssHandlePasv
+        &Ftp::Server::handlePortReply, // fssHandlePort
+        &Ftp::Server::handleDataReply, // fssHandleDataRequest
+        &Ftp::Server::handleUploadReply, // fssHandleUploadRequest
+        &Ftp::Server::handleEprtReply,// fssHandleEprt
+        &Ftp::Server::handleEpsvReply,// fssHandleEpsv
+        NULL, // fssHandleCwd
+        NULL, // fssHandlePass
+        NULL, // fssHandleCdup
+        &Ftp::Server::handleErrorReply // fssError
+    };
+    const Server &server = dynamic_cast<const Ftp::Server&>(*context->getConn());
+    if (const ReplyHandler handler = handlers[server.master->serverState])
+        (this->*handler)(reply, data);
+    else
+        writeForwardedReply(reply);
+}
+
+void
+Ftp::Server::handleFeatReply(const HttpReply *reply, StoreIOBuffer)
+{
+    if (getCurrentContext()->http->request->errType != ERR_NONE) {
+        writeCustomReply(502, "Server does not support FEAT", reply);
+        return;
+    }
+
+    HttpReply *filteredReply = reply->clone();
+    HttpHeader &filteredHeader = filteredReply->header;
+
+    // Remove all unsupported commands from the response wrapper.
+    int deletedCount = 0;
+    HttpHeaderPos pos = HttpHeaderInitPos;
+    bool hasEPRT = false;
+    bool hasEPSV = false;
+    int prependSpaces = 1;
+    while (const HttpHeaderEntry *e = filteredHeader.getEntry(&pos)) {
+        if (e->id == HDR_FTP_PRE) {
+            // assume RFC 2389 FEAT response format, quoted by Squid:
+            // <"> SP NAME [SP PARAMS] <">
+            // but accommodate MS servers sending four SPs before NAME
+
+            // command name ends with (SP parameter) or quote
+            static const CharacterSet AfterFeatNameChars("AfterFeatName", " \"");
+            static const CharacterSet FeatNameChars = AfterFeatNameChars.complement("FeatName");
+
+            Parser::Tokenizer tok(SBuf(e->value.termedBuf()));
+            if (!tok.skip('"') && !tok.skip(' '))
+                continue;
+
+            // optional spaces; remember their number to accomodate MS servers
+            prependSpaces = 1 + tok.skipAll(CharacterSet::SP);
+
+            SBuf cmd;
+            if (!tok.prefix(cmd, FeatNameChars))
+                continue;
+            cmd.toUpper();
+
+            if (!Ftp::SupportedCommand(cmd))
+                filteredHeader.delAt(pos, deletedCount);
+
+            if (cmd == cmdEprt())
+                hasEPRT = true;
+            else if (cmd == cmdEpsv())
+                hasEPSV = true;
+        }
+    }
+
+    char buf[256];
+    int insertedCount = 0;
+    if (!hasEPRT) {
+        snprintf(buf, sizeof(buf), "\"%*s\"", prependSpaces + 4, "EPRT");
+        filteredHeader.putStr(HDR_FTP_PRE, buf);
+        ++insertedCount;
+    }
+    if (!hasEPSV) {
+        snprintf(buf, sizeof(buf), "\"%*s\"", prependSpaces + 4, "EPSV");
+        filteredHeader.putStr(HDR_FTP_PRE, buf);
+        ++insertedCount;
+    }
+
+    if (deletedCount || insertedCount) {
+        filteredHeader.refreshMask();
+        debugs(33, 5, "deleted " << deletedCount << " inserted " << insertedCount);
+    }
+
+    writeForwardedReply(filteredReply);
+}
+
+void
+Ftp::Server::handlePasvReply(const HttpReply *reply, StoreIOBuffer)
+{
+    ClientSocketContext::Pointer context = getCurrentContext();
+    assert(context != NULL);
+
+    if (context->http->request->errType != ERR_NONE) {
+        writeCustomReply(502, "Server does not support PASV", reply);
+        return;
+    }
+
+    const unsigned short localPort = listenForDataConnection();
+    if (!localPort)
+        return;
+
+    char addr[MAX_IPSTRLEN];
+    // remote server in interception setups and local address otherwise
+    const Ip::Address &server = transparent() ?
+                                clientConnection->local : dataListenConn->local;
+    server.toStr(addr, MAX_IPSTRLEN, AF_INET);
+    addr[MAX_IPSTRLEN - 1] = '\0';
+    for (char *c = addr; *c != '\0'; ++c) {
+        if (*c == '.')
+            *c = ',';
+    }
+
+    // In interception setups, we combine remote server address with a
+    // local port number and hope that traffic will be redirected to us.
+    // Do not use "227 =a,b,c,d,p1,p2" format or omit parens: some nf_ct_ftp
+    // versions block responses that use those alternative syntax rules!
+    MemBuf mb;
+    mb.init();
+    mb.Printf("227 Entering Passive Mode (%s,%i,%i).\r\n",
+              addr,
+              static_cast<int>(localPort / 256),
+              static_cast<int>(localPort % 256));
+    debugs(9, 3, Raw("writing", mb.buf, mb.size));
+    writeReply(mb);
+}
+
+void
+Ftp::Server::handlePortReply(const HttpReply *reply, StoreIOBuffer)
+{
+    if (getCurrentContext()->http->request->errType != ERR_NONE) {
+        writeCustomReply(502, "Server does not support PASV (converted from PORT)", reply);
+        return;
+    }
+
+    writeCustomReply(200, "PORT successfully converted to PASV.");
+
+    // and wait for RETR
+}
+
+void
+Ftp::Server::handleErrorReply(const HttpReply *reply, StoreIOBuffer)
+{
+    if (!pinning.pinned) // we failed to connect to server
+        uri.clear();
+    // 421: we will close due to fssError
+    writeErrorReply(reply, 421);
+}
+
+void
+Ftp::Server::handleDataReply(const HttpReply *reply, StoreIOBuffer data)
+{
+    if (reply != NULL && reply->sline.status() != Http::scOkay) {
+        writeForwardedReply(reply);
+        if (Comm::IsConnOpen(dataConn)) {
+            debugs(33, 3, "closing " << dataConn << " on KO reply");
+            closeDataConnection();
+        }
+        return;
+    }
+
+    if (!dataConn) {
+        // We got STREAM_COMPLETE (or error) and closed the client data conn.
+        debugs(33, 3, "ignoring FTP srv data response after clt data closure");
+        return;
+    }
+
+    if (!checkDataConnPost()) {
+        writeCustomReply(425, "Data connection is not established.");
+        closeDataConnection();
+        return;
+    }
+
+    debugs(33, 7, data.length);
+
+    if (data.length <= 0) {
+        replyDataWritingCheckpoint(); // skip the actual write call
+        return;
+    }
+
+    MemBuf mb;
+    mb.init(data.length + 1, data.length + 1);
+    mb.append(data.data, data.length);
+
+    typedef CommCbMemFunT<Server, CommIoCbParams> Dialer;
+    AsyncCall::Pointer call = JobCallback(33, 5, Dialer, this, Ftp::Server::wroteReplyData);
+    Comm::Write(dataConn, &mb, call);
+
+    getCurrentContext()->noteSentBodyBytes(data.length);
+}
+
+/// called when we are done writing a chunk of the response data
+void
+Ftp::Server::wroteReplyData(const CommIoCbParams &io)
+{
+    if (io.flag == Comm::ERR_CLOSING)
+        return;
+
+    if (io.flag != Comm::OK) {
+        debugs(33, 3, "FTP reply data writing failed: " << xstrerr(io.xerrno));
+        closeDataConnection();
+        writeCustomReply(426, "Data connection error; transfer aborted");
+        return;
+    }
+
+    assert(getCurrentContext()->http);
+    getCurrentContext()->http->out.size += io.size;
+    replyDataWritingCheckpoint();
+}
+
+/// ClientStream checks after (actual or skipped) reply data writing
+void
+Ftp::Server::replyDataWritingCheckpoint()
+{
+    switch (getCurrentContext()->socketState()) {
+    case STREAM_NONE:
+        debugs(33, 3, "Keep going");
+        getCurrentContext()->pullData();
+        return;
+    case STREAM_COMPLETE:
+        debugs(33, 3, "FTP reply data transfer successfully complete");
+        writeCustomReply(226, "Transfer complete");
+        break;
+    case STREAM_UNPLANNED_COMPLETE:
+        debugs(33, 3, "FTP reply data transfer failed: STREAM_UNPLANNED_COMPLETE");
+        writeCustomReply(451, "Server error; transfer aborted");
+        break;
+    case STREAM_FAILED:
+        debugs(33, 3, "FTP reply data transfer failed: STREAM_FAILED");
+        writeCustomReply(451, "Server error; transfer aborted");
+        break;
+    default:
+        fatal("unreachable code");
+    }
+
+    closeDataConnection();
+}
+
+void
+Ftp::Server::handleUploadReply(const HttpReply *reply, StoreIOBuffer)
+{
+    writeForwardedReply(reply);
+    // note that the client data connection may already be closed by now
+}
+
+void
+Ftp::Server::writeForwardedReply(const HttpReply *reply)
+{
+    assert(reply != NULL);
+    const HttpHeader &header = reply->header;
+    // adaptation and forwarding errors lack HDR_FTP_STATUS
+    if (!header.has(HDR_FTP_STATUS)) {
+        writeForwardedForeign(reply); // will get to Ftp::Server::wroteReply
+        return;
+    }
+
+    typedef CommCbMemFunT<Server, CommIoCbParams> Dialer;
+    AsyncCall::Pointer call = JobCallback(33, 5, Dialer, this, Ftp::Server::wroteReply);
+    writeForwardedReplyAndCall(reply, call);
+}
+
+void
+Ftp::Server::handleEprtReply(const HttpReply *reply, StoreIOBuffer)
+{
+    if (getCurrentContext()->http->request->errType != ERR_NONE) {
+        writeCustomReply(502, "Server does not support PASV (converted from EPRT)", reply);
+        return;
+    }
+
+    writeCustomReply(200, "EPRT successfully converted to PASV.");
+
+    // and wait for RETR
+}
+
+void
+Ftp::Server::handleEpsvReply(const HttpReply *reply, StoreIOBuffer)
+{
+    if (getCurrentContext()->http->request->errType != ERR_NONE) {
+        writeCustomReply(502, "Cannot connect to server", reply);
+        return;
+    }
+
+    const unsigned short localPort = listenForDataConnection();
+    if (!localPort)
+        return;
+
+    // In interception setups, we use a local port number and hope that data
+    // traffic will be redirected to us.
+    MemBuf mb;
+    mb.init();
+    mb.Printf("229 Entering Extended Passive Mode (|||%u|)\r\n", localPort);
+
+    debugs(9, 3, Raw("writing", mb.buf, mb.size));
+    writeReply(mb);
+}
+
+/// writes FTP error response with given status and reply-derived error details
+void
+Ftp::Server::writeErrorReply(const HttpReply *reply, const int scode)
+{
+    const HttpRequest *request = getCurrentContext()->http->request;
+    assert(request);
+
+    MemBuf mb;
+    mb.init();
+
+    if (request->errType != ERR_NONE)
+        mb.Printf("%i-%s\r\n", scode, errorPageName(request->errType));
+
+    if (request->errDetail > 0) {
+        // XXX: > 0 may not always mean that this is an errno
+        mb.Printf("%i-Error: (%d) %s\r\n", scode,
+                  request->errDetail,
+                  strerror(request->errDetail));
+    }
+
+    // XXX: Remove hard coded names. Use an error page template instead.
+    const Adaptation::History::Pointer ah = request->adaptHistory();
+    if (ah != NULL) { // XXX: add adapt::<all_h but use lastMeta here
+        const String info = ah->allMeta.getByName("X-Response-Info");
+        const String desc = ah->allMeta.getByName("X-Response-Desc");
+        if (info.size())
+            mb.Printf("%i-Information: %s\r\n", scode, info.termedBuf());
+        if (desc.size())
+            mb.Printf("%i-Description: %s\r\n", scode, desc.termedBuf());
+    }
+
+    assert(reply != NULL);
+    const char *reason = reply->header.has(HDR_FTP_REASON) ?
+                         reply->header.getStr(HDR_FTP_REASON):
+                         reply->sline.reason();
+
+    mb.Printf("%i %s\r\n", scode, reason); // error terminating line
+
+    // TODO: errorpage.cc should detect FTP client and use
+    // configurable FTP-friendly error templates which we should
+    // write to the client "as is" instead of hiding most of the info
+
+    writeReply(mb);
+}
+
+/// writes FTP response based on HTTP reply that is not an FTP-response wrapper
+void
+Ftp::Server::writeForwardedForeign(const HttpReply *reply)
+{
+    changeState(fssConnected, "foreign reply");
+    closeDataConnection();
+    // 451: We intend to keep the control connection open.
+    writeErrorReply(reply, 451);
+}
+
+void
+Ftp::Server::writeControlMsgAndCall(ClientSocketContext *context, HttpReply *reply, AsyncCall::Pointer &call)
+{
+    // the caller guarantees that we are dealing with the current context only
+    // the caller should also make sure reply->header.has(HDR_FTP_STATUS)
+    writeForwardedReplyAndCall(reply, call);
+}
+
+void
+Ftp::Server::writeForwardedReplyAndCall(const HttpReply *reply, AsyncCall::Pointer &call)
+{
+    assert(reply != NULL);
+    const HttpHeader &header = reply->header;
+
+    // without status, the caller must use the writeForwardedForeign() path
+    Must(header.has(HDR_FTP_STATUS));
+    Must(header.has(HDR_FTP_REASON));
+    const int scode = header.getInt(HDR_FTP_STATUS);
+    debugs(33, 7, "scode: " << scode);
+
+    // Status 125 or 150 implies upload or data request, but we still check
+    // the state in case the server is buggy.
+    if ((scode == 125 || scode == 150) &&
+            (master->serverState == fssHandleUploadRequest ||
+             master->serverState == fssHandleDataRequest)) {
+        if (checkDataConnPost()) {
+            // If the data connection is ready, start reading data (here)
+            // and forward the response to client (further below).
+            debugs(33, 7, "data connection established, start data transfer");
+            if (master->serverState == fssHandleUploadRequest)
+                maybeReadUploadData();
+        } else {
+            // If we are waiting to accept the data connection, keep waiting.
+            if (Comm::IsConnOpen(dataListenConn)) {
+                debugs(33, 7, "wait for the client to establish a data connection");
+                onDataAcceptCall = call;
+                // TODO: Add connect timeout for passive connections listener?
+                // TODO: Remember server response so that we can forward it?
+            } else {
+                // Either the connection was establised and closed after the
+                // data was transferred OR we failed to establish an active
+                // data connection and already sent the error to the client.
+                // In either case, there is nothing more to do.
+                debugs(33, 7, "done with data OR active connection failed");
+            }
+            return;
+        }
+    }
+
+    MemBuf mb;
+    mb.init();
+    Ftp::PrintReply(mb, reply);
+
+    debugs(9, 2, "FTP Client " << clientConnection);
+    debugs(9, 2, "FTP Client REPLY:\n---------\n" << mb.buf <<
+           "\n----------");
+
+    Comm::Write(clientConnection, &mb, call);
+}
+
+static void
+Ftp::PrintReply(MemBuf &mb, const HttpReply *reply, const char *const prefix)
+{
+    const HttpHeader &header = reply->header;
+
+    HttpHeaderPos pos = HttpHeaderInitPos;
+    while (const HttpHeaderEntry *e = header.getEntry(&pos)) {
+        if (e->id == HDR_FTP_PRE) {
+            String raw;
+            if (httpHeaderParseQuotedString(e->value.rawBuf(), e->value.size(), &raw))
+                mb.Printf("%s\r\n", raw.termedBuf());
+        }
+    }
+
+    if (header.has(HDR_FTP_STATUS)) {
+        const char *reason = header.getStr(HDR_FTP_REASON);
+        mb.Printf("%i %s\r\n", header.getInt(HDR_FTP_STATUS),
+                  (reason ? reason : 0));
+    }
+}
+
+void
+Ftp::Server::wroteEarlyReply(const CommIoCbParams &io)
+{
+    if (io.flag == Comm::ERR_CLOSING)
+        return;
+
+    if (io.flag != Comm::OK) {
+        debugs(33, 3, "FTP reply writing failed: " << xstrerr(io.xerrno));
+        io.conn->close();
+        return;
+    }
+
+    ClientSocketContext::Pointer context = getCurrentContext();
+    if (context != NULL && context->http) {
+        context->http->out.size += io.size;
+        context->http->out.headers_sz += io.size;
+    }
+
+    flags.readMore = true;
+    readSomeData();
+}
+
+void
+Ftp::Server::wroteReply(const CommIoCbParams &io)
+{
+    if (io.flag == Comm::ERR_CLOSING)
+        return;
+
+    if (io.flag != Comm::OK) {
+        debugs(33, 3, "FTP reply writing failed: " << xstrerr(io.xerrno));
+        io.conn->close();
+        return;
+    }
+
+    ClientSocketContext::Pointer context = getCurrentContext();
+    assert(context->http);
+    context->http->out.size += io.size;
+    context->http->out.headers_sz += io.size;
+
+    if (master->serverState == fssError) {
+        debugs(33, 5, "closing on FTP server error");
+        io.conn->close();
+        return;
+    }
+
+    const clientStream_status_t socketState = context->socketState();
+    debugs(33, 5, "FTP client stream state " << socketState);
+    switch (socketState) {
+    case STREAM_UNPLANNED_COMPLETE:
+    case STREAM_FAILED:
+        io.conn->close();
+        return;
+
+    case STREAM_NONE:
+    case STREAM_COMPLETE:
+        flags.readMore = true;
+        changeState(fssConnected, "Ftp::Server::wroteReply");
+        if (in.bodyParser)
+            finishDechunkingRequest(false);
+        context->keepaliveNextRequest();
+        return;
+    }
+}
+
+bool
+Ftp::Server::handleRequest(String &cmd, String &params)
+{
+    HttpRequest *request = getCurrentContext()->http->request;
+    Must(request);
+
+    if (do_debug(9, 2)) {
+        MemBuf mb;
+        Packer p;
+        mb.init();
+        packerToMemInit(&p, &mb);
+        request->pack(&p);
+        packerClean(&p);
+
+        debugs(9, 2, "FTP Client " << clientConnection);
+        debugs(9, 2, "FTP Client REQUEST:\n---------\n" << mb.buf <<
+               "\n----------");
+    }
+
+    // TODO: When HttpHeader uses SBuf, change keys to SBuf
+    typedef std::map<const std::string, RequestHandler> RequestHandlers;
+    static RequestHandlers handlers;
+    if (!handlers.size()) {
+        handlers["LIST"] = &Ftp::Server::handleDataRequest;
+        handlers["NLST"] = &Ftp::Server::handleDataRequest;
+        handlers["MLSD"] = &Ftp::Server::handleDataRequest;
+        handlers["FEAT"] = &Ftp::Server::handleFeatRequest;
+        handlers["PASV"] = &Ftp::Server::handlePasvRequest;
+        handlers["PORT"] = &Ftp::Server::handlePortRequest;
+        handlers["RETR"] = &Ftp::Server::handleDataRequest;
+        handlers["EPRT"] = &Ftp::Server::handleEprtRequest;
+        handlers["EPSV"] = &Ftp::Server::handleEpsvRequest;
+        handlers["CWD"] = &Ftp::Server::handleCwdRequest;
+        handlers["PASS"] = &Ftp::Server::handlePassRequest;
+        handlers["CDUP"] = &Ftp::Server::handleCdupRequest;
+    }
+
+    RequestHandler handler = NULL;
+    if (request->method == Http::METHOD_PUT)
+        handler = &Ftp::Server::handleUploadRequest;
+    else {
+        const RequestHandlers::const_iterator hi = handlers.find(cmd.termedBuf());
+        if (hi != handlers.end())
+            handler = hi->second;
+    }
+
+    if (!handler) {
+        debugs(9, 7, "forwarding " << cmd << " as is, no post-processing");
+        return true;
+    }
+
+    return (this->*handler)(cmd, params);
+}
+
+/// Called to parse USER command, which is required to create an HTTP request
+/// wrapper. Thus, errors are handled with writeEarlyReply() here.
+bool
+Ftp::Server::handleUserRequest(const SBuf &cmd, SBuf &params)
+{
+    if (params.isEmpty()) {
+        writeEarlyReply(501, "Missing username");
+        return false;
+    }
+
+    // find the [end of] user name
+    const SBuf::size_type eou = params.rfind('@');
+    if (eou == SBuf::npos || eou + 1 >= params.length()) {
+        writeEarlyReply(501, "Missing host");
+        return false;
+    }
+
+    // Determine the intended destination.
+    host = params.substr(eou + 1, params.length());
+    // If we can parse it as raw IPv6 address, then surround with "[]".
+    // Otherwise (domain, IPv4, [bracketed] IPv6, garbage, etc), use as is.
+    if (host.find(':') != SBuf::npos) {
+        const Ip::Address ipa(host.c_str());
+        if (!ipa.isAnyAddr()) {
+            char ipBuf[MAX_IPSTRLEN];
+            ipa.toHostStr(ipBuf, MAX_IPSTRLEN);
+            host = ipBuf;
+        }
+    }
+
+    // const SBuf login = params.substr(0, eou);
+    params.chop(0, eou); // leave just the login part for the peer
+
+    SBuf oldUri;
+    if (master->clientReadGreeting)
+        oldUri = uri;
+
+    master->workingDir.clear();
+    calcUri(NULL);
+
+    if (!master->clientReadGreeting) {
+        debugs(9, 3, "set URI to " << uri);
+    } else if (oldUri.caseCmp(uri) == 0) {
+        debugs(9, 5, "kept URI as " << oldUri);
+    } else {
+        debugs(9, 3, "reset URI from " << oldUri << " to " << uri);
+        closeDataConnection();
+        unpinConnection(true); // close control connection to peer
+        resetLogin("URI reset");
+    }
+
+    return true;
+}
+
+bool
+Ftp::Server::handleFeatRequest(String &cmd, String &params)
+{
+    changeState(fssHandleFeat, "handleFeatRequest");
+    return true;
+}
+
+bool
+Ftp::Server::handlePasvRequest(String &cmd, String &params)
+{
+    if (gotEpsvAll) {
+        setReply(500, "Bad PASV command");
+        return false;
+    }
+
+    if (params.size() > 0) {
+        setReply(501, "Unexpected parameter");
+        return false;
+    }
+
+    changeState(fssHandlePasv, "handlePasvRequest");
+    // no need to fake PASV request via setDataCommand() in true PASV case
+    return true;
+}
+
+/// [Re]initializes dataConn for active data transfers. Does not connect.
+bool
+Ftp::Server::createDataConnection(Ip::Address cltAddr)
+{
+    assert(clientConnection != NULL);
+    assert(!clientConnection->remote.isAnyAddr());
+
+    if (cltAddr != clientConnection->remote) {
+        debugs(33, 2, "rogue PORT " << cltAddr << " request? ctrl: " << clientConnection->remote);
+        // Closing the control connection would not help with attacks because
+        // the client is evidently able to connect to us. Besides, closing
+        // makes retrials easier for the client and more damaging to us.
+        setReply(501, "Prohibited parameter value");
+        return false;
+    }
+
+    closeDataConnection();
+
+    Comm::ConnectionPointer conn = new Comm::Connection();
+    conn->flags |= COMM_DOBIND;
+
+    // Use local IP address of the control connection as the source address
+    // of the active data connection, or some clients will refuse to accept.
+    conn->setAddrs(clientConnection->local, cltAddr);
+    // RFC 959 requires active FTP connections to originate from port 20
+    // but that would preclude us from supporting concurrent transfers! (XXX?)
+    conn->local.port(0);
+
+    debugs(9, 3, "will actively connect from " << conn->local << " to " <<
+           conn->remote);
+
+    dataConn = conn;
+    uploadAvailSize = 0;
+    return true;
+}
+
+bool
+Ftp::Server::handlePortRequest(String &cmd, String &params)
+{
+    // TODO: Should PORT errors trigger closeDataConnection() cleanup?
+
+    if (gotEpsvAll) {
+        setReply(500, "Rejecting PORT after EPSV ALL");
+        return false;
+    }
+
+    if (!params.size()) {
+        setReply(501, "Missing parameter");
+        return false;
+    }
+
+    Ip::Address cltAddr;
+    if (!Ftp::ParseIpPort(params.termedBuf(), NULL, cltAddr)) {
+        setReply(501, "Invalid parameter");
+        return false;
+    }
+
+    if (!createDataConnection(cltAddr))
+        return false;
+
+    changeState(fssHandlePort, "handlePortRequest");
+    setDataCommand();
+    return true; // forward our fake PASV request
+}
+
+bool
+Ftp::Server::handleDataRequest(String &cmd, String &params)
+{
+    if (!checkDataConnPre())
+        return false;
+
+    changeState(fssHandleDataRequest, "handleDataRequest");
+
+    return true;
+}
+
+bool
+Ftp::Server::handleUploadRequest(String &cmd, String &params)
+{
+    if (!checkDataConnPre())
+        return false;
+
+    changeState(fssHandleUploadRequest, "handleDataRequest");
+
+    return true;
+}
+
+bool
+Ftp::Server::handleEprtRequest(String &cmd, String &params)
+{
+    debugs(9, 3, "Process an EPRT " << params);
+
+    if (gotEpsvAll) {
+        setReply(500, "Rejecting EPRT after EPSV ALL");
+        return false;
+    }
+
+    if (!params.size()) {
+        setReply(501, "Missing parameter");
+        return false;
+    }
+
+    Ip::Address cltAddr;
+    if (!Ftp::ParseProtoIpPort(params.termedBuf(), cltAddr)) {
+        setReply(501, "Invalid parameter");
+        return false;
+    }
+
+    if (!createDataConnection(cltAddr))
+        return false;
+
+    changeState(fssHandleEprt, "handleEprtRequest");
+    setDataCommand();
+    return true; // forward our fake PASV request
+}
+
+bool
+Ftp::Server::handleEpsvRequest(String &cmd, String &params)
+{
+    debugs(9, 3, "Process an EPSV command with params: " << params);
+    if (params.size() <= 0) {
+        // treat parameterless EPSV as "use the protocol of the ctrl conn"
+    } else if (params.caseCmp("ALL") == 0) {
+        setReply(200, "EPSV ALL ok");
+        gotEpsvAll = true;
+        return false;
+    } else if (params.cmp("2") == 0) {
+        if (!Ip::EnableIpv6) {
+            setReply(522, "Network protocol not supported, use (1)");
+            return false;
+        }
+    } else if (params.cmp("1") != 0) {
+        setReply(501, "Unsupported EPSV parameter");
+        return false;
+    }
+
+    changeState(fssHandleEpsv, "handleEpsvRequest");
+    setDataCommand();
+    return true; // forward our fake PASV request
+}
+
+bool
+Ftp::Server::handleCwdRequest(String &cmd, String &params)
+{
+    changeState(fssHandleCwd, "handleCwdRequest");
+    return true;
+}
+
+bool
+Ftp::Server::handlePassRequest(String &cmd, String &params)
+{
+    changeState(fssHandlePass, "handlePassRequest");
+    return true;
+}
+
+bool
+Ftp::Server::handleCdupRequest(String &cmd, String &params)
+{
+    changeState(fssHandleCdup, "handleCdupRequest");
+    return true;
+}
+
+// Convert user PORT, EPRT, PASV, or EPSV data command to Squid PASV command.
+// Squid FTP client decides what data command to use with peers.
+void
+Ftp::Server::setDataCommand()
+{
+    ClientHttpRequest *const http = getCurrentContext()->http;
+    assert(http != NULL);
+    HttpRequest *const request = http->request;
+    assert(request != NULL);
+    HttpHeader &header = request->header;
+    header.delById(HDR_FTP_COMMAND);
+    header.putStr(HDR_FTP_COMMAND, "PASV");
+    header.delById(HDR_FTP_ARGUMENTS);
+    header.putStr(HDR_FTP_ARGUMENTS, "");
+    debugs(9, 5, "client data command converted to fake PASV");
+}
+
+/// check that client data connection is ready for future I/O or at least
+/// has a chance of becoming ready soon.
+bool
+Ftp::Server::checkDataConnPre()
+{
+    if (Comm::IsConnOpen(dataConn))
+        return true;
+
+    if (Comm::IsConnOpen(dataListenConn)) {
+        // We are still waiting for a client to connect to us after PASV.
+        // Perhaps client's data conn handshake has not reached us yet.
+        // After we talk to the server, checkDataConnPost() will recheck.
+        debugs(33, 3, "expecting clt data conn " << dataListenConn);
+        return true;
+    }
+
+    if (!dataConn || dataConn->remote.isAnyAddr()) {
+        debugs(33, 5, "missing " << dataConn);
+        // TODO: use client address and default port instead.
+        setReply(425, "Use PORT or PASV first");
+        return false;
+    }
+
+    // active transfer: open a data connection from Squid to client
+    typedef CommCbMemFunT<Server, CommConnectCbParams> Dialer;
+    connector = JobCallback(17, 3, Dialer, this, Ftp::Server::connectedForData);
+    Comm::ConnOpener *cs = new Comm::ConnOpener(dataConn, connector,
+            Config.Timeout.connect);
+    AsyncJob::Start(cs);
+    return false; // ConnStateData::processFtpRequest waits handleConnectDone
+}
+
+/// Check that client data connection is ready for immediate I/O.
+bool
+Ftp::Server::checkDataConnPost() const
+{
+    if (!Comm::IsConnOpen(dataConn)) {
+        debugs(33, 3, "missing client data conn: " << dataConn);
+        return false;
+    }
+    return true;
+}
+
+/// Done establishing a data connection to the user.
+void
+Ftp::Server::connectedForData(const CommConnectCbParams &params)
+{
+    connector = NULL;
+
+    if (params.flag != Comm::OK) {
+        /* it might have been a timeout with a partially open link */
+        if (params.conn != NULL)
+            params.conn->close();
+        setReply(425, "Cannot open data connection.");
+        ClientSocketContext::Pointer context = getCurrentContext();
+        Must(context->http);
+        Must(context->http->storeEntry() != NULL);
+    } else {
+        Must(dataConn == params.conn);
+        Must(Comm::IsConnOpen(params.conn));
+        fd_note(params.conn->fd, "active client ftp data");
+    }
+
+    doProcessRequest();
+}
+
+void
+Ftp::Server::setReply(const int code, const char *msg)
+{
+    ClientSocketContext::Pointer context = getCurrentContext();
+    ClientHttpRequest *const http = context->http;
+    assert(http != NULL);
+    assert(http->storeEntry() == NULL);
+
+    HttpReply *const reply = Ftp::HttpReplyWrapper(code, msg, Http::scNoContent, 0);
+
+    setLogUri(http, urlCanonicalClean(http->request));
+
+    clientStreamNode *const node = context->getClientReplyContext();
+    clientReplyContext *const repContext =
+        dynamic_cast<clientReplyContext *>(node->data.getRaw());
+    assert(repContext != NULL);
+
+    RequestFlags reqFlags;
+    reqFlags.cachable = false; // force releaseRequest() in storeCreateEntry()
+    reqFlags.noCache = true;
+    repContext->createStoreEntry(http->request->method, reqFlags);
+    http->storeEntry()->replaceHttpReply(reply);
+}
+
+/// Whether Squid FTP Relay supports a named feature (e.g., a command).
+static bool
+Ftp::SupportedCommand(const SBuf &name)
+{
+    static std::set<SBuf> BlackList;
+    if (BlackList.empty()) {
+        /* Add FTP commands that Squid cannot relay correctly. */
+
+        // We probably do not support AUTH TLS.* and AUTH SSL,
+        // but let's disclaim all AUTH support to KISS, for now.
+        BlackList.insert(cmdAuth());
+    }
+
+    // we claim support for all commands that we do not know about
+    return BlackList.find(name) == BlackList.end();
+}
+

=== added file 'src/servers/FtpServer.h'
--- src/servers/FtpServer.h	1970-01-01 00:00:00 +0000
+++ src/servers/FtpServer.h	2014-08-10 23:18:33 +0000
@@ -0,0 +1,157 @@
+/*
+ * DEBUG: section 33    Client-side Routines
+ */
+
+#ifndef SQUID_SERVERS_FTP_SERVER_H
+#define SQUID_SERVERS_FTP_SERVER_H
+
+#include "client_side.h"
+#include "base/Lock.h"
+
+namespace Ftp
+{
+
+typedef enum {
+    fssBegin,
+    fssConnected,
+    fssHandleFeat,
+    fssHandlePasv,
+    fssHandlePort,
+    fssHandleDataRequest,
+    fssHandleUploadRequest,
+    fssHandleEprt,
+    fssHandleEpsv,
+    fssHandleCwd,
+    fssHandlePass,
+    fssHandleCdup,
+    fssError
+} ServerState;
+
+// TODO: This should become a part of MasterXaction when we start sending
+// master transactions to the clients/ code.
+/// Transaction information shared among our FTP client and server jobs.
+class MasterState: public RefCountable
+{
+public:
+    typedef RefCount<MasterState> Pointer;
+
+    MasterState(): serverState(fssBegin), clientReadGreeting(false) {}
+
+    Ip::Address clientDataAddr; ///< address of our FTP client data connection
+    SBuf workingDir; ///< estimated current working directory for URI formation
+    ServerState serverState; ///< what our FTP server is doing
+    bool clientReadGreeting; ///< whether our FTP client read their FTP server greeting
+};
+
+/// Manages a control connection from an FTP client.
+class Server: public ConnStateData
+{
+public:
+    explicit Server(const MasterXaction::Pointer &xact);
+    virtual ~Server();
+
+    // This is a pointer in hope to minimize future changes when MasterState
+    // becomes a part of MasterXaction. Guaranteed not to be nil.
+    MasterState::Pointer master; ///< info shared among our FTP client and server jobs
+
+protected:
+    friend void StartListening();
+
+    /* ConnStateData API */
+    virtual ClientSocketContext *parseOneRequest(Http::ProtocolVersion &ver);
+    virtual void processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver);
+    virtual void notePeerConnection(Comm::ConnectionPointer conn);
+    virtual void clientPinnedConnectionClosed(const CommCloseCbParams &io);
+    virtual void handleReply(HttpReply *header, StoreIOBuffer receivedData);
+    virtual int pipelinePrefetchMax() const;
+    virtual void writeControlMsgAndCall(ClientSocketContext *context, HttpReply *rep, AsyncCall::Pointer &call);
+    virtual time_t idleTimeout() const;
+
+    /* BodyPipe API */
+    virtual void noteMoreBodySpaceAvailable(BodyPipe::Pointer);
+    virtual void noteBodyConsumerAborted(BodyPipe::Pointer ptr);
+
+    /* AsyncJob API */
+    virtual void start();
+
+    /* Comm callbacks */
+    static void AcceptCtrlConnection(const CommAcceptCbParams &params);
+    void acceptDataConnection(const CommAcceptCbParams &params);
+    void readUploadData(const CommIoCbParams &io);
+    void wroteEarlyReply(const CommIoCbParams &io);
+    void wroteReply(const CommIoCbParams &io);
+    void wroteReplyData(const CommIoCbParams &io);
+    void connectedForData(const CommConnectCbParams &params);
+
+    unsigned int listenForDataConnection();
+    bool createDataConnection(Ip::Address cltAddr);
+    void closeDataConnection();
+
+    void calcUri(const SBuf *file);
+    void changeState(const Ftp::ServerState newState, const char *reason);
+    bool handleUserRequest(const SBuf &cmd, SBuf &params);
+    bool checkDataConnPost() const;
+    void replyDataWritingCheckpoint();
+    void maybeReadUploadData();
+
+    void setReply(const int code, const char *msg);
+    void writeCustomReply(const int code, const char *msg, const HttpReply *reply = NULL);
+    void writeEarlyReply(const int code, const char *msg);
+    void writeErrorReply(const HttpReply *reply, const int status);
+    void writeForwardedForeign(const HttpReply *reply);
+    void writeForwardedReply(const HttpReply *reply);
+    void writeForwardedReplyAndCall(const HttpReply *reply, AsyncCall::Pointer &call);
+    void writeReply(MemBuf &mb);
+
+    bool handleRequest(String &cmd, String &params);
+    void setDataCommand();
+    bool checkDataConnPre();
+
+    /// a method handling an FTP command; selected by handleRequest()
+    typedef bool (Ftp::Server::*RequestHandler)(String &cmd, String &params);
+    bool handleFeatRequest(String &cmd, String &params);
+    bool handlePasvRequest(String &cmd, String &params);
+    bool handlePortRequest(String &cmd, String &params);
+    bool handleDataRequest(String &cmd, String &params);
+    bool handleUploadRequest(String &cmd, String &params);
+    bool handleEprtRequest(String &cmd, String &params);
+    bool handleEpsvRequest(String &cmd, String &params);
+    bool handleCwdRequest(String &cmd, String &params);
+    bool handlePassRequest(String &cmd, String &params);
+    bool handleCdupRequest(String &cmd, String &params);
+
+    /// a method handling an FTP response; selected by handleReply()
+    typedef void (Ftp::Server::*ReplyHandler)(const HttpReply *reply, StoreIOBuffer data);
+    void handleFeatReply(const HttpReply *header, StoreIOBuffer receivedData);
+    void handlePasvReply(const HttpReply *header, StoreIOBuffer receivedData);
+    void handlePortReply(const HttpReply *header, StoreIOBuffer receivedData);
+    void handleErrorReply(const HttpReply *header, StoreIOBuffer receivedData);
+    void handleDataReply(const HttpReply *header, StoreIOBuffer receivedData);
+    void handleUploadReply(const HttpReply *header, StoreIOBuffer receivedData);
+    void handleEprtReply(const HttpReply *header, StoreIOBuffer receivedData);
+    void handleEpsvReply(const HttpReply *header, StoreIOBuffer receivedData);
+
+private:
+    void doProcessRequest();
+    void shovelUploadData();
+    void resetLogin(const char *reason);
+
+    SBuf uri; ///< a URI reconstructed from various FTP message details
+    SBuf host; ///< intended dest. of a transparently intercepted FTP conn
+    bool gotEpsvAll; ///< restrict data conn setup commands to just EPSV
+    AsyncCall::Pointer onDataAcceptCall; ///< who to call upon data conn acceptance
+    Comm::ConnectionPointer dataListenConn; ///< data connection listening socket
+    Comm::ConnectionPointer dataConn; ///< data connection
+    char uploadBuf[CLIENT_REQ_BUF_SZ]; ///< data connection input buffer
+    size_t uploadAvailSize; ///< number of yet unused uploadBuf bytes
+
+    AsyncCall::Pointer listener; ///< set when we are passively listening
+    AsyncCall::Pointer connector; ///< set when we are actively connecting
+    AsyncCall::Pointer reader; ///< set when we are reading FTP data
+
+    CBDATA_CLASS2(Server);
+};
+
+} // namespace Ftp
+
+#endif /* SQUID_SERVERS_FTP_SERVER_H */

=== added file 'src/servers/HttpServer.cc'
--- src/servers/HttpServer.cc	1970-01-01 00:00:00 +0000
+++ src/servers/HttpServer.cc	2014-08-08 03:07:03 +0000
@@ -0,0 +1,194 @@
+/*
+ * DEBUG: section 33    Client-side Routines
+ */
+
+#include "squid.h"
+#include "client_side.h"
+#include "client_side_request.h"
+#include "comm/Write.h"
+#include "HttpHeaderTools.h"
+#include "profiler/Profiler.h"
+#include "servers/forward.h"
+#include "SquidConfig.h"
+
+namespace Http
+{
+
+/// Manages a connection from an HTTP client.
+class Server: public ConnStateData
+{
+public:
+    Server(const MasterXaction::Pointer &xact, const bool beHttpsServer);
+    virtual ~Server() {}
+
+    void readSomeHttpData();
+
+protected:
+    /* ConnStateData API */
+    virtual ClientSocketContext *parseOneRequest(Http::ProtocolVersion &ver);
+    virtual void processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver);
+    virtual void handleReply(HttpReply *rep, StoreIOBuffer receivedData);
+    virtual void writeControlMsgAndCall(ClientSocketContext *context, HttpReply *rep, AsyncCall::Pointer &call);
+    virtual time_t idleTimeout() const;
+
+    /* BodyPipe API */
+    virtual void noteMoreBodySpaceAvailable(BodyPipe::Pointer);
+    virtual void noteBodyConsumerAborted(BodyPipe::Pointer);
+
+    /* AsyncJob API */
+    virtual void start();
+
+private:
+    void processHttpRequest(ClientSocketContext *const context);
+    void handleHttpRequestData();
+
+    HttpParser parser_;
+    HttpRequestMethod method_; ///< parsed HTTP method
+
+    /// temporary hack to avoid creating a true HttpsServer class
+    const bool isHttpsServer;
+
+    CBDATA_CLASS2(Server);
+};
+
+} // namespace Http
+
+CBDATA_NAMESPACED_CLASS_INIT(Http, Server);
+
+Http::Server::Server(const MasterXaction::Pointer &xact, bool beHttpsServer):
+        AsyncJob("Http::Server"),
+        ConnStateData(xact),
+        isHttpsServer(beHttpsServer)
+{
+}
+
+time_t
+Http::Server::idleTimeout() const
+{
+    return Config.Timeout.clientIdlePconn;
+}
+
+void
+Http::Server::start()
+{
+    ConnStateData::start();
+
+#if USE_OPENSSL
+    // XXX: Until we create an HttpsServer class, use this hack to allow old
+    // client_side.cc code to manipulate ConnStateData object directly
+    if (isHttpsServer) {
+        postHttpsAccept();
+        return;
+    }
+#endif
+
+    typedef CommCbMemFunT<Server, CommTimeoutCbParams> TimeoutDialer;
+    AsyncCall::Pointer timeoutCall =  JobCallback(33, 5,
+                                      TimeoutDialer, this, Http::Server::requestTimeout);
+    commSetConnTimeout(clientConnection, Config.Timeout.request, timeoutCall);
+    readSomeData();
+}
+
+void
+Http::Server::noteMoreBodySpaceAvailable(BodyPipe::Pointer)
+{
+    if (!handleRequestBodyData())
+        return;
+
+    // too late to read more body
+    if (!isOpen() || stoppedReceiving())
+        return;
+
+    readSomeData();
+}
+
+ClientSocketContext *
+Http::Server::parseOneRequest(Http::ProtocolVersion &ver)
+{
+    ClientSocketContext *context = NULL;
+    PROF_start(HttpServer_parseOneRequest);
+    HttpParserInit(&parser_, in.buf.c_str(), in.buf.length());
+    context = parseHttpRequest(this, &parser_, &method_, &ver);
+    PROF_stop(HttpServer_parseOneRequest);
+    return context;
+}
+
+void
+Http::Server::processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver)
+{
+    /* We have an initial client stream in place should it be needed */
+    /* setup our private context */
+    context->registerWithConn();
+    clientProcessRequest(this, &parser_, context, method_, ver);
+}
+
+void
+Http::Server::noteBodyConsumerAborted(BodyPipe::Pointer ptr)
+{
+    ConnStateData::noteBodyConsumerAborted(ptr);
+    stopReceiving("virgin request body consumer aborted"); // closes ASAP
+}
+
+void
+Http::Server::handleReply(HttpReply *rep, StoreIOBuffer receivedData)
+{
+    // the caller guarantees that we are dealing with the current context only
+    ClientSocketContext::Pointer context = getCurrentContext();
+    Must(context != NULL);
+    const ClientHttpRequest *http = context->http;
+    Must(http != NULL);
+
+    // After sending Transfer-Encoding: chunked (at least), always send
+    // the last-chunk if there was no error, ignoring responseFinishedOrFailed.
+    const bool mustSendLastChunk = http->request->flags.chunkedReply &&
+                                   !http->request->flags.streamError &&
+                                   !context->startOfOutput();
+    const bool responseFinishedOrFailed = !rep &&
+                                          !receivedData.data &&
+                                          !receivedData.length;
+    if (responseFinishedOrFailed && !mustSendLastChunk) {
+        context->writeComplete(context->clientConnection, NULL, 0, Comm::OK);
+        return;
+    }
+
+    if (!context->startOfOutput()) {
+        context->sendBody(rep, receivedData);
+        return;
+    }
+
+    assert(rep);
+    http->al->reply = rep;
+    HTTPMSGLOCK(http->al->reply);
+    context->sendStartOfMessage(rep, receivedData);
+}
+
+void
+Http::Server::writeControlMsgAndCall(ClientSocketContext *context, HttpReply *rep, AsyncCall::Pointer &call)
+{
+    // apply selected clientReplyContext::buildReplyHeader() mods
+    // it is not clear what headers are required for control messages
+    rep->header.removeHopByHopEntries();
+    rep->header.putStr(HDR_CONNECTION, "keep-alive");
+    httpHdrMangleList(&rep->header, getCurrentContext()->http->request, ROR_REPLY);
+
+    MemBuf *mb = rep->pack();
+
+    debugs(11, 2, "HTTP Client " << clientConnection);
+    debugs(11, 2, "HTTP Client CONTROL MSG:\n---------\n" << mb->buf << "\n----------");
+
+    Comm::Write(context->clientConnection, mb, call);
+
+    delete mb;
+}
+
+ConnStateData *
+Http::NewServer(MasterXactionPointer &xact)
+{
+    return new Server(xact, false);
+}
+
+ConnStateData *
+Https::NewServer(MasterXactionPointer &xact)
+{
+    return new Http::Server(xact, true);
+}

=== added file 'src/servers/Makefile.am'
--- src/servers/Makefile.am	1970-01-01 00:00:00 +0000
+++ src/servers/Makefile.am	2014-08-10 23:18:33 +0000
@@ -0,0 +1,10 @@
+include $(top_srcdir)/src/Common.am
+include $(top_srcdir)/src/TestHeaders.am
+
+noinst_LTLIBRARIES = libservers.la
+
+libservers_la_SOURCES = \
+	FtpServer.cc \
+	FtpServer.h \
+	HttpServer.cc \
+	forward.h

=== added file 'src/servers/forward.h'
--- src/servers/forward.h	1970-01-01 00:00:00 +0000
+++ src/servers/forward.h	2014-08-10 23:18:33 +0000
@@ -0,0 +1,36 @@
+#ifndef SQUID_SERVERS_FORWARD_H
+#define SQUID_SERVERS_FORWARD_H
+
+class MasterXaction;
+template <class C> class RefCount;
+typedef RefCount<MasterXaction> MasterXactionPointer;
+
+class ConnStateData;
+
+namespace Http
+{
+
+/// create a new HTTP connection handler; never returns NULL
+ConnStateData *NewServer(MasterXactionPointer &xact);
+
+} // namespace Http
+
+namespace Https
+{
+
+/// create a new HTTPS connection handler; never returns NULL
+ConnStateData *NewServer(MasterXactionPointer &xact);
+
+} // namespace Https
+
+namespace Ftp
+{
+
+/// accept connections on all configured ftp_ports
+void StartListening();
+/// reject new connections to any configured ftp_port
+void StopListening();
+
+} // namespace Ftp
+
+#endif /* SQUID_SERVERS_FORWARD_H */

=== modified file 'src/tests/stub_SBuf.cc'
--- src/tests/stub_SBuf.cc	2014-07-18 11:02:16 +0000
+++ src/tests/stub_SBuf.cc	2014-08-07 17:13:26 +0000
@@ -55,6 +55,6 @@
 SBuf::size_type SBuf::findFirstOf(const CharacterSet &set, size_type startPos) const STUB_RETVAL(SBuf::npos)
 SBuf::size_type SBuf::findFirstNotOf(const CharacterSet &set, size_type startPos) const STUB_RETVAL(SBuf::npos)
 int SBuf::scanf(const char *format, ...) STUB_RETVAL(-1)
-SBuf SBuf::toLower() const STUB_RETVAL(*this)
-SBuf SBuf::toUpper() const STUB_RETVAL(*this)
+void SBuf::toLower() STUB
+void SBuf::toUpper() STUB
 String SBuf::toString() const STUB_RETVAL(String(""))

=== modified file 'src/tests/stub_client_side.cc'
--- src/tests/stub_client_side.cc	2014-06-05 14:57:58 +0000
+++ src/tests/stub_client_side.cc	2014-07-30 17:33:14 +0000
@@ -53,8 +53,8 @@
 void ConnStateData::noteBodyConsumerAborted(BodyPipe::Pointer) STUB
 bool ConnStateData::handleReadData() STUB_RETVAL(false)
 bool ConnStateData::handleRequestBodyData() STUB_RETVAL(false)
-void ConnStateData::pinConnection(const Comm::ConnectionPointer &pinServerConn, HttpRequest *request, CachePeer *peer, bool auth) STUB
-void ConnStateData::unpinConnection() STUB
+void ConnStateData::pinConnection(const Comm::ConnectionPointer &pinServerConn, HttpRequest *request, CachePeer *peer, bool auth, bool monitor) STUB
+void ConnStateData::unpinConnection(const bool andClose) STUB
 const Comm::ConnectionPointer ConnStateData::validatePinnedConnection(HttpRequest *request, const CachePeer *peer) STUB_RETVAL(NULL)
 void ConnStateData::clientPinnedConnectionClosed(const CommCloseCbParams &io) STUB
 void ConnStateData::clientReadRequest(const CommIoCbParams &io) STUB

=== modified file 'src/tests/testSBuf.cc'
--- src/tests/testSBuf.cc	2014-04-21 17:14:44 +0000
+++ src/tests/testSBuf.cc	2014-08-07 17:13:26 +0000
@@ -784,7 +784,7 @@
 void
 testSBuf::testStringOps()
 {
-    SBuf sng(literal.toLower()),
+    SBuf sng(ToLower(literal)),
     ref("the quick brown fox jumped over the lazy dog");
     CPPUNIT_ASSERT_EQUAL(ref,sng);
     sng=literal;
@@ -820,7 +820,7 @@
 
     // case-insensitive checks
     CPPUNIT_ASSERT(literal.startsWith(casebuf,caseInsensitive));
-    casebuf=SBuf(fox1).toUpper();
+    casebuf=ToUpper(SBuf(fox1));
     CPPUNIT_ASSERT(literal.startsWith(casebuf,caseInsensitive));
     CPPUNIT_ASSERT(literal.startsWith(SBuf(fox1),caseInsensitive));
     casebuf = "tha quick";

=== modified file 'src/tools.cc'
--- src/tools.cc	2014-07-18 11:02:16 +0000
+++ src/tools.cc	2014-08-05 22:44:23 +0000
@@ -102,8 +102,8 @@
 {
     // Release the main ports as early as possible
 
-    // clear both http_port and https_port lists.
-    clientHttpConnectionsClose();
+    // clear http_port, https_port, and ftp_port lists
+    clientConnectionsClose();
 
     // clear icp_port's
     icpClosePorts();
@@ -1151,6 +1151,14 @@
     }
 #endif
 
+    if ((p = FtpPortList) != NULL) {
+        // skip any special interception ports
+        while (p != NULL && p->flags.isIntercepted())
+            p = p->next;
+        if (p != NULL)
+            return p->s.port();
+    }
+
     debugs(21, DBG_CRITICAL, "ERROR: No forward-proxy ports configured.");
     return 0; // Invalid port. This will result in invalid URLs on bad configurations.
 }