| Index | Alphabetical Index |
Option Name: | logformat |
---|---|
Replaces: | |
Requires: | |
Default Value: | The format definitions squid, common, combined, referrer, useragent are built in. |
Suggested Config: |
|
Usage: logformat <name> <format specification> Defines an access log format. The <format specification> is a string with embedded % format codes % format codes all follow the same basic structure where all components but the formatcode are optional and usually unnecessary, especially when dealing with common codes. % [encoding] [-] [[0]width] [{arg}] formatcode [{arg}] encoding escapes or otherwise protects "special" characters: " Quoted string encoding where quote(") and backslash(\) characters are \-escaped while CR, LF, and TAB characters are encoded as \r, \n, and \t two-character sequences. [ Custom Squid encoding where percent(%), square brackets([]), backslash(\) and characters with codes outside of [32,126] range are %-encoded. SP is not encoded. Used by log_mime_hdrs. # URL encoding (a.k.a. percent-encoding) where all URL unsafe and control characters (per RFC 1738) are %-encoded. / Shell-like encoding where quote(") and backslash(\) characters are \-escaped while CR and LF characters are encoded as \r and \n two-character sequences. Values containing SP character(s) are surrounded by quotes("). ' Raw/as-is encoding with no escaping/quoting. Default encoding: When no explicit encoding is specified, each %code determines its own encoding. Most %codes use raw/as-is encoding, but some codes use a so called "pass-through URL encoding" where all URL unsafe and control characters (per RFC 1738) are %-encoded, but the percent character(%) is left as is. - left aligned width minimum and/or maximum field width: [width_min][.width_max] When minimum starts with 0, the field is zero-padded. String values exceeding maximum width are truncated. {arg} argument such as header name etc. This field may be placed before or after the token, but not both at once. Format codes: % a literal % character byte{value} Adds a single byte with the given value (e.g., %byte{10} adds an ASCII LF character a.k.a. "new line" or "\n"). The value parameter is required and must be a positive decimal integer not exceeding 255. Zero-valued bytes (i.e. ASCII NUL characters) are not yet supported. sn Unique sequence number per log line entry err_code The ID of an error response served by Squid or a similar internal error identifier. err_detail Additional err_code-dependent error information. Multiple details are separated by the plus sign ('+'). Admins should not rely on a particular detail listing order, the uniqueness of the entries, or individual detail text stability. All those properties depend on many unstable factors, including external libraries. note The annotation specified by the argument. Also logs the adaptation meta headers set by the adaptation_meta configuration parameter. If no argument given all annotations logged. The argument may include a separator to use with annotation values: name[:separator] By default, multiple note values are separated with "," and multiple notes are separated with "\r\n". When logging named notes with %{name}note, the explicitly configured separator is used between note values. When logging all notes with %note, the explicitly configured separator is used between individual notes. There is currently no way to specify both value and notes separators when logging all notes with %note. master_xaction The master transaction identifier is an unsigned integer. These IDs are guaranteed to monotonically increase within a single worker process lifetime, with higher values corresponding to transactions that were accepted or initiated later. Due to current implementation deficiencies, some IDs are skipped (i.e. never logged). Concurrent workers and restarted workers use similar, overlapping sequences of master transaction IDs. Connection related format codes: >a Client source IP address >A Client FQDN >p Client source port >eui Client source EUI (MAC address, EUI-48 or EUI-64 identifier) >la Local IP address the client connected to >lp Local port number the client connected to >qos Client connection TOS/DSCP value set by Squid >nfmark Client connection netfilter packet MARK set by Squid transport::>connection_id Identifies a transport connection accepted by Squid (e.g., a connection carrying the logged HTTP request). Currently, Squid only supports TCP transport connections. The logged identifier is an unsigned integer. These IDs are guaranteed to monotonically increase within a single worker process lifetime, with higher values corresponding to connections that were accepted later. Many IDs are skipped (i.e. never logged). Concurrent workers and restarted workers use similar, partially overlapping sequences of IDs. la Local listening IP address the client connection was connected to. lp Local listening port number the client connection was connected to. <a Server IP address of the last server or peer connection <A Server FQDN or peer name <p Server port number of the last server or peer connection <la Local IP address of the last server or peer connection <lp Local port number of the last server or peer connection <qos Server connection TOS/DSCP value set by Squid <nfmark Server connection netfilter packet MARK set by Squid >handshake Raw client handshake Initial client bytes received by Squid on a newly accepted TCP connection or inside a just established CONNECT tunnel. Squid stops accumulating handshake bytes as soon as the handshake parser succeeds or fails (determining whether the client is using the expected protocol). For HTTP clients, the handshake is the request line. For TLS clients, the handshake consists of all TLS records up to and including the TLS record that contains the last byte of the first ClientHello message. For clients using an unsupported protocol, this field contains the bytes received by Squid at the time of the handshake parsing failure. See the on_unsupported_protocol directive for more information on Squid handshake traffic expectations. Current support is limited to these contexts: - http_port connections, but only when the on_unsupported_protocol directive is in use. - https_port connections (and CONNECT tunnels) that are subject to the ssl_bump peek or stare action. To protect binary handshake data, this field is always base64-encoded (RFC 4648 Section 4). If logformat field encoding is configured, that encoding is applied on top of base64. Otherwise, the computed base64 value is recorded as is. Time related format codes: ts Seconds since epoch tu subsecond time (milliseconds) tl Local time. Optional strftime format argument default %d/%b/%Y:%H:%M:%S %z tg GMT time. Optional strftime format argument default %d/%b/%Y:%H:%M:%S %z tr Response time (milliseconds) dt Total time spent making DNS lookups (milliseconds) tS Approximate master transaction start time in <full seconds since epoch>.<fractional seconds> format. Currently, Squid considers the master transaction started when a complete HTTP request header initiating the transaction is received from the client. This is the same value that Squid uses to calculate transaction response time when logging %tr to access.log. Currently, Squid uses millisecond resolution for %tS values, similar to the default access.log "current time" field (%ts.%03tu). busy_time Time spent in transaction-related code (nanoseconds) This cumulative measurement excludes periods of time when the transaction was waiting (e.g., for a server or helper response) while Squid worked on other transactions or was engaged in transaction-unrelated activities (e.g., generating a cache index). In other words, this measurement represents the total amount of physical time when Squid was busy working on this transaction. WARNING: This measurement relies on Squid transaction context tracking features that currently have known context leak bugs and coverage gaps. Until those features are fully implemented, logged values may significantly understate or exaggerate actual times. Do not use this measurement unless you know it works in your case. Access Control related format codes: et Tag returned by external acl ea Log string returned by external acl un User name (any available) ul User name from authentication ue User name from external acl helper un A user name. Expands to the first available name from the following list of information sources: - authenticated user name, like %ul - user name supplied by an external ACL, like %ue - SSL client name, like %us credentials Client credentials. The exact meaning depends on the authentication scheme: For Basic authentication, it is the password; for Digest, the realm sent by the client; for NTLM and Negotiate, the client challenge or client credentials prefixed with "YR " or "KK ". HTTP related format codes: REQUEST [http::]rm Request method (GET/POST etc) [http::]>rm Request method from client [http::]<rm Request method sent to server or peer [http::]ru Request URL received (or computed) and sanitized Logs request URI received from the client, a request adaptation service, or a request redirector (whichever was applied last). Computed URLs are URIs of internally generated requests and various "error:..." URIs. Honors strip_query_terms and uri_whitespace. This field is not encoded by default. Encoding this field using variants of %-encoding will clash with uri_whitespace modifications that also use %-encoding. [http::]>ru Request URL received from the client (or computed) Computed URLs are URIs of internally generated requests and various "error:..." URIs. Unlike %ru, this request URI is not affected by request adaptation, URL rewriting services, and strip_query_terms. Honors uri_whitespace. This field is using pass-through URL encoding by default. Encoding this field using other variants of %-encoding will clash with uri_whitespace modifications that also use %-encoding. [http::]<ru Request URL sent to server or peer [http::]>rs Request URL scheme from client [http::]<rs Request URL scheme sent to server or peer [http::]>rd Request URL domain from client [http::]<rd Request URL domain sent to server or peer [http::]>rP Request URL port from client [http::]<rP Request URL port sent to server or peer [http::]rp Request URL path excluding hostname [http::]>rp Request URL path excluding hostname from client [http::]<rp Request URL path excluding hostname sent to server or peer [http::]rv Request protocol version [http::]>rv Request protocol version from client [http::]<rv Request protocol version sent to server or peer [http::]>h Original received request header. Usually differs from the request header sent by Squid, although most fields are often preserved. Accepts optional header field name/value filter argument using name[:[separator]element] format. [http::]>ha Received request header after adaptation and redirection (pre-cache REQMOD vectoring point). Usually differs from the request header sent by Squid, although most fields are often preserved. Optional header name argument as for >h RESPONSE [http::]<Hs HTTP status code received from the next hop [http::]>Hs HTTP status code sent to the client [http::]<h Reply header. Optional header name argument as for >h [http::]mt MIME content type SIZE COUNTERS [http::]st Total size of request + reply traffic with client [http::]>st Total size of request received from client. Excluding chunked encoding bytes. [http::]<st Total size of reply sent to client (after adaptation) [http::]>sh Size of request headers received from client [http::]<sh Size of reply headers sent to client (after adaptation) [http::]<sH Reply high offset sent [http::]<sS Upstream object size [http::]<bs Number of HTTP-equivalent message body bytes received from the next hop, excluding chunked transfer encoding and control messages. Generated FTP listings are treated as received bodies. TIMING [http::]<pt Peer response time in milliseconds. The timer starts when the last request byte is sent to the next hop and stops when the last response byte is received. [http::]<tt Total time spent forwarding to origin servers or cache_peers (milliseconds). The timer starts when Squid decides to forward the request (to an origin server or cache_peer) and peer selection begins. The timer stops when relevant forwarding activities (including any retries) end. Between those two timer events, Squid may perform DNS lookups, query external ACL helpers, adapt responses using pre-cache RESPMOD services, and participate in other concurrent secondary activities. Most secondary activities increase peering time. In some cases, a secondary activity may start before the timer starts or end after the timer stops, leading to misleading results of simple computations like %<tt - %dt. If this logformat %code is used before its timer starts, the corresponding measurement has no value (and the %code expands to a single dash ("-") character). If this code is used while its timer is running, the time spent so far is used as the measurement value. When Squid re-forwards the request (e.g., after certain cache revalidation failures), the timer may restart. In this case, the new measurement is added to the value accumulated from previous forwarding attempts. The time interval between forwarding attempts is not added to the final result. Squid handling related format codes: Ss Squid request status (TCP_MISS etc) Sh Squid hierarchy status (DEFAULT_PARENT etc) [http::]request_attempts Number of request forwarding attempts See forward_max_tries documentation that details what Squid counts as a forwarding attempt. Pure cache hits log zero, but cache hits that triggered HTTP cache revalidation log the number of attempts made when sending an internal revalidation request. DNS, ICMP, ICP, HTCP, ICAP, eCAP, helper, and other secondary requests sent by Squid as a part of a master transaction do not increment the counter logged for the received request. SSL-related format codes: ssl::bump_mode SslBump decision for the transaction: For CONNECT requests that initiated bumping of a connection and for any request received on an already bumped connection, Squid logs the corresponding SslBump mode ("splice", "bump", "peek", "stare", "terminate", "server-first" or "client-first"). See the ssl_bump option for more information about these modes. A "none" token is logged for requests that triggered "ssl_bump" ACL evaluation matching a "none" rule. In all other cases, a single dash ("-") is logged. ssl::>sni SSL client SNI sent to Squid. ssl::>cert_subject The Subject field of the received client SSL certificate or a dash ('-') if Squid has received an invalid/malformed certificate or no certificate at all. Consider encoding the logged value because Subject often has spaces. ssl::>cert_issuer The Issuer field of the received client SSL certificate or a dash ('-') if Squid has received an invalid/malformed certificate or no certificate at all. Consider encoding the logged value because Issuer often has spaces. ssl::<cert_subject The Subject field of the received server TLS certificate or a dash ('-') if this is not available. Consider encoding the logged value because Subject often has spaces. ssl::<cert_issuer The Issuer field of the received server TLS certificate or a dash ('-') if this is not available. Consider encoding the logged value because Issuer often has spaces. ssl::<cert The received server x509 certificate in PEM format, including BEGIN and END lines (or a dash ('-') if the certificate is unavailable). WARNING: Large certificates will exceed the current 8KB access.log record limit, resulting in truncated records. Such truncation usually happens in the middle of a record field. The limit applies to all access logging modules. The logged certificate may have failed validation and may not be trusted by Squid. This field does not include any intermediate certificates that may have been received from the server or fetched during certificate validation process. Currently, Squid only collects server certificates during step3 of SslBump processing; connections that were not subject to ssl_bump rules or that did not match a peek or stare rule at step2 will not have the server certificate information. This field is using pass-through URL encoding by default. ssl::<cert_errors The list of certificate validation errors detected by Squid (including OpenSSL and certificate validation helper components). The errors are listed in the discovery order. By default, the error codes are separated by ':'. Accepts an optional separator argument. %ssl::>negotiated_version The negotiated TLS version of the client connection. %ssl::<negotiated_version The negotiated TLS version of the last server or peer connection. %ssl::>received_hello_version The TLS version of the Hello message received from TLS client. %ssl::<received_hello_version The TLS version of the Hello message received from TLS server. %ssl::>received_supported_version The maximum TLS version supported by the TLS client. %ssl::<received_supported_version The maximum TLS version supported by the TLS server. %ssl::>negotiated_cipher The negotiated cipher of the client connection. %ssl::<negotiated_cipher The negotiated cipher of the last server or peer connection. If ICAP is enabled, the following code becomes available (as well as ICAP log codes documented with the icap_log option): icap::tt Total ICAP "blocking" time for the HTTP transaction. The timer ticks while Squid checks adaptation_access and while ICAP transaction(s) expect ICAP response headers, including the embedded adapted HTTP message headers (where applicable). This measurement is meant to estimate ICAP impact on HTTP transaction response times, but it does not currently account for slow ICAP response body delivery blocking HTTP progress. Once Squid receives the final ICAP response headers (e.g., ICAP 200 or 204) and the associated adapted HTTP message headers (if any) from the ICAP service, the corresponding ICAP transaction stops affecting this measurement, even though the transaction itself may continue for a long time (e.g., to finish sending the ICAP request and/or to finish receiving the ICAP response body). When "blocking" sections of multiple concurrent ICAP transactions overlap in time, the overlapping segment is counted only once. To see complete ICAP transaction response times (rather than the cumulative effect of their blocking sections) use the %adapt::all_trs logformat code or the icap_log directive. If adaptation is enabled the following codes become available: adapt::<last_h The header of the last ICAP response or meta-information from the last eCAP transaction related to the HTTP transaction. Like <h, accepts an optional header name argument. adapt::sum_trs Summed adaptation transaction response times recorded as a comma-separated list in the order of transaction start time. Each time value is recorded as an integer number, representing response time of one or more adaptation (ICAP or eCAP) transaction in milliseconds. When a failed transaction is being retried or repeated, its time is not logged individually but added to the replacement (next) transaction. Lifetimes of individually listed adaptation transactions may overlap. See also: %icap::tt and %adapt::all_trs. adapt::all_trs All adaptation transaction response times. Same as %adapt::sum_trs but response times of individual transactions are never added together. Instead, all transaction response times are recorded individually. You can prefix adapt::*_trs format codes with adaptation service name in curly braces to record response time(s) specific to that service. For example: %{my_service}adapt::sum_trs Format codes related to the PROXY protocol: proxy_protocol::>h PROXY protocol header, including optional TLVs. Supports the same field and element reporting/extraction logic as %http::>h. For configuration and reporting purposes, Squid maps each PROXY TLV to an HTTP header field: the TLV type (configured as a decimal integer) is the field name, and the TLV value is the field value. All TLVs of "LOCAL" connections (in PROXY protocol terminology) are currently skipped/ignored. Squid also maps the following standard PROXY protocol header blocks to pseudo HTTP headers (their names use PROXY terminology and start with a colon, following HTTP tradition for pseudo headers): :command, :version, :src_addr, :dst_addr, :src_port, and :dst_port. Without optional parameters, this logformat code logs pseudo headers and TLVs. This format code uses pass-through URL encoding by default. Example: # relay custom PROXY TLV #224 to adaptation services adaptation_meta Client-Foo "%proxy_protocol::>h{224}" See also: %http::>h The default formats available (which do not need re-defining) are: logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt logformat common %>a - %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st %Ss:%Sh logformat combined %>a - %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh logformat referrer %ts.%03tu %>a %{Referer}>h %ru logformat useragent %>a [%tl] "%{User-Agent}>h" NOTE: When the log_mime_hdrs directive is set to ON. The squid, common and combined formats have a safely encoded copy of the mime headers appended to each line within a pair of brackets. NOTE: The common and combined formats are not quite true to the Apache definition. The logs from Squid contain an extra status and hierarchy code appended. |
|
| Index | Alphabetical Index |