--------------------- PatchSet 11492 Date: 2007/06/23 21:34:09 Author: hno Branch: SQUID_2_6 Tag: (none) Log: Author: Manu Garg MFC: Bug #1968: Squid hangs occasionally when using DNS search paths Squid 2.6 (tested with squid 2.6.1 and 2.6.5) hangs after running for some time (could be from 5 min to 40 min on a busy server). Squid becomes unresponsive and CPU usage becomes 90-100%. ltrace shows that it's continuously comparing strings in an infinite loop: strcmp("thumbnail.videoegg.com", "i12.ebaystatic.com") = -1 strcmp("thumbnail.videoegg.com", "i12.ebaystatic.com") = -1 strcmp("thumbnail.videoegg.com", "i12.ebaystatic.com") = -1 The problem seems to be in the way squid's internal DNS system (dns_internal.c) keeps record of looked up but not yet answered DNS queries. This bug is hit specifically when multiple search paths are used in /etc/resolv.conf. Squid caches all dns queries before sending them to avoid duplicate queries for the same name. (look at: idnsCacheQuery(q) and hash_table *idns_lookup_hash, in dns_internal.c). This mechanism works well unless multiple search paths are defined in /etc/resolv.conf. When multiple dns search paths are defined, same query object is modified and next search path is concatenated to it's name. This query is cached again and resent. Problem is that the query is not unlinked before being cached and thus linked again. Only the key of hash object (that's actually name) changes this time; object itself remains same. This corrupts the hash table of looked up queries. Merged changes: 2007/06/23 21:06:58 Manu Garg +5 -1 Bug #1968: Squid hangs occasionally when using DNS search paths Members: src/dns_internal.c:1.61->1.61.2.1 Index: squid/src/dns_internal.c =================================================================== RCS file: /cvsroot/squid/squid/src/dns_internal.c,v retrieving revision 1.61 retrieving revision 1.61.2.1 diff -u -r1.61 -r1.61.2.1 --- squid/src/dns_internal.c 17 Dec 2006 10:51:44 -0000 1.61 +++ squid/src/dns_internal.c 23 Jun 2007 21:34:09 -0000 1.61.2.1 @@ -1,6 +1,6 @@ /* - * $Id: dns_internal.c,v 1.61 2006/12/17 10:51:44 serassio Exp $ + * $Id: dns_internal.c,v 1.61.2.1 2007/06/23 21:34:09 hno Exp $ * * DEBUG: section 78 DNS lookups; interacts with lib/rfc1035.c * AUTHOR: Duane Wessels @@ -812,6 +812,10 @@ q->attempt++; } rfc1035MessageDestroy(message); + if (q->hash.key) { + hash_remove_link(idns_lookup_hash, &q->hash); + q->hash.key = NULL; + } q->start_t = current_time; q->id = idnsQueryID(); rfc1035SetQueryID(q->buf, q->id);