bind-dnsind-perf by xiaoyounan


									                   Modern DNS as a Coherent Dynamic Universal Database

                                 Paul Vixie <>
                                Akira Kato <>
                                             June, 2004

     The Domain Name System (DNS) allows zone data to change dynamically based on secure
     protocol messages, and the changed data can be globally visible after only a short delay.
     This report explores these features of DNS, including server and client implementation
     issues, and shows an example of their use in creating an automated Real Time Blackhole
     List based on reception of viruses from infected remote computers.

1. Introduction
                                                    •     Changes to DNS zone data can be trans-
DNS is a coherent autonomous distributed hier-            ferred incrementally, allowing large zones
archical database, designed in 1987 [RFC1034]             to change quickly and often [RFC1995]
to map Internet host names to their Internet
Protocol (IP) addresses and to replace the ubiq-    These new features make it possible to deliver
uitous HOSTS.TXT file. Information was en-           services through DNS that were not originally
tered into DNS using out of band mechanisms         possible. One example of a coherent dynamic
such as flat text files edited or generated by        universal database is a shared global real time
domain administrators, and was expected to          blackhole list (RTBL) for e-mail.
be changed only rarely, on the order of hours
or months.
                                                    2. DNS Changes
     After 17 years of evolution, the definition
and scope of DNS has grown. New features            While a fully detailed explaination of dynam-
to be discussed in this paper include the fol-      ic update [RFC2136], real time change noti-
lowing:                                             fication [RFC1996], incremental zone trans-
                                                    fer [RFC1995], and transaction signatures
•    New information can now enter DNS dy-
                                                    [RFC2845] would merely repeat the Inter-
     namically, using secured protocol mes-
                                                    net RFC specifications on these topics, it is
     sages [RFC2136],[RFC2845]
                                                    useful to revisit these four protocol changes
•    It is no longer necessary to periodically      in overview.
     poll for DNS zone changes, greatly reduc-           Given the perspective allowed for by sev-
     ing propagation delay [RFC1996]                eral years of implementation and deployment
                                                    experience, here is what we now know:

2.1. Dynamic Update                                      serial number. All changes to the zone that oc-
                                                         curred within this interval were published si-
In traditional DNS, all zone changes were                multaneously, as a batch. This led to either un-
made out of band, usually to a so-called mas-            fortunately short polling intervals, or unfortu-
ter file which was either machine-generated or            nately long embargoes on additions, changes,
human-edited. The [RFC2136] proposal for in-             and deletions to zone data.
band dynamic update allows zone changes to
                                                               In [RFC1996], a method was proposed
occur as a result of DNS protocol messages.
                                                         for zone master servers to signal to published
These messages can come from DHCP agents,
                                                         and/or known zone slave servers the existence
management consoles, or any other manual or
                                                         of a new zone, by transmitting only the zone’s
automated method. Updates are atomic, reli-
                                                         new serial number. If this serial number differs
able, and can be declared to be idempotent. Up-
                                                         from the one the slave server is currently us-
date authentication can be by TCP/IP source
                                                         ing, then an early refresh operation is triggered,
address, or by transaction signature (as speci-
                                                         whereby each participating zone slave server
fied in [RFC2845] and described in the follow-
                                                         can verify the zone’s serial number, and initiate
ing subsection.)
                                                         a zone transfer if appropriate.
      Updates are applied to the zone at the
                                                               Wide scale deployment of [RFC1996] has
master server, which is the apex of a distribu-
                                                         been an unqualified success, and it is safe to
tion graph containing slave servers, caching
                                                         say that no zone of any importance is served
servers, and clients. If the master and slave
                                                         using software that does not comply with the
servers for a zone all implement real time
                                                         [RFC1996]recommendation. This means most
zone change notification (per [RFC1996]) and
                                                         zone changes are signalled to slave servers
incremental zone transfer (per [RFC1995])
                                                         within a few seconds of a zone change on the
then the result will be nearly instantaneous
                                                         master server, and if incremental zone trans-
zone changes visible at all of a zone’s author-
                                                         fer is also in use, then the zone changes will be
ity servers starting from in-band DNS proto-
                                                         propagated within the same short period.
col messages specifying dynamic updates. If
a moderately low negative time-to-live (TTL)                   It’s necessary to distinguish between the
is specified as the zone’s SOA minimum (start             availability of new zone content on the author-
of authority record’s minimum field), and if an           ity servers (master and slaves), as compared to
equally low TTL is specified as the zone’s de-            the definite use of changed data, which is not
fault TTL, then caching servers and clients will         guaranteed until after the time-to-live (TTL)
see zone changes in close to real time.                  of the old data has expired. Note that for pre-
                                                         viously nonexistent data, there is an SOA min-
2.2. Real Time Change Notification                        imum that sets the maximum interval for so-
                                                         called “negative caching.” The net effect of
Another traditional constraint in DNS was that           DNS caching is that changing the zone con-
zone changes could only occur at predefined               tent on the master and slaves is useful but not a
intervals (called SOA refresh and SOA retry).            cure-all for staleness, due to positive and nega-
Zone slave servers would only contact the zone           tive caching. (See Section 3.3.1for more infor-
master server every SOA refresh seconds (or              mation on this topic.)
SOA retry if there had been a prior communi-
cations failure) to check for an updated zone

2.3. Incremental Zone Transfer                            2.4. Transaction Signatures
The traditional model for synchronizing con-              During the preparation of the dynamic update
tent between zone master servers and zone                 proposal (see [RFC2136]), it became necessary
slave servers requires that a complete copy of            to be able to authenticate a transaction’s initia-
the zone content be transmitted from the mas-             tor. A general mechanism was proposed, called
ter server to all slaves whenever this content is         TSIG ([RFC2845]), by which any cooperating
changed. For zones of nontrivial size or rate of          initiator and responder could share secrets of-
change, this “full copy” model is prohibitively           fline or out of band, and use these shared se-
expensive. Historically, this meant that large            crets to prove the identity of a transaction’s ini-
zones could only be changed one per day or                tiator and responder. While the out of band na-
even once per week, and correspondingly re-               ture of these shared secrets is a harsh scaling
quired data which had to change on a more fre-            limit, TSIG has shown its value both in enter-
quent basis to be placed in small separate zones          prise contexts (where all initiators and respon-
that could be transferred frequently without              ders are under the same administration) and in
high cost. These constraints were unfortunate,            zone hosting contexts (where a zone’s master
and acted against correctness in overall system           and slaves are operated by cooperating parties,
designs involving DNS.                                    who already have an out of band administrative
     In [RFC1995], a method was proposed for              channel.)
transferring only incremental changes from the                 Proposals to authenticate DNS transac-
zone master to each zone slave. These changes,            tions without out of band shared secrets
even if very frequent, were small enough to               include GSS-TSIG [RFC3645] and SIG(0)
be transmitted and stored at only nominal                 [RFC2931], which are beyond the scope of this
cost. After subsequent wide scale adoption                paper.
of that proposal, it is now common for zones
of arbitrary size to change their content many            2.5. Summary of Protocol Changes
times per day, sometimes even many times per
minute. The [RFC1995] proposal is now uni-                IETF’s “DNS I-N-D” project which included
versally considered a success, and is an Internet         incremental zone transfers (“I”), real time zone
Standard.                                                 change notification (“N”), and dynamic zone
      Early software implementations of                   updates (“D”) has remade DNS from its tradi-
[RFC1995] (such as ISC BIND8 and early ISC                tional static model into a modern and robust
BIND9 [ISCBIND], Microsoft DNS, and oth-                  real time global publication medium for any
ers) were constrained to only be able to trans-           DNS-compatible data.
fer incremental zone changes if the source of                  With these features now available, it has
those changes was a DNS Dynamic Update                    become possible to deploy services using DNS
[RFC2136]. As of ISC BIND9.3, any zone                    rather than constructing a new coherent au-
change including complete regeneration or hu-             tonomous distributed hierarchical database for
man editing can still be transferred as a mini-           every new service or application.
mal incremental change. From this we con-
clude that [RFC1995] has matured.

3. DNS Services Example – Dynamic                        3.2. Implementation Details
                                                         Implementation consisted of several small
Malfeasants now routinely employ viruses and             BSD UNIX shell-level utilities and some con-
worms in order to organize large numbers of              figuration and setup work for ISC BIND9 and
always-online home computers into an un-                 the Postfix mailer [POSTFIX].
witting infrastructure for launching spam, dis-
tributed denial of service, and other attacks.           3.2.1. Adding Host Addresses –
This author has observed that more than half
of all host computers who attempt to deliver             #!/bin/sh
worms or viruses will also attempt to deliver            node=‘echo $1 | awk -F. \
spam within the following four weeks.                          ’{print $4 "." $3 "." $2 "." $1}’ \
                                                              ‘; shift
      Given this correlation, it would be valu-          zone=""
able to maintain a searchable database of                server=""
“hosts who have recently tried to send us a              kf="/var/ra/Kupdate-ra.+157+43810.key"
virus,” so that subsequent e-mail launched
                                                         ( echo server $server
from these hosts can be rejected without ques-             echo zone $zone
tion. (This is useful even in the case of e-               echo prereq nxdomain $node.$zone
mail worms, since if a host is willing to send a           echo update add $node.$zone $ttl \
worm, it’s a safe bet that anything else offered           echo update add $node.$zone $ttl \
later on by the same host can be safely rejected              TXT created ‘date +%Y%m%d%H%M%S‘
                                                           if [ $# -gt 0 ]; then
or dropped.)                                                  echo update add $node.$zone $ttl \
                                                                TXT reason $@
      Given the wide spread adoption of DNS                fi
as a bearer of “real time blackhole list” (RTBL)           echo send
data [MAPSRBL], the problem can be reduced               ) | nsupdate -k $kf /dev/stdin

to:                                                      exit $?

     how can programs on our mail and                    This utility uses a shared key file (given as the
     other servers securely update a DNS                 nsupdate -k argument) to authenticate a dy-
     zone used as a “blackhole list” and                 namic update to the zone. Com-
     then grant all of our servers fast and              mand line arguments to this utility are as fol-
     coherent access to this zone?                       lows:
The full data flow necessary for this application         $1        host address
is shown in Figure 1.                                    $2..N     reason for entry

3.1. Software and Hardware Selection                     This utility adds one A RR (address record)
                                                         and one or more TXT RRs (text records) at the
The DNS server and client software used in this          name corresponding to the reversed (in-addr)
example was [ISCBIND] (version 9.3.0). The               form of the host address. For example, if host
operating system was FreeBSD 5.2.1/AMD64.       were added on 2004-05-28 at
                                                         14:12:24 (UTC) then the result would look
                                                         something like this:

 mail                                                                                mail
 server                                                                              server

  log                                                                                log
  file                                                                               file

                         dns server                       dns server
                         (primary)                        (stealth)
log file                                                                           log file
analyzer                                                                           analyzer

           dns/mail server                                           mail server

                             log file                    intrusion
                             analyzer                    detection



            web server                                                IDS server


                                      Figure 1. RTBL Data Flow

$ORIGIN                             3.2.3. Expiring Host Addresses –
137 A
    TXT "created" "20040528141224"
                                                          This PERL utility is not reproduced here. Its
Note that a prerequisite for this update is the           purpose is to output a list of hosts from our pri-
nonexistence of any records at the domain                 vate blackhole list that were added more than
name corresponding to this address. If this               45 days ago. These are then sent to the del-
condition is not met, then the update will not be utility for deletion.
applied.                                                        The design of this utility is simple. Dur-
                                                          ing a zone transfer (AXFR) of the zone, matched
                                                          sets of A (address) and TXT (text) records are
     For this “blackhole list,” only the exis-
                                                          found, and if a created date is available and
tence of an A RR (address record) matters,
                                                          if that date is more than 45 days in the past,
and the exact address does not matter. Thus is used. The TXT RR (text record)
                                                          then the host address is sent to stdout. The
                                                          CPAN [CPAN] Net::DNS [NETDNS] module
shown is used for expiry (see Section 3.2.3).
                                                          was the protocol engine for this utility and was
                                                          perfectly suitable.
3.2.2. Deleting Host Addresses –
                                                               A BSD UNIX crontab command is
#!/bin/sh                                                 used to run this utility on a nightly basis:
zone=""                                         4 2 * * * /var/ra/ | \
server=""                                           xargs /var/ra/

for N
                                                          3.2.4. Monitoring Mail Server Logs –
     echo $N |                                            Our mail server ([POSTFIX]) is configured
     awk -F.
       ’{print $4 "." $3 "." $2 "." $1}’                  with the following body_checks rule:
   echo server $server                                           /^UEsDBAoA/ DISCARD mailworm1
   echo zone $zone
   echo prereq yxrrset $node.$zone a                      This rule is one of many that cause inbound
   echo prereq yxrrset $node.$zone txt
   echo update delete $node.$zone                         mail to be checked for “worm” content. As in-
   echo send                                              dicated by the rule, such messages are discard-
done | nsupdate -k $kf -t 600 /dev/stdin
                                                          ed without notice to the sender. As a side ef-
exit $?                                                   fect, a message is sent to the syslog facility
                                                          indicating that a message was discarded due
This utility’s arguments are a list of host ad-
                                                          to this rule, and telling the IP source address
dresses whose information is to be removed
                                                          of the sending host. The
from the private blackhole list. Each one is
                                                          PERL utility, which is not reproduced here,
converted to “in-addr” format and sent to the
                                                          reads the system log files using the BSD UNIX
nsupdate utility with prerequisites of exis-
                                                          tail -F command and searches for markers
tence of A (address) and TXT (text) records,
                                                          indicating that “worm” content was discarded.
and a request for deletion of all records at that
                                                          These are parsed to find the IP source address
                                                          of the sender, and the command

is run with the following arguments:                      names lead to it), the only connections we re-
                                                          ceive are from malfeasants who are random-
$1        host address                                    ly scanning the IP address space searching for
$2        mailserver hostname                             vulnerable servers. Our customized IDS pre-
$3        “watchmaillog”                                  tends to be vulnerable, but the only actual ef-
$4        mailworm identifier                              fect of sending us a malicious payload is to
                                                          have the source address and payload logged in
Assuming that host                         a database, and to feed the utility
sent a copy of mailworm2 to mail serv-                    with new data about malicious endsystem host
er on 2004-02-28 at 23:44:57                   addresses.
(UTC), the resulting zone content would be as                  In principle and by design, many other
follows:                                                  data sources could be added.
187 A                                           3.2.6. ISC BIND9 Configuration
    TXT "reason" "" \
        "watchmaillog" "mailworm2"
    TXT "created" "20040228234457"                        In [ISCBIND] (version 9.3.0), there were three
                                                          configuration activities necessary to make this
The utility is started at                 project possible.
boot time, just after the syslog and Postfix
services.                                                       First, a new zone had to be created by
                                                          hand, starting with just an SOA RR (start of
                                                          authority record) and some NS (name serv-
     Note that the Postfix body_checks rules               er) records.
are applied to every line of every inbound mes-
                                                               Second, a new cryptorandom key had to
sage. It’s necessary for a mail server to have
                                                          be created, using the following command:
a very fast CPU, and very high main memory
bandwidth, in order to apply a moderate num-                    dnskeygen -H 128 -n update. -h

ber of rules to millions of messages per day.                   Third and finally, the named.conf file
This is true of all virus and worm detectors,             had to be amended to include knowledge of
whether commercial, open source, or home                  this key and of this zone. That configuration
grown.                                                    file change is roughly as follows:
                                                          acl mynets {
3.2.5. Other Sources of Data                                 2001:4f8::/32;
Other services such as the Apache web server                 192.5.4/23;
also produce log files, and it is reasonable to
assume that each such service ought to have               key update-ra {
                                                             algorithm hmac-md5;
its own log file watcher/postprocessor that uses              secret "84qHhYEAB7cP00OVt8YD6Q=="; to record worm sources.                        };

      We also have a customized intrusion de-             zone "" {
tection system (IDS) that listens to a large sub-           type master;
                                                            file "pri/";
set of the IPv4 address space. Because this ad-             allow-transfer { mynets; localhost; };
dress space is “dark” (meaning that no domain               allow-query { mynets; localhost; };
                                                            allow-update { key update-ra; };

};                                                 ,
Let’s walk through these configuration ele-
                                                         Here we see a reference to an exempt.regex
ments one at a time. mynets is an access con-
                                                         file, which contains a list of regular expres-
trol list covering all local IPv6 and IPv4 net-
                                                         sions to match those recipients who want to
works under the same local administrative con-
                                                         receive all of their e-mail, even if it’s spam.
trol, and therefore trusted to be allowed to see
                                                         It’s because of this requirement that our
the content of the zone. It is im-
                                                         RTBL subscriptions are declared in smt-
portant to restrict this data to trusted endsys-
                                                         pd_recipient_restrictions rather than
tems since accidental use could result in un-
                                                         in smtpd_helo_restrictions. An exam-
wanted liability.
                                                         ple exempt.regex file would be:
    update-ra is the name of the key we
generated using dnskeygen, and its se-                   /postmaster@/               OK
                                                         /abuse@/                    OK
cret is the KEY RR (key record) from the
file, which must be securely copied and stored            3.3. Difficulties Encountered
on every host system where the                A number of weaknesses were exposed in both
and utilities will be used. Since             the protocols and their software implemen-
possession of this “shared secret” grants per-           tations during the private RTBL project de-
mission to update the zone, this file should be           scribed in this section.
made readable only by the root or other priv-
ileged user id.                                          3.3.1. Negative Caching

3.2.7. Postfix Configuration                               Because of negative caching, a “worm train” of
                                                         many inbound messages from the same source
In addition to the body_checks rules de-                 will not be immediately stopped by this tech-
scribed in Section 3.2.4 above, it is nec-               nology. This is because an initial RTBL check
essary to configure every Postfix server                   will result in a cached “no such name exists”
in the local enterprise as a subscriber to               condition in the local caching recursive name
your private RTBL. This is done in the                   servers. Any queries about this same domain
/etc/postfix/ file, in the smt-                    name for the next few minutes will be an-
pd_recipient_restrictions              setting,          swered from this cache, even if a dynamic up-
usually after the standard invocation of per-            date has caused new data to appear in the zone
mit_mynetworks, and ideally before any oth-              itself.
er RTBL subscriptions. Here is an example:                    Negative caching is a necessary part of the
smtpd_recipient_restrictions =                           DNS protocol, but it may be necessary to add
  permit_sasl_authenticated,                             another type of RTBL subscription directive to
                                                         Postfix, which would attempt a dynamic update
  ...                                                    with a prerequisite of the name’s nonexistence
  check_recipient_access \                               and no actual update operations. This would
  reject_rbl_client,                          defeat negative caching by making a round trip
  reject_rbl_client \                                    to the authority server for all domain names in,
  reject_rbl_client \                                    a specified RTBL. Such a feature could be dan-

gerously destructive toward DNS caching, and               reapplied to the zone at the next name server
would need to be disabled by default.                      restart. If this journal grows beyond a calculat-
                                                           ed limit, then it is applied in real time by writ-
3.3.2. Memory Limits                                       ing an updated zone to the file system and then
                                                           truncating the journal. This design is technical-
ISC BIND9 behaves poorly when it runs low                  ly correct, but creates two serious operational
on memory. All zone data in [ISCBIND] (ver-                problems.
sion 9.3.0) is stored in the heap, including the                 First, there is a lack of scheduler granu-
currently served version of a zone, any pri-               larity during the zone rewrite process, such that
or versions still being accessed by an out-                some update requests can be lost due to serv-
bound zone transfer, and unfortunately, delet-             er timeouts or even signalled server failures.
ed records whose heap memory has not yet                   Lost updates can include new worm data, or
reused by new data. If the expiration pro-                 deletions due to the nightly expiration process.
cess described in Section 3.2.3 is not run of-             BIND9 will have to be improved to ensure that
ten enough, or if the cutoff date is so far in the         all updates are properly handled even during
past that not enough data is purged on a night-            times of heavy background maintainance. Sig-
ly basis, then BIND9 will enter a state where              nalling a server failure is better than letting the
it cannot get enough heap space to do zone                 client experience a timeout, just as an exam-
maintainance.                                              ple of what “properly handled” means in this
      Recent experience has shown that a pro-              context.
cess memory limit of 1.5 Gigabytes (1,500                        Second, there is a hard upper limit on the
Megabytes) is enough to hold a private RTBL                number of updates BIND9 can accept per sec-
of about five million host addresses includ-                ond, and this limit is enforced by storage hard-
ing the associated reason and created text                 ware. The DNS UPDATE proposal [RFC2136]
records, assuming a 45-day expiry and a “class             requires that all updates be committed to stable
B” network (with 65,535 possible host address-             storage before an server responds to the update
es) feeding an intrusion detection device. Since           request. In BSD UNIX, this means an fsync
longer expiry periods and more IDS contribu-               system call will be made, which results in phys-
tors would yield a more accurate and more use-             ical I/O. Even the fastest RAID5 storage sys-
ful RTBL, these limits will have to be relaxed             tems on the market today can only do a few
somehow.                                                   dozen or a few hundred physical I/O operations
     Future work on ISC BIND9 will include                 per second. Clearly, some way will have to be
non-heap zone data storage methods, as well                found to “batch” these I/O operations, which
as a more robust recovery process when heap                may mean bending or amending the DNS UP-
memory is exhausted.                                       DATE protocol.

3.3.3. Zone Rewrite Granularity                            3.4. Future Work

[ISCBIND] (version 9.3.0) uses a journalling               In addition to the improvements mentioned in
storage system to record zone changes. In the              Section 3.3 above, work must continue toward
event of an untimely process death (such as a              the goal of packaging up these tools and pub-
program or server crash) these changes can be              lishing them for general use by the commu-
                                                    - 10 -

nity. Now that endsystem hosts who will be                   results so far are very encouraging. The DNS
spamming you in the weeks to come are prean-                 Services model is effective, and we hope that
nouncing their intent by trying to send identi-              this paper will encourage more applications to
fiable and malicious payloads, only great good                use this model.
could come from a generally available method
of shunning traffic from these hosts. This could
make the attacks less successful overall, but
could also assist with product liability lawsuits            [CPAN]
against monopoly providers of unsecure oper-                     Comprehensive PERL Archive Network.
ating system platforms, or against malfeasants                   URL
who take advantage of these insecure endsys-
tems.                                                        [ISCBIND]
                                                                 ISC BIND Home Page, Internet Sys-
     Some consideration is also being given
                                                                 tems Consortium, Inc. (ISC). URL
to creating a robust, high availability, public
RTBL based on these tools. With dozens or
hundreds of trusted parties feeding the system               [MAPSRBL]
and subscribing to it, it may become possible                   P. Vixie. MAPS RBL Usage, Mail
to so quickly and so thoroughly “shun” hosts                    Abuse Prevention System (MAPS). URL
running malicious software (“malware”) and            
hosts who mindlessly forward these payloads,
to provide a global economic disincentive to ei-             [NETDNS]
ther own, operate, abuse, or provide the hosts                   Perl interface to the DNS resolver. URL
responsible for almost all known forms and in-         
stances of Internet abuse as of this writing.
                                                                 Wietse Venema. The Postfix Home Page.
4. Conclusion                                                    URL
The Domain Name System has made a suc-                       [RFC1034]
cessful transition from a mostly static system                   P. Mockapetris. Domain Name System –
whose content could only change due to exter-                    Concepts and Facilities, IETF, 1987.
nal human action, to a vibrantly dynamic sys-
tem whose content can change frequently and                  [RFC1995]
robotically.                                                     M. Ohta. Incremental Zone Transfer in
      This new dynamicism offers the possi-                      DNS, IETF, 1996.
bility for new services to be delivered using                [RFC1996]
DNS as a conduit. The data model offered to                      P. Vixie. A Mechanism for Prompt Notifi-
these new services includes a global query pop-                  cation of Zone Changes, IETF, 1996.
ulation, and moderately large update popula-
tions, with high coherency, reliability and per-             [RFC2136]
formance.                                                        P. Vixie, et al. Dynamic Updates in the
     At least one new application has been                       Domain Name System, IETF, 1997.
built using the DNS Services model, and the
                                               - 11 -

    P. Vixie, et al. Secret Key Transaction
    Authentication for DNS, IETF, 2000.

    D. Eastlake. DNS Request and Transac-
    tion Signatures ( SIG(0)s ), IETF, 2000.

    S. Kwan, et al. Generic Security Service
    Algorithm for Secret Key Transaction Au-
    thentication for DNS (GSS-TSIG), IETF,

To top