Learning to Talk Cisco IOS
Inferring the IOS Command Language from Router Configuration Data
Donald Caldwell Seungjoon Lee Yitzhak Mandelbaum
AT&T Labs Research AT&T Labs Research AT&T Labs Research
Abstract There are many aspects to managing a network – from
maintenance of its physical infrastructure to planning
Networks and the functionality that they deliver are of its logical architecture. A critical part of network
growing increasingly complex, making network manage- management is configuration management: managing
ment a steadily growing challenge. A key component the detailed configurations (or configs for short) of the
of network management is configuration management. hundreds of or even thousands of control components –
Router configurations capture and reflect all levels of net- mostly routers – that run the network. Router configu-
work operation, and it is highly challenging to manage rations capture and reflect all levels of network opera-
the detailed configurations of the potentially huge num- tion: from the logical structure of the (possibly many)
ber of routers that run a network. While tools exist to networks, both real and virtual, of which they are a part,
help automate network configuration management, none to the multi-faceted routing policies of those networks, to
of the tools of which we are aware directly address the the physical links, interface cards and controllers [3] that
problem of feature evolution and expansion. form the building blocks of the networks that they drive.
We present the design and implementation of a learn- Yet, looking at a single router reveals little of the entire
ing system for the Cisco IOS router configuration lan- picture. As a result, router configurations are extremely
guage and an adaptive parser built on top of the learn- brittle – small changes to a single router configuration
ing system, which constitute the first step to building a can have a massive impact on the network as a whole.
configuration management system capable of adapting to
Network operators, however, hardly have the luxury of
new and evolving network features. Based on valid con-
leaving router configurations alone. As networks evolve,
figurations, the learning system infers the IOS language,
both in terms of the users they include and the features
from which we can generate a parser for IOS. Our tools
they support, router configurations must change with
allow for incremental learning and relieve a network ana-
them. The pace of network evolution is such that the
lyst of the burden of writing an IOS parser and maintain-
set of router configurations in a network can change on
ing it as the IOS language changes. We have built a pro-
a daily basis. Network owners – particularly for large
totype system and validated its accuracy and efficiency
networks – must devote tremendous resources to ensure
by running it on the configuration files of Tier-1 ISP net-
that their networks can be simultaneously robust yet flex-
works. Our results show that from only 81 configuration
ible. Software must written, and re-written (or purchased
files, we can learn enough IOS to successfully parse all
and repurchased, as the case may be); operators must be
of the 819 IOS configurations in under 10 minutes.
trained and retrained to keep up with this constant evolu-
tion.
1 Introduction Manually configuring and analyzing router configura-
tions (even with the aid of simple commands like grep)
VPN. MPLS. Multicast. NAT. IPv6. Networks for the becomes highly challenging, time-consuming, and basi-
Internet and the functionality that they deliver are grow- cally infeasible. Automating network configuration man-
ing increasingly complex. As users become more reliant agement to the greatest extent possible is therefore criti-
on networked applications, greater-and-greater demands cal to the well being of the network (and its human op-
are placed on the network, both in terms of performance erators). Several automated network configuration and
and functionality. As a result, network management is analysis tools of this variety are proposed in the litera-
becoming steadily more difficult. ture [6, 8, 17]. However, none of the automated tools of
1
In this paper, we present the design and implementa-
tion of two components of the larger management sys-
tem: a system for learning a close approximation of
Cisco IOS– the configuration language of Cisco’s routers
– and an adaptive IOS parser that is built on top of
the learning system. Although existing network analy-
sis tools include parsers for Cisco IOS [6], their parsers
were written by hand and need to be manually updated
to incorporate new commands. In contrast, our adaptive
parser evolves with the network to account for new ele-
ments in router configurations. As features are added to
routers, the parser is able to effectively and predictably
parse their configurations into data structures or data for-
Figure 1: Design of configuration management sys- mats that are suitable for use in downstream processing
tem and its relationship with configurations and network and analysis.
tools. The focus of this paper is parsing for configuration Our learning system infers an approximation of Cisco
management (highlighted in thick line). IOS based on a set of manually selected, valid configu-
ration files, and generates a description of the inferred
format in PADS / ML, a human-and-machine readable data
description language [16]. The learning system relies
which we are aware directly address the problem of fea- upon a repository of domain knowledge about Cisco IOS
ture evolution and expansion. Instead, they address its both to parse the training configs and to guide the cre-
symptom: the complexity of network management. Yet, ation of the inferred description, so that its structure is
if these systems will succeed, they must be able to eas- predictable and human-readable. These characteristics
ily adapt to the changing features of the network, or they are critical in facilitating further configuration process-
will quickly become obsolete. ing and analysis.
Our adaptive parser efficiently parses Cisco IOS con-
figs “in the wild,” and adapts to changes in the IOS lan-
1.1 Adapting to the Network guage with little manual intervention. The key com-
We are currently designing and building an adaptable ponents of this system are the learning system and a
network configuration management system, particularly parsing harness that converts inferred descriptions into
IOS parsers that can translate configurations into a range
focusing on processing and extracting information in ex-
isting configurations in networks. The goal of this ef- of desired formats, including XML. When configura-
fort is to enable our network management tools to han- tions with unrecognized commands are encountered, our
dle evolving technologies, like new network services and parser can reinvoke the learning system (based on user
new router vendors and models, with minimal mainte- guidance) to update the previously inferred description
nance effort. In Figure 1, we depict the design of the new with knowledge of the new command.
system and its relationship with raw configurations and While the designs of our learning system and adaptive
higher-level network tools. As mentioned above, router parser are applicable to many router configuration lan-
configurations are so diverse and complex that it does not guages, our implementations are specific to Cisco’s IOS
scale for individual network tools to handle raw config- configuration language. We have chosen to focus our ef-
urations directly. On behalf of such network tools, the forts on Cisco router configurations both because Cisco
configuration management system (1) deals with router- routers are widely deployed and because their configura-
specific configuration details, (2) performs common con- tion language is particularly difficult to parse. We intend
figuration processing (e.g., generating topology informa- to expand our systems to include other configuration lan-
tion), (3) provides higher-level tools with uniform inter- guages in the near future.
faces to accessing entire configuration information, and This paper makes the following contributions:
(4) helps those tools focus on their own objectives with- • We present a learning system that infers an approx-
out dealing with highly complicated router-specific de- imation of the Cisco IOS language from valid router
tails. For example, a BGP analysis tool retrieves all the configuration files and generates a formal descrip-
BGP configuration details through the well-defined inter- tion of the language in PADS / ML.
face of the configuration management system and does
not need to know whether BGP configuration commands • We present an adaptive IOS parser that leverages the
are different across vendors (e.g., Cisco and Juniper). learning system to adapt to changes in the IOS lan-
2
key chain kcname
guage. The parser also translates parsed configs into key 2
a form suitable for downstream processing. key-string 3 04FEEDDADBADBEEF123
!
class-map match-any classname
• We have implemented the ideas presented in this description "Class description"
paper and validated them through experiments us- match ip dscp cs6
match ip dscp cs7
ing real Cisco router configurations from Tier-1 !
ISP networks. Our results show that with only a policy-map interfacename_input
class classname
small training set, the resulting parser can accu- police cir 12800 bc 2400 be 4800
rately parse hundreds of configurations (millions of conform-action transmit
exceed-action set-prec-transmit 4
lines combined) in less than 10 minutes. The en- !
tire system is implemented in less than 1100 lines interface POS2/0
description interface description
of code (including blank lines), with approximately ip vrf forwarding 123:123
600 devoted to the learning system and 500 to the ip address 135.12.34.65 255.255.255.252
ip accounting precedence input
parser. pvc 0/1919
service-policy input interfacename_input
The rest of this paper is organized as follows. In Sec- !
tion 2, we provide an overview of the Cisco IOS com- router ospf 12
log-adjacency-changes
mand language and discuss the difficulties of parsing area 0 authentication message-digest
configurations written in this language. In Section 3, we network 135.12.34.64 0.0.0.3 area 0
maximum-paths 6
give an overview of the PADS / ML system, and illustrate !
its use on a small IOS example. Section 4 presents the router bgp 7018
neighbor 12.23.34.45 activate
details of the learning system and the general IOS de- neighbor 12.23.34.45 send-community
scription on which it is based, and Section 5 describes neighbor 12.23.34.45 route-map InRMName in
neighbor 12.23.34.45 route-map OutRMName out
how the learning system is used for adaptive parsing. In !
Section 6, we present experimental results based on our rtr key-chain kcname
!
prototype system. Section 7 reviews related work, and banner exec +
Sections 8 and 9 conclude. ============= WARNING =================
This system is intended strictly for use
by authorized users.
+
2 Cisco IOS Configuration Files
Figure 2: Sections from an IOS configuration file.
IOS is the operating system of Cisco’s routers and IOS
commands are used to configure routers running IOS.
The set of IOS commands and their syntax is referred to 2.2 IOS Command Oddities
as the IOS language, and collections of IOS commands
The above description of IOS captures its logical,
are referred to as IOS configurations. In this section, we
command-independent structure. Unfortunately, that
provide a high-level overview of Cisco’s IOS language
logical structure is not expressed in configs in a consis-
and present some examples that illustrate why processing
tent manner across all commands. Rather, a set of sev-
IOS configuration files in a structured fashion is difficult.
eral common, complex syntactic structures exist, all ex-
pressing the same logical structure. In addition, some
2.1 IOS Overview commands have their own, unique structure, shared by
no other commands. Figure 2 shows several examples
IOS is a declarative language and its basic element is a
of complex structures in an IOS configuration. We now
“command”. There are simple commands, whose influ-
elaborate on those relevant to our learning system, in ad-
ence does not extend beyond a line, and mode commands,
dition to highlighting some of the oddities of the IOS for-
which change the state of the command interpreter. Each
mat that make parsing difficult.
mode has a restricted set of allowable commands, which
may be simple commands or mode commands. In Fig-
Most sections do not have a clear end. Except for
ure 2, key-string and description in lines 3 and
a few of the newer commands, IOS mode commands
6 are simple commands, and key and class-map are
do not have an explicit command to end their section,
mode commands. We call the portion of the configura-
as illustrated by interface, router ospf, and
tion in which a mode is active a section. Commands can
router bgp in the figure. In these cases, a parser must
have arguments which set particular parameters of the
either use indentation for clues, know the set of com-
router operating system. For example, the area com-
mands that implicitly indicate that the section has ended,
mand in the router ospf [4] section in Figure 2 spec-
or know the set of commands allowed in each section
ifies the authentication method to use in OSPF area 0.
after consulting the manual.
3
Each command has its own format. The variety of language, supporting functional, imperative and object-
syntax structures suggests that each organization re- oriented programming [14]. In addition to generating
sponsible for a router function designs its own com- software, PADS / ML provides a framework for develop-
mand structure to be stitched into larger recursive de- ing generic tools in OCAML– functions that can operate
scent parser. Some commands are followed by a set of on any description-derived data structure, regardless of
key/value pairs, others by a mixture of key/value pairs its type. PADS / ML does the work of specializing generic
and positional parameters (parameters whose meaning tools written in its framework to the particular data struc-
depends on their position, like the UNIX shell), and yet tures derived from a description.
others have wholly positional parameters. The extensive, end-to-end support that PADS / ML pro-
vides for processing legacy data formats makes it a nat-
Some commands have an irregular format. While
ural choice for addressing the challenges of the IOS lan-
most simple commands are on a single line, some IOS
guage. The small codebase size mentioned in the intro-
commands define argument data on lines after the com-
duction (= min_indent];
ptype cmd_args = pstring_ME("/( .*)?/") start_cmd : command; peol;
ptype indentation = pstring_ME("/ˆ\\s*/") sub_cmds : block (length indent + 1)
plist(No_sep, Error_term)
ptype command = { }
negate_opt : pstring_lit("no ") poption;
name: cmd_name; and delimited_section (args : int * delim_set) = {
arguments: cmd_args; let min_indent = fst args;
} let delim_set = snd args;
Figure 5: Utility definitions in general format of IOS con- indent_ds : [i:indentation | length i >= min_indent];
start_delim : [d:cmd_name | is_start_delim d delim_set];
figuration files. arguments_ds: pstring_SE(peor); peol;
(* ptype ... *) let end_delim = get_end_delim start_delim delim_set;
and block (min_indent : int) = let term_re = "/" ˆindent_ds ˆ" *" ˆend_delim ˆ"/";
DSection of delimited_section (min_indent,
section_delimiters) sub_cmds_ds : block (length indent_ds)
| TDSection of two_dep_section (min_indent, plist(No_sep, Regexp_term (term_re));
two_dep_sections) pre term_re; peol
| DelimitedBanner of delimited_banner (min_indent) }
| DataSection of data_section (min_indent)
| StandardSection of section (min_indent) and two_dep_section (args : int * string list) = {
| Comment of pre "/\\s*[!].*$/" precord let min_indent = fst args;
| BlankLine of pre "/\\s*$/" precord let cmd_set = snd args;
indent_tds : [i:indentation | length i >= min_indent];
ptype file_header = ... negate_opt_tds : pstring_lit("no ") poption;
ptype top_level_block = block (0) first_cmd : [c : cmd_name | List.mem c cmd_set];
ptype source = file_header ’ ’; second_cmd : cmd_name;
arguments_tds : pstring_SE(peor); peol;
* top_level_block plist(No_sep,EOF_term)
sub_cmds_tds : block (length indent_tds + 1)
plist(No_sep, Error_term)
Figure 6: Top-level definitions in general format of IOS }
configuration files.
Figure 7: Common section definitions in general format
in full in Appendix A. The description captures every- of IOS configuration files.
thing from the low-level, tokenization structure of the
configs to the high-level structure of the entire config. end of the file (EOF). The type block describes a co-
Perhaps the most important part lies between these two: herent unit of a configuration file. A “unit” can be one
the description of the command and sectioning structure; of many different syntactic constructs, so block is de-
in particular, the multiple different syntactic forms of fined using a datatype. Its alternatives include the differ-
sections. ent section forms, including indentation based sections,
In Figure 5, we show some of the basic building blocks delimited sections and two-dependency sections; a “ban-
of the IOS description. First, there are type definitions ner” form (essentially, delimited free-form text); com-
for command names, command arguments and leading ment lines; and blank lines. The min_indent argu-
indentation. These definitions are similar to the token ment of block specifies the minimum indentation re-
definitions one might find in grammar-based languages, quired of those alternatives whose parsing depends on in-
in that they are based on regular expressions: the type dentation. Note that the block definition uses the and
pstring_ME describes a string matching its regular- keyword instead of ptype because it is part of a group
expression argument. Notice that the description of of mutually-recursive types, beginning with section
command arguments matches any string (preceded by in Figure 72 .
a space) – this generality is because we usually do not The most common section forms are described in Fig-
need to examine the arguments of a command to learn ure 7. All section types are parameterized by their
the structural information in which we are interested. minimum required indentation, and their actual inden-
The last type describes a complete command: an op- tation is checked against the parameter3 . The first field
tional “no,”1 followed by the command name and its ar- (indent...) field describes the indentation preceding
guments. the first command of the section. It detects when a de-
Figure 6 contains a description of the top-level struc- 2 We have used mutual recursion to simplify the presentation, al-
ture of an IOS config. Reading from the bottom up, the though it is not yet supported by PADS / ML (it is currently under devel-
opment). The real description, as shown in Appendix A, uses single
type source describes the entire config as a header fol- recursion and polymorphism; an equivalent, if somewhat less readable,
lowed by a list of top-level blocks, terminated by the form.
3 The pervasiveness of indentation-checking is an optimization –
1 When present, “no” typically turns a feature off, indicates that it we could localize it to types describing indent-based sections, but that
has no value, or frees resources associated with a specified object. The would result in a less efficient parser. We are investigating ways of
exact meaning, though, is command-dependent. efficiently localizing indentation concerns.
7
crease in indentation level signals the end of a command mands (among other things). The tool iteratively parses
group using a constraint: a lesser indentation violates the configuration file, parsing one top level command at
the constraint, resulting in a parse error. If the pars- a time. The iterative nature of the processing allows us
ing is occurring within the context of an Error_term- to easily process large configuration files without bring-
terminated list, then the parse error will signal an end to ing the entire config into memory, thereby increasing the
the list. scalability of the tool.
The first type in Figure 7, section, describes the In the process of parsing, the parser will classify each
simplest section form: a single command, with zero or command according to its syntactic representation and
more children of greater indentation. Field start_cmd tag it accordingly. Also, the in-memory representation
describes the first command of the section, while field of parsed commands is a tree-shaped structure much like
sub_cmds describes the list of its subcommands, each our IOS model. The essential difference is that the model
of which must have a minimum indentation at least one will have at most one entry for any given command,
greater than the current indentation. Notice the de- whereas the parsed data will have as many entries as
pendence of the type of sub_cmds on the value of there are appearances of the command within its respec-
field indent: this kind of dependency cannot be ex- tive section (or the config itself, if it is a top level com-
pressed in a context-free grammar, and is a key feature mand).
of PADS / ML. Once a top level command has been parsed, we look
The next type describes a section which uses ex- up the command in the model. If it is new, then a new en-
plicit delimiting commands to indicate the start and try is created in the model for that command. If it already
end of the section, rather than depending on inden- exists, then the existing entry is updated and checked for
tation. An example such section is address family, consistency with the newly parsed data. Either way, we
which begins with address-family and ends with iterate through all of the commands in a given section
exit-address-family. One of the arguments to and recursively update each one.
this section type is a set of start-delimiter commands
which is used to check whether the current block is a
delimited section: if the first command is in the start set, 4.4 From Model to PADS/ML Description
then it is, otherwise it is not. Again, we use a constraint Once all of the commands in a config have been pro-
to express this check. cessed, the resulting model is a tree that logically repre-
The final type describes sections whose subcommand sents the elements of IOS encountered so far. The next
member-set depends on the command name and its first step is to convert this knowledge into a PADS / ML de-
parameter. We call such sections two dependency sec- scription. We have designed a straightforward conver-
tions, as explained in Section 2.2. The ip section has sion from models into PADS / ML descriptions where the
this characteristic. description exactly mirrors the model. Each IOS com-
mand is described with its own type definition. Sections
4.3 Modeling IOS are defined in terms of their structure and the list of their
valid members. The set of valid members is described
For every IOS command encountered, the essential in- with a switched datatype. Switched datatypes, like stan-
formation that we seek to record is the syntactic struc- dard datatypes, also describe format variation but for for-
ture of the command and its relationship to other com- mats in which a discriminant expression in the descrip-
mands. We represent the IOS format in memory using tion determines which variant is appropriate, rather than
a simple tree-shaped data structure. The root of the tree an analysis of the data. For section member-set descrip-
represents the config itself. The remaining nodes each tions, the command name is used as the switch discrim-
represent a particular IOS command. The tree structure inant. Every section has an Other member, used to
implicitly encodes the relationships between commands: capture lines, like comments, that are not formatted like
the children of a particular node represent the valid sub- commands. Such lines are identified when the command-
commands of the represented command. The remaining name lookahead returns an empty string. They are de-
information about the command is included in the node scribed as a string encompassing the remainder of the
itself. In particular, the node carries the command name line (technically: a string that stops with the regular ex-
and its type: a simple command or one of the complex, pression peor, defined as “$” in the built in library). The
section commands. config as a whole is described as an optional header fol-
We are now ready to describe the process of build- lowed by a list of valid top level commands, terminated
ing a model from valid configuration files. It begins by EOF.
with the general format description: from the descrip- The description has a recursive structure, with the def-
tion, PADS / ML generates a parser for top-level IOS com- initions of a command’s children grouped together with
8
(* *** Output by meta_cisco_v3, version 3.4 *** *)
open Cisco_utils much like the example in Figure 4, although it uses the
pextern cisco_command(name:string) imported standard_section type rather than a sim-
pextern cisco_unparsed
pextern delimited_banner ple plist. The description of the ip section includes
pextern data_section only one inferred child, vrf, and specifies that ip is
pextern (cmd_set) cmd_lookahead
pextern (cmd_set) standard_section a two-dependency section. Finally, the description of
pextern (cmd_set) two_dep_section (first_dep:string) the entire config contains the expected members Ip and
(* *** BEGIN SECTION config_ip *** *)
(* *** BEGIN SECTION config_ip_vrf *** *) Other and a new, catchall member Unparsed, for any
ptype config_ip_vrf_description = cisco_command precord lines formatted like commands, but not recognized as be-
ptype config_ip_vrf_export = cisco_command precord
ptype config_ip_vrf_maximum = cisco_command precord longing to any config members. The underbar ’ ’ is a
(* *** [NAME:route-target] *** *) wildcard pattern in OCAML, matching any value of the
ptype config_ip_vrf_route’target = cisco_command precord
ptype config_ip_vrf__members(next_cmd:string) = discriminant.
pmatch next_cmd with
"description" -> Description of
config_ip_vrf_description
| "export" -> Export of config_ip_vrf_export 4.5 ... and Back Again
| "maximum" -> Maximum of config_ip_vrf_maximum
| "route-target" -> Route’target of The explanation above of the form of IOS descriptions
config_ip_vrf_route’target
| "" -> Other of pstring_SE(peor) precord
can be formalized as a PADS / ML description, which we
ptype config_ip_vrf = refer to as the meta description. With such a PADS / ML
config_ip_vrf__members standard_section
(* *** END SECTION config_ip_vrf *** *)
description, the process of printing an IOS description
ptype config_ip__members(next_cmd:string) = changes from a one step process of printing the model
pmatch next_cmd with
"vrf" -> Vrf of config_ip_vrf
as an IOS description into a two step process of translat-
| "" -> Other of pstring_SE(peor) precord ing the model into the in-memory representation of an
ptype config_ip =
config_ip__members two_dep_section("ip")
IOS description (defined by PADS / ML based on the meta
(* *** END SECTION config_ip *** *) description) and then passing this data to the PADS / ML-
ptype config__members(next_cmd:string) =
pmatch next_cmd with
generated printer. As the model and the representation
"ip" -> Ip of config_ip are isomorphic, the translation code is boilerplate.
| "" -> Other of pstring_SE(peor) precord
| _ -> Unparsed of cisco_unparsed precord Why do we bother to specify a PADS / ML meta descrip-
ptype config = config__members cmd_lookahead tion of IOS? There are a number of reasons. First, we
plist(No_sep, EOF_term)
feel that the combination of translation code and meta
description is more easily understood than a function that
Figure 8: Inferred description for ip vrf section of simultaneously translates from the model to the descrip-
Figure 3. tion and prints the physical syntax of the description.
Perhaps more compelling, though, is that to support in-
the definition of the command itself, the children appear- cremental inference, we need to parse IOS descriptions
ing before the parent. Section definition beginnings and in addition to printing them. PADS / ML provides us with
endings are marked explicitly in stylized comments. To both a parser and printer for our trouble of specifying the
reduce the size of the IOS description and increase its meta description.
readability, we have factored shared command and sec- Incremental inference is essential to efficient adaptive
tion structure out of generated descriptions and into a parsing, as we will discuss in Section 5. It allows users
fixed set of polymorphic types, where possible. These to take an IOS description, reconstitute it as an IOS model
shared types have been bundled in a separate, auxiliary and then update the model based on new configurations.
description that is referenced by every generated IOS de- The updated model can then be printed as an IOS de-
scription. scription just like any other model. So, when a network
Running our learning program on the commands in analyst encounters a config that is not valid according to
Figure 3 results in the description shown in Figure 8. The the current description, she need not rerun the inference
first line opens the auxiliary description of shared types tool on all the training configs. She can run it on the new
so that subsequent lines can import particular elements config alone by providing the current description to the
of that description. Next begins the definition of the ip tool as input from which to rebuild the current model. A
section, followed immediately by the beginning of the side benefit of this feature is that it allows the tool to eas-
vrf subsection. The treatment of vrf as a subsection is ily checkpoint its inference progress when it is run on a
caused by our description of ip as a “two-dependency” very large training set.
section in the general IOS description. Next comes the Once we have a PADS / ML description of the IOS meta
description of vrf’s members, with route-target format, adding support for incremental inference and
annotated to indicate that its name has been escaped checkpointing is trivial: we need only write boilerplate
(’-’ replaced by ’’’). The section description concludes code to invert the translation from the model to represen-
9
ip vrf 1023
description VPN for Best Bank of America
route-target import 100:3
route-target import 117:7
route-target export 119207:15144
Figure 9: The relationship between domain knowledge
and learning in the inference process.
export map To_NY_VPN
maximum routes 150 80
tation and PADS / ML does the rest.
4.6 Manual Format Specification Figure 10: XML translation of ip vrf section from
For all of our efforts building a robust inference system, Figure 3.
sometimes IOS still thwarts our tool because the inden-
tation of the configuration that comes directly from the key components of this system are the incremental IOS
router is deceptive. This can be because the router breaks learning system described in Section 4 and a parsing har-
up long lines or because the portion of IOS that prints ness that translates descriptions from the learning system
the configuration fragment emits the “wrong” indenta- into efficient IOS parsers and converters. We begin with
tion. For example the police command, which resides a description of the parsing harness and its XML conver-
in the class subsection of the policy-map section sion component and then detail its interaction with the
of a configuration is sometimes broken into 3 lines with learning system.
indentation that suggests that the *-action parameters are
not parameters but subcommands. (See Figure 2.) In this
case, there are invalid configs that will teach our tool the 5.1 The Parsing Harness
wrong format. Our parsing harness has two components: a set of func-
While manually correcting the emitted description is tions that drive parsing and XML-conversion and provide
not difficult, the user must be assured that his changes the user with a simple command-line interface, and a
will remain if ever the description is incrementally up- module for converting parsed data into XML. Into this
dated. Therefore, we provide users with the ability to harness is placed the inferred IOS description from which
manually modify (or create from scratch) any command PADS / ML generates a third software component: a mod-
in an IOS description and then annotate the command ule (a library in OCAML terminology) for processing IOS
with a comment indicating that the command is locked. data.
This locking will prevent the system from modifying the As with the learning system, the driver function parses
description of the command and all its subcommands (if iteratively – one top-level command or section at a time,
any). rather than parsing the entire config at once. After pars-
In Figure 9 we illustrate the relationship between do- ing such a config element, it converts it to XML and then
main knowledge, in the form of the general format de- discards it. Some sample XML output is shown in Fig-
scription and any manual format specification, and the ure 10. The driver function also takes note of parsing
learning process, as they work together to produce a errors and logs them appropriately. Given the simplic-
PADS / ML description of the IOS language.
ity of the process, the driver function is quite small and
changes very infrequently – mostly, just to add conve-
5 Adaptively Parsing Cisco IOS nience features.
The function that converts data into XML is a generic
We now present a system for efficiently parsing Cisco function, meaning that it works on data of any type. It is
IOS configs “in the wild,” which can adapt to changes written using PADS / ML’s generic programming toolkit,
in the IOS language with little manual intervention. The specifically the original one [16]. The use of a generic
10
function is critical to the success of the the conversion 6 Evaluation
tool, for a number of reasons. First, the size and com-
plexity of the type of parsed IOS commands are directly We have built a prototype system for our learning parser
proportional to the size and complexity of the IOS for- and been using it as a part of a more general configura-
mat. Therefore, writing a tool that would work directly tion management tool being developed. In this section,
on data of this type (that is, parsed IOS commands) would we report the performance of our prototype using hun-
be a very tedious process. Second, while each IOS de- dreds of router configurations for Tier-1 ISP networks.
scription shares the same metastructure, they each de- We first describe the experiment environment before pre-
fine different PADS / ML types. Correspondingly, the in- senting the results.
memory representation of each one is different, with dif-
ferent OCAML types. Therefore, a (non-generic) func-
tion written to work on one IOS representation will not
6.1 Experimental Set-up
work on another IOS representation. This incompatabil- We use 819 Cisco router configurations in our experi-
ity would require us to modify and possibly even rewrite ments. As described in Section 4, our parser needs a
the conversion function each time a new description is subset of configurations to infer the language structure.
inferred – again, a tedious and error-prone process. The In our experiments, we first randomly select a small
generic tool framework saves us from both of these has- number of configurations and use them as input to the
sles. We write one, generic, XML converter and PADS / ML learning component. Then, using the inferred model, we
adapts it to each inferred description. try to parse the whole set of 819 configurations and see
whether the model can parse all the configurations or not.
If not, we randomly select a subset of the configurations
that the model failed and use them as input to the next
iteration of learning. We repeat this procedure until we
5.2 Adapting to Change
can parse all the 819 configurations. Although such a
selection procedure for learning was sufficient for our
A parsing system is created by running the learning sys- experiments, we envision the input selection can poten-
tem on a set of valid, pretty-printed IOS configs to create tially be further automated in the future. We perform our
a command-specific IOS description. The inferred de- experiments on SUN Fire-15000 multi-processor server
scription is then placed in a harness from which an ini- (900MHz CPU) running Solaris 5.8.
tial parsing and conversion program is built. Under nor-
mal circumstances, the parsing and conversion program
created with the harness will run without modification. 6.2 Experimental Results
Only when an unrecognized command(s) is encountered In Table 1, we report experimental results for our parser.
does it need to consider learning about that command(s) The first row of Table 1 denotes the number of configura-
so that it can correctly parse it in the future. If adapta- tions used for learning. In fact, since our parser employs
tion is warranted, then the learning system is invoked to incremental inference (Section 4.5), the values within
adapt the IOS description to the new command(s) and a parentheses are the numbers of additional configurations
new program is compiled from the result. used on top of previously inferred models. Specifically,
This feedback process is partially automated in our to complete the learning, our experiments needed five it-
prototype implementation. To facilitate the feedback erations, where we first used 4 configurations files, and
process, the parsing program outputs a list of the input subsequently added 4, 17, 25, and 31 files for respective
configs in which unrecognized commands were encoun- iterations.
tered. If the list is not empty, then manual intervention is In Table 1, we observe that we need only around 10%
required, but only to verify that each unrecognized com- of configurations (81 out of 819) to learn the language
mand is indeed a real, new command rather than corrupt model and achieve 100% success ratio. When we first
data. Our system assists the user in this process by dis- use 4 configurations for learning, we achieve only 32%
playing the relevant data to the user and only requiring success ratio (i.e., 263 out of 819). After another 4 con-
a simple accept/reject response for each config. Once figurations, the parser can parse around 61%, while the
this check has taken place, all of the uncorrupted configs success ratio becomes 93% after the parser learns from
can be passed to the inference tool, along with the most additional 17 configurations (or 25 total). Obviously, as
recent IOS description, to produce a new description ca- we increase the size of training set, the parser can han-
pable of parsing the new commands. Finally, the new dle more top-level commands. Specifically, when we
description is placed back in the harness and the parsing use only 25 configurations, the parser learns 43 top-level
and conversion program is rebuilt. commands such as interface, MPLS, NTP, etc. After us-
11
Number of Configs 4 8 25 50 81
(Increment) (4) (4) (17) (25) (31)
Parsing Success Ratio 0.32 0.61 0.93 0.96 1.00
No. of Sections Learned 37 40 43 52 56
Time to Learn Additional Configs (sec) 19.6 7.6 70.4 69.0 106.9
Table 1: Performance when we vary the number of configurations for learning
ing 81 configurations, the model contains 13 more top- Learning System Parser
Learning engine 324 Parsing engine 173
level commands (or 56 total) including SNMP, VLAN,
Specific meta-description 106 XML formatter 220
etc. This set of 56 top-level commands is sufficient to General description 146 Common types description 65
parse all the given configurations. Miscellaneous 17
Subtotal 576 Subtotal 475
Our result illustrates that our learning model can han-
dle the the following scenario well. Let us suppose that Table 2: Line count for different sections of code. The
while the current model learned from 25 configurations total number of lines is 1051.
works well with the current network configuration, a net-
work operator decided to introduce several new features
in the network (e.g., new services). To enable our parser
to parse those new commands, the network operator just 7 Related Work
needs to select some of configurations containing the
new features and re-train the parser, which in turn will A large number of network management proposals and
pick up those new commands and change its model to tools involve static analysis of router configuration.
be able to parse all the configurations including the new These tools enable network operators to detect miscon-
commands. figurations in the current network, debug problems be-
fore introducing new features, and perform “what-if”
analysis for network planning and management. Cald-
well et. al. [6] propose configuration management tool
called EDGE, which reads all router configurations in a
Running Time In Table 1, we report the time to finish network, extracts network-wide information from them,
learning from different numbers of configurations. For and creates a database and relevant reports. Although
example, it takes 19.6 seconds to infer the model from EDGE employs a parser that handles whole configura-
the first 4 configurations. The next iteration takes 7.6 tions, its parser does not learn by itself and does re-
seconds to combine the previous inferred model with the quire manual maintenance whenever changes in con-
new commands in 4 additional configurations. We ob- figuration language or new features are introduced. In
serve that as the learning involves more configurations contrast, our parser can learn such changes automati-
and a model with more commands, the running time cally, and involves minimal human interaction for main-
tends to increase. (Although not shown in the table, the tenance. Feamster and Balakrishnan [8] develop BGP
average number of lines per configuration is similar for analysis tool called rcc, which statically analyzes BGP
all iterations.) In the experiments for Table 1, we used configuration in routers and detects misconfigurations.
5 iterations, which takes a total of 273 seconds to com- The work by Xie et. al. [18] describes how to compute
plete the learning. We also performed experiments with end-to-end reachability of a given flow based on packet
only one iteration using the final set of 81 configurations, filter and routing protocol configurations. While the tools
which took around 240 seconds to generate the identical mentioned above exclusively employ static analysis of
model. This illustrates that the iterated learning does not router configurations, network management tools using
significantly degrade the running time, compared to the dynamic information (e.g., route updates) or data-plane
ideal scenario where we know which configurations to information (e.g., amount of traffic) often make use of
use for learning. Finally, we report on the performance global topology view, which is usually obtained through
of actual parsing. There are millions of command lines static analysis of router configurations [9, 17]. Our pars-
combined in the 819 configurations, and it takes around ing scheme is an essential building block for such static
9 minutes to parse all the configurations. We believe analysis, and can improve the effectiveness of these net-
this parsing speed is fast enough for typical configura- work management tools.
tion processing, which is usually done on a periodic ba- Another direction for configuration management re-
sis (e.g., daily). In Table 2, we report the breakdown of search is to specify configurations for different routers
lines of code for different parts in our implementation. in an automated fashion. Gottlieb et. al. [12] specifically
12
focus on BGP and present how to generate router config- ment system that can provide higher-level network tools
uration to provision a new BGP-speaking customer. Ex- with generic configuration information while allowing
tending automatic configuration generation to multiple for efficient handling of new features in networks. One
services, PRESTO [7] generates device-specific configu- obvious next step is to build a component that integrates
rations for a number of services and composes them into router-specific semantic knowledge with parsed router
a single router configuration. In contrast to these config- configurations to construct generic views of networks
uration generation tools, our work deals with parsing ex- from multiple perspectives (e.g., OSPF, BGP, security).
isting router configurations, based on which independent Configuration files are not the only place where IOS
tools can extract information and perform configuration commands appear. Logs of commands executed on the
analysis. router, called tacacs logs, often provide an audit trail
Outside of configuration management, there has been of IOS commands. The language learned our system
considerable work studying the problem of grammar in- should prove useful for analyzing commands extracted
duction. De La Higuera provides a recent survey [13]. from these logs.
However, this body of work is far more general than the
system that we present and their results far less appli-
9 Conclusion
cable to the problem of network configuration manage-
ment. We are starting with an accurate grammar (that Network configuration management is a daunting task,
is, description) for pretty-printed configs, which drasti- and only growing more complex. In this paper, we have
cally simplifies the problem of inferring a full descrip- presented two important additions to the network config-
tion. In fact, our description of pretty-printed configs is uration manager’s toolbox and validated them with pos-
one of the contributions of the present work. Addition- itive experimental results. We have designed and imple-
ally, we are looking for a very particular structure to the mented a system that can learn a powerful approxima-
inferred grammar – correctness alone is not enough. The tion of the IOS language. In addition, we have presented
inferred grammar must allow for straightforward transla- one application of the learning system – an efficient IOS
tion of the parse tree into XML with a known schema. parser and XML-converter that can adapt to changes in
The most closely related work from the area of gram- the IOS language with little human intervention.
mar induction is a concurrent project by Fisher et. al.,
which explores the inference of a format description
from ad hoc data without human intervention or domain 10 Availability
knowledge [11]. Once again, the generality of their result
PADS / ML is open-source software, available from
makes their work largely inapplicable to our problem.
In particular, their system cannot recognize the complex www.padsproj.org. We are pursuing permission to
functional dependencies between commands and sub- release the learning system and the adaptive parser un-
commands, and is therefore unable to infer the relation- der an open source, or non-commercial use, license. If
ships between commands which are essential to the IOS granted, the software will be made available together
language. with the PADS / ML distribution.
References
8 Future Work
[1] Cisco IOS Configuration Fundamentals Command Reference,
Currently, the process of selecting training configura- Release 12.2T.
tions is ad hoc, and overly reliant on manual effort. We [2] Cisco IOS Dial Services Command Reference, Release 12.1.
are interested in investigating methods of further assist- [3] Cisco IOS Interface and Hardware Component Command Refer-
ence, Release 12.3.
ing the user to determine the suitability of a config file.
Each version of IOS adds new commands to the lan- [4] Cisco IOS IP Command Reference, Volume 2 of 3: Routing Pro-
tocols, Release 12.2.
guage. To improve their chances for acceptance by cus-
[5] Cisco IOS Quality of Service Solutions Configuration Guide, Re-
tomers, some router manufacturers have created configu- lease 12.2.
ration languages very much like IOS, resulting in a num- [6] C ALDWELL , D., G ILBERT, A., G OTTLIEB , J., G REENBERG ,
ber of IOS “dialects.” Supporting version- and dialect- A., H JALMTYSSON , G., AND R EXFORD , J. The cutting EDGE
specific parsers may be useful for providing more rig- of ip router configuration. SIGCOMM Comput. Commun. Rev.
orous config validation and supporting a broader range 34, 1 (2004), 21–26.
of routers. A parser supporting the union of commands [7] E NCK , W., M C DANIEL , P., S EN , S., S EBOS , P., S POEREL , S.,
G REENBERG , A., R AO , S., AND A IELLO , W. Configuration
might also be useful.
Management at Massive Scale: System Design and Experience.
As mentioned in Section 1, the motivation for this In USENIX Annual Technical Conference (Santa Clara, CA, June
work was the design of a larger configuration manage- 2007).
13
ptype data_block (min_indent : int) = {
[8] F EAMSTER , N., AND BALAKRISHNAN , H. Detecting BGP (* Break out the first data line to discover the
Configuration Faults with Static Analysis. In 2nd Symp. on indentation. *)
Networked Systems Design and Implementation (NSDI) (Boston, indent_db : [ i : indentation | length i >= min_indent ];
MA, May 2005). data_token; peol;
( * data_token * peol)
[9] F ELDMANN , A., G REENBERG , A., L UND , C., R EINGOLD , N., plist(No_sep, Error_term)
AND R EXFORD , J. NetScope: Traffic engineering for IP net- }
works. IEEE Network 14, 2 (2000), 11–19. ptype data_section (min_indent : int) = {
indent_ds : [ i : indentation | length i >= min_indent ];
´
[10] F ERN ANDEZ , M., F ISHER , K., F OSTER , J. N., G REENBERG , data_cmd : command; peol;
M., AND M ANDELBAUM , Y. A generic programming toolkit (* Throw away the actual data. Just need to know that it
for PADS/ML: First-class upgrades for third-party developers. In is there. *)
data_block (String.length indent_ds + 1)
PADL (2008). }
[11] F ISHER , K., WALKER , D., Z HU , K. Q., AND W HITE , P. From ptype (block) section (min_indent : int) = {
dirt to shovels: Fully automatic tool generation from ad hoc data. indent : [ i : indentation | length i >= min_indent ];
start_cmd : command; peol;
Unpublished manuscript, July 2007. sub_cmds : block (String.length indent + 1)
[12] G OTTLIEB , J., G REENBERG , A., R EXFORD , J., AND WANG , J. plist(No_sep, Error_term)
}
Automated provisioning of BGP customers. IEEE Network 17, 6
ptype (block) two_dep_section (args : int * string list) = {
(2003). let min_indent = fst args;
[13] H IGUERA , C. D. L. Current trends in grammatical inference. let cmd_set = snd args;
indent_tds : [ i : indentation | length i >= min_indent ];
Lecture Notes in Computer Science 1876 (2001), 28–31. negate_opt_tds : pstring_lit("no ") poption;
´
[14] L EROY, X., D OLIGEZ , D., G ARRIGUE , J., R E MY, D., AND first_cmd : [c : cmd_name | List.mem c cmd_set]; ’ ’;
second_cmd : cmd_name;
VOUILLON , J. The objective caml system, release 3.10, docu-
arguments_tds: pstring_SE(peor); peol;
mentation and user’s manual, 2007. sub_cmds_tds : block (length indent_tds + 1)
[15] M ANDELBAUM , Y. The Theory and Practice of Data Descrip- plist(No_sep, Error_term)
}
tion. PhD thesis, Princeton University, Sept. 2006. type delim_set = (string * string) list
[16] M ANDELBAUM , Y., F ISHER , K., WALKER , D., F ERNANDEZ , let is_start_delim = List.mem_assoc
M., AND G LEYZER , A. PADS/ML: A functional data description let get_end_delim = List.assoc
language. In POPL (2007). ptype (block) delimited_section (args: int*delim_set) = {
[17] S HAIKH , A., AND G REENBERG , A. OSPF Monitoring: Archi- let min_indent = fst args;
let delim_set = snd args;
tecture, Design and Deployment Experience. In Proc. USENIX
Symposium on Networked Systems Design and Implementation indent_ds : [ i : indentation | length i >= min_indent ];
(NSDI) (March 2004). start_delim : [d : cmd_name | is_start_delim d delim_set];
arguments_ds: pstring_SE(peor);
[18] X IE , G., Z HAN , J., M ALTZ , D. A., Z HANG , H., G REENBERG , peol;
A., H JALMTYSSON , G., AND R EXFORD , J. On static reacha-
bility analysis of IP networks. In Proc. IEEE INFOCOM (Mar. let end_delim = get_end_delim start_delim delim_set;
2005). let term_re = "/" ˆindent_ds ˆ" *" ˆend_delim ˆ "/";
sub_cmds_ds : block (length indent_ds)
plist(No_sep, Regexp_term (term_re));
A General Cisco IOS Description pre term_re;
peol
ptype cmd_name = pstring_ME("/[A-Za-z][A-Za-z_0-9-]*/") }
ptype data_token = pre let section_delimiters =
"/([ˆA-Za-z]|([A-Za-z][A-Za-z_0-9-]*[ˆA-Za-z_0-9 -])).*$/" [("address-family","exit-address-family");
ptype cmd_args = pstring_ME("/( .*)?/") ("boot-start-marker","boot-end-marker")]
ptype indentation = pstring_ME("/ˆ\\s*/")
let two_dep_sections = ["ip"; "router"; "service"]
ptype command = {
negate_opt : pstring_lit("no ") poption; ptype block (min_indent : int) =
name: cmd_name; DSection of block delimited_section (min_indent,
arguments: cmd_args; section_delimiters)
} | TDSection of block two_dep_section (min_indent,
two_dep_sections)
let length = String.length | DelimitedBanner of delimited_banner (min_indent)
let last_contains delim reps _ = | DataSection of data_section (min_indent)
match reps with | StandardSection of block section (min_indent)
r::_ -> (String.contains r delim) | Comment of pre "/\\s*[!].*$/" precord
| _ -> false | BlankLine of pre "/\\s*$/" precord
ptype config_info = {
ptype delimited_text = { "Current configuration : "; size : pint32;
delimiter : pchar; ’ ’; unit : pstring_SE(peor)
text : pstring_SE(peor) precord }
plist(No_sep,Pred_term(last_contains delimiter)) ptype file_header = {
} "Building configuration..." precord poption;
ptype delimited_banner (min_indent : int) = { punit precord poption;
indent : [ i : indentation | length i >= min_indent ]; config_info precord poption
"banner "; }
banner_type : pstring(’ ’); ptype top_level_block = block (0)
’ ’;
banner_text: delimited_text ptype source = file_header
} * top_level_block plist(No_sep, EOF_term)
14