Learning Center
Plans & pricing Sign in
Sign Out



Misc security dump

More Info
									      Building Security In
      Editors: John Steven,
      Gunnar Peterson,

                     How	to	Do	
                     Application	Logging	Right

                                              s threats shift toward applications and as more                            logs are suitable for manual, semi-
                                                                                                                         automated, and automated analy-
                                              companies struggle with compliance man-                                    sis. Ideally, you can analyze them
                                                                                                                         without having the application
                                              dates, the need for useful, comprehensive                                  that produced them—and defi-
                                                                                                                         nitely without having the appli-
                                              application logging can only increase. Here                                cation developer on call. From
                                                                                                                         the log management viewpoint,
                     we provide guidance on application logging to application                                           the logs can be centralized for
                                                                                                                         analysis and retention. Finally,
Anton                developers and architects and to                     So, sometimes we must deal                     they won’t slow the system down
ChuvAkin             security professionals.                           with log messages like these:                     and can be proven reliable, if used
SecurityWarrior                                                                                                          as forensic evidence.
Consulting           Application                                       • Aug 11 09:11:19 xx null
                     Logging Today                                       pif ? exit! 0 (which carries                    What to Log
GunnAr               Organizations have finally gotten                   absolutely no meaning),                         So, what types of events should
Peterson             network device logging and—to                     • Message 202 User tran-                          you log?
Arctec Group         some extent—server logging un-                      sitioning             priv        level            The first type is authentication,
                     der control. However, after get-                    (which conveniently omits the                   authorization, and access events:
                     ting used to neat Cisco Adaptive                    actual user identity—the use
                     Security Appliance or other fire-                   of “secret” numeric codes also                  • successful and failed authentica-
                     wall logs and Linux “password                       causes trouble if no documenta-                   tion or authorization decisions;
                     accepted” messages, security in-                    tion is available), or                          • system access, data access, and
                     cident investigators trying to re-                • userenv[error] 1040 XYZ-                          application component access;
                     spond to the next wave of attacks                   CORP\wsupx No descrip-                            and
                     have been thrust into the horrific                  tion available (which scores                    • remote access, including from one
                     world of application logging.                       points for both honesty and use-                  application component to another
                         Problems with many applica-                     lessness).                                        in a distributed environment.
                     tion logs are truly staggering. Logs
                     are often missing, they omit criti-                   In light of this, how do we                       The second type is changes:
                     cal details, or they have no stan-                guide application developers and
                     dard form or content. On top of                   architects toward creating secu-                  • system or application changes
                     this, many security practitioners                 rity audit logs that are useful for                 (especially privilege changes),
                     must deal with debugging logs                     forensics and monitoring and that                 • data changes (including creation
                     masquerading as security audit                    help satisfy compliance mandates                    and destruction), and
                     logs. Table 1 illustrates the key                 such as the Payment Card Indus-                   • application and component in-
                     differences between the two.                      try Data Security Standard (PCI                     stallation and changes.
                         Debugging logs appear in                      DSS)? We can start by establish-
                     application frameworks more                       ing criteria for good security au-                    The third type is availability issues:
                     frequently than well-designed se-                 dit logs (which we just call “logs”
                     curity audit logs. However, using                 from now on).                                     • startups and shutdowns of sys-
                     them for investigations is often                                                                      tems, applications, and applica-
                     an exercise in frustration because                Logging Criteria                                    tion modules or components;
                     they might not contain key details                From a high level, the best logs                  • faults and errors, especially er-
                     needed for incident response and                  tell you exactly what happened,                     rors affecting the application’s
                     forensics.                                        and when, where, and how. Such                      availability; and

82	                  COPUBLISHED	BY	THE	IEEE	COMPUTER	AND	RELIABILITY	SOCIETIES							■						1540-7993/10/$26.00	©	2010	IEEE							■						JULY/AUGUST	2010
                                                                                                           Building Security In

    Table 1. Comparing security audit logs and debugging logs.

                                            Security audit logs                     Debugging logs
    Intended consumers                      Security and audit personnel            System operators and developers
    When the logger is on                   Always                                  Sometimes
    Message content                         Attacks, activities, and faults         Faults, failures, and errors
    Scope of what to log                    Known in advance                        Unknown
    Length of usefulness                    Years                                   Hours or days

• backup successes and failures             sixth W is actually an H.) This            quire much digging for a hid-
  that affect availability.                 ancient wisdom applies perfectly           den reason. Remember, the log’s
                                            to logs and helps define a useful,         customers are the security and
      The fourth type is resource issues:   unambiguous log entry.                     audit personnel.
                                                On the basis of the six Ws, the      • The action helps answer “how”
• exhausted resources, exceeded             following list provides a starting         by providing the nature of the
  capacities, and so on;                    point for what to include:                 event.
• connectivity issues and prob-                                                      • In addition, the priority helps in-
  lems; and                                 • The username helps answer                dicate the event’s importance.
• reached limits.                             “who” for those events relevant          However, a uniform scale for
                                              to user or administrator activi-         rating events by importance is
    The final type is “badness” or            ties. In addition, it’s helpful to       impossible because different or-
threats:                                      include the name of the identity         ganizations will have different
                                              provider or security realm that          priorities. (For example, dif-
• invalid inputs and other likely             vouched for the username, if             ferent companies might have
  application abuses and                      that information is available.           different policies regarding in-
• other security issues known to            • The object helps answer “what”           formation availability versus
  affect the application.                     by indicating the affected system        confidentiality.)
                                              component or other object (such
   Creating a comprehensive “what             as a user account, data resource,         So, a useful log message might
to log” list for every application            or file).                              look like this:
and organization is impossible.             • The status also helps answer
However, our list should provide a            “what” by explaining whether           2010/12/31 10:00:01AM
useful starting point for your cus-           the action aimed at the ob-            GMT+7 priority=3,
tom applications, especially those            ject succeeded or failed. (Other       system=mainserver,
dealing with regulated data such              types of status are possible, such     module=authentication,
as payment cards or sensitive per-            as “deferred.”)                        source=,
sonal information.                          • The system, application, or compo-     user=anton(idp:solar),
                                              nent help answer “where” and           action=login,
What to Include                               must provide relevant applica-         object=database,
Next, what data should you log                tion context, such as the initiator    status=failed,
for each event, and at what level of          and target systems, applications,      reason=“password
detail should you log it? The phi-            or components.                         incorrect”
losophy of relevant log details goes        • The source helps answer “from
back to ancient Greece (http://               where” for messages related to         This message has a field explain-                network connectivity or distrib-       ing the failure’s reason. Also, it
and focuses on the “Six Ws”:                  uted application operation.            isn’t in XML; human readability is
                                            • The time stamp and time zone           useful in logs, and computers can
•   Who was involved?                         help answer “when.” The time           deal with name=value pairs just
•   What happened?                            zone is essential for distribut-       as well as with XML.
•   Where did it happen?                      ed applications. In addition to
•   When did it happen?                       the time stamp and time zone,          What Not to Include
•   Why did it happen?                        some high-volume systems use a         Certain details should never be
•   How did it happen?                        transaction ID.                        logged. Some examples are ob-
                                            • The reason helps answer “why,”         vious: logs should never contain
(No, we can’t explain why the                 so that log analysis doesn’t re-       application or system passwords.

      Building Security In

                 Table 2. Defending against SQL injection attacks.

                 Layer                    Injection defense             What the logger will report
                 Presentation layer       Form input validation         An invalid-input event—that is, a failed white list or blacklist
                                                                        input validation event
                 Business logic layer     Business logic, rules, or     A failed authentication, authorization, or access event—that is,
                                          policies                      failed validation based on business logic, rules, or policies
                 Data access layer        Prepared statement or         A resource issue event—that is, an SQL syntax error event
                                          parameterized query

                (Sadly, this sometimes still hap-         structure and behaviors from the          each layer. To help correlate these
                pens with Web applications.)              business logic, business rules, en-       reports, the logger often adds a
                                                          terprise policies, and Web front          transaction or message exchange
                Centralization                            ends to the data structures and           ID to facilitate log analysis when
                As we mentioned before, easy              storage. This gives the application       reconstituting the events.
                centralization of logs is essential       context, which is everything to
                for distributed log analysis across       an incident responder.                    Sanitizing Audit Records
                either multiple systems or mul-                                                     Typically, logging subsystems are
                tiple application components of a         Locating the                              placed to detect events around
                distributed application. Although         Logging Service                           sensitive assets, which means that
                syslog has been a flawed but de           The logging service’s location re-        they’ll come into contact with
                facto standard of log centralization      lates directly to the type of events      sensitive data. Sanitizers can filter
                owing to its easy User Datagram           the logger can see and the data           and remove sensitive data from
                Protocol delivery, modern cross-          and context available to the log-         logs. A sanitizer’s location (see
                platform application frameworks           ger. The logger is responsible for        Figure 1) is important because it
                call for the publish/subscribe            discerning the event type, gather-        determines whether sensitive data
                model for log delivery. In this           ing context, and writing informa-         is filtered by the log browser or
                case, a security-monitoring tool          tion. To discern events, the logger       removed from persistent storage
                can request a subscription to a par-      must be able to view the event’s          (sensitive data is never stored in
                ticular type of logged event—and          source, its payload, and the object.      the log).
                receive all relevant logs in nearly       Additional context includes such
                real time, if needed.                     information as the authority and          Storage Forecast—
                                                          security domains.                         Cloudy with a Chance
                Know Your Customer                            Given a Web application attack        of Compromise
                When you add logging func-                such as SQL injection, a logger in        Most systems store logs inside the
                tionality to your application,            the presentation layer might be           enterprise, but as with many IT
                you’re not just building security         unable to discriminate valid input        areas, the cloud offers new op-
                in, you’re building visibility in.1       from invalid input (an attack). But       portunities and potential solu-
                That visibility is the information        in the business logic layer (where        tions and problems. The cloud
                in the log that will be viewed by         business rules are applied) or data       has proven to be an effective way
                your log’s customer—the inci-             access layer (where SQL state-            to store data. However, because
                dent responder. To maximize the           ments and data connections are            in the cloud storage model, the
                log’s usefulness, you must un-            made), increased knowledge of the         data is stored and possibly pro-
                derstand the incident responder’s         object environment provides bet-          cessed outside enterprise secu-
                requirements. These might be a            ter visibility for the application as a   rity, challenges remain owing
                particular audit record format, in-       whole, and specifically the logger,       to requirements for encryption
                tegration with network security           to flag input as a possible attack.       and other controls. For example,
                monitors, or listening for specific           Table 2 describes how a typi-         PCI DSS provides a starting point
                types of events.                          cal three-layer architecture de-          for log storage requirements, but
                   One thing is for sure: ap-             fends against SQL injection. It           meeting them might be difficult
                plication developers can gather           also gives an example of how the          in a cloud model.
                context that’s simply unavailable         visibility at each layer drives what
                anywhere else in the system.              the logger reports.
                The application is the concrete
                implementation of the software’s
                                                              The same malicious user input
                                                          might be reported differently at          I   n addition to our basic con-
                                                                                                        clusion—You must log!—we

84	             IEEE	SECURITY	&	PRIVACY
                                                                                                                      Building Security In

must remind you that logging’s
importance will only grow. In                      Logging
                                                  subsystem                    Log                    Sanitizer                  Browser
particular, the need to analyze                                                                                                              Analyst
application behavior for secu-
rity issues across distributed and,
soon, cloud-based applications
calls for us to finally get logging                Logging
                                                  subsystem                  Sanitizer                  Log                      Browser
under control.                                                                                                                               Analyst
    Software architects and devel-
opers must “get” logging; there’s
no other way. This is because
infrastructure logging from net-            Figure 1. The sanitizer’s location is important. It determines whether sensitive data is (a) filtered by
work devices and operating sys-             the log browser or (b) removed from persistent storage.
tems won’t cut it for detecting
and investigating application-level
threats. Security teams will need           and logging mechanisms while                   Anton Chuvakin is a security consul-
to guide developers and architects          ensuring that their logs record all            tant specializing in log management
through useful, effective logging.          relevant information.                          and Payment Card Industry Data Se-
    Certainly, logging standards                                                           curity Standard compliance. Contact
such as Mitre Common Event Ex-              Acknowledgments                                him at; www.
pression ( will help,         We thank Raffy Marty of Loggly for   
but several years might pass before         his thoughtful review of the draft
they develop and their adoption             article.                                       Gunnar Peterson is managing prin-
increases. Pending a global stan-                                                          cipal of Arctec Group. Contact him at
dard, organizations should quickly          Reference                            
build and implement their own                1. R. Bejtlich, “Build Visibility In,”
standard using the guidelines we                blog, 13 Aug. 2009; http://tao                    Selected CS articles and columns
presented. They should also use                           are also available for free at
standard-language APIs, libraries,              build-visibility-in.html.        

    CG &A       IEEE Computer Graphics and Applications bridges the theory and practice of computer graphics.
                From speci c algorithms to full system implementations, CG&A offers a unique combination
                of peer-reviewed feature articles and informal departments. CG&A is indispensable reading
                for people working at the leading edge of computer graphics technology and its applications
                in everything from business to the arts.

                Visit us at


To top