Document Sample
paper=opensrc-log-mgt-lasso_paperonly Powered By Docstoc
					Project LASSO for Open Source Log Management

Dr. Anton Chuvakin


Security is a rapidly changing field of human endeavor. Threats we face literally
change every day; moreover, many security professionals consider the rate of
change to be accelerating. On top of that, to be able to stay in touch with such
ever-changing reality, one has to evolve with the space as well. Thus, even
though I hope that this document will be useful for to my readers, please keep in
mind that is was possibly written years ago. Also, keep in mind that some of the
URL might have gone 404, please Google around.

Drowning in logs is all too common a sight nowadays when organizations are
trying to struggle with a combination of operational, security and compliance
requirement. A typical organization will have logs coming from a wide array of
log sources such as server operating systems (Unix and Windows), desktops,
mainframes, network gear such as routers and switches, web proxies, security
gear such as network IDS, IPS or anti-virus tools, web, email, DNS server
software as well as enterprise applications.

Log Management and Intelligence is an approach to dealing with large volumes
of computer-generated log messages (also know as audit records, audit
trails, event logs, etc) which consists of log collection, centralized aggregation,
long-term retention, log analysis (in real-time and in bulk after storage) as well
as sharing the information with the relevant parties within the organization. Such
analysis is usually performed for security, operational (such as system or network
administration) or regulatory compliance reasons.

Effectively analyzing large volumes of diverse logs faces many challenges such
as huge log volumes (reaching hundreds of gigabytes of data per day for a large
organization), log format diversity, undocumented proprietary log formats (that
hinder analysis) as well as a presence of false log records in some types of logs
(such as intrusion detection logs)

Tools to handle the log collection and analysis are sometimes build by users,
assembled from various open source components or acquired from commercial
vendors, in the form of LMI or Log Management and Intelligence solutions. So
far, the open source space have not been able to come up with a single tool to
deal with a majority of log challenges, even though there are some promising
However, an open source community has been pretty effective in building pieces
of a log management infrastructure. Syslog-NG, that enables log collection from
Unix servers and network devices as well as serves as a better replacement for
standard syslog daemons provided by the operating system vendors, is a primary
example. Also, a huge number of simple scripts and small programs such as
logwatch, logsentry, fwanalog were written by the open source community over
the years to handle specific logs or with a particular slice of a log puzzle. At
times it seems that it was easier for some to create their own script instead of
looking for one online. However, a majority of these tools focused on Unix and
Linux platforms and largely ignored Windows.

One of the recent open source solutions that enable a critical part of log
management is Project LASSO. Project Lasso is Windows-based open source
software designed to collect Windows event logs, including custom application
logs, and provide central collection and transport of Windows log data via TCP
syslog to any syslog-NG compatible log receivers. Before Project LASSO,
incorporation of Windows server and workstation logs in an overall log
management process was extremely onerous. One had to use agents installed
on every single Windows system to collect logs or be stuck with super-expensive
proprietary solutions. And deploying agents on every system is one of the most
hated pursuits in the whole domain of enterprise IT.

Open source tools such as syslog-ng existed for years to simplify log
management for Unix and Linux operating systems as well as network devices
that support syslog (such as Cisco routers and firewalls), but Windows part of the
world was largely excluded since binary Windows event logs are not syslog.
Now, Project LASSO allows remote Windows log collection (it can also be
deployed as an agent on each server, if needed) and then inclusion of such logs
into a log management systems, such as the one by LogLogic or others.

Overall, Project LASSO enables connecting the dots by enabling users to collect
analyze Windows event logs with the same ease that they are used to in the Unix
and Linux realm. After the data is collected by LASSO, one can use report and
search to review and analyze logs across all the systems in the enterprise:
Windows, Unix, network systems, application, etc. For example, one can run a
query for the same user across all the systems he or she touched. Such
capability is critical for compliance as well as incident response and forensics

This is an updated author bio, added to the paper at the time of reposting in
Dr. Anton Chuvakin ( is a recognized security expert in the
field of log management and PCI DSS compliance. Anton leads his security
consulting practice, focusing on logging,
SIEM, security strategy and compliance for security vendors and Fortune 500
He is an author of books "Security Warrior" and "PCI Compliance"
( and a contributor to "Know Your Enemy II",
"Information Security Management Handbook"; and now working on a book
about system logs. Anton has published dozens of papers on log management,
correlation, data analysis, PCI DSS, security management (see list His blog is one of the most popular in the
In addition, Anton teaches classes (including his own SANS class on log
management) and presents at many security conferences across the world; he
recently addressed audiences in United States, UK, Singapore, Spain, Russia
and other countries. He works on emerging security standards and serves on
advisory boards of several security start-ups.
Dr. Anton Chuvakin was formerly a Director of PCI Compliance Solutions at
Qualys. Previously, Anton worked at LogLogic as a Chief Logging Evangelist,
tasked with educating the world about the importance of logging for security,
compliance and operations. Before LogLogic, Anton was employed by a security
vendor in a strategic product management role. Anton earned his Ph.D. degree
from Stony Brook University.

Shared By:
Description: Misc security dump