Notes on Continuous Integration Tools
Updated 01/27/09, Updated 02/02/09, Updated 03/02/09, Updated 03/06/09, Updated 07/17/10, Updated 08/10/10
Updated 12/08/10, Updated 02/14/11, Updated 03/25/11, Updated 04/02/11, Updated 04/08/11, Updated 05/12/11
Updated 06/05/11, Updated 06/30/11, Updated 08/11/11, Updated 10/07/11, Updated 11/05/11, Updated 12/31/11
Part of the Agile manifesto, and part of many other Extreme Programming practices, Continuous Integration is
taking a known good practice to the extreme level. If it is good to integrate often, let's keep the code integrated
always. The idea is to run the build and automated tests (at least the fast ones) whenever somebody checks code
into the version control system. Usually it is done in an automated manner by a tool such as CruiseControl, one of
the earliest examples of a continuous integration tool. Another similar tool is Jenkins (formerly called Hudson) or
Similar open-source tools include LuntBuild and Continuum. Most run an Ant script under control of a timer, and
have a web-based reporting front end.
See survey at http://java-source.net/open-source/build-systems.
See also review at http://www.ibm.com/developerworks/java/library/j-ap09056/.
See survey at http://www.developer.com/open/article.php/3803646/The-Best-Continuous-Integration-Tools.htm
Introducing Continuous Integration into Your Organization
Phase 1 – No Build Server: Software is built manually on a developer’s machine, though it may use an Ant
script or similar to do so. Source code may be stored in a central source code repository, but developers do
not necessarily commit their changes on a regular basis. Integration is difficult.
Phase 2 – Nightly Builds: The team has a build server, and automated builds are scheduled on a regular
basis. However, there are no tests to verify the build, and it may be broken for extended periods.
Phase 3 – Nightly Builds and Basic Automated Tests: The build script compiles the application and runs a
set of automated unit and/or integration tests. In addition to email, the build server also alerts team
members of integration issues using more proactive channels such as Instant Messaging. Broken builds are
now generally fixed quickly.
Phase 4 – Enter the Metrics: Automated code quality and code coverage metrics are now run to help
evaluate the quality of the code base and (to some extent, at least) the relevance and effectiveness of the
tests. The code quality build also automatically generates API documentation for the application. All this
helps teams keep the quality of the code base high, alerting team members if good testing practices are
slipping. The team has also set up a “build radiator,” a dashboard view of the project status that is displayed
on a prominent screen visible to all team members.
Phase 5 – Getting More Serious about Testing: Now, practices like Test-Driven Development are more
widely practiced, resulting in a growing confidence in the results of the automated builds. The application
is no longer simply compiled and tested, but if the tests pass, it is automatically deployed to an application
server for more comprehensive end-to-end tests and performance tests.
Phase 6 – Automated Acceptance Tests and More Automated Deployment: These automated tests use
Behavior-Driven Development and Acceptance-Test Driven Development tools to act as communication
and documentation tools and documentation as much as testing tools, publishing reports on test results in
business terms that non-developers can understand. Since these high-level tests are automated at an early
stage in the development process, they also provide a clear idea of what features have been implemented,
and which remain to be done. The application is automatically deployed into test environments for testing
by the QA team either as changes are committed, or on a nightly basis; a version can be deployed (or
“promoted”) to UAT and possibly production environments using a manually-triggered build when testers
consider it ready.
Phase 7 – Continuous Deployment: Confidence in the automated unit, integration and acceptance tests is
now such that teams can apply the automated deployment techniques developed in the previous phase to
push out new changes directly into production.
CruiseControl is both a continuous integration tool and an extensible framework for creating a custom continuous
build process. It includes dozens of plugins for a variety of source controls, build technologies, and notifications
schemes including email and instant messaging. A web interface provides details of the current and previous builds.
The standard CruiseControl distribution is augmented through a rich selection of 3rd Party Tools.
The current version is 2.8.4, released September 15, 2010.
It was developed by ThoughtWorks, which also provides a professional version called Cruise.
Build details screen:
CruiseControl is composed of 3 main modules:
The build loop: core of the system, it triggers build cycles then notifies various listeners (users) using
various publishing techniques. The trigger can be internal (scheduled or upon changes in a SCM) or
external. It is configured in a XML file which maps the build cycles to certain tasks, thanks to a system of
plugins. Depending on configuration, it may produce build artifacts.
The jsp reporting application allows the users to browse the results of the builds and access the artifacts
The dashboard provides a visual representation of all project build statuses.
This modularity allows users to install CruiseControl where it will best fit their needs and environment.
Using remoting technologies (HTTP, RMI), it is possible to control and monitor the CruiseControl build loop.
Jenkins (formerly Hudson)
Jenkins monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those
things, currently Jenkins focuses on the following two jobs:
1. Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell,
Hudson provides an easy-to-use so-called continuous integration system, making it easier for developers to
integrate changes to the project, and making it easier for users to obtain a fresh build. The automated,
continuous build increases the productivity.
2. Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run
on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and
it is up to you to look at them diligently and notice when it broke. Hudson keeps those outputs and makes
it easy for you to notice when something is wrong.
Jenkins and Hudson are different forks of the same project. They each have a large community of third-party
The current Jenkins version is 1.462, released April 2012.
The next Hudson version is 3.0.0 to be released in second quarter of 2012.
The current Hudson version is 2.2.0, released in December 2011. The 2.x releases started in early 2011. The key
features to note are:
Improvements to the JSR 330 Injection mechanism including the option to disable it for extreme
Dialog based user action confirmation introduced
Plugin/extension handling improvement
Option to clean workspace before build
Improvements in maven project build runtime
See our document “Notes on Jenkins and Hudson”.
Hudson offers the following features:
1. Easy installation: Just java -jar hudson.war, or deploy it in a servlet container. No additional
install, no database.
2. Easy configuration: Hudson can be configured entirely from its friendly web GUI with extensive on-the-
fly error checks and inline help. There's no need to tweak XML manually anymore, although if you'd like
to do so, you can do that, too.
3. Change set support: Hudson can generate a list of changes made into the build from CVS/Subversion.
This is also done in a fairly efficient fashion, to reduce the load on the repository.
4. Permanent links: Hudson gives you clean readable URLs for most of its pages, including some permalinks
like "latest build"/"latest successful build", so that they can be easily linked from elsewhere.
5. RSS/E-mail/IM Integration: Monitor build results by RSS or e-mail to get real-time notifications on
6. After-the-fact tagging: Builds can be tagged long after builds are completed
7. JUnit/TestNG test reporting: JUnit test reports can be tabulated, summarized, and displayed with history
information, such as when it started breaking, etc. History trend is plotted into a graph.
8. Distributed builds: Hudson can distribute build/test loads to multiple computers. This lets you get the
most out of those idle workstations sitting beneath developers' desks.
9. File fingerprinting: Hudson can keep track of which build produced which jars, and which build is using
which version of jars, and so on. This works even for jars that are produced outside Hudson, and is ideal for
projects to track dependency.
10. Plugin Support: Hudson can be extended via 3rd party plugins. You can write plugins to make Hudson
support tools/processes that your team uses.
TeamCity (developed by the company behind intelliJ) is one of the comprehensive continuous build management
systems, offering a wide range of tests and reports. For instance, it will analyze code using intelliJ’s review tools,
and also supports immediate testing directly from your IDE.
It offers quick setup, out of the box continuous integration and unit testing, code quality analysis, pre-tested
commits, early reporting on build problems.
Build Grid uses a computing grid to run parallel builds on multiple Build Agents with different platforms
Amazon EC2 integration for rapid Build Grid scaling
Pre-tested commits help avoid breaking the code in the repository and optimize the code integration cycle
On-the-fly test results reporting with intelligent tests re-ordering
Build configuration templates for easier builds replication
Comprehensive server configuration and Build Grid management console
Role-based, per-project user administration with user groups support
LDAP support with automatic user profile synchronization Assigning and taking responsibility over the
Configurable per-user and per-user group notifications
Detecting hanging builds, with a thread dump to help investigate the cause
Customizable statistics on build duration, success rate, code quality and custom metrics
Change history browsing with a Diff Tool and "History" builds
Integration with issue tracking systems: JIRA, Bugzilla and JetBrains YouTRACK out of the box, more
with 3rd party plugins
Dedicated continuous integration for Maven and Gradle build systems
Integrated code coverage and duplicates finder for Java and .NET
Integration plugins for most major IDEs with automatic navigation between the web browser and your
There are several editions, with the Professional Edition being free for up to 20 user accounts, and then unlimited for
the Enterprise Edition. See chart below:
Important news: Starting with TeamCity 6.5 all Professional Edition restrictions were removed except one: the
maximum number of build configurations (20). TeamCity Professional edition users now have:
Unlimited number of users.
Ability to use any authentication scheme.
Adjustable per-project permissions.
This basically means that all features of the Enterprise Edition are now available in Professional Edition.
This is the dashboard that defines all of the builds and their schedule:
From here, we can trigger manual builds when needed.
This is the detail for the main build:
From here, we can review changes, review the tests that succeeded or didn’t, and view the build (console) log.
We can also view the SVN information about changes:
We can view the results from the test programs:
Luntbuild is a powerful build automation and management tool. Continuous Integration or nightly builds can be
easily set using a clean web interface. Executed builds are well managed using functions such as search,
categorization, promotion, patching, deletion, etc. It also acts as a central build artifacts repository and download
area for your whole team.
You may ask why Luntbuild, while there are already many good build automation tools such as Cruise Control,
Anthill, and others. Our answer is:
Luntbuild aims at not only being build automation, but also build management system. Executed builds
can be categorized, promoted, searched, patched, deleted, etc. This is very important for a nightly builds or
Continuous Integration builds, which generate many builds over time.
Luntbuild utilizes the project/schedule concept to easily provide build support for parallel development.
No configuration files, all jobs are done using web interface. You can set up your Luntbuild system in
less than thirty minutes.
The current version is 1.6.3, released January 4, 2010.
It is available in free and professional versions (called QuickBuild).
The web-interface is fronted with security that prevents users from seeing stuff you don't want them to see. Any
user can be a project admin (where they can edit a project), a project builder (where they can kick off builds), or a
project viewer (read-only access to a project). Or you can keep some users from even seeing some projects.
Continuum is one of the newest CI servers on the scene, and a respectable newcomer at that. The setup and
configuration of Continuum is straightforward: simply download and extract a ZIP file, run a command program,
and you're good to go. The Web-based interface makes it easy to configure a project. What's more, you don't need
to install a Web server either because Continuum embeds the Jetty Web server. Furthermore, Continuum can run as
a Windows service and also embeds context-sensitive documentation into certain sections of the application, which
provides a great deal of help.
Continuum offers the following features:
1. Easy installation : Download the standalone application and run it or deploy the Continuum war in your
servlet container. Read more
2. Easy Configuration : Projects builds are auto-configured but they can be configured easily with the web
3. SCM support : CVS, Subversion, Clearcase, Perforce, Starteam, Visual Source Safe, CM Synergy, Bazaar,
Mercurial are supported
4. Change set support : For each build result, Continuum prints all scm changes ( commit authors, commit
logs, modified files)
5. Build notification : Mail, Jabber and Google Talk, MSN, IRC, report deployment with wagon
6. Build tool support : Maven 1 and 2, ANT, shell scripts
7. External Access : External tools can interact with Continuum with XMLRPC API. Read more
8. Build type : Manual, scheduled and push (with xmlrpc) build technique are supported
9. Build template : Users can define default build templates to use by project type
10. Build queue : Users can view all projects in the queue and interrupt some builds
11. Distributed Builds : Projects can be distributed and built in multiple build agents.
12. Parallel Builds : Projects can be built simultaneously or concurrently locally using multiple build queues.
The current version is 1.3.8, released September 18, 2011.
There is an article at http://dendiz.verisux.com/2008/10/30/apache-continuum-review-and-operation/.
Set up the general configuration: