The key to successful build management in distributed environments is a foundational commitment to consistency, repeatability and portability. This is just as true for small, homogeneous development environments using in-house, scripted build systems, as it is for large, complex environments, where a new class of non-scripted, distributed build-management tools are now available.
In small- and medium-sized environments, using a properly implemented in-house build system can mitigate many of the risks and challenges associated with builds. Additionally, with a few simple steps, the burdensome task of developing and maintaining build scripts can be significantly reduced, using build tools such as Make and Ant.
Build Management Evolution
Application builds have traditionally been managed using a rules-based program derived from Make, the world's oldest, best-known build tool. Make controls the generation of executables and other non-source files from a program's source files. There are many Make versions, each with a unique file syntax that precludes portability between development tools, operating systems, or even compilers within the same operating environment.
But Java development requires a new, platform-independent build tool, which led to the creation of Apache Software Foundation's Java-based Ant, so-named by developer James Duncan Davidson because, "...it is a little thing that can build big things." Ant eliminates Make's platform-dependent wrinkles, and is extended not with shell-based commands but Java classes. Configuration files are XML-based, calling out a target tree where various tasks get executed. Each task is run by a Java class that implements a particular task interface.
Ant is powerful, but the XML configuration scripts can create limitations. XML does not handle conditional logic effectively and it is, therefore, difficult to use Ant to write "intelligent" build scripts that support robust batch processing.
Additionally, many development projects include Java and non-Java components that require both Ant and Make,
as neither handles both languages. Scripting for the two is very different. Make scripts are the input to Make programs and dictate how to build each software component.
A Make file tells the Make program which binaries are to be created from which source modules. Make rules are then "fired" based on out-of-date conditions between source and object. In contrast, Ant/XML scripting uses serial batch processing. Rules for creating Java binaries such as .jar, .war, and .ear are handled statically for each step or "task" in the XML script. Listing 1 shows the differences between Make and Ant scripts for a similar type of build task.
For either approach, the programmer must understand not only how the application is constructed, but also the specific syntax requirements of the build scripting language they are using. Additionally, Make and Ant/XML scripts are not re-usable because static application information is coded into the script.
Make And Ant/XML Challenges
As Client/Server and Java development has evolved, so has build complexity, especially with Make. When Make is used recursively (one Make file per final target executable, or binary - the most common method of managing large build processes), an application with 50 binaries would require 50 Make files plus a "driver" Make file.
The system build is completed by calling the Make program repeatedly and passing a different Make file each time. Dependency checking between the individual Make files is impossible, which means large application Make files can't be managed by a single Make file. Developers get around Make challenges through clever file-ordering to track dependencies, along with object-borrowing and multi-system parallel building techniques to reduce the associated long system build times.
Although most Ant build systems do not appear to be as complex as many Make-based build systems, it is only a matter of time. As Ant scripts suffer from being passed through different developer's hands, as new technology emerges that effects the way Ant scripts are coded or used, and as applications grow more complex, Ant will encounter many of the problems associated with a Make-based system.
The key to avoiding this is to implement best-practices for manual scripting starting with an in-house build system, while monitoring factors that would signal the need to move to an automated, non-scripting approach.
Solving Typical Scripting Problems
Scripting challenges are easier to solve in small, homogeneous development environments confined to a single language and target operating system. The first step is to shift from a developer-centric view of builds to a team-centric view, and from the notion of scripting "my build solution" to scripting "our build solution."
Build inconsistency is the toughest problem. If developers use their own build scripts in the language or tool of choice, it can be difficult to know whether problems result from bad code or a bad build. Build administrators must standardize on a single scripting approach that best suits the language being used.
The first step to reducing build inconsistency between individual developer's build scripts is to develop build script templates that can be used as a basis for all build scripts. All builds require the same basic information: source code, compiler, and final target. Individual developers can populate the build script templates with their own build specifics; i.e., the source code location, compiler location, and a final-target description. Build script templates should be well-commented and clearly organized to ease the process of populating the template with build specific information.
A major contributing factor to build inconsistency is a lack of compiler and third-party library standardization. In a disparate and distributed build environment, developers frequently build against different versions of compilers and third-party libraries. This makes it difficult to re-create builds and diagnose problems. To promote standardization, all compilers should be centralized on a network drive accessible by all developers, on a clean and "locked down" machine. The build script templates should specifically reference the standard compiler versions on the mapped network drives, ensuring that all builds occur against consistent compiler versions. All third-party libraries should similarly be consolidated on a shared network drive so the latest, approved versions are used.
