1 Introduction

As we have discussed about the Automake in the Autotools, it is seems very tempting to start generate Makefiles for your projects and generate a Makefile for each module with a master Makefile at the top of project directory structure. However, there is a very good and informative paper by Peter Miller regarding “Recursive Makefiles are Considered Harmful”. As per this paper, projects with Recursive Makefiles build system are not very optimal for building projects, especially which are very large in size. But, writing a single Makefile for a very large project or even generating it using Automake is not very manageable, which is good argument. So, a blog by Karel Zak suggest that we can create a single resulting Makefile from bunch of include files which are included in main Makefile.am. Therefore, our first goal is to create a build system which is optimal.

Figure 1.1: Project structure

2 Enabling Automake in configure.ac

Just for the introduction, lets briefly understand what’s configure.ac is?

configure.ac is the input file which is used by Autoconf to generate configure which is used to setup the environment for project compilation. In other word, this script allows us to configure our project build system. Detailed discussion over this is done in Autoconf tutorial.

Much of the configure.ac script can be generated automatically based on the existing project tree or even updated based on the the current status. This automatic generation of configure.ac script is done using autoscan script. Note, that as the project structure grows, we don’t have to create new configure.ac every time. Instead, autoscan will read the existing configure.ac and improve upon it. The resulting output of the autoscan is configure.scan and autoscan.log. configure.scan can be change to configure.ac to use by autoconf for configuration of the project.

The output of autoscan looks like:

Listing 1: configure.ac generated from autoscan.


# Process this file with autoconf to produce a configure script.    -*- Autoconf -*- 

AC_PREREQ([2.71]) 
AC_INIT([FULL-PACKAGE-NAME], [VERSION], [BUG-REPORT-ADDRESS]) 
AC_CONFIG_SRCDIR([foo/myprog.c]) 
AC_CONFIG_HEADERS([config.h]) 

# Checks for programs.
# This below is added only for C++ files.
AC_PROG_CXX AC_PROG_CC 

# Checks for libraries. 

# Checks for header files.
# Checks for typedefs, structures, and compiler characteristics. 
AC_CHECK_HEADER_STDBOOL 


# Checks for library functions. 
AC_CONFIG_FILES([Makefile]) 
AC_OUTPUT

Note that the generated script is not enabled to use Automake tool the generate the Makefiles from Makefile.am input files. Oh wait! Didn’t we just say that Automake generates Makefiles automatically for the project. Well, lets address that in the next section.

Now, to enable the Automake in this build system, we need to add one single line onto the above script.

Listing 2: Automake enabled


AC_PREREQ([2.71])

AC_INIT([FULL-PACKAGE-NAME], [VERSION], [BUG-REPORT-ADDRESS])
AC_INIT_AUTOMAKE
AC_CONFIG_SRCDIR([creational-pattern/src/MazeGame.cpp])
AC_CONFIG_HEADERS([config.h])

Executing the autoreconf -i command will now make the build system of project, Automake enabled.

3 What is a Makefile.am file?

It is just like a regular Makefile, but with comparatively less Makefile code and some syntactically specific to Automake. But, why Automake needs some input file at the first place to generate a output Makefile. Well, the main intent of the Automake tool is to generate a Makefile which mostly comprises of boilerplate code. Since GCS enforce lot of standard rules, this leads to lot of code begin boilerplate which makes it very redundant to write it by hand and also making sure that it is logically correct. Here, Automake comes in handy as it generates a “complete” Makefile which is compliant to GCS. Another benefit of using Automake is, it has automatic dependency tracking capability which allows the tracking of header files changes which are included in the source files. It is annoying problem where change in header files are often not reflected in the compiled build due to missing dependency checks and Automake handles it very gracefully to ensure the dependency checks are efficiently managed.

4 Preparing the structure

Since our goal is to create a non-recursive build system of Makefiles using Automake, all the targets rules should appear in the top-makefile which may tell you that we may need to write a big Makefile.am for a project which is at the top of project directory. Fortunately, Automake can still create a single Makefile from bunch of input *.am files. This makes it super manageable. Writing bunch of “Makemodule.am” for each sub-directories of the project and including them in top Makefile.am will help us in completing our goal.

Listing 3: Makefile.am in root directory.


bin_PROGRAMS =
man_MANS =

include foo/Makemodule.am
include bar/Makemodule.am

Here, we can see that defining the variable at the top of top Makefile.am and then using it in the Makemodule.am in the sub-directories.
For ex, in foo/Makemodule.am:

Listing 4: Makemodule.am in foo directory of project.


bin_PROGRAMS += myprog					# PLV
man_MANS += foo/myprog.8

myprog_SOURCES = foo/myprog.c \			# PSV
                 foo/myprog-utils.c

Here in Makefile.am and Makemodule.am, we are using two important types of variables. These are Product List Variable and Product Source Variable.

4.1 Product List Variable:

Products of the build systems are specified to the Makefile.am using Product List variable (PLV). These are class of variables which have special meaning to Automake. They tell what are the resultant output of the Makefiles or build system that will be generated after the successful completion of the build process. This variable consist of white-space-separated list of products and the variable (as you can see in above example) consist of two major parts separated by a underscore.
These two components are:

bin: Prefix
PROGRAMS: Primary

The bin portion of the PLV represent a installation location prefix. This is used to define the installation location for the list of products which are mentioned in the PLV. The GCS defines many installation locations. However, make variable ending with dir , whose value is a file-system location, is a viable installation location and can be used as prefix in a Automake PLV.
What we mean here is that any make variable ending with dir, for example $(bindir), with a installation location value can be used by Automake to produce install rule which will install the list of products to the $(bindir) location. However, only those products will be installed that are mentioned in the PLV which have prefix bin as mentioned in the above example (bin_PROGRAMS) and PROGRAMS is termed as Primary which represent the type of product that is expected to be produced, which in this case is the list of binary files.
Automake by default recognizes 4 type of variable which they can be treated as installation locations and they have a prefix pkg. These variables are:

pkgincludedir
pkgdatadir
pkglibexecdir
pkglibdir

These pkg version of standard libdir, includedir, datadir and libexecdir indicates that the installation of the products listed in the PLV equivalent variable should be installed in the sub-directory of these location mentioned in these standard make variable where the sub-directory is named after the package. For example, our Project which is named as Project in our Figure 1, will have binary files (like myprog) installed in the Project sub-directory of the bin system directory. And yes, we can change the installation directory from bin to any other directory by overriding the value of the $(bindir) variable in Makefile.
We can also provide our own PLV custom prefix which represent a variable, having the location for installation of some list of products. For example, for installation of some XML files to a custom location, we can define a xmldir variable having value for the installation path and a PLV xml_DATA having value a list of xml files which will be installed in the installation path.

Listing 5: Custom PLV with variable in Makefile.am

xmldir = $(datadir)/xml
xml_DATA = file1.xml file2.xml file3.xml ...

Installation location can default values which are either defined by Automake or by user itself. However, these default value can be overridden using the argument passed while running the configure script or make command. If we don’t want to install certain products to the installation location, Automake generated rules ensures to generate rules to handle such situations. For example, if we don’t want to install binary and library files and only want to install the data files, we can pass the command make bindir=’’ libdir=’’ install.

4.2 Prefix Not Associated with Installation

Sometimes, we don’t want certain products to installed anywhere. That’s why we have certain prefix which are not associated with any installation location. There are 3 main such prefix:

noinst: This signifies that the product listed in a variable with this prefix is not to be installed anywhere in the system after the build process. It could be a convenience static library which is only used to build a final product.
check: The products listed will be build only when make check command is used. These products are only build for testing purpose.
EXTRA: This signifies the list of products which may be conditionally build and eventually, installed. This types of configuration control is possible via configure script where we pass some argument to it and prompts to build some “extra” products. However, enabling such feature of autoconf and automake is not as simple as above two. Go through the below illustration:

Listing 6: A shell script code in configure.ac for building optprog product conditionally.

AC_INIT(...)
...
optional_programs=
AC_SUBST([optional_programs])
...
if test "x$(build_opt_prog)" = xyes; then
  optional_programs=$(optional_programs) optprog		### 1 ###
fi
...

Listing 7: Using the EXTRA prefix to conditionally build products in Makefile.am

EXTRA_PROGRAMS = optprog						### 2 ###
bin_PROGRAMS = myprog $(optional_programs)	  ### 3 ###

In Listing 6 ### 1 ###, optprog is appended to an Autoconf substitution variable called optional_program. The EXTRA_PROGRAMS variable at ### 2 ### in Listing 7 list optprog, as a product that may or may not be build. This redundant information of EXTRA_PROGRAMS in Makefile.am make it ensure that Automake knows that optprog is a optional product and make the resultant Makefile with appropriate script. This make sure that even if optprog is not mentioned in $(optional_programs) variable in some build.

4.3 Primaries

Primaries represent type of product which is generated by this build system. That’s it! What are these type of files?
These files can be executable, libraries, config files in terms of xml, json and event normal txt files. Wait, normal txt files? Well, what it means is that some products (not exactly) of build system are not actually generated by the build system. These are part of the project and we will need these files while executing the software in our system after the build process of that software. For example, C or C++ language have the prototype of the function or API written in header files (.h or .hpp) which allows the user of the library to make there software aware of these API before using it and enables in linking the implementation to the function calls while in the linking process of the build system. So, the main intent of this long boring example is to provide a context on why we have different kind of primaries as this helps in differentiating the type of product to Automake and hence generate rules in resultant Makefile accordingly. Interestingly, we can define our own primaries,but beware, we need to write the rules for those on our own in Makefile.am.

Let us see the default primaries supported by Automake.

PROGRAMS: These are binary executable programs.
LIBRARIES/LTLIBRARIES: The LIBRARIES primary enables Automake to generate rules for building static libraries using system compiler and librarian (ar). The LTLIBRARIES also do the same thing but to build libtool shared libraries which are generated by executing libtool scripts. The installation of these products are limited to $(libdir) or $(pkglibdir).
SCRIPTS: Scripts are generally handwritten which may be a part of project, however, Automake don’t have the assumption that it will not make any rules to build the scripts or they are static files in the project. The scripts can be generated from handwritten rules in Makefile.am files, sometimes by processing a input file using the sed or awk utility tools. Even if the static script are present in the project and is intended to be distributed with the project tarball, then dist modifier should be used with SCRIPT primary. The installation location for scripts includes $(bindir), $(sbindir), $(libexecdir), and $(pkgdatadir).
DATA: Data files are similar to what scripts files are intended for and can be distributed with project tarball and for that we can use dist modifier with DATA primary. The installation location for these type of files are $(datadir), $(sysconfdir), $(sharedstatedir) and $(localstatedir).
HEADERS: Headers files are just like static files but are used for building of the project. These files generally contained the public interface of the installed libraries which can be used by other projects for using the libraries functionality. The PLV for these type of files are pkginclude_HEADERS and include_HEADERS and the installation locations are $(includedir) and $(pkgincludedir).

There are also other types of primaries that Automake supports, but we will currently discuss about the ones which are used the most. The rest of the commonly used primaries are MANS, TEXTINFOS, PYTHON, JAVA, etc.

4.4 Product Source Variable

If you notice, the second line in Listing 7 is an example of Automake product source variable (PSV).

[modifier-list]product_SOURCES = file1 file2 ... fileN

Just like PLV which represent list of products, PSV represent list of files on which the product is dependent on. These are the source files which are used to build the products of the projects. Only characters that are allowed in make variables (letters, numbers, and the at sign) are allowed in the product tag of a PSV. So, Automake converst illegal characters into underscores.

Listing 8: Illegal make variable characters are converted to underscores in product tags.

lib_LIBRARIES = libc++.a
libc___a_SOURCES = ...

4.5 PLV and PSV Modifiers

Modifiers are optional prefix added to PLV or PSV to modify the behavior of these Automake variable which would have behaved differently if not added. It is better to understand this with examples:

Some files may be intended to be distributed and some files may not, while the distribution tarball is begin built. This can be achieved for the case of source file by adding the prefix to PSV, like the one shown below:

Listing 9: Using the dist and nodist modifiers in a Makefile.am file.
dist_myprog_SOURCES = file1.c file2.c nodist_myprog_SOURCES = file3.c file4.c
Automake normally strips relative path information from the list of header files in a HEADERS PLV. The nobase modifier restrict this behavior and let the directory structure of the project tree to be followed in the same area where the list of header files are installed (generally in system header file location).

Listing 10: Using the nobase PLV modifier in a Makefile.am file.
nobase_pkginclude_HEADERS = mylib.h sys/constants.h

This ensures that the constanst.h file is stored in $(pkginclude) directory inside sys subdirectory.
The notrans modifier is generally used for not allowing any transformation to the name of the source file. This generally occurs for man pages whose source file name is transformed from .man to .N (where N is 0,1,2,3...9).
There are certain source files which might be used for compilation of the executable product based on the condition defined in configure.ac, so, for such source file EXTRA modifier is added to the PSV to handle such cases.

Listing 11: Using the EXTRA prefix with a product SOURCES variable.
EXTRA_myprog_SOURCES = possibly.c

5 Unit Testing: Automake support for make check

Check the example below first:

Listing 12: The check target

...
check: all
		./myprog | grep “Hello from .*myprog!”
		@echo “*** ALL TESTS PASSED ***”
...

As we can see to test the sanity of the executable produced by the build system, in this case is a Makefile is used for testing. Here, it is executing the executable and using grep to check the output is as expected as shown above. Automake can also automate this by using variables specific to this functionality.

Listing 13: src/Makemodule.am: Additional code required to support the check target

bin_PROGRAMS += myprog
myprog_SOURCES += myprog.c

check_SCRIPTS = greptest.sh   @1
TESTS = $(check_SCRIPTS)	  @2

greptest.sh:				  @3
		echo ’./myprog | grep “Hello from .*jupiter!”’ > greptest.sh
		chmod +x greptest.sh

CLEANFILES = greptest.sh	  @4

The check_SCRIPTS (@1) is a PLV which refers to a script that is generated at build time. This script is only generated when make check command is executed. Also, we need to keep in mind that the generated script should be cleaned (or deleted) as they are not files which should be distributed. So, CLEANFILES(@4) variable takes care about the cleaning part.
The TESTS (@2) line is the important one in Listing 13. It tells which targets should be executed when make check is executed and since the check_SCRIPTS have complete list of these target, hence it is referenced. However, since TESTS variable ensures that appropriate rules are generated so that the test scripts are build and executed successfully. Therefore, check_SCRIPTS is redundant and can be skipped by replacing it with the contents of it. check_* PLVs becomes important only when helper scripts needs to build which are not executed directly but are executed by script which are listed in TESTS variable. Note, TESTS is actually for executing the script and Automake always build the script before executing if it is already not build.

6 Convenience Libraries: Static libraries.

Listing 14: common/Makemodule.am: To build the temporary library.

noinst_LIBRARIES += libcommon.a
libcomman_a_SOURCES = common.h print.c

The first line defines what library should be build and as you have noticed, one noinst modifier is used so that the build library is not installed into the system directory after the project is build and installed. The goal of such library is to compile common code in one static library and used in many different section of the projects. Here, libcommon.a is like a archive file which is given to compiler for compilation of executable. The only difference between .tar and .a files is that the compiler know which object files to use to compile the executable which is present inside the static library.

6.1 Product Option Variables

In order for the source files which will be used to build the executable to find the relevant code from the static library and also find the correct corresponding header files, we need to add some additional information to the Makefile.am (more specifically Makemodule.am)of the src directory.

Listing 15: src/Makemodule.am: Adding compiler and linker directives to Makefile.am files.

bin_PROGRAMS += myprog
myprog_SOURCES = main.c
myprog_CPPFLAGS = -I$(top_srcdir)/common
myprog_LDADD = ../common/libcommon.a

The two new variable added here (myprog_CPPFLAGS and myprog_LDADD) are product option variables (POVs) are used to specify product-specific options to tools that are used to build products from source code. The myprog_CPPFLAGS variable adds product-specific C-preprocessor flags to the compiler command line for all source files that are compiled for the myprog program. This add reference to the list of location that the compiler will look for header file when the source files are used for compilation. The myprog_LDADD variable is used to add libraries to the linker command line. The path ../common/libcommon.a is added to the linker command line while building the myprog product so that the relevant object files can be used to link and build the product. The variables program_LDADD and program_LIBADD is only necessary for libraries that are built as part of our own package. If our project requires libraries which are already a part of user’s system, then all we need is a macro call of AC_CHECK_LIB or AC_SEARCH_LIB in configure.ac file. This will generate appropriate script in configure that will add the references to these libraries in linker command via LIBS variable.
The set of POVs supported by Automake are listed below:

product_CPPFLAGS:
Use product_CPPFLAGS to pass flags to the C preprocessor on compiler command line.
product_CFLAGS:
Same as above but for C-compiler flags.
product_LDFLAGS:
Use product_LDFLAGS to pass global and order-independent shared library and program linker configuration flags and options to the linker, including -static, -version-info, -release, and so on.
program_LDADD:
Use program_LDADD to add libtool objects (.lo) or libraries (.la) or non-Libtool objects (.o) or archives (.a) to the linker command line when linking a program.
library_LIBADD:
Use library_LIBADD to add non-Libtool linker objects and archives to non-Libtool archives on the ar utility command line. The ar utility will incorporate archives mentioned on the command line into the product archive, so you can use this variable to gather multiple archives together into one.
ltlibrary_LIBADD:
Use ltlibrary_LIBADD to add Libtool linker objects (.lo) and Libtool static or shared libraries (.la) to a Libtool static or shared library.

We can use the last three option variables in the list to pass lists of order-dependent static and shared library references to the linker. These variable can also be used to pass -L and -l options.

6.1.1 Per-Makefile Option Variables

Some of the preprocessor flags are common for all the products that is to be build by the Makefile. So, instead of repeatedly mentioning it on product variables in .am files, these can be mentioned in the per-makefile option variable as shown below:

Listing 16: per-product and per-makefile flags.

AM_CFLAG = ... some flags ...
...
prog1_CFLAGS = ... more flags ... $(AM_CFLAGS)   @1

The first line above is a per-makefile flag while the next flag at (@1) is a per-product flag. Note, that after adding the per-makefile flag it is appended as variable to the per-product flag because the per-product flags overrides the value of per-makefile flag. In order to include both, the former is added to the latter in order to include all the mentioned flags for the products.

Why then per-makefile are needed in the first place? Can’t we directly add to the per-products flag?
Per-makefile flags generates relatively compact Makefile as compare what per-product flag generates. This is because per-product flags makes Automake to generate per-product rules instead of most of the redundant part which can be used as suffix.
Note that these variables specific to Automake don’t modify the actual CFLAGS of Makefiles. These variables are exclusively reserved for the end user to tinker with. All the flags are defined above are appended to these flags or to the Command when executed for building the product.

6.2 Building the New Library

Now the newly added static library can be build by adding the SUBDIR to the top-level makefile. This is for the case when we opt for recursive Build system. However, our intention is to create a Non-Recursive Build system for reason we have already discussed. For that we declare the PLVs on the top-level Makefile.am and we include the products list in the Makemodules.am which are in the sub-directories, which we have been doing till now.
In the case of Recursive-Makfiles, we would have to add the Makefiles of the sub-directories into the macro AC_CONFIG_FILES in configure.ac:

Listing 17: For the case of Recursive-Makefile build system.

...
AC_CONFIG_FILES([Makefile
				 common/Makefile
				 src/Makefile])
...

However, for Non-recursive case we don’t need such inclusion as there is only one Makefile which would be generated by the Automake, i.e. the top-level makefile.
On running the autoreconf -i, the updated build system will be checked and enhancements will be suggested by the Autotools scripts.

autoreconf -i
configure.ac:6: installing ’./install-sh’
configure.ac:6: installing ’./missing’
Makefile.am:2: error: library used but ’RANLIB’ is undefined
Makefile.am:2:   The usual way to define ’RANLIB’ is to add ’AC_PROG_RANLIB’
Makefile.am:2:   to ’configure.ac’ and run ’autoconf’ again.
Makefile.am: installing ’./depcomp’
autoreconf: error: automake failed with exit status: 1

As we can see, it install some scripts and mention one error where RANLIB is not added in configure.ac file to for building the project. This error can be removed by adding AC_PROG_RANLIB to configure.ac. The inclusion of RANLIB is only for making the build system portable. It is not usually not needed in modern systems.

7 What Goes into a Distribution?

Automake is quite smart enough to know what are the roles of each file in the build process and hence determine which files and directory needs to be included in the distribution created by executing make dist. This is only possible because we have mentioned every file to Automake using PLV and PSV. Be warned that don’t try to use some bash shell shortcuts to write one line to include all the source files, etc. Thats not how Automake will behave properly. Trying to hack a will be more pain than listing all the files.
Also, Automake EXTRA_DIST variable can be used to mention files or directories that also need to including while creating the distribution package.

Listing 18: directory Config to be included in distribution package.

EXTRA_DIST = Config

Note that the directories might have .git, .svn or .CVS status directories and we don’t want it to be installed with the distribution created. Fortunately, Automake handles this automatically.

8 Maintainer Mode

This is where we make our project ready to use for our user. One situation might be created that the timestamp of the source in the distribution package might be newer relative to the system clock. Because of this, when the end-user execute make command, it might see that the configuration scripts and all the tools related to Autotools are modified and the entire build system needed to be build including these configuration script, Makefile.in from Makefile.am. The problem is that the user might not have these Autotools. But, it doesn’t meant that they cannot build the project. To handle such situation, Automake has a mode called Maintainer Mode where Makefile rules are generated conditionally on whether the Autotools related generated scripts need to be rebuild. This is done via AM_MAINTAINER_MODE macro in configure.ac which disable maintainer-level make rules that we discussed just now.
For maintainer, to rebuild the entire build-system including the Autotools related script, it should use command-line option (--enable-maintainer-mode) with configure scripts that tells to generate Makefile.in templates that contains rules and commands to execute the Autotools as necessary. This is to enable the end-user to not worry about Autotools behavior and can build the project without it.

9 Suppressing Noise

Some user might want to print only relevant things while building the project and don’t want entire command-line output for building the project. This can be achieved by in two ways and either can be used:

Add the silent-rules option to the argument of AM_INIT_AUTOMAKE in configure.ac.
Call the AM_SILENT_RULES macro in configure.ac.

The user can set the verbosity by using argument --enable-silent-rules or --disable-silent-rules on the configure command line. What it does it actually it set the V (verbosity) of make output by setting it to either 1 or 0, respectively.

Ending

And thats it, we have covered hopefully enough to at least create a moderately good enough build-system using Autoconf and Automake. Mostly, we have discussed about Automake and very slightly about Autoconf. But, next write-up will be about Autoconf and how we can make the build-system configurable for end-user based on the requirement of the user and the capability of the user’s system. Farewell!

Automake

Table of Contents