Android Build System

http://elinux.org/Android_Build_System

 

Android Build System

Basics of the Android Build system were described at (Down as of 9/2011): http://source.android.com/porting/build_system.html

Note that "partner-setup" should be replaced with "choosecombo" or even "lunch" in that description.

(This information seems to be duplicated at: http://pdk.android.com/online-pdk/guide/build_system.html I'm going to assume that the source.android site is the most up-to-date, but I haven't checked.)

More information about the Android build system, and some of the rationale for it, are described at: http://android.git.kernel.org/?p=platform/build.git;a=blob_plain;f=core/build-system.html

You use build/envsetup.sh to set up a "convenience environment" for working on the Android source code. This file should be source'ed into your current shell environment. After doing so you can type 'help' for a list of defined functions which are helpful for interacting with the source.

Overview

The build system uses some pre-set environment variables and a series of 'make' files in order to build an Android system and prepare it for deployment to a platform.

Android make files end in the extension '.mk' by convention, with the main make file in any particular source directory being named 'Android.mk'.

There is only one official file named 'Makefile', at the top of the source tree for the whole repository. You set some environment variables, then type 'make' to build stuff. You can add some options to the make command line (other targets) to turn on verbose output, or perform different actions.

The build output is placed in 'out/host' and 'out/target' Stuff under 'out/host' are things compiled for your host platform (your desktop machine). Stuff under 'out/target/product/<platform-name>' eventually makes it's way to a target device (or emulator).

The directory 'out/target/product/<platform-name>/obj' is used for staging "object" files, which are intermediate binary images used for building the final programs. Stuff that actually lands in the file system of the target is stored in the directories root, system, and data, under 'out/target/product/<platform-name>'. Usually, these are bundled up into image files called system.img, ramdisk.img, and userdata.img.

This matches the separate file system partitions used on most Android devices.

Some Details

What tools are used

During the build you will be using 'make' to control the build steps themselves. A host toolchain (compiler, linker and other tools) and libraries will be used to build programs and tools that will run on the host. A different toolchain is used to compile the C and C++ code that will wind up on the target (which is an embedded board, device or the emulator). This is usually a "cross" toolchain that runs on an X86 platform, but produces code for some other platform (most commonly ARM). The kernel is compiled as a standalone binary (it does not use a program loader or link to any outside libraries). Other items, like native programs (e.g. init or toolbox), daemons or libraries will link against bionic or other system libraries.

You will be using a Java compiler and a bunch of java-related tools to build most of the application framework, system services and Android applications themselves. Finally, tools are used to package the applications and resource files, and to create the filesystem images that can be installed on a device or used with the simulator.

Telling the system where the Java toolchain is

Before you build anything, you have to tell the Android build system where your Java SDK is. (Installing a Java SDK is a pre-requisite for building).

Do this by setting a JAVA_HOME environment variable.

Specifying what to build

In order to decide what to build, and how to build it, the build system requires that some variables be set. Different products, with different packages and options can be built from the same source tree. The variables to control this can be set via a file with declarations of 'make' variables, or can be specified in the environment.

A device vendor can create definition files that describe what is to be included on a particular board or for a particular product. The definition file is called: buildspec.mk, and it is located in the top-level source directory. You can edit this manually to hardcode your selections.

If you have a buildspec.mk file, it sets all the make variables needed for a build, and you don't have to mess with options.

Another method of specifying options is to set environment variables. The build system has a rather ornate method of managing these options for you.

To set up your build environment, you need to load the variables and functions in build/envsetup.sh. Do this by 'source-ing' the file into your shell environment, like this:

$ source build/envsetup.sh

or

$ . build/envsetup.sh

You can type 'help' at this point to see some utility functions that are available to make it easier to work with the source.

To select the set of things you want to build, and what items to build for, you use either the 'choosecombo' function or the 'lunch' function. 'choosecombo' will walk you through the different items you have to select, one-by-one, while 'lunch' allows you select some pre-set combinations.

The items that have to be defined for a build are:

• the product ('generic' or some specific board or platform name)
• the build variant ('user', 'userdebug', or 'eng')
• whether you're running on a simulator ('true' or 'false')
• the build type ('release' or 'debug')

Descriptions of these different build variants are at http://source.android.com/porting/build_system.html#androidBuildVariants

The build process from a user perspective is described pretty well in this blog post: First Android platform build by CodePainters, December 2009

Actually building the system

Once you have things set up, you actually build the system with the 'make' command.

To build the whole thing, run 'make' in the top directory. The build will take a long time, if you are building everything (for example, the first time you do it).

Build tricks

Seeing the actual commands used to build the software

Use the "showcommands" target on your 'make' line:

$ make -j4 showcommands

This can be used in conjunction with another make target, to see the commands for that build. That is, 'showcommands' is not a target itself, but just a modifier for the specified build.

In the example above, the -j4 is unrelated to the showcommands option, and is used to execute 4 make sessions that run in parallel.

Make targets

Here is a list of different make targets you can use to build different parts of the system:

• make sdk - build the tools that are part of an SDK (adb, fastboot, etc.)
• make snod - build the system image from the current software binaries
• make services
• make runtime
• make droid - make droid is the normal build.
• make all - make everything, whether it is included in the product definition or not
• make clean - remove all built files (prepare for a new build). Same as rm -rf out/<configuration>/
• make modules - shows a list of submodules that can be built (List of all LOCAL_MODULE definitions)
• make <local_module> - make a specific module (note that this is not the same as directory name. It is the LOCAL_MODULE definition in the Android.mk file)
• make clean-<local_module> - clean a specific module

Speeding up the build

You can use the '-j' option with make, to start multiple threads of make execution concurrently.

In my experience, you should specify about 2 more threads than you have processors on your machine. If you have 2 processors, use 'make -j4', If they are hyperthreaded (meaning you have 4 virtual processors), try 'make -j6.

You can also specify to use the 'ccache' compiler cache, which will speed up things once you have built things a first time. To do this, specify 'export USE_CCACHE=1' at your shell command line. (Note that ccache is included in the prebuilt section of the repository, and does not have to be installed on your host separately.)

Building only an individual program or module

If you use build/envsetup.sh, you can use some of the defined functions to build only a part of the tree. Use the 'mm' or 'mmm' commands to do this.

The 'mm' command makes stuff in the current directory (and sub-directories, I believe). With the 'mmm' command, you specify a directory or list of directories, and it builds those.

To install your changes, do 'make snod' from the top of tree. 'make snod' builds a new system image from current binaries.

Setting module-specific build parameters

Some code in Android system can be customized in the way they are built (separate from the build variant and release vs. debug options). You can set variables that control individual build options, either by setting them in the environment or by passing them directly to 'make' (or the 'm...' functions which call 'make'.)

For example, the 'init' program can be built with support for bootchart logging by setting the INIT_BOOTCHART variable. (See Using Bootchart on Android for why you might want to do this.)

You can accomplish either with:

$ touch system/init/init.c
$ export INIT_BOOTCHART=1
$ make

or

$ touch system/init/init.c
$ m INIT_BOOTCHART=1

Makefile tricks

These are some tips for things you can use in your own Android.mk files.

build helper functions

A whole bunch of build helper functions are defined in the file build/core/definitions.mk

Try grep define build/core/definitions.mk for an exhaustive list.

Here are some possibly interesting functions:

• print-vars - shall all Makefile variables, for debugging
• emit-line - output a line during building, to a file
• dump-words-to-file - output a list of words to a file
• copy-one-file - copy a file from one place to another (dest on target?)

 

Add a file directly to the output area

You can copy a file directly to the output area, without building anything, using the add-prebuilt-files function.

The following line, extracted from prebuilt/android-arm/gdbserver/Android.mk copies a list of files to the EXECUTABLES directory in the output area:

$(call add-prebuilt-files, EXECUTABLES, $(prebuilt_files))

Adding a new program to build

Steps for adding a new program to the Android source tree

• make a directory under 'external'
  • e.g. ANDROID/external/myprogram
• create your C/cpp files.
• create Android.mk as clone of external/ping/Android.mk
• Change the names ping.c and ping to match your C/cpp files and program name
• add the directory name in ANDROID/build/core/main.mk after external/zlib as external/myprogram
• make from the root of the source tree
• your files will show up in the build output area, and in system images.
  • You can copy your file from the build output area, under out/target/product/..., if you want to copy it individually to the target (not do a whole install)

See http://www.aton.com/android-native-development-using-the-android-open-source-project/ for a lot more detail.