Makefile Debugging: Tracing Macro Values

 In the first of these articles I showed a technique for printing the value of any Makefile macro by defining a special rule called print-%. Now I'm going to show how to trace where a macro is used in a Makefile.

Consider this simple Makefile:

X=$(YS) hate $(ZS)
Y=dog
YS=$(Y)$(S)
Z=cat
ZS=$(Z)$(S)
S=s
all: $(YS) $(ZS)
@echo $(X)
$(YS):
@echo $(Y) $(Y)
$(ZS):
@echo $(Z) $(Z)

When run it prints

dog dog
cat cat
dogs hate cats

Tracing Macro Use

Now try to trace through and see wherethe macro $(Y) is used. It's actually used on lines 8, 9, 11, and 12 (twice). It's amazing how often macros get used! That's because Make defaults to only getting the value of a macro when needed and macros are frequently deeply nested.

Tracing such use for any real Makefile would be an impossible task, but it's possible to get Make to do the work for you. Take a look at the code which should be added to the start of the Makefile to be traced (it'll only get used when explicitly called).

ifdef TRACE
.PHONY: _trace _value
_trace: ; @$(MAKE) --no-print-directory TRACE= $(TRACE)='$$(warning TRACE $(TRACE))$(shell $(MAKE) TRACE=$(TRACE) _value)'
_value: ; @echo '$(value $(TRACE))'
endif

Before diving into understanding how it works, here's an example of using it to trace the value of $(Y) in our example Makefile. To use the tracer you tell GNU Make to run the trace target by setting the TRACE macro to the name of the macro you wanted tracked. In this example we want to watch use of the macro Y:

% gmake TRACE=Y

Makefile:8: TRACE Y
Makefile:11: TRACE Y
Makefile:12: TRACE Y
Makefile:12: TRACE Y
dog dog
cat cat
Makefile:9: TRACE Y
dogs hate cats

From the lines containing the word TRACE you can see Y being used first on line 8 (the definition of the all target references Y via the $(YS)), then on line 11(the definition of the cats target is using $(YS) which uses Y), then twice on line 12 (the two references to $(Y) itself as we execute the rule) and finally on line 9 ($(X) references $(YS) which references $(Y)).

With the power of the tracer we can try another task: finding out where $(S) is used:

% gmake TRACE=S

Makefile:8: TRACE S
Makefile:8: TRACE S
Makefile:11: TRACE S
Makefile:14: TRACE S
dog dog
cat cat
Makefile:9: TRACE S
Makefile:9: TRACE S
dogs hate cats

How the Macro Tracer Works
GNU Make has a special function called $(warning) that outputs a warning message to STDERR and returns the empty string. Conceptually the tracer changes the value of the macro to be traced to include a $(warning) message. Every time the macro is expanded the warning is printed; when a warning message is output GNU Make prints the name of the Makefile in use and the line number.

For example, if we changed the definition of Y from

Y=dog

 

to

 

Y=$(warning TRACE Y)dog

then whenever Y were expanded we would see a warning and Y would have the resulting value dog.

The trace code automates this process: it first obtains the unexpanded value of the macro to be traced, prepends it with an appropriate warning and then runs the desired Make with the specially modified value of the macro being examined. To do this it uses a Make function introduced in GNU Make 3.80 that enables you to get the unexpanded value of a macro: $(value).

In our example Makefile the macro YS is defined to be $(Y)$(S). If we ask for its value in a Makefile using the form $(YS) we get dogs, if we do $(value YS) then we'll get $(Y)$(S). $(value) returns the literal string used to define YS.

Now let's break down the tracer to understand what it does. If TRACE is defined then the block of definitions associated with the tracer is used and the first rule that the Makefile contains is the rule for _trace. Since _trace is the first target encountered, when TRACE is defined its rule will be run by default.

The _trace rule contains a single, complex command:

@$(MAKE) --no-print-directory TRACE= $(TRACE)='$$(warning TRACE $(TRACE))$(shell $(MAKE) TRACE=$(TRACE) _value)'

On the right-hand side of the command is a $(shell) invocation rerunning the Makefile with a different goal. If we are tracing YS then this $(shell) runs the command

make TRACE=YS _value

which will cause the _value rule to get run, which in turn will echo the definition of YS. _value simply echoes the definition of the macro specified by the TRACE macro. So the $(shell) ends up evaluating to $(Y)$(S).

The $(shell) is in fact inside a command-line macro definition (usually called a command-line override):

$(TRACE)='$$(warning TRACE $(TRACE))$(shell $(MAKE)TRACE=$(TRACE) _value)'

which adds the $(warning) needed to output the 'TRACE X' messages. Notice how the name of the macro being defined is a computed value: its name is contained in $(TRACE). When tracing YS this definition turns into:

YS='$(warning TRACE YS)$(Y)$(S)'

The single quotes are used to prevent the shell from seeing the $ sign, the double $ is used to prevent Make from seeing the $: in either case a macro expansion would be prevented which we want to delay until YS is actually used.

Finally the _trace rule runs

make TRACE= YS='$(warning TRACE YS)$(Y)$(S)'

which resets the value of TRACE (since this invocation of Make should actually run the real rules) and overrides the value of YS as defined in the Makefile. Recall that macros defined on the command-line override definitions in the Makefile (see article URL): even though YS is defined in the Makefile the command-line definition will be used.

Now every time $(YS) is expanded a warning is printed.

Notes
This technique doesn't work for a macro that is target-specific. GNU Make allows you to define a macro as specific to a target in the following manner:

all: FOO=foo
all: a
@echo $(FOO)

a:
@echo $(FOO)

The macro FOO will have value foo in the rule that builds all and in any prerequisites of all. The Makefile above will print foo and then foo because $(FOO) is defined in the all rule and in the a rule. The tracer is unable to obtain the value of $(FOO) and would in fact cause this Makefile to behave incorrectly.

$(warning) sends its output to STDERR which makes it possible to separate normal Make output from the tracer. Simply redirect STDERR to a trace log file. For example the following command will result in normal Make output being seen and the trace going to the log file:

% gmake TRACE=S 2> trace.log

dog dog
cat cat
dogs hate cats