Welcome to the second part of our Makefile series, where we migrate your build system to a more modular structure by including your very own libraries. In the previous article, we covered the fundamentals of creating a powerful Makefile, including directory structure, variable definition, target dependencies, and more. Now, we'll dive deeper into the realm of user-defined libraries, allowing you to unlock the full potential of your Makefile automation. By leveraging static libraries, you can enhance the modularity and reusability of your code, making it easier to manage dependencies and accelerate your development workflow. In this article, we'll walk you through the process of creating and utilizing user-defined static libraries within your Makefile, providing you with the tools to supercharge your C++ projects.
Who is this article for?
- If you have a GNU Linux development environment.
- If you are using GNU compilers such as
g++. - If you don't depend on any IDE for project configurations.
It is recommended to enable showing whitespace characters when dealing with Makefiles in your editor as it can help distinguish variable and target declarations. (Space vs Tab)
Feel free to use the suggestions below as a source of inspiration, but remember that they may not address all of your specific needs. Get creative and adapt them to suit your project requirements.
Please note: This article focuses solely on the Makefile template and does not compare or discuss other build systems.
Before we delve into the realm of user-defined static libraries, let's quickly recall the enhancements we made to the application Makefiles in the first part of this article series. These improvements are equally applicable to the library Makefiles we'll create in this part, so referring back to the previous article will provide you with the rationale behind our enhancements.
To save time, we will skip discussing topics such as folder hierarchy, header modification detection, and architecture dependency considerations. If you need a refresher on these aspects, I recommend revisiting the previous article where we extensively covered these enhancements. By applying the same principles to our library Makefiles, we ensure consistency and benefit from a streamlined and efficient build system.
Now, armed with the knowledge from the first part, let's explore how we can harness the power of user-defined static libraries to take our build system to the next level.
To ensure a common Makefile template for all your libraries, it is essential to maintain consistency in the folder hierarchy across different libraries. While you have the flexibility to create your own structure, for the purpose of this article, we will adopt a specific folder structure for our libraries.
Consider the following structure, which we'll use as a foundation for all of our libraries:
LoggerLib
├── Makefile
├── Outputs
└── SourceCode
├── Headers
│ └── LoggerLib
│ ├── Logger.hpp
│ └── Subfolder
│ └── Helper.hpp
└── Sources
├── Logger.cpp
└── Subfolder
└── Helper.cpp
In this structure, each library resides in its own directory. The Makefile serves as the entry point for building each library, the Outputs directory holds the generated object files and the archive file, while the SourceCode directory is further divided into Headers and Sources directories. The Headers directory contains the header files specific to the library, organized in a subdirectory corresponding to the library's name or namespace to prevent header collisions with other libraries. Similarly, the Sources directory contains the source code files, arranged in subdirectories if necessary. In the upcoming sections, we'll explore how this folder structure integrates with the Makefile and how it enables smooth library development and usage.
Don't worry, you will better understand this policy in the upcoming sections.
When working with user-defined static libraries, it's crucial to compile them with the same flags that were used to compile the encapsulating application. This practice ensures consistency and avoids potential issues arising from incompatible compilation settings.
Compilation flags encompass a wide range of options, including optimization levels, language standards, predefined macros, and more. These flags influence how the code is compiled and optimized, and they can have a significant impact on the behavior and performance of the resulting binary. By using consistent compilation flags, you ensure that the library and the application are built in a compatible manner, allowing them to seamlessly integrate with each other. Here are a few reasons why maintaining consistency in compilation flags is essential:
1. Compatibility: The library may contain inline functions, templates, or other constructs that rely on specific compiler flags. If the library is compiled with different flags than the application, it may lead to inconsistencies and potential compilation errors or runtime issues.
2. Binary Compatibility: When the library is built with different optimization levels or language standards, it can result in different object code and symbol names. This mismatch can cause linking errors or runtime crashes when the library is used by the application.
3. Performance and Behavior: Compilation flags can significantly impact the performance and behavior of the code. For instance, different optimization levels may produce different levels of performance or trade-off between speed and size. By compiling the library with the same flags as the application, you ensure that both components are optimized consistently.
To achieve consistency in compilation flags, it's recommended to define a set of variables or macros in the Makefile that store the desired flags. These variables can then be referenced when compiling both the library and the application, ensuring that they are built with the same flags. The next section will elaborate how we will share flags across different Makefiles.
Sharing variables between different Makefiles, i.e. the application and libraries, can be achieved using the export directive in a Makefile. This directive allows variables to be marked for exporting, making them accessible to child processes and sub-make invocations. By using export, we can share certain variables defined in the application's Makefile with the libraries.
Here are some of the variables that we can export from the application's Makefile to share with the libraries:
1. Architecture Type (ARCH_TYPE): The library needs to know the target architecture for which it is being compiled.
2. Compilation Flags (CPP_FLAGS): To ensure binary compatibility, it is crucial that both the application and the libraries are compiled with the same compilation flags.
3. Sanitizers (SANITIZERS): Sanitizers are useful tools for detecting various types of bugs and issues in the code during runtime. It is important to compile and link the libraries with the same sanitizers used by the application for safer development.
4. Symbols (DEFINES): Some user-defined libraries may rely on flags or symbols defined in the application's configuration. Sharing these symbols ensures consistency and avoids conflicts. However, caution should be exercised to prevent collisions between user-defined macros and those defined within the libraries.
5. Target RootFS (SYSTEM_ROOT_PATH): Libraries might require headers from the target file system. Exporting the target root file system path allows the libraries to locate and include the necessary headers.
By exporting these variables, we enable the libraries to access the shared values, ensuring compatibility and consistency between the application and the libraries. In the upcoming sections, we'll explore how to effectively utilize these shared variables in the library Makefile and demonstrate the benefits of their integration.
To ensure consistency among binaries and share compilation parameters with user-defined libraries, we need to make a few modifications to the application's Makefile provided in the first part of this article series. The following subsections will describe these small but impactful changes.
To share variables between Makefiles, we can use the export directive to modify the common environment. Implementing this is straightforward:
# Variables exported for user-defined libraries
export ARCH_TYPE ?= x86_64
export DEFINES
export CPP_FLAGS
export SANITIZERS
export SYSTEM_ROOT_PATHBy exporting these variables, they become accessible to the libraries, ensuring consistency in compilation parameters.
When it comes to including user-defined libraries, there are more efficient approaches than manually adding the path of each library. To create an ultimate Makefile that saves time and effort, we can automate this process. We can leverage the consistent folder hierarchy of the libraries and employ some string manipulation techniques. As an application developer, you only need to add the names of the libraries you intend to use, such as LoggerLib, AlgorithmsLib, or MyPreciousLib.
Let's assume we have the following folder structure in our development environment, with two main folders: Applications and Libraries. Inside the Libraries folder, we have multiple libraries that share the same folder structure we mentioned earlier. In this case, the application's Makefile can refer to these libraries using a single common path.
.
├── Applications
│ ├── AnotherProject
│ │ ├── Makefile
│ │ ├── Outputs
│ │ └── SourceCode
│ └── BasicProject
│ ├── Makefile
│ ├── Outputs
│ └── SourceCode
├── Libraries
│ ├── AlgorithmsLib
│ │ ├── Makefile
│ │ ├── Outputs
│ │ └── SourceCode
│ └── LoggerLib
│ ├── Makefile
│ ├── Outputs
│ └── SourceCode
In this structure, we can determine the base paths for all libraries using the following snippet. Note that PRJ_ROOT represents the exact path of the application.
USER_LIBS_BASE_PATH := $(realpath $(PRJ_ROOT)/../../Libraries)By using this base path, we can easily include any library in our application project, as shown below:
USER_LIBS := LoggerLib AlgorithmsLibWhen using a library in your application, the first step is to include its header files in your source code. To achieve this, we need to adjust the include paths accordingly. Since we have the same folder hierarchy in all our in-house libraries, we can accomplish this with a few string manipulation techniques.
### Include paths ###
# Project itself
INCLUDES += -I$(PRJ_SOURCE_CODE)
# External libraries
INCLUDES += -I$(SYSTEM_ROOT_PATH)/usr/include/cairo
# User-defined libraries
INCLUDES += $(foreach libraryName,$(USER_LIBS),-I$(USER_LIBS_BASE_PATH)/$(libraryName)/SourceCode/Headers)In the above snippet, we modify the INCLUDES variable to include the necessary paths. For our in-house libraries, we automate the inclusion of their header files using the USER_LIBS_BASE_PATH variable. With the help of a loop, we iterate over the USER_LIBS variable, which contains the names of the user-defined libraries you want to use in your application. For each library name, we construct the corresponding include path using the USER_LIBS_BASE_PATH and the library name itself, and append it to the INCLUDES variable.
When incorporating user-defined static libraries into your project, it's essential to properly link these libraries during the final compilation stage. Static libraries are typically stored as archive files containing compiled object code from the library's source files. Linking involves combining these object files with your application's object files to create the final executable. This can be done by just adding the path of the library's archive file into the linker directive as an input.
Before dealing with the linker directive, we must create a variable that contains the paths to archive files of each library given in the USER_LIBS variable. This can be done as below.
USER_LIB_ARCHIVES := $(foreach libraryName,$(USER_LIBS),$(USER_LIBS_BASE_PATH)/$(libraryName)/Outputs/$(libraryName).a)When we have this variable, the rest is straight forward. Just add it into the linker call as below.
$(CC) --sysroot="$(SYSTEM_ROOT_PATH)" -o $(EXECUTABLE) $(CPP_OBJECTS) $(SANITIZERS) $(USER_LIB_ARCHIVES) $(EXTERNAL_LIBS)Up to this point, we dealt with sharing variables, adjusting include paths, and linking with the already produced library. Above all these things, there is another thing we must do: building the library.
The key step in building a library is that its user shouldn't know much about the underlying nitty-gritty details. The user should only know how to invoke the build process and get the resulting archive file. The folder hierarchy we discussed previously can help for this purpose. Any library that is consistent with this hierarchy can be compiled by just calling make all from its root path. The command we need is exactly as below.
$(foreach libraryName,$(USER_LIBS),$(MAKE) -C $(USER_LIBS_BASE_PATH)/$(libraryName) all;)Calling this command prior to linking would make the archive files of the given libraries be ready for use. Clearing the output of libraries is also similar:
$(foreach libraryName,$(USER_LIBS),$(MAKE) -C $(USER_LIBS_BASE_PATH)/$(libraryName) clean;)Let's apply all the things described up to this point and obtain the following Makefile for our application.
ifeq ($(MAKECMDGOALS),)
$(error No targets given!)
endif
PRJ_NAME := BasicProject
### Paths ###
PRJ_ROOT := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
PRJ_SOURCE_CODE := $(PRJ_ROOT)/SourceCode
PRJ_OUTPUT_DIR := $(PRJ_ROOT)/Outputs
PRJ_OBJECTS_DIR := $(PRJ_OUTPUT_DIR)/Objects
EXECUTABLE := $(PRJ_OUTPUT_DIR)/$(PRJ_NAME).elf
# User-defined Library paths
USER_LIBS_BASE_PATH := $(realpath $(PRJ_ROOT)/../../Libraries)
USER_LIBS := LoggerLib AlgorithmsLib
### Exported Variables ###
# Variables exported for user-defined libraries
export ARCH_TYPE ?= x86_64
export DEFINES
export CPP_FLAGS
export SANITIZERS
export SYSTEM_ROOT_PATH
ifneq ($(MAKECMDGOALS),clean)
$(info Compiling the project '$(PRJ_NAME)')
### Architecture Selection ###
ifeq ($(ARCH_TYPE), x86_64)
$(info x86 64-bit architecture choosen!)
CC := g++
SZ := size
SYSTEM_ROOT_PATH := /
else ifeq ($(ARCH_TYPE), aarch64)
$(info ARM 64-bit architecture choosen!)
CC := aarch64-linux-gnu-g++
SZ := aarch64-linux-gnu-size
SYSTEM_ROOT_PATH := /path/to/aarch64-rootfs
else ifeq ($(ARCH_TYPE), arm)
$(info ARM 32-bit architecture choosen!)
CC := arm-linux-gnueabihf-g++
SZ := arm-linux-gnueabihf-size
SYSTEM_ROOT_PATH := /path/to/arm-rootfs
else
$(error Architecture type not recognized! ARCH_TYPE: '$(ARCH_TYPE)')
endif
# Check if the toolchain exist in the environment
ifeq ($(strip $(shell which $(CC))),)
$(error Toolchain($(CC)) not found in the path! Check the environment.)
endif
### Symbols ###
DEFINES += -DPRJ_NAME=$(PRJ_NAME)
### Compilation Flags ###
DEBUG_LEVEL := 3
OPTIMIZATION_LEVEL := 0
CPP_FLAGS := -O$(OPTIMIZATION_LEVEL) -g$(DEBUG_LEVEL) -Wall -std=c++17
### Sanitizers ###
ifeq ($(DEBUG_LEVEL), 3)
ifeq ($(OPTIMIZATION_LEVEL), 0)
$(warning Enabling the sanitizers..)
# Detect memory related errors
SANITIZERS += -fsanitize=address
# Detect undefined behavior
SANITIZERS += -fsanitize=undefined
# Detect memory leaks
SANITIZERS += -fsanitize=leak
# Detect thread safety issues
# SANITIZERS += -fsanitize=thread
# Detect overflows
SANITIZERS += -fsanitize=signed-integer-overflow
# Detect out-of-bound accesses
SANITIZERS += -fsanitize=bounds
endif
endif
### External Libraries ###
EXTERNAL_LIBS := -lpthread -lrt -lcairo
### User-defined Libraries ###
USER_LIB_ARCHIVES := $(foreach libraryName,$(USER_LIBS),$(USER_LIBS_BASE_PATH)/$(libraryName)/Outputs/$(libraryName).a)
### Include paths ###
# Project itself
INCLUDES += -I$(PRJ_SOURCE_CODE)
# External libraries
INCLUDES += -I$(SYSTEM_ROOT_PATH)/usr/include/cairo
# User-defined libraries
INCLUDES += $(foreach libraryName,$(USER_LIBS),-I$(USER_LIBS_BASE_PATH)/$(libraryName)/SourceCode/Headers)
### Derived Variables ###
# Source files to be compiled (Relative path required to reflect the folder hierarchy in objects folder)
CPP_SRC_FILES := $(shell find -L $(PRJ_SOURCE_CODE) -type f -name "*.cpp" -print) # All C++ files
# Object files to be linked after compilation
CPP_OBJECTS := $(patsubst $(PRJ_SOURCE_CODE)/%.cpp,$(PRJ_OBJECTS_DIR)/%.o, $(CPP_SRC_FILES)) # All C++ objects
# Dependency files (For properly detecting modifications in header files)
DEP_FILES := $(patsubst $(PRJ_OBJECTS_DIR)/%.o,$(PRJ_OBJECTS_DIR)/%.d, $(CPP_OBJECTS)) # All dependency files
-include $(DEP_FILES) # Missing targets for dependencies
endif
### Targets ###
# Complete build (Compilation and linking)
all: $(CPP_OBJECTS)
@echo "Executing target '$@' for project '$(PRJ_NAME)'"
# Call the Makefile of each library
@$(foreach libraryName,$(USER_LIBS),$(MAKE) -C $(USER_LIBS_BASE_PATH)/$(libraryName) all;)
@$(CC) --sysroot="$(SYSTEM_ROOT_PATH)" -o $(EXECUTABLE) $(CPP_OBJECTS) $(SANITIZERS) $(USER_LIB_ARCHIVES) $(EXTERNAL_LIBS)
@echo "Finished executing the target '$@' for project '$(PRJ_NAME)'"
clean:
@echo "Executing target '$@' for project '$(PRJ_NAME)'"
# Call the Makefile of each library
@$(foreach libraryName,$(USER_LIBS),$(MAKE) -C $(USER_LIBS_BASE_PATH)/$(libraryName) clean;)
rm -rf $(PRJ_OBJECTS_DIR)
@echo "Finished cleaning the target '$@' for project '$(PRJ_NAME)'"
# Generic C/C++ files build target (Object generation)
$(PRJ_OBJECTS_DIR)/%.o : $(PRJ_SOURCE_CODE)/%.cpp
@echo "Building '$<' for project '$(PRJ_NAME)'"
@mkdir -p $(@D)
$(CC) $(CPP_FLAGS) $(SANITIZERS) $(INCLUDES) $(DEFINES) -c $< -o $@ -MD -MP
@echo "Built '$<' for project '$(PRJ_NAME)'"
# Prevent confusion between files and targets
.PHONY: all cleanIf you've read this article up to this point, I'm pretty sure that you already figured out how the library Makefile is going to be. So, without any boring explanation, below is the Makefile for our libraries.
ifeq ($(MAKECMDGOALS),)
$(error No targets given!)
endif
LIB_NAME := LoggerLib
### Paths ###
LIB_ROOT := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
LIB_HEADERS := $(LIB_ROOT)/SourceCode/Headers
LIB_SOURCES := $(LIB_ROOT)/SourceCode/Sources
LIB_OUTPUT_DIR := $(LIB_ROOT)/Outputs
LIB_OBJECTS_DIR := $(LIB_OUTPUT_DIR)/Objects
ARCHIVE := $(LIB_OUTPUT_DIR)/$(LIB_NAME).a
ifneq ($(MAKECMDGOALS),clean)
ifeq ($(ARCH_TYPE),)
$(error No architecture given!)
endif
# Decide on the compiler toolchain
ifeq ($(ARCH_TYPE), x86_64)
$(info x86 64-bit architecture choosen!)
CC := g++
AR := gcc-ar
else ifeq ($(ARCH_TYPE), aarch64)
$(info ARM 64-bit architecture choosen!)
CC := aarch64-linux-gnu-g++
AR := aarch64-linux-gnu-ar
else ifeq ($(ARCH_TYPE), arm)
$(info ARM 32-bit architecture choosen!)
CC := arm-linux-gnueabihf-g++
AR := arm-linux-gnueabihf-ar
else
$(error Architecture type not recognized! ARCH_TYPE: '$(ARCH_TYPE)')
endif
# Check if the toolchain exist in the environment
ifeq ($(strip $(shell which $(CC))),)
$(error Toolchain($(CC)) not found in the path! Check the environment.)
endif
### Symbols ###
DEFINES += -DLIB_NAME=$(LIB_NAME)
### Compilation Flags ###
# Define the flags if they weren't defined in the calling environment
CPP_FLAGS ?= -O0 -g3
### Include paths ###
INCLUDES += -I$(LIB_HEADERS)
### Derived Variables ###
# Source files to be compiled (Relative path required to reflect the folder hierarchy in the objects folder)
CPP_SRC_FILES := $(shell find -L $(LIB_SOURCES) -type f -name "*.cpp" -print) # All C++ files
# Object files to be archived after compilation
CPP_OBJECTS := $(patsubst $(LIB_SOURCES)/%.cpp,$(LIB_OBJECTS_DIR)/%.o, $(CPP_SRC_FILES)) # All C++ objects
# Dependency files (For properly detecting modifications in header files)
DEP_FILES := $(patsubst $(LIB_OBJECTS_DIR)/%.o,$(LIB_OBJECTS_DIR)/%.d, $(CPP_OBJECTS)) # All dependency files
-include $(DEP_FILES) # Missing targets for dependencies
endif
### Targets ###
all: $(CPP_OBJECTS)
@echo "Executing target $@ for library $(LIB_NAME)"
# Make output directory if not exist
@mkdir -p $(LIB_OUTPUT_DIR)
# Create a new archive file
$(AR) rcs $(ARCHIVE) $(CPP_OBJECTS)
@echo "Finished target $@ for library $(LIB_NAME)"
clean:
rm -rf $(LIB_OUTPUT_DIR)
# Generic C/C++ files build target (Object generation)
$(LIB_OBJECTS_DIR)/%.o : $(LIB_SOURCES)/%.cpp
@echo "Building '$<' for library $(LIB_NAME)"
@mkdir -p $(@D)
@$(CC) $(CPP_FLAGS) $(SANITIZERS) $(INCLUDES) $(DEFINES) -c $< -o $@ -MD -MP
@echo "Built '$<' for library $(LIB_NAME)"
# Prevent confusion between files and targets
.PHONY: all cleanAs you can see that it's quite similar to the application Makefile I've proposed. The key difference is that this one is more dependent to its environment which includes the parameters or variables it needs for compilation. Also, in the all target you may realize that we don't link the compiled object files. Instead, they are archived into an .a file to be linked in the application itself.
In conclusion, this article explored the integration of user-defined static libraries into your Makefile-based build system. By maintaining consistent folder structures, sharing compilation flags, and automating library inclusion, you can streamline your workflow and enhance code modularity. Key takeaways from this article include the importance of the export directive for sharing variables between Makefiles, ensuring compatibility in compilation settings. Consistent compilation flags are vital to prevent compatibility issues and optimize performance between your application and libraries. We outlined a practical method for incorporating user-defined libraries by leveraging a common structure and automated inclusion. This approach simplifies library integration and reduces manual configuration.
In summary, user-defined static libraries offer enhanced code organization and streamlined build processes. Armed with the insights from this article, you're well-prepared to leverage libraries effectively in your Makefile projects, fostering efficient and modular C++ development. Happy coding!