
Compilation pipeline
Compiling some C files usually takes a few seconds, but during this brief period of time, the source code enters a pipeline that has four distinct components, with each of them doing a certain task. These components are as follows:
- Preprocessor
- Compiler
- Assembler
- Linker
Each component in this pipeline accepts a certain input from the previous component and produces a certain output for the next component in the pipeline. This process continues through the pipeline until a product is generated by the last component.
Source code can be turned into a product if, and only if, it passes through all the required components with success. This means that even a small failure in one of the components can lead to a compilation or linkage failure, resulting in you receiving relevant error messages.
For certain intermediate products such as relocatable object files, it is enough that a single source file goes through the first three components with success. The last component, the linker, is usually used to create bigger products, such as an executable object file, by merging some of the already prepared relocatable object files. So, building a collection of C source files can create one or sometimes multiple object files, including relocatable, executable, and shared object files.
There are currently a variety of C compilers available. While some of them are free and open source, others are proprietary and commercial. Likewise, some compilers will only work on a specific platform while others are cross-platform, although, the important note is that almost every platform has at least one compatible C compiler.
Note:
For a complete list of available C compilers, please have a look at the following Wikipedia page: https://en.wikipedia.org/wiki/List_of_compilers#C_compilers.
Before talking about the default platform and the C compiler that we use throughout this chapter, let's talk a bit more about the term platform, and what we mean by it.
A platform is a combination of an operating system running on specific hardware (or architecture), and its CPU's instruction set is the most important part of it. The operating system is the software component of a platform, and the architecture defines the hardware part. As an example, we can have Ubuntu running on an ARM-powered board, or we could have Microsoft Windows running on an AMD 64-bit CPU.
Cross-platform software can be run on different platforms. However, it is vital to know that cross-platform is different from being portable. Cross-platform software usually has different binaries (final object files) and installers for each platform, while portable software uses the same produced binaries and installers on all platforms.
Some C compilers, for example, gcc
and clang
, are cross-platform – they can generate code for different platforms – and Java bytecode is portable.
Regarding C and C++, if we say that C/C++ code is portable, we mean that we can compile it for different platforms without any change or with little modification to the source code. This doesn't mean that the final object files are portable, however.
If you have looked at the Wikipedia article we noted before, you can see that there are numerous C compilers. Fortunately for us, all of them follow the same standard compilation pipeline that we are going to introduce in this chapter.
Among these many compilers, we need to choose one of them to work with during this chapter. Throughout this chapter, we will be using gcc
7.3.0 as our default compiler. We are choosing gcc
because it is available on most operating systems, in addition to the fact that there are many online resources to be found for it.
We also need to choose our default platform. In this chapter, we have chosen Ubuntu 18.04 as our default operating system running on an AMD 64-bit CPU as our default architecture.
Note:
From time to time this chapter might refer to a different compiler, a different operating system, or a different architecture to compare various platforms and compilers. If we do so, the specification of the new platform or the new compiler will be given beforehand.
In the following sections, we are going to describe the steps in the compilation pipeline. First, we are going to build a simple example to see how the sources inside a C project are compiled and linked. Throughout this example, we will become familiar with new terms and concepts regarding the compilation process. Only after that do we address each component individually in a separate section. There, we go deep in to each component to explain more internal concepts and processes.
Building a C project
In this section, we are going to demonstrate how a C project is built. The project that we are going to work on consists of more than one source file, which is a common characteristic of almost all C projects. However, before we move to the example and start building it, we need to ensure that we understand the structure of a typical C project.
Header files versus source files
Every C project has source code, or code base, together with other documents related to the project description and existing standards. In a C code base, we usually have two kinds of files that contain C code:
- Header files, which usually have a
.h
extension in their names. - Source files, which have a
.c
extension.
Note:
For convenience, in this chapter, we may use the terms header instead of header file and source instead of source file.
A header file usually contains enumerations, macros, and typedefs, as well as the declarations of functions, global variables, and structures. In C, some programming elements such as functions, variables, and structures can have their declaration separated from their definition placed in different files.
C++ follows the same pattern, but in other programming languages, such as Java, the elements are defined where they are declared. While this is a great feature of both C and C++, as it gives them the power to decouple the declarations from definitions, it also makes the source code more complex.
As a rule of thumb, the declarations are stored in header files, and the corresponding definitions go to source files. This is even more critical with regard to function declarations and function definitions.
It is strongly recommended that you only keep function declarations in header files and move function definitions to the corresponding source files. While this is not necessary, it is an important design practice to keep those function definitions out of the header files.
While the structures could also have separate declarations and definitions, there are special cases in which we move declarations and definitions to different files. We will see an example of this in Chapter 8, Inheritance and Polymorphism, where we will be discussing the inheritance relationship between classes.
Note:
Header files can include other header files, but never a source file. Source files can only include header files. It is bad practice to let a source file include another source file. If you do, then this usually means that you have a serious design problem in your project.
To elaborate more on this, we are going to look at an example. The following code is the declaration of the average
function. A function declaration consists of a return type and a function signature. A function signature is simply the name of the function together with the list of its input parameters:
double average(int*, int);
Code Box 2-1: The declaration of the average function
The declaration introduces a function signature whose name is average
and it receives a pointer to an array of integers together with a second integer argument, which indicates the number of elements in the array. The declaration also states that the function returns a double value. Note that the return type is a part of the declaration but is not often considered a part of the function signature.
As you can see in Code Box 2-1, a function declaration ends with a semicolon ";" and it does not have a body embraced by curly brackets. We should also take note that the parameters in the function declaration do not have associated names, and this is valid in C, but only in declarations and not in definitions. With that being said, it is recommended that you name the parameters even in declarations.
The function declaration is about how to use the function and the definition defines how that function is implemented. The user doesn't need to know about the parameter names to use the function, and because of that it's possible to hide them in the function declaration.
In the following code, you can find the definition of the average
function that we declared before. A function definition contains the actual C code representing the function's logic. This always has a body of code embraced by a pair of curly brackets:
double average(int* array, int length) {
if (length <= 0) {
return 0;
}
double sum = 0.0;
for (int i = 0; i < length; i++) {
sum += array[i];
}
return sum / length;
}
Code Box 2-2: The definition of the average function
Like we said before, and to put more emphasis on this, function declarations go to headers, and definitions (or the bodies) go into source files. There are rare cases in which we have enough reason to violate this. In addition, sources need to include header files in order to see and use the declarations, which is how C and C++ work.
If you do not fully understand this now, do not worry as this will become more obvious as we move forward.
Note:
Having more than one definition for any declaration in a translation unit will lead to a compile error. This is true for all functions, structures, and global variables. Therefore, providing two definitions for a single function declaration is not permitted.
We are going to continue this discussion by introducing our first C example for this chapter. This example is supposed to demonstrate the correct way of compiling a C/C++ project consisting of more than one source file.
Example source files
In example 2.1, we have three files, with one being a header file, and the other two being source files, and all are in the same directory. The example wants to calculate the average of an array with five elements.
The header file is used as a bridge between two separate source files and makes it possible to write our code in two separate files but build them together. Without the header file, it's not possible to break our code in two source files, without breaking the rule mentioned above (sources must not include sources). Here, the header file contains everything required by one of the sources to use the functionality of the other one.
The header file contains only one function declaration, avg
, needed for the program to work. One of the source files contains the definition of the declared function. The other source file contains the main
function, which is the entry point of the program. Without the main
function, it is impossible to have an executable binary to run the program with. The main
function is recognized by the compiler as the starting point of the program.
We are now going to move on and see what the contents of these files are. Here is the header file, which contains an enumeration and a declaration for the avg
function:
#ifndef EXTREMEC_EXAMPLES_CHAPTER_2_1_H
#define EXTREMEC_EXAMPLES_CHAPTER_2_1_Htypedef enum {
NONE,
NORMAL,
SQUARED
} average_type_t;
// Function declaration
double avg(int*, int, average_type_t);
#endif
Code Box 2-3 [ExtremeC_examples_chapter2_1.h]: The header file as part of example 2.1
As you can see, this file contains an enumeration, a set of named integer constants. In C, enumerations cannot have separate declarations and definitions, and they should be declared and defined just once in the same place.
In addition to the enumeration, the forward declaration of the avg function can be seen in the code box. The act of declaring a function before giving its definition is called forward declaration. The header file is also protected by the header guard statements. They will prevent the header file from being included twice or more while being compiled.
The following code shows us the source file that actually contains the definition of the avg
function:
#include "ExtremeC_examples_chapter2_1.h"
double avg(int* array, int length, average_type_t type) {
if (length <= 0 || type == NONE) {
return 0;
}
double sum = 0.0;
for (int i = 0; i < length; i++) {
if (type == NORMAL) {
sum += array[i];
} else if (type == SQUARED) {
sum += array[i] * array[i];
}
}
return sum / length;
}
Code Box 2-4 [ExtremeC_examples_chapter2_1.c]: The source file containing the definition of avg function
With the preceding code, you should notice that the filename ends with a .c
extension. The source file has included the example's header file. This has been done because it needs the declarations of the average_type_t
enumeration and the avg
function before using them. Using a new type, in this case, the average_type_t
enumeration, without declaring it before its usage leads to a compilation error.
Look at the following code box showing the second source file that contains the main
function:
#include <stdio.h>
#include "ExtremeC_examples_chapter2_1.h"
int main(int argc, char** argv) {
// Array declaration
int array[5];
// Filling the array
array[0] = 10;
array[1] = 3;
array[2] = 5;
array[3] = -8;
array[4] = 9;
// Calculating the averages using the 'avg' function
double average = avg(array, 5, NORMAL);
printf("The average: %f\n", average);
average = avg(array, 5, SQUARED);
printf("The squared average: %f\n", average);
return 0;
}
Code Box 2-5 [ExtremeC_examples_chapter2_1_main.c]: The main function of example 2.1
In every C project, the main
function is the entry point of the program. In the preceding code box, the main
function declares and populates an array of integers and calculates two different averages for it. Note how the main
function calls the avg function in the preceding code.
Building the example
After introducing the files of example 2.1 in the previous section, we need to build them and create a final executable binary file that can be run as a program.Building a C/C++ project means that we will compile all the sources within its code base to first produce some relocatable object files (known as intermediate object files too), and finally combine those relocatable object files to produce the final products, such as static libraries or executable binaries.
Building a project in other programming languages is also very similar to doing it in either C or C++, but the intermediate and final products have different names and likely different file formats. For example, in Java, the intermediate products are class files containing Java bytecode, and the final products are JAR or WAR files.
Note:
To compile the example sources, we will not use an Integrated Development Environment (IDE). Instead, we are going to use the compiler directly without help from any other software. Our approach to building the example is exactly the same as the one that is employed by IDEs and performed in the background while compiling a number of source files.
Before we go any further, there are two important rules that we should remember.
Rule 1: We only compile source files
The first rule is that we only compile source files due to the fact that it is meaningless to compile a header file. Header files should not contain any actual C code other than some declarations. Therefore, for example 2.1, we only need to compile two source files: ExtremeC_examples_chapter2_1.c
and ExtremeC_examples_chapter2_1_main.c
.
Rule 2: We compile each source file separately
The second rule is that we compile each source file separately. Regarding example 2.1, it means that we have to run the compiler twice, each time passing one of the source files.
Note:
It is still possible to pass two source files at once and ask the compiler to compile them in just one command, but we don't recommend it and we don't do that in this book.
Therefore, for a project made up of 100 source files, we need to compile every source file separately, and it means that we have to run the compiler 100 times! Yes, that seems to be a lot, but this is the way that you should compile a C or C++ project. Believe me, you will encounter projects in which several thousand files should be compiled before having a single executable binary!
Note:
If a header file contains a piece of C code that needs to be compiled, we do not compile that header file. Instead, we include it in a source file, and then, we compile the source file. This way, the header's C code will be compiled as part of the source file.
When we compile a source file, no other source files are going to be compiled as part of the same compilation because none of them are included by the compiling source file. Remember, including source files is not allowed if we respect the best practices in C/C++.
Now let's focus on the steps that should be taken in order to build a C project. The first step is preprocessing, and we are going to talk about that in the following section.
Step 1 – Preprocessing
The first step in the C compilation pipeline is preprocessing. A source file has a number of header files included. However, before the compilation begins, the contents of these files are gathered by the preprocessor as a single body of C code. In other words, after the preprocessing step, we get a single piece of code created by copying content of the header files into the source file content.
Also, other preprocessor directives must be resolved in this step. This preprocessed piece of code is called a translation unit. A translation unit is a single logical unit of C code generated by the preprocessor, and it is ready to be compiled. A translation unit is sometimes called a compilation unit as well.
Note:
In a translation unit, no preprocessing directives can be found. As a reminder, all preprocessing directives in C (and C++) start with #
, for example, #include
and #define
.
It is possible to ask compilers to dump the translation unit without compiling it further. In the case of gcc
, it is enough to pass the -E
option (this is case-sensitive). In some rare cases, especially when doing cross-platform development, examining the translation units could be useful when fixing weird issues.
In the following code, you can see the translation unit for ExtremeC_examples_chapter2_1.c
, which has been generated by gcc
on our default platform:
$ gcc -E ExtremeC_examples_chapter2_1.c
# 1 "ExtremeC_examples_chapter2_1.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 32 "<command-line>" 2
# 1 "ExtremeC_examples_chapter2_1.c"
# 1 "ExtremeC_examples_chapter2_1.h" 1
typedef enum {
NONE,
NORMAL,
SQUARED
} average_type_t;
double avg(int*, int, average_type_t);
# 5 "ExtremeC_examples_chapter2_1.c" 2
double avg(int* array, int length, average_type_t type) {
if (length <= 0 || type == NONE) {
return 0;
}
double sum = 0;
for (int i = 0; i < length; i++) {
if (type == NORMAL) {
sum += array[i];
} else if (type == SQUARED) {
sum += array[i] * array[i];
}
}
return sum / length;
}
$
Shell Box 2-1: The produced translation unit while compiling ExtremeC_examples_chapter2_1.c
As you can see, all the declarations are copied from the header file into the translation unit. The comments have also been removed from the translation unit.
The translation unit for ExtremeC_examples_chapter2_1_main.c
is very large because it includes the stdio.h
header file.
All declarations from this header file, and further inner header files included by it, will be copied into the translation unit recursively. Just to show how big the translation unit of ExtremeC_examples_chapter2_1_main.c
can be, on our default platform it has 836 lines of C code!
Note:
The -E
option works also for the clang
compiler.
This completes the first step. The input to the preprocessing step is a source file, and the output is the corresponding translation unit.
Step 2 – Compilation
Once you have the translation unit, you can go for the second step, which is compilation. The input to the compilation step is the translation unit, retrieved from the previous step, and the output is the corresponding assembly code. This assembly code is still human-readable, but it is machine-dependent and close to the hardware and still needs further processing in order to become machine-level instructions.
You can always ask gcc
to stop after performing the second step and dump the resulting assembly code by passing the -S
option (capital S). The output is a file with the same name as the given source file but with a .s
extension.
In the following shell box, you can see the assembly of the ExtremeC_examples_chapter2_1_main.c
source file. However, when reading the code, you should see that some parts of the output are removed:
$ gcc -S ExtremeC_examples_chapter2_1.c
$ cat ExtremeC_examples_chapter2_1.s
.file "ExtremeC_examples_chapter2_1.c"
.text
.globl avg
.type avg, @function
avg:
.LFB0:
.cfi_startproc
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset 6, -16
movq %rsp, %rbp
.cfi_def_cfa_register 6
movq %rdi, -24(%rbp)
movl %esi, -28(%rbp)
movl %edx, -32(%rbp)
cmpl $0, -28(%rbp)
jle .L2
cmpl $0, -32(%rbp)
jne .L3
.L2:
pxor %xmm0, %xmm0
jmp .L4
.L3:
...
.L8:
...
.L6:
...
.L7:
...
.L5:
...
.L4:
...
.LFE0:
.size avg, .-avg
.ident "GCC: (Ubuntu 7.3.0-16ubuntu3) 7.3.0"
.section .note.GNU-stack,"",@progbits
$
Shell Box 2-2: The produced assembly code while compiling ExtremeC_examples_chapter2_1.c
As part of the compilation step, the compiler parses the translation unit and turns it into assembly code that is specific to the target architecture. By the target architecture, we mean the hardware or CPU that the program is being compiled for and is eventually to be run on. The target architecture is sometimes referred to as the host architecture.
Shell Box 2-2 shows the assembly code generated for the AMD 64-bit architecture and produced by gcc
running on an AMD 64-bit machine. The following shell box contains the assembly code generated for an ARM 32-bit architecture and produced by gcc
running on an Intel x86-64 architecture. Both assembly outputs are generated for the same C code:
$ cat ExtremeC_examples_chapter2_1.s
.arch armv5t
.fpu softvfp
.eabi_attribute 20, 1
.eabi_attribute 21, 1
.eabi_attribute 23, 3
.eabi_attribute 24, 1
.eabi_attribute 25, 1
.eabi_attribute 26, 2
.eabi_attribute 30, 6
.eabi_attribute 34, 0
.eabi_attribute 18, 4
.file "ExtremeC_examples_chapter2_1.s"
.global __aeabi_i2d
.global __aeabi_dadd
.global __aeabi_ddiv
.text
.align 2
.global avg
.syntax unified
.arm
.type avg, %function
avg:
@ args = 0, pretend = 0, frame = 32
@ frame_needed = 1, uses_anonymous_args = 0
push {r4, fp, lr}
add fp, sp, #8
sub sp, sp, #36
str r0, [fp, #-32]
str r1, [fp, #-36]
str r2, [fp, #-40]
ldr r3, [fp, #-36]
cmp r3, #0
ble .L2
ldr r3, [fp, #-40]
cmp r3, #0
bne .L3
.L2:
...
.L3:
...
.L8:
...
.L6:
...
.L7:
...
.L5:
...
.L4:
mov r0, r3
mov r1, r4
sub sp, fp, #8
@ sp needed
pop {r4, fp, pc}
.size avg, .-avg
.ident "GCC: (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609"
.section .note.GNU-stack,"",%progbits
$
Shell Box 2-3: The assembly code produced while compiling ExtremeC_examples_chapter2_1.c for an ARM 32-bit architecture
As you can see in shell boxes 2-2 and 2-3, the generated assembly code is different for the two architectures. This is despite the fact that they are generated for the same C code. For the latter assembly code, we have used the arm-linux-gnueabi-gcc
compiler on an Intel x64-86 hardware set running Ubuntu 16.04.
Note:
The target (or host) architecture is the architecture that the source is both being compiled for and will be run on. The build architecture is the architecture that we are using to compile the source. They can be different. For example, you can compile a C source for AMD 64-bit hardware on an ARM 32-bit machine.
Producing assembly code from C code is the most important step in the compilation pipeline.
This is because when you have the assembly code, you are very close to the language that a CPU can execute. Because of this important role, the compiler is one of the most important and most studied subjects in computer science.
Step 3 – Assembly
The next step after compilation is assembly. The objective here is to generate the actual machine-level instructions (or machine code) based on the assembly code generated by the compiler in the previous step. Each architecture has its own assembler, which can translate its own assembly code to its own machine code.
A file containing the machine-level instructions that we are going to assemble in this section is called an object file. We know that a C project can have several products that are all object files, but in this section, we are mainly interested in relocatable object files. This file is, without a doubt, the most important temporary product that we can obtain during the build process.
Note:
Relocatable object files can be referred to as intermediate object files.
To pull both of the previous steps together, the purpose of this assembly step is to generate a relocatable object file out of the assembly code produced by the compiler. Every other product that we create will be based on the relocatable object files generated by the assembler in this step.
We will talk about these other products in the future sections of this chapter.
Note:
Binary file and object file are synonyms that refer to a file containing machine-level instructions. Note however that the term "binary files" in other contexts can have different meanings, for example binary files vs. text files.
In most Unix-like operating systems, we have an assembler tool called as
, which can be used to produce a relocatable object file from an assembly file.
However, these object files are not executable, and they only contain the machine-level instructions generated for a translation unit. Since each translation unit is made up of various functions and global variables, a relocatable object file simply contains machine-level instructions for the corresponding functions and the pre-allocated entries for the global variables.
In the following shell box, you can see how as
is used to produce the relocatable object file for ExtremeC_examples_chapter2_1_main.s
:
$ as ExtremeC_examples_chapter2_1.s -o ExtremeC_examples_chapter2_1.o
$
Shell Box 2-4: Producing an object file from the assembly of one of the sources in example 2.1
Looking back at the command in the preceding shell box, we can see that the -o
option is used to specify the name of the output object file. Relocatable object files usually have a .o
(or a .obj
in Microsoft Windows) extension in their names, which is why we have passed a filename with .o
at the end.
The content of an object file, either .o
or .obj
, is not textual, so you would not be able to read it as a human. Therefore, it is common to say that an object file has binary content.
Despite the fact that the assembler can be used directly, like what we did in Shell Box 2-4, this is not recommended. Instead, good practice would be to use the compiler itself to call as
indirectly in order to generate the relocatable object file.
Note:
We may use the terms object file and relocatable object file interchangeably. But not all object files are relocatable object files, and, in some contexts, it may refer to other types of object files such as shared object files.
If you pass the -c
option to almost all known C compilers, it will directly generate the corresponding object file for the input source file. In other words, the -c
option is equivalent to performing the first three steps all together.
Looking at the following example, you can see that we have used the -c
option to compile ExtremeC_examples_chapter2_1.c
and generate its corresponding object file:
$ gcc -c ExtremeC_examples_chapter2_1.c
$
Shell Box 2-5: Compiling one of the sources in example 2.1 and producing its corresponding relocatable object file
All of the steps we have just done – preprocessing, compilation, and assembling – are done as part of the preceding single command. What this means for us is that after running the preceding command, a relocatable object file will be generated. This relocatable object file will have the same name as the input source file; however, it will differ by having a .o
extension.
IMPORTANT:
Note that, often, the term compilation is used to refer to the first three steps in the compilation pipeline all together, and not just the second step. It is also possible that we use the term "compilation" but actually mean "building;" encompassing all four steps. For instance, we say C compilation pipeline, but we actually mean C build pipeline.
The assembly is the last step in compiling a single source file. In other words, when we have the corresponding relocatable object file for a source file, we are done with its compilation. At this stage we can put aside the relocatable object file and continue compiling other source files.
In example 2.1, we have two source files that need to be compiled. By executing the following commands, it compiles both source files and as a result, produces their corresponding object files:
$ gcc -c ExtremeC_examples_chapter2_1.c -o impl.o
$ gcc -c ExtremeC_examples_chapter2_1_main.c -o main.o
$
Shell Box 2-6: Producing the relocatable object files for the sources in example 2.1
You can see in the preceding commands that we have changed the names of the object files by specifying our desired names using the -o
option. As a result, after compiling both of them, we get the impl.o
and main.o
relocatable object files.
At this point, we need to remind ourselves that relocatable object files are not executable. If a project is going to have an executable file as its final product, we need to use all, or at the very least, some, of the already produced relocatable object files to build the target executable file through the linking step.
Step 4 – Linking
We know that example 2.1 needs to be built to an executable file because we have a main
function in it. However, at this point, we only have two relocatable object files. Therefore, the next step is to combine these relocatable object files in order to create another object file that is executable. The linking step does exactly that.
However, before we go through the linking step, we need to talk about how we add support for a new architecture, or hardware, to an existing Unix-like system.
Supporting new architectures
We know that every architecture has a series of manufactured processors and that every processor can execute a specific instruction set.
The instruction set has been designed by vendor companies such as Intel and ARM for their processors. In addition, these companies also design a specific assembly language for their architecture.
A program can be built for a new architecture if two prerequisites are satisfied:
- The assembly language is known.
- The required assembler tool (or program) developed by the vendor company must be at hand. This allows us to translate the assembly code into the equivalent machine-level instructions.
Once these prerequisites are in place, it would be possible to generate machine-level instructions from C source code. Only then, we are able to store the generated machine-level instructions within the object files using an object file format. As an example, this could be in the form of either ELF or Mach-O.
When the assembly language, assembler tool, and object file format are clear, they can be used to develop some further tools that are necessary for us developers when doing C programming. However, you hardly notice their existence since you are often dealing with a C compiler, and it is using these tools on your behalf.
The two immediate tools that are required for a new architecture are as follows:
- C compiler
- Linker
These tools are like the first fundamental building blocks for supporting a new architecture in an operating system. The hardware together with these tools in an operating system give rise to a new platform.
Regarding Unix-like systems, it is important to remember that Unix has a modular design. If you are able to build a few fundamental modules like the assembler, compiler, and linker, you will be able to build other modules on top of them and before long, the whole system is working on a new architecture.
Step details
With all that's been said before, we know that platforms using Unix-like operating systems must have the previously discussed mandatory tools, such as an assembler and a linker, in order to work. Remember, the assembler and the linker can be run separately from the compiler.
In Unix-like systems, ld
is the default linker. The following command, which you can see in the following shell box, shows us how to use ld
directly when we want to create an executable from the relocatable object files we produced in the previous sections for example 2.1. However, as you will see, it is not that easy to use the linker directly:
$ ld impl.o main.o
ld: warning: cannot find entry symbol _start; defaulting to 00000000004000e8
main.o: In function 'main':
ExtremeC_examples_chapter3_1_main.c:(.text+0x7a): undefined reference to 'printf'
ExtremeC_examples_chapter3_1_main.c:(.text+0xb7): undefined reference to 'printf'
ExtremeC_examples_chapter3_1_main.c:(.text+0xd0): undefined reference to '__stack_chk_fail'
$
Shell Box 2-7: Trying to link the object files using the ld utility directly
As you see, the command has failed, and it has generated some error messages. If you pay attention to the error messages, they say that in three places in the Text segment ld
has encountered three function calls (or references) that are undefined.
Two of these function calls are calls to the printf
function, which we did in the main
function. However, the other one, __stack_chk_fail
, has not been called by us. It is coming from somewhere else, but where? It has been called from the supplementary code that has been put into the relocatable object files by the compiler, and this function is specific to Linux, and you may not find it in the same object files generated on other platforms. However, whatever it is and whatever it does, the linker is looking for its definition and it seems that it cannot find the definition in the provided object files.
Like we said before, the default linker, ld
, has generated these errors because it has not been able to find the definitions of these functions. Logically, this makes sense, and is true, because we have not defined printf
and __stack_chk_fail
ourselves in example 2.1.
This means that we should have given ld
some other object files, though not necessarily relocatable object files, that contain the definitions of the printf
and __stack_chk_fail
functions.
Reading what we have just said should explain why it can be very hard to use ld
directly. Namely, there are more object files and options that need to be specified in order to make ld
work and generate a working executable.
Fortunately, in Unix-like systems, the most well-known C compilers use ld
by passing proper options and specifying extra required object files. Hence, we do not need to use ld
directly.
Therefore, let's look at a much simpler way of producing the final executable file. The following shell box shows us how we can use gcc
to link the object files from example 2.1:
$ gcc impl.o main.o
$ ./a.out
The average: 3.800000
The squared average: 55.800000
$
Shell Box 2-8: Using gcc to link the object files
As a result of running these commands, we can breathe because we have finally managed to build example 2.1 and run its final executable!
Note:
Building a project is equivalent to compiling the sources firstly and then linking them together, and possibly other libraries, to create the final products.
It is important to take a minute to pause and reflect on what we have just done. Over the last few sections we have successfully built example 2.1 by compiling its sources into relocatable object files, and finally linking the generated object files to create the final executable binary.
While this process will be the same for any C/C++ code base, the difference will be in the number of times you need to compile sources, which itself depends on the number of source files in your project.
While the compilation pipeline has some steps, in each step, there is a specific component involved. The focus of the remaining sections of this chapter will be delving into the critical information surrounding each component in the pipeline.
To start this, we are going to focus on the preprocessor component.