linking with C/C++ libraries with gcc/g++ when building an existing project

linking with C C++ libraries with gcc g++ when building an existing project

Notes on linking with C/C++ libraries with gcc/g++ when building an
existing project.
                                                -- Iain Murray, 2015.


There are at least three things that can go wrong when trying to link
with a library that is in a non-standard location on your machine:

1. The compiler can't find the .h header files.
2. The linker can't find the library's binary .a or .so files.
3. At run time, your binary can't find the library's .so file.

Usually the build script for a project should sort out at least 1 and 2.
If it has a configure script, try looking at:
    ./configure --help
which may have options to point the build system to any libraries in
non-standard locations.

Unfortunately many projects have broken build systems that don't work if
libaries are in non-standard locations. So here's some notes on brute
force fixes to the three issues above.



1. Pointing the compiler to .h header files for #include lines to work
----------------------------------------------------------------------

If you are calling gcc or g++, pass the path to the .h file like this:
    gcc -I/location/of/headers ...
or
    g++ -I/location/of/headers ...

If you have a complicated build system, you may not be able to easily
specify the compilation command-line directly. Instead you can export
the CPATH environment variable, with a colon-separated list of paths.
Then call your build system.

In bash set the environment variable like this:
    export CPATH="/location/one:/location/two"
or in (t)csh like this:
    setenv CPATH "/location/one:/location/two"

Environment variables that gcc looks at are documented here:
    https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html

If you're really unlucky, the build system might reset environment
variables like CPATH. As a last resort hack, you could create a shell
script called "gcc" with the following contents (specifying the paths
you need):
    #!/bin/sh
    exec /usr/bin/gcc -I/location/one -I/location/two "$@"
make it executable with
    chmod +x gcc
and put it at the front of your path. E.g., put it in ~/bin/override
and do:
    export PATH="$HOME/bin/override:$PATH"
Then do the same for g++.


2. Pointing the linker to the library's binaries
------------------------------------------------

With access to library headers, gcc can compile object code (.o files
created when using the -c command-line option) without needing access to
the library routines themselves.

An executable will need the library routines though. These are either
included with the executable (statically linking the library's .a files
into the binary) or accessed from a .so "shared object" file.

An executable is created with a linker (usually /usr/bin/ld), but the
linker is usually called for us by gcc. We tell gcc where the .a or .so
files are by using:
    -L/path/to/library/code
We also have to tell it which libraries to link with, either by
explicitly linking in the .a file or specifying a library by name, for
example:
    -lfftw3
to link with libfftw3. However, if we're building an existing project,
its build system should already specify the library names. It's only
the locations of the libraries we'll have to fix.

If you can't easily specify paths with -L, we can again set a
colon-separated list in an environment variable:
    export LIBRARY_PATH=/location/one:/location/two
These locations are often different from the ones we previously put in
CPATH. They're directories containing library binaries (.a or .so files)
rather than the header files (.h).

If setting the environment variable doesn't work, as a last resort you
could override the gcc binary as in the previous section. This time
forcing the use of -L command-line arguments.


3. Making sure your executable can find the library's .so file
--------------------------------------------------------------

Even if the compiler knew where a shared object (.so file) was at
compile time, your resulting program can still have difficulty finding
it at run time. Even if it hasn't moved!

You can tell an executable where to look by setting LD_LIBRARY_PATH:
    export LD_LIBRARY_PATH="/location/one:/location/two"
before running your program.

Alternatively, you can make the linker bake the location into the
executable by passing:
    -Wl,-rpath,"/location/one"
to gcc (that's a letter 'l' not a one after the W). OR by setting an
environment variable:
    export LD_RUN_PATH="/location/one:/location/two"
before compiling. You then don't need to set LD_LIBRARY_PATH every time
you run the program.

You can see what shared libraries your program is trying to use with:
    ldd ./my_program