Node v4.x.y binaries have been built with compilers at versions greater than the ones used to build packages in the LTS pkgsrc repositories. They've been built with gcc 4.9 and gcc 4.8, when the compiler used to build packages from the 2014Q4 pkgsrc repository is gcc 4.7.3.
As a consequence, these node v4.x.y binaries downloaded from nodejs.org have requirements on versions of the C++ runtime that are greater than the C++ runtime provided by, for instance, the platform C++ runtime and the C++ runtime used by most (all?) 2014Q4 pkgsrc packages.,
Thus, one would think that, to run these node binaries, installing the runtime corresponding to these more recent compilers would be needed.
However, as described in this document, it turns out that these binaries run fine on a system with only the platform C++ runtime and the C++ runtime packaged with gcc 4.7's libraries.
It seems that this behavior might be the sign of a bug in the runtime linker that could become a bigger problem if these binaries stop working accidentally at some point.
A node v4.2.2 binary downloaded from nodejs.org runs fine on a SmartOS VM using a 2014Q4 pkgsrc repository (the current LTS version) and a 14.4.2 image (also the current LTS version):
[root@dev ~/node-1]# uname -a
SunOS dev 5.11 joyent_20150219T102159Z i86pc i386 i86pc Solaris
[root@dev ~/node-1]# cat /etc/motd
__ . .
_| |_ | .-. . . .-. :--. |-
|_ _| ;| || |(.-' | | |
|__| `--' `-' `;-| `-' ' ' `-'
/ ; Instance (base-64-lts 14.4.2)
`-' https://docs.joyent.com/images/smartos/base
[root@dev ~/node-1]# cat /etc/pkgsrc_version
release: 2014Q4
architecture: x86_64
[root@dev ~/node-1]# ~/node-v4.2.2-sunos-x86/bin/node -e 'console.log(process.versions)'
{ http_parser: '2.5.0',
node: '4.2.2',
v8: '4.5.103.35',
uv: '1.7.5',
zlib: '1.2.8',
ares: '1.10.1-DEV',
icu: '56.1',
modules: '46',
openssl: '1.0.2d' }
[root@dev ~/node-1]#
However, ldd indicates that the node v4.2.2 binary cannot find a C++ runtime that
matches its requirements:
[root@dev ~/node-1]# ldd -v ~/node-v4.2.2-sunos-x86/bin/node | grep CXX
libstdc++.so.6 (GLIBCXX_3.4.15) => (version not found)
libstdc++.so.6 (CXXABI_1.3) => (version not found)
[root@dev ~/node-1]#
Running that binary with LD_DEBUG=all:
55472: file=libstdc++.so.6; needed by /root/node-v4.2.2-sunos-x86/bin/node
55472: find object=libstdc++.so.6; searching
55472: trying path=/opt/local/gcc48/i486-sun-solaris2.11/lib/./libstdc++.so.6
55472: trying path=/opt/local/gcc48/lib/./libstdc++.so.6
55472: trying path=/lib/libstdc++.so.6
55472: trying path=/usr/lib/libstdc++.so.6
55472: file=/usr/lib/libstdc++.so.6 [ ELF ]; generating link map
55472: libstdc++.so.6 GLIBCXX_3.4.15
55472: libstdc++.so.6 CXXABI_1.3
we find that it uses the C++ runtime at /usr/lib/libstdc++.so.6:
[root@dev ~/node-1]# ls -l /usr/lib/libstdc++.so.6
lrwxrwxrwx 1 root root 19 Feb 19 2015 /usr/lib/libstdc++.so.6 -> libstdc++.so.6.0.13
[root@dev ~/node-1]#
That runtime is at version 6.0.13, which corresponds, according to GCC's ABI
Policy and
Guidelines, to GCC
4.4.2 and ABI versions GLIBCXX_3.4.13 and CXXABI_1.3.3.
This runtime is clearly older than the runtime version required by node v4.2.2
(GLIBCXX_3.4.15), which corresponds according to that same document to GCC
4.6.0. It's not clear why, in these conditions, a v4.2.2 binary would run on
such a platform.
According to preliminary investigations by @rmustacc, it might be due to a bug
in the linker/loader. Running node v4.2.2 should actually fail with an error
message indicating that the requirement on version GLIBCXX_3.4.15 of the C++
runtime is not met.
The situation is different for node versions between v4.0.0 and v4.2.1. These versions have been built with g++ 4.9, whereas node v4.2.2 was built with g++ 4.8. What does it change in practice?
[root@dev ~/node-1]# ldd -v ~/node-v4.2.1-sunos-x86/bin/node | grep CXX
libstdc++.so.6 (GLIBCXX_3.4.20) => (version not found)
libstdc++.so.6 (CXXABI_1.3) => (version not found)
[root@dev ~/node-1]#
These node binaries depends on even newer versions of the C++ runtime:
GLIBCXX_3.4.20 instead of GLIBCXX_3.4.15`.
The reason for that is g++ 4.9 uses __throw_out_of_range_fmt when throwing
exceptions instead of __throw_out_of_range, and that adds a dependency on version 3.4.20 of the C++ runtime. Here's the output of nm on Linux
that demonstrates that:
jgilli@ubuntu:~/dev/node$ nm ~/node-4.x-g++-4.9 | grep 3.4.20 | c++filt
00000000013a4420 D TLSv1_enc_data
00000000013b4f20 D v8::internal::FLAG_turbo_osr
00000000013b4e20 D v8::internal::FLAG_log_instruction_period
U std::__throw_out_of_range_fmt(char const*, ...)@@GLIBCXX_3.4.20
00000000013c4120 b no_eventfd.9277
00000000013c4220 b process_title
00000000013a4220 d scsv.19260
jgilli@ubuntu:~/dev/node$ nm ~/node-4.x-g++-4.8 | c++filt | grep throw_out_of_range
U std::__throw_out_of_range(char const*)@@GLIBCXX_3.4
jgilli@ubuntu:~/dev/node$
So node SmartOS binaries at versions between v4.0.0 and v4.2.1 should also not
be able to run on a SmartOS 14.4.2 image and using a 2014Q4 pkgsrc repository,
as indicated by ldd's output (note the version not found messages):
[root@dev ~/node-1]# ldd -v ~/node-v4.2.1-sunos-x86/bin/node | grep CXX
libstdc++.so.6 (GLIBCXX_3.4.20) => (version not found)
libstdc++.so.6 (CXXABI_1.3) => (version not found)
[root@dev ~/node-1]#
However, they do run, probably for the same reason that node v4.2.2 runs: by accident and due to a bug in the linker/loaded:
[root@dev ~/node-1]# ~/node-v4.2.1-sunos-x86/bin/node -p 'process.versions'
{ http_parser: '2.5.0',
node: '4.2.1',
v8: '4.5.103.35',
uv: '1.7.5',
zlib: '1.2.8',
ares: '1.10.1-DEV',
icu: '56.1',
modules: '46',
openssl: '1.0.2d' }
[root@dev ~/node-1]#
This section describes some of the potential next steps we could take to determine exactly what's going on, and to eventually fix the problem.
It's not clear yet what the linker/loader bug to which this document alludes is. It would be great if we could investigate it further to come up with a definitive answer. Who would have some time to do that, or help me investigate that issue in the next couple weeks?
If it turns out to be an actual bug in the runtime linker, fixing that bug could cause more harm than good, as it would probably make node v4.x binaries that do run today on some SmartOS platforms not run anymore.
Not fixing that bug might trigger some subtle bugs because node binaries are built with the assumption that they run with a more recent C++ runtime.
In any case, it seems that we may want to let users for node v4.x.y run node with the proper C++ runtime on their SmartOS platforms. Following are a few potential solutions for doing that.
Versions of node packaged in SmartOS pkgsrc repositories do not have that C++
runtime dependency problem as they depend on the appropriate specific C++
runtime versions. So for instance, the nodejs-v4.2.1 package includes a node
binary built by g++ 4.8 and it depends on a package provinding g++ 4.8's
runtime itself. As a result, all dependencies and requirement are met:
[root@dev ~/node-1]# pkgin install nodejs-4.2.1
calculating dependencies... done.
nothing to upgrade.
3 packages to be installed (0B to download, 94M to install):
gcc48-libs-4.8.3nb3 nodejs-4.2.1
proceed ? [Y/n]
downloading packages...
installing packages...
installing gcc48-libs-4.8.3nb3...
installing nodejs-4.2.1...
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...
updating database: 100%
marking nodejs-4.2.1 as non auto-removable
[root@dev ~/node-1]# node --version
v4.2.1
[root@dev ~/node-1]# which node
/opt/local/bin/node
[root@dev ~/node-1]# ldd `which node`
libz.so.1 => /opt/local/lib/libz.so.1
libkstat.so.1 => /lib/64/libkstat.so.1
libumem.so.1 => /lib/64/libumem.so.1
libsocket.so.1 => /lib/64/libsocket.so.1
libnsl.so.1 => /lib/64/libnsl.so.1
librt.so.1 => /lib/64/librt.so.1
libsendfile.so.1 => /lib/64/libsendfile.so.1
libstdc++.so.6 => /opt/local/gcc48/x86_64-sun-solaris2.11/lib/amd64/libstdc++.so.6
libm.so.2 => /lib/64/libm.so.2
libgcc_s.so.1 => /opt/local/gcc48/x86_64-sun-solaris2.11/lib/amd64/libgcc_s.so.1
libpthread.so.1 => /lib/64/libpthread.so.1
libc.so.1 => /lib/64/libc.so.1
libgcc_s.so.1 => /opt/local/gcc47/x86_64-sun-solaris2.11/lib/amd64/libgcc_s.so.1
libmp.so.2 => /lib/64/libmp.so.2
libmd.so.1 => /lib/64/libmd.so.1
[root@dev ~/node-1]#
As we can see from ldd's output, the C++ runtime used by that node binary is
at /opt/local/gcc48/x86_64-sun-solaris2.11/lib/amd64/libstdc++.so.6, which
is not the default C++ runtime that comes with the platform or the dependency
of packages found in the 2014Q4 (LTS) pkgsrc repository.
Removing SmartOS standalone binaries from nodejs.org would have the benefit of requiring SmartOS users to always use packages from pkgsrc to get their binaries, and guarantee that the dependencies on the C++ runtime are met.
Another potential solution could be to build node v4.x.y with g++ 4.7, because g++ 4.7 is the default compiler version for the current LTS pkgsrc repository. However, it doesn't seem to be an avenue that the Node.js project is likely to consider since:
- V8 explicitly states that it requires g++ 4.8 and later to build.
- Some previous experiments to build recent versions of node with g++ 4.7 failed.
So it seems that the only possible approach to solve that problem is to provide a C++ runtime that is compatible with g++ 4.8' runtime on SmartOS machines.
Currently, the platform C++ runtime is at version 6.0.13 (which corresponds to GCC 4.4.2), and it seems most packages have a dependency on gcc 4.7.3's runtime, so that means any binary built with gcc 4.7.3 would also find a proper runtime.
I'm wondering if it would make sense to provide more recent runtimes either in images, or in the platform's /usr/lib.
The way that the Node.js project currently solves the problem of providing binaries built with recent compilers that run on older Linux systems is by using the RedHat Developer Toolset. It's a toolchain that bundles newer features of the C/C++ runtime in the binaries themselves so that these binaries can run on systems with older runtimes. See http://www.redhat.com/summit/2012/pdf/2012-DevDay-How-To-Toolset-Newsome.pdf from slide 26 for further details.
It seems that getting something similar to that to work would be fairly complex.