flibitijibibo/flibitPackaging.md

## flibitPackaging.md

      
    Raw
  

              flibitPackaging.md
            
          
    A week ago I was CC'd in on a thread about Linux packaging, and how to avoid doing it the wrong way (i.e. RPM, Deb, etc.). I've always used MojoSetup and I've never forced distributions to do any additional work, but this is still a new concept to a lot of people. Additionally, Amos suggested that I expand on Itch's FNA appendix, so here's a guide on how I package my games.
This is a bit of an expansion on my MAGFest 2016 presentation, which you can find here:
http://www.flibitijibibo.com/magfest2016/
https://www.youtube.com/watch?v=B83CWUh0Log
I would recommend looking at that first! After that, read on...
On any operating system you should always assume one thing: Absolutely none of your game's dependencies will be on the system except for the C runtime and a graphics driver (hopefully it's a new one, but that's another story). For whatever reason this surprises a lot of people, particularly Windows developers that already do this on Windows! This may be the result of people discovering things like apt and dnf, where Linux users can grab software via repositories and not have to pull in a giant blob of things from the web whenever they do clean installs, or what have you.
The first mistake you can make is thinking that you need to integrate the package manager into your game.
The software you grab from a repository is NOT the same thing as what you grab from a website! These repositories fulfill a very important role, and it actually has very little to do with convenience. They're actually doing something much more important: ensuring compatibility across the entire system and everything that might run on it. The way this is done is by self-compiling everything:
https://bodhi.fedoraproject.org/
So when Fedora updates Mono, for example...
https://fedoraproject.org/wiki/Changes/Mono_4
... they're not grabbing a blob from the Mono project website, they're pulling the source in and building against Fedora's system libraries, which were also self-built, and then building everything that depends on Mono against the new version, all with an ungodly amount of checks, tests, and all sorts of things that make build engineers sweat with joy.
Your game is not doing any of that, and it's certainly not doing it for every single distribution. This is likely because A: your game is proprietary, B: you probably just want to type make once and be done with it like an Earth-dwelling being, and C: you don't even have a grasp of just how many distributions are out there, and that's just limiting it to the ones that exist today that will be running your game.
Sounds like an insane ecosystem to support, but guess what: Just treat Linux deps like you do Windows deps and all of this goes away! With this in mind, let's look at an example.
Here's the dependency tree for Super Hexagon, generated by ldd:
[flibitijibibo@flibitTower data]$ ldd x86_64/superhexagon.x86_64 
	linux-vdso.so.1 (0x00007ffd30363000)
	libGL.so.1 => /lib64/libGL.so.1 (0x00007f434a094000)
	libGLU.so.1 => /lib64/libGLU.so.1 (0x00007f4349e23000)
	libSDL2-2.0.so.0 => x86_64/libSDL2-2.0.so.0 (0x00007f4349b00000)
	libGLEW.so.1.6 => x86_64/libGLEW.so.1.6 (0x00000031f7e00000)
	libopenal.so.1 => x86_64/libopenal.so.1 (0x00007f4349855000)
	libvorbisfile.so.3 => x86_64/libvorbisfile.so.3 (0x0000003d47200000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f43494d3000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f43491d0000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f4348fb9000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4348d9c000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f43489da000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f43487d6000)
	libGLX.so.0 => /lib64/libGLX.so.0 (0x00007f43485a5000)
	libGLdispatch.so.0 => /lib64/libGLdispatch.so.0 (0x00007f43482bb000)
	libX11.so.6 => /lib64/libX11.so.6 (0x00007f4347f7b000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f4347d73000)
	libvorbis.so.0 => /usr/lib64/libvorbis.so.0 (0x00007f4347b45000)
	libogg.so.0 => /usr/lib64/libogg.so.0 (0x00007f434793e000)
	/lib64/ld-linux-x86-64.so.2 (0x00005651c1fe0000)
	libXext.so.6 => /lib64/libXext.so.6 (0x00007f434772b000)
	libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f4347509000)
	libXau.so.6 => /lib64/libXau.so.6 (0x00007f4347304000)

This might seem daunting at first, but that's because you didn't build this game. I did, and I know exactly what I linked to. The link line looks like this:
g++ -O2 -pthread (SECRETS) -o x86_64/superhexagon.x86_64 (STATICLIBS) -lGL -lGLU -lSDL2 -lGLEW -lopenal -lvorbisfile

Here's what we care about:
-lGL -lGLU -lSDL2 -lGLEW -lopenal -lvorbisfile

-lGL and -lGLU are pretty self-explanatory; obviously you don't need to ship OpenGL drivers, so you can skip those outright. But SDL2, GLEW, OpenAL, and Vorbisfile are definitely DLLs you would ship on Windows (well, maybe you statically link GLEW, but you get the idea).
But what's all that other crap up there? Even after trimming down our stuff, we're left with this:
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f43494d3000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f43491d0000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f4348fb9000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4348d9c000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f43489da000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f43487d6000)
	libGLX.so.0 => /lib64/libGLX.so.0 (0x00007f43485a5000)
	libGLdispatch.so.0 => /lib64/libGLdispatch.so.0 (0x00007f43482bb000)
	libX11.so.6 => /lib64/libX11.so.6 (0x00007f4347f7b000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f4347d73000)
	libvorbis.so.0 => /usr/lib64/libvorbis.so.0 (0x00007f4347b45000)
	libogg.so.0 => /usr/lib64/libogg.so.0 (0x00007f434793e000)
	/lib64/ld-linux-x86-64.so.2 (0x00005651c1fe0000)
	libXext.so.6 => /lib64/libXext.so.6 (0x00007f434772b000)
	libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f4347509000)
	libXau.so.6 => /lib64/libXau.so.6 (0x00007f4347304000)

That's still a lot of junk, and you may not know what you need to ship...
The easy way to check is to just ldd your own dependencies and see where it leads. I'll give you a quick hint and say you can immediately skip the C/C++ runtime and friends, AKA these guys:

libc
libm
libdl
librt
libgcc_s
libstdc++(*)
libpthread
Those weird Linux objects (linux-vdso, ld-linux, etc.)

Already we're down to this:
	libGLX.so.0 => /lib64/libGLX.so.0 (0x00007f43485a5000)
	libGLdispatch.so.0 => /lib64/libGLdispatch.so.0 (0x00007f43482bb000)
	libX11.so.6 => /lib64/libX11.so.6 (0x00007f4347f7b000)
	libvorbis.so.0 => /usr/lib64/libvorbis.so.0 (0x00007f4347b45000)
	libogg.so.0 => /usr/lib64/libogg.so.0 (0x00007f434793e000)
	libXext.so.6 => /lib64/libXext.so.6 (0x00007f434772b000)
	libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f4347509000)
	libXau.so.6 => /lib64/libXau.so.6 (0x00007f4347304000)

We've not even looked at our dependencies yet and already it's looking good! But let's start checking those out. Here's libGL and libGLU:
[flibitijibibo@flibitTower x86_64]$ ldd /usr/lib64/libGL.so.1
	linux-vdso.so.1 (0x00007ffc8a6c7000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f095108b000)
	libGLX.so.0 => /lib64/libGLX.so.0 (0x00007f0950e59000)
	libGLdispatch.so.0 => /lib64/libGLdispatch.so.0 (0x00007f0950b70000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f09507af000)
	/lib64/ld-linux-x86-64.so.2 (0x000055ca83bf3000)
	libX11.so.6 => /lib64/libX11.so.6 (0x00007f095046e000)
	libXext.so.6 => /lib64/libXext.so.6 (0x00007f095025c000)
	libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f095003a000)
	libXau.so.6 => /lib64/libXau.so.6 (0x00007f094fe35000)
[flibitijibibo@flibitTower x86_64]$ ldd /usr/lib64/libGLU.so.1
	linux-vdso.so.1 (0x00007fff1f18d000)
	libGL.so.1 => /lib64/libGL.so.1 (0x00007ff4433d4000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007ff443051000)
	libm.so.6 => /lib64/libm.so.6 (0x00007ff442d4f000)
	libc.so.6 => /lib64/libc.so.6 (0x00007ff44298e000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff442776000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007ff442572000)
	libGLX.so.0 => /lib64/libGLX.so.0 (0x00007ff442341000)
	libGLdispatch.so.0 => /lib64/libGLdispatch.so.0 (0x00007ff442057000)
	/lib64/ld-linux-x86-64.so.2 (0x00005581f1329000)
	libX11.so.6 => /lib64/libX11.so.6 (0x00007ff441d17000)
	libXext.so.6 => /lib64/libXext.so.6 (0x00007ff441b05000)
	libxcb.so.1 => /lib64/libxcb.so.1 (0x00007ff4418e2000)
	libXau.so.6 => /lib64/libXau.so.6 (0x00007ff4416de000)

Right away you see libGLX, libGLdispatch, and all those libX* libs. These were needed for the driver so those can be assumed as available, and thus we can narrow it down further:
	libvorbis.so.0 => /usr/lib64/libvorbis.so.0 (0x00007f4347b45000)
	libogg.so.0 => /usr/lib64/libogg.so.0 (0x00007f434793e000)

All that's left is Ogg and Vorbis, and I'll hazard a guess that you know where these guys came from:
[flibitijibibo@flibitTower x86_64]$ ldd libvorbisfile.so.3 
	linux-vdso.so.1 (0x00007ffd2b34b000)
	libvorbis.so.0 => /usr/lib64/libvorbis.so.0 (0x00007f007c5d2000)
	libogg.so.0 => /usr/lib64/libogg.so.0 (0x00007f007c3ca000)
	libc.so.6 => /usr/lib64/libc.so.6 (0x00007f007c009000)
	libm.so.6 => /usr/lib64/libm.so.6 (0x00007f007bd07000)
	/lib64/ld-linux-x86-64.so.2 (0x000056162ecab000)

So along with Vorbisfile we need to ship Ogg and Vorbis too. Big surprise, right? Basically anything that's not a standard runtime lib and not something that was needed by the driver needs to go into your game. There are very few exceptions to this rule - for example, I'm pretty sure a Linux machine without zlib is a work of fiction at this point. But libpng, which uses zlib, should probably be included with your game due to the various versions of that lib (1.2, 1.5, etc.).
At the end of the day, that giant ldd we started with actually looks like this in the official shipping version of Super Hexagon:
[flibitijibibo@flibitTower data]$ ls x86_64/
libGLEW.so.1.6
libogg.so.0
libopenal.so.1
libSDL2-2.0.so.0
libvorbisfile.so.3
libvorbis.so.0
superhexagon.x86_64

This all might seem complicated, but keep in mind that this is only as complicated as your dependency tree is. If you use libraries that are also conscious of this you'll be fine, but not everyone is like this. So unfortunately, all of you using stuff like SFML or even Chromium are in for a real treat:
[flibitijibibo@flibitTower lib64]$ ldd libcef.so 
	linux-vdso.so.1 (0x00007ffc1fd84000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f4ff5ade000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f4ff58d9000)
	libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f4ff56d5000)
	libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f4ff5483000)
	libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f4ff5149000)
	libfontconfig.so.1 => /lib64/libfontconfig.so.1 (0x00007f4ff4f05000)
	libfreetype.so.6 => /lib64/libfreetype.so.6 (0x00007f4ff4c5b000)
	libpangocairo-1.0.so.0 => /lib64/libpangocairo-1.0.so.0 (0x00007f4ff4a4d000)
	libcairo.so.2 => /lib64/libcairo.so.2 (0x00007f4ff4724000)
	libpango-1.0.so.0 => /lib64/libpango-1.0.so.0 (0x00007f4ff44d9000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f4ff41d6000)
	libX11.so.6 => /lib64/libX11.so.6 (0x00007f4ff3e96000)
	libXi.so.6 => /lib64/libXi.so.6 (0x00007f4ff3c86000)
	libXcomposite.so.1 => /lib64/libXcomposite.so.1 (0x00007f4ff3a82000)
	libXext.so.6 => /lib64/libXext.so.6 (0x00007f4ff3870000)
	libnss3.so => /lib64/libnss3.so (0x00007f4ff3547000)
	libnssutil3.so => /lib64/libnssutil3.so (0x00007f4ff3318000)
	libsmime3.so => /lib64/libsmime3.so (0x00007f4ff30f1000)
	libnspr4.so => /lib64/libnspr4.so (0x00007f4ff2eb2000)
	libasound.so.2 => /lib64/libasound.so.2 (0x00007f4ff2bba000)
	libXdamage.so.1 => /lib64/libXdamage.so.1 (0x00007f4ff29b7000)
	libXfixes.so.3 => /lib64/libXfixes.so.3 (0x00007f4ff27b1000)
	libXtst.so.6 => /lib64/libXtst.so.6 (0x00007f4ff25aa000)
	libgconf-2.so.4 => /lib64/libgconf-2.so.4 (0x00007f4ff2378000)
	libatk-1.0.so.0 => /lib64/libatk-1.0.so.0 (0x00007f4ff2152000)
	libXcursor.so.1 => /lib64/libXcursor.so.1 (0x00007f4ff1f46000)
	libXrender.so.1 => /lib64/libXrender.so.1 (0x00007f4ff1d3b000)
	libXss.so.1 => /lib64/libXss.so.1 (0x00007f4ff1b37000)
	libXrandr.so.2 => /lib64/libXrandr.so.2 (0x00007f4ff192b000)
	libexpat.so.1 => /lib64/libexpat.so.1 (0x00007f4ff1701000)
	libdbus-1.so.3 => /lib64/libdbus-1.so.3 (0x00007f4ff14b1000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4ff1293000)
	libcups.so.2 => /lib64/libcups.so.2 (0x00007f4ff100a000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f4ff0c88000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f4ff0a70000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f4ff06af000)
	/lib64/ld-linux-x86-64.so.2 (0x000055b337c66000)
	libffi.so.6 => /lib64/libffi.so.6 (0x00007f4ff04a7000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f4ff0290000)
	libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f4ff0080000)
	libpng16.so.16 => /lib64/libpng16.so.16 (0x00007f4fefe4d000)
	libpangoft2-1.0.so.0 => /lib64/libpangoft2-1.0.so.0 (0x00007f4fefc36000)
	libthai.so.0 => /lib64/libthai.so.0 (0x00007f4fefa2d000)
	libgthread-2.0.so.0 => /lib64/libgthread-2.0.so.0 (0x00007f4fef82b000)
	libharfbuzz.so.0 => /lib64/libharfbuzz.so.0 (0x00007f4fef5c8000)
	libpixman-1.so.0 => /lib64/libpixman-1.so.0 (0x00007f4fef321000)
	libEGL.so.1 => /lib64/libEGL.so.1 (0x00007f4fef11d000)
	libxcb-shm.so.0 => /lib64/libxcb-shm.so.0 (0x00007f4feef18000)
	libxcb-render.so.0 => /lib64/libxcb-render.so.0 (0x00007f4feed0e000)
	libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f4feeaec000)
	libGL.so.1 => /lib64/libGL.so.1 (0x00007f4fee85c000)
	libplc4.so => /lib64/libplc4.so (0x00007f4fee657000)
	libplds4.so => /lib64/libplds4.so (0x00007f4fee452000)
	libdbus-glib-1.so.2 => /lib64/libdbus-glib-1.so.2 (0x00007f4fee227000)
	libsystemd.so.0 => /lib64/libsystemd.so.0 (0x00007f4fee19e000)
	libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f4fedf50000)
	libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f4fedc68000)
	libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f4feda35000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f4fed831000)
	libgnutls.so.30 => /lib64/libgnutls.so.30 (0x00007f4fed4fd000)
	libavahi-common.so.3 => /lib64/libavahi-common.so.3 (0x00007f4fed2ef000)
	libavahi-client.so.3 => /lib64/libavahi-client.so.3 (0x00007f4fed0de000)
	libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f4fecea8000)
	libdatrie.so.1 => /lib64/libdatrie.so.1 (0x00007f4fecc9f000)
	libgraphite2.so.3 => /lib64/libgraphite2.so.3 (0x00007f4feca73000)
	libGLdispatch.so.0 => /lib64/libGLdispatch.so.0 (0x00007f4fec789000)
	libXau.so.6 => /lib64/libXau.so.6 (0x00007f4fec585000)
	libGLX.so.0 => /lib64/libGLX.so.0 (0x00007f4fec354000)
	libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007f4febfd3000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f4febdb0000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f4febb95000)
	liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f4feb96e000)
	libgcrypt.so.20 => /lib64/libgcrypt.so.20 (0x00007f4feb689000)
	libgpg-error.so.0 => /lib64/libgpg-error.so.0 (0x00007f4feb475000)
	libdw.so.1 => /lib64/libdw.so.1 (0x00007f4feb22a000)
	libcap.so.2 => /lib64/libcap.so.2 (0x00007f4feb025000)
	libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f4feae16000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f4feac11000)
	libp11-kit.so.0 => /lib64/libp11-kit.so.0 (0x00007f4fea9ab000)
	libidn.so.11 => /lib64/libidn.so.11 (0x00007f4fea776000)
	libtasn1.so.6 => /lib64/libtasn1.so.6 (0x00007f4fea563000)
	libnettle.so.6 => /lib64/libnettle.so.6 (0x00007f4fea32c000)
	libhogweed.so.4 => /lib64/libhogweed.so.4 (0x00007f4fea0ff000)
	libgmp.so.10 => /lib64/libgmp.so.10 (0x00007f4fe9e86000)
	libfreebl3.so => /lib64/libfreebl3.so (0x00007f4fe9c83000)
	libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f4fe9a12000)
	libelf.so.1 => /lib64/libelf.so.1 (0x00007f4fe97fa000)
	libattr.so.1 => /lib64/libattr.so.1 (0x00007f4fe95f3000)

If you really hated reading this and want to avoid doing anything like this at all costs, your solution is to simply calm down on the dependency tree. Use OS portability libs like SDL which only depend on the C runtime (or in the case of Windows, not even that). Use SDL_GL_GetProcAddress instead of linking to libGL directly (for that matter, don't use GLU or GLEW). Use stb_image or LodePNG instead of linking to libjpeg, libpng, etc. Use something like stb_vorbis instead of libvorbisfile. Don't use GTK/Qt/wx at all. You can even find libs like miniz which can replace seemingly tiny dependencies like zlib! These are really tiny things you can do to get your dependency tree looking sparkling clean without having to do any real work at all, and it makes your game super portable as well.
Now that we know what files we need to ship, where do we put them? Is it just like throwing DLLs next to the EXE?
Linux library paths are a little bit strange, but this can work to our advantage. Odds are you're shipping both 32-bit and 64-bit, and you probably don't want to make 2 packages just for each architecture. RPM/Deb might force you to do this, but we're better than that. We can bundle 32/64-bit together at once simply by using an executable script instead of directly launching the executable binary. It'll look something like this:
GameName/
	- GameName <- This is our script!
	- Content.blob
	- x86/
		- gamename.x86
		- libSDL2-2.0.so.0
	- x86_64/
		- gamename.x86_64
		- libSDL2-2.0.so.0

And the script will look something like this:
https://gist.github.com/flibitijibibo/5365145
And that's it! We move to the location of the game, determine which architecture to pick, and add the real executable folder to the library search path before launching the binary.
There is only one thing you must be careful about: If you're doing most of your deployment from Windows, BE SURE that you did not save this script with Windows line endings! Windows is fantastically stupid and uses '\r\n' as a new line instead of just '\n', and this will rightfully confuse the system when trying to read the script. Very often I get reports from FNA developers that miss this and as a result, none of their players can play the game.
From here you're totally done putting the game together; this would be exactly what goes into your SteamPipe build folder, and you could throw this into a .tar.gz or .tar.bz2 file and be done with it. But we want an installer, right?
Finally we can talk about MojoSetup, which is probably a MUCH smaller part of packaging than you thought. Here's how it works: Grab this archive here:
http://www.flibitijibibo.com/flibitMojoSetup.tar.bz2
Extract it somewhere, navigate to the location in a terminal, and run this:
./build.sh examplegame
Congratulations, you just made a MojoSetup package. Seriously, that's it. Run ./examplegame-installer and check it out!
What you're looking at is a self-extracting zipfile (try it: unzip examplegame-installer) with both graphical and command-line interfaces, which allow users to install wherever they have permission to write while NEVER touching the system's package database, and it will automatically generate local desktop icons and an uninstaller that never leaves a footprint, while also prominently showing them the README right at the beginning of the install process.
And you probably wanted to ship a .deb file. Ha ha.
Getting this to work with your game is as trivial as just ripping off the examplegame/ folder. Simply edit the files for the needs of your game's title, description, etc. and copy your game folder into data/, and your packaging environment is complete.
The two files that are notably important are the example FEZ.bmp and Linux.README. For that particular bmp, you can use PNG if you want; it just needs to be large enough to work as a high-resolution desktop icon (which you should already have a VERY large version of if you ship for the Mac already). The Linux.README can have whatever you like inside, but that's the format that I've always used for my games.
From there, just upload the package and users can download your game, installing or unzipping as they please, and you'll never have to fight the distro's package manager to get users started. If you ship as often as I do, this matters a LOT. More importantly, being able to keep the build environment static is important too - having to do anything more than copying data and running ./build.sh becomes a HUGE pain when you have a large library of games to maintain over a period of many MANY years (and counting). Do whatever you can to keep your packaging process small!
MojoSetup's been a standard for myself and for many long-time Linux game developers - its roots go back to the late 1990s and Loki Software! I hope that it will be your standard as well.
(*) - At one point it was a tradition to ship with libstdc++ due to horrible ABI incompatibilities, but this appears to have gotten less horrible over the last few years, and some distributors like Valve will actually recommend NOT shipping with it.