Skip to content

Instantly share code, notes, and snippets.

@gubatron
Last active August 29, 2015 14:06
Show Gist options
  • Save gubatron/e6c10f984965b6bd412a to your computer and use it in GitHub Desktop.
Save gubatron/e6c10f984965b6bd412a to your computer and use it in GitHub Desktop.
Help building libtorrent python bindings on MacOSX

Just compiled libtorrent 1.0.2 for development purposes.

$ ./configure --enable-tests --enable-examples --enable-python-binding --enable-debug

$ make

$ sudo make install

python bindings are there, but when I try to import libtorrent or run the simple_client.py, I get the following error:

$ python simple_client.py 
Traceback (most recent call last):
  File "simple_client.py", line 7, in <module>
    import libtorrent as lt
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/libtorrent.so, 2): Symbol not found: __ZNKSt5ctypeIcE13_M_widen_initEv
  Referenced from: /usr/local/lib/libtorrent-rasterbar.8.dylib
  Expected in: /usr/lib/libstdc++.6.dylib
 in /usr/local/lib/libtorrent-rasterbar.8.dylib

I've seen a similar error in Linux and it was solved by simply updating the LD_LIBRARY_PATH to where the symbols were, like this:

export LD_LIBRARY_PATH=/usr/local/lib

This doesn't seem to be working on MacOSX, even though the libraries are in the same /usr/local/lib folder:

$ ls -l /usr/local/lib | grep torrent
-rwxr-xr-x   1 root      admin   7962564 Sep 11 10:36 libtorrent-rasterbar.8.dylib
-rw-r--r--   1 root      admin  97019960 Sep 11 10:36 libtorrent-rasterbar.a
lrwxr-xr-x   1 root      admin        28 Sep 11 10:36 libtorrent-rasterbar.dylib -> libtorrent-rasterbar.8.dylib
-rwxr-xr-x   1 root      admin      1132 Sep 11 10:36 libtorrent-rasterbar.la

those are the libraries I just built, so why can't python find them is the question you must ask.

If you look at the error again closely

ImportError: dlopen(/usr/local/lib/python2.7/site-packages/libtorrent.so, 2)

python says is failed to open that 'libtorrent.so' library which lives in its site-packages directory /usr/local/lib/python2.7/site-packages/libtorrent.so

Let's look at that file.

$ ls -l '/usr/local/lib/python2.7/site-packages/libtorrent.so'
-rwxr-xr-x  1 root  admin  5216208 Sep 11 03:07 /usr/local/lib/python2.7/site-packages/libtorrent.so

it looks like it was compiled around 3am when I left the computer building libtorrent before going to bed. the files above have a different date, and it is because I've reissued the 'sudo make install' command again after telling homebrew to remove previous versions of libtorrent (thinking that was the issue)

maybe I should re-build the python bindings again, to make sure everything has been built under the same conditions.

So I go inside the libtorrent 'bindings/python' folder and issue a make command:

$ make
/usr/local/bin/python setup.py build
running build
running build_ext

and then a 'sudo make install' command

$ sudo make install
/usr/local/bin/python setup.py build
running build
running build_ext
/usr/local/bin/python setup.py install --prefix=/usr/local
running install
running build
running build_ext
running install_lib
copying build/lib.macosx-10.9-x86_64-2.7/libtorrent.so -> /usr/local/lib/python2.7/site-packages
running install_egg_info
Removing /usr/local/lib/python2.7/site-packages/python_libtorrent-1.0.2-py2.7.egg-info
Writing /usr/local/lib/python2.7/site-packages/python_libtorrent-1.0.2-py2.7.egg-info
make[1]: Nothing to be done for `install-data-am'.

let's look at that file again now, see if the date changed:

$ ls -l '/usr/local/lib/python2.7/site-packages/libtorrent.so'
-rwxr-xr-x  1 root  admin  5216208 Sep 11 03:07 /usr/local/lib/python2.7/site-packages/libtorrent.so

it looks unchanged... let's remove the bindings, maybe 'sudo make clean' will do it

$ sudo make clean
rm -rf .libs _libs
/usr/local/bin/python setup.py clean --all
running clean
removing 'build/temp.macosx-10.9-x86_64-2.7' (and everything under it)
removing 'build/lib.macosx-10.9-x86_64-2.7' (and everything under it)
'build/bdist.macosx-10.9-x86_64' does not exist -- can't clean it
'build/scripts-2.7' does not exist -- can't clean it
removing 'build'
rm -f *.lo

is the file still there?

$ ls -l '/usr/local/lib/python2.7/site-packages/libtorrent.so'
-rwxr-xr-x  1 root  admin  5216208 Sep 11 03:07 /usr/local/lib/python2.7/site-packages/libtorrent.so

hmm, that didn't delete it.

let's make clean the whole of libtorrent and see

$ sudo make clean
Making clean in include/libtorrent
rm -rf .libs _libs
rm -f *.lo
Making clean in src
test -z "libtorrent-rasterbar.la" || rm -f libtorrent-rasterbar.la
rm -f ./so_locations
rm -rf .libs _libs
rm -rf ../ed25519/src/.libs ../ed25519/src/_libs
rm -rf kademlia/.libs kademlia/_libs
rm -f *.o
rm -f ../ed25519/src/*.o
rm -f ../ed25519/src/*.lo
rm -f kademlia/*.o
rm -f kademlia/*.lo
rm -f *.lo
Making clean in examples
 rm -f client_test dump_torrent make_torrent simple_client rss_reader upnp_test connection_tester
rm -rf .libs _libs
rm -f *.o
rm -f *.lo
Making clean in test
 rm -f test_bitfield test_file test_file_storage test_privacy test_auto_unchoke test_bandwidth_limiter test_bdecode_performance test_bencoding test_buffer test_checking test_fast_extension test_hasher test_http_connection test_ip_filter test_dht test_lsd test_metadata_extension test_pe_crypto test_peer_priority test_pex test_piece_picker test_xml test_string test_primitives test_http_parser test_magnet test_packet_buffer test_read_piece test_rss test_ssl test_storage test_swarm test_threads test_torrent test_torrent_parse test_tracker test_trackers_extension test_transfer test_upnp enum_if test_utp test_session test_web_seed test_url_seed test_remap_files test_gzip test_utf8 test_socket_io test_sliding_average
rm -rf .libs _libs
test -z "libtest.la" || rm -f libtest.la
rm -f ./so_locations
rm -f *.o
test -z "test_bitfield.log test_file.log test_file_storage.log test_privacy.log test_auto_unchoke.log test_bandwidth_limiter.log test_bdecode_performance.log test_bencoding.log test_buffer.log test_checking.log test_fast_extension.log test_hasher.log test_http_connection.log test_ip_filter.log test_dht.log test_lsd.log test_metadata_extension.log test_pe_crypto.log test_peer_priority.log test_pex.log test_piece_picker.log test_xml.log test_string.log test_primitives.log test_http_parser.log test_magnet.log test_packet_buffer.log test_read_piece.log test_rss.log test_ssl.log test_storage.log test_swarm.log test_threads.log test_torrent.log test_torrent_parse.log test_tracker.log test_trackers_extension.log test_transfer.log test_upnp.log enum_if.log test_utp.log test_session.log test_web_seed.log test_url_seed.log test_remap_files.log test_gzip.log test_utf8.log test_socket_io.log test_sliding_average.log" || rm -f test_bitfield.log test_file.log test_file_storage.log test_privacy.log test_auto_unchoke.log test_bandwidth_limiter.log test_bdecode_performance.log test_bencoding.log test_buffer.log test_checking.log test_fast_extension.log test_hasher.log test_http_connection.log test_ip_filter.log test_dht.log test_lsd.log test_metadata_extension.log test_pe_crypto.log test_peer_priority.log test_pex.log test_piece_picker.log test_xml.log test_string.log test_primitives.log test_http_parser.log test_magnet.log test_packet_buffer.log test_read_piece.log test_rss.log test_ssl.log test_storage.log test_swarm.log test_threads.log test_torrent.log test_torrent_parse.log test_tracker.log test_trackers_extension.log test_transfer.log test_upnp.log enum_if.log test_utp.log test_session.log test_web_seed.log test_url_seed.log test_remap_files.log test_gzip.log test_utf8.log test_socket_io.log test_sliding_average.log
test -z "test_bitfield.trs test_file.trs test_file_storage.trs test_privacy.trs test_auto_unchoke.trs test_bandwidth_limiter.trs test_bdecode_performance.trs test_bencoding.trs test_buffer.trs test_checking.trs test_fast_extension.trs test_hasher.trs test_http_connection.trs test_ip_filter.trs test_dht.trs test_lsd.trs test_metadata_extension.trs test_pe_crypto.trs test_peer_priority.trs test_pex.trs test_piece_picker.trs test_xml.trs test_string.trs test_primitives.trs test_http_parser.trs test_magnet.trs test_packet_buffer.trs test_read_piece.trs test_rss.trs test_ssl.trs test_storage.trs test_swarm.trs test_threads.trs test_torrent.trs test_torrent_parse.trs test_tracker.trs test_trackers_extension.trs test_transfer.trs test_upnp.trs enum_if.trs test_utp.trs test_session.trs test_web_seed.trs test_url_seed.trs test_remap_files.trs test_gzip.trs test_utf8.trs test_socket_io.trs test_sliding_average.trs" || rm -f test_bitfield.trs test_file.trs test_file_storage.trs test_privacy.trs test_auto_unchoke.trs test_bandwidth_limiter.trs test_bdecode_performance.trs test_bencoding.trs test_buffer.trs test_checking.trs test_fast_extension.trs test_hasher.trs test_http_connection.trs test_ip_filter.trs test_dht.trs test_lsd.trs test_metadata_extension.trs test_pe_crypto.trs test_peer_priority.trs test_pex.trs test_piece_picker.trs test_xml.trs test_string.trs test_primitives.trs test_http_parser.trs test_magnet.trs test_packet_buffer.trs test_read_piece.trs test_rss.trs test_ssl.trs test_storage.trs test_swarm.trs test_threads.trs test_torrent.trs test_torrent_parse.trs test_tracker.trs test_trackers_extension.trs test_transfer.trs test_upnp.trs enum_if.trs test_utp.trs test_session.trs test_web_seed.trs test_url_seed.trs test_remap_files.trs test_gzip.trs test_utf8.trs test_socket_io.trs test_sliding_average.trs
test -z "test-suite.log" || rm -f test-suite.log
rm -f *.lo
Making clean in bindings
Making clean in python
rm -rf .libs _libs
/usr/local/bin/python setup.py clean --all
running clean
'build/lib.macosx-10.9-x86_64-2.7' does not exist -- can't clean it
'build/bdist.macosx-10.9-x86_64' does not exist -- can't clean it
'build/scripts-2.7' does not exist -- can't clean it
rm -f *.lo
rm -rf .libs _libs
rm -f *.lo
Making clean in tools
 rm -f parse_hash_fails parse_request_log
rm -rf .libs _libs
rm -f *.o
rm -f *.lo
rm -rf .libs _libs
rm -f *.lo

and guess what... the file was still there... so I just removed it manually (sudo rm ...), and now I'm going to rebuild everything from scratch again #patience

So I've just built everything again, I've witnessed the creation of the libtorrent.so python binding library during the build output

copying build/lib.macosx-10.9-x86_64-2.7/libtorrent.so -> /usr/local/lib/python2.7/site-packages

let's see all the libraries created:

$ ls -l /usr/local/lib/python2.7/site-packages/libtorrent.so 
-rwxr-xr-x  1 root  admin  5216208 Sep 11 15:15 /usr/local/lib/python2.7/site-packages/libtorrent.so

$ ls -l /usr/local/lib/libtorrent-rasterbar.*
-rwxr-xr-x  1 root  admin   7962564 Sep 11 16:36 /usr/local/lib/libtorrent-rasterbar.8.dylib
-rw-r--r--  1 root  admin  97019960 Sep 11 16:36 /usr/local/lib/libtorrent-rasterbar.a
lrwxr-xr-x  1 root  admin        28 Sep 11 16:36 /usr/local/lib/libtorrent-rasterbar.dylib -> libtorrent-rasterbar.8.dylib
-rwxr-xr-x  1 root  admin      1132 Sep 11 16:36 /usr/local/lib/libtorrent-rasterbar.la

the python binding was created over an hour earlier, let's see if it works now:

$ python 
Python 2.7.8 (default, Aug 24 2014, 21:26:19) 
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import libtorrent
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/libtorrent.so, 2): Symbol not found: __ZNKSt5ctypeIcE13_M_widen_initEv
  Referenced from: /usr/local/lib/libtorrent-rasterbar.8.dylib
  Expected in: /usr/lib/libstdc++.6.dylib
 in /usr/local/lib/libtorrent-rasterbar.8.dylib
>>>

FML... same exact error message.

Help.

@gubatron
Copy link
Author

ok, turns out that the boost libraries that come with homebrew didn't have Boost.Python, now I'm trying this and rebuilding:

brew reinstall boost --with-python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment