Problems compiling native gems under Snow Leopard + RVM

1. Problems compiling native gems under Snow Leopard + Homebrew

Hopefully somebody here can help with this, I've asked a few other places with no success.

I've got Snow Leopard running on an older macbook, with Homebrew.

My problem is that some native gems have problems compiling for the correct architecture when I use a compiled ruby (e.g. RVM) instead of the system ruby.

Under System ruby, it works (mysql_api.bundle can load):

$ sudo env ARCHFLAGS="-arch x86_64" gem install mysql -v 2.8.1 -- --with-mysql-config=/usr/local/bin/mysql_config
$ file /Library/Ruby/Gems/1.8/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle
/Library/Ruby/Gems/1.8/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle: Mach-O 64-bit bundle x86_64
$ ruby -e "require '/Library/Ruby/Gems/1.8/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle'"

Under RVM 1.8.7, it fails (wrong architecture when loading mysql_api.bundle):

$ rvm use 1.8.7@test1
info: Using ruby 1.8.7 p174 with gemset test1
$ env ARCHFLAGS="-arch x86_64" gem install mysql -v 2.8.1 -- --with-mysql-config=/usr/local/bin/mysql_config
$ file /Users/woolley/.rvm/gems/ruby-1.8.7-p174@test1/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle
/Users/woolley/.rvm/gems/ruby-1.8.7-p174@test1/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle: Mach-O 64-bit bundle x86_64
$ ruby -e "require '/Users/woolley/.rvm/gems/ruby-1.8.7-p174@test1/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle'"
/Users/woolley/.rvm/gems/ruby-1.8.7-p174@test1/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle: dlopen(/Users/woolley/.rvm/gems/ruby-1.8.7-p174@test1/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle, 9): no suitable image found.  Did find: (LoadError)
	/Users/woolley/.rvm/gems/ruby-1.8.7-p174@test1/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle: mach-o, but wrong architecture - /Users/woolley/.rvm/gems/ruby-1.8.7-p174@test1/gems/mysql-2.8.1/ext/mysql_api/mysql_api.bundle
	from -e:1


Here's the error from mkmf.log:

"gcc -o conftest -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.47/include/mysql   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common   conftest.c  -L. -L/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib -L.     -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib  -L/usr/local/Cellar/mysql/5.1.47/lib/mysql -lmysqlclient  -lz -lm  -lruby-static -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib  -L/usr/local/Cellar/mysql/5.1.47/lib/mysql -lmysqlclient -lz -lm  -ldl -lobjc  "
ld: warning: in /Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/libruby-static.a, file was built for unsupported file format which is not the architecture being linked (x86_64)
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { mysql_ssl_set(); return 0; }
/* end */


Looking at the 'file' architecture for other .bundle files is inconclusive.  Some are i386, some of x86_64, and the apple-provided ones are universal/multi architecture.

Looking at these two links, I'm not even sure which one my machine should be.  I can't boot into 64-bit kernel by holding 6+4 when I boot, which makes sense since my machine is not on the list in the second article:

http://www.macobserver.com/tmo/article/checking_32_or_64-bit_kernel_boot_mode_in_snow_leopard/
http://www.osnews.com/story/22009/Snow_Leopard_Seeds_Use_32bit_Kernel_Drivers_by_Default

I don't know enough to dig into the C compilation and debug what is wrong.  Any help is appreciated.

2._MySQL_Info

$ which mysql_config
/usr/local/bin/mysql_config

$ mysql -e 'status'
--------------
mysql  Ver 14.14 Distrib 5.1.47, for apple-darwin10.3.0 (i386) using readline 5.1


Why are some libraries under MySQL x86_64 arch, when mysql status reports i386 as the arch?

$ file /usr/local/Cellar/mysql/5.1.47/lib/mysql/libmysqlclient.dylib 
/usr/local/Cellar/mysql/5.1.47/lib/mysql/libmysqlclient.dylib: Mach-O 64-bit dynamically linked shared library x86_64

3._Ruby_Info

$ rvm system
$ file `which ruby`
/usr/bin/ruby: Mach-O universal binary with 3 architectures
/usr/bin/ruby (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/bin/ruby (for architecture i386):	Mach-O executable i386
/usr/bin/ruby (for architecture ppc7400):	Mach-O executable ppc


$ rvm 1.8.7@test1
$ file `which ruby`
/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/bin/ruby: Mach-O executable i386

4._various_architectures

$ file /Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0/readline.bundle 
/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0/readline.bundle: Mach-O bundle i386

$ file /Users/woolley/.rvm/rubies/ruby-1.8.7-p174/bin/ruby
/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/bin/ruby: Mach-O executable i386

$ file /usr/local/lib/mysql/libmysqlclient.dylib
/usr/local/lib/mysql/libmysqlclient.dylib: Mach-O 64-bit dynamically linked shared library x86_64

$ file /usr/local/bin/mysql
/usr/local/bin/mysql: Mach-O 64-bit executable x86_64

5._universal_mysql_binary_attempt

On the homebrew issue tracker ( http://github.com/mxcl/homebrew/issues/issue/1733/#comment_293680 ),
adamv suggested compiling a universal binary:

brew install --universal mysql

This worked, but now the mysql gem compilation fails altogether.  Here's the output:


mkmf.log of mysql gem attempting to compile against homebrew universal mysql install:
have_func: checking for mysql_ssl_set()... -------------------- yes

"gcc -o conftest -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common   conftest.c  -L. -L/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib -L.     -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -lruby-static -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -ldl -lobjc  "
conftest.c: In function ‘t’:
conftest.c:3: error: ‘mysql_ssl_set’ undeclared (first use in this function)
conftest.c:3: error: (Each undeclared identifier is reported only once
conftest.c:3: error: for each function it appears in.)
conftest.c: In function ‘t’:
conftest.c:3: error: ‘mysql_ssl_set’ undeclared (first use in this function)
conftest.c:3: error: (Each undeclared identifier is reported only once
conftest.c:3: error: for each function it appears in.)
lipo: can't figure out the architecture type of: /var/folders/SB/SB7Z8KLVFjqhYIoHhOGyLk+++TM/-Tmp-//cc5ko5Qh.out
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { void ((*volatile p)()); p = (void ((*)()))mysql_ssl_set; return 0; }
/* end */

"gcc -o conftest -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common   conftest.c  -L. -L/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib -L.     -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -lruby-static -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -ldl -lobjc  "
ld: warning: in /Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/libruby-static.a, file was built for unsupported file format which is not the architecture being linked (x86_64)
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { mysql_ssl_set(); return 0; }
/* end */

--------------------

have_func: checking for rb_str_set_len()... -------------------- no

"gcc -o conftest -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common   conftest.c  -L. -L/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib -L.     -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -lruby-static -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -ldl -lobjc  "
conftest.c: In function ‘t’:
conftest.c:3: error: ‘rb_str_set_len’ undeclared (first use in this function)
conftest.c:3: error: (Each undeclared identifier is reported only once
conftest.c:3: error: for each function it appears in.)
conftest.c: In function ‘t’:
conftest.c:3: error: ‘rb_str_set_len’ undeclared (first use in this function)
conftest.c:3: error: (Each undeclared identifier is reported only once
conftest.c:3: error: for each function it appears in.)
lipo: can't figure out the architecture type of: /var/folders/SB/SB7Z8KLVFjqhYIoHhOGyLk+++TM/-Tmp-//ccwOQ4Ld.out
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { void ((*volatile p)()); p = (void ((*)()))rb_str_set_len; return 0; }
/* end */

"gcc -o conftest -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common   conftest.c  -L. -L/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib -L.     -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -lruby-static -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -ldl -lobjc  "
ld: warning: in /Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/libruby-static.a, file was built for unsupported file format which is not the architecture being linked (x86_64)
Undefined symbols for architecture x86_64:
  "_rb_str_set_len", referenced from:
      _t in ccWfbYX7.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
lipo: can't open input file: /var/folders/SB/SB7Z8KLVFjqhYIoHhOGyLk+++TM/-Tmp-//ccclkbLw.out (No such file or directory)
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { rb_str_set_len(); return 0; }
/* end */

--------------------

have_func: checking for rb_thread_start_timer()... -------------------- no

"gcc -o conftest -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common   conftest.c  -L. -L/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib -L.     -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -lruby-static -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -ldl -lobjc  "
conftest.c: In function ‘t’:
conftest.c:3: error: ‘rb_thread_start_timer’ undeclared (first use in this function)
conftest.c:3: error: (Each undeclared identifier is reported only once
conftest.c:3: error: for each function it appears in.)
conftest.c: In function ‘t’:
conftest.c:3: error: ‘rb_thread_start_timer’ undeclared (first use in this function)
conftest.c:3: error: (Each undeclared identifier is reported only once
conftest.c:3: error: for each function it appears in.)
lipo: can't figure out the architecture type of: /var/folders/SB/SB7Z8KLVFjqhYIoHhOGyLk+++TM/-Tmp-//ccsuznHH.out
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { void ((*volatile p)()); p = (void ((*)()))rb_thread_start_timer; return 0; }
/* end */

"gcc -o conftest -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common   conftest.c  -L. -L/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib -L.     -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -lruby-static -L/usr/local/Cellar/readline/6.0/lib -L/usr/local/Cellar/readline/6.0/lib -arch i386 -arch x86_64  -L/usr/local/Cellar/mysql/5.1.48/lib/mysql -lmysqlclient -lz -lm  -ldl -lobjc  "
ld: warning: in /Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/libruby-static.a, file was built for unsupported file format which is not the architecture being linked (x86_64)
Undefined symbols for architecture x86_64:
  "_rb_thread_start_timer", referenced from:
      _t in ccYwp6PT.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
lipo: can't open input file: /var/folders/SB/SB7Z8KLVFjqhYIoHhOGyLk+++TM/-Tmp-//ccMOB6Ez.out (No such file or directory)
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { rb_thread_start_timer(); return 0; }
/* end */

--------------------

have_header: checking for mysql.h... -------------------- no

"gcc -E -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common    conftest.c -o conftest.i"
gcc-4.2: -E, -S, -save-temps and -M options are not allowed with multiple -arch flags
checked program was:
/* begin */
1: #include <mysql.h>
/* end */

--------------------

have_header: checking for mysql/mysql.h... -------------------- no

"gcc -E -I. -I/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/lib/ruby/1.8/i686-darwin9.6.0 -I.  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -I/usr/local/Cellar/mysql/5.1.48/include/mysql  -arch i386 -arch x86_64   -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT  -DDONT_DECLARE_CXA_PURE_VIRTUAL -g -O2 -pipe -fno-common    conftest.c -o conftest.i"
gcc-4.2: -E, -S, -save-temps and -M options are not allowed with multiple -arch flags
checked program was:
/* begin */
1: #include <mysql/mysql.h>
/* end */

--------------------

6._SOLVED

So, the root of the problem was that RVM/Ruby had incorrectly compiled for the i386 architecture.

No idea how this happened, when I removed and reinstalled Ruby 1.8.7-p174 via RVM with the defaults, 
it compiled for the x86_64 architecture, not i386.  Now it is correct:

$ rvm 1.8.7@test1
$ file `which ruby`
/Users/woolley/.rvm/rubies/ruby-1.8.7-p174/bin/ruby: Mach-O 64-bit executable x86_64

I usually put this line in my root .rvmrc file to always compile in x86_64

export rvm_archflags="-arch x86_64"