Last active
October 24, 2023 21:39
-
-
Save ChadJPetersen/e5215da9c00b20fabff05b4a0e43e385 to your computer and use it in GitHub Desktop.
After hours of slaving I finally got Xapian working on windows with a ruby binding (I use it for Redmine). I know this is far from the right way to do it, and is rather hacky. But it works! Also I found little starting points online when trying to compile Xapian on windows. So hoping this helps someone else a bit!
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#In Powershell. | |
# Install chocolatey https://chocolatey.org/install to be the software package manager. Right now the following line is the installer. But it would be good to go grab the new command from the site provided. | |
#Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1')) | |
choco install git ghostscript xpdf-utils gnuwin 7zip cyg-get msys2 graphviz activeperl | |
choco install ruby --version=2.6.5.1 | |
# This sets up ruby to be a dev kit | |
ridk install 1 | |
cyg-get install catdoc unrtf unzip antiword xlsx2csv libvisio-tools | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# SWITCH YOUR CONSOLE!!!!!! | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# In MSYS2 console | |
pacman -Syu | |
# close and reopen if prompted | |
pacman -Su | |
# Close and reopen if prompted | |
# if getting errors on the below you can try the --force option. | |
pacman -Syuu | |
pacman -S base-devel --needed | |
pacman -S mingw-w64-x86_64-toolchain zlib file pcre mingw64/mingw-w64-x86_64-file mingw64/mingw-w64-x86_64-pcre mingw-w64-x86_64-tesseract-ocr mingw-w64-x86_64-tesseract-data-eng git doxygen tcl mingw-w64-x86_64-graphviz mingw-w64-x86_64-python-docutils python3-pip python3-pygments | |
pacman -R libtool | |
pacman -S mingw64/mingw-w64-x86_64-libtool mingw-w64-x86_64-cmake gcc mingw64/mingw-w64-x86_64-zlib msys/zlib-devel msys/libuuid-devel pcre-devel | |
pip3 install --upgrade pip | |
pip install sphinx docutils | |
pip3 install sphinx | |
# i686 versions of pacman packages are for a 32 bit build. | |
# pacman -S mingw-w64-i686-toolchain mingw64/mingw-w64-i686-libtool mingw32/mingw-w64-i686-file mingw32/mingw-w64-i686-pcre mingw-w64-i686-cmake mingw32/mingw-w64-i686-zlib | |
git clone https://github.com/xapian/xapian.git /C/Tools/Xapian/src | |
cd /C/Tools/Xapian/src | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# SWITCH YOUR CONSOLE!!!!!! | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# After installing things, switch from msys2 console to admin MINGW64 | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# Go through all xapian projects and change /dev/null to NUL in file types: | |
# *.cc, *.h, *.cpp, *.c | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# BE SURE TO UPDATE PATH'S TO WHERE YOU WANT THEM!!! | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
./bootstrap | |
# make distclean # can be helpful if a previous build was here. | |
# You may also have to add the dot command to the path. This is part of graphviz and can be installed via chocolaty in powershell. | |
PATH=$PATH:/c/ProgramData/chocolatey/bin | |
# After running the next line you may get some issues about unexpected fi token. Open up the config file that was currently being configured and remove the empty else statement. | |
# -L/C/Tools/msys64/mingw32/lib | |
./configure --prefix=/C/Tools/Xapian --with-ruby RUBY="/C/Tools/ruby26/bin/ruby.exe" RDOC="/C/Tools/Ruby26/bin/rdoc.cmd" RUBY_INC="/C/Tools/Ruby26/include/ruby-2.6.0" RUBY_INC_ARCH="/C/Tools/Ruby26/include/ruby-2.6.0/x64-mingw32" CPPFLAGS="-D_FORTIFY_SOURCE=0" | |
# --enable-shared --disable-static | |
# LDFLAGS="-Wl,--no-undefined -Wl,-ldxerr9 " CFLAGS=-fPIC CXXFLAGS=-fPIC | |
# CC=x86_64-w64-mingw32-gcc CC_FOR_BUILD=x86_64-w64-mingw32-gcc CPP="x86_64-w64-mingw32-gcc -E" CXX=x86_64-w64-mingw32-g++ ac_ct_CC=x86_64-w64-mingw32-gcc ac_ct_CXX=x86_64-w64-mingw32-g++ CXXCPP="x86_64-w64-mingw32-g++ -E" | |
# You may need to add a file for mingw for the time strip function. | |
# you can do this by adding the line | |
# #define _XOPEN_SOURCE 700 | |
# before the includes in scriptindex.cc | |
# You may also need to delete whitespace before some multi line commands if you get errors like 'recipe commences before first target. Stop.' | |
# However I would suggest running make first before just changing things willy nilly. | |
# to supress unused parameter errors use (void)(<unused parameter name>); | |
make dist | |
# This you may need to run as admin | |
make install | |
cd /C/Tools/Xapian | |
mkdir ./xapian-ruby | |
mkdir ./xapian-ruby/ext | |
mkdir ./xapian-ruby/lib | |
mkdir ./xapian-ruby/ext/lib | |
mkdir ./xapian-ruby/ext/_xapian | |
mkdir ./xapian-ruby/ext/_xapian/lib | |
mkdir ./xapian-ruby/ext/_xapian/xapian | |
echo "require 'rake/extensiontask' | |
spec = Gem::Specification.load('xapian.gemspec') | |
Rake::ExtensionTask.new do |ext| | |
ext.name = '_xapian' # indicate the name of the extension. | |
ext.ext_dir = 'ext/_xapian' # search for 'hello_world' inside it. | |
# ext.lib_dir = 'lib/my_lib' # put binaries into this folder. | |
# ext.config_script = 'extconf.rb' # use instead of the default 'extconf.rb'. | |
ext.tmp_dir = 'tmp' # temporary folder used during compilation. | |
ext.source_pattern = \"*.{c,cpp,cc}\" # monitor file changes to allow simple rebuild. | |
# ext.config_options << 'CPPFLAGS=\"-I/C/Tools/Xapian/include\"' # supply additional options to configure script. | |
ext.gem_spec = spec # optionally indicate which gem specification will be used. | |
end" > ./xapian-ruby/Rakefile | |
echo 'Gem::Specification.new do |s| | |
s.name = "_xapian" | |
s.version = "1.4.13" | |
s.summary = "Xapian for Ruby" | |
s.author = "Edited by Chad Petersen" | |
s.files = Dir.glob("ext/**/*.{ c,rb}") + | |
Dir.glob("lib/**/*.rb") | |
s.extensions << "ext/_xapian/extconf.rb" | |
s.add_development_dependency "rake-compiler" | |
end' > ./xapian-ruby/xapian.gemspec | |
# you will need to update your version number | |
cp ./lib/libxapian.a ./xapian-ruby/ext/lib/libxapian.a | |
cp ./lib/libxapian.a ./xapian-ruby/ext/_xapian/lib/libxapian.a | |
echo "require \"mkmf\" | |
LIBDIR = RbConfig::CONFIG['libdir'] | |
INCLUDEDIR = RbConfig::CONFIG['includedir'] | |
HEADER_DIRS = [INCLUDEDIR] | |
# setup constant that is equal to that of the file path that holds that static libraries that will need to be compiled against | |
LIB_DIRS = [LIBDIR, File.expand_path(File.join(File.dirname(__FILE__), \"lib\"))] | |
# array of all libraries that the C extension should be compiled against | |
libs = ['-lxapian','-lz','-lrpcrt4','-lws2_32','-luuid','-lstdc++'] #if xapian changes required libraries this line will have to change. Double check dependency_libs in C:\Tools\xapian-bindings-1.4.7\ruby\.libs\_xapian.la that the needed libraries are on this line. | |
dir_config('_xapian', HEADER_DIRS, LIB_DIRS) | |
# iterate though the libs array, and append them to the \$LOCAL_LIBS array used for the makefile creation | |
libs.each do |lib| | |
\$LOCAL_LIBS << \"#{lib} \" | |
end | |
create_makefile('_xapian/_xapian') | |
" > ./xapian-ruby/ext/_xapian/extconf.rb | |
#more version updating | |
cp ./src/xapian-bindings/ruby/xapian_wrap.cc ./xapian-ruby/ext/_xapian/_xapian.cc | |
cp ./src/xapian-bindings/ruby/xapian_wrap.h ./xapian-ruby/ext/_xapian/_xapian.h | |
sed -i -e 's/xapian_wrap/_xapian/g' ./xapian-ruby/ext/_xapian/_xapian.cc | |
#more version updating | |
cp ./lib/libxapian.a ./xapian-ruby/ext/lib/libxapian.a | |
cp -a ./include/. ./xapian-ruby/ext/_xapian/ | |
cp ./src/xapian-bindings/ruby/xapian.rb ./xapian-ruby/lib/xapian.rb | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# !!!!!!Change shell to powershell!!!!!!!! | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
cd C:/Tools/Xapian/xapian-ruby | |
gem install rake-compiler | |
rake compile | |
cp ./lib/_xapian.so C:/Tools/ruby26/lib/ruby/site_ruby/2.6.0/_xapian.so | |
cp ./lib/xapian.rb C:/Tools/ruby26/lib/ruby/site_ruby/2.6.0/xapian.rb | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
# !!!!!!Change shell to mingw64!!!!!!!!!!! | |
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | |
cd /C/Tools/ruby26/lib/ruby/site_ruby/2.6.0 | |
echo "Make sure the following files exist in the ruby bin folder." | |
ldd * |
I was trying to compile xapian-core on msys2 and I always got "xapian configure: error: no 32 bit integer type found"
This part of the instructions solved my problem. But I'm not sure what the reason for the failure was.
# In MSYS2 console
pacman -Syu
# close and reopen if prompted
pacman -Su
# Close and reopen if prompted
# if getting errors on the below you can try the --force option.
pacman -Syuu
pacman -S base-devel --needed
pacman -S mingw-w64-x86_64-toolchain zlib file pcre mingw64/mingw-w64-x86_64-file mingw64/mingw-w64-x86_64-pcre mingw-w64-x86_64-tesseract-ocr mingw-w64-x86_64-tesseract-data-eng git doxygen tcl mingw-w64-x86_64-graphviz mingw-w64-x86_64-python-docutils python3-pip python3-pygments
pacman -R libtool
pacman -S mingw64/mingw-w64-x86_64-libtool mingw-w64-x86_64-cmake gcc mingw64/mingw-w64-x86_64-zlib msys/zlib-devel msys/libuuid-devel pcre-devel
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I managed to get the integration working with Delphi from the C# Integration. I called the Xapian.DLL and called stdcall extern CSharp* functions