Sonic Pi does an in-source build and on Windows uses two frameworks to assemble its dependencies: ruby/gem
and vcpkg
. Additionally three embedded applications, SuperCollider server & sc3-plugins, Erlang and Osmid need to be added to the package. Erlang and SuperCollider can be downloaded, osmid needs to be built using cmake. The Sonic Pi executable, using Qt, qscintilla and qwt for the editor, is built using qmake
and nmake
. Some of the deployment is handled by windeployqt
, proper deployment and installer creation is not finalized yet. The build description assumes a 64-bit build using Visual Studio 2017, everything is done on the command line though. As to versioning, simply use the latest of everything (VS2017, ruby 2.41-2, cmake 3.8.2, erlang 20, supercollider 3.8) and the C/C++ libraries as provided in the vcpkg precombiled download (qt5.8, aubio 0.46, qscintilla 2.10, qwt 6.13, boost 1.64)
Prior knowledge required: basic command of git, basic familiarity with Windows command line (open command line, traverse file system, execute commands, copy and move files, create and run batch-file), basic understanding of running build tools, a basic understanding of the function of search paths.
-
Download and install build tools: git, cmake, 7zip, Visual Studio 2017 (Visual C/C++ component and Windows SDK). You also need an editor like Notepad++ or Atom that supports unix line endings
-
Clone the Sounic-pi Source:
git clone https://github.com/samaaron/sonic-pi.git
Take a minute to get accustomed to the unusual file tree. The build is done inside this tree, the embedded applications are installed or copied into the tree as well. The two main folders are
sonic-pi\app\server\native\win
(placement of embedded apps) andsonic-pi\app\gui\qt
(build folder) -
Download Ruby (2.42 latest), SuperCollider 3.8 64-bit & sc3-plugins, Erlang
-
Clone and build osmid as described in the accompanying Readme.
-
Download the prebuilt vcpkg libraries. Contained are: qt5.8, qscintilla, qwt, aubio (using libsndfile and ffmpeg), boost 1.64. For SuperCollider there are also fftw3 and portaudio, but they are not used in the Sonic pi build. Qt5.8 comes with a lot of additional libraries (glib, zlib, icu and many more)
The folder in which everything is rooted is:
sonic-pi/app/server/native/win
The folder needs to be created. Copy the apps so that this file/folder structure results:
sonic-pi/app/server/native/win/ruby (all of ruby native goes here, the bin/include/lib folders go here)
sonic-pi/app/server/native/win/erlang (the bin/lib folders go here)
sonic-pi/app/server/native/win/osmid (place the two execs o2m and m2o here)
sonic-pi/app/server/native/win/scsynth.exe (the libs required by sc also go into the root folder)
sonic-pi/app/server/native/win/plugins (sc-core- and sc3-plugins)
From SuperCollder are required: scsynth.exe, fftw3f.dll, libsndfile.dll and the Windows runtime libraries (msvcp120.dll, msvcr120.dll, also contained in the SC application folder). All stock- and sc3-plugins go into the plugins folder.
If you build SuperCollider with VS2017 (undocumented, but possible), the list of files is: scsynth.exe, fftw3f.dll, libsndfile-1.dll, ogg.dll, flac.dll, vorbis.dll, vorbisenc.dll, concrt140.dll, msvcp140.dll.
Chose a "safe" location, without potentially problematic elements like spaces and non-ascii characters in the path, e.g. C:\vcpkg. If you expand to the root folder of the drive (C:) and rename the top level foldername (vcpkg-export-2017....7z) to vcpkg
the path statements given in the scripts below should work without adjustment.
If for some reason you want to build the vcpkg libraries yourself, note that building Qt5 takes quite some time (4-8 hours). Clone vcpkg from https://github.com/MicroSoft/vcpkg.git
and follow the instructions to bootstrap the system (basically just run bootstrap-vcpkg.bat
) Vcpkg controls build features like target architecture and library linkage (dynamic vs static) via so called triplets
(little cmake files contained in vcpkg\triplets). By default vcpkg builds 32-bit dynamic libraries. The corresponding triplet is called x86-windows
. As this build describes a 64-bit build, we need to append the triplet name x64-windows
to each "install" command. For qt5 run:
vcpkg install qt5:x64-windows
The whole set of libraries required for SP would be:
vcpkg install qt5:x64-windows qwt:x64-windows aubio:x64-windows boost:x64-windows
Note that aubio will automatically pull in libsndfile and ffmpeg. Libsndfile will in turn pull in ogg, flac and vorbis.
Qscintilla was not included in above command, because it needs special treatment. The SP-build requires qscintilla to be provided as static build. You would expect that to be done by running vcpkg install qscintilla:x64-windows-static
. But this will try to trigger a rebuild of all qscintilla dependencies (qt5 and its dependencies) as static libraries, which would not only take a long time, but would fail, because qt5 cannot be built as static libraries with vcpkg yet. The solution is to formulate an exception in the triplet-file used to define our build: vcpkg\triplets\x64-windows.cmake. Edit it and append the following lines:
if(PORT STREQUAL "qscintilla")
set(VCPKG_LIBRARY_LINKAGE static)
endif()
After this modification qscintilla can be built the "normal" way:
vcpkg install qscintilla:x64-windows
This looks like the command to build a dynamic library, but because of the exception it will be built statically and nevertheless get installed to the same target folder as the other (dynamic) libs.
While this is not ideal, the Sonic Pi build depends on certain executables to be available via the search path. These are ruby, gem, qmake, lrelease, nmake, and the C/C++-toolchain executables. Also some libraries must either be found in the path or pointed to by an environment variable. Toolchain access is best handled by running a MS provided batch file that sets up all required paths (vcvarsallXXX.dll). The example below assumes using a 64-bit host (Win64) to build a 64-bit version of SP. There are other vcvarsall-batch files that support other combinations of build host and target architecture. For the SP-specific search locations we must prepend path statements to the search path generated by vcvarsall. This also makes sure that only and exactly the executables are found that should be used in the build, but not similar executables that may also be installed on your systen. This is best done by running a batch-file when opening the command-line. This batch file (for unixers: script) will call the above mentioned MS-batch-file that sets the VS-required paths, and then add the paths specific to the SP build. The example below assumes that vcpkg is extracted to C:\vcpkg, VS2017 is installed the default way, and the Sonic Pi source resides in C:\Projects\sp\sonic-pi. The batch-file also sets an environment-variable AUBIO_LIB that points to the dynamic library aubio.dll in C:\vcpkg\installed\x64-windows\bin. Adding the ruby binary folder to the path should make sure that this particular ruby install is used rather than any other that might be available (note though that file-associations can override search path order, so make sure to prepend the execution of each .rb-file by the command ruby
. Thereby the system will search for ruby following the search-path order, and not look up a file assiciation):
echo off
set VCPKG_DIR=C:\vcpkg
set PROJECT_ROOT=C:\Projects\sp
echo Running vcvars64.bat to complete environment setup! Building 64-bit executable on 64-bit host.
call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat"
echo Using VS2017 64-bit compiler and default WinSDK.
echo Using libraries (including Qt) from vcpkg
echo Setting up search path...
set PATH=%VCPKG_DIR%\installed\x64-windows\bin;%VCPKG_DIR%\installed\x64-windows\tools\qt5;%PROJECT_ROOT%\sonic-pi\app\server\native\win\ruby\bin;%PATH%
set CMAKE_PREFIX_PATH=%VCPKG_DIR%\installed\x64-windows
set AUBIO_LIB=%VCPKG_DIR%\installed\x64-windows\bin\aubio.dll
echo Done!
cd /D %PROJECTS_DIR%\sp\sonic-pi\app\gui\qt
Note: if you have multiple rubies installed on your system, you have to make sure that the correct ruby-install is used in the following steps. If you only have one ruby install (and no other msys2 install), and the search path points to the bin folder of that ruby install, you can ignore this, you can run the command below from anywhere. If you have multiple installs, and especially if the file type .rb is registered to a different install from the one used in Sonic Pi, you have to take precautions. The ruby commands used are ruby
, gem
(for installing additional ruby libraries 'gems') and optionally ridk
(for installing the development kit). All three commands call executables contained in the bin
-folder of the ruby install:
<your prefix\>sonic-pi\app\server\native\win\ruby\bin
The safest is to prepend the commands with the absolute path:
<your prefix\>sonic-pi\app\server\native\win\ruby\bin\ruby
<your prefix\>sonic-pi\app\server\native\win\ruby\bin\gem
<your prefix\>sonic-pi\app\server\native\win\ruby\bin\ridk install
Another option is to use relative paths from your current location. If you are in the "build-folder" sonic-pi\app\gui\qt, then these path-expressions make sure that the correct executable is used:
..\..\server\native\win\ruby\bin\ruby
..\..\server\native\win\ruby\bin\gem
..\..\server\native\win\ruby\bin\ridk install
-
Install a fresh Ruby development-kit (msys2): if you installed ruby with the installer (making sure that it ends up in the correct "native\win\ruby" folder, you likely already installed the dk. If not, or if you just extracted the rubyinstaller archive, you need to run the script
ridk install
(remember the introductory note). Go through all three steps provided by the script one-by-one. (i.e. type 1 first and wait for the script to return, then 2 - return - 3 - done. It's quite a long process, a full C/C++ dev environment (msys2 and mingw-toolchain) is installed) -
Install the gems listed below (see note below, copy paste command)
gem install win32-process windows-pr fast_osc rugged
-
Remove duplicate gems from the generic gems folder in
sonic-pi\app\server\vendor
- rugged
- fast_osc
- ffi
- did_you_mean
- minitest
Note: there used to be a longer list of gems to be installed. Further research is required if this is still necessary:
gem install win32-process windows-pr fast_osc ffi hamster wavefile rubame kramdown rugged multi_json ruby-beautify memoist
The minimum of gems that need to be removed from the vendor folder is:
- rugged
- fast_osc
You will get build errors if they are not removed. Note that in case of duplicate gems (in vendor and native) the generic one in the vendor folder will be used. The native gems are installed to
sonic-pi\app\server\native\win\ruby\lib\ruby\gems\2.4.0\gems
-
Change to build directory
cd sonic-pi\app\gui\qt
-
Run i18n
ruby ..\..\server\bin\i18n-tool.rb -t
(remember the introductory remark. You might want to replaceruby
by<your prefix\>sonic-pi\app\server\native\win\ruby\bin\ruby
) -
Copy ruby_help template to ruby_help.h
copy ruby_help.tmpl ruby_help.h
-
Build docs
ruby ..\..\server\bin\qt-doc.rb -o ruby_help.h
Check that folder
info
and fileshelp_files.qrc
andinfo_files.qrc
have been created. You should not have gotten any warnings, otherwise something went wrong.
- Adjust win32-section in .\SonicPi.pro to current environment (not tested critically, adjust paths to your environment):
# Windows only (adjusted to use with vcpkg)
win32 {
LIBS += -lqscintilla2 -lqwt
QMAKE_CXXFLAGS += -IC:\vcpkg\installed\x64-windows\include -IC:\vcpkg\installed\x64-windows\include\qwt
QMAKE_LFLAGS += /LIBPATH:C:\vcpkg\installed\x64-windows\lib
DEFINES += _CRT_SECURE_NO_WARNINGS _WINSOCK_DEPRECATED_NO_WARNINGS
}
Note: likely boost is not used properly in the build. Shared memory access seems not to be working
- Copy folder
sonic-pi\etc\doc\images\tutorial
to.\images
(the existing dummy file does not work as link) - Run (qt5 tools folder must be in path! (see above)):.
lrelease SonicPi.pro
qmake -o Makefile SonicPi.pro
nmake
nmake install
cd release
windeployqt .\sonic-pi.exe -printsupport
Note: there is a batch-file win-build-app.bat
which will automate the steps from running i18n to windeployment. It will run successfully if the tutorial folder was copied to the images folder, the file SinocPi.pro adjusted to this build, and the foldername windows
replaced by win
.
- Continue using the search paths and AUBIO_LIB from the build environment (just continue using the batch file)
- Configure erlang_bin_path in
sonic-pi/app/server/sonicpi/lib/sonicpi/util.rb:329
), e.g. (just to avoid an error, this does not work yet 😄):
when :windows
erlang_bin_path = File.join(native_path, "erlang", "bin", "erl.exe")
"\"#{erlang_bin_path}\""
cd sonic-pi\app\gui\qt\release
.\sonic-pi.exe
Note: at this stage you cannot run Sonic Pi from the file-explorer. The file explorer uses the default Windows search part which does not contain the path to the vcpkg bin-folder. SP has to be run from the command line. Making SP startable from the file-explorer requires a proper install that has not been defined yet.
- copy qwt.dll to current folder (release)
- copy Qt5OpenGL.dll and several dlls (zlib1, pcre, harfbuzz, and more (test)) to current folder
- aubio.dll (and deps: libsndfile, ffmpeg, ogg, vorbis, flac - where does aubio.dll (and sibblings) go to work in the gem)
- Windows runtime libraries
- Qt platform plugins, dlls, plugins
Note: qscintilla2 is statically linked Note: for now runtime access to libraries is handled via search path
Done!
- Visual Studio 2017
- Windows SDK (corresponding, mind backwards compatibility, part of VS2017 install)
- Gem & dev environment (MSYS2, atuomatically acquired by ruby 2.41)
- VCPKG
- git
- cmake
- qmake, lrelease, windeployqt (comes with qt5)
- namke (comes with VS)
- 7z
- Ruby
- Erlang
- SuperCollider
If built rather than downloaded:
- supercollider: https://github.com/supercollider/supercollider.git
- sc3-plugins: https://github.com/supercollider/sc3-plugins.git
- Ruby: https://rubyinstaller.org/
- Erlang: https://www.erlang.org/downloads
- SuperCollider: http://supercollider.github.io/download
- sc3-plugins: http://supercollider.github.io/download
- vcpkg precompiled libs (temporarily, unofficial): https://github.com/bagong/dummy/releases
- Visual Studio Community: https://www.visualstudio.com/downloads/
- Cmake: https://cmake.org/download/
- Git for Windows: https://git-scm.com/download/win
- remove rugged-0.24.0 from sonic-pi/app/server/vendor
- remove fast_osc from sonic-pi/app/server/vendor
- do not remove "prerelease-aubio" from vendor folder, the more recent "native" version doesn't work yet
- only when building supercollider: place ASIO-SDK next to portaudio folder
- add special include for qwt in SonicPi.pro
- don't forget libsndfile, flac, vorbis, vorbisenc, ogg ffmpeg when deploying aubio
?
- FIX: WHY DOESN'T SOUND WORK YET? LIKELY SHARED MEMORY ACCESS DOESN'T WORK - boost - shared memory
- Configure Erlang use properly
- Follow up missing qtvirtualkeyboard plugin (required?)
- Fix non-working enter in editor
- Create installer
- use only native gems? (install all, or remove the ones from generic vendor which are installed natively?)
- or use as many generic gems as possible and only install the ones required as native (windows specific, fast_osc, rugged), and more?
- Erlang otp requires a ms runtime environment installed - that's a complication for the SP install...
Please give feedback if you've used this and got questions, problems, or solutions. I'll try to enhance this over time. Comment either below or enquire in the Gitter group sonic-pi.
Version 3? 😄