Skip to content

Instantly share code, notes, and snippets.

@heetbeet
Last active September 4, 2019 10:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save heetbeet/53b5f4e0b7d93af2455cbaa4bb0e4e2b to your computer and use it in GitHub Desktop.
Save heetbeet/53b5f4e0b7d93af2455cbaa4bb0e4e2b to your computer and use it in GitHub Desktop.
Emdw restructure

Emdw code restructure

Johan du Preez and I are in the process of restructuring emdw with the following aims:

  1. easier integration with external libraries and projects, less reinventing the wheel
  2. more use of Boost (less reinventing the wheel) https://github.com/boostorg https://www.boost.org/
  3. moving our tests from PatrecII to Google Test https://github.com/google/googletest
  4. moving our linear algebra and matrix libraries from gLinear and PatrecII to xtensor https://github.com/QuantStack/xtensor
  5. removing gLinear and PatrecII as external libraries
  6. easier housekeeping and modularisation
  7. separating core components from work-in-progress
  8. support for clang and cling https://github.com/QuantStack/xeus-cling
  9. better Windows osX support

Code changes

We use CMake's new FetchContent to directly retrieve external dependencies upon the first cmake call. Furthermore we include watered down versions of gLinear and PatrecII directly, with the idea to slowly transition away from them. We also use the new CMake format to allow for easier nested directory structures. Our current directory structure is something like this:

file structure

Upgrading your code

- WARNING - first try on a cleanly downloaded emdw repo!

We are going to upgrade the main emdw brances to this new structure. In order for you to effectively merge and pull using these branches, you must restructure your code to fit ours.

Upgrading CMake

The CMake version must be 3.13 or higher, which is unfortunately not yet available on apt-get (Aug 2019). To quickly get CMake > 3.13 up and running, I recommend you to:

  1. make sure apt-get hasn't caught up yet (this might save you some hassle),
  2. otherwise download the cmake-3.15.2-Linux-x86_64.tar.gz binaries from https://cmake.org/download/ ,
  3. extract the folder that's in the .tar.gz to somewhere like /path/to/cmake-3.14.6-Linux-x86_64, and
  4. add the binaries to your system PATH by adding this line to ~/.bashrc:
export PATH=/path/to/cmake-3.14.6-Linux-x86_64/bin:$PATH
  1. (you can apply these changes without restarting your terminal window with: source ~/.bashrc.)

If you are using an IDE with an older build-in cmake (like CLion 2017), a simple hack is to replace your IDE's cmake binaries with this download.

Patching your project

To restructure your emdw layout, we provide the following upgrade helper emdw-restructure-patch-routine.py. Just run it under Python 3 and enter your emdw location in the prompt. The helper will take care of the following:

  1. move your emdw ./src files into their new subdirectories (without touching your file content)
  2. inject all CMakeLists.txt files into their correct location
  3. replace the ./cmake dir with a new ./cmake dir
  4. inject the ./src/dep-patreclite and ./src/dep-glinear libraries
  5. safely replace the variable FAIL to PATREC_FAIL within the whole code base (it conflicts with googletest)
  6. replace the usage #include "gLinear/foo.hpp" with #include "foo.hpp" within the whole code base
  7. comment out *.txt in .gitignore to make sure git picks up all the new CMakeLists.txt files

You should now be able to run and compile tstemdw as usual (note that the compiled bin will be located under the .\build\src\bin directory)

cd build           #go to build
cmake ..           #run cmake
make tstemdw -j8   #compile tstemdw
src/bin/tstemdw    #run tstemdw

If CMake throws an error, it might be necessary to first delete and recreate (mkdir) the ./build directory.

Finding a new home for your files

The files not listed by emdw-restructure-patch-routine.py will still reside in ./src. These are probably your files and will need to be moved into the subdirectories of your choice. You may use these pre-defined subdirectories if you files belong there, or create new ones. For work in progress, I would suggest using ./src/wip-your-features-name.

You will also need to add to CMakeList.txt to accomodate your files. To get an idea of how the new CMake structure work, take a look at what is inside the other CMakeLists.txt files, specifically ./src/CMakeLists.txt, ./src/emdw-core/CMakeLists.txt and ./src/emdw-base/CMakeLists.txt.

Making sure these changes reflect in git

Since a lot has changed in the git repo, it is easy to forget to commit everything. A good sequence to ensure all your wanted changes are in a commit, I suggest you add:

git add -u
git add src
git add cmake

We need your help

With this new development there are numerous things that needs to be done. For instance, we have ported the emdw tests to Google Test and they work, but are still a bit sloppy and not yet merge ready. If you would like to help with any of the emdw restructuring, please let us know.

Kind Regards,
Simon Streicher (sfstreicher at gmail)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment