Skip to content

Instantly share code, notes, and snippets.

@SupaMic

SupaMic/compiling.md

Last active May 24, 2023 22:36
Show Gist options
  • Save SupaMic/03504089005285ae020fe7e3ae92ad42 to your computer and use it in GitHub Desktop.
Save SupaMic/03504089005285ae020fe7e3ae92ad42 to your computer and use it in GitHub Desktop.
Download ZIP
Compiling NIFs on Windows
Raw
compiling.md

Compiling bcrypt_elixir or argon2_elixir on a Windows machine in 2022

I'm going outline the full process and what I can decipher is the reasoning behind each step of this process for successfully compiling the bcrypt_elixir or argon2_elixir dependencies on a Windows machine in the hopes that when inevitably Microsoft changes where and how they package build libraries, Elixir devs will have a better understanding when they try to troubleshoot.

The cause of all this headache is that bcrypt_elixir and argon2_elixir are actually just wrappers around C++ implementations of these password hashing algorithms implemented in NIFs (Native Implemented Functions). Some C++ NIF background [https://andrealeopardi.com/posts/using-c-from-elixir-with-nifs/] A peek at C code in the bcrypt_elixir repo [https://github.com/riverrun/bcrypt_elixir/tree/master/c_src]

Before I go ahead I want to implore the Elixir community to collaborate and build native implementations of these or similar password hashing algorithms because the following process is so problematic that we are excluding a huge swath of potential professional and young developers from participating in the ecosystem since now these are practically a requirement to use the Phoenix framework.

Multiple other native implementations exist that might be easier for some to reason about if they want to do the hard work of refactoring into Elixir but aren't C++ experts: Javascript [https://github.com/dcodeIO/bcrypt.js/] Golang [https://pkg.go.dev/golang.org/x/crypto/bcrypt]

Another promising algorithm I found to refactor was Pufferfish which was supposed to be an improvement on bcrypt (FYI: the b stands for Blowfish). https://github.com/epixoip/pufferfish

PROBLEM When we run mix deps.compile this error occurs:

    ==> bcrypt_elixir
    could not compile dependency :bcrypt_elixir, "mix compile" failed. You can recompile this dependency with "mix deps.compile bcrypt_elixir", update it with "mix deps.update bcrypt_elixir" or clean it with "mix deps.clean bcrypt_elixir" 
    ** (Mix) "nmake" not found in the path. If you have set the MAKE environment variable, please make sure it is correct.

According to this error we need nmake.exe which you may have learned by now is required (along with a few other things) to compile this package and get our Elixir app running locally.

On with the Windows Adventure! First I'm assuming you do not want to install Visual Studio because if you are not using it already, why would you want a 15GB application just to compile around 40kb of files maybe once a month.

Unfortunately this process will not save you any of that storage space because Microsoft...but we should be able to get the tools we need without Visual Studio trying to embed itself as your IDE of choice by setting all your file extension defaults to itself or frustratingly constantly asking you to update because ... Microsoft.

Instead of installing the latest version of Visual Studio, you'll want to install "Build Tools for Visual Studio 2022", currently the link to the file exists https://visualstudio.microsoft.com/downloads/ under the Tools for Visual Studio dropdown/accordian. I also found a more reliable link for all the past versions here https://my.visualstudio.com/Downloads?q=build%20tools

This should download vs_BuildTools.exe which is the Visual Studio Installer program and which allows you to download(~115MB) and install the Visual Studio Build Tools 2022. Great lets try again -> mix deps.compile bcrypt_elixir

Same error =(

If we do a search of the files installed(C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools), we'll see no nmake.exe was created.

We need to add more MS packages to get the files we need, so in the Visual Studio Installer (reopen vs_BuildTools.exe or use start menu) click the Modify button inline with Visual Studio Build Tools. Under Workloads tab you'll need to click on the "Universal Windows Platform Build Tools" (~12GB) and in the left optional panel a "Windows 10 SDK" (~2GB) and "C++ (v143) Universal Windows Platform tools" ~4GB. Select Modify and hope your internet connection is decent. ~35min to download and install at a solid 10MB/s

Now search for nmake.exe again and you'll find we have not only 1 but 8 versions located in folder combinations of 2 parents under Hostx64 & Hostx86 with child folders named arm, arm64, x64 and x86 so which folder to do we add to the Windows PATH variable to let it know where nmake.exe lives. From what I understand the Hosts are the compiling system and the child folders are the target system so since you are running locally hosted version of these files if you have a 64bit windows computer get the filepath (your version will be different): C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\Hostx64\x64

Now, just because you installed a Microsoft program on the Microsoft Operating System doesn't mean it knows you have so you'll need to add to your environment variables manually. In the control panel find the link to "Edit System Environment Variables", open the PATH variable in User and System listings and add the aformentiond path to the appropriate make program. (just the directory path not including nmake.exe)

Try again, same error =( Reboot!

Try again... Yay... NEW Error!

==> bcrypt_elixir

    Microsoft (R) Program Maintenance Utility Version 14.33.31629.0
    Copyright (C) Microsoft Corporation.  All rights reserved.

    del /Q /F priv
    erl -eval "io:format(\"~s~n\", [lists:concat([\"ERTS_INCLUDE_PATH=\", code:root_dir(), \"/erts-\", erlang:system_info(version), \"/include\"])])" -s init stop -noshell > Makefile.auto.win
    nmake /                   /F Makefile.win priv\bcrypt_nif.dll

    Microsoft (R) Program Maintenance Utility Version 14.33.31629.0
    Copyright (C) Microsoft Corporation.  All rights reserved.

    if NOT EXIST "priv" mkdir "priv"
            cl /O2 /EHsc /I"c_src" /I"c:/Program Files/erl-24.1.7/erts-12.1.5/include" /LD /MD /Fepriv\bcrypt_nif.dll  c_src\bcrypt_nif.c c_src\blowfish.c
    Microsoft (R) C/C++ Optimizing Compiler Version 19.33.31629 for x64
    Copyright (C) Microsoft Corporation.  All rights reserved.

    bcrypt_nif.c
    c_src\bcrypt_nif.c(52): fatal error C1083: Cannot open include file: 'stdio.h': No such file or directory
    blowfish.c
    c:\phx\acalog-api\deps\bcrypt_elixir\c_src\blf.h(37): fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory
    Generating Code...
    NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\Hostx64\x64\ccould not compile dependency :bcrypt_elixir, "mix compile" failed. You can recompile this dependency with "mix deps.compile bcrypt_elixir", update it with "mix deps.update bcrypt_elixir" or clean it with "mix deps.clean bcrypt_elixir"
    l.EXE"' : return code '0x2'
    Stop.
    NMAKE : fatal error U107** (Mix) Could not compile with "nmake" (exit status: 2).
    7One option is to install a recent version of
    :[Visual C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
    either manually or using [Chocolatey](https://chocolatey.org/) -
    `choco install VisualCppBuildTools`.
    '
    After installing Visual C++ Build Tools, look in the "Program Files (x86)"
    "directory and search for "Microsoft Visual Studio". Note down the full path
    Cof the folder with the highest version number. Open the "run" command and
    :type in the following command (make sure that the path and version number
    \are correct):

    P    cmd /K "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64

    This should open up a command prompt with the necessary environment variables
    set, and from which you will be able to run the "mix compile", "mix deps.compile",
    and "mix test" commands.

    Another option is to install the Linux compatiblity tools from [MSYS2](https://www.msys2.org/).

    After installation start the msys64 bit terminal from the start menu and install the
     C/C++ compiler toolchain. E.g.:

      pacman -S --noconfirm pacman-mirrors pkg-config
      pacman -S --noconfirm --needed base-devel autoconf automake make libtool git \
        mingw-w64-x86_64-toolchain mingw-w64-x86_64-openssl mingw-w64-x86_64-libtool
    
    This will give you a compilation suite nearly compatible with Unix' standard tools.
    
    es (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\Hostx64\x64\nmake.EXE"' : return code '0x2'
    Stop.

Now this is alot of gobbledigook and offers some other pathways for success (none of which I've been able to get working) but the indicator of our solution is the line that includes "Cannot open include file: 'stdio.h': No such file or directory".

Lets do a search for this file, surely its in the 20GB of files that contain the required C++ headers that are included during compile time.

Search for "stdio.h" in "C:\Program Files (x86)\Microsoft Visual Studio"... Nope...nothing

Do we have to download more packages? no, we already got the files we need with the Windows 10 SDK but it lives elsewhere on the system, namely in "C:\Program Files (x86)\Windows Kits\10". A search locates it in "C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt"

But now how do we tell the nmake.exe that the header files it needs are in this other folder...you can't but it knows if you access it in the correct way.

Close whatever command line tool you are using because it just aint gonna work for you.
Open the Visual Studio Installer and next to Visual Studio Build Tools click Launch, try "mix --version" and if you get response you can then navigate to your Elixir app using DOS commands (for example: cd c:\phx\my_app) then mix deps.compile bcrypt_elixir (OR mix compile if you have a more NIFs to nmake)

This is what success should look like...

==> bcrypt_elixir

Microsoft (R) Program Maintenance Utility Version 14.33.31629.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        del /Q /F priv
        erl -eval "io:format(\"~s~n\", [lists:concat([\"ERTS_INCLUDE_PATH=\", code:root_dir(), \"/er
ts-\", erlang:system_info(version), \"/include\"])])" -s init stop -noshell > Makefile.auto.win
        nmake /                   /F Makefile.win priv\bcrypt_nif.dll

Microsoft (R) Program Maintenance Utility Version 14.33.31629.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        if NOT EXIST "priv" mkdir "priv"
        cl /O2 /EHsc /I"c_src" /I"c:/Program Files/erl-24.1.7/erts-12.1.5/include" /LD /MD /Fepriv\b
crypt_nif.dll  c_src\bcrypt_nif.c c_src\blowfish.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.33.31629 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

bcrypt_nif.c
blowfish.c
Generating Code...
Microsoft (R) Incremental Linker Version 14.33.31629.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/dll
/implib:priv\bcrypt_nif.lib
/out:priv\bcrypt_nif.dll
bcrypt_nif.obj
blowfish.obj
   Creating library priv\bcrypt_nif.lib and object priv\bcrypt_nif.exp
Compiling 3 files (.ex)
Generated bcrypt_elixir app

Now you can close the Developer Command Prompt for VS 2022 and go back to your cmdline application of preference secure in the knowledge that now you can compile C++ NIFs when needed and Elixir will recognize that this package has been compiled into the build files so you won't need to do it again...for a while.

So nmake.exe is about 100kb, it probably uses an additional dll or two so add on another few hundred kb, stdio.h

@SupaMic
Copy link
Author

SupaMic commented May 24, 2023

This can still fail for some reason and the nif needs to be built with just the right x64 architecture.
For example in bcrypt compiliation even though you compiled from the Microsft build tool dozens of times as described here, and the dll's seem to compile as x64 architecture ok, but when you do some digging the the error from the erlang load nif function
:erlang.load_nif(path, 0)

{:error,
 {:load_failed,
  'Failed to load NIF library c:/Users/admin/project/_build/dev/lib/bcrypt_elixir/priv/bcrypt_nif: \'Unspecified error\''}}

# \deps\bcrypt_elixir\lib\bcrypt\base.ex
  defp load_nif do
    path = :filename.join(:code.priv_dir(:bcrypt_elixir), 'bcrypt_nif')
    :erlang.load_nif(path, 0)
    |> IO.inspect(label: ":erlang.load_nif(path, 0)") ## I added this line to see what is actually coming back from the BEAM
  end

Try running...

cmd /K "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" amd64

(remember to use your respective Visual Studio install year)

and then go to your Repo and run

mix deps.compile bcrypt_elixir --force

to rebuild it (this is also handy to add some IO.inspects into the dependency package and get the "Unspecified Error" which indicates most likely that nmake.exe used the wrong architecture for Erlang)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment