Fix evil-winrm error on macOS M1: "Error: An error of type OpenSSL::Digest::DigestError happened, message is Digest initialization failed: initialization error"

openssl3_no_md4.txt

# @author Pichaya Morimoto (p.morimoto@sth.sh)

# gem install evil-winrm
# evil-winrm -u "${user}" -p "${pass}"  -i "${ip}"

Evil-WinRM shell v3.4

Info: Establishing connection to remote endpoint

Error: An error of type OpenSSL::Digest::DigestError happened, message is Digest initialization failed: initialization error

Error: Exiting with code 1

# Root cause:
OpenSSL 3.0 has retired a number of algorithms including MD4 function, which was used in evil-winrm.

In Linux, we can configure the file /etc/ssl/openssl.cnf.

```bash
[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1
```

However, this seems not working with our case. Maybe because macOS/Ruby is using LibreSSL by default.

# Solution:
So, we will compile ruby with the old openssl 1.1 instead.
I intentionally select the ruby version 3.1.2 to match with the Kali Linux at the time of writing this.

brew install rbenv ruby-build openssl@1.1
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(rbenv init -)"' >> ~/.zshrc
source ~/.zshrc

LDFLAGS="-L/opt/homebrew/opt/capstone/lib" CPPFLAGS="-I/opt/homebrew/opt/capstone/include" RUBY_CONFIGURE_OPTS="--with-openssl-dir=$(brew --prefix openssl@1.1)" rbenv install 3.1.2

rbenv global 3.1.2
gem install evil-winrm

# and it works !
evil-winrm -u "${user}" -p "${pass}"  -i "${ip}"

Evil-WinRM shell v3.4

Info: Establishing connection to remote endpoint

*Evil-WinRM* PS C:\Users\user\Documents>



Note:

I also tested with RUBY_CONFIGURE_OPTS="--with-openssl-dir=$(brew --prefix openssl@3)"
in order to allow the MD4 algorithm in the /opt/homebrew/etc/openssl@3/openssl.cnf file
along with getting the new openssl (1.1 > 3) but it failed to build :(

commenting here for when i'm looking again for a solution

1. /etc/ssl/openssl.cnf is for the openssl bundled with macOS, not for the one installed with homebrew
2. openssl version -d to get the current openssl directory
3. edit that file (in case of openssl@3 "/opt/homebrew/etc/openssl@3" is returned by the command above) by adding the legacy support
4. enjoy ✌️