Skip to content

Instantly share code, notes, and snippets.

@starbelly
Last active November 18, 2021 14:11
Show Gist options
  • Save starbelly/e7c0604513e41d0a2af456aa7abcc8a2 to your computer and use it in GitHub Desktop.
Save starbelly/e7c0604513e41d0a2af456aa7abcc8a2 to your computer and use it in GitHub Desktop.
How to test fetching and publishing private packages with rebar3 and a local hexpm instance.

rebar3 vs hex_core vs rebar3_hex vs hexpm... let them fight.

Laziness is a virtue : Janky automated integration testing script thingy.

Huh?

Not unlike integration testing itself; the bash script below is brittle and makes some assumptions about your environment:

  • You have a postgres instance running and are able to connect to it using postgres:postgres on localhost:5432
  • You have erlang and elixir installed
  • You have bash, git, and all the other odds and ends required for some kind of development setup

The following script will:

  • Back up your ~/.config/rebar3/hex.config
  • Make the test dir rebar3_hex_qa
  • Download hexpm and set it up
  • Download rebar3 and build it from the pertinent branch
  • Create a user and an org
  • Start the hexpm app in the background
  • Create a rebar3 project to test publishing a private package
    • creates _checkouts for rebar3_hex and hex_core on the pertinent branches
    • runs rebar3 hex user auth
    • runs rebar3 hex publish
    • creates a rebar3 project to test fetching a private package
      • has private package created in the previous project as a dep
      • runs rebar3 compile

Cross your fingers, follow the prompts, and verify the results... if crossing your fingers worked.

Copy and paste the following script and perform chmod +x on it.

The script

#!/bin/bash
base_dir=$(pwd)
rebar="$base_dir/rebar_hex_qa/rebar3/rebar3"
mkdir -p rebar_hex_qa
cd rebar_hex_qa

user="hexfoo"
passwd="hexfoo1"
org="jellybeam"
email="${user}\@foo.bar"
backup_hex="~/.config/rebar3/hex.config-`date +%F-%H%M%S`"

echo "Backing up ~/.config/rebar3/hex.config to $backup_hex"
cp ~/.config/rebar3/hex.config ~/.config/rebar3/hex.config.bak

# Setup hexpm
git clone https://github.com/hexpm/hexpm.git --depth 1 &> /dev/null
cd hexpm
git pull
mix do deps.get, compile, ecto.drop, ecto.setup; cd assets; npm install; cd ..
EXFILE=$(cat <<EOF
Supervisor.terminate_child(Hexpm.Supervisor, Hexpm.Billing.Report)
{:ok, user} = Hexpm.Accounts.Users.add(%{username: "${user}", emails: [%{email: "${email}"}], password: "${passwd}"}, audit: {"user", "agent"})
Hexpm.Accounts.Users.verify_email(user.username, hd(user.emails).email, hd(user.emails).verification_key)
{:ok, org} = Hexpm.Accounts.Organizations.create(user, %{name: "jellybeam", billing_active: true}, audit: {user, "TEST"})
change = Ecto.Changeset.change(org, %{billing_active: true})
Hexpm.Repo.update!(change)
EOF
)
echo "$EXFILE" > setup.exs
echo -n "Starting hexpm app..."
mix phx.server setup.exs &> /dev/null &
echo $! > phoenix_pid_file
echo "ok"
cd ..

# Build rebar3
echo -n "Snagging rebar3..."
git clone https://github.com/erlang/rebar3.git &> /dev/null
echo "ok"
cd rebar3
git checkout fix-repo-url-for-custom-repos
./bootstrap
export PATH=$(pwd):$PATH
cd ..

# Publish test
echo ""
echo "Let's publish a private package..."
echo ""
$rebar new lib private_publish_package_test
cd private_publish_package_test
mkdir -p _checkouts
cd _checkouts
git clone https://github.com/starbelly/rebar3_hex.git &> /dev/null
git clone https://github.com/hexpm/hex_core.git &> /dev/null
cd ..
PUBFILE=$(cat <<EOF
{erl_opts, [debug_info]}.
{project_plugins, [rebar3_hex]}.
{hex, [{repos, [
              #{name => <<"hexpm:${org}">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
               <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false},
             #{name => <<"hexpm">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
               <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false}]}]}.
EOF
)
echo "$PUBFILE" > rebar.config
echo ""
echo "========================================================================="
echo ""
echo "Attempting to authenticate with local hexpm you will be prompted for a username and password."
echo ""
echo "Note your username is '$user' and your password is '$passwd'."
echo ""
echo "You can set the local password to anything"
$rebar hex user auth
$rebar hex publish
cd ..

# Fetch test
echo ""
echo "Let's try to download that private package..."
echo ""
$rebar new lib private_package_fetch_test
cd private_package_fetch_test
FETCH_FILE=$(cat <<EOF
{erl_opts, [debug_info]}.
{hex, [{repos, [
              #{name => <<"hexpm:${org}">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
               <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false},
             #{name => <<"hexpm">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
               <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false}]}]}.
EOF
)
echo "$FETCH_FILE" > rebar.config
$rebar compile
cd ..

echo ""
echo "========================================================================="
echo ""
echo "Login to http://localhost:4000 with $user / $passwd"
echo ""
echo "Verify the the following:"
echo ""
echo "    - The private package published does not show up on the home page"
echo ""
echo "    - Your user has no packages at http://localhost:4000/users/$user"
echo ""
echo "    - Three keys were created for your user at http://localhost:4000/dashboard/keys :"
echo "        * api key"
echo "        * api read key"
echo "        * orgs key"
echo ""
echo "    - Your org $org has one package at http://localhost:4000/dashboard/orgs/$org"
echo ""
echo "Press enter to stop hexpm app when you're done inspecting."
read

echo ""
echo "Goodbye and remember your hex.config prior to this test was backed up to $backup_hex"
echo ""
# Stop hexpm
kill -9 `cat hexpm/phoenix_pid_file`

Doing things the silly manual way

Setup a local instance of hexpm

  1. Ensure you have a postgres instance running

  2. git clone https://github.com/hexpm/hexpm.git; mix do deps.get, compile, ecto.setup; cd assets && npm install; mix phx.server

  3. Visit http://localhost:4000/signup and enter in some stuff and things

  4. Look back to your console for the verification link, copy/paste/click it Example:

    [debug] Sending email with Bamboo.LocalAdapter:
    %Bamboo.Email{assigns: %{email: "starbelly@eh.foo", key: "12baece30b3ce87033e6c41a5bbc03cb", username: "starbelly"},...
         You can do so by following <a href=\"http://localhost:4000/email/verify?username=
         email=starbelly%40eh.foo& key=12baece30b3ce87033e6c41a5bbc03cb\">this link</a> or by pasting this link in your web browser: http://localhost:4000/email/verify?username=starbelly&email=starbelly%40eh.foo&key=12baece30b3ce87033e6c41a5bbc03cb\n</p>\n\n<p>\n...     
    
  5. Log in and go to http://localhost:4000/dashboard/orgs and create an org

Test steps

  1. git clone https://github.com/erlang/rebar3.git; cd rebar3; ./bootstrap
  2. rebar3 local install (or just add pwd to the beginning of your $PATH)
  3. Create a new test project with rebar3 new lib private_publish_package_test
  4. cd private_publish_package_test mkdir _checkouts; cd _checkouts; git clone https://github.com/starbelly/rebar3_hex.git; git clone https://github.com/hexpm/hex_core.git; cd ..
  5. Construct a rebar.config like the following:
    {erl_opts, [debug_info]}.
    {project_plugins, [rebar3_hex]}.
    {hex, [{repos, [
                  #{name => <<"hexpm:myorg">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
                   <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false},
                 #{name => <<"hexpm">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
                   <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false}]}]}.
    
  6. run rebar3 hex user auth and ensure you authenticated with your local hexpm instance vs hex.pm by going to http://localhost:4000/dashboard/keys
  7. run rebar3 hex repo generate and ensure it created a key on your local hexpm instance vs hex.pm by going to http://localhost:4000/dashboard/keys
  8. run rebar3 hex repo auth -k <key_outputed_from_previous_step>
  9. run rebar3 hex publish and ensure this went to your org and not (local) hexpm public repository.
  10. Create a new project to test fetching the private package: rebar3 new lib private_package_fetch_test
  11. Construct a rebar.config like the following:
    {erl_opts, [debug_info]}.
    {project_plugins, [rebar3_hex]}.
    {deps, [private_publish_package_test]}.
    {hex, [{repos, [
               #{name => <<"hexpm:myorg">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
                <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false},
              #{name => <<"hexpm">>, api_url => <<"http://localhost:4000/api">>, repo_url =>
                <<"http://localhost:4000">>, repo_verify => false, repo_verify_origin => false}]}]}.
    
  12. rebar3 compile - it should fetch your private package. If it fails to fetch your private package:

Go to Jail!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment