Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Parse a .env (dotenv) file directly using BASH
# Pass the env-vars to MYCOMMAND
eval $(egrep -v '^#' .env | xargs) MYCOMMAND
# … or ...
# Export the vars in .env into your shell:
export $(egrep -v '^#' .env | xargs)
@kowalk

This comment has been minimized.

Copy link

@kowalk kowalk commented Feb 16, 2018

If you want just set a single variable in your script just use code below:
MY_VAR=$(grep MY_VAR .env | xargs)
IFS='=' read -ra MY_VAR <<< "$MY_VAR"
MY_VAR=${MY_VAR[1]}

@rafaelbeckel

This comment has been minimized.

Copy link

@rafaelbeckel rafaelbeckel commented Mar 17, 2018

read_var() {
    VAR=$(grep $1 $2 | xargs)
    IFS="=" read -ra VAR <<< "$VAR"
    echo ${VAR[1]}
}

MY_VAR=$(read_var MY_VAR .env)
@desprit

This comment has been minimized.

Copy link

@desprit desprit commented Mar 26, 2018

MY_VAR=$(grep MY_VAR .env | xargs)
MY_VAR=${MY_VAR#*=}

dunno about possible drawbacks though

@pronebird

This comment has been minimized.

Copy link

@pronebird pronebird commented May 27, 2018

It won't work if you have spaces, i.e FOO="BAR BUZ" will be returned without quotes from xargs

@revolter

This comment has been minimized.

Copy link

@revolter revolter commented May 30, 2018

Use xargs -d '\n' on GNU systems or xargs -0 on BSD systems to handle values with spaces.

https://stackoverflow.com/a/32589977/865175

@amberdiehl

This comment has been minimized.

Copy link

@amberdiehl amberdiehl commented Aug 10, 2018

I have nothing technical to add; just saying thank you. This is exactly what I needed.

@dovidweisz

This comment has been minimized.

Copy link

@dovidweisz dovidweisz commented Aug 30, 2018

Here's a one-liner for getting one variable:

MY_VAR=$(grep MY_VAR .env | cut -d '=' -f2)

One issue i see with all these solutions is: what happens if i have two similar variables. I need to fix the Grep part to only match ^MY_VAR=. I'm just not that good with grep.

Also... i would never have thought about using grep here, so Thank You.

@duncanmcclean

This comment has been minimized.

Copy link

@duncanmcclean duncanmcclean commented Nov 10, 2018

Perfect! Exactly what I was looking for :)

@twalling

This comment has been minimized.

Copy link

@twalling twalling commented Dec 14, 2018

I had to change the one-liner by @dovidweisz so that all of my var was included. If you have equal signs in your var which we're also using as the delimiter then you have to tell cut to include all the rest of the fields

MY_VAR=$(grep MY_VAR .env | cut -d '=' -f 2-)
@msmans

This comment has been minimized.

Copy link

@msmans msmans commented Dec 20, 2018

With GNU grep, you can simply do MY_VAR=$(grep -oP '^MY_VAR=\K.*' .env)

@Jules-Baratoux

This comment has been minimized.

Copy link

@Jules-Baratoux Jules-Baratoux commented Dec 26, 2018

I use the following to avoid overriding already set variable:

source <(grep -v '^#' .env | sed -E 's|^(.+)=(.*)$|: ${\1=\2}; export \1|g')
@heycarsten

This comment has been minimized.

Copy link

@heycarsten heycarsten commented Jan 2, 2019

I’ve been doing it like this:

# Load up .env
set -o allexport
[[ -f .env ]] && source .env
set +o allexport

It seems to work great… are their drawbacks to this?

In my case I’m just loading in the variables to use for in-project commands (in bin)

✌️

@dazza-codes

This comment has been minimized.

Copy link

@dazza-codes dazza-codes commented Feb 10, 2019

When this is in a script, allow an existing ENV variable or CLI to override the .env file and only try to use a .env file if it exists:

test -z "$VAR" && test -f .env && \
    VAR=$(grep -e '^VAR=.*' .env | cut -d '=' -f2)

Add an export VAR if necessary.

@seeekr

This comment has been minimized.

Copy link

@seeekr seeekr commented Mar 5, 2019

If you're having trouble with spaces still messing up your loading of the variables, use this:

IFS='
'
export $(egrep -v '^#' .env | xargs -0)
IFS=

(Replace xargs -0 with xargs -d '\n' if you're not on macOS.)

Explanation: IFS changes what bash splits strings on. By default it splits on spaces, but using the above it only does so on newlines. We reset its behavior after we're done.

@TechupBusiness

This comment has been minimized.

Copy link

@TechupBusiness TechupBusiness commented Apr 7, 2019

This works fine (as long as you don't use "=" in your values):

read_var() {
    VAR=$(grep "^$1=" $2 | xargs)
    IFS="=" read -ra VAR <<< "$VAR"
    IFS=" "
    echo ${VAR[1]}
}
@miguelmota

This comment has been minimized.

Copy link

@miguelmota miguelmota commented Jun 24, 2019

Here's a version to make the filename optional and use .env as default:

read_var() {
  if [ -z "$1" ]; then
    echo "environment variable name is required"
    return 1
  fi

  local ENV_FILE='.env'
  if [ ! -z "$2" ]; then
    ENV_FILE="$2"
  fi

  local VAR=$(grep $1 "$ENV_FILE" | xargs)
  IFS="=" read -ra VAR <<< "$VAR"
  echo ${VAR[1]}
}

Example usage:

local my_var=$(read_var MY_VAR)
local my_other_var=$(read_var MY_VAR staging.env)
@AdamGerthel

This comment has been minimized.

Copy link

@AdamGerthel AdamGerthel commented Jul 10, 2019

God I wish it was possible to upvote the different solutions here. I have no idea which one to pick.

@judy2k

This comment has been minimized.

Copy link
Owner Author

@judy2k judy2k commented Jul 10, 2019

@AdamGerthel bear in mind that there are actually two different problems being solved in these solutions. The original is parsing a whole .env file - many of the other 'solutions' are extracting a single value from a .env file.

(This was never intended to be a popular/controversial gist - I was just putting it here for my own use!)

@AdamGerthel

This comment has been minimized.

Copy link

@AdamGerthel AdamGerthel commented Jul 10, 2019

@judy2k right - and I eventually found this on SO: https://stackoverflow.com/a/20909045/930998

@cdcabrera

This comment has been minimized.

Copy link

@cdcabrera cdcabrera commented Jul 12, 2019

Awesome @miguelmota much appreciated! I updated it a little

read_var() {
  local ENV_FILE="${2:-./.env}"
  local VAR=$(grep $1 "$ENV_FILE" | xargs)

  IFS="=" read -ra VAR <<< "$VAR"
  echo ${VAR[1]}
}
@Norcoen

This comment has been minimized.

Copy link

@Norcoen Norcoen commented Aug 8, 2019

I’ve been doing it like this:

# Load up .env
set -o allexport
[[ -f .env ]] && source .env
set +o allexport

It seems to work great… are their drawbacks to this?

In my case I’m just loading in the variables to use for in-project commands (in bin)

v

Personally I use

set -a
[ -f .env ] && . .env
set +a

it does the same, but it should be POSIX compatible

@peteratticusberg

This comment has been minimized.

Copy link

@peteratticusberg peteratticusberg commented Oct 26, 2019

export $(cat .env) works for basic use cases

@cocowalla

This comment has been minimized.

Copy link

@cocowalla cocowalla commented Nov 15, 2019

export $(cat .env) works for basic use cases

What kind of cases does it not work for

EDIT: Answering my own question: it doesn't work for quoted strings

@dobesv

This comment has been minimized.

Copy link

@dobesv dobesv commented Nov 19, 2019

@AlexSkrypnyk

This comment has been minimized.

Copy link

@AlexSkrypnyk AlexSkrypnyk commented Jan 20, 2020

What is important is to make sure that environment variables' values takes precedence over values in .env file.

Using this script solves this:

t=$(mktemp) && export -p > "$t" && set -a && . ./.env && set +a && . "$t" && rm "$t" && unset t
@ktomk

This comment has been minimized.

Copy link

@ktomk ktomk commented Feb 9, 2020

using a dot env file format like in the docker project (supports comments, setting and imports [1]):

<.env.dist | sed -n 's/^[^#]/export /p' | source /proc/self/fd/0

Your mileage may vary. source requires a file-name.

[1]: Docker Run: Set environment variables (-e, --env, --env-file) https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file

@rsoury

This comment has been minimized.

Copy link

@rsoury rsoury commented Feb 13, 2020

read_var() {
    VAR=$(grep $1 $2 | xargs)
    IFS="=" read -ra VAR <<< "$VAR"
    echo ${VAR[1]}
}

MY_VAR=$(read_var MY_VAR .env)

Thanks, this answer worked well, however, for exact matching with grep:

read_var() {
     VAR=$(grep -w $1 $2 | xargs)
     IFS="=" read -ra VAR <<< "$VAR"
     echo ${VAR[1]}
}
 
MY_VAR=$(read_var MY_VAR .env)
@abdennour

This comment has been minimized.

Copy link

@abdennour abdennour commented Feb 27, 2020

This single line had worked with me since 2 years and I am using it everywhere now , even MacOS:

export $(cat .env | xargs)

Do not forget "xargs".

@judy2k

This comment has been minimized.

Copy link
Owner Author

@judy2k judy2k commented Mar 3, 2020

export $(cat .env | xargs)

@abdennour This is the same as the original solution, but the original also filters out comment lines beginning with #

@sarink

This comment has been minimized.

Copy link

@sarink sarink commented Mar 16, 2020

@Blackjacx

This comment has been minimized.

Copy link

@Blackjacx Blackjacx commented Mar 23, 2020

Why not just using source .env

@loopmode

This comment has been minimized.

Copy link

@loopmode loopmode commented Mar 24, 2020

I had trouble using the initial method for values that were URLs, very simple ones actually (e.g HOST=http://localhost:3000)

I then went with source .env and all is good now.

@bmmuller

This comment has been minimized.

Copy link

@bmmuller bmmuller commented Apr 3, 2020

The function below takes the env file as a parameter and handles spaces, comments and single/double quotes properly.

function export_envs() {
  local envFile=${1:-.env}
  while IFS='=' read -r key temp || [ -n "$key" ]; do
    local isComment='^[[:space:]]*#'
    local isBlank='^[[:space:]]*$'
    [[ $key =~ $isComment ]] && continue
    [[ $key =~ $isBlank ]] && continue
    value=$(eval echo "$temp")
    eval export "$key='$value'";
  done < $envFile
}
@ktomk

This comment has been minimized.

Copy link

@ktomk ktomk commented Apr 4, 2020

@bmmuller: does it handle imports? e.g. only export a variable if it exists yet (not null-ify is)? and what is intended the portability of your code example (/bin/bash or /bin/sh)?

@mgrachev

This comment has been minimized.

Copy link

@mgrachev mgrachev commented Jul 7, 2020

In addition to using environment variables I can recommend the tool https://github.com/dotenv-linter/dotenv-linter  — it’s a lightning-fast linter for .env files. Written in Rust. Maybe it would be useful for you.

@m-b-davis

This comment has been minimized.

Copy link

@m-b-davis m-b-davis commented Jul 8, 2020

Here's a version to make the filename optional and use .env as default:

read_var() {
  if [ -z "$1" ]; then
    echo "environment variable name is required"
    return
  fi

  local ENV_FILE='.env'
  if [ ! -z "$2" ]; then
    ENV_FILE="$2"
  fi

  local VAR=$(grep $1 "$ENV_FILE" | xargs)
  IFS="=" read -ra VAR <<< "$VAR"
  echo ${VAR[1]}
}

Example usage:

local my_var=$(read_var MY_VAR)
local my_other_var=$(read_var MY_VAR staging.env)

I had two problems with this

First - This was failing for me when having a var which is a substring of another var e.g

API_PORT=5000
PORT=4000

I would get PORT as 5000 PORT instead of 4000.

Second: if an env var was named incorrectly e.g PORTX=4000 it would still load PORT

To fix, I used a carat before the name to match on the start of the line and an equals after (see grep $1)

read_var() {
  if [ -z "$1" ]; then
    echo "environment variable name is required"
    return
  fi

  local ENV_FILE='.env'
  if [ ! -z "$2" ]; then
    ENV_FILE="$2"
  fi

  local VAR=$(grep ^$1= "$ENV_FILE" | xargs)
  IFS="=" read -ra VAR <<< "$VAR"
  echo ${VAR[1]}
}
@GaMoCh

This comment has been minimized.

Copy link

@GaMoCh GaMoCh commented Jul 16, 2020

Remove spaces and lines that start with '#'

export $(sed 's/[[:blank:]]//g; /^#/d' .env | xargs)
@sandhawke

This comment has been minimized.

Copy link

@sandhawke sandhawke commented Jul 24, 2020

Why not just using source .env

Because the syntax of .env files (X=y) is used in the Bourne shell and its derivatives to set local shell variables, not environment variables. Shell variables just effect the behavior of that shell process, where environment variables affect programs run from that shell, which is what you want with .env.

To set an environment variable you need to use the 'export' command (like "export X=y") or the allexport (-a) setting seen in the solutions above.

@ltfschoen

This comment has been minimized.

Copy link

@ltfschoen ltfschoen commented Aug 22, 2020

After reading through the different approaches here, I used the one by @miguelmota and modified it so that from any Shell script file, I could call a get-env.sh Shell script and specify the environment variable name whose value I wanted to retrieve from the .env file and output to the console.

First I created a .env file with the following contents:

AWS_ACCOUNT_ID=123
MY_NAME="Luke"

Then I created a get-env.sh file and made it executable with chmod +x get-env.sh

#!/bin/bash

# Use this script in other Shell scripts to interpret
# and use the value of an environment variable's name that is provided as
# the first argument to this Shell script that is contained in a .env file
# by running the following, where in this example `my_var` is returned from
# this Shell script and AWS_ACCOUNT_ID is the environment variable name in
# the .env file whose value we want to use in a Shell script:
# ```
# . ./scripts/get-env.sh AWS_ACCOUNT_ID && \
#   export AWS_ACCOUNT_ID=$my_var && \
#   echo $AWS_ACCOUNT_ID
# ```
# Note: This script does not interpret equal signs in the value i.e. `VAR="12=34"`
VAR="$1"

read_var() {
  if [ -z "$1" ]; then
    echo "environment variable name is required"
    return
  fi

  local ENV_FILE='.env'
  # If user provides second argument to read_var use that file instead of .env
  if [ ! -z "$2" ]; then
    ENV_FILE="$2"
  fi

  local VAR=$(grep ^$1= "$ENV_FILE" | xargs)
  IFS="=" read -ra VAR <<< "$VAR"
  echo ${VAR[1]}
}

my_var=$(read_var $VAR ./.env)
# echo $my_var

Then I created a test.sh Shell script and made it executable with chmod +x test.sh

#!/bin/bash

# Retrieve value of environment variable `AWS_ACCOUNT_ID` from .env file
. ./scripts/get-env.sh AWS_ACCOUNT_ID && \
  export AWS_ACCOUNT_ID=$my_var && \
  echo $AWS_ACCOUNT_ID 

# Retrieve value of environment variable `MY_NAME` from .env file
. ./scripts/get-env.sh MY_NAME && \
  export MY_NAME=$my_var && \
  echo $MY_NAME 

And lastly I ran it with ./test.sh
And it output the values of those two environment variables from the .env file to the terminal

123
Luke
@carlosonunez

This comment has been minimized.

Copy link

@carlosonunez carlosonunez commented Aug 25, 2020

Another way of exporting your dotenv to a single command: eval “$(cat $ENV_FILE | tr ‘\n’ ‘ ‘) $COMMAND”

@DenisGorbachev

This comment has been minimized.

Copy link

@DenisGorbachev DenisGorbachev commented Sep 1, 2020

Oneliner:

MY_VAR=$(grep MY_VAR .env | grep -v -P '^\s*#' | cut -d '=' -f 2-)

@tvitcom

This comment has been minimized.

Copy link

@tvitcom tvitcom commented Sep 18, 2020

in

I’ve been doing it like this:

# Load up .env
set -o allexport
[[ -f .env ]] && source .env
set +o allexport

It seems to work great… are their drawbacks to this?
In my case I’m just loading in the variables to use for in-project commands (in bin)
v

Personally I use

set -a
[ -f .env ] && . .env
set +a

it does the same, but it should be POSIX compatible

I tryed that with errors but this in Debian9 ok for me:

#!/bin/sh
set -a
test -f ./.env && . ./.env
set +a
#For example, using os.Getenv("MY_X_PARAM_1") in go files:
go run main.go
@herbaltealeaf

This comment has been minimized.

Copy link

@herbaltealeaf herbaltealeaf commented Oct 22, 2020

For me many of these solutions don't work if you have some crazy envs that will escape and cut off the values.

example 1

  set -a
  source .env
  set +a

This will not work if you have an ampersand character '&' and you'll likely see something like this:

[1] 21850

Command 'r@a' not found, did you mean:

  command 'ra' from deb argus-client

Try: sudo apt install <deb name>

[1]+  Done                    export DB_PASSWORD=gAH47J8Ahhvx

Where the actual contents in the env file are
DB_PASSWORD=r@a&gAH47J8Ahhvx

Example 2:

export $(sed 's/[[:blank:]]//g; /^#/d' .env | xargs)
xargs: unmatched double quote; by default quotes are special to xargs unless you use the -0 option

Example 3:

set -o allexport
[[ -f .env ]] && source .env
set +o allexport
gAH47J8Ahhvx: command not found
[1]+  Done                    AMPERSAND_FAIL=zr@a
bash: .env: line 24: unexpected EOF while looking for matching ``'
bash: .env: line 25: syntax error: unexpected end of file

Example 4:

source <(grep -v '^#' .env | sed -E 's|^(.+)=(.*)$|: ${\1=\2}; export \1|g')
bash: ${          # ANOTHER=ENV}: bad substitution
bash: /dev/fd/63: line 22: unexpected EOF while looking for matching `)'
bash: /dev/fd/63: line 23: syntax error: unexpected end of file
source <(grep -v '^#' .env | sed -E 's|^(.+)=(.*)$|: ${\1=\2}; export \1|g')

I've tried most of these methods and the one which seems to work every case I could think to throw at it is this new function - which I derived from @bmmuller 's export_envs function. Sadly the original one failed ampersands chars '&'.

function export_envs() {
  local envFile=${1:-.env}
  local isComment='^[[:space:]]*#'
  local isBlank='^[[:space:]]*$'
  while IFS= read -r line; do
    [[ $line =~ $isComment ]] && continue
    [[ $line =~ $isBlank ]] && continue
    key=$(echo "$line" | cut -d '=' -f 1)
    value=$(echo "$line" | cut -d '=' -f 2-)
    eval "export ${key}=\"$(echo \${value})\""
  done < <( cat "$envFile" )
}

Test .env file


# a comment

# COMMENTED_FAIL=ENV
        # a comment
          # ANOTHER_COMMENT_FAIL=ENV

TWO_PLUS_EQUALS_FAIL=5432=dfg=sdfg=sdf=dfsg=g
AMPERSAND_FAIL=r@a&gAH47J8Ahhvx
QUOTING_FAIL='a''"'""''''''"""""""'"""'"''$bb$a""$""$"$"'$/$??$?$?$??$?4//$$$a?$/$?$?$?$/$/$/$///$?'$?/$'''$?$'""""?$"""''''\$sdfg"'"/$dfg"""''"""''asdf   ""'''/${avar}"""'' ''""\${bvar}"""'''
SUPER_FAIL=ag^^&&&fghhag^27@%$%$^&*&^*=&)(_()_+_*%^#    @%!$@========#$!@#$!@#WTertJKRY``~`~~~~)~~~``|\\|[{Pp#*$(#(@@!$%#$%&     &^*^&()&(*_)))}]":"??>?</././l';l     'l\[p][}P{P][pl,/,/.,mN<n,n<MB]}]/?<">{}>{}!@#$%^&*()_+====sdfgsd

I'm not sure of the security implications of having to use eval, however it was important to me that the import .env function is able to handle any combination of 'plain-old' (and visible) ASCII chars.

  • This was tested on ubuntu 18.04 in bash. Copy pasta'd into the terminal
@Norcoen

This comment has been minimized.

Copy link

@Norcoen Norcoen commented Oct 22, 2020

Where the actual contents in the env file are
DB_PASSWORD=r@a&gAH47J8Ahhvx

why wouldn't you put that in quotes? Quotes were made for this...

I dont think any solution should try to handle malformed user input. The best way might be a hard fail/exit with error message and return != 0

@herbaltealeaf

This comment has been minimized.

Copy link

@herbaltealeaf herbaltealeaf commented Oct 22, 2020

Where the actual contents in the env file are
DB_PASSWORD=r@a&gAH47J8Ahhvx

why wouldn't you put that in quotes? Quotes were made for this...

I dont think any solution should try to handle malformed user input. The best way might be a hard fail/exit with error message and return != 0

quoting or double-quoting DB_PASSWORD still fails. Its reasonable to expect an export function to handle '&' chars as content.

@Norcoen

This comment has been minimized.

Copy link

@Norcoen Norcoen commented Oct 23, 2020

quoting or double-quoting DB_PASSWORD still fails. Its reasonable to expect an export function to handle '&' chars as content.

$ cd /tmp
$ vim .env
$ cat .env 
DB_PASSWORD='r@a&gAH47J8Ahhvx'
$ set -a
$ . .env
$ set +a
$ echo $DB_PASSWORD 
r@a&gAH47J8Ahhvx

why would it fail though? No problems in my case

@liwo

This comment has been minimized.

Copy link

@liwo liwo commented Oct 28, 2020

why wouldn't you put that in quotes? Quotes were made for this...

Personally, I agree. But the docker developers unfortunately don't.

From the docker-compose documentation (https://docs.docker.com/compose/env-file/):

  • There is no special handling of quotation marks. This means that they are part of the VAL.

Bummer. If you use the same .env file with docker-compose quotes must be avoided.

@Norcoen

This comment has been minimized.

Copy link

@Norcoen Norcoen commented Oct 28, 2020

doing it wrong just because one idiot from docker thought it would be clever to do shit like that... Id rather have a script generating the yaml file from my correct .env file than using it like that
Do you know if podman copied the same behavior or fixed it?

@virgilwashere

This comment has been minimized.

Copy link

@virgilwashere virgilwashere commented Oct 29, 2020

@herbaltealeaf @bmmuller

What about the use case when values build on preceding entries in the dotenv file?

Source file: gitlab.env

CI_SERVER_HOST=gitlab.selfhosted.domain
CI_SERVER_PROTOCOL=https
CI_SERVER_URL=${CI_SERVER_PROTOCOL}://${CI_SERVER_HOST}
CI_API_V4_URL=${CI_SERVER_URL}/api/v4
CI_REGISTRY=${CI_SERVER_HOST}:4567
CI_REGISTRY_USER=gitlab-ci-token
GITLAB_URL=${CI_SERVER_URL}

I get literals as values.

$ export_envs gitlab.env
$ env | grep -Fe CI_ -e GITLAB_ - | grep -Fv TOKEN
CI_REGISTRY_USER=gitlab-ci-token
CI_API_V4_URL=${CI_SERVER_URL}/api/v4
CI_PAGES_DOMAIN=pages.selfhosted.domain
CI_REGISTRY=${CI_SERVER_HOST}:4567
CI_SERVER_URL=${CI_SERVER_PROTOCOL}://${CI_SERVER_HOST}
CI_SERVER_PROTOCOL=https
CI_SERVER_HOST=gitlab.selfhosted.domain
GITLAB_URL=${CI_SERVER_URL}

Whereas sourcing the file evaluates the values correctly:

$ source gitlab.env
$ env | grep -Fe CI_ -e GITLAB_ - | grep -Fv TOKEN
CI_REGISTRY_USER=gitlab-ci-token
CI_API_V4_URL=https://gitlab.selfhosted.domain/api/v4
CI_PAGES_DOMAIN=pages.selfhosted.domain
CI_REGISTRY=gitlab.selfhosted.domain:4567
CI_SERVER_URL=https://gitlab.selfhosted.domain
CI_SERVER_PROTOCOL=https
CI_SERVER_HOST=gitlab.selfhosted.domain
GITLAB_URL=https://gitlab.selfhosted.domain
@leydson-vieira

This comment has been minimized.

Copy link

@leydson-vieira leydson-vieira commented Jan 19, 2021

god bless you

@kolypto

This comment has been minimized.

Copy link

@kolypto kolypto commented Feb 9, 2021

This version withstands every special character in values:

set -a
source <(cat development.env | sed -e '/^#/d;/^\s*$/d' -e "s/'/'\\\''/g" -e "s/=\(.*\)/='\1'/g")
set +a

Explanation:

  • -a means that every bash variable would become an environment variable
  • /^#/d removes comments (strings that start with #)
  • /^\s*$/d removes empty strings, including whitespace
  • "s/'/'\\\''/g" replaces every single quote with '\'', which is a trick sequence in bash to produce a quote :)
  • "s/=\(.*\)/='\1'/g" converts every a=b into a='b'

As a result, you are able to use special characters :)

To debug this code, replace source with cat and you'll see what this command produces.

@weisk

This comment has been minimized.

Copy link

@weisk weisk commented Feb 14, 2021

Awesomest discussion in a gist ever

@loopmode

This comment has been minimized.

Copy link

@loopmode loopmode commented Feb 14, 2021

Epic gist indeed :)

@abij

This comment has been minimized.

Copy link

@abij abij commented Feb 16, 2021

@kolypto my initial version was using single quotes in the .env file and is double enriched with your version, so i tweaked it a bit.

#input from .env:
SECRET1='bla'
SECRET2  = "sap"
SECRET3 =buz
SECRET_4 =b'u"z

# wrong output:
SECRET1=''\''bla'\'''
SECRET2  =' "sap"'
SECRET3 ='buz'
SECRET_4 ='b'\''u"z'

Improved sed rules:

  • /^#/d (same) removes comments (strings that start with #)
  • /^\s*$/d (same) removes empty strings, including whitespace
  • s/\(\w*\)[ \t]*=[ \t]*\(.*\)/\1=\2/ keep group 1 and 2 before and after the = ignoring the whitespace.
  • "s/=['\"]\(.*\)['\"]/=\1/g" value part, removes start/ending single or double quotes
  • "s/'/'\\\''/g" (same) replaces every single quote with ''', which is a trick sequence in bash to produce a quote :)
  • "s/=\(.*\)/='\1'/g" (same) converts every a=b into a='b'
set -o allexport
eval $(cat '.env' | sed -e '/^#/d;/^\s*$/d' -e 's/\(\w*\)[ \t]*=[ \t]*\(.*\)/\1=\2/' -e "s/=['\"]\(.*\)['\"]/=\1/g" -e "s/'/'\\\''/g" -e "s/=\(.*\)/='\1'/g")
set +o allexport
# Correct output, ready for export!
SECRET1='bla'
SECRET2='sap'
SECRET3='buz'
SECRET_4='b'\''u"z'
@rochacbruno

This comment has been minimized.

Copy link

@rochacbruno rochacbruno commented Mar 5, 2021

@Jules-Baratoux thanks.

your solution

# avoid override already set variables
source <(grep -v '^#' .env | sed -E 's|^(.+)=(.*)$|: ${\1=\2}; export \1|g')

Works great for me

🥳

@kolserdav

This comment has been minimized.

Copy link

@kolserdav kolserdav commented Mar 7, 2021

For get value of specified variable by name:

echo $(grep -v '^#' .env | grep -e "YOUR_VARIABLE_NAME_FROM_DOT_ENV_FILE" | sed -e 's/.*=//')
@rochacbruno

This comment has been minimized.

Copy link

@rochacbruno rochacbruno commented Mar 15, 2021

This is the tricky the worked for me on bash and zsh

set variables from .compose.env but don't override existing exported vars

eval "$(grep -v '^#' .compose.env | sed -E 's|^(.+)=(.*)$|export \1=${\1:-\2}|g' | xargs -L 1)"
@LuanP

This comment has been minimized.

Copy link

@LuanP LuanP commented Mar 18, 2021

Considering only the first =:

eval "$(grep -v '^#' .env | sed -E '0,/^(.+)=/s/^(.+)=(.*)$/export \1=${\1:-\2}/g' | xargs -L 1)"
@j0eii

This comment has been minimized.

Copy link

@j0eii j0eii commented Mar 23, 2021

read_var(){
  echo $(grep -v '^#' .env | grep -e "$1" | sed -e 's/.*=//')
}

CUR_APP_ENV=$(read_var "APP_ENV")

Not to override APP_ENV caz it will breaks .env hot reload.

@i3130002

This comment has been minimized.

Copy link

@i3130002 i3130002 commented Mar 30, 2021

Used export $(grep -v '^#' .env | xargs -d '\r\n') to trim \r from lines.

@ko1nksm

This comment has been minimized.

Copy link

@ko1nksm ko1nksm commented Apr 12, 2021

Seeing that this thread has been going on for long years, I figured we need a dotenv tool for the shell.

And I wrote it.
https://github.com/ko1nksm/shdotenv

There is no formal specification for .env, and each is slightly different, but shdotenv supports them and correctly parses comments, whitespace, quotes, etc. It is a single file shell script that requires only awk and runs lightly.

There is no need to waste time on trial and error anymore.

@smac89

This comment has been minimized.

Copy link

@smac89 smac89 commented Apr 15, 2021

The following worked well for me in github actions:

eval "cat <<EOF
$(egrep -v '^#' .env)
EOF
" | tee --append $GITHUB_ENV

Tips for using this (things that bit me):

  • Don't quote anything unless absolutely necessary; the quotes will be taken literally
  • Avoid interpolation which uses a variable inside the .env file. So if your file contains FOO and you try to use in in another variable like FOOBAR=${FOO}bar, all you will get is FOOBAR=bar

Apart from these, it supports pretty much anything you can do regularly including using variables that store the output of a command

@adriancuadrado

This comment has been minimized.

Copy link

@adriancuadrado adriancuadrado commented Apr 15, 2021

Here is my contribution, which I developed with no idea someone already did something similar (I saw too many comments and I decided to develop it instead of reading all of them):

eval "$(
    cat <(
        grep -vE -e '^\s*#' -e '^\s*$' < .env |
        grep -E "^[A-Z0-9_]+=['\"]"
    ) <(
        grep -vE -e '^\s*#' -e '^\s*$' -e "^[A-Z0-9_]+=['\"]" < .env |
        sed -r "s/^([A-Z0-9_]+=)(.*)/\\1'\\2'/"
    )
)"

Explanation:

  1. Remove all comments and empty lines:
    • Regex for comments: ^\s*#
    • Regex for empty lines: ^\s*$
    • Grep command:
      grep -vE -e '^\s*#' -e '^\s*$' < .env
      
  2. Get all variables that do not need treatment (i.e. those which are already inside quotes):
    grep -E "^[A-Z0-9_]+=['\"]"
    
  3. Pipe step 1 to step 2:
    grep -vE -e '^\s*#' -e '^\s*$' < .env |
    grep -E "^[A-Z0-9_]+=['\"]"
    
  4. Use process substitution to pass the result as a parameter to cat (I could have used a temporal file, but meh):
    <(
        grep -vE -e '^\s*#' -e '^\s*$' < .env |
        grep -E "^[A-Z0-9_]+=['\"]"
    )
    
  5. Get all variables that do need treatment:
    grep -vE -e '^\s*#' -e '^\s*$' -e "^[A-Z0-9_]+=['\"]" < .env
    

    Notice how the 3 regular expressions are the same as in steps 1 and 2. The only difference here is that I reuse the -v option with the third regex.

  6. Put the single quotes to avoid problems with special characters:
    • Regex for the variable with '=': ^([A-Z0-9_]+=)
    • Regex for the rest of the contents that must be inside single quotes: (.*)
    • Put the first group as is and the second between quotes: \\1'\\2'
    • Sed command:
      sed -r "s/^([A-Z0-9_]+=)(.*)/\\1'\\2'/"
      
  7. Pipe step 5 to step 6:
    grep -vE -e '^\s*#' -e '^\s*$' -e "^[A-Z0-9_]+=['\"]" < .env |
    sed -r "s/^([A-Z0-9_]+=)(.*)/\\1'\\2'/"
    
  8. Do pretty much the same as step 4:
    <(
        grep -vE -e '^\s*#' -e '^\s*$' -e "^[A-Z0-9_]+=['\"]" < .env |
        sed -r "s/^([A-Z0-9_]+=)(.*)/\\1'\\2'/"
    )
    
  9. Concat and eval:
eval "$(
    cat <(
        grep -vE -e '^\s*#' -e '^\s*$' < .env |
        grep -E "^[A-Z0-9_]+=['\"]"
    ) <(
        grep -vE -e '^\s*#' -e '^\s*$' -e "^[A-Z0-9_]+=['\"]" < .env |
        sed -r "s/^([A-Z0-9_]+=)(.*)/\\1'\\2'/"
    )
)"

This is basically what @kolypto does here, but more complex and worse because it doesn't take into consideration the variables that contain single or double quotes inside them, because I should have been able to reuse the result of some grep commands and because maybe there is also a way to not have to feed 2 commands with the contents of the same file (I wrote < .env twice). I had some fun coding this at least ;)

@adriancuadrado

This comment has been minimized.

Copy link

@adriancuadrado adriancuadrado commented Apr 15, 2021

If you are seeking for an aswer and have scrolled to the bottom to find it, here it is (thanks to @abij. You can see his answer here):

set -o allexport
eval $(cat '.env' | sed -e '/^#/d;/^\s*$/d' -e 's/\(\w*\)[ \t]*=[ \t]*\(.*\)/\1=\2/' -e "s/=['\"]\(.*\)['\"]/=\1/g" -e "s/'/'\\\''/g" -e "s/=\(.*\)/='\1'/g")
eval $(cat '.env.local' | sed -e '/^#/d;/^\s*$/d' -e 's/\(\w*\)[ \t]*=[ \t]*\(.*\)/\1=\2/' -e "s/=['\"]\(.*\)['\"]/=\1/g" -e "s/'/'\\\''/g" -e "s/=\(.*\)/='\1'/g")
set +o allexport

Just make sure you execute that monster in the same folder that you have your .env file.
The second eval is to read the .env.local file, whose variables should override the ones in .env

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment