Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 90 You must be signed in to star a gist
  • Fork 8 You must be signed in to fork a gist
  • Save gene1wood/06a64ba80cf3fe886053f0ca6d375bc0 to your computer and use it in GitHub Desktop.
Save gene1wood/06a64ba80cf3fe886053f0ca6d375bc0 to your computer and use it in GitHub Desktop.
Python relative imports in AWS Lambda fail with `attempted relative import with no known parent package`

Python relative imports in AWS Lambda fail with attempted relative import with no known parent package

The Problem

In AWS Lambda if I attempt an explicit relative import like this

.
├── lambda_file.py
└── example.py
# lambda_file.py
from .example import lambda_handler
# example.py
def lambda_handler(event, context):
    return True

And I configure AWS Lambda's handler to lambda_file.lambda_handler I get the following errors

  • Python 2.7 : Attempted relative import in non-package
  • Python 3.7 : attempted relative import with no known parent package

Why use explicit relative imports

PEP008 says :

Implicit relative imports should never be used and have been removed in Python 3.

How to workaround by using implicit relative imports

If I change lambda_file.py to contain the following, it works, but no longer uses explicit relative imports

# lambda_file.py
from example import lambda_handler

How to correctly solve the problem

The solution is to ensure that the "Handler" value that you configure in AWS Lambda contain at least 2 . periods. To achieve this you need to put your code within a directory in your AWS Lambda code zip file and make that directory a module by adding an empty __init__.py file. The resulting structure looks like this

.
├── app
│   ├── __init__.py
│   ├── lambda_file.py
│   └── example.py

And you now change the "Handler" value from lambda_file.lambda_handler to app.lambda_file.lambda_handler

Additional Notes

@chandra-goka
Copy link

Thanks a lot, its worked.

@tecklund
Copy link

tecklund commented Jan 8, 2021

Thank you for saving me hours of confusion :)

@CarlosDomingues
Copy link

Thank you so much!

@yucer-elbt
Copy link

Thank you !

@mebibou
Copy link

mebibou commented Feb 18, 2021

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))

Done this for Google as moving code to a folder seems to not work at all! This was the only solution that worked for GCP (which forces you to have a main.py file at the root...)

@gene1wood
Copy link
Author

@mebibou Indeed, the solution above is for AWS Lambda. Sounds like GCP's equivalent may work differently.

@EyalPerry
Copy link

thank you!

@pagpires
Copy link

Thanks for the explanation, this is very helpful. Any ideas with numpy-like package? It doesn't allow relative importing by complaining:

# when I do `from . import numpy` 
Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

Seems the reason being that it will needs C compilation and relative importing will mess it up.

Really appreciate any tips!

@Ghost93
Copy link

Ghost93 commented May 24, 2021

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))

@Jordan-Eckowitz
works great!

@AustinGilkison
Copy link

Deploying with container images
If you are Deploying container images, you may also need to make these modifications.
This is before breaking into multiple files.

.
├── app.py
|── Dockerfile
└── requirements.txt

After with the lambda_handler in the __init__.py

.
├── app
│   ├── __init__.py
│   ├── models.py
│   ├── exceptions.py
│   └── utils.py
|── Dockerfile
└── requirements.txt

You will also need to rework the Dockerfile COPY commands from

COPY app.py requirements.txt ./

RUN python3.8 -m pip install -r requirements.txt -t .

to

COPY requirements.txt ./
RUN python3.8 -m pip install -r requirements.txt -t .

COPY app/ ./app/

Via this method, you can keep the CMD command the same CMD ["app.lambda_handler"]
This is what I had to do to get it to work for me.

@balajimaniv
Copy link

@AustinGilkison
Thanks for your input, it works perfectly fine for files that exist inside container.
Let say If I want to call the handler file located in EFS file system and its mounted to Lambda function with the mount path "/mnt/xxxfolder" .

I have copied the 'init.py' in all the folder structure defined till handler function, but Its still failing with

[ERROR] TypeError: the 'package' argument is required to perform a relative import for '.mnt.xxxfolder.yyfolder.lambda_handler'
Traceback (most recent call last):
  File "/var/lang/lib/python3.9/importlib/init.py", line 122, in import_module
    raise TypeError(msg.format(name))
    
 Appreciate your comments. Thanks.

@finesse-fingers
Copy link

finesse-fingers commented Feb 2, 2022

And you now change the "Handler" value from lambda_file.lambda_handler to app.lambda_file.lambda_handler

Worked a treat. I also had to stick a bunch of __init__.py, due to deeply nested structure of some of the lambdas.

@nk9
Copy link

nk9 commented Jul 17, 2022

Thanks so much, this quickly helped me solve my problem!

@whoizNiKHiL
Copy link

Thanks .

@SpicySoftware
Copy link

Thanks a lot!

@MZuk543
Copy link

MZuk543 commented Feb 2, 2024

Thank you!

@thomasuebel
Copy link

Thanks a bunch! I spent too much time figuring this out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment