Python relative imports in AWS Lambda fail with `attempted relative import with no known parent package`

aws-lambda-relative-import-no-known-parent-package.md

Python relative imports in AWS Lambda fail with attempted relative import with no known parent package

The Problem

In AWS Lambda if I attempt an explicit relative import like this

.
├── lambda_file.py
└── example.py

# lambda_file.py
from .example import lambda_handler

# example.py
def lambda_handler(event, context):
    return True

And I configure AWS Lambda's handler to lambda_file.lambda_handler I get the following errors

• Python 2.7 : Attempted relative import in non-package
• Python 3.7 : attempted relative import with no known parent package

Why use explicit relative imports

PEP008 says :

Implicit relative imports should never be used and have been removed in Python 3.

How to workaround by using implicit relative imports

If I change lambda_file.py to contain the following, it works, but no longer uses explicit relative imports

# lambda_file.py
from example import lambda_handler

How to correctly solve the problem

The solution is to ensure that the "Handler" value that you configure in AWS Lambda contain at least 2 . periods. To achieve this you need to put your code within a directory in your AWS Lambda code zip file and make that directory a module by adding an empty __init__.py file. The resulting structure looks like this

.
├── app
│   ├── __init__.py
│   ├── lambda_file.py
│   └── example.py

And you now change the "Handler" value from lambda_file.lambda_handler to app.lambda_file.lambda_handler

Additional Notes

• AWS Lambda loads your handler as a module import not as a top-level script
• This solution came after reading this explanation of explicit relative imports by Brendan Barnwell

Thank you very much for documenting this - it saved me a number of heart beats :-)

AWS Lambda loads your handler as a module import not as a top-level script

Do you have a link to the docs for this or is it something you just figured out? Maybe just by reading the lambda Python runtime source code?

@hawkins I think this is something that I intuited by looking at how you instruct AWS Lambda on how to begin invoking your code. You do so by giving it a string (e.g. lambda_function.lambda_handler) which is a Python . delimited value that an import statement could use. This AWS development guide says

The lambda_function file exports a function named lambda_handler that takes an event object and a context object. This is the handler function that Lambda calls when the function is invoked. The Python function runtime gets invocation events from Lambda and passes them to the handler. In the function configuration, the handler value is lambda_function.lambda_handler

which alludes to this fact.

Thank you very much for documenting this - it saved me a number of heart beats :-)

@mukosi , you're welcome =)

Thank you !!

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))

Another thank you for documenting this! I was doing folder/handler.func rather than folder.handler.func... Would've taken forever to figure out without this simple Gist 👍

Another thank you for documenting this! I was doing folder/handler.func rather than folder.handler.func... Would've taken forever to figure out without this simple Gist 👍

me, too.. Thanks a lot!

Thanks a lot !!!!
It saved my day !!!!

Thanks a lot, its worked.

Thank you for saving me hours of confusion :)

Thank you so much!

Thank you !

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))

Done this for Google as moving code to a folder seems to not work at all! This was the only solution that worked for GCP (which forces you to have a main.py file at the root...)

@mebibou Indeed, the solution above is for AWS Lambda. Sounds like GCP's equivalent may work differently.

thank you!

Thanks for the explanation, this is very helpful. Any ideas with numpy-like package? It doesn't allow relative importing by complaining:

# when I do `from . import numpy` 
Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

Seems the reason being that it will needs C compilation and relative importing will mess it up.

Really appreciate any tips!

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))

@Jordan-Eckowitz
works great!

Deploying with container images
If you are Deploying container images, you may also need to make these modifications.
This is before breaking into multiple files.

.
├── app.py
|── Dockerfile
└── requirements.txt

After with the lambda_handler in the __init__.py

.
├── app
│   ├── __init__.py
│   ├── models.py
│   ├── exceptions.py
│   └── utils.py
|── Dockerfile
└── requirements.txt

You will also need to rework the Dockerfile COPY commands from

COPY app.py requirements.txt ./

RUN python3.8 -m pip install -r requirements.txt -t .

to

COPY requirements.txt ./
RUN python3.8 -m pip install -r requirements.txt -t .

COPY app/ ./app/

Via this method, you can keep the CMD command the same CMD ["app.lambda_handler"]
This is what I had to do to get it to work for me.

@AustinGilkison
Thanks for your input, it works perfectly fine for files that exist inside container.
Let say If I want to call the handler file located in EFS file system and its mounted to Lambda function with the mount path "/mnt/xxxfolder" .

I have copied the 'init.py' in all the folder structure defined till handler function, but Its still failing with

[ERROR] TypeError: the 'package' argument is required to perform a relative import for '.mnt.xxxfolder.yyfolder.lambda_handler'
Traceback (most recent call last):
  File "/var/lang/lib/python3.9/importlib/init.py", line 122, in import_module
    raise TypeError(msg.format(name))
    
 Appreciate your comments. Thanks.

And you now change the "Handler" value from lambda_file.lambda_handler to app.lambda_file.lambda_handler

Worked a treat. I also had to stick a bunch of __init__.py, due to deeply nested structure of some of the lambdas.

Thanks so much, this quickly helped me solve my problem!

Thanks .

Thanks a lot!

Thank you!

Thanks a bunch! I spent too much time figuring this out.