Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Python relative imports in AWS Lambda fail with `attempted relative import with no known parent package`

Python relative imports in AWS Lambda fail with attempted relative import with no known parent package

The Problem

In AWS Lambda if I attempt an explicit relative import like this

.
├── lambda_file.py
└── example.py
# lambda_file.py
from .example import lambda_handler
# example.py
def lambda_handler(event, context):
    return True

And I configure AWS Lambda's handler to lambda_file.lambda_handler I get the following errors

  • Python 2.7 : Attempted relative import in non-package
  • Python 3.7 : attempted relative import with no known parent package

Why use explicit relative imports

PEP008 says :

Implicit relative imports should never be used and have been removed in Python 3.

How to workaround by using implicit relative imports

If I change lambda_file.py to contain the following, it works, but no longer uses explicit relative imports

# lambda_file.py
from example import lambda_handler

How to correctly solve the problem

The solution is to ensure that the "Handler" value that you configure in AWS Lambda contain at least 2 . periods. To achieve this you need to put your code within a directory in your AWS Lambda code zip file and make that directory a module by adding an empty __init__.py file. The resulting structure looks like this

.
├── app
│   ├── __init__.py
│   ├── lambda_file.py
│   └── example.py

And you now change the "Handler" value from lambda_file.lambda_handler to app.lambda_file.lambda_handler

Additional Notes

@hawkins

This comment has been minimized.

Copy link

@hawkins hawkins commented Feb 26, 2020

Thanks for this info - I was just running into this for the N-th time today and couldn't recall the solution!

AWS Lambda loads your handler as a module import not as a top-level script

Do you have a link to the docs for this or is it something you just figured out? Maybe just by reading the lambda Python runtime source code?

@mukosi

This comment has been minimized.

Copy link

@mukosi mukosi commented Jun 11, 2020

Thank you very much for documenting this - it saved me a number of heart beats :-)

@gene1wood

This comment has been minimized.

Copy link
Owner Author

@gene1wood gene1wood commented Jun 11, 2020

AWS Lambda loads your handler as a module import not as a top-level script

Do you have a link to the docs for this or is it something you just figured out? Maybe just by reading the lambda Python runtime source code?

@hawkins I think this is something that I intuited by looking at how you instruct AWS Lambda on how to begin invoking your code. You do so by giving it a string (e.g. lambda_function.lambda_handler) which is a Python . delimited value that an import statement could use. This AWS development guide says

The lambda_function file exports a function named lambda_handler that takes an event object and a context object. This is the handler function that Lambda calls when the function is invoked. The Python function runtime gets invocation events from Lambda and passes them to the handler. In the function configuration, the handler value is lambda_function.lambda_handler

which alludes to this fact.

Thank you very much for documenting this - it saved me a number of heart beats :-)

@mukosi , you're welcome =)

@hareesh0909

This comment has been minimized.

Copy link

@hareesh0909 hareesh0909 commented Jun 26, 2020

Thank you !!

@Jordan-Eckowitz

This comment has been minimized.

Copy link

@Jordan-Eckowitz Jordan-Eckowitz commented Jun 30, 2020

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))
@brikis98

This comment has been minimized.

Copy link

@brikis98 brikis98 commented Aug 27, 2020

Another thank you for documenting this! I was doing folder/handler.func rather than folder.handler.func... Would've taken forever to figure out without this simple Gist 👍

@dev-owner

This comment has been minimized.

Copy link

@dev-owner dev-owner commented Dec 3, 2020

Another thank you for documenting this! I was doing folder/handler.func rather than folder.handler.func... Would've taken forever to figure out without this simple Gist 👍

me, too.. Thanks a lot!

@threadstonesecure

This comment has been minimized.

Copy link

@threadstonesecure threadstonesecure commented Jan 5, 2021

Thanks a lot !!!!
It saved my day !!!!

@chandra-goka

This comment has been minimized.

Copy link

@chandra-goka chandra-goka commented Jan 5, 2021

Thanks a lot, its worked.

@tecklund

This comment has been minimized.

Copy link

@tecklund tecklund commented Jan 8, 2021

Thank you for saving me hours of confusion :)

@CarlosDomingues

This comment has been minimized.

Copy link

@CarlosDomingues CarlosDomingues commented Jan 12, 2021

Thank you so much!

@yucer-elbt

This comment has been minimized.

Copy link

@yucer-elbt yucer-elbt commented Feb 18, 2021

Thank you !

@mebibou

This comment has been minimized.

Copy link

@mebibou mebibou commented Feb 18, 2021

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))

Done this for Google as moving code to a folder seems to not work at all! This was the only solution that worked for GCP (which forces you to have a main.py file at the root...)

@gene1wood

This comment has been minimized.

Copy link
Owner Author

@gene1wood gene1wood commented Feb 18, 2021

@mebibou Indeed, the solution above is for AWS Lambda. Sounds like GCP's equivalent may work differently.

@EyalPerry

This comment has been minimized.

Copy link

@EyalPerry EyalPerry commented Apr 4, 2021

thank you!

@pagpires

This comment has been minimized.

Copy link

@pagpires pagpires commented Apr 25, 2021

Thanks for the explanation, this is very helpful. Any ideas with numpy-like package? It doesn't allow relative importing by complaining:

# when I do `from . import numpy` 
Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

Seems the reason being that it will needs C compilation and relative importing will mess it up.

Really appreciate any tips!

@Ghost93

This comment has been minimized.

Copy link

@Ghost93 Ghost93 commented May 24, 2021

Instead of folder restructuring you can add the line below to lambda_file.py. This adds the Lambda file's directory to the path.
You can then use absolute, rather than relative, imports inside lambda_file.py.

sys.path.append(os.path.join(os.path.dirname(__file__)))

@Jordan-Eckowitz
works great!

@AustinGilkison

This comment has been minimized.

Copy link

@AustinGilkison AustinGilkison commented Oct 8, 2021

Deploying with container images
If you are Deploying container images, you may also need to make these modifications.
This is before breaking into multiple files.

.
├── app.py
|── Dockerfile
└── requirements.txt

After with the lambda_handler in the __init__.py

.
├── app
│   ├── __init__.py
│   ├── models.py
│   ├── exceptions.py
│   └── utils.py
|── Dockerfile
└── requirements.txt

You will also need to rework the Dockerfile COPY commands from

COPY app.py requirements.txt ./

RUN python3.8 -m pip install -r requirements.txt -t .

to

COPY requirements.txt ./
RUN python3.8 -m pip install -r requirements.txt -t .

COPY app/ ./app/

Via this method, you can keep the CMD command the same CMD ["app.lambda_handler"]
This is what I had to do to get it to work for me.

@balajimaniv

This comment has been minimized.

Copy link

@balajimaniv balajimaniv commented Oct 14, 2021

@AustinGilkison
Thanks for your input, it works perfectly fine for files that exist inside container.
Let say If I want to call the handler file located in EFS file system and its mounted to Lambda function with the mount path "/mnt/xxxfolder" .

I have copied the 'init.py' in all the folder structure defined till handler function, but Its still failing with

[ERROR] TypeError: the 'package' argument is required to perform a relative import for '.mnt.xxxfolder.yyfolder.lambda_handler'
Traceback (most recent call last):
  File "/var/lang/lib/python3.9/importlib/init.py", line 122, in import_module
    raise TypeError(msg.format(name))
    
 Appreciate your comments. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment