Topic Thread

[AWS] Add python librairies to AWS lambda

  • 1.  [AWS] Add python librairies to AWS lambda

    Posted 11-08-2018 07:31

    Working on a project I wanted to use python librairies not installed natively on AWS lambda. 
    For example, I used paramiko to generate a SSH session to an EC2 device. 
    I used the following process:

    1. Create a standard linux EC2 device in AWS. You can search for  "linux" on marketplace and take the first available result.

    2. Use pre-installed yum application to install "python-devel" and "openssl-devel" as follows:
    #sudo yum install python-devel openssl-devel

    3. Get "pip" application which you will use to install virtualenv application later.
    #curl -o

    4. Install pip
    #sudo python

    5. Install a virtual environment on your linux device to store all python librairies and dependencies inside a local repository which you will then upload to AWS.
    #sudo pip install virtualenv

    6. Create your environment called "mylambda" for example
    #sudo virtualenv mylambda

    7. Move to that new environment.
    #cd mylambda

    8. Activate the new environment.
    #source bin/activate

    9. Now you can install any python application/code/featrue as you would do on a standard linux device. All dependencies will be established within the virtual environment (i.e the local repository). Here I install "paramiko" for ssh connections.
    #pip install paramiko
    =>(do not use "sudo" here. Otherwise the installation will involve the library in the root directory)

    10. Move to the directory where all the libraries and useful files have just been installed.
    #cd lib/python2.7/site-package/

    11. Here you need to zip all files to a directory without changing the folder strucutre. Usual issue is that dezipping is generating a subfolder (Ex: ===unzip===> mylambda) which will make librairies impossible to find by native AWS lambda service. Using zip -r9 avoids that issue.
    #zip -r9 /home/ec2-user/ .

    12. Do the same for all lib64 files.
    #cd ../../../lib64/python2.7/site-package/
    #zip -r9 /home/ec2-user/ .

    13. Now you need to add your freshly coded lambda python function to that directory. You will get a nice directory containing all librairies and your lambda function.
    #zip /home/ec2-user/ <your python which is the main script containing the handler. Ex:>

    => that's it !

    14. Now you can upload the whole zip package to AWS lambda. WARNING: Use s3 bucket to upload the file to lambda ! Do not use the "upload zip file" feature from lambda AWS. 


    Emmanuel [LastName] [Designation]
    [City] [State]