Skip to content

Instantly share code, notes, and snippets.

@oginga
Last active February 14, 2019 22:50
Show Gist options
  • Save oginga/da29a1a99527dff59cf61ec687a5c963 to your computer and use it in GitHub Desktop.
Save oginga/da29a1a99527dff59cf61ec687a5c963 to your computer and use it in GitHub Desktop.
Importing third party(including your own) python modules on PL/Python

Problem

I needed to perfom some linear programming calculations with my local Postgres installation whenever there was change(insert/update) in any row for my events table. This prompted the creation of a trigger function(NB: the example below is a UDF that doesn't return a trigger) and binding it to the table.I edited a dummy function to test if my third party module pulp was actually being imported by the PL/Python embedded interpreter.The lib was installed on my systemwide local site-package dir /home//.local/lib/python2.7/site-packages - Elementary Juno and visible on pip freeze

	CREATE OR REPLACE FUNCTION pyversion() RETURNS text AS $$
	import sys,os
	from pulp import *
	#sys.path.remove('/home/<username>/.local/lib/python2.7/site-packages')
	return sys.version + '\n' + '\n'.join(sys.path)+'\n'.join(os.environ)
	$$ LANGUAGE plpythonu;

and ran it using <psql-database># select pyversion(); which returned an error

ERROR:  ImportError: No module named pulp
CONTEXT:  Traceback (most recent call last):
  PL/Python function "pyversion", line 3, in <module>
    from pulp import *
PL/Python function "pyversion"

Attempt 1

Use python sys.path.append/insert to append the lib path

	CREATE OR REPLACE FUNCTION pyversion() RETURNS text AS $$
	import sys,os
	sys.path.append('/home/<username>/.local/lib/python2.7/site-packages')
	from pulp import *
	return sys.version + '\n' + '\n'.join(sys.path)+'\n'.join(os.environ)
	$$ LANGUAGE plpythonu;

Got the same error as above.But its worth noting that when the append and import lines were commented out and script re-run,the output indicated the existence of our previously appended path

-----------------------------------------------------
 2.7.15rc1 (default, Nov 12 2018, 14:31:15)         +
 [GCC 7.3.0]                                        +
 /usr/lib/python2.7                                 +
 /usr/lib/python2.7/plat-x86_64-linux-gnu           +
 /usr/lib/python2.7/lib-tk                          +
 /usr/lib/python2.7/lib-old                         +
 /usr/lib/python2.7/lib-dynload                     +
 /usr/local/lib/python2.7/dist-packages             +
 /usr/lib/python2.7/dist-packages                   +
 /usr/lib/python2.7/dist-packages/gtk-2.0           +
 /home/<username>/.local/lib/python2.7/site-packagesLANG+
 LC_NUMERIC                                         +
 PG_OOM_ADJUST_FILE                                 +
 PGSYSCONFDIR                                       +
 PG_GRANDPARENT_PID                                 +
 LC_COLLATE                                         +
 LC_CTYPE                                           +
 LC_MONETARY                                        +
 PGDATA                                             +
 PWD                                                +
 PGLOCALEDIR                                        +
 LC_TIME                                            +
 LC_MESSAGES
(1 row)


Attempt 2

Created a symbolic link of the site-packages dir in the ** /usr/lib/python2.7** dir.

ln -s /home/<username>/.local/lib/python2.7/site-packages/pulp /usr/lib/python2.7/pulp

but still got the same error

SOLUTION Attempt 3

I copied the lib to one of the python library paths # cp -r /home/<username>/.local/lib/python2.7/site-packages/pulp /usr/lib/python2.7/pulp as a superuser and my function finally worked

I'M NOT SURE IF THIS IS A NEAT SOLUTION TO THE PROBLEM,SO THIS GIST IS A CALL FOR NEATER SOLUTION AND A FALLBACK FOR THOSE IN MY SITUATION

@sinnud
Copy link

sinnud commented Feb 14, 2019

I met the same issue recently and got solved by:
chmod 777 /home/<username>/.local/lib/python2.7/site-packages
chmod 666 /home/<username>/.local/lib/python2.7/site-packages/*.py
Please try to see if it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment