Skip to content

Instantly share code, notes, and snippets.

@simonw
Created February 2, 2025 20:45
Show Gist options
  • Save simonw/f8283d68e9bd7ad3f140d52cad6874a7 to your computer and use it in GitHub Desktop.
Save simonw/f8283d68e9bd7ad3f140d52cad6874a7 to your computer and use it in GitHub Desktop.
pylimbo
========
pylimbo is an in-process, SQLite‑compatible OLTP (Online Transaction Processing) database management system written in Rust and exposed to Python via the pyo3/maturin toolchain. It implements a subset of Python’s DB‑API 2.0, allowing you to use standard SQL commands to work with your data in a lightweight embedded database engine.
Features
--------
• High‑performance, in‑process database engine written in Rust
• SQLite‑compatible SQL interface
• Standard Python DB‑API 2.0–style connection and cursor objects
• Detailed error handling with custom exception classes
• In‑memory or file‑based databases
• Easy integration into Python applications
Note: In this alpha version, some advanced features (such as transactions, executemany(), and fetchmany()) are not supported; calls to these functions will raise NotSupportedError.
Installation
------------
Requirements:
• Python 3.9 or later
• Rust toolchain (if building from source)
• maturin (version 1.x; see pyproject.toml for version details)
There are two ways to install pylimbo:
A. Install from PyPI (when available)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pip install pylimbo
B. Build and install from source
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Clone the repository:
git clone https://github.com/penberg/limbo.git
cd limbo
2. Build and install the extension (in “develop” mode so that changes are picked up immediately):
maturin develop
Alternatively, you can build a wheel and install it using pip:
maturin build --release
pip install target/wheels/pylimbo-*.whl
Installation via Makefile
-------------------------
The provided Makefile offers convenience targets to set up your environment. For example, to install all Python dependencies:
make install
This command installs both production and development dependencies from the requirements files.
Quickstart
----------
Below is a basic example that demonstrates how to connect to a database, execute a query, and fetch data.
Example: Connecting to a File‑Based Database
---------------------------------------------
# example.py
import limbo
# Connect to a file‑based database; if the file does not exist, it will be created.
con = limbo.connect("sqlite.db")
# Create a cursor object to run SQL commands
cursor = con.cursor()
# Execute a SQL query; for DDL or DML statements the query runs immediately.
cursor.execute("SELECT * FROM users")
# Fetch the first row from the result set
user = cursor.fetchone()
print("First user:", user)
Example: Using an In‑Memory Database
-------------------------------------
import limbo
# Connect to an in-memory database (data will not be persisted)
con = limbo.connect(":memory:")
cursor = con.cursor()
# Create a new table
cursor.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, username TEXT)")
# Insert a record
cursor.execute("INSERT INTO users VALUES (1, 'alice')")
# Retrieve the data
cursor.execute("SELECT * FROM users")
user = cursor.fetchone()
print("Inserted user:", user)
Detailed API Usage
------------------
Connection
~~~~~~~~~~
• connect(path: str) → Connection
  Establishes a connection to the database at the specified path. Use ":memory:" for an in‑memory database.
• connection.cursor() → Cursor
  Creates a new cursor object for executing SQL statements.
• connection.close()
  Closes the connection (in this version, the connection is released automatically when no longer used).
• connection.commit() and connection.rollback()
  Currently not supported; these methods will raise NotSupportedError as transactions have not yet been implemented.
Cursor
~~~~~~
• cursor.execute(sql: str, parameters: Optional[Tuple[Any, ...]] = None) → Cursor
  Prepares and executes the given SQL statement.
  For DDL (CREATE, ALTER, DROP) and DML (INSERT, UPDATE, DELETE) statements the command is executed immediately.
• cursor.fetchone() → Optional[Tuple[Any, ...]]
  Retrieves the next row of the query result, or returns None if no further data is available.
• cursor.fetchall() → List[Tuple[Any, ...]]
  Retrieves all remaining rows from the query result.
• cursor.executemany(sql: str, parameters: Optional[List[Tuple[Any, ...]]]) and cursor.fetchmany(size: Optional[int])
  Not available in this version; calling these functions will raise NotSupportedError.
Error Handling
~~~~~~~~~~~~~~
pylimbo exposes a set of custom exceptions that mirror standard database errors. These include:
• Warning
• Error
• InterfaceError
• DatabaseError
• DataError
• OperationalError
• IntegrityError
• InternalError
• ProgrammingError
• NotSupportedError
For example, you can handle errors as follows:
from limbo import connect, ProgrammingError
try:
con = connect("non_existing.db")
except ProgrammingError as err:
print("Failed to connect:", err)
Testing and Development
-----------------------
pylimbo includes a robust suite of tests (located in the tests/ directory) to ensure correct behavior. You can run the tests using either the Makefile or pytest directly.
To run tests using the Makefile, execute:
make test
For linting and code formatting, use:
make lint
The Makefile also supports targets for checking and compiling requirements:
• make check-requirements
• make compile-requirements
To set up your development environment:
1. Install development dependencies:
pip install -r requirements.txt -r requirements-dev.txt
2. Build the extension for development:
maturin develop
Contributing
------------
Contributions are welcome! To contribute:
1. Fork the repository.
2. Create your feature branch (git checkout -b feature/your-feature).
3. Write comprehensive tests for your changes.
4. Open a pull request with a detailed explanation of your modifications.
License
-------
pylimbo is distributed under the MIT License. See the LICENSE file for additional details.
Additional Resources
--------------------
• Homepage: https://github.com/penberg/limbo
• Source Code: https://github.com/penberg/limbo
Acknowledgements
----------------
pylimbo is built upon limbo_core and leverages the robust integration capabilities of Rust and pyo3. We appreciate the efforts of the community and contributors who help improve and mature this project. Enjoy using pylimbo in your projects!
Happy coding!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment