Skip to content

Instantly share code, notes, and snippets.

@sss-ng
Last active October 11, 2023 00:30
Show Gist options
  • Save sss-ng/1283c85d1010264132985156c148439b to your computer and use it in GitHub Desktop.
Save sss-ng/1283c85d1010264132985156c148439b to your computer and use it in GitHub Desktop.
why does sqlalchemy UUID succeed with mysql BINARY(16) but fails with postgres BYTEA

Here a docker compose to test

version: '3.4'

volumes:
  pg_vol:
    driver: local
  my_vol:
    driver: local

services:
  db:
    container_name: 'pg'
    image: postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: pass
      POSTGRES_ROOT_PASSWORD: pass
      POSTGRES_USER: user
      POSTGRES_DB: db
    volumes:
      - pg_vol:/var/lib/postgresql/data
    ports:
      - 5432:5432

  db2:
    container_name: 'my'
    image: mysql
    command: --default-authentication-plugin=mysql_native_password
    environment:
      MYSQL_PASSWORD: pass
      MYSQL_ROOT_PASSWORD: pass
      MYSQL_USER: user
      MYSQL_DATABASE: db
    ports:
      - 3306:3306
    volumes:
      - my_vol:/var/lib/mysql

The postgres database is populated

db=# select * from t;
                 id                 
------------------------------------
 \xfb2f42c0670711eea94d0242ac1a0002
 \xfd3b61aa670711eea94d0242ac1a0002
 \xfdaab2c7670711eea94d0242ac1a0002
 \xfe073a8d670711eea94d0242ac1a0002
 \xfe5d9ba4670711eea94d0242ac1a0002

and mysql

mysql> select * from t;
+------------------------------------+
| id                                 |
+------------------------------------+
| 0xFB2F42C0670711EEA94D0242AC1A0002 |
| 0xFD3B61AA670711EEA94D0242AC1A0002 |
| 0xFDAAB2C7670711EEA94D0242AC1A0002 |
| 0xFE073A8D670711EEA94D0242AC1A0002 |
| 0xFE5D9BA4670711EEA94D0242AC1A0002 |

Here is the python script to test

from sqlalchemy import create_engine, Column
from sqlalchemy.orm import sessionmaker, declarative_base
from sqlalchemy_utils import UUIDType
# from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy import UUID

Base = declarative_base()

class T(Base):
    __tablename__ = "t"
    id = Column(UUIDType(binary=True), primary_key=True)
  # id = Column(UUID, primary_key=True)


if __name__ == "__main__":
    engine = create_engine("postgresql://user:pass@localhost:5432/db")
    # engine = create_engine('mysql://user:pass@127.0.0.1:3306/db')
    Session = sessionmaker(bind=engine)
    session = Session()

    # Query the table and print the UUIDs
    results = session.query(T).all()
    for row in results:
        print(row.id)

Postgres

Running this with id = Column(UUIDType(binary=True), primary_key=True)

user@dell:~$ python3 scrap.py 

Traceback (most recent call last):
  File "/home/git/mysql-uuid/scrap.py", line 26, in <module>
    results = session.query(T).all()
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2688, in all
    return self._iter().all()  # type: ignore
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/engine/result.py", line 1786, in all
    return self._allrows()
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/engine/result.py", line 554, in _allrows
    rows = self._fetchall_impl()
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/engine/result.py", line 1693, in _fetchall_impl
    return self._real_result._fetchall_impl()
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/engine/result.py", line 2293, in _fetchall_impl
    return list(self.iterator)
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/orm/loading.py", line 191, in chunks
    fetch = cursor._raw_all_rows()
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/engine/result.py", line 547, in _raw_all_rows
    return [make_row(row) for row in rows]
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/engine/result.py", line 547, in <listcomp>
    return [make_row(row) for row in rows]
  File "lib/sqlalchemy/cyextension/resultproxy.pyx", line 16, in sqlalchemy.cyextension.resultproxy.BaseRow.__init__
  File "lib/sqlalchemy/cyextension/resultproxy.pyx", line 73, in sqlalchemy.cyextension.resultproxy._apply_processors
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy/sql/type_api.py", line 2153, in process
    return fixed_process_value(value, dialect)
  File "/home/.local/lib/python3.10/site-packages/sqlalchemy_utils/types/uuid.py", line 111, in process_result_value
    return uuid.UUID(value)
  File "/usr/lib/python3.10/uuid.py", line 174, in __init__
    hex = hex.replace('urn:', '').replace('uuid:', '')
AttributeError: 'memoryview' object has no attribute 'replace'

Then if i run it with id = Column(UUID, primary_key=True)

user@dell:~$ python3 scrap.py 

<memory at 0x7f40cf98ed40>
<memory at 0x7f40cf98eec0>
<memory at 0x7f40cf98ef80>
<memory at 0x7f40cf98f040>
<memory at 0x7f40cf98f100>
<memory at 0x7f40cf98f1c0>
<memory at 0x7f40cf98f280>
<memory at 0x7f40cf98f340>
<memory at 0x7f40cf98f400>
<memory at 0x7f40cf98f4c0>
<memory at 0x7f40cf98f580>
<memory at 0x7f40cf98f640>
<memory at 0x7f40cf98f700>
<memory at 0x7f40cf98f7c0>
<memory at 0x7f40cf98f880>
<memory at 0x7f40cf98f940>
<memory at 0x7f40cf98fa00>
<memory at 0x7f40cf98fac0>
<memory at 0x7f40cf98fb80>
<memory at 0x7f40cf98fc40>
<memory at 0x7f40cf98fd00>

Mysql

if you run the above with mysql, it succeeds... Im wondering why it succeeds for mysql but not for postgres.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment