Skip to content

Instantly share code, notes, and snippets.

View jdavcs's full-sized avatar
🐧

John Davis jdavcs

🐧
  • Johns Hopkins University
  • Baltimore, MD
View GitHub Profile
@jdavcs
jdavcs / test_change_state.py
Last active March 24, 2020 18:07
Performance test for galaxy's JobWrapper change_state()
from scripts.db_shell import *
def create_jobs_with_datasets(n, datasets_per_job):
jobs = []
for i in range(n):
job = Job()
jobs.append(job)
for j in range(datasets_per_job):
ds = Dataset(state=Dataset.states.NEW)
@jdavcs
jdavcs / 0_indexes.py
Last active September 30, 2020 20:49
compare indexes in db and defined in mapping.py
def test_indexes(self):
from sqlalchemy import create_engine, MetaData
from galaxy.model import mapping
# metadata loaded from mapping.py
md1 = mapping.metadata
ix1 = set()
for t in md1.tables.values():
ix1 |= t.indexes # mapping only: 422
@jdavcs
jdavcs / check_model.py
Created October 1, 2020 01:43
Compare db indexes defined in mapping.py to those in the database. Call check_model.sh from galaxy root.
import os
import sys
sys.path.insert(1, os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir, 'lib')))
from sqlalchemy import create_engine, MetaData
from galaxy.model import mapping
from galaxy.model.orm.scripts import get_config
@jdavcs
jdavcs / poetry.out
Created March 24, 2021 18:10
`poetry lock -vvv` after adding `circus = "*"`
Using virtualenv: /home/sergey/0dev/galaxy/_galaxy/dev-deps-nosqlalchemyutils/.venv
Updating dependencies
Resolving dependencies...
1: fact: galaxy is 21.01
1: derived: galaxy
1: fact: galaxy depends on aiofiles (*)
1: fact: galaxy depends on async-generator (*)
1: fact: galaxy depends on async-exit-stack (*)
1: fact: galaxy depends on Babel (*)
1: fact: galaxy depends on bdbag (*)
@jdavcs
jdavcs / sa_defer_simple.py
Created April 4, 2021 16:42
SA deferred load possible bug / basic demo
from sqlalchemy import (
create_engine,
inspect,
Column,
ForeignKey,
Integer
)
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import (
defer,
@jdavcs
jdavcs / sa_defer_galaxy.py
Created April 4, 2021 16:43
SA deferred load possible bug / galaxy simplified example, classical mapping
from sqlalchemy import (
create_engine,
inspect,
Column,
ForeignKey,
Integer,
MetaData,
String,
Table,
)
@jdavcs
jdavcs / gist:ad7f31bd780ea9c8abd87f3ab480bb0c
Last active April 20, 2021 05:20
pre and post flush difference between SA 1.3 and 1.4 at checkpoint 4
PRE-FLUSH:
(Pdb) pp items
<galaxy.model.MetadataFile(None) at 0x7fb433652b20>,
<galaxy.model.Job(1) at 0x7fb4336496a0>,
<galaxy.model.User(1) at 0x7fb4336432e0>,
<galaxy.model.JobToOutputDatasetAssociation(1) at 0x7fb4336a5cd0>,
<galaxy.model.HistoryDatasetAssociation(1) at 0x7fb4336a5880>,
<galaxy.model.Dataset(1) at 0x7fb4336a5d00>
@jdavcs
jdavcs / subquery_vs_no_subquery.py
Created May 21, 2021 04:28
subquery load options
from sqlalchemy import (
create_engine,
inspect,
Column,
ForeignKey,
Integer,
MetaData,
String,
Table,
)
@jdavcs
jdavcs / gist:331bfc06c029a9d125b44001e150c240
Created September 22, 2021 17:09
UI performance data
history,pairs,ui click,ui screen done,ui total,request 1,request 2,request total
old,10,860,969,109,52,18,70
old,10,779,884,105,55,17,72
old,10,720,821,101,52,18,70
old,10,670,781,111,51,18,69
old,10,671,769,98,39,14,53
old,10,870,971,101,55,12,67
old,10,563,665,102,38,17,55
old,10,649,729,80,37,11,48
old,10,311,397,86,42,19,61
1 ==================================================================================
2 TESTING: new db/sqlite, pre-conflict dev (current dev minus 3a29, caa7)
3 ==================================================================================
4 - manage_db.sh init
5
6 - manage_db.sh version
7 9540a051226e (gxy) (head)
8 d4a650f47a3c (tsi) (head)
9 - manage_db.sh dbversion
10 9540a051226e (head)