Skip to content

Instantly share code, notes, and snippets.

View Eric-Arellano's full-sized avatar

Eric Arellano Eric-Arellano

View GitHub Profile
# Copyright 2020 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Any, ClassVar, Iterable, List, Optional, Type, Union
from pants.engine.rules import UnionRule, union
from pants.util.collections import ensure_str_list
from pants.util.memo import memoized_property
```#!/usr/bin/env python3
import re
from pathlib import Path
def has_multiline(p: Path) -> bool:
text = p.read_text()
has_triple_quotes = bool(re.search('"""', text))
if not has_triple_quotes:
@Eric-Arellano
Eric-Arellano / linter_performance_audit.py
Last active February 17, 2020 07:08
Script to test how V2 linters differ when running in one single invocation vs. a new invocation for each file
#!/usr/bin/env python3.7
"""To run, first run `pex tqdm -o tqdm.pex --interpreter-constraint='CPython==3.7.*'` then run
`./tqdm.pex linter_performance_audit.py`.
"""
import shutil
import subprocess
from dataclasses import dataclass
from pathlib import Path
@Eric-Arellano
Eric-Arellano / flake8-pantsbuild_py37-failure.txt
Last active January 5, 2020 21:20
Ran `pex -vvvvvvvvv flake8 . --entry-point flake8 --interpreter-constraint='CPython>=3.6' --output-file flake8.pex`
This file has been truncated, but you can view the full file.
pex: Building pex
pex: Building pex :: Resolving interpreters
pex: Constraints on interpreters: ['CPython>=3.6'], Matching Interpreter: /Users/eric/.pyenv/versions/3.6.8/bin/python3.6
pex: Building pex :: Resolving interpreters :: Isolating pex
pex: Executing: PYTHONPATH=/Users/eric/.pex/isolated/0edff9f980790256c0e238e65ccca12cffa9085b /Users/eric/.pyenv/versions/3.6.8/bin/python3.6m -s -c import sys
from pex.interpreter import PythonIdentity
sys.stdout.write(PythonIdentity.get().encode())
pex: Could not identify /Users/eric/.pyenv/versions/3.6.8/bin/python3.6m: received exit code 1 during execution of `['/Users/eric/.pyenv/versions/3.6.8/bin/python3.6m', '-s', '-c', 'import sys\nfrom pex.interpreter import PythonIdentity\n\nsys.stdout.write(PythonIdentity.get().encode())\n']` while trying to execute `['/Users/eric/.pyenv/versions/3.6.8/bin/python3.6m', '-s', '-c', 'import sys\nfrom pex.interpreter import PythonIdentity\n\nsys.stdout.write(PythonIdentity.get().encode())\n']`
@Eric-Arellano
Eric-Arellano / contextutil_results.txt
Created June 4, 2019 00:13
Performance of V2 Pytest runner, showing that the requirements PEX causes ~75% of the performance cost.
./pants --no-v1 --v2 test tests/python/pants_test/util:contextutil
time elapsed for pex download: 0.01029515266418457
time elapsed for getting closure: 0.04847383499145508
time elapsed for requirements pex: 8.583088159561157
time elapsed for sources: 0.009785652160644531
time elapsed for __init__: 0.01706719398498535
time elapsed for merging files: 0.0031499862670898438
time elapsed for pytest: 2.261147975921631
total elapsed time: 10.933007955551147
@Eric-Arellano
Eric-Arellano / d.diff
Last active May 16, 2019 16:46
V2 test O(t + b) memory consumption, where t is # targets and b is # of BUILD files
diff --git a/3rdparty/python/requirements.txt b/3rdparty/python/requirements.txt
index e50da50f4..c76c110a9 100644
--- a/3rdparty/python/requirements.txt
+++ b/3rdparty/python/requirements.txt
@@ -10,6 +10,7 @@ fasteners==0.14.1
faulthandler==2.6 ; python_version<'3'
future==0.17.1
futures==3.0.5 ; python_version<'3'
+objgraph
Markdown==2.1.1
size: 1056, obj: <class 'pants.engine.fs.Digest'>
size: 1056, obj: <class 'pants.engine.fs.Snapshot'>
size: 1056, obj: <class 'pants.engine.fs.FileContent'>
size: 1056, obj: <class 'abc.Collection.of(FileContent)'>
size: 1056, obj: <class 'pants.engine.isolated_process.FallibleExecuteProcessResult'>
size: 493, obj: b'============================= test session starts ==============================\nplatform darwin -- Python 3.6.8, pytest-3.6.4, py-1.8.0, pluggy-0.7.1\nrootdir: /Users/eric/DocsLocal/code/projects/pants/.pants.d/process-executionBU0vrV, inifile:\nplugins: cov-2.4.0, timeout-1.2.1\ncollected 5 items\n\npants_test/util/test_strutil.py ..... [100%]\n\n=========================== 5 passed in 0.04 seconds ===========================\n'
size: 1022, obj: b"# Copyright 2014 Pants project contributors (see CONTRIBUTORS.md).\n# Licensed under the Apache License, Version 2.0 (see LICENSE).\n\n# see/edit requirements.txt in this directory to change deps.\npython_requiremen
16:33:50 [INFO] setting up service <pants.pantsd.service.fs_event_service.FSEventService object at 0x1116f82b0>
16:33:50 [INFO] setting up service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x10fca4ef0>
16:33:50 [INFO] watching invalidating files: ('contrib/avro/src/python/pants/__init__.py', 'contrib/avro/src/python/pants/__init__.pyc', 'contrib/avro/src/python/pants/__pycache__/__init__.cpython-36.pyc', 'contrib/avro/src/python/pants/__pycache__/__init__.cpython-37.pyc', 'contrib/avro/src/python/pants/contrib/__init__.py', 'contrib/avro/src/python/pants/contrib/__init__.pyc', 'contrib/avro/src/python/pants/contrib/__pycache__/__init__.cpython-36.pyc', 'contrib/avro/src/python/pants/contrib/__pycache__/__init__.cpython-37.pyc', 'contrib/avro/src/python/pants/contrib/avro/BUILD', 'contrib/avro/src/python/pants/contrib/avro/__init__.py', 'contrib/avro/src/python/pants/contrib/avro/__init__.pyc', 'contrib/avro/src/python/pants/contrib/avro/__pycache__/__init__.cpython-36.pyc', 'contrib/a
14:00:17 00:14 [pytest]
Invalidated 19 targets.
scrubbed PYTHONPATH=/Users/eric/DocsLocal/code/projects/pants/src/python: from py.test environment
14:00:17 00:14 [run]
.......................................... [ 18%]
.......................................... [ 37%]
.......................................... [ 56%]
.......................................... [ 75%]
....................................F..... [ 94%]
............. [100%]
20:14:16 00:16 [pytest]
Invalidated 19 targets.
scrubbed PYTHONPATH=/Users/eric/DocsLocal/code/projects/pants/src/python: from py.test environment
20:14:16 00:16 [run]
============== test session starts ===============
platform darwin -- Python 3.6.8, pytest-3.6.4, py-1.7.0, pluggy-0.7.1 -- /Users/eric/.pyenv/versions/3.6.8/bin/python3.6
cachedir: .pants.d/.pytest_cache
rootdir: /Users/eric/DocsLocal/code/projects/pants/.pants.d, inifile: /Users/eric/DocsLocal/code/projects/pants/.pants.d/test/pytest-prep/CPython-3.6.8/ef674d9b47ad4c4617aa75e878049022ce988507/pytest.ini
plugins: cov-2.4.0, timeout-1.2.1
collecting ... collected 223 items