Skip to content

Instantly share code, notes, and snippets.

View twobraids's full-sized avatar

K Lars Lohn twobraids

View GitHub Profile
ennis:…vct ᚥ ./create-test-environment
wrote 1977229 bytes to /home/lars/work/wf/vct/venv/virtualenv-16.1.0.tar.gz
wrote 12154 bytes to /home/lars/work/wf/vct/venv/virtualenv-16.1.0/src/virtualenv_support/appdirs-1.4.3-py2.py3-none-any.whl
wrote 21083 bytes to /home/lars/work/wf/vct/venv/virtualenv-16.1.0/src/virtualenv_support/packaging-18.0-py2.py3-none-any.whl
wrote 1323545 bytes to /home/lars/work/wf/vct/venv/virtualenv-16.1.0/src/virtualenv_support/pip-18.1-py2.py3-none-any.whl
wrote 59028 bytes to /home/lars/work/wf/vct/venv/virtualenv-16.1.0/src/virtualenv_support/pyparsing-2.3.0-py2.py3-none-any.whl
wrote 569846 bytes to /home/lars/work/wf/vct/venv/virtualenv-16.1.0/src/virtualenv_support/setuptools-40.5.0-py2.py3-none-any.whl
wrote 10702 bytes to /home/lars/work/wf/vct/venv/virtualenv-16.1.0/src/virtualenv_support/six-1.11.0-py2.py3-none-any.whl
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in /home/lars/work/wf/vct/venv/bin/python2.7
@twobraids
twobraids / example in use on the command line
Created February 19, 2016 23:39
an example of getting configuration information into a file
$ python t0.py --help
usage:
t0.py [OPTIONS]...
OPTIONS:
--admin.conf
the pathname of the config file (path/filename)
--admin.dump_conf
#==============================================================================
class ConnectionContextWithHost(S3ConnectionContext):
"""an alternative base class that specific implementations of Boto
connection can derive. It adds "host" to the configuration"""
required_config = Namespace()
required_config.add_option(
'host',
doc="The hostname",
default="",
@twobraids
twobraids / argument collapsing
Created October 16, 2015 00:08
config change for arguments in signatures
consulate kv set socorro/processor/resource.signature.collapse_arguments True
@twobraids
twobraids / socorro_app_as_crontabber_app.py
Created October 12, 2015 21:43
socorro apps as crontabber apps
from crontabber.base import BaseCronApp
from crontabber.mixins import as_backfill_cron_app
from configman import Namespace, class_converter
@as_backfill_cron_app
class SocorroAppAsCrontabberApp(BaseCronApp):
required_config = Namespace()
required_config.add_option(
@twobraids
twobraids / ERROR on master
Created September 24, 2015 15:03
unit test failure on master
======================================================================
ERROR: test_get_with_zero (socorro.unittest.external.es.test_supersearch.IntegrationTestSuperSearch)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/lars/socorro/socorro/unittest/external/es/test_supersearch.py", line 1457, in test_get_with_zero
_results_number=0,
File "/home/lars/socorro/socorro/external/es/supersearch.py", line 503, in get
results = search.execute()
File "/home/lars/socorro/socorro-virtualenv/local/lib/python2.7/site-packages/elasticsearch_dsl/search.py", line 543, in execute
**self._params
@twobraids
twobraids / gist:82496fcb2a52ccaaf1d2
Created September 2, 2015 22:08
this is even stranger. a return value of 0 from the position function apparently means "not found"
breakpad=> select uuid, position('\u0000' in raw_crash::text) from raw_crashes_20150824 where raw_crash::text like '%\u0000%';
uuid | position
--------------------------------------+----------
77c9e17c-39b8-409e-be5b-b63982150825 | 0
e3e9f28b-eff4-48fb-b4aa-bb8822150826 | 0
05a5a22e-21d7-4b85-aef0-3de3a2150828 | 0
b1b54310-402a-4bff-8cc2-f2c9b2150829 | 0
(4 rows)
@twobraids
twobraids / gist:13169bf340ef3152ff97
Created September 2, 2015 21:34
how could this possibly be true?
breakpad=> select uuid, position('\u0000' in raw_crash::text) from raw_crashes_20150831 where raw_crash::text like '\u0000%';
uuid | position
------+----------
(0 rows)
breakpad=> select uuid, position('\u0000' in raw_crash::text) from raw_crashes_20150831 where raw_crash::text like '%\u0000%';
uuid | position
--------------------------------------+----------
808ed71a-499b-45d9-ae86-7d9642150831 | 0
@twobraids
twobraids / gist:091323605cf713068345
Created September 2, 2015 19:27
this looks reasonable
import datetime
import mock
op = mock.Mock()
today = datetime.datetime.utcnow().date()
for i in range(28, 0, -1):
then = today - datetime.timedelta(days=i)
till = then + datetime.timedelta(days=1)
@twobraids
twobraids / reprocessing
Created August 24, 2015 22:54
push directly into reprocessing queue
$ . /data/socorro/socorro-virtualenv/bin/activate
$ envconsul -prefix socorro/common -prefix socorro/processor socorro submitter --destination.crashstorage_class=socorro.external.rabbitmq.crashstorage.RabbitMQCrashStorage --destination.routing_key=socorro.reprocessing --source.temporary_file_system_storage_path=./--new_crash_source.new_crash_source_class=socorro.collector.submitter_app.DBSamplingCrashSource --new_crash_source.sql="select 'ce8410a1-b080-4c77-8d80-e64fb2150817'"