Skip to content

Instantly share code, notes, and snippets.

@jalessio

jalessio/openaddresses-machine-test-1480613168.log

Created December 1, 2016 17:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jalessio/7288a86e11c3c1c0dc04a8310dba1911 to your computer and use it in GitHub Desktop.
Save jalessio/7288a86e11c3c1c0dc04a8310dba1911 to your computer and use it in GitHub Desktop.
Download ZIP
output after running clean install on branch migurski/fix-botocore-dependency from https://github.com/openaddresses/machine/pull/469
Raw
openaddresses-machine-test-1480613168.log
/usr/local/lib/python3.4/dist-packages/pq/utils.py:132: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn(notice)
...........2016-12-01 17:26:12,823 ERROR: Failed to check merged status of openaddresses/openaddresses gobbledygook: 'base_commit'
..........................2016-12-01 17:26:33,171 WARNING: Skipping file with unknown conform: test
2016-12-01 17:26:33,172 WARNING: Skipping file with unknown conform: test
..................2016-12-01 17:26:33,206 WARNING: Conform named bar.txt as file but we could not find it.
..2016-12-01 17:26:33,208 WARNING: No GDB found in ['nope.txt']
2016-12-01 17:26:33,208 WARNING: Multiple GDBs found, but source has no file attribute.
.2016-12-01 17:26:33,210 WARNING: No JSON found in ['nope.txt']
2016-12-01 17:26:33,210 WARNING: Found more than one JSON file in source, can't pick one
.2016-12-01 17:26:33,211 WARNING: No shapefiles found in ['nope.txt']
2016-12-01 17:26:33,212 WARNING: Multiple shapefiles found, but source has no file attribute.
2016-12-01 17:26:33,212 WARNING: Unknown source conform type broken
.....................................2016-12-01 17:26:34,081 WARNING: Found http://s3.amazonaws.com/data-testpreviews.openaddresses.io/runs/10/preview.png in an existing comment: https://api.github.com/repos/openaddresses/hooked-on-sources/issues/comments/259845585
...2016-12-01 17:26:34,310 WARNING: Mismatched /hook signatures: sha1=79c627d0a3f18ffb913fbbca70f6d9d983aa7c83 vs. sha1=80c4b24a90baa10a63811bd05f396b658a5759bc, sha1=94c05fcb02b3790f99dd17248070863c63ac685c
. 20561 615.6 WARNING: Error in process_one.process()
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/openaddr/process_one.py", line 51, in process
if json.load(file).get('skip', None):
File "/usr/lib/python3.4/json/__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.4/json/decoder.py", line 359, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting ',' delimiter: line 12 column 5 (char 242)
20561 627.1 ERROR: Expecting ',' delimiter: line 12 column 5 (char 242)
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/openaddr/process_one.py", line 288, in main
file_path = process(source, destination, args.render_preview, mapzen_key=args.mapzen_key)
File "/usr/local/lib/python3.4/dist-packages/openaddr/process_one.py", line 95, in process
cache_result, conform_result, preview_path, temp_dir)
File "/usr/local/lib/python3.4/dist-packages/openaddr/process_one.py", line 206, in write_state
source_data = json.load(file)
File "/usr/lib/python3.4/json/__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.4/json/decoder.py", line 359, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting ',' delimiter: line 12 column 5 (char 242)
20561 629.9 INFO: process shutting down
2016-12-01 17:26:40,656 ERROR: Failed to check merged status of openaddresses/hooked-on-sources badjson20f08890960f50f863626e1062f922522: 'base_commit'
...2016-12-01 17:26:41,051 WARNING: No files
...2016-12-01 17:26:41,360 WARNING: No /hook signature provided
.2016-12-01 17:26:41,700 ERROR: Failed to check merged status of openaddresses/hooked-on-sources e91fbc420f08890960f50f863626e1062f922522: Expecting ',' delimiter or '}': line 1 column 64956 (char 64955)
...2016-12-01 17:26:42,175 ERROR: Oops
Traceback (most recent call last):
File "/home/vagrant/machine/openaddr/ci/__init__.py", line 117, in process_github_payload
is_rerun, owner, repo, status_url, comments_url)
File "/home/vagrant/machine/openaddr/ci/__init__.py", line 735, in create_queued_job
task_files = add_files_to_queue(queue, job_id, job_url, files, commit_sha, rerun)
File "/usr/local/lib/python3.4/dist-packages/mock/mock.py", line 1062, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/mock/mock.py", line 1128, in _mock_call
ret_val = effect(*args, **kwargs)
File "/home/vagrant/machine/openaddr/tests/ci.py", line 1651, in job_queue_fails
raise Exception('Nope')
Exception: Nope
.2016-12-01 17:26:42,419 ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
2016-12-01 17:26:42,480 ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
.2016-12-01 17:26:42,727 ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
2016-12-01 17:26:42,791 ERROR: Failed to check merged status of openaddresses/hooked-on-sources ded44ed5f1733bb93d84f94afe9383e2d47bbbaa: Expecting ',' delimiter or '}': line 1 column 57572 (char 57571)
...2016-12-01 17:26:47,241 WARNING: Source is missing a conform object
2016-12-01 17:26:47,241 WARNING: Error doing conform; skipping
Traceback (most recent call last):
File "/home/vagrant/machine/openaddr/__init__.py", line 185, in conform
csv_path, addr_count = task4.convert(data, decompressed_paths, workdir)
File "/home/vagrant/machine/openaddr/conform.py", line 521, in convert
source_path = find_source_path(source_definition, source_paths)
File "/home/vagrant/machine/openaddr/conform.py", line 404, in find_source_path
conform = source_definition["conform"]
KeyError: 'conform'
2016-12-01 17:26:47,251 WARNING: Nothing processed
.2016-12-01 17:26:47,283 WARNING: Could not download source data
2016-12-01 17:26:47,284 WARNING: Error in process_one.process()
Traceback (most recent call last):
File "/home/vagrant/machine/openaddr/process_one.py", line 56, in process
cache_result = cache(temp_src, temp_dir, extras)
File "/home/vagrant/machine/openaddr/__init__.py", line 109, in cache
downloaded_files = task.download(source_urls, workdir, data.get('conform'))
File "/home/vagrant/machine/openaddr/cache.py", line 280, in download
raise DownloadError('{} response from {}'.format(resp.status_code, source_url))
openaddr.cache.DownloadError: 404 response from http://www.ci.berkeley.ca.us/uploadedFiles/IT/GIS/No-Parcels.zip
.2016-12-01 17:26:47,975 WARNING: Nothing previewed
..2016-12-01 17:26:48,094 WARNING: Nothing previewed
.2016-12-01 17:26:48,140 WARNING: Nothing previewed
.2016-12-01 17:26:48,173 WARNING: Nothing previewed
.2016-12-01 17:26:48,240 WARNING: Nothing previewed
.2016-12-01 17:26:48,307 WARNING: Nothing previewed
.2016-12-01 17:26:48,352 WARNING: Nothing previewed
.2016-12-01 17:26:48,394 WARNING: Error doing conform; skipping
Traceback (most recent call last):
File "/home/vagrant/machine/openaddr/__init__.py", line 185, in conform
csv_path, addr_count = task4.convert(data, decompressed_paths, workdir)
File "/home/vagrant/machine/openaddr/conform.py", line 525, in convert
rc = conform_cli(source_definition, source_path, dest_path)
File "/home/vagrant/machine/openaddr/conform.py", line 1131, in conform_cli
extract_to_source_csv(source_definition, source_path, extract_path)
File "/home/vagrant/machine/openaddr/conform.py", line 1080, in extract_to_source_csv
csv_source_to_csv(source_definition, source_path, extract_path)
File "/home/vagrant/machine/openaddr/conform.py", line 684, in csv_source_to_csv
with csvopen(source_path, 'r', encoding=enc) as source_fp:
File "/home/vagrant/machine/openaddr/compat.py", line 125, in csvopen
return io.open(filename, mode=mode, encoding=encoding)
IsADirectoryError: [Errno 21] Is a directory: '/tmp/testOA-3_9t1jhi/process_one-oldi8s1i/conform-mc8qcmat/unzipped/07000-12.0a'
2016-12-01 17:26:48,395 WARNING: Nothing processed
.2016-12-01 17:26:48,481 WARNING: Nothing previewed
.2016-12-01 17:26:48,539 WARNING: Nothing previewed
.2016-12-01 17:26:48,596 WARNING: Nothing previewed
.2016-12-01 17:26:48,656 WARNING: Nothing previewed
.2016-12-01 17:26:48,697 WARNING: Nothing previewed
.2016-12-01 17:26:48,754 WARNING: Nothing previewed
.2016-12-01 17:26:48,805 WARNING: Unknown source conform type broken
2016-12-01 17:26:48,807 WARNING: Nothing processed
../home/vagrant/machine/openaddr/conform.py:611: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
_L.warn("No encoding given and OGR couldn't guess. Trying ISO-8859-1, YOLO!")
2016-12-01 17:26:48,855 WARNING: No encoding given and OGR couldn't guess. Trying ISO-8859-1, YOLO!
2016-12-01 17:26:49,335 WARNING: Nothing previewed
.2016-12-01 17:26:49,558 WARNING: Nothing previewed
.2016-12-01 17:26:49,614 WARNING: Nothing previewed
.2016-12-01 17:26:50,360 WARNING: Nothing previewed
.2016-12-01 17:26:50,652 WARNING: Nothing previewed
..2016-12-01 17:26:54,777 WARNING: Error doing excerpt; skipping
Traceback (most recent call last):
File "/home/vagrant/machine/openaddr/__init__.py", line 176, in conform
data_sample, geometry_type = task3.excerpt(decompressed_paths, workdir, conform)
File "/home/vagrant/machine/openaddr/conform.py", line 280, in excerpt
return ExcerptDataTask._excerpt_csv_file(data_path, encoding, csvsplit)
File "/home/vagrant/machine/openaddr/conform.py", line 357, in _excerpt_csv_file
data_sample = [row for (row, _) in zip(input, range(6))]
File "/home/vagrant/machine/openaddr/conform.py", line 357, in <listcomp>
data_sample = [row for (row, _) in zip(input, range(6))]
_csv.Error: field larger than field limit (1)
2016-12-01 17:26:54,782 WARNING: Error doing conform; skipping
Traceback (most recent call last):
File "/home/vagrant/machine/openaddr/__init__.py", line 185, in conform
csv_path, addr_count = task4.convert(data, decompressed_paths, workdir)
File "/home/vagrant/machine/openaddr/conform.py", line 525, in convert
rc = conform_cli(source_definition, source_path, dest_path)
File "/home/vagrant/machine/openaddr/conform.py", line 1131, in conform_cli
extract_to_source_csv(source_definition, source_path, extract_path)
File "/home/vagrant/machine/openaddr/conform.py", line 1085, in extract_to_source_csv
csv_source_to_csv(source_definition, source_path, extract_path)
File "/home/vagrant/machine/openaddr/conform.py", line 713, in csv_source_to_csv
num_fields = len(reader.fieldnames)
File "/usr/lib/python3.4/csv.py", line 96, in fieldnames
self._fieldnames = next(self.reader)
_csv.Error: field larger than field limit (1)
2016-12-01 17:26:54,786 WARNING: Nothing processed
2016-12-01 17:26:54,803 WARNING: Nothing previewed
.2016-12-01 17:26:54,843 WARNING: Nothing previewed
.2016-12-01 17:26:54,953 WARNING: Nothing previewed
.2016-12-01 17:26:55,000 WARNING: Nothing previewed
....................................2016-12-01 17:26:55,070 ERROR: Failed to download http://s3.amazonaws.com/openaddresses/000.csv
...../home/vagrant/machine/openaddr/parcels/utils.py:73: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/vagrant/machine/openaddr/tests/parcels/sources/us/ca/berkeley.json' mode='r' encoding='UTF-8'>
source_json = json.loads(open('{}/sources/{}'.format(config.openaddr_dir, source)).read())
..../home/vagrant/machine/openaddr/parcels/utils.py:88: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/vagrant/machine/openaddr/tests/parcels/sources/us/ca/berkeley.json' mode='r' encoding='UTF-8'>
source_json = json.loads(open('{}/sources/{}'.format(config.openaddr_dir, source)).read())
........./usr/lib/python3/dist-packages/osgeo/ogr.py:2970: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/output-6eud0mep/us/ca/berkeley.csv' mode='w' encoding='UTF-8'>
self.__dict__[key] = value
...........................2016-12-01 17:27:19,128 WARNING: Got an error from ftp02.portlandoregon.gov: 'NoneType' does not support the buffer interface
2016-12-01 17:27:19,133 WARNING: Got an error from ftp02.portlandoregon.gov: 'NoneType' does not support the buffer interface
........
----------------------------------------------------------------------
Ran 232 tests in 69.733s
OK
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/65018/us/ca/alameda.zip
http://s3.amazonaws.com/data.openaddresses.io/runs/117966/us/ca/santa_clara.zip
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment