dmtucker/1.17

Last active April 25, 2020 04:07

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/dmtucker/bffe536fabd27b5ebf43db112a89fa6c.js"></script>
Save dmtucker/bffe536fabd27b5ebf43db112a89fa6c to your computer and use it in GitHub Desktop.

Download ZIP

Raw

1.17

What's up with Python 1.17 in the PyPI stats database?

Author

dmtucker commented Mar 5, 2018 •

edited

Where does 1.17 come from?

Standard urllib seems like the most likely source (which is present in at least one fork):

$ python -c 'import sys, urllib, urllib2;  print(sys.version[:3], urllib.__version__, urllib2.__version__)'; python3 -c 'import urllib.request; print(urllib.request.__version__)'
('2.7', '1.17', '2.7')
3.5

PyPI associates urllib with the Python version.
That would make this issue specific to Python 2.
- The top downloads all seem to support it still.
- urllib3 is also a possibility.
  - By default, urllib3 does not set a user agent.

What agents may be using urllib?

bandersnatch?
conda?
easy_install switched to urllib2 in 0.4a2.
pip?
requests vendors urllib3 and skipped 1.17.

Author

dmtucker commented Mar 6, 2018

Where is 1.17 being used from?

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 365 --where 'details.python = "1.17"' '' country
Served from cache: False
Data processed: 81.80 GiB
Data billed: 81.80 GiB
Estimated cost: $0.40

| country | download_count |
| ------- | -------------- |
| US      |      3,216,292 |
| JP      |         55,160 |
| CN      |         34,118 |
| GB      |         24,925 |
| IN      |          8,064 |
| None    |          7,190 |
| FR      |          6,131 |
| DE      |          5,703 |
| IL      |          5,109 |
| ES      |          4,710 |

Author

dmtucker commented Mar 6, 2018 •

edited

Notes

Switching PyPI to HTTPS-only should mean that occurrences dropped in Oct/Nov 2017 (but they didn't).

urllib.urlretrieve uses FancyURLopener which inherits from URLopener which sets the User-Agent.

easy_install's user-agent logic:
https://github.com/pypa/setuptools/blob/97ff22f31ace57f4eabb6f1e77c9c553de0d1c24/setuptools/package_index.py#L50

log parser user agent logic:
https://github.com/pypa/linehaul/blob/master/linehaul/user_agents.py

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 365 --where 'details.python = "1.17"' '' system distro
Served from cache: False
Data processed: 146.60 GiB
Data billed: 146.60 GiB
Estimated cost: $0.72

| system_name | distro_name | download_count |
| ----------- | ----------- | -------------- |
| None        | None        |      3,405,944 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 365 --where 'details.python = "1.17"' '' installer installer-version
Served from cache: False
Data processed: 153.04 GiB
Data billed: 153.04 GiB
Estimated cost: $0.75

| installer_name | installer_version | download_count |
| -------------- | ----------------- | -------------- |
| None           | None              |      3,405,944 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 365 --where 'details.python = "1.17"' '' impl impl-version
Served from cache: False
Data processed: 155.91 GiB
Data billed: 155.91 GiB
Estimated cost: $0.77

| implementation | impl_version | download_count |
| -------------- | ------------ | -------------- |
| None           | None         |      3,405,944 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 365 --where 'details.python = "1.17"' '' setuptools-version
Served from cache: False
Data processed: 51.03 GiB
Data billed: 51.03 GiB
Estimated cost: $0.25

| setuptools_version | download_count |
| ------------------ | -------------- |
| None               |      3,405,944 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 365 --where 'details.python = "1.17"' '' openssl
Served from cache: False
Data processed: 173.99 GiB
Data billed: 173.99 GiB
Estimated cost: $0.85

| openssl_version | download_count |
| --------------- | -------------- |
| None            |      3,405,944 |

Python Version Trends

None of these seem to correspond with 1.17 trends:

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 1095 --limit 36 --order download_month --where 'details.python = "2.6"' '' month
Served from cache: False
Data processed: 186.30 GiB
Data billed: 186.30 GiB
Estimated cost: $0.91

| download_month | download_count |
| -------------- | -------------- |
| 2018-03        |        146,529 |
| 2018-02        |        828,609 |
| 2018-01        |      1,141,411 |
| 2017-12        |      1,614,495 |
| 2017-11        |      1,371,796 |
| 2017-10        |      2,023,681 |
| 2017-09        |      2,289,735 |
| 2017-08        |      2,704,477 |
| 2017-07        |      2,828,229 |
| 2017-06        |      2,889,239 |
| 2017-05        |      2,043,001 |
| 2017-04        |      2,457,703 |
| 2017-03        |      2,680,095 |
| 2017-02        |      2,576,469 |
| 2017-01        |      3,066,075 |
| 2016-12        |      3,053,485 |
| 2016-11        |      3,591,192 |
| 2016-10        |      3,270,814 |
| 2016-09        |      3,209,299 |
| 2016-08        |      4,005,600 |
| 2016-07        |      3,946,704 |
| 2016-06        |      4,171,922 |
| 2016-05        |      2,034,229 |
| 2016-03        |      1,815,018 |
| 2016-02        |      9,519,408 |
| 2016-01        |      2,959,021 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 1095 --limit 36 --order download_month --where 'details.python = "2.7"' '' month
Served from cache: False
Data processed: 186.30 GiB
Data billed: 186.30 GiB
Estimated cost: $0.91

| download_month | download_count |
| -------------- | -------------- |
| 2018-03        |      4,668,812 |
| 2018-02        |     22,073,881 |
| 2018-01        |     30,490,386 |
| 2017-12        |     31,818,101 |
| 2017-11        |     30,530,294 |
| 2017-10        |     42,417,489 |
| 2017-09        |     47,877,172 |
| 2017-08        |     61,447,951 |
| 2017-07        |     63,732,403 |
| 2017-06        |     66,186,931 |
| 2017-05        |     68,284,527 |
| 2017-04        |     71,634,346 |
| 2017-03        |     75,668,717 |
| 2017-02        |     72,209,408 |
| 2017-01        |     80,589,481 |
| 2016-12        |     71,611,590 |
| 2016-11        |     78,672,198 |
| 2016-10        |     68,610,169 |
| 2016-09        |     74,346,875 |
| 2016-08        |     72,597,214 |
| 2016-07        |     62,769,844 |
| 2016-06        |     50,753,857 |
| 2016-05        |     17,155,656 |
| 2016-03        |      9,538,368 |
| 2016-02        |     48,248,300 |
| 2016-01        |     15,114,515 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 1095 --limit 36 --order download_month --where 'details.python = "3.4"' '' month
Served from cache: False
Data processed: 186.30 GiB
Data billed: 186.30 GiB
Estimated cost: $0.91

| download_month | download_count |
| -------------- | -------------- |
| 2018-03        |        453,018 |
| 2018-02        |      1,966,827 |
| 2018-01        |      2,435,945 |
| 2017-12        |      2,517,948 |
| 2017-11        |      2,310,739 |
| 2017-10        |      2,491,198 |
| 2017-09        |      2,332,760 |
| 2017-08        |      2,697,636 |
| 2017-07        |      2,397,228 |
| 2017-06        |      1,834,341 |
| 2017-05        |      1,863,789 |
| 2017-04        |      1,848,761 |
| 2017-03        |      2,238,398 |
| 2017-02        |      1,794,172 |
| 2017-01        |      1,793,060 |
| 2016-12        |      1,519,826 |
| 2016-11        |      1,539,866 |
| 2016-10        |      1,660,522 |
| 2016-09        |      1,602,485 |
| 2016-08        |      1,999,833 |
| 2016-07        |      1,976,705 |
| 2016-06        |        792,186 |
| 2016-05        |        363,550 |
| 2016-03        |        188,615 |
| 2016-02        |        977,270 |
| 2016-01        |        258,988 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 1095 --limit 36 --order download_month --where 'details.python = "3.5"' '' month
Served from cache: False
Data processed: 186.30 GiB
Data billed: 186.30 GiB
Estimated cost: $0.91

| download_month | download_count |
| -------------- | -------------- |
| 2018-03        |        419,007 |
| 2018-02        |      2,124,505 |
| 2018-01        |      2,331,339 |
| 2017-12        |      2,649,399 |
| 2017-11        |      2,759,223 |
| 2017-10        |      2,312,185 |
| 2017-09        |      2,016,890 |
| 2017-08        |      2,438,073 |
| 2017-07        |      2,391,372 |
| 2017-06        |      3,384,902 |
| 2017-05        |      2,832,727 |
| 2017-04        |      3,037,491 |
| 2017-03        |      2,466,478 |
| 2017-02        |      2,367,826 |
| 2017-01        |      2,417,595 |
| 2016-12        |      2,464,993 |
| 2016-11        |      1,820,581 |
| 2016-10        |      2,006,853 |
| 2016-09        |      1,761,309 |
| 2016-08        |      1,242,226 |
| 2016-07        |      1,004,178 |
| 2016-06        |        603,266 |
| 2016-05        |        333,189 |
| 2016-03        |         89,432 |
| 2016-02        |        489,781 |
| 2016-01        |        105,938 |

(david-YYe3LIIa) david@kahuna:~ $ pypinfo --days 1095 --limit 36 --order download_month --where 'details.python = "3.6"' '' month
Served from cache: False
Data processed: 186.30 GiB
Data billed: 186.30 GiB
Estimated cost: $0.91

| download_month | download_count |
| -------------- | -------------- |
| 2018-03        |        460,217 |
| 2018-02        |      2,271,910 |
| 2018-01        |      2,236,416 |
| 2017-12        |      2,251,264 |
| 2017-11        |      5,606,128 |
| 2017-10        |      2,226,206 |
| 2017-09        |      2,054,285 |
| 2017-08        |      2,339,605 |
| 2017-07        |      1,661,571 |
| 2017-06        |      1,726,659 |
| 2017-05        |      1,152,307 |
| 2017-04        |        902,785 |
| 2017-03        |        792,730 |
| 2017-02        |        580,227 |
| 2017-01        |        391,527 |
| 2016-12        |         71,144 |
| 2016-11        |         26,199 |
| 2016-10        |         23,336 |
| 2016-09        |         23,020 |
| 2016-08        |         23,118 |
| 2016-07        |         20,472 |
| 2016-06        |         10,369 |
| 2016-05        |          3,076 |
| 2016-03        |          1,319 |
| 2016-02        |          7,425 |
| 2016-01        |          1,597 |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment