Skip to content

Instantly share code, notes, and snippets.

@tirkarthi
Last active February 18, 2020 16:21
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tirkarthi/fc038c383592344cbb087d16a98515de to your computer and use it in GitHub Desktop.
Save tirkarthi/fc038c383592344cbb087d16a98515de to your computer and use it in GitHub Desktop.
Python 3.8 and 3.9 issues

Do take one issue per person to help others :) Also checkout Fedora tracker for more easy issues : https://bugzilla.redhat.com/show_bug.cgi?id=1785415 . I tried to mark the issues complete but it's more maintenance so sorry if the issue is fixed or already has a PR and is not updated here.

@rec
Copy link

rec commented Jan 18, 2020

I'm taking the first three - in the nltk toolkit.

(Because it makes more sense for one person to do all the issues in one project as most of the time is setting up...)

@serhiy-storchaka
Copy link

2 and 3 has the same link.

@tirkarthi
Copy link
Author

I raised a PR for time.clock removal before. Yeah, I missed filing an issue for deprecation warning due to invalid sequence. You can use find . -iname "*.py" | xargs -I{} python3.8 -Wall -m py_compile {} to find the deprecation warnings in nltk.

@tirkarthi
Copy link
Author

Updated 3rd issue to be moto related but similar issue exists for nltk too. Sorry for the confusion.

@rec
Copy link

rec commented Jan 18, 2020

Anyway, nltk cannot run a developer install on Mac OS. :-/

Details here: https://www.reddit.com/r/Python/comments/eqc9wg/list_of_easy_open_source_issues_for_starters_on/fes7nf2/

@hugovk
Copy link

hugovk commented Jan 19, 2020

You can use find . -iname "*.py" | xargs -I{} python3.8 -Wall -m py_compile {} to find the deprecation warnings

Any idea why I'm not getting warnings or errors on macOS here?

$ cat 1.py
from collections import Mapping
$ python --version
Python 3.8.0
$ python 1.py
1.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Mapping
$ python -Wall -m py_compile 1.py
$
$ python --version
Python 3.9.0a2+
$ python 1.py
Traceback (most recent call last):
  File "/private/tmp/openslide-python/1.py", line 1, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/Users/hugo/.pyenv/versions/3.9-dev/lib/python3.9/collections/__init__.py)
$ python -Wall -m py_compile 1.py
$

@tirkarthi
Copy link
Author

The warning reported are compile time warning and I guess py_compile can be used to detect only those warnings. This is over executing code and hence running the test suites is the way to go for these. I use the below regex to get some patterns.

rg "collections\.(Awaitable|Coroutine|AsyncIterable|AsyncIterator|AsyncGenerator|Hashable|Iterable|Iterator|Generator|Reversible|Sized|Container|Callable|Collection|MutableSet|Mapping|MutableMapping|MappingView|KeysView|ItemsView|ValuesView|Sequence|MutableSequence|ByteString)\b"

@rec
Copy link

rec commented Jan 19, 2020

So if this is really going to be a project, there needs to be a better organizing tool than a gist!

@tirkarthi
Copy link
Author

@rec Agreed. I used gist as a todo list for small list but it keeps getting longer and didn't expect much interest from Reddit. I will try to host it some place where people can possibly subscribe and has minimum maintenance from my end. Suggestions welcome on tools if any. I am not sure if making updates send email to people who subscribe so I have filed below issues and updated main gist.

@hugovk
Copy link

hugovk commented Jan 19, 2020

I subscribed and got an email and notification.

Perhaps a wiki page so people can edit when they've made a PR?

@tirkarthi
Copy link
Author

I didn't realize updates to gist trigger emails. I will check the wiki idea. Thanks @hugovk for the top packages data that I am using for this task :) https://hugovk.github.io/top-pypi-packages/top-pypi-packages-365-days.json, https://github.com/hugovk/top-pypi-packages

@hugovk
Copy link

hugovk commented Jan 20, 2020

You're welcome!

I've also made a script that uses that list to clone a certain number of top repos, and run a test command in each of them, eg. hugovk/pypi-tools#16.

If it was possible to create a Flake8 plugin to detect some of the issues you've been finding, that would work really well with this.

Or another script/command (with a non-zero exit) to find potential problems. For example, contains collections import Mapping and not collections.abc import Mapping.

@hugovk
Copy link

hugovk commented Jan 21, 2020

@rec
Copy link

rec commented Jan 22, 2020 via email

@tirkarthi
Copy link
Author

So I have triaged the fedora report as of today and filed around 30 issues upstream and confirmed several fixes in the respective issues in fedora. I also added the project issues there. Click on the "report" word for the issue that should have the source hyperlinked. I am picking up several issues for PR. Just comment here if you are taking a block of issues to avoid rework. Issues updated as below

Thank you all for the contributions!

@hugovk
Copy link

hugovk commented Jan 23, 2020

Can be checked off:
dask/zict#30
Mic92/python-mpd2#108
meejah/txtorcon#341

@tirkarthi
Copy link
Author

I fixed several issues yesterday and checked them off as they have PR. Sorry, I forgot to update here. It seems the fedora tracker is updated daily with new bugs as found so will try and see if there are additional issues that can be added here.

@hugovk
Copy link

hugovk commented Jan 23, 2020

  • confirmed - report - source Bug 1791951 - jpype fails to build with Python 3.9: 'array.array' object has no attribute 'fromstring'

--> PR jpype-project/jpype#572

@hugovk
Copy link

hugovk commented Jan 25, 2020

@rec
Copy link

rec commented Jan 26, 2020 via email

@hugovk
Copy link

hugovk commented Feb 16, 2020

  • Importing ABC from collections module directly was removed in Python 3.9 #285

Can be ticked: the PR for this has been fixed: google/apitools#286

  • Importing ABC directly from collections was removed in Python 3.9 #189

And this: sqlalchemy-redshift/sqlalchemy-redshift#190

  • Importing ABC from collections module directly was removed in Python 3.9 #4

And this: fcurella/cookiejar#4

@tirkarthi
Copy link
Author

Thanks @hugovk for the ping and PRs on the issues. I have updated the gist by reading all the open issues. Unfortunately I am not updating this list anymore due to effort required. Still there are many PRs to make. I am also trying to run some scripts on around 100 repos per week to file issues if found regarding deprecation and syntax warnings.

@tirkarthi
Copy link
Author

As a stat for the gist 51 issues remain open and 64 issues have been resolved :)

@hugovk
Copy link

hugovk commented Feb 18, 2020

You're welcome, and thanks for starting the list and making issues and PRs!

The aforementioned Fedora tracker is a good place for an up-to-date list:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment