Skip to content

Instantly share code, notes, and snippets.

@obestwalter
Last active February 20, 2022 17:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save obestwalter/e82068b95a9576024d0f9504587e24d6 to your computer and use it in GitHub Desktop.
Save obestwalter/e82068b95a9576024d0f9504587e24d6 to your computer and use it in GitHub Desktop.
When inspecting self in this way in a property it endlessly recursess
import traceback
class SomeClass:
a_class_attr = "don't find me"
def an_instance_method(self):
...
def _a_private_method(self):
...
def make_name2callable(obj):
n2c = {n: getattr(obj, n) for n in dir(obj)}
return {k: v for k, v in n2c.items() if not k.startswith("_") and callable(v)}
class ThisWorks(SomeClass):
def name2callable(self):
return make_name2callable(self)
class ThisDoesnt(SomeClass):
@property
def name2callable(self):
return make_name2callable(self)
print(ThisWorks().name2callable())
try:
print(ThisDoesnt().name2callable)
except RecursionError:
print("why does this cause a recursion?")
traceback.print_exc()
@obestwalter
Copy link
Author

obestwalter commented Feb 20, 2022

output (run with python3.8)

{'an_instance_method': <bound method SomeClass.an_instance_method of <__main__.ThisWorks object at 0x7fb20ce71d30>>, 'name2callable': <bound method ThisWorks.name2callable of <__main__.ThisWorks object at 0x7fb20ce71d30>>}

why does this cause a recursion?
Traceback (most recent call last):
  File "/home/ob/oss/xonsh/recursion_reproducer.py", line 33, in <module>
    print(ThisDoesnt().name2callable)
  File "/home/ob/oss/xonsh/recursion_reproducer.py", line 27, in name2callable
    return make_name2callable(self)
  File "/home/ob/oss/xonsh/recursion_reproducer.py", line 15, in make_name2callable
    n2c = {n: getattr(obj, n) for n in dir(obj)}
  File "/home/ob/oss/xonsh/recursion_reproducer.py", line 15, in <dictcomp>
    n2c = {n: getattr(obj, n) for n in dir(obj)}
  File "/home/ob/oss/xonsh/recursion_reproducer.py", line 27, in name2callable
    return make_name2callable(self)
  File "/home/ob/oss/xonsh/recursion_reproducer.py", line 15, in make_name2callable
    n2c = {n: getattr(obj, n) for n in dir(obj)}
  File "/home/ob/oss/xonsh/recursion_reproducer.py", line 15, in <dictcomp>
    n2c = {n: getattr(obj, n) for n in dir(obj)}
  [... and on, and on, and on ...]

@obestwalter
Copy link
Author

obestwalter commented Feb 20, 2022

Ok, it is actually quite obvious. In that loop getattr(self, "name2callable") will fetch the property function object, see it is a descriptor and call __get__ on it, which will lead to an infinite recursion. It's the same as writing self.name2callable. D'oh.

@obestwalter
Copy link
Author

obestwalter commented Feb 20, 2022

This was courtesy of me playing with xonsh and starting to write my own .xonshrc. I use this code now to automatically inject all methods in a class in the xonsh aliases.

class _XOLI_AUTO_ALIAS:
    """On Instantiation: inject all public methods into aliases"""

    def __init__(self):
        """create map from alias to callable.

        Transform names from pythony snake_case to bashy slug-case
        """
        from xonsh.built_ins import XSH

        XSH.aliases.update(
            {
                k.replace("_", "-"): v
                for k, v in {n: getattr(self, n) for n in dir(self)}.items()
                if not k.startswith("_") and callable(v)
            }
        )

class _XOLI_ALIASES(_XOLI_AUTO_ALIAS):
    """Inject my collection of aliases"""

    @staticmethod
    def gh_rebase_upstream():
        # follows https://gitlab.com/palisade/palisade-release/-/wikis/How-to-rebase-your-branch-from-the-master-branch
        with oli.swap("-e", "-x"):
            assert $(git status --porcelain), f"FATAL: not clean!\n" + $(git status)
            git_branches = $(git branch -l)
            git_base = "_main" if "_main" in git_branches else "master"
            assert git_base in git_branches, (
                f"FATAL: no _main/master branch => {git_branches}"
            )
            git_feature_branch = $(git rev-parse --abbrev-ref HEAD)
            assert git_feature_branch != git_base, (
                f"FATAL: not on feature branch but on {git_base}"
            )
            assert "master" in git_branches, f"no known _main branch in {git_branches}"
            remotes = $(git remote)
            assert "upstream" in remotes and "origin" in remotes, remotes 
            git checkout @(git_base)
            git pull upstream @(git_base)
            git checkout @(git_feature_branch)
            git pull upstream @(git_feature_branch)
            git rebase @(git_base) @(git_feature_branch)


    @staticmethod
    def reload():
        source ~/.xonshrc
        echo reloaded ~/.xonshrc "(but did not reset namespace)"

   ...


_XOLI_ALIASES()

@obestwalter
Copy link
Author

obestwalter commented Feb 20, 2022

And this would be a better approach: https://docs.python.org/3/library/inspect.html#fetching-attributes-statically

Fetching attributes statically

Both getattr() and hasattr() can trigger code execution when fetching or checking for the existence of attributes. Descriptors, like properties, will be invoked and getattr() and getattribute() may be called.

For cases where you want passive introspection, like documentation tools, this can be inconvenient. getattr_static() has the same signature as getattr() but avoids executing code when it fetches attributes.

@obestwalter
Copy link
Author

For completeness here is a little test/demo that uses this:

import os
import sys

import pytest


class _XOLI_AUTO_ALIAS:
    """On Instantiation: inject all public methods into aliases"""

    def __init__(self):
        """create map from alias to callable.

        Transform names from pythony snake_case to bashy slug-case
        """
        from xonsh.built_ins import XSH

        XSH.aliases.update(
            {
                k.replace("_", "-"): v
                for k, v in {n: getattr(self, n) for n in dir(self)}.items()
                if not k.startswith("_") and callable(v)
            }
        )


# end of production code
#########################################################################################


class FakeXonshBuiltIns(type(os)):
    DEFAULT_ALIASES = {"default": os.getcwd}

    def __init__(self):
        self.XSH = self._XSH()

    class _XSH:
        def __init__(self):
            self.aliases = FakeXonshBuiltIns.DEFAULT_ALIASES.copy()


@pytest.fixture(autouse=True)
def provide_global_xonsh_aliases():
    sys.modules["xonsh.built_ins"] = FakeXonshBuiltIns()
    from xonsh.built_ins import XSH

    assert isinstance(XSH, FakeXonshBuiltIns._XSH)
    assert XSH.aliases == FakeXonshBuiltIns.DEFAULT_ALIASES
    assert XSH.aliases is not FakeXonshBuiltIns.DEFAULT_ALIASES


def test_auto_alias():
    class Oliasses(_XOLI_AUTO_ALIAS):
        a_public_class_atribute = "dont find me!"

        def an_instance_method(self):
            ...

        def another_instance_method(self):
            ...

        @classmethod
        def a_class_method(cls):
            ...

        @staticmethod
        def a_static_method():
            ...

        def _a_private_method(self):
            ...  # don't find me!

    class Oliherited(Oliasses):
        def a_method_on_inheritor(self):
            ...

    from xonsh.built_ins import XSH

    # ensure defaults are as expected
    assert XSH.aliases == FakeXonshBuiltIns.DEFAULT_ALIASES

    # trigger alias generation
    Oliherited()

    # ensure existing aliases are not clobbered
    assert XSH.aliases.pop("default") == os.getcwd

    # ensure new aliases are what is expected
    assert len(XSH.aliases) == 5

    # ensure all method names are in there and transformed properly
    for name in [
        "an_instance_method",
        "another_instance_method",
        "a_class_method",
        "a_static_method",
        "a_method_on_inheritor",
    ]:
        assert name not in XSH.aliases
        obj = XSH.aliases[name.replace("_", "-")]
        assert obj.__name__ == name
        assert callable(obj)

    # ensure_no_privates_and_no_non_callables
    private_attrs = ["a_public_class_atribute", "_a_private_method"]
    for name in private_attrs:
        assert name not in XSH.aliases
        assert name.replace("_", "-") not in XSH.aliases

    # general check that nothing weird ended up in aliases
    for k, v in XSH.aliases.items():
        assert not k.startswith("_")
        assert callable(v)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment