abadger/basic.rst

## basic.rst
For the glossary:

distribution
    In some operating systems, the kernel of the OS is produced by one
    organization but this is combined with other necessary tools by a second
    organization.  This second organization is called a distribution.

native string
    Python2 and Python3 have different types for unadorned string literals and many
    strings operations.  In Python2, these are byte strings.  In Python3,
    these are text strings.


.. py:data:: BOOLEANS_TRUE

    All the values that parameter parsing converts to True.  This is a mixture
    of boolean, integer, and native string types.

    In basic.py this is a list but in the refactor we should make this
    a frozenset

.. py:data:: BOOLEANS_FALSE

    All the values that parameter parsing converts to False.  This is
    a mixture of boolean, integer, and native string types.

    In basic.py this is a list but in the refactor we should make this
    a frozenset

.. py:data:: BOOLEANS

    All the values that parameter parsing consider to be valid booleans.  This
    is the superset of :py:data:`BOOLEANS_TRUE` and :py:data:`BOOLEANS_FALSE`.

    In basic.py this is a list but in the refactor we should make this
    a frozenset

.. note:: In very old code, modules would specify that a parameter was
    a boolean by listing something like this in the :ref:`argument_spec`::

        module = AnsibleModule(
            argument_spec=dict(
                return_status=dict(choices=BOOLEANS),
            ),
        )

    This pattern is not very flexible and can cause issues in cornercases.
    It should no longer be used.  Switch to using the type field instead::

        module = AnsibleModule(
            argument_spec=dict(
                return_status=dict(type='bool'),
            ),
        )

.. py:data:: AVAILABLE_HASH_ALGORITHMS

    Mapping of hash algorithms supported by this version of python to
    a constructor for the hash algorithm.  These hash algorithms all follow
    the API that is available for the stdlib :py:mod:`stdib:hashlib` hash
    algorithms.

.. py:data:: FILE_COMMON_ARGUMENTS

    This is the argument_spec for anything that implements similar
    functionality to the file module.  Note that currently, using this means
    that the module will accept all of these parameters but may not actually
    utilize them.  For this reason, this may be deprecated in the future in
    favor of an argument_spec and supporting utility functions that make sure
    that both are valid.

.. py:function:: get_platform() -> NativeString

    :rtype: NativeString
    :returns: Name of the platform the module is running on

    Returns a native string that labels the platform ("Linux", "Solaris",
    etc).  Currently, this is the result of calling :py:func:`stdlib:platform.system`.

.. py:function:: get_distribution() -> Union[NativeString, None]

    :rtype: NativeString or None
    :returns: Name of the distribution the module is running on

    This function attempts to determine what the :term:`distribution` is and
    return a string representing that value.  If it cannot determine
    a distribution, it returns None.

.. py:function:: get_distribution_version() -> Union[NativeString, None]

    :rtype: NativeString or None
    :returns: A string representation of the version of the distribution

    This function attempts to find the version of the distribution and return
    it as a native string.  If it cannot determine a version, it returns None.

.. py:function:: get_all_subclasses(cls: type) -> Iterable[type]

    :arg cls: A python class
    :rtype: iterable
    :returns: An iterable of python classes which are the subclasses of `cls`.

    In python, you can use a class's :py:meth:`__subclasses__` method to
    determine what subclasses of a class exist.  However, `__subclasses__`
    only goes one level deep.  This function searches each child class's
    `__subclasses__` method to find all of the descendent classes.  It then
    returns an iterable of the descendent classes.

.. warn:: Core team members, should we change get_all_subclasses() to return
        a set/frozenset?  Right now, it returns a list and elements could be
        repeated

.. py:function:: load_platform_subclass(cls: type, *args: Any, **kwargs: Any) -> type

    :arg cls: Class to find an appropriate subclass for
    :arg \*args: Positional arguments to initialize the subclass with
    :arg \*\*kwargs: Keyword arguments to initialize the subclass with
    :returns: An instantiated class

    Some Ansible modules have different implementations depending on the
    platform they run on.  This function is used to select between the various
    implementations and choose one.  You can look at the implementation of the
    Ansible :ref:`user` module for an example of how to use this.

.. warn:: Core team members, do we want to change this API?  Instead of
    returning an instantiated class, return the class and make the caller
    instantiate it.  Doing that will mean we don't have to pass in ``*args``
    and ``**kwargs``.

Internals
---------

These globals are not for use by other code.  Documenting here for people who
need to modify :file:`basic.py`.

.. py:data:: _ANSIBLE_ARGS

    This is an internal global variable that holds the parameters to the
    module once AnsibleModule has read them in.  It is a native string type.


Deprecated
==========

All Python2 and Python3 compat
------------------------------

Analogs to these are either in six or one of the
ansible.module_utils.pycompatXY modules::

Deprecated    New Way to Achieve this
----------    -----------------------
imap          ansible.module_utils.six.moves.map
basestring    ansible.module_utils.six.string_types or (ansible.module_utils.six.text_type, ansible.module_utils.six.binary_type) [*]_
unicode       ansible.module_utils.six.text_type or (ansible.module_utils._text.to_native, ansible.module_utils._text.to_text) [@]_
bytes         ansible.module_utils.six.binary_type or (ansible.module_utils._text.to_bytes, ansible.module_utils._text.to_native) [@]_
iteritems     ansible.module_utils.six.iteritems
reduce        ansible.module_utils.six.moves import reduce
NUMBERTYPES   ansible.module_utils.six.integer_types + (float,) [+]_
NoneType      ansible.module_utils.pycompat27.NoneType      ## Need to move
Sequence      ansible.module_utils.pycompat24.Sequence      ## Need to move
Mapping       ansible.module_utils.pycompat24.Mapping       ## Need to move
SEQUENCETYPE  ansible.module_utils.pycompat27.SEQUENCETYPE  ## Need to move
json          ansible.module_utils.pycompat24.json          ## Need to move
literal_eval  ansible.module_utils.pycompat24.literal_eval
get_exception ansible.module_utils.pycompat24.get_exception


.. _[*]: What to replace basestring with depends on the usage.  Most code that
    uses basestring looks like one of these::

        if not isinstance(variable, basestring):
            # Get the string representation of the object
            variable = str(variable)

        if not isinstance(variable, unicode):
            # Make sure a string is unicode for the API we're calling
            variable = unicode(variable, 'utf-'8')

    Those should be replaced with code like this::

        if not isinstance(variable, (six.binary_type, six.text_type)):
            # Get the string representation of the object
            variable = to_native(variable, errors='surrogate_or_strict')

        if not isinstance(variable, six.text_type):
            # Make sure a string is the platform's text type (unicode on py2, str on py3)
            variable = to_text(variable, 'utf-8')

.. _[@]: unicode and bytes have even more potential replacements than
    basestring depending on how they are used.  Here's a quick overview.  For
    more information, see the documentation of the referenced replacement
    functions.

    For converting between text and bytes use :func:`ansible.module_utils._text.to_text` and
    :func:`ansible.module_utils._text.to_bytes`::

        # Original:
        text_string = unicode("Toshio wrote this", encoding='utf-8')
        byte_string = bytes(u"Toshio wrote this", 'utf-8')

        # New:
        text_string = to_text("Toshio wrote this", errors='surrogate_or_strict')
        byte_string = to_bytes(u"Toshio wrote this", errors='surrogate_or_strict')

    For converting between an unknown object and its string representation
    (for instance, for printing an informational error message), use the
    ``nonstring`` argument to `to_text` and `to_bytes`::

        # Original:
        text_msg = u'Error: Can not process: %s' % unicode({'An object': 'value'}, encoding='utf-8')
        byte_msg = 'Error: Can not process: %s' % bytes({'An object': 'value'}, 'utf-8')

        # New:
        text_msg = u'Error: Can not process: %s' % to_text({'An object': 'value'}, nonstring='simplerepr')
        byte_msg = to_bytes('Error: Can not process: %s' % to_native({'An object': 'value'}, nonstring='simplerepr'))
        # byte_msg is different because the original was not safe on python3

    For testing whether a value is a text or byte string::

        # Original:
        if isinstance('string', unicode): pass
        if isinstance('string', bytes): pass

        # New:
        if isinstance('string', text_type): pass
        if isinstance('string', binary_type): pass


.. _[+]: NUMBERTYPES includes floating point numbers while six's equivalent is
    just for integers.  Here's a sample of how to change your code::

        # Old code:
        from ansible.module_utils.basic import NUMBERTYPES
        if isinstance(obj, NUMBERTYPES):
            return str(obj)

        from ansible.module_utils.six import integer_types
        # New code:
        if isinstance(obj, integer_types + (float,)):
            return str(obj)

.. todo:: Move the deprecated values that belong in pycompatXY into those
    files.

.. warn:: Ansible Core Team -- do we want to make an
    ansible.module_utils.json with this and other json-related functions? (Or
    put this into a pycompat* as it's only for python-2.4 which doesn't have
    the stdlib json library)?


Private Globals
---------------

.. py:data:: PASSWD_ARG_RE

    Defines what a password looks like.  We use this to scan output for things
    that we should obfuscate.

.. py:data:: PERM_BITS

    Bitmask of all the stat.mode bits pertaining to UNIX permissions

.. py:data:: EXEC_PERM_BITS

    Bitmask of all the stat.mode bits pertaining to UNIX execute permissions

.. py:data:: DEFAULT_PERM

    Bitmask for all file permissions.  This is combined with umask in the
    actual code to come out with the default permissions.

.. todo:: Rename these to have a leading underscore to mark them as private


RST Notes
=========
If we switch to inline documentation,  attributes are documented like this::

  #: All the values that parameter parsing converts to True
  BOOLEANS_TRUE = ['yes', 'on', '1', 'true', 'True', 1, True]

  #: All the values that parameter parsing converts to False
  BOOLEANS_FALSE = ['no', 'off', '0', 'false', 'False', 0, False]

Pending
=======

This list of functions needs to be further documented

Move to _text
-------------

.. py:function:: json_dict_unicode_to_bytes(d: dict, encoding='utf-8': NativeString, errors='surrogate_or_strict': NativeString) -> dict

    * This function recursively walks through a container converting any strings inside of the
      container into byte strings.  Handles any container type which is json serializable.
    * Rename this function.  Perhaps ``container_to_bytes()`` or ``recursive_to_bytes``?
    * First argument can be any of the known container type, not just a dict
    * Change this to handle sets as well (since we serialize sets in jsonify)
    * Change isinstance checks to use collections.*
    * Change this from recursion to iteration so we don't run up against the recursion limit

.. py:function:: json_dict_bytes_to_unicode(d: dict, encoding='utf-8': NativeString, errors='surrogate_or_strict': NativeString) -> dict

    * This function recursively walks through a container converting any strings inside of the
      container into text strings.
    * Rename this function.  Perhaps ``container_to_text()`` or ``recursive_to_text``?
    * First argument can be any of the known container type, not just a dict
    * Change this to handle sets as well (since we serialize sets in jsonify)
    * Change isinstance checks to use collections.*
    * Change this from recursion to iteration so we don't run up against the recursion limit

.. py:function:: container_to_native(container: Container, encoding='utf-8': NativeString, errors='surrogate_or_strict': NativeString) -> dict

    * New function that is an alias for one of the above two functions depending on whether we're
      running on Python2 or Python3

Logging related
---------------

Sanitization Related
~~~~~~~~~~~~~~~~~~~~

.. py:function:: return_values(obj: Any) -> (iterator)NativeStrings

  * turn text into native strings (recursively)
  * handle import types: string_types, sequencetypes (have to look into this), mappings, bool,
    numbertypes
  * This is used to turn params from the argspec listed as no_log into a string that will later be
    used to filter out values from returns.
    * this is a normalization step so comparisons will work better later
    * since this comes from the module parameters, the types are constrained to what's legal in
      a json dict

.. py:function:: _remove_values_conditions(value: Any, no_log_strings: Set, deferred_removals: List) -> Any

    Helper function for :meth:`remove_values`.

    :arg value: The value to check for strings that need to be stripped
    :arg no_log_strings: set of strings which must be stripped out of any values
    :arg deferred_removals: List which holds information about nested
        containers that have to be iterated for removals.  It is passed into
        this function so that more entries can be added to it if value is
        a container type.  The format of each entry is a 2-tuple where the first
        element is the ``value`` parameter and the second value is a new
        container to copy the elements of ``value`` into once iterated.
    :returns: if ``value`` is a scalar, returns ``value`` with two exceptions:
        1. :class:`~datetime.datetime` objects which are changed into a string representation.
        2. objects which are in no_log_strings are replaced with a placeholder
            so that no sensitive data is leaked.
        If ``value`` is a container type, returns a new empty container.

    ``deferred_removals`` is added to as a side-effect of this function.

    .. warning:: It is up to the caller to make sure the order in which value
        is passed in is correct.  For instance, higher level containers need
        to be passed in before lower level containers. For example, given
        ``{'level1': {'level2': 'level3': [True]} }`` first pass in the
        dictionary for ``level1``, then the dict for ``level2``, and finally
        the list for ``level3``.

.. py:function:: remove_values(value: Any, no_log_strings: Set) -> Any

  * remove no_log_strings from a value.
  * Currently used for the values returned by :func:`exit_json` so it needs to return typed data
    (Cannot return string representations)

.. py:function:: heuristic_log_sanitize(data: String, no_log_values=None: Set) -> NativeString

  * attempt to sanitize data using :func:`remove_values` and a regex that looks for potential password2.
  * Currently used by logging functions so returns string representations of values.


Formatting of strings
---------------------

.. seealso::
    :ref:`Move to _text` heading.  Should we make ``text.converters`` for what's there
    currently and ``text.formatters`` for these?

.. py:function:: bytes_to_human(size, isbits=False, unit=None)

.. py:function:: human_to_bytes(number, default_unit=None, isbits=False)

.. py:function:: _lenient_lowercase(lst)

File handling
-------------

.. py:function:: is_executable(path)

  * "does a file have an executable bit set"

.. py:function:: format_attributes(attributes)

  * For a passed in list of attribute flags (from lsattr), create a list of long attr names

.. py:function:: get_flags_from_attributes(attributes)

  * For a list of long attr names, return the list of flags

Hashing
-------

Module Parameter handling
-------------------------

.. py:function:: _load_params()

 * Read the module's parameters and store them in a global var
 * Can be called by very dynamic custom modules that want to process the params
 * Sets :ref:`_ANSIBLE_ARGS`
 * Since the params come from stdin, this is used to keep a copy of the params for later use.

.. py:function:: env_fallback(\*args, \*\*kwargs)

  * Load a param's default value from the environment.

.. py:class:: AnsibleFallbackNotFound(Exception)

  * Exception thrown when a param specifies a nonexistent fallback

Module Return handling
----------------------

On Module Start
---------------

On Module Exit
--------------

Process management
------------------

Json
----
.. py:class:: _SetEncoder(json.JSONEncoder)

  * A json encoder that also understands sets.
  * Needed for jsonify()
  * Possible categories?  text?  json?  params?  returns?
  * Maybe replace this with a function for the default parameter to the jsonEncoder (see jsonify)


Uncategorized
-------------

.. py:function:: get_module_path()

  * Only used by git and _accelerate
  * Find the directory name that the module lives in (when running)


AnsibleModule
-------------

Version 2 of AnsibleModule should contain functionality that sets up a Module similar to how GUI
frameworks have an ``App`` class that sets up an Application environment.  Parameter parsing and
registering how to exit from the Module seem like things which are AnsibleModule responsibilities.

* Entrypoint for current AnsibleModules
* Going to need a new AnsibleModule
* Going to split most methods out of it
* New focus should be
  * parameter handling.
  * return values?
  * log_sanitization?

.. py:meth:: `__init__(self, argument_spec, bypass_checks=False, no_log=False, check_invalid_arguments=True,
         mutually_exclusive=None, required_together=None, required_one_of=None,
         add_file_common_args=False, supports_check_mode=False, required_if=None)`

  * Initialization is all about handling parameters


From AnsibleModule
------------------

Same categories as for the functions.  Many of these will need to be ported from being methods to
being functions.

Move to _text
-------------

.. py:meth:: bytes_to_human(self, size)
.. py:meth:: human_to_bytes(self, number, isbits=False)

  * Light wrappers around the toplevel functions.  Remove these for the new AnsibleModule

Logging related
---------------

.. py:meth:: _log_to_syslog(self, msg)

.. py:meth:: debug(self, msg)

.. py:meth:: log(self, msg, log_args=None)

Sanitization Related
~~~~~~~~~~~~~~~~~~~~

Formatting of strings
---------------------

File handling
-------------
.. py:meth:: selinux_mls_enabled(self)
.. py:meth:: selinux_enabled(self)
.. py:meth:: selinux_initial_context(self)
.. py:meth:: selinux_default_context(self, path, mode=0)
.. py:meth:: selinux_context(self, path)

  * Find selinux settings
  * Uses fail_json rather than exceptions which is how it still ties to module

.. py:meth:: user_and_group(self, path, expand=True)

  * Get uid and gid of a filepath

.. py:meth:: find_mount_point(self, path)

  * Is a path a mount point?

.. py:meth:: is_special_selinux_path(self, path)

  * Is the path on an selinux special filesystem

.. py:meth:: set_default_selinux_context(self, path, changed)

  * Set system's default selinux context on it

.. py:meth:: set_context_if_different(self, path, context, changed, diff=None)
.. py:meth:: set_owner_if_different(self, path, owner, changed, diff=None, expand=True)
.. py:meth:: set_group_if_different(self, path, group, changed, diff=None, expand=True)
.. py:meth:: set_mode_if_different(self, path, mode, changed, diff=None, expand=True)
.. py:meth:: set_attributes_if_different(self, path, attributes, changed, diff=None, expand=True)

  * Sets various file attributes and reports back whether the file was changed or not

.. py:meth:: get_file_attributes(self, path)

  * Retrieve file system extended attributes

.. py:meth:: _symbolic_mode_to_octal(cls, path_stat, symbolic_mode)
.. py:meth:: _apply_operation_to_mode(user, operator, mode_to_apply, current_mode)
.. py:meth:: _get_octal_mode_from_symbolic_perms(path_stat, user, perms, use_umask)

  * helpers for setting file modes

.. py:meth:: or_reduce(mode, perm)

  * helper for :meth:`_get_octal_mode_from_symbolic_perms`

.. py:meth:: set_fs_attributes_if_different(self, file_args, changed, diff=None, expand=True)
.. py:meth:: set_directory_attributes_if_different(self, file_args, changed, diff=None, expand=True)
.. py:meth:: set_file_attributes_if_different(self, file_args, changed, diff=None, expand=True)

  * Note: make this an alias to set_fs_attributes_if_different.

.. py:meth:: backup_local(self, fn)

  * Make a backup of a file with a date extension

.. py:meth:: cleanup(self, tmpfile)

  * This is an ``os.unlink()`` that does not error
  * Rename: try_remove()?
  * Also needs to not log the failure to sys.stderr.  Instead use the module warnings facility

.. py:meth:: preserved_copy(self, src, dest)
.. py:meth:: atomic_move(self, src, dest, unsafe_writes=False)
.. py:meth:: _unsafe_writes(self, src, dest)

  * About writing files

.. py:meth:: append_to_file(self, filename, str)

  * Why do we have this?  Only used by known_hosts.py

.. py:meth:: load_file_common_arguments(self, params)

  * Should be moved into something for file-type modules.  Like ec2.py/aws.py is for amazon modules.

===> hashing

.. py:meth:: digest_from_file(self, filename, algorithm)

  * Possibly a part of file_handling

.. py:meth:: md5(self, filename)
.. py:meth:: sha1(self, filename)
.. py:meth:: sha256(self, filename)

 * I believe these should be deprecated.  Just use hashlib
 * One thing that is useful is that md5 says that MD5 is not available potentially because of FIPS
   mode.

Module Parameter handling
-------------------------

.. py:meth:: _handle_aliases(self, spec=None, param=None)
.. py:meth:: _handle_no_log_values(self, spec=None, param=None)
.. py:meth:: _check_arguments(self, check_invalid_arguments, spec=None, param=None, legal_inputs=None)
.. py:meth:: _count_terms(self, check, param=None)
.. py:meth:: _check_mutually_exclusive(self, spec, param=None)
.. py:meth:: _check_required_one_of(self, spec, param=None)
.. py:meth:: _check_required_together(self, spec, param=None)
.. py:meth:: _check_required_arguments(self, spec=None, param=None)
.. py:meth:: _check_required_if(self, spec, param=None)
.. py:meth:: fail_on_missing_params(self, required_params=None)

  * We should have a more flexible way to let the module author decide whether to fail if arguments are missing.

.. py:meth:: _check_argument_types(self, spec=None, param=None)
.. py:meth:: _check_argument_values(self, spec=None, param=None)

  * Check the parameters against the argspec
  * Want ``check_argument_values()`` to be pluggable and the plugins live in a separate file(s)

.. py:meth:: _check_type_str(self, value)
.. py:meth:: _check_type_list(self, value)
.. py:meth:: _check_type_dict(self, value)
.. py:meth:: _check_type_bool(self, value)
.. py:meth:: _check_type_int(self, value)
.. py:meth:: _check_type_float(self, value)
.. py:meth:: _check_type_path(self, value)
.. py:meth:: _check_type_raw(self, value)

  * Base parameter types.  Should be built into the file that implements the arguments.  Would like
    ~90% of modules to only require the base.  That way, there isn't overhead of boilerplate in
    multiple files.

.. py:meth:: _check_type_bytes(self, value)
.. py:meth:: _check_type_bits(self, value)
.. py:meth:: _check_type_jsonarg(self, value)

  * Few modules use these so push them to other module_utils files

.. py:meth:: safe_eval(self, value, locals=None, include_exceptions=False)

  * Only used by _check_type_dict().
  * This mostly doesn't seem to be needed.  We eventually call literal_eval which should eliminate
    some of the things this checks for (like imports)
  * Can we get rid of this in AnsibleModulev2 (leave in AnsibleModulev1 for backwards compat)?

.. py:meth:: _handle_options(self, argument_spec=None, params=None)

  * option subspecs.

.. py:meth:: _set_defaults(self, pre=True, spec=None, param=None)
.. py:meth:: _set_fallbacks(self, spec=None, param=None)

  * More setting of things from argspec

.. py:meth:: _load_params(self)

  * sets the params as a module attribute

Module Return handling
----------------------

.. py:meth:: warn(self, warning)
.. py:meth:: deprecate(self, msg, version=None)

  * Adds message to a queue that will be added to the json output when we exit
  * logs the message as well
  * currently does not sanitize the message (done by caller)

.. py:meth:: add_path_info(self, kwargs)

  * Retrieves stat type information about a path in 'path' key for kw args and returns it in kwargs.
    * Should change the API of this so that we pass in a discrete path and return the information
      about it.

.. py:meth:: add_cleanup_file(self, path)

  * add a file to a queue of things to be removed on exit

.. py:meth:: _return_formatted(self, kwargs)

  * Help to put state information (warnings, and deprecations) into the json return

.. py:meth:: exit_json(self, \*\*kwargs)
.. py:meth:: fail_json(self, \*\*kwargs)
  * These are returning things.  Should we rework this?
  * Possibly, alikins excepthook
    * Why should exit_json be used with exit_json?  So that we always invoke the cleanup functions.

On Module Start
---------------
.. py:meth:: _check_locale(self)

  * Validate locale and set to C if have any problems.
    * Called from the current AnsibleModule's __init__
    * Research why this was needed?

.. py:meth:: _log_invocation(self)

  * Log the invocation of the module

.. py:meth:: _set_cwd(self)

On Module Exit
--------------

.. py:meth:: do_cleanup_files(self)

  * On exit, cleanup the files

Process management
------------------

.. py:meth:: get_bin_path(self, arg, required=False, opt_dirs=[])

  * Set the cwd to a directory that we have read access to.

.. py:meth:: _read_from_pipes(self, rpipes, rfds, file_descriptor)
.. py:meth:: `run_command(self, args, check_rc=False, close_fds=True, executable=None, data=None,
            binary_data=False, path_prefix=None, cwd=None, use_unsafe_shell=False, prompt_regex=None,
            environ_update=None, umask=None, encoding='utf-8',    errors='surrogate_or_strict')`

Json
----

.. py:meth:: jsonify(self, data)
.. py:meth:: from_json(self, data)

Uncategorized
-------------

.. py:meth:: boolean(self, arg)

  * Possibly part of parameters

## common.rst

      
    Raw
  

              common.rst
            
          
common/
common/__init__.py
common/module.py
common/file.py
common/hashing.py
common/json.py
common/logging.py
common/process.py
common/sys_info.py
common/connection.py
params/__init__.py
params/common.py
params/size_validators.py
params/serialized_validators.py
text/
text/__init__.py
text/convert.py
text/format.py


common/module.py


Contains an AnsibleModule class
Parses parameters
Sets up the Module Environment (exit functions, cleanup functions)
It's expected that all modules will import core/module.py (either directly or through
a dependency).

.. py:data:: _ANSIBLE_ARGS

.. py:function:: _load_params()


.. py:class:: AnsibleModule

  * Same constructor as the present AnsibleModule but a lot less utility methods


.. py:meth:: warn(self, warning)

.. py:meth:: deprecate(self, msg, version=None)

.. py:meth:: add_path_info(self, kwargs)

.. py:meth:: _return_formatted(self, kwargs)

.. py:meth:: add_cleanup_file(self, path)


.. py:meth:: exit_json(self, \*\*kwargs)

  * Move these to module-level functions so that we can access them without access to a module.
    This is similar to how Python's warning module works (with a global var to store the
    information) so it doesn't seem too bad.


.. py:meth:: fail_json(self, \*\*kwargs)

  * Possibly move this to sys.excepthook instead.  An exception allows other code to catch the
    exception.


.. py:function:: cleanup()

  * Replacement for do_cleanup_files()
  * New function that should be used with atexit.register() to cleanup files on program exit.
    Should probably allow other code to hook the cleanup functions (Or should they just call atexit
    themselves?)


.. py:meth:: _check_locale(self)

.. py:meth:: _log_invocation(self)

.. py:meth:: _set_cwd(self)


.. py:function:: get_module_path()

  * This is only used by git module and _accelerate.  In git, it doesn't do what it was meant to do
    since AnsiBallZ was introduced.  Maybe we should leave it in basic.py for compat but not port it
    to the new API.


common/file.py

.. py:data:: PERM_BITS

.. py:data:: EXEC_PERM_BITS

.. py:data:: DEFAULT_PERM

  * Are these private? (and should be marked as such)


.. py:func:: get_file_arg_spec()

    Replacement for FILE_COMMON_ARGUMENTS.  This may need to be broken up into several different
    functions (since not all args are used by the same arg spec)


.. py:meth:: load_file_common_arguments(self, params)


.. py:function:: is_executable(path)

.. py:function:: format_attributes(attributes)

.. py:function:: get_flags_from_attributes(attributes)

.. py:meth:: selinux_mls_enabled(self)

.. py:meth:: selinux_enabled(self)

.. py:meth:: selinux_initial_context(self)

.. py:meth:: selinux_default_context(self, path, mode=0)

.. py:meth:: selinux_context(self, path)

.. py:meth:: user_and_group(self, path, expand=True)

.. py:meth:: find_mount_point(self, path)

.. py:meth:: is_special_selinux_path(self, path)

.. py:meth:: set_default_selinux_context(self, path, changed)

.. py:meth:: set_context_if_different(self, path, context, changed, diff=None)

.. py:meth:: set_owner_if_different(self, path, owner, changed, diff=None, expand=True)

.. py:meth:: set_group_if_different(self, path, group, changed, diff=None, expand=True)

.. py:meth:: set_mode_if_different(self, path, mode, changed, diff=None, expand=True)

.. py:meth:: set_attributes_if_different(self, path, attributes, changed, diff=None, expand=True)

.. py:meth:: get_file_attributes(self, path)

.. py:meth:: _symbolic_mode_to_octal(cls, path_stat, symbolic_mode)

.. py:meth:: _apply_operation_to_mode(user, operator, mode_to_apply, current_mode)

.. py:meth:: _get_octal_mode_from_symbolic_perms(path_stat, user, perms, use_umask)

.. py:meth:: or_reduce(mode, perm)

.. py:meth:: set_fs_attributes_if_different(self, file_args, changed, diff=None, expand=True)

.. py:meth:: set_directory_attributes_if_different(self, file_args, changed, diff=None, expand=True)

.. py:meth:: set_file_attributes_if_different(self, file_args, changed, diff=None, expand=True)

.. py:meth:: backup_local(self, fn)

.. py:meth:: try_remove(self, tmpfile)

    * Replacement for cleanup

.. py:meth:: preserved_copy(self, src, dest)

.. py:meth:: atomic_move(self, src, dest, unsafe_writes=False)

.. py:meth:: _unsafe_writes(self, src, dest)

.. py:meth:: append_to_file(self, filename, str)


common/hashing.py

.. py:data:: AVAILABLE_HASH_ALGORITHMS

.. py:meth:: digest_from_file(self, filename, algorithm)


Note
Perhaps this file should all be deprecated


common/json.py

.. py:class:: _SetEncoder(json.JSONEncoder)

.. py:meth:: jsonify(self, data)

.. py:meth:: from_json(self, data)


Also include things in json_utils.py

common/logging.py

.. py:data:: PASSWD_ARG_RE

.. py:function:: return_values(obj: Any) -> (iterator)NativeStrings

.. py:function:: _remove_values_conditions(value: Any, no_log_strings: Set, deferred_removals: List) -> Any

.. py:function:: remove_values(value: Any, no_log_strings: Set) -> Any

.. py:function:: heuristic_log_sanitize(data: String, no_log_values=None: Set) -> NativeString


Note
These are all used by AnsibleModule when returning information so perhaps they should
remain in core/module.py

.. py:meth:: debug(self, msg)

.. py:meth:: _log_to_syslog(self, msg)

.. py:meth:: log(self, msg, log_args=None)

    * These interact with syslog.  They make use of the above sanitization routines.  Check if
      they're used by AnsibleModule


common/process.py

.. py:meth:: get_bin_path(self, arg, required=False, opt_dirs=[])

.. py:meth:: _read_from_pipes(self, rpipes, rfds, file_descriptor)

.. py:meth:: `run_command(self, args, check_rc=False, close_fds=True, executable=None, data=None,
            binary_data=False, path_prefix=None, cwd=None, use_unsafe_shell=False, prompt_regex=None,
            environ_update=None, umask=None, encoding='utf-8',    errors='surrogate_or_strict')`


common/sys_info.py

.. py:function:: get_platform() -> NativeString

.. py:function:: get_distribution() -> Union[NativeString, None]

.. py:function:: get_distribution_version() -> Union[NativeString, None]


.. py:function:: _get_all_subclasses(cls: type) -> Iterable[type]

    Made this private so we can move it in the future if other things need it


.. py:function:: load_platform_subclass(cls: type, *args: Any, **kwargs: Any) -> type

    Does this belong here?  It's used by a few classes like service and user to implement code on
    different platforms.  But that code seems like it might be overly complicated.  I bet there's
    a simpler way to achieve the same results.


common/connection.py

Move the toplevel connection.py file to here.

params/common.py

Put parameter parsing and validation into its own module so that we can use it for action plugins in
addition to modules.  ansible.module_utils.core.module will have to import it, though.
.. py:data:: BOOLEANS_TRUE

.. py:data:: BOOLEANS_FALSE

.. py:data:: BOOLEANS

.. py:meth:: boolean(self, arg)

.. py:function:: env_fallback(\*args, \*\*kwargs)

.. py:class:: AnsibleFallbackNotFound(Exception)

.. py:meth:: _handle_aliases(self, spec=None, param=None)

.. py:meth:: _handle_no_log_values(self, spec=None, param=None)

.. py:meth:: _check_arguments(self, check_invalid_arguments, spec=None, param=None, legal_inputs=None)

.. py:meth:: _count_terms(self, check, param=None)

.. py:meth:: _check_mutually_exclusive(self, spec, param=None)

.. py:meth:: _check_required_one_of(self, spec, param=None)

.. py:meth:: _check_required_together(self, spec, param=None)

.. py:meth:: _check_required_arguments(self, spec=None, param=None)

.. py:meth:: _check_required_if(self, spec, param=None)

.. py:meth:: fail_on_missing_params(self, required_params=None)

.. py:meth:: _check_argument_types(self, spec=None, param=None)

.. py:meth:: _check_argument_values(self, spec=None, param=None)

.. py:meth:: _check_type_str(self, value)

.. py:meth:: _check_type_list(self, value)

.. py:meth:: _check_type_dict(self, value)

.. py:meth:: _check_type_bool(self, value)

.. py:meth:: _check_type_int(self, value)

.. py:meth:: _check_type_float(self, value)

.. py:meth:: _check_type_path(self, value)

.. py:meth:: _check_type_raw(self, value)

.. py:meth:: _safe_eval(self, value, locals=None, include_exceptions=False)

  * Only copy _safe_eval over if we need it for _check_type_dict()


.. py:meth:: _handle_options(self, argument_spec=None, params=None)

.. py:meth:: _set_defaults(self, pre=True, spec=None, param=None)

.. py:meth:: _set_fallbacks(self, spec=None, param=None)


params/size_validators.py

.. py:meth:: _check_type_bytes(self, value)

.. py:meth:: _check_type_bits(self, value)


params/serialized_validators.py

.. py:meth:: _check_type_jsonarg(self, value)


text/convert.py

.. py:function:: to_bytes()

.. py:function:: to_text()

.. py:function:: to_native()

.. py:function:: container_to_bytes(d: dict, encoding='utf-8': NativeString, errors='surrogate_or_strict':   NativeString) -> dict

    replacement for json_dict_unicode_to_bytes


.. py:function:: container_to_text(d: dict, encoding='utf-8': NativeString, errors='surrogate_or_strict':   NativeString) -> dict

    replacement for json_dict_bytes_to_unicode


.. py:function:: container_to_native(container: Container, encoding='utf-8': NativeString,                           errors='surrogate_or_strict': NativeString) -> dict


text/format.py

.. py:function:: bytes_to_human(size, isbits=False, unit=None)

.. py:function:: human_to_bytes(number, default_unit=None, isbits=False)

.. py:function:: _lenient_lowercase(lst)