real-python-interesting-resources.md

ace-python-interview.ipynb

logging.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logging in Python\n",
    "\n",
    "https://realpython.com/lessons/logging-python-introduction/\n",
    "\n",
    "## 01. Intro\n",
    "\n",
    "## 02. The logging module\n",
    "\n",
    "- severity levels\n",
    "\n",
    "  Only log events from each severity level and upwards\n",
    "\n",
    "- root\n",
    "\n",
    "  `root` is the name of the default logger\n",
    "\n",
    "## 03. Basic Configuration\n",
    "\n",
    "- `basicConfig(**kwargs)`\n",
    "\n",
    "  ```python\n",
    "  # main.py\n",
    "  import logging\n",
    "\n",
    "  logging.basicConfig(\n",
    "    filename='app.log',\n",
    "    filemode='w',\n",
    "    format='%(name)s - %(levelname)s - %(message)s',\n",
    "    level=logging.DEBUG)\n",
    "  logging.debug('This will get logged') # the default is logging.warning('something')\n",
    "  ```\n",
    "\n",
    "  Output:\n",
    "\n",
    "  ```bash\n",
    "    $ python main.py\n",
    "    $ cat app.log\n",
    "    root - DEBUG - This will get logged\n",
    "  ```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(\n",
    "  filename='app.log',\n",
    "  filemode='w',\n",
    "  format='%(name)s - %(levelname)s - %(message)s',\n",
    "  level=logging.DEBUG)\n",
    "logging.debug('This will get logged') # the default is logging.warning('something')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 04. Formatting the output\n",
    "\n",
    "- Can also add\n",
    "  - the process id to formatted output\n",
    "    ```python\n",
    "    logging.basicConfig(format='%(process)d - %(levelname)s - %(message)s' )\n",
    "    ```\n",
    "  - the creation time\n",
    "    ```python\n",
    "    logging.basicConfig(\n",
    "        format='%(asctime)s - %(levelname)s - %(message)s'\n",
    "        dtfmt='%d-%b-%y %H:%M:%S' )\n",
    "    ```\n",
    "\n",
    "## 05. Logging variable data\n",
    "\n",
    "- Use f-strings\n",
    "\n",
    "## 06. Capturing stack traces\n",
    "\n",
    "- pass `exc_info=True`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "a = 5\n",
    "b = 0\n",
    "\n",
    "try:\n",
    "    c = a/b\n",
    "except Exception as e:\n",
    "    # or, shorthand = `logging.exception()`\n",
    "    logging.error('Exception occurred', exc_info=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "  Output:\n",
    "\n",
    "  ```bash\n",
    "      $ python main.py\n",
    "\n",
    "      ERROR:root:Exception occurred\n",
    "      Traceback (most recent call last):\n",
    "      File \"/home/jon/code/playground/real-python-videos-logging/exception.py\", line 6, in <module>\n",
    "          c = a/b\n",
    "              ~^~\n",
    "      ZeroDivisionError: division by zero\n",
    "  ```\n",
    "\n",
    "## 07. Classes and Functions\n",
    "\n",
    "- For more customisable, instead of root: `logging` module contains:\n",
    "\n",
    "  - `logging.Logger`\n",
    "\n",
    "  which creates:\n",
    "\n",
    "  - `logging.LogRecord` events\n",
    "\n",
    "  - `logging.Handler`\n",
    "\n",
    "    - sends the `LogRecord` to required output detination like the console or file with subclass\n",
    "\n",
    "  - `logging.Formatter`\n",
    "    - the string format of the output\n",
    "\n",
    "- Use the `getLogger()` method to create a custom logger to create a new logger object\n",
    "\n",
    "  ```python\n",
    "  import logging\n",
    "  # create a new custom logger object.\n",
    "  logger = logging.getLogger('example_logger')\n",
    "\n",
    "  # create a warning level even on this new logger\n",
    "  # by default, a custom logger has no formatting so we\n",
    "  # don't see severity level\n",
    "  logging.warning('This is a warning')\n",
    "  ```\n",
    "\n",
    "## 08. Creating a custom logger\n",
    "\n",
    "Than can output to stdout and a file simultaneously.\n",
    "\n",
    "- create a custom logger\n",
    "- create some handlers that represent output destinations\n",
    "  - attach formatters to those handlers\n",
    "  - and attach the handers to the logger so we can use it\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "__main__ - WARNING - This is a warning\n",
      "__main__ - ERROR - This is an error\n"
     ]
    }
   ],
   "source": [
    "# 08-customer-logger-1.py\n",
    "import logging\n",
    "\n",
    "# create the custom logger object\n",
    "# `__name__` represents the name of the current module\n",
    "logger = logging.getLogger(__name__)\n",
    "\n",
    "# first, create handlers\n",
    "\n",
    "console_handler = logging.StreamHandler()\n",
    "file_handler = logging.FileHandler('file.log')\n",
    "\n",
    "# set the minimium severity level\n",
    "# so only events (LogRecords) that are WARNING or more severe will be logged in the console\n",
    "console_handler.setLevel(logging.WARNING)\n",
    "# ..only events that are marked as ERROR or more severe, will be logged to the external file.\n",
    "file_handler.setLevel(logging.ERROR)\n",
    "\n",
    "# create a formatter for each handler\n",
    "console_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')\n",
    "file_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')\n",
    "\n",
    "# attach formatting objects to each handler\n",
    "console_handler.setFormatter(console_format)\n",
    "file_handler.setFormatter(file_format)\n",
    "\n",
    "logger.addHandler(console_handler)\n",
    "logger.addHandler(file_handler)\n",
    "\n",
    "logger.warning('This is a warning')\n",
    "logger.error('This is an error')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 08. Logger from Config File\n",
    "\n",
    "Stored in `logging.conf`\n",
    "\n",
    "```conf\n",
    "[loggers]\n",
    "keys=root,sampleLogger\n",
    "\n",
    "[handlers]\n",
    "keys=consoleHandler\n",
    "\n",
    "[formatters]\n",
    "keys=sampleFormatter\n",
    "\n",
    "[logger_root]\n",
    "level=DEBUG\n",
    "handlers=consoleHandler\n",
    "\n",
    "[logger_sampleLogger]\n",
    "level=DEBUG\n",
    "handlers=consoleHandler\n",
    "qualname=sampleLogger\n",
    "propagate=0\n",
    "\n",
    "[handler_consoleHandler]\n",
    "class=StreamHandler\n",
    "level=DEBUG\n",
    "formatter=sampleFormatter\n",
    "args=(sys.stdout,)\n",
    "\n",
    "[formatter_sampleFormatter]\n",
    "format=%(asctime)s - %(name)s - %(levelname)s - %(message)s\n",
    "\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 09. logging config from a dictionary\n",
    "\n",
    "```python\n",
    "version: 1\n",
    "formatters:\n",
    "  simple:\n",
    "    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'\n",
    "handlers:\n",
    "  console:\n",
    "    class: logging.StreamHandler\n",
    "    level: DEBUG\n",
    "    formatter: simple\n",
    "    stream: ext://sys.stdout\n",
    "loggers:\n",
    "  sampleLogger:\n",
    "    level: DEBUG\n",
    "    handlers: [console]\n",
    "    propagate: no\n",
    "root:\n",
    "  level: DEBUG\n",
    "  handlers: [console]\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

real-python-interesting-resources.md

Real Python Interesting Resources

1. Threading in Python

   • https://realpython.com/courses/threading-python/

2. Logging in Python

   • https://realpython.com/courses/logging-python/
   • see attached notebook

3. Ace your Python Interview

   • https://realpython.com/learning-paths/python-interview/

4. Django Ninja

   • https://realpython.com/courses/rest-apis-with-django-ninja/

5. What are Python generators

   • https://realpython.com/lessons/what-are-python-generators/

6. Python profiling (article)

   • https://realpython.com/python-profiling/

---

Threading in Python