Demo of interactive config API

importable_config_object_overview-ep-2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importable config object overview\n",
    "\n",
    "https://github.com/ESMValGroup/ESMValCore/pull/868\n",
    "\n",
    "The main goal of this branch is to get an importable config object (`CFG`) for interactive use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/stef/r/esmvalcore/esmvalcore/future/config/_config_validators.py:235: ESMValToolDeprecationWarning: `write_plots` will be deprecated in 2.2.0.\n",
      "  'write_plots': deprecate(validate_bool, 'write_plots', '2.2.0'),\n",
      "/home/stef/r/esmvalcore/esmvalcore/future/config/_config_validators.py:236: ESMValToolDeprecationWarning: `write_netcdf` will be deprecated in 2.2.0.\n",
      "  'write_netcdf': deprecate(validate_bool, 'write_netcdf', '2.2.0'),\n",
      "/home/stef/r/esmvalcore/esmvalcore/future/config/_config_validators.py:237: ESMValToolDeprecationWarning: `output_file_type` will be deprecated in 2.2.0.\n",
      "  'output_file_type': deprecate(validate_string, 'output_file_type',\n"
     ]
    }
   ],
   "source": [
    "from esmvalcore.future import CFG"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The config object is essentially a Python dictionary with some extra functions, similar to `matplotlib.rcParams`.\n",
    "There are a few changes with the current config. One of them is that it warns us when parameters have been deprecated. The other changes are addressed in the rest of this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ESMValCoreConfig({'auxiliary_data_dir': PosixPath('/mnt/e/eScience/climwip/auxiliary_data'),\n",
       "                  'check_level': <CheckLevels.DEFAULT: 3>,\n",
       "                  'compress_netcdf': False,\n",
       "                  'config_developer_file': None,\n",
       "                  'config_file': PosixPath('/home/stef/.esmvaltool/config-user.yml'),\n",
       "                  'diagnostics': set(),\n",
       "                  'drs': {'CMIP5': 'default',\n",
       "                          'CMIP6': 'default',\n",
       "                          'OBS': 'default',\n",
       "                          'OBS6': 'default',\n",
       "                          'native6': 'default'},\n",
       "                  'exit_on_warning': False,\n",
       "                  'log_level': 'info',\n",
       "                  'max_datasets': None,\n",
       "                  'max_parallel_tasks': 1,\n",
       "                  'max_years': None,\n",
       "                  'output_dir': PosixPath('/mnt/e/eScience/climwip/esmvaltool_output'),\n",
       "                  'output_file_type': 'png',\n",
       "                  'profile_diagnostic': False,\n",
       "                  'remove_preproc_dir': False,\n",
       "                  'rootpath': {'CMIP5': '~/default_inputpath',\n",
       "                               'CMIP6': '~/default_inputpath',\n",
       "                               'OBS6': '~/obs_inputpath',\n",
       "                               'default': '~/default_inputpath',\n",
       "                               'native6': '~/obs_inputpath'},\n",
       "                  'run_diagnostic': True,\n",
       "                  'save_intermediary_cubes': False,\n",
       "                  'skip-nonexistent': False,\n",
       "                  'synda_download': False,\n",
       "                  'write_netcdf': True,\n",
       "                  'write_plots': True})"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CFG"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can update values likes you would expect from a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "CFG['remove_preproc_dir'] = False\n",
    "CFG['output_dir'] = '~/esmvaltool_output'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the type of `output_dir` changed from `str` to `Path`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PosixPath('/home/stef/esmvaltool_output')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CFG['output_dir']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The configuration is validated\n",
    "\n",
    "One of the main differences is that every configuration parameter is validated. This means that invalid parameters will raise an error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "`output_directory` is not a valid config parameter.\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    CFG['output_directory'] = '~/esmvaltool_output'\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As will invalid values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Key `max_parallel_tasks`: Could not convert '🐜' to int\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    CFG['max_parallel_tasks'] = '🐜'\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some values will be corrected to the correct type automatically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "int"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CFG['max_years'] = '123'\n",
    "type(CFG['max_years'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sessions can be started from the config\n",
    "\n",
    "For ESMValCore, we want to start each recipe in a new session. A session is defined by a session directory (i.e. `recipe_name_20201022_123456`), and associated sub-directories (`run_dir`, `work_dir`, etc.).\n",
    "\n",
    "A session can be started from the `CFG`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = CFG.start_session(name='recipe_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`session` is essentially a copy of the CFG, but with some extra functions. Because they are both Python dictionaries, the contents are the same, until we change it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "print(session == CFG)  # True\n",
    "\n",
    "session['output_dir'] = '~/session_output'\n",
    "\n",
    "print(session == CFG)  # False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Session({'auxiliary_data_dir': PosixPath('/mnt/e/eScience/climwip/auxiliary_data'),\n",
       "         'check_level': <CheckLevels.DEFAULT: 3>,\n",
       "         'compress_netcdf': False,\n",
       "         'config_developer_file': None,\n",
       "         'config_file': PosixPath('/home/stef/.esmvaltool/config-user.yml'),\n",
       "         'diagnostics': set(),\n",
       "         'drs': {'CMIP5': 'default',\n",
       "                 'CMIP6': 'default',\n",
       "                 'OBS': 'default',\n",
       "                 'OBS6': 'default',\n",
       "                 'native6': 'default'},\n",
       "         'exit_on_warning': False,\n",
       "         'log_level': 'info',\n",
       "         'max_datasets': None,\n",
       "         'max_parallel_tasks': 1,\n",
       "         'max_years': 123,\n",
       "         'output_dir': PosixPath('/home/stef/session_output'),\n",
       "         'output_file_type': 'png',\n",
       "         'profile_diagnostic': False,\n",
       "         'remove_preproc_dir': False,\n",
       "         'rootpath': {'CMIP5': '~/default_inputpath',\n",
       "                      'CMIP6': '~/default_inputpath',\n",
       "                      'OBS6': '~/obs_inputpath',\n",
       "                      'default': '~/default_inputpath',\n",
       "                      'native6': '~/obs_inputpath'},\n",
       "         'run_diagnostic': True,\n",
       "         'save_intermediary_cubes': False,\n",
       "         'skip-nonexistent': False,\n",
       "         'synda_download': False,\n",
       "         'write_netcdf': True,\n",
       "         'write_plots': True})"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The session directories can be accessed from different properties of the session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "session_dir: /home/stef/esmvaltool_output/recipe_name_20201120_135232\n",
      "run_dir:     /home/stef/esmvaltool_output/recipe_name_20201120_135232/run\n",
      "work_dir:    /home/stef/esmvaltool_output/recipe_name_20201120_135232/work\n",
      "preproc_dir: /home/stef/esmvaltool_output/recipe_name_20201120_135232/preproc\n",
      "plot_dir:    /home/stef/esmvaltool_output/recipe_name_20201120_135232/plots\n"
     ]
    }
   ],
   "source": [
    "print(f\"session_dir: {session.session_dir}\")\n",
    "print(f\"run_dir:     {session.run_dir}\")\n",
    "print(f\"work_dir:    {session.work_dir}\")\n",
    "print(f\"preproc_dir: {session.preproc_dir}\")\n",
    "print(f\"plot_dir:    {session.plot_dir}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## config-developer\n",
    "\n",
    "`config-developer` settings are loaded by default. Updating the `CFG['config_developer_file']` parameter triggers reloading `config-developer` and the CMOR tables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'CMIP6': {'cmor_strict': True,\n",
       "  'input_dir': {'default': '/',\n",
       "   'BADC': '{activity}/{institute}/{dataset}/{exp}/{ensemble}/{mip}/{short_name}/{grid}/{latestversion}',\n",
       "   'DKRZ': '{activity}/{institute}/{dataset}/{exp}/{ensemble}/{mip}/{short_name}/{grid}/{latestversion}',\n",
       "   'ETHZ': '{exp}/{mip}/{short_name}/{dataset}/{ensemble}/{grid}/'},\n",
       "  'input_file': '{short_name}_{mip}_{dataset}_{exp}_{ensemble}_{grid}*.nc',\n",
       "  'output_file': '{project}_{dataset}_{mip}_{exp}_{ensemble}_{short_name}',\n",
       "  'cmor_type': 'CMIP6'},\n",
       " 'CMIP5': {'cmor_strict': True,\n",
       "  'input_dir': {'default': '/',\n",
       "   'BADC': '{institute}/{dataset}/{exp}/{frequency}/{modeling_realm}/{mip}/{ensemble}/{latestversion}/{short_name}',\n",
       "   'CP4CDS': '{institute}/{dataset}/{exp}/{frequency}/{modeling_realm}/{mip}/{ensemble}/{short_name}/latest/',\n",
       "   'DKRZ': '{institute}/{dataset}/{exp}/{frequency}/{modeling_realm}/{mip}/{ensemble}/{latestversion}/{short_name}',\n",
       "   'ETHZ': '{exp}/{mip}/{short_name}/{dataset}/{ensemble}/',\n",
       "   'SMHI': '{dataset}/{ensemble}/{exp}/{frequency}',\n",
       "   'RCAST': '{exp}/{mip}/{short_name}/{dataset}/{ensemble}/',\n",
       "   'BSC': '{type}/{project}/{exp}/{dataset.lower}'},\n",
       "  'input_file': '{short_name}_{mip}_{dataset}_{exp}_{ensemble}*.nc',\n",
       "  'output_file': '{project}_{dataset}_{mip}_{exp}_{ensemble}_{short_name}',\n",
       "  'institutes': {'ACCESS1-0': ['CSIRO-BOM'],\n",
       "   'ACCESS1-3': ['CSIRO-BOM'],\n",
       "   'bcc-csm1-1': ['BCC'],\n",
       "   'bcc-csm1-1-m': ['BCC'],\n",
       "   'BNU-ESM': ['BNU'],\n",
       "   'CanAM4': ['CCCma'],\n",
       "   'CanCM4': ['CCCma'],\n",
       "   'CanESM2': ['CCCma'],\n",
       "   'CCSM4': ['NCAR'],\n",
       "   'CESM1-BGC': ['NSF-DOE-NCAR'],\n",
       "   'CESM1-CAM5': ['NSF-DOE-NCAR'],\n",
       "   'CESM1-CAM5-1-FV2': ['NSF-DOE-NCAR'],\n",
       "   'CESM1-FASTCHEM': ['NSF-DOE-NCAR'],\n",
       "   'CESM1-WACCM': ['NSF-DOE-NCAR'],\n",
       "   'CFSv2-2011': ['COLA-CFS', 'NOAA-NCEP'],\n",
       "   'CMCC-CESM': ['CMCC'],\n",
       "   'CMCC-CM': ['CMCC'],\n",
       "   'CMCC-CMS': ['CMCC'],\n",
       "   'CNRM-CM5': ['CNRM-CERFACS'],\n",
       "   'CNRM-CM5-2': ['CNRM-CERFACS'],\n",
       "   'CSIRO-Mk3-6-0': ['CSIRO-QCCCE'],\n",
       "   'CSIRO-Mk3L-1-2': ['CSIRO-QCCCE'],\n",
       "   'EC-EARTH': ['ICHEC'],\n",
       "   'FGOALS-g2': ['LASG-CESS'],\n",
       "   'FGOALS-gl': ['LASG-IAP'],\n",
       "   'FGOALS-s2': ['LASG-IAP'],\n",
       "   'FIO-ESM': ['FIO'],\n",
       "   'fio-esm': ['FIO'],\n",
       "   'GEOS-5': ['NASA-GMAO'],\n",
       "   'GFDL-CM2p1': ['NOAA-GFDL'],\n",
       "   'GFDL-CM3': ['NOAA-GFDL'],\n",
       "   'GFDL-ESM2G': ['NOAA-GFDL'],\n",
       "   'GFDL-ESM2M': ['NOAA-GFDL'],\n",
       "   'GFDL-HIRAM-C180': ['NOAA-GFDL'],\n",
       "   'GFDL-HIRAM-C360': ['NOAA-GFDL'],\n",
       "   'GISS-E2-H': ['NASA-GISS'],\n",
       "   'GISS-E2-H-CC': ['NASA-GISS'],\n",
       "   'GISS-E2-R': ['NASA-GISS'],\n",
       "   'GISS-E2-R-CC': ['NASA-GISS'],\n",
       "   'HadCM3': ['MOHC'],\n",
       "   'HadGEM2-A': ['MOHC'],\n",
       "   'HadGEM2-AO': ['NIMR-KMA'],\n",
       "   'HadGEM2-CC': ['MOHC'],\n",
       "   'HadGEM2-ES': ['INPE', 'MOHC'],\n",
       "   'inmcm4': ['INM'],\n",
       "   'IPSL-CM5A-LR': ['IPSL'],\n",
       "   'IPSL-CM5A-MR': ['IPSL'],\n",
       "   'IPSL-CM5B-LR': ['IPSL'],\n",
       "   'MIROC-ESM': ['MIROC'],\n",
       "   'MIROC-ESM-CHEM': ['MIROC'],\n",
       "   'MIROC4h': ['MIROC'],\n",
       "   'MIROC5': ['MIROC'],\n",
       "   'MPI-ESM-LR': ['MPI-M'],\n",
       "   'MPI-ESM-MR': ['MPI-M'],\n",
       "   'MPI-ESM-P': ['MPI-M'],\n",
       "   'MRI-AGCM3-2H': ['MRI'],\n",
       "   'MRI-AGCM3-2S': ['MRI'],\n",
       "   'MRI-CGCM3': ['MRI'],\n",
       "   'MRI-ESM1': ['MRI'],\n",
       "   'NICAM-09': ['NICAM'],\n",
       "   'NorESM1-M': ['NCC'],\n",
       "   'NorESM1-ME': ['NCC']}},\n",
       " 'CMIP3': {'cmor_strict': True,\n",
       "  'input_dir': {'default': '/',\n",
       "   'BADC': '{institute}/{dataset}/{exp}/{frequency}/{modeling_realm}/{short_name}/{ensemble}/{latestversion}',\n",
       "   'DKRZ': '{exp}/{modeling_realm}/{frequency}/{short_name}/{dataset}/{ensemble}'},\n",
       "  'input_file': '{short_name}_*.nc',\n",
       "  'output_file': '{project}_{institute}_{dataset}_{mip}_{exp}_{ensemble}_{short_name}_{start_year}-{end_year}',\n",
       "  'cmor_type': 'CMIP3',\n",
       "  'institutes': {'bccr_bcm2_0': ['BCCR'],\n",
       "   'cccma_cgcm3_1': ['CCCMA'],\n",
       "   'cccma_cgcm3_1_t63': ['CCCMA'],\n",
       "   'cnrm_cm3': ['CNRM'],\n",
       "   'csiro_mk3_0': ['CSIRO'],\n",
       "   'csiro_mk3_5': ['CSIRO'],\n",
       "   'gfdl_cm2_0': ['GFDL'],\n",
       "   'gfdl_cm2_1': ['GFDL'],\n",
       "   'ingv_echam4': ['INGV'],\n",
       "   'inmcm3_0': ['INM'],\n",
       "   'ipsl_cm4': ['IPSL'],\n",
       "   'iap_fgoals1_0_g': ['LASG'],\n",
       "   'miub_echo_g': ['MIUB-KMA'],\n",
       "   'mpi_echam5': ['MPIM'],\n",
       "   'mri_cgcm2_3_2a': ['MRI'],\n",
       "   'giss_aom': ['NASA'],\n",
       "   'giss_model_e_h': ['NASA'],\n",
       "   'giss_model_e_r': ['NASA'],\n",
       "   'ncar_ccsm3_0': ['NCAR'],\n",
       "   'ncar_pcm1': ['NCAR'],\n",
       "   'miroc3_2_hires': ['NIES'],\n",
       "   'miroc3_2_medres': ['NIES'],\n",
       "   'ukmo_hadcm3': ['UKMO'],\n",
       "   'ukmo_hadgem1': ['UKMO'],\n",
       "   'BCM2': ['BCCR'],\n",
       "   'CGCM3-1-T47': ['CCCMA'],\n",
       "   'CGCM3-1-T63': ['CCCMA'],\n",
       "   'BCC-CM1': ['CMA'],\n",
       "   'CM3': ['CNRM', 'INM'],\n",
       "   'MK3': ['CSIRO'],\n",
       "   'MK3-5': ['CSIRO'],\n",
       "   'CM2': ['GFDL'],\n",
       "   'CM2-1': ['GFDL'],\n",
       "   'ECHAM4': ['INGV'],\n",
       "   'CM4': ['IPSL'],\n",
       "   'FGOALS-G1-0': ['LASG'],\n",
       "   'ECHO-G': ['MIUB-KMA'],\n",
       "   'ECHAM5': ['MPIM'],\n",
       "   'GISS-AOM': ['NASA'],\n",
       "   'GISS-EH': ['NASA'],\n",
       "   'GISS-ER': ['NASA'],\n",
       "   'CCSM3': ['NCAR'],\n",
       "   'PCM': ['NCAR'],\n",
       "   'MIROC3-2-HI': ['NIES'],\n",
       "   'MIROC3-2-MED': ['NIES'],\n",
       "   'HADCM3': ['UKMO'],\n",
       "   'HADGEM1': ['UKMO']}},\n",
       " 'OBS': {'cmor_strict': False,\n",
       "  'input_dir': {'default': 'Tier{tier}/{dataset}',\n",
       "   'BSC': '{type}/{institute.lower}/{dataset.lower}/{freq_folder}/{short_name}{freq_base}',\n",
       "   'RCAST': '{dataset}'},\n",
       "  'input_file': {'default': '{project}_{dataset}_{type}_{version}_{mip}_{short_name}[_.]*nc',\n",
       "   'BSC': '{short_name}_*.nc',\n",
       "   'RCAST': '{short_name}_{mip}_{type}_{dataset}_*.nc'},\n",
       "  'output_file': '{project}_{dataset}_{type}_{version}_{mip}_{short_name}',\n",
       "  'cmor_type': 'CMIP5'},\n",
       " 'OBS6': {'cmor_strict': False,\n",
       "  'input_dir': {'default': 'Tier{tier}/{dataset}',\n",
       "   'BSC': '{type}/{institute.lower}/{dataset.lower}/{freq_folder}/{short_name}{freq_base}'},\n",
       "  'input_file': {'default': '{project}_{dataset}_{type}_{version}_{mip}_{short_name}[_.]*nc',\n",
       "   'BSC': '{short_name}_*.nc'},\n",
       "  'output_file': '{project}_{dataset}_{type}_{version}_{mip}_{short_name}',\n",
       "  'cmor_type': 'CMIP6'},\n",
       " 'native6': {'cmor_strict': False,\n",
       "  'input_dir': {'default': 'Tier{tier}/{dataset}/{latestversion}/{frequency}/{short_name}'},\n",
       "  'input_file': {'default': '*.nc'},\n",
       "  'output_file': '{project}_{dataset}_{type}_{version}_{mip}_{short_name}',\n",
       "  'cmor_type': 'CMIP6',\n",
       "  'cmor_default_table_prefix': 'CMIP6_'},\n",
       " 'obs4mips': {'cmor_strict': False,\n",
       "  'input_dir': {'default': 'Tier{tier}/{dataset}', 'RCAST': '/'},\n",
       "  'input_file': '{short_name}_{dataset}_{level}_{version}_*.nc',\n",
       "  'output_file': '{project}_{dataset}_{level}_{version}_{short_name}',\n",
       "  'cmor_type': 'CMIP6',\n",
       "  'cmor_path': 'obs4mips',\n",
       "  'cmor_default_table_prefix': 'obs4MIPs_'},\n",
       " 'ana4mips': {'cmor_strict': False,\n",
       "  'input_dir': {'default': 'Tier{tier}/{dataset}', 'RCAST': '/'},\n",
       "  'input_file': '{short_name}_{mip}_{type}_{dataset}_*.nc',\n",
       "  'output_file': '{project}_{mip}_{type}_{dataset}_{short_name}',\n",
       "  'cmor_type': 'CMIP5'},\n",
       " 'EMAC': {'input_dir': {'default': '{dataset}'},\n",
       "  'input_file': '',\n",
       "  'output_file': '{dataset}_{ensemble}_{short_name}',\n",
       "  'cmor_type': 'CMIP5'},\n",
       " 'CORDEX': {'input_dir': {'default': '/',\n",
       "   'spec': '{domain}/{institute}/{driver}/{exp}/{ensemble}/{dataset}/{rcm_version}/{mip}/{short_name}',\n",
       "   'BADC': '{domain}/{institute}/{driver}/{exp}/{ensemble}/{dataset}/{rcm_version}/{mip}/{short_name}/{latestversion}'},\n",
       "  'input_file': '{short_name}_{domain}_{driver}_{exp}_{ensemble}_{dataset}_{rcm_version}_{mip}*.nc',\n",
       "  'output_file': '{short_name}_{dataset}_{exp}_{ensemble}_{rcm_version}_{mip}',\n",
       "  'cmor_type': 'CMIP5',\n",
       "  'cmor_path': 'cordex'}}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from esmvalcore._config import CFG as CFG_DEV\n",
    "CFG_DEV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading a different config file\n",
    "\n",
    "Different config files can be loaded through the config object. Notice that the `config_file` value reflects the given file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ESMValCoreConfig({'auxiliary_data_dir': PosixPath('/mnt/e/eScience/climwip/auxiliary_data'),\n",
       "                  'check_level': <CheckLevels.DEFAULT: 3>,\n",
       "                  'compress_netcdf': False,\n",
       "                  'config_developer_file': None,\n",
       "                  'config_file': PosixPath('/home/stef/.esmvaltool/config-user_example.yml'),\n",
       "                  'diagnostics': set(),\n",
       "                  'drs': {'CMIP5': 'default',\n",
       "                          'CMIP6': 'default',\n",
       "                          'OBS': 'default',\n",
       "                          'OBS6': 'default',\n",
       "                          'native6': 'default'},\n",
       "                  'exit_on_warning': False,\n",
       "                  'log_level': 'info',\n",
       "                  'max_datasets': None,\n",
       "                  'max_parallel_tasks': 1,\n",
       "                  'max_years': None,\n",
       "                  'output_dir': PosixPath('/mnt/e/eScience/climwip/example'),\n",
       "                  'output_file_type': 'png',\n",
       "                  'profile_diagnostic': False,\n",
       "                  'remove_preproc_dir': False,\n",
       "                  'rootpath': {'CMIP5': '~/example',\n",
       "                               'CMIP6': '~/example',\n",
       "                               'OBS6': '~/example',\n",
       "                               'default': '~/example',\n",
       "                               'native6': '~/example'},\n",
       "                  'run_diagnostic': True,\n",
       "                  'save_intermediary_cubes': False,\n",
       "                  'skip-nonexistent': False,\n",
       "                  'synda_download': False,\n",
       "                  'write_netcdf': True,\n",
       "                  'write_plots': True})"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CFG.load_from_file('~/.esmvaltool/config-user_example.yml')\n",
    "CFG"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Changes to `config-user.yml`\n",
    "\n",
    "No changes 🥳"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "esmvaltool",
   "language": "python",
   "name": "esmvaltool"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}