Skip to content

Instantly share code, notes, and snippets.

@tjburch

tjburch/tabpfn_census_compare.ipynb

Last active October 24, 2022 14:23
Show Gist options
  • Save tjburch/c0abe475f399f8d0b9fb74a7cb414d07 to your computer and use it in GitHub Desktop.
Save tjburch/c0abe475f399f8d0b9fb74a7cb414d07 to your computer and use it in GitHub Desktop.
Download ZIP
tabpfn_census_compare.ipynb
Display the source blob
Display the rendered blob
Raw
tabpfn_census_compare.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"collapsed_sections": [],
"authorship_tag": "ABX9TyOfeG2JwIM3ckY6fCWTM7PV",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/tjburch/c0abe475f399f8d0b9fb74a7cb414d07/tabpfn_census_compare.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"# TabPFN Test\n",
"\n",
"Benchmarking TabPFN against XGBoost for a real-world datasets."
],
"metadata": {
"id": "PvTLowEpdteO"
}
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"collapsed": true,
"id": "-wE_gQ9MWxth",
"outputId": "62059822-3d1f-4f6e-ab02-ab44402a4638"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting tabpfn\n",
" Downloading tabpfn-0.1.3-py3-none-any.whl (136 kB)\n",
"\u001b[K |████████████████████████████████| 136 kB 35.8 MB/s \n",
"\u001b[?25hCollecting hyperopt>=0.2.5\n",
" Downloading hyperopt-0.2.7-py2.py3-none-any.whl (1.6 MB)\n",
"\u001b[K |████████████████████████████████| 1.6 MB 59.2 MB/s \n",
"\u001b[?25hRequirement already satisfied: torch>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from tabpfn) (1.12.1+cu113)\n",
"Collecting gpytorch>=1.5.0\n",
" Downloading gpytorch-1.8.1-py2.py3-none-any.whl (361 kB)\n",
"\u001b[K |████████████████████████████████| 361 kB 71.4 MB/s \n",
"\u001b[?25hRequirement already satisfied: seaborn>=0.11.2 in /usr/local/lib/python3.7/dist-packages (from tabpfn) (0.11.2)\n",
"Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.7/dist-packages (from tabpfn) (4.64.1)\n",
"Collecting configspace>=0.4.21\n",
" Downloading ConfigSpace-0.6.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)\n",
"\u001b[K |████████████████████████████████| 4.9 MB 46.9 MB/s \n",
"\u001b[?25hRequirement already satisfied: numpy>=1.21.2 in /usr/local/lib/python3.7/dist-packages (from tabpfn) (1.21.6)\n",
"Requirement already satisfied: scikit-learn>=0.24.2 in /usr/local/lib/python3.7/dist-packages (from tabpfn) (1.0.2)\n",
"Requirement already satisfied: pyyaml>=5.4.1 in /usr/local/lib/python3.7/dist-packages (from tabpfn) (6.0)\n",
"Collecting openml>=0.12.2\n",
" Downloading openml-0.12.2.tar.gz (119 kB)\n",
"\u001b[K |████████████████████████████████| 119 kB 66.7 MB/s \n",
"\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from configspace>=0.4.21->tabpfn) (4.1.1)\n",
"Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from configspace>=0.4.21->tabpfn) (1.7.3)\n",
"Requirement already satisfied: cython in /usr/local/lib/python3.7/dist-packages (from configspace>=0.4.21->tabpfn) (0.29.32)\n",
"Requirement already satisfied: pyparsing in /usr/local/lib/python3.7/dist-packages (from configspace>=0.4.21->tabpfn) (3.0.9)\n",
"Collecting py4j\n",
" Downloading py4j-0.10.9.7-py2.py3-none-any.whl (200 kB)\n",
"\u001b[K |████████████████████████████████| 200 kB 68.7 MB/s \n",
"\u001b[?25hRequirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->tabpfn) (1.15.0)\n",
"Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->tabpfn) (2.6.3)\n",
"Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->tabpfn) (0.16.0)\n",
"Requirement already satisfied: cloudpickle in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.2.5->tabpfn) (1.5.0)\n",
"Collecting liac-arff>=2.4.0\n",
" Downloading liac-arff-2.5.0.tar.gz (13 kB)\n",
"Collecting xmltodict\n",
" Downloading xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from openml>=0.12.2->tabpfn) (2.23.0)\n",
"Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from openml>=0.12.2->tabpfn) (2.8.2)\n",
"Requirement already satisfied: pandas>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from openml>=0.12.2->tabpfn) (1.3.5)\n",
"Collecting minio\n",
" Downloading minio-7.1.12-py3-none-any.whl (76 kB)\n",
"\u001b[K |████████████████████████████████| 76 kB 3.0 MB/s \n",
"\u001b[?25hRequirement already satisfied: pyarrow in /usr/local/lib/python3.7/dist-packages (from openml>=0.12.2->tabpfn) (6.0.1)\n",
"Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.0->openml>=0.12.2->tabpfn) (2022.4)\n",
"Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.24.2->tabpfn) (3.1.0)\n",
"Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.24.2->tabpfn) (1.2.0)\n",
"Requirement already satisfied: matplotlib>=2.2 in /usr/local/lib/python3.7/dist-packages (from seaborn>=0.11.2->tabpfn) (3.2.2)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.2->seaborn>=0.11.2->tabpfn) (1.4.4)\n",
"Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.2->seaborn>=0.11.2->tabpfn) (0.11.0)\n",
"Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/dist-packages (from minio->openml>=0.12.2->tabpfn) (1.24.3)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from minio->openml>=0.12.2->tabpfn) (2022.9.24)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->openml>=0.12.2->tabpfn) (2.10)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->openml>=0.12.2->tabpfn) (3.0.4)\n",
"Building wheels for collected packages: openml, liac-arff\n",
" Building wheel for openml (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for openml: filename=openml-0.12.2-py3-none-any.whl size=137326 sha256=e67e04c06f4bf0456685b7216c10ec809c9d9f2accaeaa3ba09a95d4fc95a2da\n",
" Stored in directory: /root/.cache/pip/wheels/6a/20/88/cf4ac86aa18e2cd647ed16ebe274a5dacee9d0075fa02af250\n",
" Building wheel for liac-arff (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for liac-arff: filename=liac_arff-2.5.0-py3-none-any.whl size=11732 sha256=871be641904f199795284946400d52c018c10303c55b83e9979af4e6df27dec6\n",
" Stored in directory: /root/.cache/pip/wheels/1f/0f/15/332ca86cbebf25ddf98518caaf887945fbe1712b97a0f2493b\n",
"Successfully built openml liac-arff\n",
"Installing collected packages: xmltodict, py4j, minio, liac-arff, openml, hyperopt, gpytorch, configspace, tabpfn\n",
" Attempting uninstall: hyperopt\n",
" Found existing installation: hyperopt 0.1.2\n",
" Uninstalling hyperopt-0.1.2:\n",
" Successfully uninstalled hyperopt-0.1.2\n",
"Successfully installed configspace-0.6.0 gpytorch-1.8.1 hyperopt-0.2.7 liac-arff-2.5.0 minio-7.1.12 openml-0.12.2 py4j-0.10.9.7 tabpfn-0.1.3 xmltodict-0.13.0\n"
]
}
],
"source": [
"! pip install tabpfn;"
]
},
{
"cell_type": "code",
"source": [
"import matplotlib.pyplot as plt\n",
"plt.style.use(\"bmh\")\n",
"import numpy as np\n",
"import pandas as pd\n",
"from pathlib import Path\n",
"from sklearn.preprocessing import LabelEncoder, LabelBinarizer\n",
"from sklearn.metrics import accuracy_score, roc_auc_score\n",
"from tqdm import tqdm\n",
"import torch\n",
"from xgboost import XGBClassifier"
],
"metadata": {
"id": "KB0DJj3bXZIX"
},
"execution_count": 2,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Test Case: Census Income Data\n",
"\n",
"Check for dataset with tens-of-thousands of points: train set size ~30,000, test set size ~16,000. Classification of income bracket based on various identifiers of reported person.\n",
"\n",
"Thanks to Bojan Tunguz (@tunguz) for maintaining the very nice [TabularBenchmarks](https://github.com/tunguz/TabularBenchmarks/) repository, which houses this data. I will replicate the data preprocessing performed within that repository."
],
"metadata": {
"id": "KT-SHMitdtp_"
}
},
{
"cell_type": "code",
"source": [
"CSV_HEADER = [\n",
" \"age\",\n",
" \"workclass\",\n",
" \"fnlwgt\",\n",
" \"education\",\n",
" \"education_num\",\n",
" \"marital_status\",\n",
" \"occupation\",\n",
" \"relationship\",\n",
" \"race\",\n",
" \"gender\",\n",
" \"capital_gain\",\n",
" \"capital_loss\",\n",
" \"hours_per_week\",\n",
" \"native_country\",\n",
" \"income_bracket\",\n",
"]\n",
"\n",
"df_train = (\n",
" pd.read_csv(\"https://raw.githubusercontent.com/tunguz/TabularBenchmarks/main/datasets/census_income/input/adult.data\", header=None, names=CSV_HEADER)\n",
" .assign(income_bracket=lambda df_:(df_.income_bracket.str[:-1].str.strip() == \">50\").astype(int))\n",
")\n",
"df_test =(\n",
" pd.read_csv(\"https://raw.githubusercontent.com/tunguz/TabularBenchmarks/main/datasets/census_income/input/adult.test\", header=None, names=CSV_HEADER)\n",
" .assign(income_bracket=lambda df_:(df_.income_bracket.str[:-1].str.strip() == \">50K\").astype(int))\n",
") "
],
"metadata": {
"id": "8VAeqxr-W71e"
},
"execution_count": 3,
"outputs": []
},
{
"cell_type": "code",
"source": [
"print(\"Train set size:\")\n",
"print(df_train.income_bracket.value_counts())\n",
"print(\"Test set size:\")\n",
"print(df_test.income_bracket.value_counts())"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "6dPZu3dJXklR",
"outputId": "d55984ea-0806-4040-c65e-5d41604bd5cc"
},
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Train set size:\n",
"0 24720\n",
"1 7841\n",
"Name: income_bracket, dtype: int64\n",
"Test set size:\n",
"0 12435\n",
"1 3846\n",
"Name: income_bracket, dtype: int64\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Note that the ratio of those earning less than 50K and those making more than 50K (encoded as 0 and 1 respectively) is a bit over 3 to 1, so a baseline all-0 model would be about 75% accurate."
],
"metadata": {
"id": "ir5Pl0ULe3bm"
}
},
{
"cell_type": "code",
"source": [
"zeros_acc = accuracy_score(df_test['income_bracket'], np.zeros(len(df_test)))\n",
"print(f\"All-Zero Accuracy: {zeros_acc}\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1a4uU3oNZaZ9",
"outputId": "b3e18370-a09f-4641-e8b9-04d4e147f83c"
},
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"All-Zero Accuracy: 0.7637737239727289\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"### Data Preprocessing"
],
"metadata": {
"id": "NrMFavL1fVCx"
}
},
{
"cell_type": "code",
"source": [
"def preprocess_categoricals(df, encoders=None):\n",
" \n",
" # Binaries and str\n",
" df = (\n",
" df\n",
" .assign(gender=(df.gender == ' Male').astype(int))\n",
" .assign(native_country=(df.native_country == ' United-States').astype(int))\n",
" )\n",
" cat_columns = ['workclass', 'marital_status', 'occupation', 'relationship', 'race']\n",
"\n",
" if encoders is None:\n",
" encoders = {}\n",
" for col in cat_columns:\n",
" le = LabelEncoder()\n",
" df[col] = le.fit_transform(df[col])\n",
" encoders[col] = le\n",
" return df, encoders\n",
" else:\n",
" for col in cat_columns:\n",
" df[col] = encoders[col].transform(df[col])\n",
" return df, encoders\n",
"\n",
"df_train, label_encoders = preprocess_categoricals(df_train)\n",
"df_test, _ = preprocess_categoricals(df_test, label_encoders)"
],
"metadata": {
"id": "dxqJ4U5SYE6w"
},
"execution_count": 6,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### TabPFN Model\n",
"\n",
"We'll load the TabPFN Classifier and fit the data with a random sampling of 1024 points from the dataset, the maximum currently workable for TabPFN"
],
"metadata": {
"id": "AeO0ySRrfa9i"
}
},
{
"cell_type": "code",
"source": [
"from tabpfn.scripts.transformer_prediction_interface import TabPFNClassifier"
],
"metadata": {
"id": "6cqlTES9YLzJ"
},
"execution_count": 7,
"outputs": []
},
{
"cell_type": "code",
"source": [
"columns = ['education_num', 'age', 'capital_gain', 'capital_loss', 'hours_per_week', 'gender', 'native_country',\n",
" 'workclass', 'marital_status', 'occupation', 'relationship', 'race']\n",
"\n",
"np.random.seed(1234)\n",
"sample_idx = np.random.randint(0, len(df_train), size=1024)\n",
"tabpfn_clf = TabPFNClassifier(device='cuda')\n",
"tabpfn_clf.fit(\n",
" df_train[columns].iloc[sample_idx], df_train.iloc[sample_idx]['income_bracket']\n",
")\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "XLkAItJBYrnv",
"outputId": "e9146025-1471-4496-cefe-e39a404b9bd9",
"collapsed": true
},
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"We have to download the TabPFN, as there is no checkpoint at /usr/local/lib/python3.7/dist-packages/tabpfn/models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"It has about 100MB, so this might take a moment.\n",
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dca887320>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dca8879e0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dcaac2050>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dca9188c0>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dca887200>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dca887290>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dca887320>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"TabPFNClassifier(device='cuda')"
]
},
"metadata": {},
"execution_count": 8
}
]
},
{
"cell_type": "markdown",
"source": [
"Right now the colab GPU has memory issues trying to predict the whole dataset at once, so process it in four batches"
],
"metadata": {
"id": "Pr1pCSoigplc"
}
},
{
"cell_type": "code",
"source": [
"preds = np.zeros(len(df_test))\n",
"batches = np.linspace(0, len(df_test), 4)\n",
"for i in range(len(batches)):\n",
" if batches[i] == len(df_test):\n",
" continue\n",
" \n",
" torch.cuda.empty_cache()\n",
" low = int(batches[i])\n",
" hi = int(batches[i+1])\n",
"\n",
" y_eval = tabpfn_clf.predict(df_test[columns].values[low:hi], return_winning_probability=False)\n",
" preds[low:hi] = y_eval"
],
"metadata": {
"id": "k2BWX5YhYuwW"
},
"execution_count": 9,
"outputs": []
},
{
"cell_type": "code",
"source": [
"def pct_gain(a,b):\n",
" return (100*(b-a)) / a\n",
"tabpfn_acc = accuracy_score(df_test['income_bracket'], preds)\n",
"print(f\"TabPFN Accuracy {tabpfn_acc}\")\n",
"print(f\"Pct improvement over zeros: {pct_gain(zeros_acc, tabpfn_acc)}%\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "C2gBp49YY1Wm",
"outputId": "560532ca-76c2-4533-9d2a-86f33d7a7ac7"
},
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"TabPFN Accuracy 0.8341010994410663\n",
"Pct improvement over zeros: 9.20788098110173%\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"### XGBoost Comparisons\n",
"\n",
"1. Accuracy using the same datapoints?"
],
"metadata": {
"id": "bhZeT__6hxi6"
}
},
{
"cell_type": "code",
"source": [
"def train_test_xgb(indices):\n",
"\n",
" params ={\n",
" \"n_estimators\": 300, \n",
" \"subsample\": 0.99, \n",
" \"learning_rate\": 0.05, \n",
" \"colsample_bytree\": 1, \n",
" \"max_depth\": 7, \n",
" \"random_state\": 1234, \n",
" \"tree_method\": \"gpu_hist\"\n",
" } # Borrowed HPs from Bojan's notebook on this dataset, not optimized further\n",
"\n",
" clf = XGBClassifier(**params)\n",
"\n",
" clf.fit(\n",
" df_train.iloc[indices][columns].values, \n",
" df_train.iloc[indices]['income_bracket']\n",
" )\n",
" test_preds = clf.predict(df_test[columns].values)\n",
" return test_preds\n",
"\n",
"\n",
"xgb_samesize_preds = train_test_xgb(sample_idx)\n",
"xgb_samesize_acc = accuracy_score(df_test['income_bracket'], xgb_samesize_preds)\n",
"print(f\"\"\"\n",
" XGB(with same datapoints) improvement over TabPFN: \n",
" {pct_gain(tabpfn_acc, xgb_samesize_acc)}%\n",
"\"\"\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5sjv3H3qh3Ul",
"outputId": "56cc378e-b37a-4044-ae1e-ac756f56679c"
},
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
" XGB(with same datapoints) improvement over TabPFN: \n",
" 1.2960235640648086%\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"2. Accuracy without imposing training set size contraint"
],
"metadata": {
"id": "iYJf5PJBjQ1t"
}
},
{
"cell_type": "code",
"source": [
"xgb_full_preds = train_test_xgb(np.arange(len(df_train)))\n",
"xgb_acc = accuracy_score(df_test['income_bracket'], xgb_full_preds)\n",
"print(f\"\"\"\n",
" XGB (using all datapoints) improvement over TabPFN: \n",
" {pct_gain(tabpfn_acc, xgb_acc)}%\n",
"\"\"\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "guxo1o2-i0N5",
"outputId": "903f69a9-01bc-4fd0-9c68-1d4a100621fa"
},
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
" XGB (using all datapoints) improvement over TabPFN: \n",
" 4.764359351988223%\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"3. Accuracy of XGB as a function of dataset size compared to TabPFN"
],
"metadata": {
"id": "2ytVGk8_ldNm"
}
},
{
"cell_type": "code",
"source": [
"train_sizes = [\n",
" 25, 50, 60, 70, 90, 100, 125, 150, 200, 300, 400, 500, 600,\n",
" 700, 800, 900, 1000, 1300, 1600\n",
"]\n",
"accuracies = []\n",
"for train_size in tqdm(train_sizes):\n",
" i_preds = train_test_xgb(np.random.randint(0, len(df_train), size=train_size))\n",
" accuracies.append(\n",
" accuracy_score(df_test['income_bracket'], i_preds)\n",
" )\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "83v0iJeCYHFr",
"outputId": "4511487e-85aa-44a1-db72-ca8b2c3734dd"
},
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"100%|██████████| 19/19 [00:25<00:00, 1.34s/it]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"plt.figure(figsize=(8,6))\n",
"plt.plot(train_sizes,accuracies, label=\"XGB Accuracy\")\n",
"plt.plot([1024], [tabpfn_acc], marker=\"o\",markersize=14, color=\"firebrick\")\n",
"plt.axhline(tabpfn_acc, \n",
" label=\"TabPFN Accuracy w/1,024 training points\",\n",
" color=\"firebrick\", linestyle=\"--\")\n",
"plt.legend(fontsize=12, frameon=False)\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.xlabel(\"XGB training set size\");"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"id": "adgM5VGvbobU",
"outputId": "a7d234f2-d4ac-4bf5-89f3-cdfb29c344c5"
},
"execution_count": 14,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 576x432 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAFzCAYAAAAjVEDpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxjV33//9eRLNnyJq9jzz5DFiaTmex7oCmdQCAppB1oSFgaaEpbvgO/loZ+fyxtOmxfaGnDl36hK+VHgC8MIQWStqEESGlJCNlmEjKTkD2OZ5F3y5ItW7J0fn9osbxLHuno6KPzfjzmMZZ0r3Secz3++Nx7zrlKa42Li4uLi4uLnHgq3QAXFxcXFxeX0sYVdxcXFxcXF2Fxxd3FxcXFxUVYXHF3cXFxcXERFlfcXVxcXFxchMUVdxcXFxcXF2Gpq3QDSpWf/OQnur6+vuj9tNYopcrQIvORZAFZHkkWkOWRZAHnsTmltkxNTQ3v2bOne6nXxBT3+vp6duzYUfR+w8PDdHV1laFF5iPJArI8kiwgyyPJAs5jc0ptOXjwYN9yr9X8afnJyclKN6FkkWQBWR5JFpDlkWQB57E5Ji01X9x7e3sr3YSSRZIFZHkkWUCWR5IFnMfmmLTUfHEPhUKVbkLJIskCsjySLCDLI8kCzmNzTFpqvrj7fL5KN6FkkWQBWR5JFpDlkWQB57E5Ji01X9yDwWClm1CySLKALI8kC8jySLKA89gck5aaL+7Dw8OVbkLJIskCsjySLCDLI8kCzmNzTFpqvri73wrtjSSPJAvI8kiygPPYHNdzN5h4PF7pJpQskiwgyyPJArI8kizgPDbHpKXmi3ssFqt0E0oWSRaQ5ZFkAVkeSRZwHptj0lLzxd3NobQ3kjySLCDLI8kCzmNz3Dx3g3FzKO2NJI8kC8jySLKA89gcN8/dYPx+f6WbULJIsoAsjyQLyPJIsoDz2JqU1swYvJ2LmBvHrDUtLS2VbkLJIskCsjySLCDLI8kCzmMyWmumEinGYglGp2YzfycYj80yGkswFptldCr991gsQUrDv797PT5v+fvVNV/cR0ZGaG5urnQzShJJFpDlkWQBWR5JFnCeUmRmNl2ws0U5W7jHprJFO/PaVIKZpC74fZt8iomZJJ2NrriXPe3t7ZVuQskiyQKyPJIsIMsjyQLOs1ySKc34dLog5/eqc73sqbne9mQ8WfD7NtR56Gisoz3goz3gy33dEaijvdFHeyD9uC1Qx/RklNZGM0vQ1nxxj8VitLa2VroZJYkkC8jySLKALI8kC9SWR2tNZCaZK8rpwp3+eyyW//Us4elZCu1j13kU7YE6OvKK89zj+YU74PMWbBkzeGxqvrhPT09XugkliyQLyPJIsoAsjyQLyPDEEsncqfBnXp7AO+xhNJbpZWeKdfbxbKqwkq2Atoa6uV52Y6ZIZwp3/uOWei9KqZK7TB6bmi/ubg6lvZHkkWQBWR5JFrDXE0+m5hXn+T3rvOvasVmmZ1ML9o4u+77Nfu9c77qxjo78v/NOkwcb6vB6Sl+wi4nJY1PzxT0UCrF169ZKN6MkkWQBWR5JFpDlkWQBs55kSjMxvXhk+MJe9lgsQWSm8OvY9V6V6Un7qCfOxs4gHYE62uZd0073uP111TOj2+Sxqfni3tDQUOkmlCySLCDLI8kCsjySLFA+z9hUgsMDkxwORXlycJKhaJzx6VkKPCuOV5EbWJa9lt2Rf3o87/p2o8+TOy0+MDBAT09PWUymY/J7reaLeyAQqHQTShZJFpDlkWQBWR5JFiiNR2vNiUicw6Eoh0OTHB6IcjQ8s+S2wYa6XFGeGzU+/3FHY/o6tmcN17ElHR+Tlpov7mNjY2JGlkqygCyPJAvI8kiywNo8Ka15cTSWLuShKIcHJhmZSszbpr7OwxnrGtnV08zu3mY2t9XTFvBRV+br2JKOj0lLzRf3zs7OSjehZJFkAVkeSRaQ5ZFkgcI88WSKZ4emeGIg3TM/MjC5aG53a72XM3ub2d3TxJm9zZzW1Vj2Qr5UJB0fk5aaL+6RSETMak6SLCDLI8kCsjySLLC0Zyqe5MnBSZ7InGZ/emiS+IKV1dY1+9jV08yu3mZ29zaxua1hTafRSx1Jx8ekpeaLezwer3QTShZJFpDlkWQBWR5JFkh7xmKJ3Cn2J0JRXhiNLRr4trWtgV29TZli3sy6Zjtv0CLp+Ji01Hxxt3VO6FoiyQKyPJIsIMtT7RatNaFIPNcrfyIU5djE4XnbeBW8sruR3b3N7Opt4syeZoIN1fHjv9qPT37cPHeDkTTHVZIFZHkkWUCWp9osKa15aXSawwPRXEFfbfDbjnWNRS2TalOq7fisFDfP3WDcNAt7I8kjyQKyPLZbEskUzwxP5U6zHxmYJLpg8FtLvTdzvbyJjfUJLjptY0UGv5Ujth+fYuKmwhmM32/ndaa1RJIFZHkkWUCWxzZLdvBbdo75L5cY/Nbd5MtdK9/V28SWvMFv4+PjYgo72Hd8TiYmLTVf3MPhMG1tbZVuRkkiyQKyPJIsIMtTactYLMGR0GRmWlqU50dObvBbpT2ljiSPSUvNF/eurq5KN6FkkWQBWR5JFpDlMWnRWhOKzq389kRo8cpvnpMc/Cbp2IAsj0lLzRf3cDhMU1NTpZtRkkiygCyPJAvI8pTTkj/4LVvQhxcOfvMqzuhpKtngN0nHBmR5TFpqvrgnEonVN6qSSLKALI8kC8jylNJS7OC3XWVY+U3SsQFZHpOWmi/ubg6lvZHkkWQBWZ6TsZzs4LdyRNKxAVkeN8/dYNwcSnsjySPJArI8xVgKGfy2JTv4LXOavafF7GhvSccGZHncPHeDkXItB2RZQJZHkgVkeZazFDP4bVdP+hT7rt7Kr/wm6diALI9Ji7HvQqXU64HPA17gS1rrzyx4fQtwG9CW2eZDWuu7F7z+JLBfa/1XpWqX11udqzYtFUkWkOWRZAFZnqwlpTV9Y9OZVd+WH/y2Y11T7hT7GeuarFv5TdKxAVkekxYjxV0p5QW+CLwWOAo8rJS6S2v9ZN5mfwrcrrX+O6XUTuBuYFve67cC3y912yYmJmhvby/121YkkiwgyyPJAjI8WmteGI3x48MD9E+P8uTgJJGZlQe/ndoZwOf1VKjFhUXCscmPJI9Ji6me+0XAc1rrFwCUUgeAa0n3xLPRQPYu9kHgePYFpdRvAC8Ck6VuWHd3d6nfsmKRZAFZHkkWqF5PLJHk0PEID748wcP9E3k98xhgfvBbOVKtx2a5SPKYtJgq7huB/rzHR4GLF2yzH7hHKfV+oAm4EkAp1Qz8v6R7/R8sdcNGR0dpbGws9dtWJJIsIMsjyQLV5TkWnuGh/jAP9U/wixNREnkj4Doa69jZXsdlp/ZUZPBbOVJNx6aQSPKYtNg0oO4G4Cta679WSl0KfE0ptYt00f+c1jqqVvgNenBwkJtuuom6ujqSySR79+5l3759hEIhmpqa8Hq9TExM0N3dzejoKFpruru7GRsbyy3mH41G6enpYWhoCKUUHR0dDA0N0draSjKZZHJykt7eXkKhED6fj2AwyPDwMMFgkHg8TiwWy73u9/tpaWlhZGSE9vZ2YrEY09PTudcbGhoIBAKMjY3R2dlJJBIhHo/nXg8EAvj9fsLhMF1dXYTDYRKJRO71pUzj4+N0dXUxMDBAc3Nz1ZvC4XDuOFW7aWxsjI6Ojnnfe9Vsmpqaoq+vb9H/JxtMrW0dPPryCAdPTPHUuObYxNw9tBVwemcDuzq97Gz3ctFpG3n22WdZ357COztJX98JK03FHKdUKkVfX9+KP/eqyTQ2NkZ3d/eqP8urwTQ7O8vx48dX/VleqGmlKK31ihuUIplivV9rfVXm8YcBtNafztvmCPB6rXV/5vELwCXAvwCbM5u1ASngFq31F/I/44EHHtA7duwoum3T09M0NDQUvZ+NkWQBWR5JFrDPMzKV4KH+CR56OczB4xFiiVTutWa/l/M3tXDx5iAXbGqhLeCbt69tlpON89ibUlsOHjz46J49ey5Y6jVTPfeHgdOUUtuBY8D1wNsWbPMysAf4ilLqDKABGNJavzq7gVJqPxBdWNhPJgMDA2LmUEqygCyPJAtU3pNMaZ4ZnuLBl9On258bic17fVt7AxdvbuWiLUF2rmvCu8IKcJW2lDrOY29MWowUd631rFLqfcAPSE9z+7LW+ohS6uPAI1rru4CbgX9SSn2A9OC6d2kDpxWypzwkRJIFZHkkWaAynsjMLI8cjfBQf5hHjkYIT8/mXqv3Ks7Z0MLFW4JctLl1xbumLYw7NnZHksekxdg198yc9bsXPHdL3tdPApev8h77y9I4FxcX66K15qWxaR7qn+DB/jBPDkzOWw2ut8XPxZtbuXBzK2evb6G+zu4pai4uJmPTgLqKJBqN0tnZWelmlCSSLCDLI8kCpfNorRmNzRKKzHBiIk4oGmcg8/XRiWlGp+Z6514FZ69v5qLNrVy8OcjmtnpWGmRbaNyxsTuSPCYtNV/ce3p6Kt2EkkWSBWR5JFmgOM9kPEkoMkMoEudEJFO8I3FCma9nkstffWtrqOOiza1ctKWV8ze20uQv/QpftXxsqiGSPCYtNV/ch4aG2Lx58+obVkEkWUCWR5IF5nsSyRSD0US6950p2NlCHorMMLFg1beFaa330ttSz/oWP70tfnpb6+lt9qefa/WXfREZycdGQiR5TFpqvriX4rSeLZFkAVkeKZZ4MsU9z4xy8KUoE48/Syg6w/BkYtGd0fLj9yp6W+rpbfGzvsVPT34hb6kvS2+8mOQfG51KMf7zn3P0q18l8sQTpOJxPH4/Lbt3s+m3f5u2Sy5Beey+ti/ley0bSR6Tlpov7qstBFBNkWQBWZ5qt2it+e8Xx/nyw8c5EYnPe02RXrZ1faaAZ3ve61vTxbs9UGf1Eq7ZYxPr6+Pwvn0kJiZITU3lXk/NzBB+5BEiTz6Jr7WVXV/8IgGLp2ZV+/fawkjymLTUfHEfGhoSM4dSkgXK79Fa8+xIDJ9HsbG1Hn8ZR1tX87E5Eoryjw8d46nBdMHb0tbAq3o97Nq2nvUt9axr9ll/M5WVMjQ0xDrgsRtvJDk5CcvMwE1NTTETi/HYjTdyzm23WVvgq/l7balI8pi01Hxxb21tXX2jKokkC5TXMxlP8tn/6uNnfWEg3ftc1+xnU7CeTcF6NgYbMn/Xs67Jv+IiKIWkGo/NsfA0//zwce57Kf1v1B6o453nrecNr+xkIjxOe3v1mZZKS3Mzh1cp7LloTXJyksPvex8X3Hmnlafoq/F7baVI8pi01HxxTyZXHuxTTZFkgfJ5nh+Z4hM/fonjEzM0+jy0B3yciMwwEI0zEI3z6LHIvO19HsWG1volC39bQ11B19Gq6diEp2f5+sEQ//bUEEmdXiDmLWf18Fu719GYuT5eTZ7VMvHwwyQmJlYv7NloTSIcZvzBB2m/9NLyNm4NkXRsQJbHpKXmi/vk5CRdXV2VbkZJIskC5fHc88wIf3N/P/Gk5pTOAH+2ZzsbWuuZTWlCkRmOhtN/joWnc1+PTCXoG5+mb3x60fs1+71szC/6rfW5wh/wzQ0Uq4ZjMzOb4ntHhvjmYyGmEikUcNXpHdx4/nq6muav+FYNnkIzdODAvGvshSQ1NcXR226zsrhLOjYgy2PSYuTGMSZyzz//s575X/9ryddO/chH6N27F4DQd77Dc8tsB/CqRx7JfX3oHe9g8pe/XHK7nt/8TU776EcBiD71FI+9853Lvuc5X/sazWecAcCzn/oUA9/97pLbNe3Ywblf/3ru8X0XLHk/AMCZijXFZ1P8f7f/F2fe+icnbYpufAX//jufoH98mqlEilv+/j3Lvudjb/xdknuuZkNzHWce/imj//uzJTNB6Y5Tavup/MPeP2VoMn1/85VMp37kI7Rfcw319fVivvfWEk99PYHt2601uZ8R8k36j/+44jeOcXGpWE5MzPCJH7/IZF+YM0vwfj0tfv7Pta9Ea8349CxH/n75bY+GZzj4yxEAznsyxK+X4PPLkVAkztBkgld0NPC7F21kegUTQCgUEjPIaa1JxeOrb+TiUqGI6bmv9Zavx48fZ8OGDWVokflIskBpPD9/Ocxf/qSPaDzJ+hY/f7ZnO6d2NZaohcsnmdIMRuOZU/vTPPbyCE8MJ4jG5665rWv2cdnWNi7fGmRXb/NJD9orJn1jMb700HEe7J8AoLPRx7suWM+Vp3YU1A5J32v3X3YZeg2F2lNfz2X331+GFp1cJB0bkOUptcWGW75am2AwWOkmlCySLHBynmRK89VHT/DNxwcAuHRLkD+5YgvN9Wa+5b0exfrWeta31nPh5lZet72J+kAjjx+PcH9fmJ/1jTMYTfC9I0N878gQrfVeLt0a5LKtbZy/saVs0/JGpxJ87eAJvv/0CCkNAZ+H687qYe+u7nljBFaLpO+1pjPPJHroUNH7tezeXYbWnHwkHRuQ5TFpqfniPjw8TFNTU6WbUZJIssDaPWNTCT79k5d47HgUj4J3X7CB3zprXUUXUhkeHmbr1q2cv6mV8ze18r7LNvH00BT3vzTOfS+FOT4xww+eGeUHz4zSUOfhos2tXL4tyEWbgyVZwS2WSPIvh4e4/fEBpmdTeBT8+o4u3nleL+2NvjV5pHyvNVxzDVNPP13UoDpPYyObbryxjK1aeyQdG5DlMWmp+eLufiu0N2vxHAlF+eS9LzEylaA9UMdHXrONsze0lL5xRWahxaMUZ6xr4ox1Tdx04QZeGptO9+hfGue5kRj//eI4//3iOHUexTkbmrl8WxuXbQkWXYiTKc0Pnx3ltkdPMDKVHix3yZZWfvfCjWxpbyiZp5rT8yu/QuRLX2ImFitsOpxS+IJB2i6+uPyNW0MkHRuQ5XE9d4OJCxoUI8kCxXm01nzn8BBfeugYSQ27epr46K9tp7Op+F5pObKSRSnF9o4A2zsCvOPcXkKRGX7WF+b+l8IcDkV55GiER45G+Bv62dnTxOXb2rh8W5D1LfUrfuYjRyf4pweP8eJYegrfaV0Bfu+ijSX5ZUfS91pidpZdX/ziqivUAaAU3qYmdn3hC1YuYAOyjg3I8pi01Hxxj8VilW5CySLJAoV7JuNJbv3py/z0xXEA3rJ7Hb9z4QbqDA5QWy3FHJvelnr27lrH3l3rGIsl+HlfmPv7whw6FuHIwCRHBib5xweP8YqOAJdvC3L51ja2dzTkFtN5YSTGPz10LLcYz7pmH+++YAOvOaW9ZJcmJH2vxWIx1m3dyjm33bbk2vLZeBob8QWD7PrCF6xdehZkHRuQ5TFpqfnR8jMzM9TXr9wDqpZIskBhnhdHY3zixy9yNJxebe6Dv7KVV21vM9TCwlOKYzMZT/Jw/wT3943zUP8EsUQq99r6Fj+Xb2sjMjPLPc+MooFGn4e3ndPLb5zZXfIBepK+1/ItOpVi/MEHOXrbbYvvCnfjjbRdfLG1PfZsJB0bkOUptcWNll8hkubrSrLA6p4fPTvK5+/vZ2Y2xfb2Bv7syu1sCq79OnI5U4pj0+T38quntPOrp7QTT6Z47HiE+14M88DLYU5E4tzxxCAAXgVv3NnN28/tJdhQnv/ikr7X8i3K46H90kutXHmu0Eg6NiDLY9JS88Xd7/evvlGVRJIFlvfEkyn+/ufH+LenhgG48rQO/p/LN9NQxru6nWxKfWz8Xg8XbU6Ppk+mNEcGJvlZ3ziJpGbvrm42lvmXHEnfa5Is4Dw2x6Sl5ot7S0vlR1KXKpIssLRnIBLnk/e+yNNDU/g8iv9x2SaufmVnQTdvqWTKeWy8HsVZ65s5a31z2T5jYSR9r0mygPPYHJMWe7s6hjIyMlLpJpQskiyw2PNQf5j/8b1f8vTQFD3Nfj73xtO5ZkeX9YUd5B+bao4kCziPzTFpqfmee3t7e6WbULJIssCcJ5nSfP1QiG8cCqGBiza38j+v2Eprma4nlyNSj42ESLKA89gck5aa77m7aRb2JhaLEZ6e5aM/eJ7/eygEwLvOX8/HX/eKqirsIPPYSIkkCziPzTFpqa6fkGXI9PTie3RXa0xYUloTnUni8yrqPOk/5TotfiQU5Uv3DjE8mSDYUMeHX7OV8za2luWzyh1J32cgyyPJAs5jc0xaar649/b2VroJJUs5LcmU5t7nR/nawRChyPxVlnweNVfsvQqfx5N77FvwOP3cwtfTz829rpiMJ/nO4QlmU7BzXRMf3bON7qbqHTUr6fsMZHkkWcB5bI5JS80XdzeHcuVorflZX5ivPHqCvswypvWZKWezyRRJDYmUJpEqz2JIv3FmN++5aAM+b3VfQZL0fQayPJIs4Dw2x81zN5iGBjsXPVlLSm157HiELz98nF8OpZfi7Gn289vn9/Jrp8zd8zuZ0iQzxT2RTDGb0iSS6cezSU0ilcr8nX5+NpV+Lvd1cm6/+fum2NaY5KqzNpXUVKlI+j4DWR5JFnAem2PSUvPFPRAIVLoJJUupLM8MTfHlR45zMLM2eVtDHW87t5erd3TiX9CD9noUXo8ifcL85G9Nmp+JiYmSvl8lI+n7DGR5JFnAeWyOSUvNF/exsTFaW6tzkNbCnKzl5fFpvvLICe57KX0Dlkafh+vO6uE3d3UT8JW2cBcSd2zsjSSPJAs4j80xaan54t7Z2VnpJpQsa7UMRuN87eAJfvjsKCkNfq/i2p3dvPXsnopOOXPHxt5I8kiygPPYHJOWmi/ukUiE5mZzy3aWM8VaxmMJvvn4AP/25DCJlMaj4Jodnbz93F66LBiZXsvHxvZI8kiygPPYHJOWmi/u8Xh89Y2qJIVaJuNJ/uWJQf7l8GDutqGvOaWd3z5vPRuD9txasRaPTbVEkkeSBZzH5pi01Hxxr6U5lPHZFHc9NcyBx0JMzCSB9FKu775gPad0NppoYlGppWNTbZHkkWQB57E5Ji3VPXm4BAmFQpVuQsmynCWZ0nz/l8O869tP8o8PHmNiJsmunib++tdP45NXnWJlYYfaODbVGkkeSRZwHptj0lLzPXfJ0yxSWnPfi+N85dETHA3PAPCKjgC/c+F6LtzUav3d1CQfm2qPJI8kCziPzXFT4QzG76/8wLFSJWvRWvPosfQCNM+NpG9UsKHVz43nr+eKV7TjsbyoZyPx2EiJJI8kCziPzTFpqfniHg6HaWtrq3QzSpJwOMyJuI8vP3ycx09EAehs9PH2c3t5/Ss7qfNUR1HPRtqxkWIBWR5JFnAem2PSUvPFvaurq9JNKEmGJ+P845MzPHzsGQBa6r289ewert3ZnVsLvtoi5diALAvI8kiygPPYHJOW6vypX8KEw+FKN6Ek+dsHjvLwsUnq6zzccE4Pt123k+vO6qnawg5yjg3IsoAsjyQLOI/NMWmp+Z57IpGodBNOOrMpnVsH/u9/85VsDMq40YKEY5ONJAvI8kiygPPYHJOW6u3WlSgS5lD+cnCSqUSKzUG/mMIOMo5NNpIsIMsjyQLOY3PcPHeDkTCHMttrP621ugbMrRYJxyYbSRaQ5ZFkAeexOSYtxoq7Uur1SqmnlVLPKaU+tMTrW5RS/6mUOqSU+oVS6urM869VSj2qlHoi8/evlbJdTU1NpXy7iiRb3M/d0FLhlpQ2Eo5NNpIsIMsjyQLOY3NMWoxcc1dKeYEvAq8FjgIPK6Xu0lo/mbfZnwK3a63/Tim1E7gb2AYMA2/UWh9XSu0CfgBsLFXbvF7ztzItZSbjSX45NIlHwe4eO1eaW2uq/djkR5IFZHkkWcB5bI5Ji6me+0XAc1rrF7TWceAAcO2CbTSQvdFtEDgOoLU+pLU+nnn+CBBQSpXs7iYTExOlequK5BcnoqQ0nLGuiUQsWunmlDTVfmzyI8kCsjySLOA8NsekxdRo+Y1Af97jo8DFC7bZD9yjlHo/0ARcucT7vBk4qLWeKVXDuru7S/VWFcnBY+lvlvM2ttDdHaxwa0qbaj82+ZFkAVkeSRZwHptj0mLTVLgbgK9orf9aKXUp8DWl1C6tdQpAKXUm8BfA65baeXBwkJtuuom6ujqSySR79+5l3759hEIhmpqa8Hq9TExM0N3dzejoKFpruru7ef7559mwYQMA0WiUnp4ehoaGUErR0dHB0NAQra2tJJNJJicn6e3tJRQK4fP5CAaDDA8PEwwGicfjxGKx3Ot+v5+WlhZGRkZob28nFosxPT2de72hoYFAIMDY2BidnZ1EIhHi8Xju9UAggN/vJxwO09XVRTgcJpFI5F7Pmh58aRSAzb5pXnhhhFNPPZWBgYHcPYOr0ZQ9Ti+++CKtra10d3dXveno0aOccsop8773qtk0NjaG3+9f9P+pGk3PP/8869evX/ZnRLWZtNYopVb8uVdNppGREU4//fRVf5ZXgymZTNLQ0LDiz71iTCtFaa2LrsLFJlOs92utr8o8/jCA1vrTedscAV6vte7PPH4BuERrPaiU2gTcC7xba33/Up/xwAMP6B07dhTdtv7+fjZv3lz0fjZkMBrnHQeO0OjzcMc7z+LEsaNVa1kq1XxsFkaSBWR5JFnAeWxOqS0HDx58dM+ePRcs9Zqpa+4PA6cppbYrpfzA9cBdC7Z5GdgDoJQ6A2gAhpRSbcC/Ax9arrCfTKr5lM+h4+lR8mevb6HOo6raslQkeSRZQJZHkgWcx+aYtBgp7lrrWeB9pEe6P0V6VPwRpdTHlVJvymx2M/AepdTjwDeBd+n0aYX3AacCtyilHsv8WVeqtg0MDJTqrYwnOwXuvI3pKXDVbFkqkjySLCDLI8kCzmNzTFqMXXPXWt9Nenpb/nO35H39JHD5Evt9EvhkudqVvZ5RbUnpuSVnz80U92q1LBdJHkkWkOWRZAHnsTkmLTW/Ql215sXRGOHpWbqafGwOlmxmoIuLi4uLgNR8cY9Gq3NueLbXfv7GFpRKLztbrZblIskjyQKyPJIs4Dw2x6Sl5ot7T09PpZuwpiy15Gy1WpaLJI8kC8jySLKA89gck5aaL+5DQ0OVbkLRic+mOBxK/waYvd4O1WlZKZI8kiwgyyPJAs5jc0xaar64Z09pV1OODE4yk9S8oiNAe8CXe74aLStFkkeSBWR5JFnAeWyOSUvNF/fVVvmxMQunwGVTjZPDc38AACAASURBVJaVIskjyQKyPJIs4Dw2x6Sl5ot7NZ7yObRMca9Gy0qR5JFkAVkeSRZwHpvjTssbTGtr6+obWZSJ6VmeHZ7C51Hs6p0/Z7LaLKtFkkeSBWR5JFnAeWyOSUvNF/dkMlnpJhSVx45H0MDOniYa6uYfvmqzrBZJHkkWkOWRZAHnsTkmLTVf3CcnJyvdhKJy8PjSp+Sh+iyrRZJHkgVkeSRZwHlsjklLzRf33t7ein12MqWZTRV3V765xWsWn96ppKUckeSRZAFZHkkWcB6bY9JS88U9FApV5HO11rz/zqfZ991fkiywwB+fmCEUidNS7+WUzsCi1ytlKVckeSRZQJZHkgWcx+aYtNR8cff5fKtvVIZEZpI8NxLjxbFpnhmeKmifbK/9nA0teD2L50tWylKuSPJIsoAsjyQLOI/NMWmp+eIeDAYr8rkD0Xju60czRXu1LDe/PZtKWcoVSR5JFpDlkWQB57E5Ji01X9yHh4cr8rmDecX94LGJVbdPpjSPn8gU9w1LF/dKWcoVSR5JFpDlkWQB57E5Ji01X9wr9VthfnF/amCSqfjKUySeG5kiMpNkfYuf9a1L3+JV0m+4IMsjyQKyPJIs4Dw2x/XcDSYej6++URmSX9yTGh4/sfKtAFc7JQ+Vs5QrkjySLCDLI8kCzmNzTFpqvrjHYrGKfO7gZAKATcF0L3y1U/O5W7yuUNwrZSlXJHkkWUCWR5IFnMfmmLTUfHGv1BzKbM/9Da/sBFYeVBdLJHlyYBIFnLN++eIuaT4oyPJIsoAsjyQLOI/NcfPcDaZScyiHMsX91dvbaPR5OBqeYSCy9Cmbw6FJEinN6d2NtDbULfuekuaDgiyPJAvI8kiygPPYHDfP3WD8fr/xz4zPphiNzeJV0N3k55zM6PflTs1nnz93mVHy2VTCUs5I8kiygCyPJAs4j80xaan54t7SsnLBLEeGMtfbu5r8eD0qN0ju4DKn5g+tsJ58fiphKWckeSRZQJZHkgWcx+aYtNR8cR8ZGTH+mYOT6dPv65rTv8Vl14k/eDyyaCnasakEL4xOU+9V7OxpWvF9K2EpZyR5JFlAlkeSBZzH5pi01Hxxb29vN/6Z2cF065rTSxFuaPXT2+InMpPk+ZH5oymzd4Hbvb4Zv3flw1UJSzkjySPJArI8kizgPDbHpKXmi3slplnkintTuueu1Nyp+UcXXHfPzW9f5Xo7yJoyArI8kiwgyyPJAs5jc9xUOIOZnp42/pm54t4yN7gid2o+77q71ppDucVrFt/idWEqYSlnJHkkWUCWR5IFnMfmmLTUfHGvxBzKhT13gHM2NONRcGRgklgivRRt//gMw1MJ2hrq2NbRsOr7SpoPCrI8kiwgyyPJAs5jc9w8d4OpxBzKwWh6tHz2mjtAS30dp3c1MpvS/CKzFG32FP25G1vwqMW3eF0YSfNBQZZHkgVkeSRZwHlsjpvnbjANDav3iEsZrfWi0fLZnL9p/qn57BS481eZApeNaUu5I8kjyQKyPJIs4Dw2x6Sl5ot7IBAw+nnj07MkkpqWei8Bn3fea3OD6iLzevArrSefH9OWckeSR5IFZHkkWcB5bI5JS80X97GxMaOfNzcNbvFKRWesayLg8/Dy+DT3vTjOVCLF5mA93U2FrWpk2lLuSPJIsoAsjyQLOI/NMWmp+eLe2dlp9PNy19uXKNh1HsXZ65sB+Mqjx4HCRslnY9pS7kjySLKALI8kCziPzTFpqfniHoksfze2cmSlnjvMTYk7PpHebrUlZ/Nj2lLuSPJIsoAsjyQLOI/NMWmp+eIejy99J7ZyZeHqdAtz/qa5Yu5RcFamJ19ITFvKHUkeSRaQ5ZFkAeexOSYtNV/cTc+hXK3nvrG1Plf4z1jXRJPfu+R2S0XSfFCQ5ZFkAVkeSRZwHpvj5rkbjOk5lMtNg8tGKcUFmSlxhU6By0bSfFCQ5ZFkAVkeSRZwHptj0lJn7JMsjelpFnML2Cw/Av5d569nQ0s9b9zZVdR7S5oyArI8kiwgyyPJAs5jc0xaar64+/2FTTMrRaZnU4SnZ6nzKNoDy//TtwV8XHd2T9Hvb9JiIpI8kiwgyyPJAs5jc0xaav60fDgcNvZZQ5nr7d1NvoKWky02Ji0mIskjyQKyPJIs4Dw2x6SloOKulDq73A2pVLq6ijv1fTIZWGUw3cnGpMVEJHkkWUCWR5IFnMfmmLQU2nP/kVLqcaXUB5VS68vaIsM52d+kkinNx3/0AnceGVp126EyF3dJv+GCLI8kC8jySLKA89gc63ruwHrgFuBi4Fml1D1KqXcopRrL1zQzSSQSJ7V/f3ia+14Kc8cTg6tuOzi5+mC6k8nJWmyLJI8kC8jySLKA89gck5aCirvWelZrfafW+reAjcDtwP8EBpRSX1VKXV7ORpYzJzvvMJnSQPqGMKtltTnuJxtJ80FBlkeSBWR5JFnAeWyOtfPclVLNwG8A1wObgAPAs8D/VUp9cZV9X6+Uelop9ZxS6kNLvL5FKfWfSqlDSqlfKKWuznvtw5n9nlZKXVVMm1fLyc47TKbSf8/MpoglkitumyvuTUuvTneykTQfFGR5JFlAlkeSBZzH5lh3P3el1DVKqQPAMeCtwJeADVrr92itPwGcB9y4wv5e4IvAG4CdwA1KqZ0LNvtT4Hat9bmkf3n428y+OzOPzwReD/xt5v1KkqampoK3DU/P8qHvP8fP+sZzzyW1zn09Flu5917unnsxlmqIJI8kC8jySLKA89gck5ZCe+6fAR4Fdmitr9ZaH9BaT2df1FqPAn+0wv4XAc9prV/QWsdJ9/ivXbCNBrK3QAsCxzNfXwsc0FrPaK1fBJ7LvF9J4vUW/nvCgy+HOXgswj3PjOaeS6Xyi/vy11NSWjOUuebeXabiXoylGiLJI8kCsjySLOA8NsekpdBr7ru11p/VWp9YYZsvrfAWG4H+vMdHM8/lZz/wDqXUUeBu4P1F7LvmTExMFLxtKJLueSfzCnqhPfexqVlmU5pgQx0NdeVZXqAYSzVEkkeSBWR5JFnAeWyOSUtBK9Qppb4DfE5r/dO8514N/KHW+i0lassNwFe01n+tlLoU+JpSalehOw8ODnLTTTdRV1dHMplk79697Nu3j1AoRFNTE16vl4mJCbq7uxkdHUVrTXd3N4lEgpGREQCi0Sg9PT0MDQ2hlKKjo4OhoSFaW1tJJpM8F0r32KNTMY4fP04wGOT4iYFcG54/OsCF6wOEQiH8fj8tLS2MjIzQ3t7OL4+np0B0N9bR19dHQ0MDgUCAsbExOjs7iUQixONxent7CYVCBAIB/H4/4XCYrq4uwuEwiUQi9/pSpmQyyfT0NAMDAzQ3NxdkmpyczL2nz+cjGAwyPDxMMBgkHo8Ti8Vyry80xWIxpqenc6+X2pRKpejv76e7u7vqTYlEgqmpqXnfe9Vsqq+vp6+vb9H/p2o0JRIJhoeHl/0ZUW2mYDBIX1/fij/3qs00MzOz6s/yajA1Nzdz/PjxVX+WF2paKUrn9TyX3UipEWCd1jqZ91wdMKC1XvXu85livV9rfVXm8YcBtNafztvmCPB6rXV/5vELwCXATfnbKqV+kHmvB/I/44EHHtA7duxY1bIwR48eZdOmTQVt+0d3PcOTg5Ocu6GZv7j6NAAeOTrBR/7jeQDeeV4v7zxv6WUA/uuFMT5170tcvjXIn7/2FUW3s5AUY6mGSPJIsoAsjyQLOI/NKbXl4MGDj+7Zs+eCpV4r9PzwNLBwJEAzUOikvYeB05RS25VSftID5O5asM3LwB4ApdQZQAMwlNnueqVUvVJqO3Aa8FCBn7tqCvnlJptQdAaYGyGf/rqw0/K51elayre2cDGWaogkjyQLyPJIsoDz2ByTlkKL+w+Af1BKtQJk/v4C8B+F7Ky1ngXel3mfp0iPij+ilPq4UupNmc1uBt6jlHoc+CbwLp3OEdLz6p/MfN6+/DMIJ5vu7u6CtpueTTE6lS7e+dfZ878eX2FAXW51uqbyFfdCLdUSSR5JFpDlkWQB57E5Ji2FFvebSY9kH1VKDQKjpEe0rzRCfl601ndrrU/XWp+itf5U5rlbtNZ3Zb5+Umt9udb6bK31OVrre/L2/VRmv1dqrb9f6GcWkoGBgdU3AgYiM7mv83vrqbxe/Eo990Ju9XqyKdRSLZHkkWQBWR5JFnAem2PSUtCAOq31GHBNZl35TUC/1lrEygLZwQqr5URmpDxAXm1f0HNfobhPZue4l2cBGyjcUi2R5JFkAVkeSRZwHptj0lLU/dy11ieUUiFAKaU8medSq+wmIicm8nru+aflC5znXu4FbFxcXFxcXLIpdIW6DUqp72ZGzc+SHkiX/VPViUajBW0Xis713Jeb5z6VSBGfXfy7zlQ8SWQmic+raGso6vepolKopVoiySPJArI8kizgPDbHpKXQa+7/AMRJj2aPkl5u9i7gD8rULmPp6ekpaLvQxDLFfUEtX+q6+9Dk3GA6pdQaWllYCrVUSyR5JFlAlkeSBZzH5pi0FFrcLwN+R2v9GKC11o+Tnn9+c9laZihDQ6vfhx3gRP6AumWuucPSp+YHDJ2SL9RSLZHkkWQBWR5JFnAem2PSUmhxT5I+HQ8wrpTqBiYp4TKwlUohPWmt9bwBdfN77guL++Ke+9xI+fINpoPCLNUUSR5JFpDlkWQB57E5Ji2FFvcHgewtWH8AfAv4DvBIORplMqst4Qfpe7XP5F1Lz++tL6jtS851HzLUcy/EUk2R5JFkAVkeSRZwHptj0lJocX8n8F+Zr/8IuBc4DLytHI0ymUJOk2RvGNPRmB4Mlyqy5+5Oy68tkjySLCDLI8kCzmNzTFpWHbqduXf654HfA9Bax4BPlrldxtLa2rrqNtlpcJtaGxidii55zd2r0tfilzwtP2mmuBdiqaZI8kiygCyPJAs4j80xaVm1555Z6vV1gMj57Mnk6ivZZq+3bwzWp/dZoufe0Zi+nj4+vdRp+cw19zIuPQuFWaopkjySLCDLI8kCzmNzTFoKPS3/OeBjSqnyjgirQCYnJ1fdJpQZKZ8r7vPWlk//3dWUKe4Leu7JlM5NhetuKu8/XyGWaookjyQLyPJIsoDz2ByTlkJXVHk/0Av8sVJqCMhVN631lnI0zFR6e3tX3SZ7zX3TEj337PX3zkY/MLXotPzIVIKUho5AHf66Qn+XWlsKsVRTJHkkWUCWR5IFnMfmmLQUWm3eAVwJXJX5+p15f6o6odDqS+Rn57hvam0Alj4tn+25L5znnh0p321g2dlCLNUUSR5JFpDlkWQB57E5Ji2F3jjmv1bfqjrj8618qjyRTDEUTeBR0NuaLtBJnZ77rpQilTlF39ZQh0dBZCbJbEpT50nPZzQ1mA5Wt1RbJHkkWUCWR5IFnMfmmLQUVNyVUh9f7jWt9S2la475BIPBFV8fjMbRpAfD+b0eFOlrEik9N0IewOdVBBvqGIvNMh5L0JUZPDc0me7Jd5X5ejusbqm2SPJIsoAsjyQLOI/NMWkp9LT85gV/LgQ+CJxSpnYZy/Dw8IqvZ0fK97aki7U30yPPDqrLnpb3ehTtgfTvSvnX3Sem01+X84Yx2axmqbZI8kiygCyPJAs4j80xaSn0tPy7Fz6nlHo9cEPJW2Q4q/0mlR1Mt74lPZjOq9Lr8GYvu8/Nc1e0BXzA9Lzr7uFMcQ8aKO6SfsMFWR5JFpDlkWQB57E5Nvbcl8o9wG+UqiGVSjweX/H17AI2i3ruqcU992zvPH86XLa4txoo7qtZqi2SPJIsIMsjyQLOY3NMWgq95v6KBU81kl56tr/kLTKcWCy24uvZ0/LrW5cr7untvIrcafnxeafl04sWmDgtv5ql2iLJI8kCsjySLOA8NsekpdCK8xzpcWTZW9pMAYeAG8vRKJNZbd5hdgGb3sxpeY9acM0987fHo2gPLJ4OZ7LnLmk+KMjySLKALI8kCziPzbFunrvW2qO19mb+9mitm7XWr9ZaP1ruBpY7q807nLvmnum5Z369SWV67Kl519yXGFA3Y+6au6T5oCDLI8kCsjySLOA8NsekpaDirpQ6Rym1ecFzm5VSZ5enWebi9y8//zwyM0s0nqShzpMrzp5lR8uT13NPF/TZlCYyk8SjoNnvLZshm5Us1RhJHkkWkOWRZAHnsTkmLYUOqPs6sHCith/4WmmbYz4tLS3LvnYir9euMqfjvWrBNffMqHmvUnnX3NOn5SOZU/It9XW5a/XlzEqWaowkjyQLyPJIsoDz2ByTlkKL+xat9Qv5T2itnwe2lbxFhjMyMrLsa6HsSPnW+txzy81zn3/NPV3Uw5lT8q315e+1w8qWaowkjyQLyPJIsoDz2ByTlkKL+1Gl1Hn5T2QeHy99k8ymvb192dcWXm+HuWvui6bCKUUw03OfmJklmdKEM0U++3y5s5KlGiPJI8kCsjySLOA8NsekpZhbvt6plHq/UupqpdT7ge8Ct5avaWay0tSE7A1jsgvYQP5UuPTj3CI2HqjzKFrrvaR0emW6bM89WG+muEuaMgKyPJIsIMsjyQLOY3Osmwqntf4npdQ4cBPp5Wf7gZu11neUs3EmMj09vexrC5eehaVOy6efz06Rawv4mJhJMhabzc1xNzENDla2VGMkeSRZQJZHkgWcx+aYtBRcdbTW3wa+Xca2VCQrzTsMLdVzXzCgLjcVLlP02wN1vDwO49OJ3Bx3EwvYgKz5oCDLI8kCsjySLOA8Nse6ee5Kqb9RSl224LnLlFL/uzzNMpfl5h0mU5qBTM+9Z17PPfP6gkVsvLme+9xcd5ML2ICs+aAgyyPJArI8kizgPDbHunnupG8Q88iC5x4lvQRtVaehoWHJ54cnEyQ1dDb6qK+b+2ea67mnH2cXs8kW/fwR8yZvGgPLW6o1kjySLCDLI8kCzmNzTFoKLe56iW29RexvbQKBwJLPn4jMv2FMNouuuS/ouefPdZ/I9dzNTIVbzlKtkeSRZAFZHkkWcB6bY9JSaHH+KfBJpZQHIPP3xzLPV3XGxsaWfP7EEtPgYG7gXGqJu8IBmdu+zu+5tzUsXP+nPFnOUq2R5JFkAVkeSRZwHptj0lLo+eI/BP4NOKGU6gO2kp7j/sZyNcxUOjs7l3w+t4BN3mA6WOKae3YRm8z89/bcNfdE3jV3Mz335SzVGkkeSRaQ5ZFkAeexOSYthd445ihwHnAt8Fngt4D/BB4qX9PMJBKJLPl8KDr/Vq/ZLLrmnl1+1rPwtPxs7rS8qWvuy1mqNZI8kiwgyyPJAs5jc0xaiqk6ncDFwLuAs0ifkv/DMrTJaOLx+JLPn1i2577yNffsKfhQJM5MUuP3KhrqzAxNWM5SrZHkkWQBWR5JFnAem2PSsmJxV0r5gDeRLuhXkb6v+zeBLcB1WuvBcjew3Flu3uFy19yzxX25a+7Znns0PreATfamM+WOpPmgIMsjyQKyPJIs4Dw2x6Z57gPAPwBPA5dorXdqrT8BiPlVaql5h1PxJOHpWXxeRUfj/MFwubXll+m5++s8NPrm/llNnZIHWfNBQZZHkgVkeSRZwHlsjk3z3H8BtJE+HX+hUkrOCv6ZLDU1IXvDmN5mf250fDaL1pZfMM8d5ua6g9niLmnKCMjySLKALI8kCziPzbFmKpzW+leBU4B7gA8CIaXUvwJNLL6/e1XG7/cvei4UXfp6O8xNhVs4Wt6b90tAe95d4EwW96Us1RxJHkkWkOWRZAHnsTkmLauO9NJa92mtP6G1Pg3YA5wAUsDjSqm/LHcDy51wOLzouRMTS4+Uh8Vry2eLvMczV9zb8nrurYbuCAdLW6o5kjySLCDLI8kCzmNzTFqKGsattb5Pa/17QC/wfmB3WVplMF1dXYueC0WW77nn5rnnbhyTeT7v7P28nruhe7nD0pZqjiSPJAvI8kiygPPYHJOWNc3R0lpPa62/qbV+Q6kbZDpL9tyXGSkP+VPh0o8XjpaHBcW93swCNiDrN1yQ5ZFkAVkeSRZwHptjbc9dYhKJxKLnpjLT2BauKw/Ln5bPv+beVqEBdUtZqjmSPJIsIMsjyQLOY3NMWoxVHqXU64HPk77hzJe01p9Z8PrngNdkHjYC67TWbZnX/hK4hvQvIz8E/lDrTFU9ySw17/DWN55OLJHE7138u8+iRWyW6Lm3VWhAnaT5oCDLI8kCsjySLOA8Nsemee4liVLKC3wReAOwE7hBKbUzfxut9Qe01udorc8B/g/wncy+lwGXk14VbxdwIXBFqdq23LzDgM87r2Bnk5vnntJorXPX3D3LXHM3dS93kDUfFGR5JFlAlkeSBZzH5tg0z71UuQh4Tmv9gtY6DhwgvU79crmB9Ep4kL7dbAPgB+pJT8EbKFXDmpqaitrek3fNPb+wq3lT4SpzWr5Yi+2R5JFkAVkeSRZwHptj0mKquG8E+vMeH808tyhKqa3AduBeAK31A6RvUnMi8+cHWuunStUwr7e4AW/evFu+LjXHHSrXcy/WYnskeSRZQJZHkgWcx+aYtJirPIXneuAOrXUSQCl1KnAGsCnz+g+VUq/WWs+7l/zg4CA33XQTdXV1JJNJ9u7dy759+wiFQjQ1NeH1epmYmKC7u5vR0VG01nR3d9Pf308qlV5mLhqN0tPTw9DQEEopOjo6GBoaorW1lWQyyeTkJKTSByc8ESEy2Qyke+6Dg4PEYjF6e3sZDIW4+hVNNDb4Odb/Mu3t7cRiMaanp+nt7SUUCtHQ0EAgEGBsbIzOzk4ikQjxeDz3eiAQwO/3Ew6H6erqIhwOk0gkcq8vZRobGyMQCDAwMEBzc3PBpux7+nw+gsEgw8PDBINB4vF4zhQKhfD7/bS0tDAyMmLEdPToUaLRKN3d3VVv6u/vp76+ft73XjWbhoeHl/z/VI2m/v5+ksnksj8jqs2UTCaZmJhY8edeNZmGh4dpbGxc9Wd5NZiy773az/JCTStFlWhc2sofotSlwH6t9VWZxx8G0Fp/eoltDwH7tNY/yzz+E6Ahs6Y9SqlbgGmt9bwFdB544AG9Y8eOots2NTVFY2NjwdsfeDzElx8+wVvPWsf15/Tym1/9BY0+D9+78eyiP7vUKdZieyR5JFlAlkeSBZzH5pTacvDgwUf37NlzwVKvmTot/zBwmlJqu1LKT7p3ftfCjZRSO4B24IG8p18GrlBK1WXuUncFULLT8qOjo0Vtn5sKp5ceKV/JFGuxPZI8kiwgyyPJAs5jc0xajBR3rfUs8D7gB6QL8+1a6yNKqY8rpd6Ut+n1wIEF09zuAJ4HngAeBx7XWv9rCdtW1PZzN45Z/pp7pWLiLIzJSPJIsoAsjyQLOI/NMWkxds1da303cPeC525Z8Hj/Evslgd8vV7u6u7uL2t6bd+OY3AI2lvTci7XYHkkeSRaQ5ZFkAeexOSYtNb9C3cBAcbPq5vfc089ZUtuLttgeSR5JFpDlkWQB57E5Ji01X9yzIxELzdwiNljXcy/WYnskeSRZQJZHkgWcx+aYtNR8cS82+cvPppZYV97FxcXFxaXSqfniHo1Gi9reo5YYUGdJz71Yi+2R5JFkAVkeSRZwHptj0lLzxb2np6eo7XP3c9dz19y9dtT2oi22R5JHkgVkeSRZwHlsjklLzRf3oaGhorafu+Wrfdfci7XYHkkeSRaQ5ZFkAeexOSYtNV/cVZHXyz1519xtOy1frMX2SPJIsoAsjyQLOI/NMWmp+eK+2vq8CzPvxjGZnrsltb1oi+2R5JFkAVkeSRZwHptj0lLzxb3o0/JLXnO3o7pLOn0FsjySLCDLI8kCzmNz3Gl5g2ltbS1qe2/eaPmUZdfci7XYHkkeSRaQ5ZFkAeexOSYtNV/ck8lkUdvPrVA3d+MYjyU992IttkeSR5IFZHkkWcB5bI5JS80X98nJyaK2zxb31Ly15UverDWlWIvtkeSRZAFZHkkWcB6bY9JiSVmqXHp7e4vaft6NYyy75l6sxfZI8kiygCyPJAs4j80xaan54h4KhYraPjegzsJ57sVabI8kjyQLyPJIsoDz2ByTlpov7j6fr6jt83vuKcuuuRdrsT2SPJIsIMsjyQLOY3NMWmq+uAeDwaK2n3fLV8uuuRdrsT2SPJIsIMsjyQLOY3NMWiwpS5XL8PBwUdt75904Zv5zlU6xFtsjySPJArI8kizgPDbHpKXmi3vxPff030mNdfPcJf2GC7I8kiwgyyPJAs5jc1zP3WDi8XhR23vyT8un7Lqfe7EW2yPJI8kCsjySLOA8NsekpeaLeywWK2r7eVPhdOY5S/4Vi7XYHkkeSRaQ5ZFkAeexOSYtlpSlyqXoee6evBvHWHZXOEnzQUGWR5IFZHkkWcB5bI6b524wxc47zNbxpCbvrnB2FHdJ80FBlkeSBWR5JFnAeWyOm+duMH6/v6jt5904JnfNveTNWlOKtdgeSR5JFpDlkWQB57E5Ji01X9xbWlqK2j43z33eNXc7qnuxFtsjySPJArI8kizgPDbHpKXmi/vIyEhR23stHi1frMX2SPJIsoAsjyQLOI/NMWmp+eLe3t5e1PbZU/DzV6izo7gXa7E9kjySLCDLI8kCzmNzTFpqvrgXPRUud1o+737ulhR3SVNGQJZHkgVkeSRZwHlsjpsKZzDT09NFbe9Rimwpn7VsQF2xFtsjySPJArI8kizgPDbHpKXmi/ta5h1me+/xpF3X3CXNBwVZHkkWkOWRZAHnsTlunrvBrGXeYbannsjcOcaW0/KS5oOCLI8kC8jySLKA89gcN8/dYBoaGoreZ3HPvaRNWnPWYrE5kjySLCDLI8kCzmNzTFpqvrgHAoGi95kr7ql5jyudtVhsjiSPJAvI8kiygPPYHJOWmi/uY2NjRe+TXW4213O3pLivxWJzJHkkWUCWR5IFnMfmmLTUfHHv7Owsep/sXeCy19xtGVC3FovNkeSRZAFZHkkWcB6bY9JS88U9EokUvU+2mMdnszeO+2Ta1AAAIABJREFUKWmT1py1WGyOJI8kC8jySLKA89gck5aaL+7xeLzofWy95r4Wi82R5JFkAVkeSRZwHptj0lLzxX1N89wzPfeEZWvLS5oPCrI8kiwgyyPJAs5jc9w8d4NZ0zz3bM991q6eu6T5oCDLI8kCsjySLOA8NsfNczeYtUxNyNbyudHypWzR2iNpygjI8kiygCyPJAs4j81xU+EMxu/3F71Ptqdu22j5tVhsjiSPJAvI8kiygPPYHJOWmi/u4XC46H28ls5zX4vF5kjySLKALI8kCziPzTFpqfni3tXVVfQ+2dPw2dHyltT2NVlsjiSPJAvI8kiygPPYHJOWmi/uJ9NzT1h2VzhJv+GCLI8kC8jySLKA89gckT13pdTrlVJPK6WeU0p9aInXP6eUeizz5xml1Hjea1uUUvcopZ5SSj2plNpWqnYlEomi98mehtcLHlc6a7HYHEkeSRaQ5ZFkAeexOSYtdSY+RCnlBb4IvBY4CjyslLpLa/1kdhut9Qfytn8/cG7eW3wV+JTW+odKqWYgVaq2rWXeoWdBT92W4i5pPijI8kiygCyPJAs4j82ROM/9IuA5rfULWus4cAC4doXtbwC+CaCU2gnUaa1/CKC1jmqtp0rVsLXNc1/w2JLT8pLmg4IsjyQLyPJIsoDz2ByJ89w3Av15j49mnlsUpdRWYDtwb+ap04FxpdR3lFKHlFKfzZwJKEmampqK3mdhMbdlnvtaLDZHkkeSBWR5JFnAeWyOSYuR0/JF5nrgDq11MvO4Dng16dP0LwPfAt4F/HP+ToODg9x0003U1dWRTCbZu3cv+/btIxQK0dTUhNfrZWJigu7ubkZHR9Fa093dzcjICCpTrKPRKD09PQwNDaGUoqOjg6GhIVpbW0kmk0xOTtLb28vMdGxegyejUQaTUWKxGL29vYRCIfx+Py0tLYyMjNDe3k4sFmN6ejr3ekNDA4FAgLGxMTo7O4lEIsTj8dzrgUAAv99POBymq6uLcDhMIpHIvb6UaWpqiubmZgYGBmhubi7KFAqF8Pl8BINBhoeHCQaDxOPxiprGxsaIxWJ0d3dXvWlkZITGxsZ533vVbIrFYvT19S36/1SNppGREYBlf0ZUmykQCNDX17fiz71qMkUiEVpaWlb9WV4NJr/fz/Hjx1f9WV6oaaUorfWKG5QiSqlLgf1a66syjz8MoLX+9BLbHgL2aa1/lnl8CfAXWusrMo/fCVyitd6Xv98DDzygd+zYUXTb+vr62Lp1a1H7fOLHL/LTF3Pj/fjSm89gS3tD0Z9d6qzFYnMkeSRZQJZHkgWcx+aU2nLw4MFH9+zZc8FSr5k6ofwwcJpSartSyk+6d37Xwo2UUjuAduCBBfu2KaW6M49/DXhy4b5rTXd39+obLYh3wSV2W07Lr8VicyR5JFlAlkeSBZzH5pi0GClLWutZ4H3AD4CngNu11keUUh9XSr0pb9PrgQM673RC5vT8B4EfK6WeABTwT6Vq2+joaNH7LBwd77FktPxaLDZHkkeSBWR5JFnAeWyOSYuxa+5a67uBuxc8d8uCx/uX2feHwFllalfR+ywaUGfJaHkTl1hMRpJHkgVkeSRZwHlsjkmLJSeUK5c1nZb32DnPXdLpK5DlkWQBWR5JFnAemyPutLzNGRgYKHqfxT33UrXm5LIWi82R5JFkAVkeSRZwHptj0lLzxT07zaCYLBxAt3DFukplLRabI8kjyQKyPJIs4Dw2x6Sl5ov7WrJwAJ0tp+VdXFxcXFzAFXei0WjR+yxeoc6O4r4Wi82R5JFkAVkeSRZwHptj0lLzxb2np6fofRbNc7ejtq/JYnMkeSRZQJZHkgWcx+aYtNR8cR8aGip6H1tPy6/FYnMkeSRZQJZHkgWcx+aYtNR8cVdrGAyXf1peYc+AurVYbI4kjyQLyPJIsoDz2ByTlpov7qstvr9U8nvqtvTaYW0WmyPJI8kCsjySLOA8NsekpeaL+1pOk+RPhbOotos6fQWyPJIsIMsjyQLOY3PcaXmDaW1tLXqf/NPyNvXc12KxOZI8kiwgyyPJAs5jc0xaar64J5PJ1TdakHmn5S26HrQWi82R5JFkAVkeSRZwHptj0lLzxX1ycrLofWztua/FYnMkeSRZQJZHkgWcx+aYtNR8ce/t7S16n/k991K25uSyFovNkeSRZAFZHkkWcB6bY9JS88U9FAoVvU9+QbflXu6wNovNkeSRZAFZHkkWcB6bY9JS88Xd5/MVvY+t19zXYrE5kjySLCDLI8kCzmNzTFpqvrgHg8Gi9/HMu+ZeytacXNZisTmSPJIsIMsjyQLOY3NMWiwqTZXJ8PBw0fvk99xtWZ0O1maxOZI8kiwgyyPJAs5jc0xaar64r+U3qfzeuk2j5SX9hguyPJIsIMsjyQLOY3Ncz91g4vF40fvMmwpnUc99LRabI8kjyQKyPJIs4Dw2x6Sl5ot7LBYrep/5a8uXsjUnl7VYbI4kjyQLyPJIsoDz2ByTFotKU2WypnnulvbcJc0HBVkeSRaQ5ZFkAeexOW6eu8GsZd5h/mV2m665S5oPCrI8kiwgyyPJAs5jc9w8d4Px+/1F72PraPm1WGyOJI8kC8jySLKA89gck5aaL+4tLS1F72PrNfe1WGyOJI8kC8jySLKA89gckxaLSlNlMjIyUvQ+tl5zX4vF5kjySLKALI8kCziPzTFpqfni3t7eXvQ+ts5zX4vF5kjySLKALI8kCziPzTFpqfnivqapcJb23CVNGQFZHkkWkOWRZAHnsTluKpzBTE9PF72Prdfc12KxOZI8kiwgyyPJAs5jc0xaLCpNlcnJznO3abS8pPmgIMsjyQKyPJIs4Dw2x81zN5g13c/d0mvukuaDgixPuSzRaJSzzz6bb3/727nnIpEIu3fv5s4778w9d+jQIa6//nq2b9/Otm3buOSSS/jkJz/J+Pg4AN/4xjfo6upi8+bNbN68mXPPPZcvf/nLq3r6+vro7Ozk5ptvLovPRCR9n4Hz2Bw3z91gGhoait7HM+9+7qVszcllLRabI8lTLktzczO33norH/nIR3J3nNq/fz/nnnsu1157LQAPPvggb3rTm7j44ot58MEHeemll7jjjjvwer0cPnw4914XXngh/f399Pf3c9ttt7F//35+8YtfrOg5cOAAbW1tfPe732VmZqYsxuWSTCZL8j6Svs/AeWyOSUvNF/dAIFD0PvMG1FnUc1+LxeZI8pTTsmfPHl73utfxoQ99iPvuu4/vfe97fPazn829vn//ft72trfxgQ98gHXr1gGwadMmPvzhD/OqV71qyfc866yzOP3003nmmWeWfD0QCKC15lvf+hYf/ehH8fl8/Md//Me8be6++25+5Vd+hS1btnDeeefxox/9CICxsTH27dvHzp072b59O+94xzuA9NmDN7zhDfPeo6OjgxdeeAGAffv2cfPNN3PdddexadMmfvrTn3LPPfdwxRVXsGXLFnbt2sVnPvOZefv//Oc/56qrrmLbtm3s2rWLb3zjGxw8eJBXvvKVuV8OAoEA//qv/8qrX/3qgv69bY+k/zcgy2PSUmfskyzN2NgYra2tRe1j6wp1a7HYHJs9r/vSISOfc8/vnlvQdp/61Ke49NJL+clPfsLHP/5xenp6AJicnOThhx/mox/9aFGfe/DgQZ577jnOOeecJV8fGxvjyJEjHD9+nL179/L0009z4MCB3NmCRx99lPe+97185Stf4YorriAUChGNRgH4gz/4A5qamvjZz35GU1MTDz30UMHtuuOOO/jWt77FgQMHiMfjPPLII/zd3/0dO3bs4KmnnmLv3r3s3r2ba665hv7+fq677jo+97nP8aY3vYlIJMKxY8fYvXs37e3t3Hvvvbz2ta9lbGyM22+/nbe+9a1F/RvZGpv/36wlkjwmLTXfc+/s7Cx6H6+la8uvxWJzpHnKmba2Nnbs2EEsFuONb3xj7vnx8XFSqVSuxw7w53/+52zbto1NmzbxV3/1V7nnH3nkEbZt28aWLVu48soreetb38opp5yy5Od1dnbyzW9+kyuvvJK2tjbe/OY38+Mf/5ihoSEAvv71r/P2t7+d17zmNXg8HjZs2MDpp59OKBTiRz/6EbfeeittbW34fD4uv/zygp1XX301l1xyCR6Ph4aGBl71qlexc+dOPB4PZ555Jnv37uX+++8H0r8IXHHFFbz5zW/G5/PR0dHB7t27Abjhhhty4xS8Xi/33nsvb3nLWwpuh82R9v9GksekpeZ77pFIhObm5qL2mTcVzqKe+1osNsdmT6E96mxOnDjB+vXry9QauP3223n55Ze54oor2L9/P7feeiuQLvoej4eBgQFOP/10AD72sY/xsY99jN///d9ndnY29x4XXHAB3//+9wEYHBzkPe95D5/4xCe45ZZbFn3e0NAQd955J5///OcBuOiii9i0aRN33HEH733ve///9s48Pooi/f/vh0AC4ZAEAjGEEO6AgCggATTKAkpckEPQrLDCb1dBxEW/cigSWQRdrlVZhe8CniuKwQ2HyCGgYr5yyqHIfRNyciZADkgC9fuje8ZJmJyEyaRT79drXjNdVV39fLqm++k6uoqEhAR69ep1034JCQn4+PhQu3btEuls0KBBru2dO3cydepUDh48SFZWFllZWfbWg4SEBIKDg53mM3jwYN5++23S09OJjo4mNDTUMqOy3fm6KQlW0uNKLRW+5p6VlVXsfXK9CudGZ7AkWtwZK+m5nVrOnTvHpEmTmDNnDu+88w4rVqxg69atAFSvXp0OHTqwatWqYuVZr149+vbte1M/uo1169Zx5coVxo8fT0hICCEhISQlJREVFQUYTvjkyZM37degQQNSUlK4dOnSTXHe3t65Jvk4c+bMTWkkz8P0iBEj6N27N3v37iU2Npbhw4ejlLIf69SpU07tDwgIoFOnTqxatYqvv/7aMk3yYK3rBqylx5Va3Mg1lQ0les/dTWvuVql52LCSntupZcKECTz66KM88MAD+Pv7M2XKFF566SX76PU33niDL774gjlz5tibzRMSEoiNjc03z4sXL7J69WpCQkKcxq9bt44hQ4awadMmYmJiiImJYe3atezbt48DBw4wdOhQFi9eTExMDDdu3CAxMZEjR47g7+9Pz549GTduHKmpqWRnZ7NlyxYA2rRpw6FDh9i7dy9Xr15l5syZhWpPS0vDx8eHqlWrsmvXLpYuXWqPGzx4MDExMSxfvpycnBwuXrzI3r177fERERG89957HD9+nD59+hR+ossJVrpuwFp69HvuLqRk77m752h5K70PCtbSc7u0rF69mu3btzN16lR72NNPP42/v799xHxoaCgrVqxgy5YtdOrUieDgYAYPHsz999/PiBEj7Pvt2LHD/p57aGgoderUcepgExMT2bRpE6NGjaJ+/fr2T/v27enRowdffvklHTp0YO7cuUyaNIng4GD69u1LXFwcAPPnz6dKlSp07tyZFi1aMH/+fACaNWvG+PHjGTBgAB07diQ0NLRQ/bNnz2b69OkEBQUxe/Zs+vfvb48LDAxkyZIlzJs3jyZNmhAWFpbr1T/boLuHHnoIb2/vYp5598VK1w1YS48rtYitCau8s3XrVpVfLaMgzp49m2uwUVG4oRS9P/oVgKfa12d4x4BiH/d2UBIt7oyV9FhJC1hHz7333svkyZNzPRSUd6xSNjaspKe0tezevXtXjx49OjqLq/A1d09Pz2LvU0kEW33dnWruJdHizlhJj5W0gDX0rFy5EhEhLCysrE0pVaxQNo5YSY8rtVR45+5sYE9RsDl1d+pzL6kWd8VKeqykBcq/nr59+zJu3DhmzZrFlStXytqcUqW8l01erKTHlVoq/KtwdevWLdF+HgI5uFfNvaRa3BUr6bGSFij/er755hv77/T09DK0pPQp72WTFyvpcaUWl9XcRaS3iBwWkWMi8qqT+HdF5Ffzc0REUvPE1xKReBGZW5p23WrN3Y18u6WecMFaeqykBaylx0paQOtxZyxXcxcRD2Ae0AuIB3aIyEql1AFbGqXU/zik/xuQd5aQacD/lbZt2dnZJdrP3izvRt69pFrcFSvpsZIWsJYeK2kBrcedcaUWV9Xc7wOOKaVOKKWygCigXwHp/wR8adsQkQ5AfWB9aRtW0vcObXPKu1Ofu5XeBwVr6bGSFrCWHitpAa3HnbHie+4NgDiH7Xgz7CZEpBHQGPjB3K4EvA2Mux2GlfS9Q9ua7u5Uc7fS+6BgLT1W0gLW0mMlLaD1uDOu1OKOA+oigGillG2x5ueBNUqp+LxTTzpy9uxZ/vrXv1K5cmWuX7/OwIEDGT16NMnJyVSvXh0PDw8uX76Mn58fFy9eRCmFn58fGRkZXLhwATBmu6pfvz7nzp1DRPD19eXcuXPUqlWL69evk56ejr+/v1FAN24AkHrxAqmplcnKyiIzM9Me7+npSc2aNblw4QI+Pj5kZmZy9epVe3zVqlWpVq0aKSkp1KlThytXrpCVlWWPr1atGp6enly6dIm6dety6dIlsrOz7fHONNmOcebMGfv8xcXRVKVKFe644w7Onz/PHXfcUeaarl69SlxcHH5+fuVeU0ZGBhkZGbn+e+VZk1KK2NjYm66n8qgpIyOD8+fP53uPKG+aPD09iY2NLfC+V540ZWRkcO3atULv5eVBk4eHB4mJiYXey4uqqSBcMomNiHQBpiilHjG3JwIopaY7SfsLMFoptcXc/gJ4ALgB1AA8gf9VSuUalFfSSWxSUlLw8fEp9n5PL9lP8pUsxoUF8XAL91i1qKRa3BUr6XE3Lb6+vuzcuZMmTZqUaH9303MrWEkLlB898fHxdOnShVOnTuHh4ZFvupSUFNLT04uU1tUMHjyYgQMH8qc//alI6Uu7bNxhEpsdQHMRaSwinhi185V5E4lICOADbLWFKaWGKKWClFLBGE3zn+V17LfC5cuXS7Sfra/dndZzL6kWd8VKem6XFtt0sQ0bNqROnToEBATYt21LmhaXxYsXU7duXRo2bEhQUBBhYWGsW7cOgE2bNlGnTh3atGljP47txjZjxgx8fX1Zvny5Pa+cnBx8fX05ffp0gcccPXo0fn5+ZdIE647/s3fffZdp06aRlZXFsGHDuPvuu/H19WXTpk0F7peSksIzzzxDYGAg7dq1Izo62h63fv16wsPDCQ4OJiQkhDFjxjh9xz8lJYXmzZsTHh6e73EWL15cYHxRCAwMJC4urlBnffny5SKndTX//e9/i+zY+/bty6effnp7DXLAJc5dKZUDvACsAw4CXyml9ovIVBF5zCFpBBClXDgnrp+fX4n2+320fGlac2uUVIu7YiU9t0tLXFyc/RMYGMjixYvt24MHDy5xvp06dSIuLo5Tp04xdOhQ/vKXv5Caaryd6u/vz9GjR+3H+fJL+9hXfHx8mDFjBtevX88v65tIT0/nm2++oVatWnz11Vcltrkk5OTkuOX/bP369fYlc0NDQ1mwYAH169cvdL/x48dTo0YNDh06xMKFCxk7diwHDx4EDCc5duxYDhw4wLZt20hKSuLvf//7TXlMmTLFvjzwrVCc/0BBuGP5lJSaNWu67Fguc01KqTVKqRZKqaZKqbfMsMlKqZUOaaYUVCtXSn2qlHqhNO26ePFiifbzENu3+9TcS6rFXbGSHldr2bVrFw8//DDBwcG0atWKCRMm3LTc5IYNG7jnnnto1qwZkydP5oY5jsSRSpUqMWTIEDIzM3Mt4Zqfnh49euDp6VksJ21z7OPHj7cvGWsjJSWF0aNH07p1axo3bszQoUPtcWvWrCEsLIygoCDuvfdevvvuOwDuvvtufvzxR3u6GTNmMHLkSABOnz6Nr68vixYtom3btvTr14+LFy8yfPhwQkJCaNSoEX/84x/tDhEgMzOTyMhI2rVrR6NGjQgPDyczM5Mnn3yShQsX5rL3/vvvd7q87vPPP8/cucYUHYmJifj6+vLhhx8CcPLkSZo0aWI//6mpqRw/fpxOnTrh6enJqFGjCA0NpVIh60vbHpKeffZZatSoQWhoKOHh4fayGDRoED179sTb25vatWvz9NNPs3379lx5bN++nYMHD/LUU0/le5zDhw8zduxY+0JDwcHBgNH6MnbsWJ544gkCAwP56aefWL9+PQ8++CBBQUG0adOGGTNm2POxlUVOTg5g1GzfeustevfuTVBQEAMHDuTChQtcvHixyGltREVF0a5dO5o2bcrs2bNv+k84Mnr0aF5++WUGDBhAUFAQffr0sS9yZDsnPXr0oFGjRvTo0SPXOevbty+fffYZ8Htrxuuvv07jxo1p3749GzZsAODNN99k69atREZG0rBhQyZMmIBSitdee40WLVoQFBREt27dOHDgAKWFG9U7y4aSNhLYJ7Fxo9HyVlkEyIa769nUsWO+n+Rly+zpkpct41T//vmmdeSXoUOdhhcXDw8P3nrrLY4dO8a6deuIiYnho48+ypVm9erV/PDDD/z444+sXbuWzz///KZ8cnJyWLRoETVq1KBp06b28PzKRkR47bXXmDVrVpHf6Y2KiuLxxx9n4MCBHD16lF9//dUe99xzz5GZmcmWLVs4cuQIo0aNAoyHl1GjRvHGG29w6tQpVq1aRVBQUJGOB7Blyxa2bdtGdHQ0Sil69uzJjh07OHLkCO3atbM/DABMnjyZPXv28O2333LixAmmTJlCpUqViIiIyPUQs2/fPpKSknj44YdvOl7Xrl3ZvHmz/djBwcH2pW43b95Mly5d7M77+++/JywsrNhN0MePH6dy5co0atTIHnbXXXdx6NChfM+B4zil69ev88orrzBr1iwKGrzcsmVL3n777VytOzaio6N5+eWXOX36NKGhoXh7e/Pvf/+bU6dOsWTJEj755BNWr16db95Lly5l7ty5HD58mOzsbObOnZvvf81ZWoBDhw4xfvx4Fi5cyMGDB7l8+TJJSUn5HtNm9/jx4zl69Cht2rSxr5aYkpJCREQEI0aM4Pjx4zz//PNERETk+3C7a9cumjVrxrFjxxgzZgwvvvgiSikiIyPp0qULEydOJC4ujlmzZvHDDz+wdetWduzYQWxsLB9//HGhg+SKQ4V37rfcLO9GNXcrNV+B9fS4kvbt29OpUycqV65MUFAQw4cPtzsTG2PGjMHHx4fAwECee+45ljk8kOzcudPeN7t06VI+++wzatWqBRiv8zzwwAMEBwcTHBycq48dIDw8nDp16rBo0aJC7YyPj+enn35i0KBB1KtXj7CwMHvtPTk5me+++4533nmH2rVrU6VKFbp16wbA559/zpAhQ+jevTuVKlUiICCgWE3Jr7zyCtWrV6datWr4+fkxdOhQatasiZeXF6+++ir79u3j8uXL3Lhxgy+++ILp06cTEBCAh4cHnTt3xsvLi/DwcI4fP87x48cBWLJkCQMGDHC6OEi3bt3Ytm0bN27cYMuWLYwZM8ZeA9yyZQtdu3a1p92wYYO9Sb44pKenU7NmzVzXTa1atUhLS7sp7caNG4mKimLixIn2sAULFtChQwfat29f7GPbePTRR+2tDFWrVuX++++ndevWVKpUibvuuouBAwfaH3Kc8dRTT9GsWTOqVatG//792bt3b773AWdpwVgQ6JFHHiE0NBRPT08mTpxY4MMKQK9evejatSteXl5ERkayY8cO4uPjWb9+PU2bNuXJJ5+kcuXKPP744zRv3pxvv/3WaT4NGzZk2LBheHh4EBERQXJyMmfPnrXH20a+A1SpUoW0tDSOHDmCUoqWLVuW6nvw7vgqnEs5c+ZMrifdomJz6u7U515SLe6Ku+u5f+fOIqXzHziQax06FEnLPU5qzyXh2LFjREZG8uuvv5KRkcH169e5++67c6Vp0OD3qSYaNmyYq3bTsWNH1q5d6zRvf39/1qxZU6CeSZMm8cILL/DEE08UaOeSJUto0aIFbdu2BYzRx6+//jrTpk0jISEBHx8fateufdN+CQkJJXKANhy1JyYmsmjRIr7++mvOnz9vr0FfuHCBa9eucfXqVXvTsyNVq1ZlwIABfPXVV7zyyissW7aMTz75xOnxGjdujLe3N3v37mXr1q2MGzeOzz//nKNHj7J582Z7TfHGjRv8+OOPvPnmm8XWVL16da5cuZLrurly5UouhwKwY8cORowYwaeffkqzZs0ASEpKYuHChWzcuLHYx3XE8byC8ZA4depUDh48SFZWFllZWfTrl//8ZY7LoVarVo309HTOnDnj1Dk7SwvGQ6GjHd7e3oXWiB3T16hRAx8fH5KTk0lOTiYwMDBX2rzXSn42eXt7A7nXLnAcvBkWFsYzzzzDhAkTiIuLo0+fPkydOtX+EH2ruJFrKhvy/vGLiq37y51q7iXV4q5YSY+rtYwbN47mzZuzY8cOTp8+TWRk5E3NmwkJCfbf8fHx3HnnnUXOvzA93bt3p0mTJnz88ccFpluyZAmxsbGEhIQQEhJCZGQkFy5cYMOGDTRo0ICUlBSn83E3aNAg1xgAR7y9vcnMzLRvO9acbDg6i40bN7JmzRqWL19ObGwse/bsAYyuhzp16lC1atVcTc+OREREEB0dTUxMDNWqVeO+++7LV2u3bt1YuXIl2dnZBAQE0LVrV6KiokhNTbU/3OzevZvAwMASLTDStGlTcnJyOH/+vD1s//79uZref/vtN4YMGcL777/Pgw8+aA/fvXs3Z86coUuXLoSEhDBx4kR2795NSEiI04Fx+dWE84aPGDGC3r17s3fvXmJjYxk+fHixu9uKe+3Ur1+fxMRE+3ZmZmahY14cr4W0tDRSUlLw9/fH39+f+Pj4XGmLe63YEBGqVq2aK2zkyJFs3LiRrVu3cvz4cd5///1i55sfFd65l5SankajR02vCt/4oXFD0tLSqFmzJjVq1ODIkSNOa5Rz584lNTWV+Ph4FixYQP/+/UvVhkmTJvHee+/lG//zzz/vceIDAAAQ40lEQVRz8uRJvvvuO2JiYoiJiWHz5s0MGjSIqKgo/P396dmzJ+PGjSM1NZXs7Gx718LQoUNZvHgxMTEx3Lhxg8TERI4cOQJA27ZtWbZsGdnZ2fzyyy+sXHnTW7e5SE9Px8vLCx8fHzIyMpg2bZo9zjagMDIykqSkJK5fv87PP//MtWvXALjvvvuoVKkSr7/+eqGtFF27duWDDz6gS5cugDH47oMPPiA0NNTev75hw4ab+uxtrQcAWVlZXL161e4gFy9ebG+RqV69On369GHOnDmkp6ezbds21qxZY7frwIEDDB48mBkzZtC7d+9cx+jZsye//vqrvRwmTpxI27ZtiYmJcdr37+fnR2Ji4k2DNPOSlpaGj48PVatWZdeuXSxdurTA9KXBY489xrp169i+fTtZWVnMnDmz0AeKDRs2sG3bNrKysvjHP/5Bx44dCQwMpFevXhw7dozo6GhycnJYtmwZhw8f5pFHHim2XX5+frkG6u3evZudO3eSnZ2Nt7c3Xl5ehQ6aLA4V3rk7648qCs+FNmBi90Y0r1utlC0qOSXV4q5YSY+rtUydOpWlS5cSFBTESy+95NRxh4eH0717dx588EF69erFn//85yLnXxQ9oaGh3HvvvfnGR0VFER4eTuvWralfv779M3LkSNavX09KSgrz58+nSpUqdO7cmRYtWjB//nwAOnTowNy5c5k0aRLBwcH07dvXfuN87bXX7CPQZ8yYwaBBgwq0s1evXjRs2JA2bdrQpUsXOuYZzDh16lRatWpFz549adKkCW+88UauNwuefPJJDhw4UKhz79atG2lpafb+9dDQUDIzMwvtb7/vvvsICAggKSmJQYMGERAQYNeakJBA586d7Wn/+c9/kp6eTsuWLXn22Wd5++23adWqFQDz5s3j/PnzvPjii/Y5CmwPGl5eXrnKoFatWlSpUiXf1+/CwsLsrS22pn1nzJ49m+nTpxMUFMTs2bNL9ABZ3GunVatWzJgxg2eeeYZWrVpRvXp1/Pz88PLyynefQYMGMXPmTJo2bcqePXtYsGABYEz2FBUVxbx582jatCnvv/8+UVFR1KlT/InLRo4cyZo1a2jcuDGvvvoqV65c4aWXXqJJkyb2eQz+9re/FTvf/HDJDHWuoKQz1F29evWmppLyipW0gLX0WEkLWEvPrWqJioriP//5T75jFIrK2bNneeihh9i/f3+hA8BsDBw4kOnTp9OyZUt7mJXKBm5dT1paGo0bN2bnzp1Ox4mMHj2agIAAJk2adCtmFonSLht3mKHObTl37lxZm1BqWEkLWEuPlbSAtfTcipaMjAw++ugjhg0bdst2XL58mWnTphXZsQMsW7Ysl2MHa5UNlEzPt99+S0ZGBunp6UyePJnWrVsX61XJ24Ury6bCO/fiXEjujpW0gLX0WEkLWEtPSbV8//33tGjRgnr16hXa9F8UmjVrxuOPP37L+VipbKBketasWUPr1q1p3bo1J06c4MMPP3SL8+JKGyp8s3xGRob9lYXyjpW0gLX0WEkLWEuPlbSA1uPOlLYW3SxfAFZqwrKSFrCWHitpAWvpsZIW0HrcGd0s70JKa8IAd8BKWsBaeqykBaylx0paQOtxZ1yppcI799JaucgdsJIWsJYeK2kBa+mxkhbQetwZV2qp8M7dcWrA8o6VtIC19FhJC1hLj5W0gNbjzrhSS4V37qU5UX9ZYyUtYC09VtIC1tJjJS2g9bgzrtRS4Z17cnJyWZtQalhJC1hLj5W0gLX0WEkLaD3ujCu1VHjnvmLFirI2odSwkhawlh4raQFr6bGSFtB63BlXaqnwzt1xDevyjpW0gLX0WEkLWEuPlbSA1uPOuFJLhXfuOTk5ZW1CqWElLWAtPVbSAtbSYyUtoPW4M67UYpkZ6r7//vtzQGxx97t48WJdX1/f84WndH+spAWspcdKWsBaeqykBbQed+Y2aGnUo0cPP2cRlnHuGo1Go9FoDCp8s7xGo9FoNFZDO3eNRqPRaCxGhXXuItJbRA6LyDERebWs7SkKItJQRDaKyAER2S8iL5rhviKyQUSOmt8+ZriIyHumxt9E5N6yVXAzIuIhIr+IyCpzu7GIbDdtXiIinma4l7l9zIwPLku7nSEitUUkWkQOichBEelSXstGRP7H/I/tE5EvRaRqeSobEflYRM6KyD6HsGKXhYgMM9MfFZFbX7S99LTMNv9nv4nIchGp7RA30dRyWEQecQh3i3ueMz0OcWNFRIlIXXPbrcvGtMOpHhH5m1lG+0VklkO4a8pHKVXhPoAHcBxoAngCe4DWZW1XEey+E7jX/F0TOAK0BmYBr5rhrwIzzd+PAmsBAUKB7WWtwYmml4HFwCpz+ysgwvw9Hxhl/n4emG/+jgCWlLXtTrT8B3jG/O0J1C6PZQM0AE4C1RzKZHh5KhsgDLgX2OcQVqyyAHyBE+a3j/nbx020PAxUNn/PdNDS2ryfeQGNzfuchzvd85zpMcMbAuswBkbXLQ9lU0D5dAe+A7zM7XquLp+KWnO/DzimlDqhlMoCooB+ZWxToSilkpRSu83fV4CDGDfifhiOBfO7v/m7H/CZMtgG1BaRO11sdr6ISCDwR+BDc1uAPwDRZpK8Wmwao4EeZnq3QETuwLjIPwJQSmUppVIpp2UDVAaqiUhlwBtIohyVjVLq/4CLeYKLWxaPABuUUheVUinABqD37bc+N860KKXWK6Vs71VtAwLN3/2AKKXUNaXUSeAYxv3Obe55+ZQNwLvABMBxlLdblw3kq2cUMEMpdc1Mc9YMd1n5VFTn3gCIc9iON8PKDWbT5z3AdqC+UirJjEoG6pu/3V3nHIyL+Ya5XQdIdbhpOdpr12LGXzLTuwuNgXPAJ2J0M3woItUph2WjlEoA/gmcxnDql4BdlN+ysVHcsnDbMsrDXzBqt1BOtYhIPyBBKbUnT1S51AO0AB4wu6liRKSTGe4yPRXVuZdrRKQGsBR4SSl12TFOGW0/bv9+o4j0Ac4qpXaVtS2lRGWMprl/K6XuAdIxmn7tlKOy8cGoNTQGAoDqlFGt6HZRXsqiMERkEpADfFHWtpQUEfEGXgMml7UtpUhljC6DUGA88JWrW7MqqnNPwOjfsRFohrk9IlIFw7F/oZSyzWV4xtaka37bmoDcWWc34DEROYXRBPUH4F8YzW6VzTSO9tq1mPF3ABdcaXAhxAPxSqnt5nY0hrMvj2XTEziplDqnlMoGlmGUV3ktGxvFLQt3LiNEZDjQBxhiPqxA+dTSFONBco95PwgEdouIP+VTDxj3g2Vmd8LPGK2TdXGhnorq3HcAzc3Rv54Yg4BWlrFNhWI++X0EHFRKveMQtRKwjRYdBnztEP60OeI0FLjk0CxZpiilJiqlApVSwRjn/wel1BBgIzDITJZXi03jIDO929S8lFLJQJyItDSDegAHKIdlg9EcHyoi3uZ/zqalXJaNA8Uti3XAwyLiY7ZmPGyGlTki0hujS+sxpVSGQ9RKIEKMNxgaA82Bn3Hje55Saq9Sqp5SKti8H8RjDBxOphyWjckKjEF1iEgLjEFy53Fl+dzKaLzy/MEYhXkEY4TipLK2p4g234/RlPgb8Kv5eRSjf/N74CjGCE1fM70A80yNe4GOZa0hH10P8fto+Sbmn/0Y8F9+H21a1dw+ZsY3KWu7nehoD+w0y2cFxijeclk2wBvAIWAfsAhjdG+5KRvgS4zxAtkYzuKvJSkLjP7sY+bn/7mRlmMYfbS2+8B8h/STTC2HgXCHcLe45znTkyf+FL+PlnfrsimgfDyBz83rZzfwB1eXj55+VqPRaDQai1FRm+U1Go1Go7Es2rlrNBqNRmMxtHPXaDQajcZiaOeu0Wg0Go3F0M5do9FoNBqLoZ27RqNBRNJEpElpp3V3RGRtWa4optHcLrRz12hKARGpISKnRGSIQ1hNETktIoMcwjqKyCoRSRGRVDGW731Lfl9+dLiIXDcdaJqInBCRUQUc9yERib9V+5VSNZRSJ0o7rSsQkR9F5JmS7KuUCldK/afwlBpN+UI7d42mFFBKpQEjgTki4mcGzwJ2KqWiAUSkK/AjsBkIUUrVxpizPQe42yG7raYDrQE8DswSkXtKapvDlLEajaaCoJ27RlNKKKXWAauB90TkIeAJjLXObcwCPlFKTVdKnTH3Oa2U+rtS6sd88vwFY2nfVnnjzFXn1gIBDjX9ABGZIiLRIvK5iFwGhovIfSKy1WwtSBKRueY0l7a8lIg0M39/KiLzRGS1iFwxV7ZqWsK0D4vIYRG5JCL/K8YKWU5r2aaNO0XksoicEZF3HOJCRWSLaf8e8/wiIm8BDwBzTf1zneRb1TwXF8z9d4hIfTPOXus3801z+CiH4zg9vkbjrmjnrtGULv+DMZ1uNDBOGfNj2xxxF4xFf4qMGEtFtsCY1jYXSql0IBxItNX0lVKJZnQ/04baGCuGXTdtq2va0YPcDx55icCYgtYHY3rPt4qbVkTqmjZMxJj69TDQtYB8/gX8SylVC2Mxka/MfBpgPDS9ibHS1jhgqYj4KaUmAT8BL5j6X3CS7zCMxWwamnY8B2TmTaSUutuhxeRl097dBR2/AC0aTZminbtGU4oopVKA/YA3xmpqNnwwrrdkW4CIzDJrgukiEumQNtQMv4IxV/sijPnQi8NWpdQKpdQNpVSmUmqXUmqbUipHKXUKWAA8WMD+y5VSPytjffYvMObNL27aR4H9SqllZtx7jvqdkA00E5G6Sqk0pdQ2M3wosEYptcbUswHjYefRQs6BY751gGZKqevmubicX2IRuR/DkT9mprvV42s0Lkc7d42mFBGRoUAwxsIkMx2iUjCWfbzTFqCUmmD2uy/HWP/ZxjalVG2lVE3AH7gL+EcxTYnLY1cLcyBfstlU/w+MWnx+ODrhDKBGCdIGONqhjIUsChr891eMVopDZtN5HzO8ETDYfOBJFZFUjEWU7swvozwswlgxLEpEEs2HqirOEopIQ4wWg2FKqSOldHyNxuVo567RlBIiUg94F3gWY3DdEyLyANib0LcDA4uTp9k3vxTom1+SIob/G2OVt+Zms/drGCtu3U6SMNalBuxLFgfml1gpdVQp9SegHsaDUbTZnREHLDIfeGyf6kqpGbZdCzJCKZWtlHpDKdUao1ugD/B03nQiUg1jNb85Sqm1DlGFHV+jcTu0c9doSo+5wAql1EZlrDk9AfhARLzM+AnAX0TkVfNBABEJBBrnl6GI1AEGYDT1O+MMUEdE7ijEtprAZSBNREKAfF+vK0VWA21FpL85Yn80RkuEU0RkqNmPfgNINYNvYCyd2VdEHhERD3OA3EPmuQPjHOT73r2IdBeRtiLigXEOss188/IxcEgpNStPeGHH12jcDu3cNZpSQET6YzTVjreFKaU+BBKByeb2JuAPQBhwxGze/Rbj9bj3HbLrYhuxjTFS/hzwN2fHVUodwlhP+oTZZByQj4njgKeAK8AHwJKSKS06SqnzwGCMtwQuAK0x+qqv5bNLb2C/qftfQIQ5XiAOY4DgaxjnIg7jPNvuX/8CBokxd8B7TvL1xxjYdxnjfMZgNNXnJQIYkGfE/ANFOL5G43bo9dw1Go1LEJFKGH3uQ5RSG8vaHo3GyugnT41Gc9swm7Jrm10Ttn7+bYXsptFobhHt3DUaze2kC3AcOI8xKLC/Uuqmd8w1Gk3popvlNRqNRqOxGLrmrtFoNBqNxdDOXaPRaDQai6Gdu0aj0Wg0FkM7d41Go9FoLIZ27hqNRqPRWAzt3DUajUajsRj/HyFmZfsSBHspAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"source": [
"acc_differences = np.array(accuracies) - tabpfn_acc\n",
"xgb_better = acc_differences > 0\n",
"first_better = xgb_better.argmax()\n",
"print(\n",
" f\"\"\"\n",
" First XGB train size better than tabpfn: {train_sizes[first_better]} \n",
" {round((1 - train_sizes[first_better]/1024)*100)}% smaller\n",
" \"\"\"\n",
")\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "7EMZT3VvqlMH",
"outputId": "6d31497e-e4fb-44b7-c686-a11b16974157"
},
"execution_count": 15,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
" First XGB train size better than tabpfn: 500 \n",
" 51% smaller\n",
" \n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"### Where might TabPFN do better?\n",
"\n",
"Train a bunch of TabPFN models on increasing data sizes and compare to XGB at the same training set size."
],
"metadata": {
"id": "tD_qUf-ELMJZ"
}
},
{
"cell_type": "code",
"source": [
"def train_test_tabpfn(indices):\n",
"\n",
" clf = TabPFNClassifier(device='cuda')\n",
" clf.fit(\n",
" df_train[columns].iloc[indices], df_train.iloc[indices]['income_bracket']\n",
" )\n",
"\n",
" preds = np.zeros(len(df_test))\n",
" for i in range(len(batches)):\n",
" if batches[i] == len(df_test):\n",
" continue\n",
" \n",
" torch.cuda.empty_cache()\n",
" low = int(batches[i])\n",
" hi = int(batches[i+1])\n",
" y_eval = clf.predict(df_test[columns].values[low:hi], return_winning_probability=False)\n",
" preds[low:hi] = y_eval\n",
" \n",
" del clf\n",
" return preds\n",
"\n",
"\n",
"tabpfn_train_sizes = [\n",
" 15, 25, 50, 75, 100, 150, 200, 300, 400, 500, 600,\n",
" 700, 800, 900, 1000\n",
"]\n",
"\n",
"tabpfn_accuracies = []\n",
"for train_size in tqdm(tabpfn_train_sizes):\n",
" i_preds = train_test_tabpfn(np.random.randint(0, len(df_train), size=train_size))\n",
" tabpfn_accuracies.append(\n",
" accuracy_score(df_test['income_bracket'], i_preds)\n",
" )\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "SAde-SMxLTgW",
"outputId": "45da354a-b87b-4e06-d312-b4b752de7569",
"collapsed": true
},
"execution_count": 17,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 0%| | 0/15 [00:00<?, ?it/s]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6066ef0>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc6051cb0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc61abb00>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc61ab5f0>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6066dd0>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6066e60>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6066ef0>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 7%|▋ | 1/15 [00:08<01:57, 8.42s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc7157170>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc605b320>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc610f200>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc610f170>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc7157830>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc7157680>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc7157170>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Nan caught in MLP model x: tensor(20, device='cuda:0') y: tensor(10, device='cuda:0')\n",
"{'is_causal': False, 'num_causes': 4, 'prior_mlp_hidden_dim': 111, 'num_layers': 3, 'noise_std': 0.026010460754866297, 'y_is_effect': False, 'pre_sample_weights': True, 'prior_mlp_dropout_prob': 0.8798318739130685, 'pre_sample_causes': True}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 13%|█▎ | 2/15 [00:17<01:54, 8.83s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc719bd40>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc719bb00>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc616f5f0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc616f9e0>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc713d9e0>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dca856b00>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc719bd40>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 20%|██ | 3/15 [00:27<01:50, 9.22s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605b710>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc60817a0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc01677a0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc61af4d0>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605b5f0>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605b680>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605b710>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 27%|██▋ | 4/15 [00:36<01:41, 9.22s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc719bb00>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc719acb0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc6179ef0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc6179e60>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc617f950>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc719b320>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc719bb00>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 33%|███▎ | 5/15 [00:45<01:32, 9.21s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605bb00>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc61810e0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc013e0e0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc013e200>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605b710>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605b170>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc605bb00>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 40%|████ | 6/15 [00:55<01:24, 9.35s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179b90>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dca8b5a70>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc610f0e0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc610fd40>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179ef0>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179e60>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179b90>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 47%|████▋ | 7/15 [01:05<01:16, 9.51s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6051f80>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc61ba7a0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc010ff80>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dcaab4440>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6051a70>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc60517a0>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6051f80>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Nan caught in MLP model x: tensor(10, device='cuda:0') y: tensor(10, device='cuda:0')\n",
"{'is_causal': False, 'num_causes': 4, 'prior_mlp_hidden_dim': 61, 'num_layers': 4, 'noise_std': 0.14386383872542202, 'y_is_effect': False, 'pre_sample_weights': False, 'prior_mlp_dropout_prob': 0.5410763686413531, 'pre_sample_causes': False}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 53%|█████▎ | 8/15 [01:15<01:09, 9.94s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc014b170>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc0171950>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc713d3b0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc713d950>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc014b710>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc014b0e0>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc014b170>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 60%|██████ | 9/15 [01:27<01:02, 10.47s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc0171b00>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc014b9e0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc61ab0e0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc61ab710>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc0171200>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc0171320>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc0171b00>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 67%|██████▋ | 10/15 [01:40<00:56, 11.21s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc01673b0>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dca856b00>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc6055050>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc6055170>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc01675f0>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc01670e0>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc01673b0>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 73%|███████▎ | 11/15 [01:54<00:48, 12.03s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179dd0>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc60518c0>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc0139f80>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc0139ef0>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179b90>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179f80>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6179dd0>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Nan caught in MLP model x: tensor(20, device='cuda:0') y: tensor(10, device='cuda:0')\n",
"{'is_causal': False, 'num_causes': 2, 'prior_mlp_hidden_dim': 104, 'num_layers': 4, 'noise_std': 0.047356857520838784, 'y_is_effect': False, 'pre_sample_weights': False, 'prior_mlp_dropout_prob': 0.864927320126097, 'pre_sample_causes': False}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 80%|████████ | 12/15 [02:09<00:39, 13.01s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6181f80>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc719bc20>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc010f4d0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc010f710>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6181290>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc61813b0>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6181f80>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 87%|████████▋ | 13/15 [02:25<00:27, 13.86s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6051a70>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dc6181440>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc61795f0>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc6179b00>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6051290>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6051f80>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc6051a70>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\r 93%|█████████▎| 14/15 [02:42<00:14, 14.83s/it]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Loading models_diff/prior_diff_real_checkpoint_n_0_epoch_100.cpkt\n",
"Loading....\n",
"Using style prior: True\n",
"MODEL BUILDER <module 'tabpfn.priors.differentiable_prior' from '/usr/local/lib/python3.7/dist-packages/tabpfn/priors/differentiable_prior.py'> <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc013ecb0>\n",
"Using cuda device\n",
"init dist\n",
"Not using distributed\n",
"DataLoader.__dict__ {'num_steps': 8192, 'get_batch_kwargs': {'batch_size': 1, 'eval_pos_seq_len_sampler': <function train.<locals>.eval_pos_seq_len_sampler at 0x7f3dd2321440>, 'seq_len_maximum': 10, 'device': 'cuda', 'num_features': 100, 'hyperparameters': {'lr': 0.0001, 'dropout': 0.0, 'emsize': 512, 'batch_size': 1, 'nlayers': 12, 'num_features': 100, 'nhead': 4, 'nhid_factor': 2, 'bptt': 10, 'eval_positions': [972], 'seq_len_used': 50, 'sampling': 'mixed', 'epochs': 400, 'num_steps': 8192, 'verbose': False, 'mix_activations': True, 'nan_prob_unknown_reason_reason_prior': 1.0, 'categorical_feature_p': 0.2, 'nan_prob_no_reason': 0.0, 'nan_prob_unknown_reason': 0.0, 'nan_prob_a_reason': 0.0, 'max_num_classes': 10, 'num_classes': 2, 'noise_type': 'Gaussian', 'balanced': False, 'normalize_to_ranking': False, 'set_value_to_nan': 0.1, 'normalize_by_used_features': True, 'num_features_used': <function load_model.<locals>.<lambda> at 0x7f3dc6181e60>, 'num_categorical_features_sampler_a': -1.0, 'differentiable_hyperparameters': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'prior_type': 'prior_bag', 'differentiable': True, 'flexible': True, 'aggregate_k_gradients': 8, 'recompute_attn': True, 'bptt_extra_samples': None, 'dynamic_batch_size': False, 'multiclass_loss_type': 'nono', 'output_multiclass_ordered_p': 0.0, 'normalize_with_sqrt': False, 'new_mlp_per_example': True, 'prior_mlp_scale_weights_sqrt': True, 'batch_size_per_gp_sample': None, 'normalize_ignore_label_too': True, 'differentiable_hps_as_style': False, 'max_eval_pos': 1000, 'random_feature_rotation': True, 'rotate_normalized_labels': True, 'canonical_y_encoder': False, 'total_available_time_in_s': None, 'train_mixed_precision': True, 'efficient_eval_masking': True, 'multiclass_type': 'rank', 'done_part_in_training': 0.8425, 'categorical_features_sampler': <function load_model.<locals>.<lambda> at 0x7f3dc6181ef0>, 'num_features_used_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb5e0>', 'num_classes_in_training': '<function <lambda>.<locals>.<lambda> at 0x7fc575dfb550>', 'batch_size_in_training': 8, 'bptt_in_training': 1024, 'bptt_extra_samples_in_training': None, 'prior_bag_get_batch': (<function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc013ee60>, <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc013edd0>), 'prior_bag_exp_weights_1': 2.0, 'normalize_labels': True, 'check_is_compatible': True}, 'batch_size_per_gp_sample': None, 'get_batch': <function get_model.<locals>.make_get_batch.<locals>.new_get_batch at 0x7f3dc013ecb0>, 'differentiable_hyperparameters': {'prior_bag_exp_weights_1': {'distribution': 'uniform', 'min': 1000000.0, 'max': 1000001.0}, 'num_layers': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 6, 'min_mean': 1, 'round': True, 'lower_bound': 2}, 'prior_mlp_hidden_dim': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 130, 'min_mean': 5, 'round': True, 'lower_bound': 4}, 'prior_mlp_dropout_prob': {'distribution': 'meta_beta', 'scale': 0.9, 'min': 0.1, 'max': 5.0}, 'noise_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 0.3, 'min_mean': 0.0001, 'round': False, 'lower_bound': 0.0}, 'init_std': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 0.01, 'round': False, 'lower_bound': 0.0}, 'num_causes': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 12, 'min_mean': 1, 'round': True, 'lower_bound': 1}, 'is_causal': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'pre_sample_weights': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'y_is_effect': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'prior_mlp_activations': {'distribution': 'meta_choice_mixed', 'choice_values': [<class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>, <class 'torch.nn.modules.activation.Tanh'>], 'choice_values_used': [\"<class 'torch.nn.modules.activation.Tanh'>\", \"<class 'torch.nn.modules.linear.Identity'>\", '<function get_diff_causal.<locals>.<lambda> at 0x7fc575dfb670>', \"<class 'torch.nn.modules.activation.ELU'>\"]}, 'block_wise_dropout': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sort_features': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'in_clique': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'sampling': {'distribution': 'meta_choice', 'choice_values': ['normal', 'mixed']}, 'pre_sample_causes': {'distribution': 'meta_choice', 'choice_values': [True, False]}, 'outputscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'lengthscale': {'distribution': 'meta_trunc_norm_log_scaled', 'max_mean': 10.0, 'min_mean': 1e-05, 'round': False, 'lower_bound': 0}, 'noise': {'distribution': 'meta_choice', 'choice_values': [1e-05, 0.0001, 0.01]}, 'multiclass_type': {'distribution': 'meta_choice', 'choice_values': ['value', 'rank']}}}, 'num_features': 100, 'epoch_count': 0}\n",
"Nan caught in MLP model x: tensor(40, device='cuda:0') y: tensor(10, device='cuda:0')\n",
"{'is_causal': False, 'num_causes': 4, 'prior_mlp_hidden_dim': 19, 'num_layers': 6, 'noise_std': 0.005091342359247765, 'y_is_effect': False, 'pre_sample_weights': False, 'prior_mlp_dropout_prob': 0.06181411513321087, 'pre_sample_causes': False}\n",
"Style definition of first 3 examples: None\n",
"Using a Transformer with 25.82 M parameters\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"100%|██████████| 15/15 [03:00<00:00, 12.01s/it]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"plt.figure(figsize=(8,6))\n",
"plt.plot(train_sizes,accuracies, label=\"XGB\")\n",
"\n",
"plt.plot(tabpfn_train_sizes, tabpfn_accuracies, label=\"TabPFN\", color=\"firebrick\")\n",
"plt.legend(fontsize=12, frameon=False)\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.xlabel(\"Number of Training Records\")\n",
"plt.xlim(left=0);"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"id": "6dtsjwJNNLs0",
"outputId": "8500a59e-2786-40e9-819a-16f5a67c171f"
},
"execution_count": 19,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 576x432 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAFzCAYAAAAjVEDpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9eXycVb34/z6zZrLvSfeWttiNfd/Valn8su8IiAsqAl68bihXrIg/uXhFXLji9wJfFC8CiiKySBFZCwKlFGgLtKVtmi5JJnsyM8ls5/fHLJlMJskks57Deb9eeSUzzzPPnHdn4POccz6fc4SUEoPBYDAYDPpgKXQDDAaDwWAwZBcT3A0Gg8Fg0AwT3A0Gg8Fg0AwT3A0Gg8Fg0AwT3A0Gg8Fg0AwT3A0Gg8Fg0AxboRuQLZ577jnpdDoL3YysIqVECFHoZmQV46QOOnoZJzUwTunh9Xo7V65c2ZDqmDbB3Wq1smTJkkI3I6t0dnZSX19f6GZkFeOkDjp6GSc1ME7psX79+pbxjmkzLB8KhQrdhKzj8XgK3YSsY5zUQUcv46QGxilztAnudru90E3IOs3NzYVuQtYxTuqgo5dxUgPjlDnaBPdAIFDoJmSdtra2Qjch6xgnddDRyzipgXHKHG2Cu27JF6DnaIRxUgcdvYyTGhinzNEmuFut1kI3IetUVVUVuglZxzipg45exkkNjFPmaBPcg8FgoZuQdTo7OwvdhKxjnNRBRy/jpAbGKXO0Ce6m564GxkkddPQyTmpgnDJHm+Cu4770fr+/0E3IOsZJHXT0Mk5qYJwyR5vgHg6HC92ErOPz+QrdhKxjnNRBRy/jpAbGKXO0Ce46ZleaWk810NEJ9PQyTmpgnDJHm+CuYp374OAgBx10EH/84x/jzw0MDHDAAQfw17/+lba2Nt58800uuugiFixYwPz58zn66KO5+eab6e3tBeD++++nvr6eOXPmMGfOHA455BDuueeeQilNiqlfVQcdvYyTGhinzNEmuFss6qmUl5dz22238d3vfjeeSbl69WoOOeQQzjzzTDZt2sQZZ5zBUUcdxauvvsrOnTv505/+hNVqZePGjfHrHHHEEbS2ttLa2spvf/tbVq9ezdtvv10orQlxOByFbkLW0dEJ9PQyTmqgm5OUkpCwEQjlb/pYm41jVAzuACtXrmTVqlVcf/31XHHFFTzyyCO8/PLLANx+++1ccsklfO1rX4ufP3v2bL7zne+Me70DDzyQ/fffny1btnDggQfmvP1TpaKiotBNyDo6OoGeXsZJDYrZKSwlg8MhBoaD9A2F6B8O0j8U/RkO0TcUZGA4SP9QiL7hIAPR54NhyU9Oq+Sgmflx0ya4q1zn/qMf/YhjjjmG5557jptuuommpiY8Hg/r16/nxhtvnNK11q9fz7Zt2zj44INz1NrM6Orqory8vNDNyCo6OoGeXsZJDfLlFArLeCDuHw7SFw3EA0Oxv0eOxYL3wHCQ8DSKs5xWGDY996ljs6WnsuquN3PckghrvnBI2udWV1ezZMkSXnvtNU4//XQAent7CYfDNDY2xs/7/ve/z29/+1uCwSDXXXcd3/jGNwBYt24d8+fPJxwOMzg4yJVXXsnChQuzK5QlampqCt2ErKOjE+jpZZzUYDpOgVCY/uHQOL3oIH2jgnYkSA8MT2830TKHlaoSKxVOG5VOW+TvEhtVThuVJTYqS6xUxo/ZqCixMuQZpLKyclrvNx20Ce4ql8I99NBD7Nq1i5NOOonVq1dz2223UV1djcViob29nf333x+AH/zgB/zgBz/gS1/60qiRisMPP5wnn3wSgI6ODq688kp++MMfTrnXnw98Pl9ev+D5QEcn0NPLOKlB36CXYWtJNFCHRnrRo4L3yLGB4SDewNRjgAAqnNZIQHYmBOVogK5y2qgoGQngldHHNsvU9zLpyfPn9KEL7lPpUecDt9vNDTfcwD333MPixYs59thjOf/88znmmGNYsWIFjz32GCeccELa12tsbOT000/nnnvuKcrgPjQ0VOgmZB0dnUBPL+OUX6SU+ALhMYE5EpAThsKT5q6HQxLYN6X3sghG9ZRjf1fGgndSAK8qsVHmsGKdRqCeDvn+nLQJ7qrWuX/rW9/itNNOiwfw1atXc9111/HCCy9w0003cckllzBjxgw+/elP09DQwJ49e2hpaWHBggUpr9fd3c3jjz/OkiVL8qmRNqZ+VR109DJO00dKiccfiieRDSQG5hQ960gyWYjANCaobRaRIjCP9LCrSpJ62U4rZQ5rUe8Omu/vnjbBXcU698cff5xXX32VV155Jf7c5ZdfzsMPP8xPfvITLrvsMh555BFuvfVWbr/9dgBmzpzJaaedxhe/+MX4a15//XXmzJkDgMvl4sQTT+THP/5xfmXSpK2tjXnz5hW6GVlFRyfQ08s4pYcvEGJH9xDburxs6/SxrctLS8/QtAK102ah0mmN9Khjw9vxXvToAF4V7Vm372ll/vz5WXUqNPn+7gld1mR/4YUX5AEHHFDoZmSV9vZ2mpqaCt2MrGKc1EFHL+M0lsHhINu6fJGfTi/bunzs7htKmRFearekHN6uiAbpqsRj0b+dtqmXKZvPKT3Wr1//xsqVKw9PdUybnruqde4T4XK5Ct2ErGOc1EFHrw+7U483wNYuLx90+dga7ZG3DYzd0MQqYL/aEhbVlbKovpRFdS72q3VR6sjP7psf9s8pG2gT3FWucx+Pnp4e7bJgjZM66Oj1YXGSUtIxGIgMqyf0yLu8Y6cvHVbBgloXi+tKWVTvYlFdKfNrSnBMo8edLT4sn1Mu0Sa4p1vnrhJ1dXWFbkLWMU7qoKOXjk41tbXs7huK9MSjQXxblzdlDXep3cLCeBCPBPK51SV5yxhPFx0/p3w7aRMRVa5zH4+BgQHtVp4yTuqgo5fqTsGwZFdPJNFta6ePD7q8bOv0MhQaO0FeVWKLBPDosPqiulJmVDqwFHFGeQzVP6dU5NvJBPcixu8fOxemOsZJHXT0UslpOBhmR7cv3hPf1uljR4+PQIpAXl9mZ3FdKQvrXCyuj/TM60vtRV0aNhEqfU7pkm8nbYK7qnXuE2FqctVARyfQ06tYnTz+EB/Egnh0jnxXb+qM9ZmVThbXuVgYnR+fW2Glsaos/43OIcX6OWWCqXOfJirWuU+GqclVAx2dQE+vYnDq9QXY1uWLBPNOL1u7fOztHx5znkXA/JqShKH1SM+8LCljvaWlBTQL7sXwOWWbfDtpE9xNKZwaGCd10NErn05SSjq9gfgiMLHfbs/YjojdIpgfLT1bXB8J4gtqXZSkkbFuPic1MKVw00TVuaWJcDgcY56rra1l3bp17LfffgVoUeakclIdHZ1AT69cOYWlZF+/nw+6Ij3xWNZ639DYEt0Sm4WF0QS3xfUuFta5mFfjmtZmJGA+J1XIt5M2wT0Umt7WfYUitlwsgNfrxel0YrVGhttuu+02zj//fPr6+qiurk77mvfffz9f/epXcblcCCGYP38+N9xwAyeffDIvvfQSZ555JqWlpfHzjz/+eP7whz9wyy23cOutt3L33Xdz9tlnA5F1AxobG9mwYQNz587NkjVTdlIBHZ1AT69sOIXCkl29Q5GFYLq8fBDtkafalazCaY1nqsdqyGdWOrNaemY+JzXIt5M2wV21OvfW1tb43wcddBA///nP+ehHPzrqnPr6+ilf94gjjuDJJ58kHA5z11138bnPfY5NmzYBkYSO2N/J1NTUcMstt3DGGWfEbzJywXScih0dnUAvr9DQEPseeABrXR1MYd7THwqzs2co0hOPBvHt3T78KTLWa0tt0YVgolnrdaU0luc+Y12nzymGccqcvEVEIcQpwM8BK3CXlPKWpONzgd8C1dFzrpdSPpF0fDOwWkr5X8nXV63nPh5vvPEG3/nOd9iyZQsOh4OzzjqLm2++edSQztNPP82dd97JwMAAl1xyCatXrx6Tc2CxWPj0pz/N9ddfz44dOyZ935UrV/Luu+/y0EMPcfHFF2fdK0ZfXx9lZXol/+joBHp4hf1+2v7yF1rvuYdAVxe2xkZmnnIKlhTVNb5AiO1dPrZ2RerHt3b6aOnxkSKO01zhiPTG61zxHnltaWEqdnT4nJIxTpmTl+AuhLACdwCfBHYDrwshHpVSbk447T+Ah6SUvxZCLAOeAOYnHL8NeHK899BlAxyr1cqPfvQjDjnkEF5//XWuu+467r77bq666qr4OY8//jj//Oc/8Xg8nH322SxatIjLL7981HWCwSD33Xcf5eXlLFy4kLfffnvC9xVC8N3vfpfvfve7nHfeeTnrbehY1aCjE6jtJYNBOp54gl3/8z8M74vsC16+bBmuc89F2Gz0DwVHlZ5t7fSyp2+Y5P+LCGBudTRjPZq1vrDORYWzeEYKVf6cxsM4ZU6+vqFHAtuklNsBhBAPAGcS6YnHkEBs4d0qYG/sgBDiLGAH4BnvDdKtc3/p8JQb6GSd49etm9brDj744Pjfhx56KFdccQUvv/zyqOD+1a9+lZqaGmpqavjyl7/Mn//853hwX7duHfPnz8dms7FgwQJ+97vfxdczbmtrG7WN4s9+9rP4HDvAqaeeyk9/+lPuu+++MTcL2cLUr6qDil4yHKbzmWfYdeed+FpaAHDttx9cdAWb5x/C+24P2x/cTPvg2AVFbBbB/JqSkYVg6kpZUFuCy56fzVKmi4qf02QYp8zJV3CfBbQmPN4NHJV0zmpgjRDiWqAM+ASAEKIc+DaRXv83xnsDXe70tm3bxn/8x3+wYcMGPB4P4XCYgw46aNQ5s2bNiv89Z84c9kV7JgCHH344Tz6ZeoBjojn3GDfccAPXXHMNF1xwQQYW42PqV9VBJS8pJT1r19Ly3/+NZ8sWAPz1zWw4/hyeaT6YQLeA7rb4+U6rYGG0bjy2POu8mhIcVvVKalX6nNLFOGVO8YwtwcXAvVLKnwohjgHuE0KsIBL0fyalHJxoqLi3t5fjjjsOm81GKBTinHPO4eqrr6atrY2ysjKsViv9/f0c+sILdHd3I6WkoaGB9vb2+Hq/g4ODNDU14Xa7EUJQW1uL2+2msrKSUCiEx+OhubmZtrY27HY7VVVVdHZ2UlVVhd/vx+fzxY/v27ePiooKurq6qKmpwefzMTQ0FD9eUlKCy+Wip6cHKSXd3d20tLTw7//+7yxYsIAnnngCq9XK//7v//L888+zd+/e+A3Mm2++SUNDA1arlbfeeovGxkZ2795Nd3c34XCYlpaWMU4dHR2EQiG8Xu8Yp9jvvXv3cuSRRzJz5kzuuOMOAPbs2UNTUxNtbW04HI60nerq6hgYGMDv98ePx+o8W1paqK+vp6+vj0AgED+e+Dk1NDTk5XMyTqmdHA4HXq8Xj8dT9E7vPPE0A3+4H/v2SFDvL6vmhcNOZ8NHjiVstUWG1ivtLGtw0WDxccTimZRLH6FgkObmxogTEk//MPuKxEnHz8k4Zd9pIkQ+5qqjwXq1lPLk6OPvAEgpf5xwzibgFClla/TxduBo4GEgVjdWDYSBG6WUv0p8jxdffFGuWLEi1yo5ITFb/hOf+ASrVq3im9/8Jm+88QZf+cpXqKuri/fGa2trOfHEE7n33nsZHBzk3HPP5aqrruKKK67g/vvv57777kvZc3/ppZf40pe+lLLnfsstt7Bjxw5+85vfAPCvf/2LSy+9lO7u7qyXwvX09FBTU5O16xUD+XIK+XyEh4ex56mcphg/KyklrX3DbGwbZOdrb1L7198zuyXynfaUlPPSIafx9gEfZeGMalY0l7OiuYxljWWUR+fIi9EpU4yTGuTCaf369W+sXLky5VxzvnrurwOLhRALgD3ARcAlSefsAlYC9wohlgIlgFtKeULsBCHEamAwObCDPtnyN910E1/72tf45S9/yf77789ZZ53Fiy++OOqcU089lY997GP09/dz8cUXc9lll2W1DUcffTSHHnoo//jHP7J6XYD+/n7t/qPNtZN3+3b2PvggHY8/TnhoCGdzM+XLllGxfDnly5dTvmQJthzsNlUMn1UoLNnW5WVjm4eNbYNsbPfg3LOTj77+Vw7cuQGAIYeLlhNOp+zs87hgfiPfqy8ddy/yYnDKNsZJDfLtlJeeO4AQ4jTgdiJlbvdIKX8khLgJWCelfDSaIf8/QDmR5LpvSSnXJF1jNZHgPqYUbu3atXLZsmW51sgrXq931KIzOmCc0kOGw/SsXcveBx6g99VX489bnE7Cw0nrkAuBa968SMBftiwS8PffH4vTmVEbCvFZDQXDvNcRCeTvtHl4t8PDUDCyOExtbzsnrXuUFdteRyCRDiflZ5/H0i98lpKa9EYzzPdPDYxTehRDz51ozfoTSc/dmPD3ZuC4Sa6xerxjweDYZR5Vp7u7W7svuHGamODgIO2PPsq+hx5iaPduACwlJTR+6lPMvPBCXPPm4d25k8FNmxjYvJnBzZvxbNmCb+dOfDt34n4i8p+YsFopXbRoJNgvW0bZfvshprDYU64/K38wTJc3wM6eoWivfJCtnT6CSVuhLWaQlW8+TsPr/0SEwwi7neZzzmHOZz+LY4oLg5jvnxoYp8wppoQ6QxK61O4nYpxS4925k30PPkj7Y48R9vkAcM6cyYwLLqD5jDOwVVbGzy1buJCyhQtpOuMMILJQi2frVgY3b44E/E2b8O7Ygef99/G8/z785S9ApNdfvmQJ5dGAX7FsGSVz5oy7psF0vaSUDAyH6PQE6PT66fIE6PQGIo89Abq8fjo9AfqHx06lWQQsqnNF5stdfur//id6/voXZCAAVitNZ5zBnCuvpGTGjGm3TTeMkxrk20mb4K7a8rPp0NDQUOgmZB3jNIIMh+l5+WX2Pvggva+8En++6ogjmHnhhdSecAIijaWALQ4HFcuXU7F8ObGQF/J6GXzvvXiwH9i8meE9e+h/6y3633or/lprRQUVS5fGe/cVy5bhaGxECJHSKxiWdMcCdTRIRwL26MCdannWZKwCakvtzKhwsrypjBXN5SxrKsM55GH3ffex94EH6I7e6NSvWsW8L30JV4alROb7pwbGKXO0iYi61Lkn0t7erl2tp3GKDL13PPYYex98kKHoHgMWp5PG005jxoUXUrZoUcZtspaWUnXooVQdemj8uUBv76je/cDmzQS6uuh97TV6X3stfp6trg7/vEV0OysIVNbQZy+j2+rCbXHhFi68JWV4Syrw250wTq+/1G6hvsxBXamd+jI79aV26srsNJQ5qIs+rnbZsCS8PuT1svf3v2X3ffcRGhgAoPaEE5h71VWU779/xv8mYL5/qmCcMkeb4J7LzU4KRXkOMqALzYfZybdrVyTr/W9/I+T1AuBsbmbGBRfQdOaZ2KuqctlM7NXV1Bx7LDXHHgtEhgn9HR3xufuedzbSv2kzdHVh6eoiNps93gB42GolVFaJrKjEWlWFo6oaV201ZXW1lNZWYyupxl5Zjb26GltVFfbqCqzl5WOmAcLDw+z785/Z/f/+H4HubiAyejHvK1+h8oADsvpv8GH+/qmEccocbYK7wRAjFJY8t72HgeEQTeUOGsvtNJU74rXO+USGw/S++ip7H3iAnrVr489XHXYYMy66iLoTTphSkls2EULgbGpir72Shy378U/XcQQPC1Hb5+awQDuzwwPUhv2UDntw+gaweQYQA/0E+/oI9PbC0BCW/h7o74E9EAIGoz/jYrVir6zEVl2NvaoKW3U1g+++i7+9HYCKFSuY95WvUH3kkXn4FzAY9EWb4K5LnXsig4OD1NXVFboZWSXXTlvcXm5/aRfbunxjjpXaLdFg76C5IvI79rip3EG1yzatDXNSOQU9Hjoef5x9Dz4YX+Pc4nTScOqpzLzwQsoWL56eYJaQUrJ+zwAPb+xg3e7IELhFwPELajnvgCUsayqjpaVlwmHE0NBQJND39cUD/ni/A319BHt7CXk8BHp6CPT0kPgJlS5ezLyrrorkGeRwi1Tz35QaGKfM0Sa4p7txjEo0NTUVuglZJ1dOXn+Ie9/Yx6Ob3YQlNJU7OGx2BR2DfjoGA7QP+vEGwuzoGWJHz1DKazisgsbysUE/9ru+zI7VMjbwJDr5du+OZL0/+ighT2SfI0dTEzPOP5/ms87K2+py4xEIhXn2gx7+vLGD7d2RfwenzcIp+9dy9opGZlaO1MZP9llZS0qwlpTgnMJnGg4ExgR+a1kZ1UceibDkfl1389+UGhinzNEmuOtY5+52u5kzZ87kJypELpzW7uzljpd30+kNYBFw3gGNXHZo86jdvKSU9A+HaB/00zHgj/weHP17YDjE7r5hdvcNp3wfi4CGsliwt8eDvsXXy0GeDroe/iM9L70E0ZKXykMOYeaFF1L30Y8WbOg9xsBwkMfe7eSvm910eyP/rdS6bJy5vIFPLamnsmRs+3LxWVnsdhz19VOuT88W5r8pNTBOmaNNcNeRXA5PFopsOrk9fu54eTcvt/QB8JGGUq47fg4L68YuFCGEoKrERlWJjf3rUy8k4fWH6PBEgn3bwNjg3+0N0h79+53oa+bu3cKnXvw923oiO/OFbXYcJ63kI5d/mprlS7PmOl329Q/z541untrSFV/pbX5NCecd0MhHF9ZMuAua+f6pgXFSg3w7aRPcdaxzn2zXHxXJhlMoLHl0s5t739iHLxDGZbfw2cNncvrS+pTD5ulS6rAy3+Fifo0r5XF/KIx7MBAP9oMvPk/jE7djCQboL6tm3fKPsn7piXhdFdhfG2Z5y1YOnlHBwTMr2L+hFFsGbZsqm9s9/OmdDl5u6SW24Nuhsyo474BGDptVkdb/aMz3Tw2Mkxrk20mbiKhjnbvb7dau1jNTp22dXm5/qZUtnZFSsuPmVfGVY2fTUObIVhPHxWG1MKvKyawqJ22PPMK2/3sLhMOUn3IKB3/3e9R1DtG0d4AN+wb5oMvHhr2DbNg7CG/sw2W3cEBzOQfPKOfgmRXsV+caVeOdDUJhySstffzpnQ42d0Tm+20WwcpFNZy7opH96lLftIyH+f6pgXFSg3w7aRPcdaxzr0xYclQXpuvkC4S4b30bf97YQVhCfZmda46dzbHz8pugJqVk97330hLd737uF79I+XnnUV7q5Ki5To6aG6lV7xsK8va+QTbsHeDNvQPs7hvmtdZ+XmvtB6DCaeWgaKA/eEYFc6qd0x628wVCrNnSzV82dbC33w9AucPKp5bWc9ayBurKppdsar5/amCc1CDfTtoEdx3RsbxvOk6v7urjly+30jEYSZg7e3kDnzlsBqWO/N7QyXCYHbffzt777wchWPitbzHj/PPp7Owcc25ViY0TFlRzwoLIzUenx8+GvYO8tS8S7DsGA7y0s4+XdkbyBWpdNg6aGRnCP3hmOTMqJt/Rrcsb4NFNbh57r5OB6DrtzRUOzlnRyMn7145KKJwO5vunBsZJDfLtpE1w1/HL4PF4qC9QVnGumIpTlyfAr/+1mxd29AKRDUWuO34u+zfkf7eocDDI1h/8APeTTyJsNva/6SYaVq0C0nOqL3PwicW1fGJxLVJK2gb8bIgO4W/YO0C3L8izH/Tw7Ac9QKSU7+CZIz37xN73jm4fD7/TwbMf9BCITqgvbSzl3AMaOW5edUZ5B4l82L9/qmCc1CDfTtoEdx3r3JubmwvdhKyTjlNYSh57t5N7Xt+LNxCmxGbhM4fN4KzlDVkLXFMhNDTEe9/+Nj1r12JxuVj6k59Qc/TR8eNT/ZyEEMyodDKj0smpS+qRUrKrdyg6Rz/AW/sGaR/089SWbp7aElmOdU6Vk4NnVrBvYDi+6IwAjp9fxbkHNLK8KftLW35Yv3+qYZzUIN9O2gR3HRPq2tratEsqmcxpR7eP21/axbsdkYS5o+ZUcs2xc2iqyH3CXCoCfX1s/trXGHj7bWxVVSz/xS+oWL581DmZfk5CCObVuJhX4+LM5Q2EwpLt3b74fP3GNg+tfcO0RuvvnTYLJ+9fy9nLG5lVNfnw/XT5MH7/VMQ4qUG+nbQJ7jrWReo4GjGe01AwzP+u38ef3ukgJKG21MbVx8zh+PlVBftshzs62HTNNXi3b8fZ1MTyO+6gdP78Medl+3OyWgSL60tZXF/K+Qc2EQiF2eL2smHfIE6bhVWLa1MuOpNtPkzfP5UxTmqQbydtgruO2fJVOd4lrBCkclq3u59frG2lbcCPAM5YVs9nD59JWZ4T5hLxtbSw8ZprGN63D9eCBaz41a/GXWY115+T3WpheXM5y5vzu6vUh+X7pzrGSQ3y7aRNcNdx+dnOzk7KysoK3YyskujU4w1w56t74klk+9WW8G/Hz2VpY2GdB999l43XXkuwt5eKFStYdvvtE64Jr+PnBHp6GSc1ME6Zo01wNz13NaiqqiIsJX9/v4u7XtvLoD+E0yq47NAZnHNAY15XcUtF7+uv8+7Xv07I66X6mGNYeuutWF0TL/6i4+cEenoZJzUwTpmjTXCX0c06dMLv9+f8PYaH/bzxj5cZrq7HNnM2NouI/1gT/k71nNUSGTKOPWcVk+c+7OjycN+LHWxsj6ygdvjsCq49bk5add25pvMf/+D9730PGQjQcPLJLF69Gksa82T5+JwKgY5exkkNjFPmaBPcw+FwoZuQdXy+sXuSZ4vOgSGe//1fcDz8e2p627EDexrmsWnRkWxeeDj95dNbB9kqIkue2qyW+A1A/GZACPb2DxOSUOOy8eWjZ/PR/aqLIhly38MP88Ett4CUzLjwQvb7+tfT3oI0l59TIdHRyzipgXHKHG2Cu47Zlbmoi3y3fZCXHnyM+sfup6l7LwCDlXWUDHmY5W5hlruFVa/8ka65H6F1+TG0LDkST2klwbAkGJaEEn4HEh4Hw5KwhJCEUEgyPMGiQqctqePzR8ykwln4r5+Ukta772bXnXcCMO+qq5j9uc9N6YZDx5pc0NPLOKmBccqcwv/fNUuYOvfxCYTCvLi9h389+g8WPf0gSzt3ATBUXU/1ZVdwzCXnQihEz8sv437qKXpeeom6Xe9Tt+t9Dn7qd1Qffjj1q1ZR94mPYZ9g3igsR98ApLoJ6Olo45CPzM3YKRvIcJjtP/0p+x58MLKc7PXXM+Pcc6d8HR1rckFPL+OkBjp0TOMAACAASURBVMYpc7QJ7pY0h1BVwuHIbOGWHm+Ax9/r5M01L3L4iw9zYvsHAAQra5hxxedYfNG5WGLvYbdT//GPU//xjxP0eOh+4QXca9bQ+8or9L72Gr2vvcYHt9xC9THH0LBqFbUnnogtKfPTIgQOq4AJchudQyUZOWWLcCDA1tWrcT/1FMJu5yM//CH1n/jEtK6V6edUrOjoZZzUwDhljgnuRUxFRcW0Xrel08sjm9xsW7uOE179C2fveQ+AcEUVc6+4gjkXno+1ZPwgaysro/HUU2k89VQCfX10PfssnWvW0LtuHT0vvkjPiy9icTqpOf54Glatoua44ya8XjacsknI5+Pdb32L3ldewVpaytKf/pTqI46Y9vWKwSkX6OhlnNTAOGWONsFdxzr3rq4uysvTW7gkGJa8tKOXRza56d60mY+9/leO3fVO5GBZGXMvv5yZF100prc9GfaqKprPOovms87C39VF5zPP0LlmDf0bNtD1zDN0PfMM1tJSak86iYaTT6b6qKMmzDCfilMuCPT2svm66xjYuBF7TQ3Lf/ELypcuzeiahXbKFTp6GSc1ME6Zo01wt9m0UYlTU1Mz6Tm9vgBPvNfF397txLJ7Jyete5Rl29cDIEpczL7kYmZdeim2LOwl7KirY+YFFzDzggsYbmvD/Y9/0LlmDYObN+N+8kncTz6JrbKSuo9/nIZVq6g67DBE0voD6TjliuG2NjZeey2+HTtwzpjBil/9ClcW5sAK6ZRLdPQyTmpgnDJHm4ioaylc5ThBeVt06P3Z7T1UdLVx4ht/44CtryGQCIeTmRecz+zPfAZ7jr5QzuZmZl96KbMvvRRfayuda9bgXrMG7wcf0P7II7Q/8gj2ujrqV66kYdUqKg48EGGxTOiUS7w7d7Lp6qsZbm+ndL/9WP6rX+FsbMzKtQvllGt09DJOamCcMscE9yJmaGho1ONQWLJ2Z2TofWO7h6qBLk554zEO2vIylnAYYbPRfPbZzP7c53A2NOStna45c5jz+c8z5/Ofx7NtG51PP417zRqGWlvZ99BD7HvoIZxNTdSvWkVgyRJqDjsMe21t2nXkmTKwcSOb/u3fCPb1UXHggSz72c8mzPqfKsmfky7o6GWc1MA4ZY7QZWW3tWvXymXLlhW6GVlleHgYp9NJ31CQJ97r5G/vdtLpCVDu6eVjG57koM0vYAkFwWql6f/8H+Z84QuUzJhR6GYDkfpxz3vv4Y726P3t7aOOC6sVR0MDjsZGHA0NOBsbcTQ24ow9F/3b4sxs5bqef/2Ld7/5TcI+HzXHHceS//zPtJP/0iX2OemGjl7GSQ2MU3qsX7/+jZUrVx6e6pg2PXcd69xf37qb17rt/PODbvwhSalvgLM3P82KDc8gAn4QgoZTT2XulVfimlscteMxhBCUL11K+dKlzL/2Wgbefhv3U0/R+dpryN5egn19DLe1MdzWNuF1bFVVo4N+QwPOpqaRG4KGBmxVqbeFdT/9NFu+9z1kMEjDqaey+Pvfx5KD3Awda3JBTy/jpAbGKXO0Ce66lcL9+pXd/GVTPwAlwx4+veN5Fv7rSYgO7dR9/OPM/dKXKFu4sJDNTAthsVB58MFUHnww5e3tNDU1ER4exu92M9zRgb+jg2G3G39Hx6jn/G43wb4+gn19eLduHff6FqczPgoQuwmQ4TB7778fpGTmJZew4LrrcjYNUJLlkYBiQUcv46QGxilzTHAvQnq8AR7Z5MYVGOKy9n8x87m/Eh4cBKDm+OOZ9+UvU75kSYFbOT1c0R3WLE4nJbNnUzJ79rjnynCYQE/P6ICffCPQ3k7I42Fo926Gdu8ec41511zD7M98Jqfr17sm2TVOVXT0Mk5qYJwyR5vgrlOd+9od3Rz+zjN8/M3HcXoHCANVRx7JvC9/mcoDDyx08zKip6cn7YxRYbHgqKvDUVc34c1MyOuNBP9o0B/u6MDf1UX1EUdQd9JJ2Wr6uEzFSSV09DJOamCcMidvwV0IcQrwcyKLk94lpbwl6fhc4LdAdfSc66WUTwghPgncAjgAP/BNKeU/k6+vS52754MPCN9wA6fu3gZAxYEHMu8rX6H68JQ5E8pRV1eX9WtaS0spnT+f0vnzs37tdMiFUzGgo5dxUgPjlDl5iYhCCCtwB/BJYDfwuhDiUSnl5oTT/gN4SEr5ayHEMuAJYD7QCZwupdwrhFgBPAXMSn4P1UvhwoEAu++9l9a776Y2GGSgrJo511zF8vPOKYotUbPFwMCAditP6egEenoZJzUwTpmTr+7ukcA2KeV2ACHEA8CZQGJwl0BszKIK2AsgpXwz4ZxNgEsI4ZRSDie+gcrBfWDjRrb+8Id4P4hs7LJu2Yn0nPNZlh1SqVVgB/D7/YVuQtbR0Qn09DJOamCcMidfwX0W0JrweDdwVNI5q4E1QohrgTIg1RZd5wLrkwM7qLmfe8jno+XOO9n7hz9AOEzJnDk8+8kreNoxl39fOpPmZr3uXMHs06wSOnoZJzUwTplTTBPVFwP3Sil/KoQ4BrhPCLFCShkGEEIsB/4TWJXqxR0dHVx55ZXYbDZCoRDnnHMOV199NW1tbZSVlWG1Wunv76ehoYHu7m6klDQ0NNDe3h4fKhkcHKSpqQm3240QgtraWtxuN5WVlYRCITweD83NzbS1tWG326mqqqKzs5Oqqir8fj8+ny9+3OFwUFFRQVdXFzU1Nfh8PoaGhuLHw++/j/uXv8S/bx9YLFSfey6ccTbP/MuHFdi/LMDOnTspKSmhvr6evr4+AoFA/PXF6FRSUoLL5aKnp4e6ujoGBgbw+/3x47Fjxqn4nRwOB7t27WLhwoXGyTgZpyJ1moi8rFAXDdarpZQnRx9/B0BK+eOEczYBp0gpW6OPtwNHSyk7hBCzgX8Cn5VSrk31Hi+88II84IADcmySOcGBAXb8/Oe0P/IIAKWLF7P4e9+jYtky/v5+F7e9uIvDZlXw41MX0dHRQWOW1j8vFoyTOujoZZzUwDilRzGsUPc6sFgIsQDYA1wEXJJ0zi5gJXCvEGIpUAK4hRDVwONEsudTBnZAibnpruee44NbbsHf2Ymw25n7hS8w6zOfia+a9sKOHgBOXFANgMPhKFhbc4VxUgcdvYyTGhinzMnLyi9SyiBwDZFM93eJZMVvEkLcJIQ4I3ra14ErhRBvAX8ArpCRYYVrgEXAjUKIDdGfMbc/oVAoHyrTwt/VxXvXX8+73/gG/s5OKg48kEPuv585n/98PLAPDAd5c88AFgHHzo8E976+vkI2OycYJ3XQ0cs4qYFxypy8zblLKZ8gUt6W+NyNCX9vBo5L8bqbgZsnu36x1LkHenrwbNmCZ+vW+I93+3ZkMIjF5WL+Ndcw4/zzxyyF+kpLHyEJh8wsp6ok4lJfX18IhZxinNRBRy/jpAbGKXOKIyJmgXz33MPBIL6dOyMBfMsWPNu24dmyhUBXV8rza44/noXf/va4u7a9uKMXgBMWjOy/3tfXR1lZWfYbX0CMkzro6GWc1MA4ZY42wT0fiYGB3l5afv1rBt55B++OHcgUO9FZS0spXbSIsv33p2zx4sjPokVYS0vHva7HH+KN6JD8cfNG9hnXcac746QOOnoZJzUwTpmjTXDPR5379v/6L9x//3v8ccmsWfEgXrp4MeWLF+OcOXPKu4+90tJHMCw5aEY5NaUjHqbWUw10dAI9vYyTGhinzNEmuOf6rsi7cyfup55C2Gws//nPKV+xAluWhlhGhuSrRz1v9jRWAx2dQE8v46QGxilztAnuVqs1p9dvvesukJKmM86g+qjkxfWmj8cfYt2efgRw3PzRwV23OScwTiqho5dxUgPjlDn6bIKeQ7w7d+JeswZhszH7s5/N6rVfa+0jEJIsby6jrnT01EKub1gKgXFSBx29jJMaGKfM0Sa45zJbvvXuuyEcpumMM8bNdp8uL2yPDMmfmJAlH6O/vz+r71UMGCd10NHLOKmBccocbYJ7rhLqEufas91r9wVCvL478oEfP79qzPGGhoasvl8xYJzUQUcv46QGxilztAnuwWAwJ9eN99pPP33CXntLj49XWqa2AtGru/rxhyTLm8qoLxu7NGF3d/eU21vsGCd10NHLOKmBccocbYJ7Loj32q1WZn/ucxOee+vzLXz/6e1sbvekff0Xd6bOko+Rj9r9fGOc1EFHL+OkBsYpc7QJ7rlYfjbdufawlOzsGQLg+e09aV3bFwjxWmtsSD51cDdDU2qgoxPo6WWc1MA4ZY42wT3bde5T6bW7BwMEQpG7shd29BJO4w7t9d39DAfDLG0spbE89W5B7e3tU294kWOc1EFHL+OkBsYpc7QJ7tkuM5hKhvye/qH4313eAO+mMTQfX7hmnF47QHl5eZqtVQfjpA46ehknNTBOmaNNcM8mvpaWkV57Ghnyu/uGRz1+Phq4x2M4GObVXdEh+XHm2w0Gg8FgmC7aBPds1rnvivbaG884g5KZMyc9f09/JLjHytlenGRo/vXd/QwFw3ykoZTmCue45w0ODk6x5cWPcVIHHb2MkxoYp8zRJrhnq8492N+P++9/R1itzEmzrn1vtOf+8YW1NJbb6fIGJsyaT2dIHqCpqSnNVquDcVIHHb2MkxoYp8zRJrhnq8490NcH4TDOGTPS6rXDSM99VpUzvtLcC+MMzfuDYV7dFamHH68ELobb7U632cpgnNRBRy/jpAbGKXO0Ce7ZIjwcCdQWR+oM9mRCYcm+aHCfWenkxGjAHm9o/o09A3gDYRbVuZhROf6QPIAQYipNVwLjpA46ehknNTBOmaNNcM9WnXs8uDsnDrwx2gb8hCQ0lNlx2ix8pKGUpnLHuEPzL+6I1MFP1msHqK2tnULL1cA4qYOOXsZJDYxT5mgT3LNV5x72+4H0e+6xMrhZVZGbASFEPHA/v3300Lw/FOaVaJb8iWkEdzM0pQY6OoGeXsZJDYxT5mgT3LNV5x7ruYs0e+57osl0sytL4s/Fh+Z39owamn9zzwAef4j9al3MqiphMiorK9NutyoYJ3XQ0cs4qYFxyhxtgnu2mGrPfW9svr1q5GYgNjTf7Q2yKWFoPp4ln2Ztey63sS0UxkkddPQyTmpgnDJHm+CerX84GQvuafbcYwvYzEpIjkscmo/t1x4IhXk5umtcOkPyAB5P+pvQqIJxUgcdvYyTGhinzNEmuGerzn2q2fKJZXCJnLTf6KH5DXsHGfSHmF9TwpzqyYfkAZqbm9NttjIYJ3XQ0cs4qYFxyhxtgnvWE+rS6LkHQmE6Bv1YBMyoGH0zsH/96KH5qQ7JA7S1tU2h5WpgnNRBRy/jpAbGKXO0Ce7ZqiGcSs9934CfsISmcgd26+h/SiFEfPj92W09rG2JBPd0h+Qhe6MRxYRxUgcdvYyTGhinzNEmuGctW34KPfdYpnzykHyME6ND80+838nAcIi51SXMq3Gl3Zaqqqq0z1UF46QOOnoZJzUwTpmjTXDP1vKzU+m57+mL1riPs9JcbGg+HK2Gm8qQPEBnZ+eUzlcB46QOOnoZJzUwTpmjTXDPes89neCesOxsKhKH5mFqQ/Jg7l5VQUcn0NPLOKmBccocbYK7nGCL1akwlUVsxsuUT+RjCyMbycyrKWF+TXpZ8jH80RsNnTBO6qCjl3FSA+OUOdlZkL0ICIfD2bnOFHrusRr32ROsNreovpT/+tRiGsvtU0768/l8UzpfBYyTOujoZZzUwDhljjY991SZiL6WFjZcfjk9L7+c9nXS3ThmKBim0xPAKiLZ8hNx4IxymivSWxQnEVPrqQY6OoGeXsZJDYxT5mgT3FPVufe8+iqDmzfT+cwzaV8n3RXqYtu8zqh0YrXkZis/U+upBjo6gZ5exkkNjFPmaBPcLZaxKuGhSDZ7OM25jp09Pt7c2RW53iTD8ntSLDubbRxprpKnEsZJHXT0Mk5qYJwyR+/gHh1ijwX5yVjX2o/PGzl30uCeYsOYbFNRUZGzaxcK46QOOnoZJzUwTpmTt+AuhDhFCPG+EGKbEOL6FMfnCiGeFUK8KYR4WwhxWsKx70Rf974Q4uRU109V5x4P7tHfkxGUElsoMrw/2bD87miN++wc9ty7urpydu1CYZzUQUcv46QGxilz8hLchRBW4A7gVGAZcLEQYlnSaf8BPCSlPAS4CPjv6GuXRR8vB04B/jt6vVHYbGMT/ycK7i09Pp58r3NUCV0wDPZYcE+z5z5RGVym1NTU5OzahcI4qYOOXsZJDYxT5uSr534ksE1KuV1K6QceAM5MOkcCsd3sq4C90b/PBB6QUg5LKXcA26LXG0WqUrj4nHuK4P7zta387KVWtnaOlCeEwhJrML2e+974nPvUatengikHUQMdnUBPL+OkBsYpc/IV3GcBrQmPd0efS2Q1cKkQYjfwBHDtFF6bMriHYj33pIQ6KSXbuyL/0B7/yD7wwfDIsLzfMv6Kd15/iG5fELtV0FCeu80AhtLMFVAJ46QOOnoZJzUwTplTTIvYXAzcK6X8qRDiGOA+IcSKdF/c19fHcccdh81mIxQKcc4553By9E4p4PHQ09NDf38/DQ0NfLDXjTcQuRnY29bG3JJI8O/p7aMxGty3tXWzYrYXt9tNZWUloVAIj8dDc3Mz67bsBqC5zE7rrl1UVVXh9/vx+Xw0NzfT1taGw+GgoqKCrq4uampq8Pl8DA0NxY+XlJTgcrno6emhrq6OgYEB/H5//LjL5aK0tJSWlhbq6+vp6+sjEAjEj5eVlWG1WuNO3d3dSClpaGigvb2d8vJyAAYHB2lqasLtdiOEoLa2NqVTW1sbdrudqqoqOjs7jdOH3MnhcMTf1zgZJ+NUnE4TIbK1bOuEbxIJ1qullCdHH38HQEr544RzNgGnSClbo4+3A0cDn088VwjxVPRaryS+x3PPPScPOuigUe+76brr6HnpJZxNTRzx+OPx59/cM8C3n9wGwA8+uR/HzIus+fvrV3Yz/9ufocLbh+ueP3LYgQtS+jz3QQ//37M7OXZeFas/ud90/1kmpaWlhXnz5uXs+oXAOKmDjl7GSQ2MU3qsX7/+jZUrVx6e6li+huVfBxYLIRYIIRxEEuQeTTpnF7ASQAixFCgB3NHzLhJCOIUQC4DFwGvJbzBRnXsoac69pXdkeCQ0KqFOYovOuXcFx1+YJp5Ml8NMeYCSktzN5xcK46QOOnoZJzUwTpmTl2F5KWVQCHEN8BRgBe6RUm4SQtwErJNSPgp8HfgfIcTXiCTXXSEjwwqbhBAPAZuBIHC1lDKU/B4T1rknBfddicE9nBTco8PyXYE0gnsOM+UBXK70935XBeOkDjp6GSc1ME6Zk7c5dynlE0QS5RKfuzHh783AceO89kfAjya6/oR17kkJdbt6RoJ7MCG4h8PheCmce4LS+Mn2cc8WPT09VFZWTn6iQhgnddDRyzipgXHKHG1WqEtZ5x7LTgyFCCcE//F67qHo+vRBiw23b+zNQoz40rM57rnX1dXl9PqFwDipg45exkkNjFPmaBPcU9a5JwzHxwJ9/1CQ3qGRwD2q5x49P2iz4/aM3YgGYGA4SP9wCKfNQl1p7srgAAYGBnJ6/UJgnNRBRy/jpAbGKXO0Du6JiXSxwJ3Ya4fRwZ3o8H3QaqPTk3qzmcQNY6a6P/tU8ae54Y1KGCd10NHLOKmBccocbYJ7qv3cwymCe0tScE8clpfx4G6nfzjEcHDsDUO+kunA7GmsCjo6gZ5exkkNjFPmaBPck/dzl1KODu7RwB3ruce2YB8vuAN0phiaz8dWrzHMnsZqoKMT6OllnNTAOGWONsE9uRROBoOQMFQfm3OPZcrHgnNQpgjutlhwHzuMks+euykHUQMdnUBPL+OkBsYpc7QJ7snz38m17clz7vvVRv6hE0feRSAazO2RHeFSJdXls+fumGRnOhUxTuqgo5dxUgPjlDnaBPdQaPS6NuGkRfrDw8N4/SHcngB2i2B2dWS1oFHD8tHgHtsRzu0Zu+FMPnvufX19OX+PfGOc1EFHL+OkBsYpc9IK7kKIgyY/q7Ak17mn6rm3RhefmV3lxGGN9PRTZcvbosE9ec69dyiIxx+i1G6huiT36//U19fn/D3yjXFSBx29jJMaGKfMSbfn/g8hxFtCiG8IIWbktEXTZEzPPUVwb4nOt8+tLsEazahL7LnHhuUdrtTBfW/C4jW5LoMDc/eqCjo6gZ5exkkNjFPmpBvcZwA3AkcBW4UQa4QQlwohSnPXtKmRvLtdKMWwfGt0vn1uTQm2VME92nN3uiJD9snD8vnaMCZGcgWADhgnddDRyzipgXHKnLSCu5QyKKX8q5TyfGAW8BDwLaBdCPE7IUTKNeHzSXKde8qee29Cz12kGJaP9txLSiPBPbnn3jYQOT4jT8Hd1HqqgY5OoKeXcVID45Q5U0qoE0KUA2cR2bJ1NvAAsBX4XyHEHdlvXvok3xUlB/fQ8DC7eiPPJQ7LJwZ3SzASvF2lJVhEZI7dHxpJp+/2Rd4j18vOxjC1nmqgoxPo6WWc1MA4ZU66CXWfEkI8AOwBLgTuAmZKKa+UUv4QOBT4TO6aOTlWq3XU4+Rs+YBviLaBYSwiMmeeclg+eoNgdTriAbzLO3LT0B39uzZPwb2srCwv75NPjJM66OhlnNTAOGVOuj33W4A3gCVSytOklA9IKePRU0rZDVyXiwZOl+See2+/l7CEmZVOHFZLPLgn9txFMBbcnTSURWoSE4fmu72RDWdqXfkJ7sk3LDpgnNRBRy/jpAbGKXPSnXM/QEr5EynlvgnOuSt7zZo6k2XL9/V7AJgTrW+PZ8snJOJZonPuVmcJ9WWRAO4eHEmqG+m5574MDqC/vz8v75NPjJM66OhlnNTAOGVOusPyfxZCnJD03AlCiD/lpllTZ7KEusEBLwDzosE91bC8JTgyLN9QNnp9+bCU9ETn3PPVc29oaMjL++QT46QOOnoZJzUwTpmT7rD8ScDLSc+9Anwsu82ZPsFgcNTj2Hav1tJItZ5n0AdEkukArFHzYIo6d5vTSX3Z6CVo+4aChCRUOK04bPlZ2K+7uzsv75NPjJM66OhlnNTAOGVOulFqCEjOBigHirYYMZZQZ6usBMDnjQb3mtE998Tgbo313EucCT33SMCPD8nnqdcOY2v3dcA4qYOOXsZJDYxT5qQb3J8CfiOEqASI/v4V8PdcNWyqjLf8bCy4B3yRx3Oia8LH6txDo4L72J57ZzSox5Pp8jTfDmZoShV0dAI9vYyTGhinzEk3uH8dqAS6hRAdQDdQRRFlyI9X526rqgIigbup3IHLHslYHOm5R8+XcqTn7nSMJNTFeu6+/JbBAbS3t+ftvfKFcVIHHb2MkxoYp8xJqxsqpewBPhVdV3420CqlLKpVBsbUuceCe0VF5HfQz5zqkZXlkhPqgmGJLRTpnVudTqpK7VgE9HiDBMOyIMPy5eXleXuvfGGc1EFHL+OkBsYpc6Y0xiyl3CeEaAOEEMISfS48ycsKQnLP3R4KxDPlYWwpXCgssYUiAdzijCxyU+Oy0+UN0OUJJAzL5y+4GwwGg8EwHdIthZsphPiLEKILCBJJpIv9FAXj1bnbo3PutqA/nikPjFl+dlRwd0Tm2xOT6goxLD84OJi398oXxkkddPQyTmpgnDIn3Tn33wB+YCUwSGS52UeBL+eoXVMmuc49lJQtbwsF45nyMDZbPhiW2IIjPXdgVDlcbFi+Lo8JdU1NTXl7r3xhnNRBRy/jpAbGKXPSDe7HAp+TUm4ApJTyLeDzRBLtioLkOvdYz906Sc89Nucekowaloeknnue15UHcLvdeXuvfGGc1EFHL+OkBsYpc9IN7iEiw/EAvUKIBsBDZPvXoiRW5+6xuwBwhAJUOEd63ck991TD8iMZ84GCJNSJaLmeThgnddDRyzipgXHKnHSD+6vAadG/nwIeBP4MrMtFo6bDeHXunZZIb90RGp0ekHJYPpotL5KG5Vt6hxgOSUpsFkod+Vv8v7a2Nm/vlS+Mkzro6GWc1MA4ZU66wf0y4Pno39cB/wQ2ApfkolHTYbw697ZwdNGaoH/U8diwfHjUnHvknOSEuq2dkXXp87mADZihKVXQ0Qn09DJOamCcMmfSaCWEsAI/B74IIKX0ATfnuF1TZrw69z0BK7XCgiUcRgaDiGgP3xodIUlnWH5gOJKJn88heYDKaL6AThgnddDRyzipgXHKnEl77lLKELAKKMp69vGIBfddPgjYIkE5lLBTXOo598iwfCyhrq7UTuIsSb5r3JPL+3TAOKmDjl7GSQ2MU+akOyz/M+AHQoiiXcFlvDr3Fk+YoNUx6jkYmy0fCASwhkNIRLx3b7daqHGNDG7kO7h7PJ68vl8+ME7qoKOXcVID45Q56U4iXws0A/8uhHAD8d1WpJRzc9GwqTJmP/dotnxv2Eoo2nMPT9RzH47Mt4ds9lFZjfVlDrp9+d80BqC5uTmv75cPjJM66OhlnNTAOGVOuj33S4FPACdH/74s4acoSEyok8EgMhQCi4WwxRrPfk8V3EMyshVfMHosbBt9kxCbd4f8z7m3tRXV8v1ZwTipg45exkkNjFPmpLtxzPOTn1VYEnvbsbl1aXeAEFhTBHchBBYBYRn5iQX3kN0x6roNicE9z8PyyaMROmCc1EFHL+OkBsYpc9IK7kKIm8Y7JqW8Mc1rnEIk694K3CWlvCXp+M+Aj0UflgKNUsrq6LFbgU8RGWl4Gvg3KaVMfH1itnwsiFtKnFx+aDNl/3QhGR3cIdJ794ckwbAkNBQZlh/bcx8J9nV5Du5V0U1vdMI4qYOOXsZJDYxT5qQ7LD8n6ecI4BvAwnReHC2nuwM4FVgGXCyEWJZ4jpTya1LKg6WUBwO/JLJIDkKIY4HjgAOBFdH3Pin5PRKXn40FcUdJCZceOoPKyrJRz8dI3DwmNucuk3ru9QXsuXd2dub1/fKBcVIHHb2MkxoYp8xJn9iZoAAAIABJREFUd1j+s8nPRXviF6f5PkcC26SU26OvfQA4E9g8zvkXA9+PvT1QAjgAAdiBMbvej+q5R5PpYiVtsbr1VD13iGTMh4cjr0nuuceG5W0WQaUzf6vTgbl7VQUdnUBPL+OkBsYpc9LtuadiDXBWmufOAloTHu9mnHXphRDzgAVEVsFDSvkK8CywL/rzlJTy3eTXJY7SjwzLR5aejQX5WNCPYRUjPffgOD335orIaxvLHXlfG9jv909+kmIYJ3XQ0cs4qYFxypx059z3S3qqlMjSs60pTs+Ui4A/RRfPQQixCFgKzI4ef1oIcYKU8sXEF3V1dXHcccdhs9mYHQhwFYDdTktLS3zT+Y69eyn1eunu7kZKiTV6a9PZ1Y2nt5cqIj331tZWhBDU1tbi63Jz7RH11JVYaGlpobm5mba2Nux2O1VVVXR2dlJVVYXf78fn88WPOxwOKioq6OrqoqamBp/Px9DQUPx4SUkJLpeLnp4e6urqGBgYwO/3x4/Hjvl8Purr6+nr6yMQCMSPl5WVYbVa6e/vp6GhIe7U0NBAe3s75eXlQGQP4aamJtxud9zJ7XZTWVlJKBTC4/EYJ+M0xsnhcMTbb5yMk3EqTqeJEEl5aalPEiJMZHg81nX1Am8C10kp30jj9ccAq6WUJ0cffwdASvnjFOe+CVwtpXw5+vibQImU8ofRxzcCQ1LKWxNft3btWrlsWWQav/fVV9l49dVUHX44B9x5J1tvvpn2Rx5h0Q030Hz22fHXXPbAJtoH/fz2wmVsevwflPzX9+lZeiin3/d/J/03yQfDw8M4o6MOumCc1EFHL+OkBsYpPdavX//GypUrD091LK1heSmlRUppjf62SCnLpZQnpBPYo7wOLBZCLBBCOIj0zh9NPkkIsQSoAV5JeHoXcJIQwhZdIe8kYMywfGKde6wULj7nnqIUDpLm3KNDJtI2eli+kJhaTzXQ0Qn09DJOamCcMiet4C6EOFgIMSfpuTlCiIPSeb2UMghcQ2S72HeBh6SUm4QQNwkhzkg49SLggaQytz8BHwDvAG8Bb0kp/zZGxDKiEk4O7rGEuuQ594Rs+XB0zh1H8QR3RxG1JVsYJ3XQ0cs4qYFxypx011P9PXBG0nMO4D4iJWqTIqV8Angi6bkbkx6vTvG6EPClya4/YXCPJtaFxvTcI79DYYn0Jyx8UyRUVFQUuglZxzipg45exkkNjFPmpJstPzdWxhZDSvkBMD/rLZomo+rcoz10a1K2vPSn3tM9GJYjx4rojrGrq6vQTcg6xkkddPQyTmpgnDIn3eC+WwhxaOIT0cd7s9+k6WGzjQxCjFcKN7bnnjAsHwvuRdRzr6mpKXQTso5xUgcdvYyTGhinzJnKlq9/FUJcK4Q4TQhxLfAX4LbcNW1qhMMj280nD8unWlseErd9JT4sL4qo5+7z+QrdhKxjnNRBRy/jpAbGKXPSXaHuf4QQvcDniSw/2wp8XUr5p1w2bipMFNzFOAl1idnyRHvuxRTch5LaqwPGSR109DJOamCcMiftDcqllH8E/pjDtmRE4o47Y3ru0eH5MT13kWLOvYiG5c2exmqgoxPo6WWc1MA4ZU66pXC/iG7gkvjcsUKI23PTrKmTWOc+Jls+NiyflFCXOOdOoPh67qbWUw10dAI9vYyTGhinzEl3zv1iYF3Sc28QWYK2KEgshQslbxwz6Zz7yLC8pYiCe0l0xEEnjJM66OhlnNTAOGVOusFdpjjXOoXX55xRde6x4J68ccwE2fJEe/7CUTxLHrpcrkI3IesYJ3XQ0cs4qYFxypx0g/OLwM1CCAtA9PcPos8XBan2cx/Tcx9nhbqQlIjosLylpHiCe09PT6GbkHWMkzro6GWc1MA4ZU66CXX/BjwG7BNCtADziNS4n56rhk2VVHXu1imsLV+Mc+51dXWFbkLWMU7qoKOXcVID45Q56W4csxs4FDgT+AlwPpE91l/LXdOmxkSlcOMl1CVmy8d67lZn8QT3gYGBQjch6xgnddDRyzipgXHKnLRL4YA64CjgCiLryb9IpEdfFKQV3CeYc7fGgnsRzbn7k25GdMA4qYOOXsZJDYxT5kwY3KNbrJ5BJKCfDGwD/gDMBS6QUnbkuoHpkrLOfZKEusRseXs0oc5SRD13U+upBjo6gZ5exkkNjFPmTDYs3w78BngfOFpKuUxK+UOg6G6r0trPfcwKdZHficPysXOLAVPrqQY6OoGeXsZJDYxT5kwW3N8GqokMxx8hhCja1fwnKoUTNhtYLMhQCJmQVW9LyJa3BCM3B7YiCu6mHEQNdHQCPb2MkxoYp8yZMLhLKT8KLATWAN8A2oQQfwPKAPsEL807IpocB2Oz5YUQKZPqRrZ8JR7ciymhzlFEmfvZwjipg45exkkNjFPmTJotL6VskVL+UEq5GFgJ7APCwFtCiFtz3cB0CYVC8b+TE+oS/06cd0+cc7cEY9nyxdNz7+vrK3QTso5xUgcdvYyTGhinzJnSCnNSypeklF8EmoFrgQNy0qppEKtzl6EQMhAAIUbVrKfa0z0xW94SnbO3FdEiNvX19YVuQtYxTuqgo5dxUgPjlDnTWj5WSjkkpfyDlPLUbDdousR67uGENeITh+pTJdXFe+6hMNYinHM3d69qoKMT6OllnNTAOGVO0awNnylSSmBsGVyMVMPy8YS6QACBJGSxjlrprtAkVgDognFSBx29jJMaGKfMKZ5IliGxOndbRQWHP/roqKx4GJlLT0yoiwX3WMAPWu1Yi+h2x9R6qoGOTqCnl3FSA+OUOUUUyjIjdlckrFZKZs7ENXfuqOMTJdTFAn7Qao8H/GLA1HqqgY5OoKeXcVID45Q52gR3q9U64fGUc+7ROXkZ67nbiiu4l5WVFboJWcc4qYOOXsZJDYxT5mgT3Cdjojl3Ge+527AUUXCf7IZFRYyTOujoZZzUwDhljjbBPbHOPRUTDcvLIh2W7+/vL3QTso5xUgcdvYyTGhinzNEmuCduHJOKVCvUje2527GJ4gnuDQ0NhW5C1jFO6qCjl3FSA+OUOdoE92BSdnwyqebc47306KYxQZs93psvBrq7uwvdhKxjnNRBRy/jpAbGKXO0Ce6TkXpYPvJbBkZ67sUU3GO1+zphnNRBRy/jpAbGKXO0Ce6TLT6TMrjHhuD9Iz33YppzN0NTaqCjE+jpZZzUwDhljjbBfbLVfybKlqdIE+ra29sL3YSsY5zUQUcv46QGxilztAnuade5p0ioIzCyQl0RxXbKy8sL3YSsY5zUQUcv46QGxilztAnukzHRxjHSH+n1h2z2UZvNGAwGg8GgItoE93Tr3FNt+Sr9kefCtonL6fLN4OBgoZuQdYyTOujoZZzUwDhljjbBPe069wkWsSm24N7U1FToJmQd46QOOnoZJzUwTpmTt+AuhDhFCPG+EGKbEOL6FMd/JoTYEP3ZIoToTTg2VwixRgjxrhBisxBifvLr065zTznnHgvujqmL5RC3213oJmQd46QOOnoZJzUwTpmTly1fhRBW4A7gk8Bu4HUhxKNSys2xc6SUX0s4/1rgkIRL/A74kZTyaSFEORCeahusE/TcRSy4T9L7zzc6zv8bJ3XQ0cs4qYFxypx89dyPBLZJKbdLKf3AA8CZE5x/MfAHACHEMsAm///2zj08zqu+85+vR5KtKLYiS8LGSdY4XBtSCBTSpIGWxVzCpcnW29Kk3MJtu2AupQQKpUAIC2wKlIVCYZ8thVIuIZsNqaHuE9JCIAUCIQ6EmJDECTF2EtmyZMu2IlmW/Ns/3jPy68nYmtE71rzn5Pd5Hj2a9zbz+8yR5jfnvOdidh2Ame03swdrL2h4nHvdGeqyDnWHOstVc1++fHm7Q2g57hQPKXq5Uxy4U3EWKrmfDGzLbW8P+x6CpNXAGuDbYdfjgD2SrpZ0i6SPhJaAI5jPOPdqzb1jJrvWSpbcvWkqDlJ0gjS93CkO3Kk4C9Is3yQXAleZWbX7ewfwTLJm+l8DXwMuBj6Xv2jPnj2ce+65dHR0MDMzw7p161i/fj1DQ0P09PRwsLpm+8QE27dvx8xYvLQPgM7pLLlPL1rE5OQkw8PDSGL58uUMDw+zbNkyZmZmGB8fZ+XKlQwNDdHZ2Ulvby+7du2it7eXqakpJiYmZo93dXWxdOlSRkZG6OvrY2JigsnJydnjS5Ysobu7m927d9Pf38++ffuYmpqaPd7d3U2lUmHr1q0MDAwwNjbGwYMHZ4/39PRQqVTYu3cvg4ODjI6OYmYMDg6yY8eO2TGV+/fvZ8WKFe7kTk05dXV1MTk5yfj4uDu5kzuV1OlYaCHmu5V0DnCpmT0/bL8LwMw+XOfcW4D1ZvaDsH02cLmZ/V7Yfjlwtpmtz193ww032BlnnHHUGCbvu4+fXHABi1et4ukbNgAwPjXDH3zxVv7rdf+bJ979E757/ht493tf3QrllrBr1y4GBgbaHUZLcad4SNHLneLAnRpj06ZNN69du/Zp9Y4tVLP8TcBjJa2R1EVWO99Qe5KkJwB9wA9rrj1JUnVi3mcDv6i9tsh67mVtlh8fH293CC3HneIhRS93igN3Ks6CJHczmwbeCFwL3A5caWabJV0m6fzcqRcCV1iuOSE0z18C/LuknwMC/k/ta8w5zn3JEqBmhrrQn64jNMvTVa7kvnLlynaH0HLcKR5S9HKnOHCn4izYOHcz22hmjzOzR5vZB8O+95rZhtw5l5rZQ8bAm9l1ZvYkM/tNM7s49Lg/gjk71IXEXW/hmI6ZMEa+ZEPhhoaG2h1Cy3GneEjRy53iwJ2Kk8wMdXONIVRnJ0jY9DQWmvAlsUjlbZafqzUiRtwpHlL0cqc4cKfiJJPc51oVTtJRZ6mrhOSukjXL9/b2tjuEluNO8ZCilzvFgTsVJ5nkPtf0s3D0Nd1n77mXrOa+a9eudofQctwpHlL0cqc4cKfiJJPc56q5w1E61S3SbLO8uhYfn+DmiX97jYMUnSBNL3eKA3cqTjLJvZHx+vU61VWk2UlsytYsPzX1kH6D0eNO8ZCilzvFgTsVJ5nkfujQ3GvJHG1N946S3nOfmJhodwgtx53iIUUvd4oDdypOMsm9kZ6I1eRuuW9QRzTLLy5Xs7yP9YyDFJ0gTS93igN3Kk4yyX2uce6Qq7nn7rl3YlQOZUPjFpVs+IWP9YyDFJ0gTS93igN3Kk4yyX3RorlVKtUOdblm+S7LetkfrHTS0cBzLCRdJbtN0ArcKR5S9HKnOHCn4pQrmxWgkeSuOh3qug5lyX260nl4ffeSsHTp0naH0HLcKR5S9HKnOHCn4iST3BsZ516pM86961BY7rWjk0rJ3o2RkZF2h9By3CkeUvRypzhwp+KULJ3Nn46OuZemrzeJzeGae8fsKnFloa+vr90htBx3iocUvdwpDtypOMkk92aGwuWTe+dMtebeVbpmeR8OEgcpOkGaXu4UB+5UnIdXcq/Xoa7aLF/ppDLH4jMLzWSuV38quFM8pOjlTnHgTsVJJrk3NM69Xoe6mcMd6srWLO9jPeMgRSdI08ud4sCdipNMcm9mnPsRC8dUm+UrHaVrlvexnnGQohOk6eVOceBOxUkmuTcyFK5+ci/vULgl4TZCSrhTPKTo5U5x4E7Fedgn98Md6jpZVLLk3t3d3e4QWo47xUOKXu4UB+5UnGSSe0PrudfpUHe4Wb58Nffdu3e3O4SW407xkKKXO8WBOxUnmeTe0Dj3Oh3qOkuc3Pv7+9sdQstxp3hI0cud4sCdipNMcp/3kq8l7i2/b9++dofQctwpHlL0cqc4cKfiPCyTu+WSe2U6N/1suXI7U7mlaVPBneIhRS93igN3Kk4yyb2hce517rlXprM3vIzN8j7WMw5SdII0vdwpDtypOMkk90bGuVfqNcvnau5lS+4+1jMOUnSCNL3cKQ7cqTjJJPf5Lvk62yxfwnvuPhwkDlJ0gjS93CkO3Kk4ySR3NTAvfL0lXxfNNsuXb1W4rvBlJCXcKR5S9HKnOHCn4iST3GdmZuY8p+499xIPhRsbG2t3CC3HneIhRS93igN3Kk4yyX2+67kvOpjrLV+y5D4wMNDuEFqOO8VDil7uFAfuVJxkkntDNfc699wXTZd3yVf/9hoHKTpBml7uFAfuVJxkkruZzXmOurpAwg4exMKXgeo995kSNss3MgIgNtwpHlL0cqc4cKfiJJPcGxnnLulw7T1MKFBtlj9YwqFwPtYzDlJ0gjS93CkO3Kk4yST3Rr8V1Xaq08HDk9hUSvZu+FjPOEjRCdL0cqc4cKfilCydzZ9KpdLQebWd6o5M7uWquff09LQ7hJbjTvGQopc7xYE7FSeZ5N4otZ3qdLC8M9Q1+oUlJtwpHlL0cqc4cKfiJJPcG+ktD3Vq7tPlrbnv3bu33SG0HHeKhxS93CkO3Kk4C5bcJZ0n6Q5JWyS9s87xj0v6afi5U9KemuPLJG2X9Kl6z99IhzqoM9Z96vAMdWWruQ8ODrY7hJbjTvGQopc7xYE7FWdBkrukCvBp4AXA6cBFkk7Pn2NmbzWzM83sTOBvgatrnuYDwPeO9hrT09MNxXLMDnUlG+c+Ojra7hBajjvFQ4pe7hQH7lSchaq5nwVsMbN7zGwKuAK44BjnXwR8tboh6beAFcC3igbykJp7ie+5NzJ2PzbcKR5S9HKnOHCn4ixUcj8Z2Jbb3h72PQRJq4E1wLfD9iLgY8Alx3qBRqafhcMd6mYOHMje7BL3lvemqThI0QnS9HKnOHCn4jSWEReWC4GrzKzaQ+4NwEYz236sld927tzJ6173Ojo6OpiZmWHdunWsX7+eoaEhenp6qFQq7N27FwtfAnbdfz8njI/DoUMc0iJsUYX7t2/j1FUrGR4eRhLLly9neHiYZcuWMTMzw/j4OCtXrmRoaIjOzk56e3vZtWsXvb29TE1NMTExMXu8q6uLpUuXMjIyQl9fHxMTE0xOTs4eX7JkCd3d3ezevZv+/n727dvH1NTU7PHqsSVLljAwMMDY2BgHDx6cPZ53GhwcZHR0FDNjcHCQHTt2cOKJJwKwf/9+VqxY4U7u1JRTV1cXv/71r3n0ox/tTu7kTiV1OhZaiKYCSecAl5rZ88P2uwDM7MN1zr0FWG9mPwjbXwaeCRwCTgS6gL8zsyM65d1www12xhlnzBnLne9/Pzu/8Q0e81d/xcBznsONz3oWBzoXc/lrPsXGV59Zqqb5kZER+vv72x1GS3GneEjRy53iwJ0aY9OmTTevXbv2afWOLVTN/SbgsZLWAPeR1c7/pPYkSU8A+oAfVveZ2Utzxy8Gnlab2JuhUu1QNzU1OwXtdCXraV8pT153HMdxnHmzIPfczWwaeCNwLXA7cKWZbZZ0maTzc6deCFxh82hOmM8493xyryibe75M7N+/v90htBx3iocUvdwpDtypOAt2z93MNgIba/a9t2b70jme4wvAF+oda3ice26GOgs95svYmQ5gxYoV7Q6h5bhTPKTo5U5x4E7FSWaGuobHuderuZdwGBzA8PBwu0NoOe4UDyl6uVMcuFNxkknujTKb3CcnZ8e6T1c6SllzL9ttglbgTvGQopc7xYE7FSeZ5N7wOPc6HepmKuWsuc811CFG3CkeUvRypzhwp+Ikk9wbXs+9TrN8Z/cSzjp12XGLbb5401QcpOgEaXq5Uxy4U3HKOInNvGh4Pfdch7pqs/zpJ5/EH/3u6uMW23xZtqx8XziK4k7xkKKXO8WBOxUnmZp7o9S7514d+142Gh3eFxPuFA8perlTHLhTcZJJ7g2Pc8+tCldtlleozZeN8fHxdofQctwpHlL0cqc4cKfiJJPcm17PfWpqtuZe3Vc2Vq5c2e4QWo47xUOKXu4UB+5UnGSSe5EOdYtKWnMfGhpqdwgtx53iIUUvd4oDdypOMsm90TGE9WaoK2tyb7Q1IibcKR5S9HKnOHCn4iST3BvuLR/uuc/ka+4lbZbv7e1tdwgtx53iIUUvd4oDdypOMsm96elnJydL3yy/a9eudofQctwpHlL0cqc4cKfiJJPcG625V+p1qCtpcvdvr3GQohOk6eVOceBOxUkmuTe6SmzdDnUlbZafCvGlhDvFQ4pe7hQH7lScZJL7oUOHGjqvOqbdpqY4NDkJlDe5T0xMtDuEluNO8ZCilzvFgTsVJ5nk3mhPREmzyXx6716gvM3yPtYzDlJ0gjS93CkO3Kk4yST3Rse5w+Ga+vS+fQCopDV3H+sZByk6QZpe7hQH7lScZJL7okWNq9Qm97LW3LtKGlcR3CkeUvRypzhwp+J4cqe899yXLl3a7hBajjvFQ4pe7hQH7lScZJJ7o+Pc4XBNvez33EdGRtodQstxp3hI0cud4sCdipNMcu/oaHxp+tmV4aq95Uua3Pv6+todQstxp3hI0cud4sCdipNMcm90KBw8tBm+rM3yPhwkDlJ0gjS93CkO3Kk4ntwpb819MrQspIQ7xUOKXu4UB+5UnGSSezMr7sRSc/exnnGQohOk6eVOceBOxUkmuTc1zr2mpl7W5O5jPeMgRSdI08ud4sCdipNMcm9qKFzoUDe7XdJm+SU1caaAO8VDil7uFAfuVJyHZ3KvqamrpMm9u7u73SG0HHeKhxS93CkO3Kk4yST3psa555K7urqQdDxCKszu3bvbHULLcad4SNHLneLAnYqTTHJvZpx7JZfcy9okD9Df39/uEFqOO8VDil7uFAfuVJxkknszQ+HyzfBl7UwHsC9Mj5sS7hQPKXq5Uxy4U3Eelsk936GuzDX3qampdofQctwpHlL0cqc4cKfiJJPcmxnnHkuzvI/1jIMUnSBNL3eKA3cqTjLJfT7rudc+Lhs+1jMOUnSCNL3cKQ7cqTjJJPemhsJFcs/dh4PEQYpOkKaXO8WBOxUnmeTezHC2WO65d5U4tvniTvGQopc7xYE7FWfBkruk8yTdIWmLpHfWOf5xST8NP3dK2hP2nynph5I2S7pV0h/Xe/6ZmZmGY4mlWX5sbKzdIbQcd4qHFL3cKQ7cqTiNDw4vgKQK8GngucB24CZJG8zsF9VzzOytufPfBDwlbD4IvMLM7pK0CrhZ0rVmtif/Gk2t514ziU1ZGRgYaHcILced4iFFL3eKA3cqzkLV3M8CtpjZPWY2BVwBXHCM8y8CvgpgZnea2V3h8f3ATmCw9oJ519xLnNz922scpOgEaXq5Uxy4U3EWKrmfDGzLbW8P+x6CpNXAGuDbdY6dBXQBd9ceM7OGg4mlWb6ZEQCx4E7xkKKXO8WBOxVnQZrlm+RC4CozO6IqLumRwD8BrzSzh8xYMzY2xrnnnktHRwczMzOsW7eO9evXMzQ0RE9PD5VKhb179zI4OMjO3By/4wcOMDIyAsD+/ftZsWIFw8PDSGL58uUMDw+zbNkyZmZmGB8fZ+XKlQwNDdHZ2Ulvby+7du2it7eXqakpJiYmZo93dXWxdOlSRkZG6OvrY2JigsnJydnjS5Ysobu7m927d9Pf38++ffuYmpqaPd7d3c0JJ5zA1q1bGRgYYGxsjIMHD84er3UaHR3FzBgcHGTHjh2ceOKJ7uRO83bq6uqafV13cid3KqfTsVAzNd75Iukc4FIze37YfheAmX24zrm3AOvN7Ae5fcuA64EPmdlV9V7j+uuvtyc/+ckNxXNg505ueuELAVh10UWc9ra3NeWzUGzdupXVq1e3O4yW4k7xkKKXO8WBOzXGpk2bbl67du3T6h1bqGb5m4DHSlojqYusdr6h9iRJTwD6gB/m9nUBXwe+eLTEDlCpVBoOJpZm+Z6ennaH0HLcKR5S9HKnOHCn4ixIcjezaeCNwLXA7cCVZrZZ0mWSzs+deiFwhR3ZnPAS4HeBi3ND5c4sEk8syb2ZLyyx4E7xkKKXO8WBOxVnwe65m9lGYGPNvvfWbF9a57ovAV+a6/mb6i2fn6GuxL3l9+7dS19fX7vDaCnuFA8perlTHLhTcZKZoa6ZhWO0aNHs+PYy19wHBx8y4i963CkeUvRypzhwp+Ikk9ynp6ebOr+a1Mtccx8dHW13CC3HneIhRS93igN3Kk4yyb1Zqsu+lnmGuoUYybDQuFM8pOjlTnHgTsVJJrk3M/0s5Gru3iy/oLhTPKTo5U5x4E7FSSa5Nzv7TwzN8jt27Gh3CC3HneIhRS93igN3Kk4yyb3ZYQYx1NyrMxOlhDvFQ4pe7hQH7lScZJJ7s3SGIQmdJ53U5kgcx3Ecp7Ukk9ybGecOcNrb387jP/hBeh7/+OMUUXH279/f7hBajjvFQ4pe7hQH7lScMi4cMy+aGecO0H3qqXSfeupxiqY1rFixot0htBx3iocUvdwpDtypOMnU3Jsd5x4Dw8PD7Q6h5bhTPKTo5U5x4E7FSSa5p4ikdofQctwpHlL0cqc4cKfiJJPcmx3nHgNzrdcbI+4UDyl6uVMcuFNxkknuzY5zjwFvmoqDFJ0gTS93igN3Kk4yyT3FJQKXLVvW7hBajjvFQ4pe7hQH7lScZJJ7ijQ7vC8G3CkeUvRypzhwp+Ikk9xT/GMYHx9vdwgtx53iIUUvd4oDdypOMsm92XHuMbBy5cp2h9By3CkeUvRypzhwp+Ikk9xT7FA3NDTU7hBajjvFQ4pe7hQH7lScZJL7nj172h1Cy7nmmmvaHULLcad4SNHLneLAnYrjyb3EXH311e0OoeW4Uzyk6OVOceBOxUkmuZtZu0NoOSlOqetO8ZCilzvFgTsVR6kkxY0bN+5bvHjxHe2Oo5WMjo4OLF++fFe742gl7hQPKXq5Uxy4U8OsXrt27WC9A8kkd8dxHMdxMpJplnccx3EcJ8OTu+M4juMkRvTJXdJ5ku6QtEXSO9sdT6NIOlXSdyT9QtJmSW8J+5dLuk7SXeF3X9gvSZ8MnrdKemp7DY6OpIqkWyR9M2yvkfSjEPvXJHWF/YvD9pZw/FHtjPtYSDpJ0lWSfinpdknnxF5Wkt4a/vZuk/RVSUun+caTAAALXElEQVRiKytJ/yBpp6TbcvuaLhdJrwzn3yXple1wycVSz+kj4W/vVklfl3RS7ti7gtMdkp6f21+qz8Z6Xrljb5NkkgbCdrRlFfa/KZTXZkl/ndu/cGVlZtH+ABXgbuA0oAv4GXB6u+NqMPZHAk8Nj5cCdwKnA38NvDPsfydweXj8QuBfAQFnAz9qt8Mx3P4c+ArwzbB9JXBhePxZ4PXh8RuAz4bHFwJfa3fsx3D6R+C14XEXcFLMZQWcDPwK6M6V0cWxlRXwu8BTgdty+5oqF2A5cE/43Rce95XM6XlAR3h8ec7p9PC5txhYEz4PK2X8bKznFfafClwLbAUGEiir/wz8G7A4bD+iHWUVe839LGCLmd1jZlPAFcAFbY6pIczsATPbFB7vA24n+8C9gCyREH7/l/D4AuCLlnEjcJKkRy5w2HMi6RTgRcDfh20BzwauCqfUOlVdrwLWhvNLhaResn/izwGY2ZSZ7SHysgI6gG5JHcAJwANEVlZm9j1gtGZ3s+XyfOA6Mxs1s93AdcB5xz/6+tRzMrNvmVl1LNWNwCnh8QXAFWZ2wMx+BWwh+1ws3WfjUcoK4OPAO4B87+5oywp4PfA/zexAOGdn2L+gZRV7cj8Z2Jbb3h72RUVo4nwK8CNghZk9EA4NASvC41hc/xfZP+qhsN0P7Ml9MOXjnnUKx8fC+WVjDTAMfF7Z7Ya/l9RDxGVlZvcBHwV+TZbUx4Cbib+soPlyKX151fBqslotRO4k6QLgPjP7Wc2hmL0eBzwz3L76rqSnh/0L6hR7co8eSScC/w/4MzPbmz9mWVtONGMVJb0Y2GlmN7c7lhbTQdb09hkzewowTtbcO0uEZdVHVjtYA6wCemhjDeh4EVu5zIWkdwPTwJfbHUtRJJ0A/CXw3nbH0mI6yG4bnA28HbiyHa1csSf3+8ju11Q5JeyLAkmdZIn9y2ZWnZtwR7UJN/yuNunE4HoucL6ke8malp4NfIKsSa0jnJOPe9YpHO8FRhYy4AbZDmw3sx+F7avIkn3MZfUc4FdmNmxmB4Grycov9rKC5sslhvJC0sXAi4GXhi8tELfTo8m+XP4sfGacAmyStJK4vbYDV4dbCj8ma8UcYIGdYk/uNwGPDT18u8g6+mxoc0wNEb7JfQ643cz+JndoA1DtAfpK4J9z+18RepGeDYzlmh5LgZm9y8xOMbNHkZXFt83spcB3gD8Mp9U6VV3/MJxfulqWmQ0B2yQ9PuxaC/yCiMuKrDn+bEknhL/FqlPUZRVotlyuBZ4nqS+0aDwv7CsNks4ju911vpk9mDu0AbhQ2WiGNcBjgR8TwWejmf3czB5hZo8KnxnbyToZDxFxWQHXkHWqQ9LjyDrJ7WKhy6poj7x2/5D1qryTrLfhu9sdTxNxP4OsufBW4Kfh54Vk9zH/HbiLrMfl8nC+gE8Hz58DT2u3wxx+z+Jwb/nTwh/xFuD/crgX6ZKwvSUcP63dcR/D50zgJ6G8riHrqRt1WQHvB34J3Ab8E1kv3qjKCvgqWZ+Bg2TJ4TXzKRey+9hbws+rSui0hey+bPWz4rO5898dnO4AXpDbX6rPxnpeNcfv5XBv+ZjLqgv4Uvi/2gQ8ux1l5dPPOo7jOE5ixN4s7ziO4zhODZ7cHcdxHCcxPLk7juM4TmJ4cnccx3GcxPDk7jiO4ziJ4cndcRYISV+Q9D/a9NqS9HlJuyX9+Di+zjMl3dHqc1NB0r2SntPuOJz08eTuPGwJH7Q7wzzx1X2vlXR9G8M6XjwDeC5wipmdlT8g6S8l7Q8/k5Jmctubm3kRM7vBzB4/95nNndsskq4PLvsl7ZJ0dUkX73Gc44Ind+fhTgV4S7uDaBZJlSYvWQ3ca2bjtQfM7ENmdqKZnQj8d+CH1W0ze2LuNSUpps+MNwanxwAnki2UsyDkpvB1nLYQ0z+q4xwPPgJcIumk2gOSHiXJ8h/UoUb42vD4Yknfl/RxSXsk3SPpd8L+baFV4JU1Tzsg6TpJ+5StGLU699xPCMdGJd0h6SW5Y1+Q9BlJGyWNE6a3rIl3laQN4fotkl4X9r+GbAnec0JN9v2NvjnB94OSvg88CJwm6VWSbg8O90j609z5z5K0Pbd9r6RLJN0qaUzS1yQtafbccPwdkh6QdH9oYTFJj5nLwbLlea8hm2Ww+lzHeq+7JX1M0tYQx39I6g7Hzpe0OZT39ZJ+oyb+v5B0KzAuqUPSy8PzjChb9CX/3p4l6SeS9kraISk/DbXjFMKTu/Nw5yfA9cAl87z+t8mmpO0HvkK2YM7TyWqLLwM+pWzlvyovBT5AtpDETwmre4VbA9eF53gE2fzSfyfp9Ny1fwJ8EFgK/EedWK4gmwJzFdn87x+S9Gwz+xxH1sjf16Tjy4H/Fl53K9lCLC8GlgGvAj4u6anHuP4lZCvOrQGeBFzc7LnK5lb/c7IFbx5DNr1xQ0jqB9aRTVfayHv9UeC3gN8hW93rHcAhZfOEfxX4M2AQ2Ah8Q9l84FUuAl4EnES29OdnyN6/VWR/I6fkzv0E8AkzW0a2iMqVjTo5zlx4cnecbMnJN0kanMe1vzKzz5vZDPA1stWdLjOzA2b2LWCKLBlV+Rcz+56ZHSCbZ/ocSaeSJct7w3NNm9ktZCsG/lHu2n82s++b2SEzm8wHEZ7jXOAvzGzSzH5KVlt/xTycavmCmW0OcR00s38xs7st47vAt4BnHuP6T5rZ/WY2CnyDXA26iXNfAnw+xPEgcGkDcX9S0hjZoh0DwJvC/qO+1+G2w6uBt5jZfWY2Y2Y/COX1x2Tld51lK+l9FOgm+xKQj3+bmU2QfcH6Zq6830O2QliVg8BjJA2Y2X4zu7EBJ8dpCE/uzsMeM7sN+CY1a7Q3yI7c44nwfLX78jX3bbnX3Q+MktXqVgO/HZp790jaQ1bLX1nv2jqsAkbNbF9u31bg5CZcjsYRryvpBZJuDE3ae8gWvRg4xvVDuccPcuT70ei5q2riONZ7UeXNZtZL1gLQx+Fa87He6wGyRXLurvN8q8jeUwDM7FCII/8eb6s5P1/e4xy5TO5ryGr3v5R0k6QXN+DkOA3hnT4cJ+N9ZCs4fSy3r9r57ARgb3icT7bzYXbd5tBcvxy4nywJfNfMnnuMa4+1ytP9wHJJS3MJ/j/RmrWuZ19X0mKyWu4ryFoSDkq6hmwVr+PJAxzZpH3q0U6sxcx+rmwI4qfD7YOjvteh5j5J1kz+s5rD9wO/mTtXIY78e5wvoweA/D35E8ia5qtx3QVcFF5zHXCVpP56nR4dp1m85u44gJltIWtWf3Nu3zDZB/fLJFUkvZrsQ78IL5T0jHCf9gPAjWa2jazl4HGhA1Zn+Hl6vsPWHPFvA34AfFjSEklPIqsZfqlgvLV0kS0NOwxMS3oB2Zrax5srgVdJ+o2QJN/T5PX/CKwAzucY73Wojf8D8DfKOihWJJ0TvtRcCbxI0lpJncDbgANk73s9rgJenCvvy8h95kp6maTB8Jp7wu5DdZ7HcZrGk7vjHOYyoKdm3+uAt5M1pz6Ro3+QN8pXyFoJRsk6bb0MINS2n0fWuet+subpy8kSaaNcBDwqXP914H1m9m8F4z2CEOebyRLdbrJOfhta+RpHed1/BT4JfIesY1z1/vSBBq+fIuvA9p4G3utLyNYQv4msnC4HFpnZHWTl9bdk9/F/H/j98Nz1XnMzsJ6szB8ge7+25045D9gsaX+I7cJwr95xCuPruTuOEx2hReM2YLGZTbc7HscpG15zdxwnCiT9gaTFkvrIatPf8MTuOPXx5O44Tiz8KdkY+7uBGeD17Q3HccqLN8s7juM4TmJ4zd1xHMdxEsOTu+M4juMkhid3x3Ecx0kMT+6O4ziOkxie3B3HcRwnMTy5O47jOE5i/H9v1dV+DwUoxQAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"source": [
"The stabilization at ~100 training records for TabPFN is nice to see, a regime in which it's beating this current XGB configuration. Would be interesting to do a parameter search for XGB at 100-200 training records."
],
"metadata": {
"id": "MXbxbMkJXnPC"
}
},
{
"cell_type": "markdown",
"source": [
"## Results:\n",
"\n",
"TabPFN exhibits accuracy within 1.5% of XGBoost using the same 1,024 training points. While this seems pretty close, XGBoost can achieve matching accuracy to theTabPFN using 51% fewer training samples (500 points). Leveraging the full dataset (32,561 training points), XGBoost achieves about 5% better accuracy than TabPFN. \n",
"\n",
"There is a small regime in which TabPFN might perform better (less than 200 records), however the comparsion is an unoptimized XGBoost, in fact, using HPs for a larger training set, so a more optimized XGBoost would be a better comparison."
],
"metadata": {
"id": "3sDObEXapjij"
}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment