| name: tf27 | |
| channels: | |
| - pytorch | |
| - defaults | |
| dependencies: | |
| - python=2.7 | |
| - tensorflow-gpu=1.12.* | |
| - keras | |
| - numpy | |
| - pandas |
They basically all suggest that apparent improvements to the state of the art in ML and related fields are often not real, or at least the result of factors other than what the authors claim.
The state of sparsity in deep neural networks
What is the state of neural network pruning?
On the State of the Art of Evaluation in Neural Language Models
Do Transformer Modifications Transfer Across Implementations and Applications?
| import numpy as np | |
| np.random.seed(1234) | |
| class FullyConnectedLayer(object): | |
| def __init__(self, num_inputs, num_outputs): | |
| pass |
Revisiting ResNets: Improved Training and Scaling Strategies
Bello, Irwan, William Fedus, Xianzhi Du, Ekin D. Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, and Barret Zoph. "Revisiting resnets: Improved training and scaling strategies." arXiv preprint arXiv:2103.07579 (2021).
@article{bello2021revisiting,
title={Revisiting resnets: Improved training and scaling strategies},
author={Bello, Irwan and Fedus, William and Du, Xianzhi and Cubuk, Ekin D and Srinivas, Aravind and Lin, Tsung-Yi and Shlens, Jonathon and Zoph, Barret},
journal={arXiv preprint arXiv:2103.07579},
| from typing import * # for convenience; also, this is valid for typing | |
| import warnings | |
| import torch | |
| import torch.nn as nn | |
| import torch.nn.functional as F | |
| from torch import Tensor # shorten type signatures | |
| # def try_compile(f: Callable): | |
| # def try_compile(): |