amartya18x

Saisons de La Vigne Blanc (£7.25, classic, easy drinking Southern French style you're familiar with).
Blockhead Chenin Blanc (£7.25, South African, fresh and easy drinking).
Chilinero Sauvignon Blanc (£8.99, Herby, fresh and limey style).
Vitese Zibbibo (£9.99, Peach and jasmine - light and delicious: the perfect springtime white).
Terroir Picpoul de Pinet (£12.99, smooth, appley and long on the palate. Delightfun grape).
Monte Real White Rioja (£12.99, Creamy and oaky, rich and indulgent).

Saisons de la Vigne Rouge (£7.25, smooth and medium bodied - St Hugh's house wine).
Blockhead Shiraz (£7.25, fruity and slightly peppery, a perfect bbq red).

amartya18x

Understanding deep learning requires rethinking generalization (https://openreview.net/forum?id=Sy8gdB9xx)
Spectrally-normalized margin bounds for neural networks (https://papers.nips.cc/paper/7204-spectrally-normalized-margin-bounds-for-neural-networks.html)
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks (https://openreview.net/forum?id=Skz_WfbCZ)
Stronger Generalization Bounds for Deep Nets via a Compression Approach (http://proceedings.mlr.press/v80/arora18b.html)
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks (http://proceedings.mlr.press/v97/arora19a.html)
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian (https://arxiv.org/abs/1906.05392)
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks (https://arxiv.org/abs/1903.11680)

amartya18x

Theory

DL Theory

1. [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer NNs](http://proceedings.mlr.press/v97/arora19a.html]
   • Analysis by NTK and how training speed depends on the projection of y on the eigenvectors of the NTK.
   • Projections on top eigen values decrease faster than smaller eigenvalues.
   • Hence, loss on correct labels decrease faster than incorrect.
   • Gen bound depends on the NTK, which is data-dependant.
2. Gradient Descent Finds Global Minima of Deep Neural Networks

• Also, NTK based analysis

amartya18x

• Cubic regularization of Newton method and its global performance Yurii Nesterov, B. T. Polyak (https://link.springer.com/article/10.1007/s10107-006-0706-8)
• Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition Rong Ge, Furong Huang, Chi Jin, Yang Yuan (https://arxiv.org/abs/1503.02101)
• First-order Methods Almost Always Avoid Saddle Points Jason D. Lee, Ioannis Panageas, Georgios Piliouras, Max Simchowitz, Michael I. Jordan, Benjamin Recht (https://arxiv.org/abs/1710.07406)
• How to Escape Saddle Points Efficiently

amartya18x

amartya18x

• Terminal - https://thorsten-hans.com/setting-up-iterm2-with-oh-my-zsh-and-powerline-on-osx-c51bd149272f
• Then I will give you my emacs config
  • And you look at it and install everything one by one
• Set up your email client as well
  • You might want to look at Airmail3 if you want to purchase an email client
  • Or you can use "Spark" if you want a free one
• Also, maybe you can get itunes
• Setup the double finger right click from settings and the other touchpad gestures

amartya18x

%% (c) 2018 Amartya Sanyal (2018) University of Oxford
%% 
%% Draws upon style elements from
%% the style file for the course CS773 by Purusottam Kar
%% the style file for the course CS289 by Alexander Sherstov
%% the class file for the publication NOW FnT(R)-ML by Neal Parikh
%% the style file for the proceedings of the NIPS 2015 conference

%% By using this style file or taking parts of it, you agree to acknowledge the
%% author of this style file.

amartya18x

% Theorem Environments

\newtheorem{thm}{Theorem}
\newtheorem{lem}[thm]{Lemma}
\newtheorem{corollary}[thm]{Corollary}
\newtheorem{claim}[thm]{Claim}
\newtheorem{proposition}[thm]{Proposition}
\newtheorem{remark}{Remark}
\newtheorem{defn}{Definition}

amartya18x

local opt = lapp [[
Train a CNN classifier on CIFAR-10 using AllReduceSGD.
   --nodeIndex         (default 1)         node index
   --numNodes          (default 1)         num nodes spawned in parallel
]]
local torch = require 'torch'
local grad = require 'autograd'
local dataset = require 'dataset'
local optim = require 'autograd.optim'
local Coptim = require 'optim'

amartya18x

local torch = require 'torch'
local grad = require 'autograd'
require 'lua_data'
local optim = require 'optim'
local _ = require 'moses'
-- Helper function for broadcasting
local types = {'Byte', 'Char', 'Short', 'Int', 'Long', 'Float', 'Double'}
local Tensor = {}

function broadcastAs(tensor, template)