Recently I have been posting videos of seminar talks and lectures online to a YouTube channel. The equipment and software that I use:
First you'll have to install the Git command line tool on your machine, following these instructions. Then find the repository that you want to contribute to, copy its address from the green "Clone or Download" button, and on your local machine run e.g.
git clone https://github.com/dmurfet/difflinearlogic.git
To see a list of what has changed (optional) run git status. Then
| General | |
| ======= | |
| https://www.youtube.com/watch?v=1X7Koxx4qJE (from 16:40 Hassabis on why DeepMind is unusual) | |
| https://www.youtube.com/watch?v=PW09L6-75ig (Shklyarov) | |
| Higher education | |
| ================ | |
| https://www.youtube.com/watch?v=yUGn5ZdrDoU (Clayton Christensen on disruption in higher ed) |
written in December 2018.
The practical development of deep learning and its associated infrastructure has initiated a broad re-examination of the practice of computer programming. In this document we briefly survey how this discussion has evolved over the past few years, and then describe our point of view on the underlying mathematics.
We begin with some appeals to authority, in the form of the following references:
As a PhD student you are optimising for a goal with a long time horizon (in the first case to complete a PhD, but then perhaps also to obtain a permanent research position, which could take much longer) and it is hard to determine the correlation between any given intermediate action and eventual success (whatever you define that to be, but two large components could be prove beautiful theorems and get a job). This brute fact lies at the root of much stress and uncertainty. How does one prove beautiful theorems? How does one get a job?
Well, who knows, but certainy not by trying to directly optimise for a goal with a decade long time horizon, and this degree of uncertainty! You have to develop shorter term proxy goals, and it seems to me that part of the job of a supervisor is to assist in that development. If you want to prove beautiful theorems and get a job, then since it is difficult to infer from first principles the algorithm for doing either of those things, a r
The rough area at the moment is moduli of A-infinity structures in geometry.
In early 2019 I decided to try to understand the University of Melbourne a little better. I have recorded some observations here in case they are useful for other academics. For updates in early 2020 see down the page. The notes are taken from various University of Melbourne (UoM) official documents, primarily
To a first approximation, if you want to understand the University I think you should read the report, ignore the glossy bits, and pay close attention to the statistics on p.13 and the financial data reported beginning on p.124. All references in this section are to the report, unless specified otherwise.
I am making publicly available my hand-written working notes for the paper "Constructing A-infinity categories of matrix factorisations" in the same spirit that I made available the other notes on my webpage The Rising Sea. Obviously you should not expect these notes to be as coherent, or readable, as the final paper, but those marked on the first page as (checked) are indeed checked, to the same level of rigour that I apply to any of my published papers. And they often contain more details than the paper. I hope you find them useful!
The main references, written in the same notation and from the same outlook as the final paper, are given below. You should probably start with (ainfmf28). Some of these PDF files are large, you have been warned.
The optimisation algorithm used in most of DeepMind's deep RL papers is RMSProp (e.g in the Mnih et al Atari paper, in the IMPALA paper, in the RL experiments of the PBT paper, in the Zambaldi et al paper). I have seen speculation online that this is because RMSProp may be well-suited to deep learning on non-stationary distributions. In this note I try to examine the RMSProp algorithm and specifically the significance of the epsilon hyperparameter. The references are
Often in the literature RMSProp is presented as a variation of AdaGrad (e.g. in the deep learning textbook and in Karpathy's class). However, I think this is misleading, and that the explanation in Hinton's lecture is (not surprisingl