dmurfet

Deep Learning in Australia

This document has moved to its own page: antipode.ai and is no longer being updated on Github gists.

dmurfet

Optimisation algorithms for deep RL

The optimisation algorithm used in most of DeepMind's deep RL papers is RMSProp (e.g in the Mnih et al Atari paper, in the IMPALA paper, in the RL experiments of the PBT paper, in the Zambaldi et al paper). I have seen speculation online that this is because RMSProp may be well-suited to deep learning on non-stationary distributions. In this note I try to examine the RMSProp algorithm and specifically the significance of the epsilon hyperparameter. The references are

• Karpathy's CS231n notes
• Hinton's lecture notes
• The deep learning textbook Chapter 8

Often in the literature RMSProp is presented as a variation of AdaGrad (e.g. in the deep learning textbook and in Karpathy's class). However, I think this is misleading, and that the explanation in Hinton's lecture is (not surprisingl

dmurfet

Spaces of programs and synthesis

written in December 2018.

The practical development of deep learning and its associated infrastructure has initiated a broad re-examination of the practice of computer programming. In this document we briefly survey how this discussion has evolved over the past few years, and then describe our point of view on the underlying mathematics.

Program synthesis

We begin with some appeals to authority, in the form of the following references:

dmurfet

Getting started

First you'll have to install the Git command line tool on your machine, following these instructions. Then find the repository that you want to contribute to, copy its address from the green "Clone or Download" button, and on your local machine run e.g.

git clone https://github.com/dmurfet/difflinearlogic.git

Committing changes

To see a list of what has changed (optional) run git status. Then

dmurfet

The Rising Sea HOWTO

Videos

Recently I have been posting videos of seminar talks and lectures online to a YouTube channel. The equipment and software that I use:

• Two Sony HDR-CX405 video cameras (around $300 in 2018) on cheap generic tripods (under $50).
• Sennhesier ClipMic digital lapel microphone (around $200 in 2018).
• An iPhone (which the ClipMic digital records to).
• A bunch of 32Gb microSD cards.

dmurfet

# Science 12-2-22

Basics of genetics
https://ocw.mit.edu/courses/biology/7-01sc-fundamentals-of-biology-fall-2011/genetics/mendels-laws/
* Watched Excerpt 1 and answered the question
* Planned to plant peas

# Future

High school biology