cedrickchee

Google-owned Kaggle adds free GPUs to online coding service.

Google says users of Colaboratory, its live coding mashup that works like a cross between a Jupyter Notebook and a Google Doc, now comes with free GPUs. Users can write a few code snippets, detailed here, and get access to two vCPUs with 13GB of RAM and, the icing on the cake - an NVIDIA K80 GPU, according to a comment from an account linked to Michael Piatek at Google.

Access Colaboratory here.

This is awesome! This definitely increases the usability of Colab notebooks, which I have already started using in my day-to-day experiments. I especially like the ability to save multiple checkpoints - a features sorely lacking in standard Jupyter Notebooks.

Free GPUS with very few strings attached:

• Jupyter notebooks max running time of 12 hours, according to someone (Michael Piatek?) on the Colab team. What this means is,

cedrickchee

class AMSgrad(Optimizer):
    """AMSGrad optimizer.

    Default parameters follow those provided in the Adam paper.

    # Arguments
        lr: float >= 0. Learning rate.
        beta_1: float, 0 < beta < 1. Generally close to 1.
        beta_2: float, 0 < beta < 1. Generally close to 1.
        epsilon: float >= 0. Fuzz factor.

cedrickchee

cedrickchee

git commit --amend --author "John Doe <john.doe@gmail.com>" --no-edit
git rebase --continue
git push origin master --force

cedrickchee

Originally forked from Philipp Bayer's gist. All credits goes to him.

This gist convert the original text to markdown for better readability.

Problems and Ideas:

1. Gene function prediction - given a predicted protein or gene sequence, what is the function?

The classic approach is to use something like BLAST to compare with known sequences, but this has many drawbacks. For starters, in plants the databases lean very heavily towards Arabidopsis thaliana, not more common plants such as maize or wheat.

cedrickchee

Project Title

One Paragraph of project description goes here

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

cedrickchee

This gist was being mentioned by Jeremy in fast.ai deep learning part 2 2018, lesson 9 video. In this lesson, we are learning multi-class object detection (computer vision) using the pascal-multi.ipynb notebook.

One of the fast.ai's students pointed out that by using Pandas, we can do things much simpler than using Python collections.defaultdict and shared this gist.

>The more you get to know Pandas, the more often you realize it is a good way to solve lots of different problems.

cedrickchee

My Notes for the Talk

A talk by Andrej Karpathy at Train AI 2018 conference - machine learning for a human world.

Training Datasets

The part around building and managing datasets is very interesting. We don't get to hear about these problems often.

Software 2.0 Integrated Development Enviroments (IDEs)

What IDEs including code editors will look like?

cedrickchee

My notes on self-improvement plan.

Ways to change habits:

1. Get more sleep
2. Make time to exercise
3. Drink more water
4. Eat less sugar
5. Stay teachable
6. Read and write more

cedrickchee

Discriminative Learning Rate

This paper, Large Batch Training of Convolutional Networks by Boris Ginsburg et. al has discriminative learning rate algorithm known as Layer-wise Adaptive Rate Scaling (LARS).

It was used to train ImageNet with very very large batch sizes by looking at the ratio between the gradient and the mean at each layer and using that to change the learning rate of each layer automatically. They found that they could use much larger batch sizes.

Code Implementation

A training algorithm based on LARS implemented as an optimizer in PyTorch follows: