Title | Prompt |
---|---|
Brainstorm Ideas Based On This | Brainstorm 5 project ideas based on this text: |
Create Action Items | Generate a markdown list of action items to complete based on the following text, using a unique identifier for each item as bold headings. If there are any errors in the text, make actions items to fix them. In a sublist of each item, provide a description, priority, estimated level of difficulty, and a reasonable duration for the task. Here is the text: |
Create Flashcards | Create 3 Anki flashcards based on the following text. Format the response as markdown with the bold questions and plaintext answers. Separate each entry with ‘—‘. Here’s the text: |
Generate Cheatsheet | Generate a concise cheatsheet for the concepts in this text. Add additional details based on your own knowledge of the topic. |
def top_k_top_p_filtering(logits, top_k=0, top_p=0.0, filter_value=-float('Inf')): | |
""" Filter a distribution of logits using top-k and/or nucleus (top-p) filtering | |
Args: | |
logits: logits distribution shape (vocabulary size) | |
top_k >0: keep only top k tokens with highest probability (top-k filtering). | |
top_p >0.0: keep the top tokens with cumulative probability >= top_p (nucleus filtering). | |
Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751) | |
""" | |
assert logits.dim() == 1 # batch size 1 for now - could be updated for more but the code would be less clear | |
top_k = min(top_k, logits.size(-1)) # Safety check |
This gist contains a list of points I found very useful while watching the fast.ai "Practical deep learning for coders" and "Cutting edge deep learning for coders" MOOC by Jeremy Howard and team. This list may not be complete as I watched the video at 1.5x speed on marathon but I did write down as many things I found to be very useful to get a model working. A fair warning the points are in no particular order, you may find the topics are all jumbled up.
Before beginning, I want to thank Jeremy Howard, Rachel Thomas, and the entire fast.ai team in making this awesome practically oriented MOOC.
-
Progressive image resolution training: Train the network on lower res first and then increase the resolution to get better performance. This can be thought of as transfer learning from the same dataset but at a different resolution. There is one paper by NVIDIA as well that used such an approach to train GANs.
-
Cyclical learning rates: Gradually increasing the learning rate initially helps to avoid getting stuc
# Remove anything linked to nvidia | |
sudo apt-get remove --purge nvidia* | |
sudo apt-get autoremove | |
# Search for your driver | |
apt search nvidia | |
# Select one driver (the last one is a decent choice) | |
sudo apt install nvidia-370 |
""" | |
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy) | |
BSD License | |
""" | |
import numpy as np | |
# data I/O | |
data = open('input.txt', 'r').read() # should be simple plain text file | |
chars = list(set(data)) | |
data_size, vocab_size = len(data), len(chars) |
import os | |
def get_size(start_path = '.'): | |
total_size = 0 | |
for dirpath, dirnames, filenames in os.walk(start_path): | |
for f in filenames: | |
fp = os.path.join(dirpath, f) | |
total_size += os.path.getsize(fp) | |
return total_size | |
print get_size() |