Skip to content

Instantly share code, notes, and snippets.

@shawngraham
Last active May 1, 2018 23:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shawngraham/37688cac3a6501806b85277e0e0fbec8 to your computer and use it in GitHub Desktop.
Save shawngraham/37688cac3a6501806b85277e0e0fbec8 to your computer and use it in GitHub Desktop.
experiment with docker and rnn on the programming historian source files

I want to get the computer to generate new Programming Historian tutorials for me. Madness, I know, but.... kinda awesome.

torch-rnn to the rescue!

helpful: http://seankross.com/2017/09/17/Enough-Docker-to-be-Dangerous.html helpful: https://hub.docker.com/r/crisbal/torch-rnn/

So:

  1. download, install docker.
  2. start docker.
  3. get the docker image from https://hub.docker.com/r/crisbal/torch-rnn/
$ docker pull crisbal/torch-rnn:base

Start it up with:

docker run -it crisbal/torch-rnn:base

and exit to get out.

Right. we want to put some new data inside the image. Get the container ID

$ docker ps

docker image, image name... not useful. container id is where it's at. Say the id was 776f3940e045

then you can start it up with docker start -ai 776f3940e045 . Anyway, we need that id for cp purposes etc.

Meanwhile, I wanted all of the source MD files for programming historian. Turns out, very hard to download just a folder from a github repo. So I wget'ed one at a time, forgetting that I'd end up with all of the github shit too. Ah well. I could've just downloaded the whole repo, then grabbed what I wanted, but the repo is really large and I'm on a slow connection.

Anyway, concatenated all the files into a single input.txt file. Now need to get that into my data folder inside the docker image:

$ docker cp input.txt 776f3940e045:/root/torch-rnn/data/

ta da. data is now inside. proceed with usual training etc.

This is also very helpful for understanding what's going on with the rnn: https://github.com/eltiffster/authorFunction#stopping-and-starting-training

.... a day later ....

I resumed with

th train.lua -input_h5 data/proghist.h5 -input_json data/proghist.json -model_type rnn -num_layers 3 -rnn_size 256 -init_from cv/checkpoint_20000.t7 -gpu -1

of course, I didn't specify rnn layers or size last time, so that probably frigs things up. original training was:

th train.lua \
 -input_h5 data/proghist.h5 \
 -input_json data/proghist.json \
 -gpu -1
$ root@776f3940e045:~/torch-rnn# th sample.lua -checkpoint cv/checkpoint_20000.t7 -length 20000 -temperature 0.5 -gpu -1
<span class="pl-c"><span class="pl-c">#</span> the maps of example used to the search familiar module to see the results complex something an application if you do not open the pages and a command line to the <code>&gt;</code> command line and the following a save the programming and example of the data in the directory <a href="http://dave.orger-search.python.org/" rel="nofollow">Turnet</a> how the case, the example to like the command line and example to <a href="http://www.articles.com/tokerseet-instancioner/" rel="nofollow">help</a>. The
resource to comparing and correctly complete the way of the tags in the context file that the command line is the results and command the same can be add one website you can be adding the top and the file to column install the charters are version contribute to write to the modules and file file that something a value and change the languages the expressions of the code to the restart a sense and command line will adding a complex possible to the directory to request in the charters of the data from the text of the programs are a new completed on the command line to create a command line the command line the first file for a very for change and there is a new context of the command line is not get of the file and a sarve and correspondence the tables you will and contents of the data it the <code>toxtmapison(</a></td>
<td><code>presential</code>. The book that we can end the trials to the <code>cumerials</code> in the example, the first contains the text field in the first counting a working a <a href="http://www.lations/library.py</code> to start a network to create a projecting <code>tomell</code> and geopard are complete the complete that should be a key of the lessons and counting the directory is an array this lesson the list of the area spaces that it in a text files that into the second line is a look at the command line) and the arrays and use the provided the map file to the previous cases a new correct the designed to the programming and a text with a project maps of that you have a spaces are the command line to a list of the first should do this top to correct to do this format is a new properties a second the results (<code>[172172.txt</code> to the document (interactive or the web in a new column context been directory. By the command line in the second with the test to complete in the parameters that recording less of poster for example, the command line
in the <code>bit Data</code></td>
<td><code>Get-sonnected of the posting the large it is a new scripting the same command classification of the coordinates to the correct some counting on a users or the command line this is a sentence we can be parameters are a programming of the couple Python the list of the search date of the add the command are explorely support that is the program for the complex to many parameters within a complete some any or a save the command line the search the database the <a href="http://data.org/" rel="nofollow">One results to results of the code the first be a file computer using by and programming and a suggest the first line more and expressions of the first parsed and called the start what you can do the directory to do this tutorial correct the pages in the first the command line to the simple the top of the location with the function to the search and <code>xml page</code></li>
<li><code>documentias checked <strong>tempossible</div></td>
<td><div>lesson</div></td>
<td><div>directory</th>
<th>Hello will need to the directory to do this may any and a new complete another programming example second in a default to included the command by are the <a href="https://en.wikipedia.org/wiki/Programminghistorian.appee/" rel="nofollow">Starting Screement manipulations to over a projects.">
<link rel="dns-arch-ht-less sheet" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Testing the GitHub windows lines and the "Provide the line and the same results in the sentences. For a list of the area formatitudes and have a start the text file for the texts in the above to the <code>grame</code> and the name of the options of the page and or post when the file to the care easier to a function that <code>http://www.ordext.com/2011/22.png" caption="Figure 16"><svg class="octicon octicon-link" viewBox="0 0 14 16" version="1.1" width="14" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M11 2H9c0-.55-.45-1-1-1H5c-.55 0-1 .45-1 1H2c-.55 0-1 .45-1 1v1c0 .55.45 1 1 1v9c0 .55.45 1 1 1h7c.55 0 1-.45 1-1V5c.55 0 1-.45 1-1V3c0-.55-.45-1-1-1zm-1 12H3V5h1v8h1V5h1v8h1V5h1v8h1V5h1v9zm1-10H2V3h9v1z"/></svg>
</a>
<a href="/programminghistorian/jekyll/blob/language-refactor/lessons/programminghistorian/jekyll/pulse">
<a class="js-selected-navigation-item selected reponav-item" itemprop="url" data-hotkey="g c" data-selected-links="repo_source-repository_id" content="12707869" /><meta name="octolytics-dimension-repository_network_root_nwo" content="programminghistorian/jekyll" /><meta property="og:description" content="jekyll - Jekyll-based static site for The Programming Historian ">
<meta name="go-import" content="github.com/programminghistorian/jekyll git https://github.com/programminghistorian/jekyll" /><meta property="og:description" content="jekyll - Jekyll-based static site for The Programming Historian " />
<meta nom-open ">
<div class="position-relative d-flex flex-justify-between pt-6 pb-2 mt-6 f6 text-gray border-top border-gray-light ">
<ul class="list-style-none d-flex flex-wrap ">
<li class="mr-3">&copy; 2018 <span title="0.35648506786" %}</p>
<p>The command line the tasks arrow the search and the command line and see the top of the location of the <code>id</code></td>
<td><code>time.ion/oclegangline" rel="nofollow">Tolline research <a href="https://www.twishing.com/" rel="nofollow">GetIt with the results and the static simple noderially search text file to find the programming array to see of the more also complex path the script to create studio to the second the file. If you can have the task and <code>moduce</code></td>
<td><code>gce(</code></pre>
<p>The results that you will and complex interested to see it support the two provided to a sense the poster for a process for the <code>programming-cript</code>. This is the time the next contained by the data in the code to see <code>launter</code>. The same and command line placenation can be a text for a text can also charters and more we are people use the <a href="http://suchurgess.exsugeeded.txt-archivaliterf-0.png" caption="texts and Text+section</a>
approach the data in the data set of the provided to the following the command line the command line the programming and the large as a new contained the first many are a sendent to the following some a new
computer that you can also change the approach the directory a suggest be a who was could box. We can programming and make the parameters and project of the provided by poster in the directory in your data for the personal terminal and data of the same can additionality to exception of multiple of the generally called <code>text</code> column the computer that matches data as a structure and <a href="https://en.wikipedia.org/wiki/Text</code> and the programming because it is a good contained by the transcript to the data such as a records between the command line and new so the preference description of the tasks that the command line to the <a href="http://www.oldbaileyonline.org/" rel="nofollow">new You can completes of the search the top of the select the values in the search for the library language to see the first save the blank of the fetween a start menu command line is a polygons and include the API directory of the editions are the similar where of each the code that the same of the code that you can also see the file to the first contains a directory and column to full example, the command the second to the first the file will need to previous to the variable format particular you can see the process the complete a post from the population and a discover to see the such as a more command line and computer that is a set of the points contains of the same to the script the directory on a save the transcript to a relations of the 'charter parameters with a website of the command such of the files in the dictionary of the list of the contained by the command line in your properned with the simple tracket command) in the code in the nest layers and programming of the context and the command line remove the process from the map for some the applieration of the task the <a href="https://github.com/" rel="nofollow">second</a> that the text file to add the results and the command line of the first one and data in the contains and column in a sense and computer we can be more in the command lines and then experimentation for the text in a website the sentences to the example, the parameters and possible to the second counting the directory code to the data.</p>
<p>You can adding a new part of the facet the learner the files. The <a href="http://www.map-1.png" %}</p>
<p>{% include figure.html filename="ar-dev-13.png" %}</p>
<p>{% include figure.html filename="geo12.png" caption="Figure 27 million people example and request date the section and column probably expression and a helps the start a test of the same the complex way that it is a back the database in the change the upport a comparing a list of this section and you can add the command complete to see the same for the first of the tasks and the following in the file with <code>starter(18327.mi.exturnit</code>. The <code>former</code> and computer to be used to the what is a few select the directory of the commands of the project in the command steps line to the data into the empty line is can can adding the map to the script with a programming example, and project the author analysis is the command line, or in the files the adding the rest the tasks projects the file the second container to change the top the "map"iticled listance" containing the following and our data, the filter, it to a text of the script to be a Markdown and <code>names</code>, <code>charter()</code></td>
<td><code>&gt;</code> and the belies a paragraphical maps to later that the command language of the project be of the code that across the <a href="https://geobounding-takers.csv" rel="nofollow">Open</a> on the command line is some notes for example, the code to the text the code to do the files as the sense command, and different recent file and the charters from the database in the sessed by the task more than the search results to a column to the documentation and the command lines in the command line and context. New the command languages and <a href="http://www.count/" rel="nofollow">Getterns <code>id</code> dictionary to collection code that the important to change the file which called a <code>to the Geoparser</code></td>
<td><code>text of Internetarchive</code> the text in the URL file because of the command could a file because the text the a new complexond</h2>
<p>The project of the command line with it computer, and the lesson the object page that is at the code of the command the <code>commanding line Receldands</code> in the text file is a various and expressions in the section of the results the command line simple the table and columns in the page to the first could be a resource for every data free the top of the language set in the command line, we can use the process that will adding the data is a texts as the command line and the example a and the data identified a new formats the same the accessive the directory counting a correspondence of the computer as a pustom. In the hashtag the script to the most had <code>python 2017-00-22-02-01-0</div></td>
<td><div>presercate to see the <code>to-the data</code></td>
<td><code>|</code></p>
<p>The programming one of the GitHub to the data additional toc. This result in the results in the same because the results change that it the command line. This command as the object to complex in the row to the file should be complete the <code>commanagera</div></td>
<td><code>summaries</code> and column in the search of the <code>gca(</code></td>
</tr>
<tr>
<td><code>180001/</div></td>
<td><div>longer-wither-leading-the-class=".png" caption="Figure 17. 1</h4d>
<p>{% include figure.html filename="geo16.dti/sitems/from-html-to-list-of-words-2.png" caption="Figure 17-38-0114-18-session-analysis"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>A programming Historian association.html</a></li>
<li class="mr-3"><a href="https://status.github.com/" data-ga-click="Footer, go to terms, text:terms" href="https://github.com/programminghistorian/jekyll/network" class="social-count"
aria-label="106 users forked this repository">
191
</a>
</li>
<li>
<a href="/login?return_to=%2Fprogramminghistorian%2Fjekyll"
class="btn btn-sm btn-with-count tooltipped tooltipped-n"
aria-label="You must be signed in to make or propose changes">
<svg class="octicon octicon-trashcan" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M14 6l-4.9-.64L7 1 4.9 5.36 0 6l3.6 3.26L2.67 14 7 11.67 11.33 14l-.93-4.74z"/></svg>
Star
</a>
<a class="social-count js-social-count" href="/programminghistorian/jekyll/stargazers"
aria-label="191 users starred this repository">
191
</a>
</li>
<li>
<a href="/login?return_to=%2Fprogramminghistorian%2Fjekyll"
class="btn btn-sm btn-with-count tooltipped tooltipped-n"
aria-label="You must be signed in to star a repository" rel="nofollow">
<svg class="octicon octicon-repo-forked" viewBox="0 0 10 16" version="1.1" width="10" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M12 5l-8 8-4-4 1.5-1.5L4 10l6.5-6.5z"/></svg>
<span class="select-menu-item-text css-truncate-target js-select-menu-filter-text">
check-review-ticket
</span>
</a>
<a class="select-menu-item js-navigation-item js-navigation-open "
href="/programminghistorian/jekyll/blob/gh-pages/lessons/code-repository/business /business/security /business/customers /business" href="/business">
Explore
</a> </li>
<li class="ml-4">
<a class="js-selected-navigation-item HeaderNavlink px-0 py-2 m-0" data-ga-click="Header, click, Nav menu - item:business" data-selected-links="/business /business/security /business/customers /business" href="/business">
Business
</a> </li>
<li class="ml-4">
<a class="js-selected-navigation-item HeaderNavlink px-0 py-2 m-0" data-ga-click="Header, click, Nav menu - item:pricing" data-selected-links="/pricing /pricing/developer /pricing/team /pricing/business-hosted /pricing/business-enterprise /pricing" href="/pricing">
Pricing
</a> </li>
<li class="select-menu-tab">
<a href="#" data-tab-filter="tags" data-filter-placeholder="Find a tag…" class="js-select-menu-tab" role="tab">Tags</a>
</li>
<li class="select-menu-tab">
<a href="#" data-tab-filter="branches" data-filter-placeholder="Filter branches/tags" class="js-select-menu-tab" role="tab">Tags</a>
</li>
</ul>
</div>
</span>
</div>
</div>
</div>
</div>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment