Skip to content

Instantly share code, notes, and snippets.

@brendano

brendano/.RData

Last active June 27, 2019 17:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save brendano/b93e1571c38fea904c51d13f470b894e to your computer and use it in GitHub Desktop.
Save brendano/b93e1571c38fea904c51d13f470b894e to your computer and use it in GitHub Desktop.
Download ZIP
google ngram books plot
Raw
.Rhistory
d=read.csv("out.csv")
head(d)
names(d)
d[1,]
3[,c(1,4,8)]
d[,c(1,4,8)]
x=d[,c(1,4,8)]
head(x)
melt(x)
spread(x)
?spread
gather
?gather
gather(d[,c(1,4,8)])
d$t=1980:(1980-1+nrow(d))
d
gather(d[,c(1,4,8)])
gather(d[,c(1,4,8)],key='t')
head(d,1)
gather(d[,c(1,4,8,'t')],key='t')
ncol(d)
gather(d[,c(1,4,8,11)],key='t')
gather(d[,c(1,4,8,'t')],key='t') %>% head
gather(d[,c(1,4,8,11)],key='t') %>% head
gather(d[,c(1,4,8,11)],key='t',value='ngram') %>% head
x=gather(d[,c(1,4,8,11)],key='t'); names(x)[2]="ngram"
head(x)
qplot(t,value,data=x,colour='ngram')
gg()
qplot(t,value,data=x,colour='ngram')
qplot(t,value,data=x,colour=ngram)
qplot(t,value,data=x,colour=ngram, geom=)
qplot(t,value,data=x,colour=ngram, geom=c('line','point'))
qplot(t,value,data=x,colour=ngram, geom=c('line','point'), log='y')
Raw
README.txt
Data from
https://books.google.com/ngrams/graph?content=%22cognitive+science%22%2C%22artificial+intelligence%22%2C%22machine+learning%22&case_insensitive=on&year_start=1980&year_end=2008&corpus=15&smoothing=3&share=&direct_url=t4%3B%2C%22%20cognitive%20science%20%22%3B%2Cc0%3B%2Cs0%3B%3B%22%20cognitive%20science%20%22%3B%2Cc0%3B%3B%22%20Cognitive%20Science%20%22%3B%2Cc0%3B.t4%3B%2C%22%20artificial%20intelligence%20%22%3B%2Cc0%3B%2Cs0%3B%3B%22%20artificial%20intelligence%20%22%3B%2Cc0%3B%3B%22%20Artificial%20Intelligence%20%22%3B%2Cc0%3B%3B%22%20Artificial%20intelligence%20%22%3B%2Cc0%3B.t4%3B%2C%22%20machine%20learning%20%22%3B%2Cc0%3B%2Cs0%3B%3B%22%20machine%20learning%20%22%3B%2Cc0%3B%3B%22%20Machine%20Learning%20%22%3B%2Cc0#t4%3B%2C%22%20cognitive%20science%20%22%3B%2Cc0%3B%2Cs0%3B%3B%22%20cognitive%20science%20%22%3B%2Cc0%3B%3B%22%20Cognitive%20Science%20%22%3B%2Cc0%3B.t4%3B%2C%22%20artificial%20intelligence%20%22%3B%2Cc0%3B%2Cs0%3B%3B%22%20artificial%20intelligence%20%22%3B%2Cc0%3B%3B%22%20Artificial%20Intelligence%20%22%3B%2Cc0%3B%3B%22%20Artificial%20intelligence%20%22%3B%2Cc0%3B.t4%3B%2C%22%20machine%20learning%20%22%3B%2Cc0%3B%2Cs0%3B%3B%22%20machine%20learning%20%22%3B%2Cc0%3B%3B%22%20Machine%20Learning%20%22%3B%2Cc0
copy and paste "var data" line into => data.txt
Commands after that:
~/Desktop/ngram_trends_ai % python get.py > out.csv
$ R
> d=read.csv("out.csv")
> d$t=1980:(1980-1+nrow(d))
> x=gather(d[,c(1,4,8,11)],key='t'); names(x)[2]="ngram"
> qplot(t,value,data=x,colour=ngram, geom=c('line','point'), log='y')
Output in: Rplot.pdf
Raw
data.txt
var data = [{"ngram": "" cognitive science &quot (All)", "type": "CASE_INSENSITIVE", "timeseries": [1.988285345394525e-09, 1.7889791814607038e-09, 2.3565464302155448e-09, 2.5652787751160945e-09, 3.0793829728211925e-09, 3.296852761030295e-09, 3.7145815253079662e-09, 4.1995486433592845e-09, 4.690783701942282e-09, 4.800356035291768e-09, 5.018061000458143e-09, 5.3018152903642044e-09, 5.530524453083758e-09, 5.431870035115567e-09, 5.432463155906144e-09, 5.503084684372566e-09, 5.539179206584929e-09, 5.2331939932275566e-09, 4.703808584701921e-09, 4.348608251270036e-09, 4.178451701705325e-09, 3.749049818606893e-09, 3.348010010102114e-09, 3.2263143387048896e-09, 3.001070704444775e-09, 2.8223902371972615e-09, 2.7316326411489884e-09, 2.7214903677852397e-09, 2.7750903189988074e-09], "parent": ""}, {"ngram": "" cognitive science &quot", "type": "EXPANSION", "parent": "" cognitive science &quot (All)", "timeseries": [1.8215109143238095e-09, 1.6158894555751147e-09, 2.148176812359163e-09, 2.38667624552491e-09, 2.7813012765902964e-09, 3.0312720825495123e-09, 3.4260253760949003e-09, 3.856114990811019e-09, 4.3320366341756e-09, 4.47677070017757e-09, 4.594069505766843e-09, 4.879946859302566e-09, 5.054121819471545e-09, 4.944040795809965e-09, 4.94592842871709e-09, 5.02822031630704e-09, 5.054027862883004e-09, 4.78122798495763e-09, 4.3752795037781715e-09, 4.038539058726087e-09, 3.8807024895147285e-09, 3.5146172956095664e-09, 3.1455563867918303e-09, 3.0244904358767144e-09, 2.857193307345288e-09, 2.666465239085726e-09, 2.5918068432180994e-09, 2.58461434476942e-09, 2.6398108698977296e-09]}, {"ngram": "" Cognitive Science &quot", "type": "EXPANSION", "parent": "" cognitive science &quot (All)", "timeseries": [1.6677443107071532e-10, 1.73089725885589e-10, 2.0836961785638172e-10, 1.7860252959118432e-10, 2.98081696230896e-10, 2.6558067848078283e-10, 2.8855614921306586e-10, 3.4343365254826553e-10, 3.587470677666817e-10, 3.235853351141975e-10, 4.239914946913004e-10, 4.218684310616386e-10, 4.764026336122131e-10, 4.878292393056022e-10, 4.865347271890538e-10, 4.748643680655259e-10, 4.851513437019252e-10, 4.5196600826992647e-10, 3.2852908092374985e-10, 3.1006919254394916e-10, 2.977492121905964e-10, 2.344325229973267e-10, 2.024536233102836e-10, 2.018239028281751e-10, 1.438773970994868e-10, 1.5592499811153563e-10, 1.3982579793088887e-10, 1.3687602301581948e-10, 1.352794491010778e-10]}, {"ngram": "" artificial intelligence &quot (All)", "type": "CASE_INSENSITIVE", "timeseries": [1.4524584965314169e-08, 1.645942996342242e-08, 1.926327901820753e-08, 2.0718613547021647e-08, 2.3277992263917976e-08, 2.644595978714851e-08, 2.8628305345382698e-08, 3.116548560492102e-08, 3.1549647592856086e-08, 3.121157824256419e-08, 3.0578806058707235e-08, 2.9286710437733835e-08, 2.651584762869419e-08, 2.414005418576488e-08, 2.2005876934175613e-08, 2.0355296674402978e-08, 1.7849577851495165e-08, 1.6332075703744716e-08, 1.502922459103598e-08, 1.4232912804683958e-08, 1.411945155635078e-08, 1.3288917989827198e-08, 1.273601415996417e-08, 1.269111644148462e-08, 1.2006758347480963e-08, 1.1474028727143095e-08, 1.1197858575580828e-08, 1.0778746017958697e-08, 1.0733540584118018e-08], "parent": ""}, {"ngram": "" artificial intelligence &quot", "type": "EXPANSION", "parent": "" artificial intelligence &quot (All)", "timeseries": [1.2215100930745848e-08, 1.3739098747578282e-08, 1.5681708672351153e-08, 1.6791667485043556e-08, 1.860523620881135e-08, 2.052665643808333e-08, 2.2170917546304736e-08, 2.390068257592962e-08, 2.3838826278068026e-08, 2.296393729141398e-08, 2.258575675690671e-08, 2.165806378684465e-08, 2.013926346324167e-08, 1.8447448774898996e-08, 1.711443964325099e-08, 1.610155107657972e-08, 1.4687722784044257e-08, 1.3396221021366728e-08, 1.2396914225324248e-08, 1.1422203627375893e-08, 1.1169280230863673e-08, 1.022643861655784e-08, 9.646290496537598e-09, 9.750849270874693e-09, 9.136875256388066e-09, 8.666873377701937e-09, 8.427884390371787e-09, 8.01124580007695e-09, 8.115831451327438e-09]}, {"ngram": "" Artificial Intelligence &quot", "type": "EXPANSION", "parent": "" artificial intelligence &quot (All)", "timeseries": [1.3226829354007208e-09, 1.6135288882779263e-09, 2.1782737280595654e-09, 2.5423264287433814e-09, 3.24110765210455e-09, 4.3825620117082e-09, 5.046234509618687e-09, 5.813224663547041e-09, 6.33315730017411e-09, 6.905929225301374e-09, 6.692562916553259e-09, 6.355629673342784e-09, 5.21912795521067e-09, 4.5926490071274935e-09, 3.9493915915400635e-09, 3.3685355933193203e-09, 2.5777147242119926e-09, 2.4519504680599523e-09, 2.2320659447722893e-09, 2.3688007904405187e-09, 2.465377679783387e-09, 2.570266729609979e-09, 2.6725859303411426e-09, 2.58853922423847e-09, 2.4379214995968415e-09, 2.418349568219672e-09, 2.4285986370090975e-09, 2.4660518294084e-09, 2.31252980653629e-09]}, {"ngram": "" Artificial intelligence &quot", "type": "EXPANSION", "parent": "" artificial intelligence &quot (All)", "timeseries": [9.868010991676002e-10, 1.1068023275662142e-09, 1.403296617796812e-09, 1.3846196332347104e-09, 1.4316484030020759e-09, 1.536741337356976e-09, 1.4111532894592722e-09, 1.4515783654443582e-09, 1.3776640146139535e-09, 1.3417117258488327e-09, 1.3004863852472681e-09, 1.2730169775464007e-09, 1.157456210241849e-09, 1.099956403738389e-09, 9.420456993845606e-10, 8.852100045039382e-10, 5.841403432389163e-10, 4.839042143180348e-10, 4.00244420939444e-10, 4.419083868675447e-10, 4.847936457037199e-10, 4.922126436593796e-10, 4.1713773308542963e-10, 3.5172794637145747e-10, 4.3196159149605506e-10, 3.888057812214843e-10, 3.4137554819994403e-10, 3.0144838847334656e-10, 3.051793262542901e-10]}, {"ngram": "" machine learning &quot (All)", "type": "CASE_INSENSITIVE", "timeseries": [4.844133115466942e-10, 3.8753064923735535e-10, 3.550062975069916e-10, 4.601145040369506e-10, 6.809790548916489e-10, 9.72570417058703e-10, 1.2469679772879924e-09, 1.712779319167674e-09, 1.8437270955993868e-09, 1.9548805411741453e-09, 1.8994633108456884e-09, 1.7959548630010401e-09, 1.5588977053671016e-09, 1.3116974371753442e-09, 7.284004669269198e-10, 7.253682892433793e-10, 7.221464775370683e-10, 7.561819563374682e-10, 7.785945202057728e-10, 1.0126548211086828e-09, 1.2013968986313017e-09, 1.3418451905167217e-09, 2.048401289859011e-09, 2.089761356024614e-09, 2.1129831257698277e-09, 2.0331732012276606e-09, 2.0353418117569078e-09, 2.009601096641944e-09, 2.2075689470879567e-09], "parent": ""}, {"ngram": "" machine learning &quot", "type": "EXPANSION", "parent": "" machine learning &quot (All)", "timeseries": [3.2129586144402644e-10, 2.5703668915522113e-10, 2.4626133077187973e-10, 2.370517056458711e-10, 3.8187721635678836e-10, 4.304757780396581e-10, 5.683286024333561e-10, 7.151002607524199e-10, 8.242234077788473e-10, 8.759610160240833e-10, 9.303153802875386e-10, 8.27868464257721e-10, 8.156260389302616e-10, 7.637784016356619e-10, 4.82764568400716e-10, 4.535886686093552e-10, 4.3455206298416623e-10, 4.3489211636644447e-10, 4.3946887597266585e-10, 5.233604482330583e-10, 6.140120297125153e-10, 6.791196652652534e-10, 1.1311481196152329e-09, 1.1789929033056978e-09, 1.2096197377076123e-09, 1.207196850419986e-09, 1.2681073321048804e-09, 1.2744093225158792e-09, 1.449749334470063e-09]}, {"ngram": "" Machine Learning &quot", "type": "EXPANSION", "parent": "" machine learning &quot (All)", "timeseries": [1.6311745010266776e-10, 1.3049396008213422e-10, 1.0874496673511184e-10, 2.230627983910795e-10, 2.9910183853486064e-10, 5.420946390190449e-10, 6.786393748546362e-10, 9.97679058415254e-10, 1.0195036878205395e-09, 1.078919525150062e-09, 9.6914793055815e-10, 9.68086398743319e-10, 7.4327166643684e-10, 5.479190355396822e-10, 2.4563589852620385e-10, 2.717796206340241e-10, 2.8759441455290206e-10, 3.212898399710237e-10, 3.39125644233107e-10, 4.892943728756244e-10, 5.873848689187864e-10, 6.627255252514682e-10, 9.172531702437783e-10, 9.107684527189162e-10, 9.033633880622155e-10, 8.259763508076747e-10, 7.672344796520273e-10, 7.351917741260649e-10, 7.578196126178938e-10]}];
Raw
get.py
import json,re
d=open("data.txt").read()
x=json.loads(re.sub(r";\s*$","", re.sub(r".*= ","",d)))
print ','.join(z['ngram'] for z in x)
for t in range(len(x[0]['timeseries'])):
print ','.join(str(z['timeseries'][t]) for z in x)
Raw
out.csv
" cognitive science &quot (All) " cognitive science &quot " Cognitive Science &quot " artificial intelligence &quot (All) " artificial intelligence &quot " Artificial Intelligence &quot " Artificial intelligence &quot " machine learning &quot (All) " machine learning &quot " Machine Learning &quot
1.98828534539e-09 1.82151091432e-09 1.66774431071e-10 1.45245849653e-08 1.22151009307e-08 1.3226829354e-09 9.86801099168e-10 4.84413311547e-10 3.21295861444e-10 1.63117450103e-10
1.78897918146e-09 1.61588945558e-09 1.73089725886e-10 1.64594299634e-08 1.37390987476e-08 1.61352888828e-09 1.10680232757e-09 3.87530649237e-10 2.57036689155e-10 1.30493960082e-10
2.35654643022e-09 2.14817681236e-09 2.08369617856e-10 1.92632790182e-08 1.56817086724e-08 2.17827372806e-09 1.4032966178e-09 3.55006297507e-10 2.46261330772e-10 1.08744966735e-10
2.56527877512e-09 2.38667624552e-09 1.78602529591e-10 2.0718613547e-08 1.6791667485e-08 2.54232642874e-09 1.38461963323e-09 4.60114504037e-10 2.37051705646e-10 2.23062798391e-10
3.07938297282e-09 2.78130127659e-09 2.98081696231e-10 2.32779922639e-08 1.86052362088e-08 3.2411076521e-09 1.431648403e-09 6.80979054892e-10 3.81877216357e-10 2.99101838535e-10
3.29685276103e-09 3.03127208255e-09 2.65580678481e-10 2.64459597871e-08 2.05266564381e-08 4.38256201171e-09 1.53674133736e-09 9.72570417059e-10 4.3047577804e-10 5.42094639019e-10
3.71458152531e-09 3.42602537609e-09 2.88556149213e-10 2.86283053454e-08 2.21709175463e-08 5.04623450962e-09 1.41115328946e-09 1.24696797729e-09 5.68328602433e-10 6.78639374855e-10
4.19954864336e-09 3.85611499081e-09 3.43433652548e-10 3.11654856049e-08 2.39006825759e-08 5.81322466355e-09 1.45157836544e-09 1.71277931917e-09 7.15100260752e-10 9.97679058415e-10
4.69078370194e-09 4.33203663418e-09 3.58747067767e-10 3.15496475929e-08 2.38388262781e-08 6.33315730017e-09 1.37766401461e-09 1.8437270956e-09 8.24223407779e-10 1.01950368782e-09
4.80035603529e-09 4.47677070018e-09 3.23585335114e-10 3.12115782426e-08 2.29639372914e-08 6.9059292253e-09 1.34171172585e-09 1.95488054117e-09 8.75961016024e-10 1.07891952515e-09
5.01806100046e-09 4.59406950577e-09 4.23991494691e-10 3.05788060587e-08 2.25857567569e-08 6.69256291655e-09 1.30048638525e-09 1.89946331085e-09 9.30315380288e-10 9.69147930558e-10
5.30181529036e-09 4.8799468593e-09 4.21868431062e-10 2.92867104377e-08 2.16580637868e-08 6.35562967334e-09 1.27301697755e-09 1.795954863e-09 8.27868464258e-10 9.68086398743e-10
5.53052445308e-09 5.05412181947e-09 4.76402633612e-10 2.65158476287e-08 2.01392634632e-08 5.21912795521e-09 1.15745621024e-09 1.55889770537e-09 8.1562603893e-10 7.43271666437e-10
5.43187003512e-09 4.94404079581e-09 4.87829239306e-10 2.41400541858e-08 1.84474487749e-08 4.59264900713e-09 1.09995640374e-09 1.31169743718e-09 7.63778401636e-10 5.4791903554e-10
5.43246315591e-09 4.94592842872e-09 4.86534727189e-10 2.20058769342e-08 1.71144396433e-08 3.94939159154e-09 9.42045699385e-10 7.28400466927e-10 4.82764568401e-10 2.45635898526e-10
5.50308468437e-09 5.02822031631e-09 4.74864368066e-10 2.03552966744e-08 1.61015510766e-08 3.36853559332e-09 8.85210004504e-10 7.25368289243e-10 4.53588668609e-10 2.71779620634e-10
5.53917920658e-09 5.05402786288e-09 4.85151343702e-10 1.78495778515e-08 1.4687722784e-08 2.57771472421e-09 5.84140343239e-10 7.22146477537e-10 4.34552062984e-10 2.87594414553e-10
5.23319399323e-09 4.78122798496e-09 4.5196600827e-10 1.63320757037e-08 1.33962210214e-08 2.45195046806e-09 4.83904214318e-10 7.56181956337e-10 4.34892116366e-10 3.21289839971e-10
4.7038085847e-09 4.37527950378e-09 3.28529080924e-10 1.5029224591e-08 1.23969142253e-08 2.23206594477e-09 4.00244420939e-10 7.78594520206e-10 4.39468875973e-10 3.39125644233e-10
4.34860825127e-09 4.03853905873e-09 3.10069192544e-10 1.42329128047e-08 1.14222036274e-08 2.36880079044e-09 4.41908386868e-10 1.01265482111e-09 5.23360448233e-10 4.89294372876e-10
4.17845170171e-09 3.88070248951e-09 2.97749212191e-10 1.41194515564e-08 1.11692802309e-08 2.46537767978e-09 4.84793645704e-10 1.20139689863e-09 6.14012029713e-10 5.87384868919e-10
3.74904981861e-09 3.51461729561e-09 2.34432522997e-10 1.32889179898e-08 1.02264386166e-08 2.57026672961e-09 4.92212643659e-10 1.34184519052e-09 6.79119665265e-10 6.62725525251e-10
3.3480100101e-09 3.14555638679e-09 2.0245362331e-10 1.273601416e-08 9.64629049654e-09 2.67258593034e-09 4.17137733085e-10 2.04840128986e-09 1.13114811962e-09 9.17253170244e-10
3.2263143387e-09 3.02449043588e-09 2.01823902828e-10 1.26911164415e-08 9.75084927087e-09 2.58853922424e-09 3.51727946371e-10 2.08976135602e-09 1.17899290331e-09 9.10768452719e-10
3.00107070444e-09 2.85719330735e-09 1.43877397099e-10 1.20067583475e-08 9.13687525639e-09 2.4379214996e-09 4.31961591496e-10 2.11298312577e-09 1.20961973771e-09 9.03363388062e-10
2.8223902372e-09 2.66646523909e-09 1.55924998112e-10 1.14740287271e-08 8.6668733777e-09 2.41834956822e-09 3.88805781221e-10 2.03317320123e-09 1.20719685042e-09 8.25976350808e-10
2.73163264115e-09 2.59180684322e-09 1.39825797931e-10 1.11978585756e-08 8.42788439037e-09 2.42859863701e-09 3.413755482e-10 2.03534181176e-09 1.2681073321e-09 7.67234479652e-10
2.72149036779e-09 2.58461434477e-09 1.36876023016e-10 1.0778746018e-08 8.01124580008e-09 2.46605182941e-09 3.01448388473e-10 2.00960109664e-09 1.27440932252e-09 7.35191774126e-10
2.775090319e-09 2.6398108699e-09 1.35279449101e-10 1.07335405841e-08 8.11583145133e-09 2.31252980654e-09 3.05179326254e-10 2.20756894709e-09 1.44974933447e-09 7.57819612618e-10
Display the source blob
Display the rendered blob
Raw
Rplot.pdf
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment