Skip to content

Instantly share code, notes, and snippets.

@seandavi
Created July 22, 2021 16:52
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save seandavi/651904b3060e01cba51e89168b72eee2 to your computer and use it in GitHub Desktop.
download all of EBI biosample as json
start_date='2000-01-01'
end_date = '2021-12-31'
datefilter = function(date) {
startdate = format(date,'%Y-%m-%d')
return(sprintf("dt:release:from=%suntil=%s",startdate,startdate))
}
download_biosample = function(date) {
require(httr)
require(jsonlite)
j = 0
pagesize=2500
while(TRUE) {
x = httr::GET("https://www.ebi.ac.uk/biosamples/samples",
query=list(page=j,size=pagesize,filter=datefilter(date)))
res = content(x)$`_embedded`$samples
if(length(res)==0) break
txt = paste(sapply(res,
jsonlite::toJSON),
collapse = '\n')
write(txt,file = sprintf('/tmp/biosample_%s.json',date),append=TRUE)
j = j+1
}
}
library(parallel)
parallel::mclapply(seq(as.Date(start_date),as.Date(end_date),by=1),
download_biosample,mc.cores=16)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment