View wdatebash
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
wdate () { | |
search=$1; | |
zoneinfo=/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pytz/zoneinfo/; | |
format='%a %F %T'; | |
find -L $zoneinfo -type f \ |
View 1-introducing-datasets.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View python_request_create_gist.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
''' | |
HTTP Reuests has following parameters: | |
1)Request URL | |
2)Header Fields | |
3)Parameter | |
4)Request body | |
PREREQUISITE | |
* head over to https://github.com/settings/tokens and generate a new token with 'gists' permissions |
View load_logs_from_s3.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
# -*- coding: utf-8 -*- | |
""" | |
iterate a bucket in s3 containing JSON log files | |
get each file, massage json a little and uploade to elasticsearc | |
""" | |
View split_json.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
FILENAME=records_111901_00000000.jsonl | |
split -a 5 -l 1 $FILENAME temp | |
n=1 | |
for f in temp* | |
do | |
cat $f >> Part${n}.json | |
rm $f | |
((n++)) | |
done |
View splitter.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
FILENAME=filename_here.csv | |
HDR=$(head -1 $FILENAME) | |
split -l 100 $FILENAME xyz | |
n=1 | |
for f in xyz* | |
do | |
if [ $n -gt 1 ]; then | |
echo $HDR > Part${n}.csv | |
fi |
View scigraph_context.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# source: https://github.com/springernature/scigraph/blob/master/2018Q3/context/scigraph.json | |
{ | |
"@context": { | |
"@vocab": "http://schema.org/", | |
"@base": null, | |
"sg": "http://scigraph.springernature.com/id/", | |
"sgo": "http://scigraph.springernature.com/ontologies/core/", | |
"rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#", | |
"rdfs": "http://www.w3.org/2000/01/rdf-schema#", |
View grant.jsonld
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"@context": { | |
"@vocab": "http://schema.org/", | |
"@base": null, | |
"sg.articles": "http://scigraph.springernature.com/things/articles/", | |
"sg.persons": "http://scigraph.springernature.com/things/persons/", | |
"sg.grants": "http://scigraph.springernature.com/things/grants/", | |
"sg": "http://scigraph.springernature.com/ontologies/core/" | |
}, | |
"@id": "sg.grants:grant.2691278", |
View SPARQL queries to obtain statistics.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Obtain the label of a given class (:class1). | |
SELECT DISTINCT ?c (STR(?l) AS ?lb) | |
WHERE { | |
?c a :class1 ; | |
<http://www.w3.org/2000/01/rdf-schema#label> ?l . | |
} | |
# Obtain a list of classes. | |
SELECT DISTINCT ?c | |
WHERE { |
NewerOlder