Skip to content

Instantly share code, notes, and snippets.

View 1-introducing-datasets.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lambdamusic
lambdamusic / python_request_create_gist.py
Last active Jul 28, 2019 — forked from joshisumit/python_request_create_gist.py
Create GIST from your python code with python requests module and OAuth token.
View python_request_create_gist.py
'''
HTTP Reuests has following parameters:
1)Request URL
2)Header Fields
3)Parameter
4)Request body
PREREQUISITE
* head over to https://github.com/settings/tokens and generate a new token with 'gists' permissions
View load_logs_from_s3.py
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
iterate a bucket in s3 containing JSON log files
get each file, massage json a little and uploade to elasticsearc
"""
@lambdamusic
lambdamusic / split_json.sh
Created Feb 27, 2019
Split jsonl file by line with 10000+ lines and rename files
View split_json.sh
#!/bin/bash
FILENAME=records_111901_00000000.jsonl
split -a 5 -l 1 $FILENAME temp
n=1
for f in temp*
do
cat $f >> Part${n}.json
rm $f
((n++))
done
@lambdamusic
lambdamusic / splitter.sh
Created Sep 24, 2018 — forked from maesa/splitter.sh
Bash script for splitting large CSV files while keeping the header into 100 lines a piece using Split. Outputs as Part1, Part2, ... while keeping its Header
View splitter.sh
#!/bin/bash
FILENAME=filename_here.csv
HDR=$(head -1 $FILENAME)
split -l 100 $FILENAME xyz
n=1
for f in xyz*
do
if [ $n -gt 1 ]; then
echo $HDR > Part${n}.csv
fi
@lambdamusic
lambdamusic / scigraph_context.json
Last active Sep 21, 2018
scigraph context (preview)
View scigraph_context.json
# source: https://github.com/springernature/scigraph/blob/master/2018Q3/context/scigraph.json
{
"@context": {
"@vocab": "http://schema.org/",
"@base": null,
"sg": "http://scigraph.springernature.com/id/",
"sgo": "http://scigraph.springernature.com/ontologies/core/",
"rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
"rdfs": "http://www.w3.org/2000/01/rdf-schema#",
@lambdamusic
lambdamusic / grant.jsonld
Created Jul 19, 2018
SciGraph Grants schema.org JSONLD
View grant.jsonld
{
"@context": {
"@vocab": "http://schema.org/",
"@base": null,
"sg.articles": "http://scigraph.springernature.com/things/articles/",
"sg.persons": "http://scigraph.springernature.com/things/persons/",
"sg.grants": "http://scigraph.springernature.com/things/grants/",
"sg": "http://scigraph.springernature.com/ontologies/core/"
},
"@id": "sg.grants:grant.2691278",
View SPARQL queries to obtain statistics.
# Obtain the label of a given class (:class1).
SELECT DISTINCT ?c (STR(?l) AS ?lb)
WHERE {
?c a :class1 ;
<http://www.w3.org/2000/01/rdf-schema#label> ?l .
}
# Obtain a list of classes.
SELECT DISTINCT ?c
WHERE {
@lambdamusic
lambdamusic / querymanager.py
Created Feb 22, 2018
Python Index Slicing implementation
View querymanager.py
def __getitem__(self, key):
"""
Basic slicing support up to 100 (QueryManager.BATCH_DEFAULT_SIZE)
https://docs.python.org/3/reference/datamodel.html#object.__getitem__
This is just quick way to get samples items, as it works only for the initial batch of results.
Use the next() method to go through all the resultset.
"""
if isinstance( key, slice ) :
@lambdamusic
lambdamusic / Django 1.8 static tips
Last active Feb 1, 2018
Django 1.8 static tips
View Django 1.8 static tips
# usual way
```
<link href="{% static 'libs/bootswatch3.2/{{ THEME }}/bootstrap.min.css' %}" rel="stylesheet">
```
# more manual but works if you have variables in it
```