Skip to content

Instantly share code, notes, and snippets.

View raprasad's full-sized avatar

Raman Prasad raprasad

View GitHub Profile
@raprasad
raprasad / k8s_notes.md
Last active July 4, 2018 07:04
k8s notes

(1) We have a pod with multiple containers. Rough example:

- Pod

-- container 1:  webapp
      exposes port: 80

-- container 2: r-service
 exposes port: 8070
@raprasad
raprasad / NamedTemporaryFile_example.py
Last active June 28, 2018 19:25
example of NamedTemporaryFile
import os
import tempfile
def dashes(): print('-' * 40)
def msgt(amsg): dashes(); print(amsg); dashes()
# ---------------------------
# Create the temp file object
# ---------------------------
named_temp_file = tempfile.NamedTemporaryFile(delete=False)
@raprasad
raprasad / dv_add_file.md
Last active June 27, 2018 16:14
Dataverse: add a file to a Dataset using a persistentID
@raprasad
raprasad / gce_secrets_note.md
Created June 1, 2018 15:53
secrets from file; to env variable

example of putting file contents in a secrets

@cloudshell:~ (raven2-186120)$ echo husky > dogs.txt

@cloudshell:~ (raven2-186120)$  kubectl create secret generic test-env-variable --from-file=TEST_ENV_VARIABLE=./dogs.txt                                                      
secret "test-env-variable" created
@raprasad
raprasad / gce_notes.md
Last active May 25, 2018 16:56
slow test

one suggestion to get used to the process:

Don't change anything yet

branch from master

  1. add this to settings/base.py:
@raprasad
raprasad / schema_draft.json
Created May 11, 2018 17:33
schema draft 2
{
"type":"object",
"properties":{
"dataset":{
"type":"object",
"properties":{
"description":{
"type":"string"
},
"unitOfAnalysis":{
@raprasad
raprasad / variable_schema_d1.md
Last active May 11, 2018 12:15
Variable schema, initial draft (incomplete)

This describes a single variable within the variables section.

Note: The interpretation and tworavens sections are placeholders in this version

{
   "type":"object",
   "patternProperties":{
      "^[_a-zA-Z0-9]+$":{
         "type":"object",
@raprasad
raprasad / message_w_preprocess.md
Created May 4, 2018 17:14
Preprocess info is under "data"
{
   "success":true,
   "message":"Metadata retrieved",
   "data":{
      "self":{
         "description":"Generated by https://github.com/TwoRavens/raven-metadata-service",
         "created":"2018-05-04 13:12:12",
         "preprocess_id":1782,
         "version":1
def get_dataframe(input_file, start_idx=0, num_rows=100, delimiter=None, is_excel=False):
"""Retrieve a pandas dataframe based on format"""
try:
if is_excel:
dataframe = pd.read_excel(input_file) # add nrows, skiprows + lineterminator, if needed
else:
dataframe = pd.read_csv(input_file,
delimiter=delimiter) # add nrows, skiprows + lineterminator, if needed
return ok_resp(dataframe)
except pd.errors.ParserError as err_obj:
@raprasad
raprasad / response1-success.md
Created March 14, 2018 19:24
create pipelines resp 1
[{"responseInfo": {"status": {"code": "OK", "details": "Pipeline Submitted"}}, "progressInfo": "SUBMITTED", "pipelineId": "27a0f05d-fc46-4497-bc55-ecfd27e52e83"}, {"responseInfo": {"status": {"code": "OK", "details": "Pipeline Submitted"}}, "progressInfo": "SUBMITTED", "pipelineId": "22548ad8-b38c-4145-9656-f8e7d555c4b4"}, {"responseInfo": {"status": {"code": "OK", "details": "Pipeline Submitted"}}, "progressInfo": "SUBMITTED", "pipelineId": "2f7688a0-e121-463c-89bf-585b1848ff3d"}, {"responseInfo": {"status": {"code": "OK", "details": "Pipeline Submitted"}}, "progressInfo": "SUBMITTED", "pipelineId": "244ce547-5ec3-4f5e-a0c3-1a0efff6e269"}, {"responseInfo": {"status": {"code": "OK", "details": "Pipeline Submitted"}}, "progressInfo": "SUBMITTED", "pipelineId": "de7ed878-e475-454a-93b3-a6d1ba95757f"}, {"responseInfo": {"status": {"code": "OK", "details": "Pipeline Submitted"}}, "progressInfo": "SUBMITTED", "pipelineId": "27b5f716-8275-465d-819e-a6c135a14e47"}, {"responseInfo": {"status": {"code": "OK",