magic-lantern

magic-lantern

magic-lantern

    # using a data.frame for output results
    # base time 26 seconds with 60k rows

    # this block takes 27 seconds with 100k rows
    # if (sum(df[i,]))
    #   df$ccc_flag[i] <- 1L

    # this one line takes abotut 10 seconds with 60k rows
    # df$ccc_flag[i] <- ccc_flag

magic-lantern

# install.packages('bigrquery')
library('bigrquery')

project <- "sandbox-nlp" # put your project ID here
sql <- "SELECT *
FROM `sandbox-nlp.MIMIC3_V1_4.ADMISSIONS`
LIMIT 100"
gbq_cursor <- bq_project_query(project, sql)
result <- bq_table_download(gbq_cursor)

magic-lantern

Introduction

Fast Healthcare Interoperability Resources (http://hl7.org/fhir) - FHIR – is a standards framework created by Health Level 7 (HL7). FHIR combines the features of HL7's previous health data interchange standards including HL7 v2 , HL7 v3 and CDA. FHIR improves on previous standards by using more modern web standards for data exchange, supports many programming languages, and promostes human readability.

Documentation

Start by reading through this first: "FHIR Drills" https://fhir-drills.github.io/index.html

• Python Client: http://docs.smarthealthit.org/client-py/
• Java Client: http://hapifhir.io/doc_rest_client.html

magic-lantern

Steps to upgrade r:

From linux command-line remove all existing R packages:

sudo apt-get update
sudo apt-get remove r-base r-base-dev r-base-core r-base-html r-cran-boot r-cran-class r-cran-cluster r-cran-codetools r-cran-foreign r-cran-kernsmooth r-cran-lattice r-cran-mass r-cran-matrix r-cran-mgcv r-cran-nlme r-cran-nnet r-cran-rpart  r-cran-spatial r-cran-survival r-doc-html r-recommended

Next, update /etc/apt/sources.list and add these lines (comment out old R version, add new pkg list):

magic-lantern

library(future.apply)

# set parallel_processing to TRUE if parallelization desired
parallel_processing <- TRUE
num_workers <- availableCores() # this option will automatically scale to fit current machine

# default plan for future.apply/future is sequential (no parallelization)
# plan(multiprocess) should pick recommended option based on OS
if (parallel_processing) {
  plan(multiprocess, workers = num_workers)

magic-lantern

import pandas as pd
import numpy as np
import multiprocessing
from multiprocessing import Pool

num_processes = multiprocessing.cpu_count()
# on some systems, these next 2 lines will give better count for CPU intensive tasks
# import psutil
# num_processes = psutil.cpu_count(logical=False)
num_partitions =   num_processes * 2 #smaller batches to get more frequent status updates

magic-lantern

{"name":"SethMac","device":1,"model":3,"type":"layout","data":{"layer0":[41,30,31,32,33,34,35,36,37,38,39,45,46,42,43,20,26,8,21,23,28,24,12,18,19,47,48,49,57,4,22,7,9,10,11,13,14,15,51,52,40,225,29,27,6,25,5,17,16,54,55,56,229,224,226,227,44,231,192,193,228],"layer1":[53,58,59,60,61,62,63,64,65,66,67,68,69,76,0,0,82,0,0,0,0,0,82,0,70,74,77,0,0,80,81,79,0,0,0,80,81,79,75,78,0,0,0,0,0,0,0,0,0,0,73,76,0,0,0,0,0,0,192,193,0],"layer2":[53,200,201,202,203,0,170,169,168,241,240,244,243,76,0,0,82,0,0,0,0,0,82,0,70,74,77,0,0,80,81,79,0,0,0,80,81,79,75,78,0,0,0,0,0,0,0,0,0,0,73,76,0,0,0,0,0,0,192,193,0],"taps":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,53,0,0,0,0,0,0,0,0,0,0,82,0,0,0,0,0,80,81,79]},"crc":"da16ca4b"}

magic-lantern

Setting up an SSH proxy on Windows

No administrative privileges required.

Steps:

1. Download Cygwin setup application from https://cygwin.com

2. Run Cygwin setup file

   • Select “Install For Just Me”