Skip to content

Instantly share code, notes, and snippets.

@derms
derms / support-collect.sh
Created August 26, 2022 13:15
TigerGraph support-collect
#!/bin/bash
# TigerGraph Support Collector v1.0
# Revised 3/17/2022
supportdir="support_collection_"$(date +%F_%H%M)
echo; read -p "Would you like to collect logs (this may take some time)? [y/n]"
if [[ $REPLY =~ ^[Yy]$ ]]; then
echo; echo "Collecting logs..."
gcollect -o ./$supportdir -c gpe,gse,gsql,rest -t 86400 collect
@derms
derms / fraud_detection_with_gnn_v4.ipynb
Last active May 20, 2022 09:12
Fraud_Detection_with_GNN_v4.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BEGIN
CREATE LOADING JOB load_job_bi_data FOR GRAPH dataops {
DEFINE FILENAME MyDataSource = "/home/tigergraph/tigergraph/data/gui/loading_data/bi_data.csv";
LOAD MyDataSource
TO VERTEX object VALUES(gsql_concat($0, ":", $1, ":", $3), $0, $1, $3, _, _)
, TO VERTEX object VALUES(gsql_concat($0, ":", $4, ":", $5), $0, $4, $5, _, _) WHERE $4 != "Logical Table"
, TO EDGE relates_to VALUES(gsql_concat($0, ":", $1, ":", $3), gsql_concat($0, ":", $4, ":", $5)) WHERE $4 != "Logical Table"
USING SEPARATOR = ","
@derms
derms / 1_convertModel.py
Last active February 29, 2020 12:49
Sample Sentiment model converted into ONNX and run in MarkLogic. Original Model : https://github.com/cocoa-ai/SentimentCoreMLDemo/blob/master/SentimentPolarity/Resources/SentimentPolarity.mlmodel
import coremltools
# Load a Core ML model
coreml_model = coremltools.utils.load_spec('SentimentPolarity.mlmodel')
# Convert the Core ML model into ONNX
onnx_model = onnxmltools.convert_coreml(coreml_model, 'Sentiment Polarity')
# Save as protobuf
onnxmltools.utils.save_model(onnx_model, 'SentimentPolarity.onnx')
@derms
derms / wait_for_http_200.sh
Created February 20, 2020 13:35 — forked from rgl/wait_for_http_200.sh
Wait for an HTTP endpoint to return 200 OK with Bash and curl
bash -c 'while [[ "$(curl -s -o /dev/null -w ''%{http_code}'' localhost:9000)" != "200" ]]; do sleep 5; done'
# also check https://gist.github.com/rgl/c2ba64b7e2a5a04d1eb65983995dce76
@derms
derms / build.gradle
Created February 14, 2020 09:25
Skip ml-gradle mlUndeploy when production envirornment
def isProd(taskName) {
def isProdEnv = "prod".equalsIgnoreCase(findProperty("environmentName"))
if (isProdEnv) {
println "Skipping ${taskName} as is prod envrionment"
}
return isProdEnv
}
mlUndeploy.onlyIf { !isProd("mlUndeploy") }
mlUndeploy.dependsOn.each { taskList ->
taskList.each { taskName ->
@derms
derms / 1_import_onnx_model.sjs
Last active February 27, 2020 15:25
Example of using ONNX Machine Learning Models in MarkLogic 10 (10.0-4 and above). BiDAF model: https://github.com/onnx/models/tree/master/text/machine_comprehension/bidirectional_attention_flow.
'use strict';
// Simple way to download the model and insert it into the database.
// Normally would use a tool like mlcp to do this. This only needs to be done once
declareUpdate()
//downloads the BiDAF model
let model = fn.subsequence(
xdmp.httpGet("https://onnxzoo.blob.core.windows.net/models/opset_9/bidaf/bidaf.onnx")
, 2, 1)
//inserts the BiDAF model into the database at the URI /onnx/model/bidaf.onnx
@derms
derms / clear-caches.xqy
Created July 18, 2019 12:32
marklogic clear caches
xquery version "1.0-ml";
(xdmp:list-cache-clear(),
xdmp:expanded-tree-cache-clear(),
xdmp:compressed-tree-cache-clear(),
xdmp:triple-cache-clear(),
xdmp:triple-value-cache-clear(),
xdmp:cache-status(xdmp:host()))
@derms
derms / sample.sh
Created May 22, 2019 15:04
Find files with certain extension(s) and find the size of those files
find . -name "*.pdf" -o -name "*.jpg" -o -name "*.png" -o -name "*.tif" -o -name "*.PNG" | sed 's| |\\ |g' | xargs ls -sk | sed 's/ .\/.*\//,/g'
@derms
derms / build.gradle
Last active April 19, 2019 11:12
MarkLogic - Multiple Delete Collections tasks
class CustomDeleteCollectionsTask extends com.marklogic.gradle.task.datamovement.DataMovementTask {
String[] collections
@org.gradle.api.tasks.TaskAction
void deleteCollections() {
project.ext.collections = (collections != null && collections.length > 0) ? collections.join(",") : null
runQueryBatcherJob(new com.marklogic.client.ext.datamovement.job.DeleteCollectionsJob())
}
}