Skip to content

Instantly share code, notes, and snippets.

@pacarvalho
pacarvalho / python_trader_for_medium_article_v1.py
Last active August 31, 2022 06:49
Simple python script for trading currency pair on forex.com
from gcapi import GCapiClient # From https://github.com/rickykim93/gcapi-python
from gcapi.gcapi_tools import format_date # From https://github.com/rickykim93/gcapi-python
from datetime import datetime
from time import sleep
import numpy
def check_sell_criteria(prices):
# TODO: Write your algorithm to detect sell condition here
return False
@pacarvalho
pacarvalho / slurm-enqueue-job.yml
Last active March 23, 2024 13:40
Sample Github Action Workflow To Enqueue Job With SLURM Cluster
name: SLURM Enqueue Workflow
on:
push:
branches:
- 'master'
jobs:
enqueue:
runs-on: ubuntu-latest
@pacarvalho
pacarvalho / app.py
Created February 19, 2021 16:02
Simple Lambda Collaborative Filter for Recommendation Engine
# Library used for training of the model
# NOTE: We require the environment variable SURPRISE_DATA_FOLDER to designate the
# the directory used by Surprise. Else it will err due to the default Lambda home
# directory being read only.
from surprise import SVD
from surprise import Dataset
from surprise import Reader
import os # Get environment variables
import psycopg2 # PSQL connector library
import csv # Format we will export ratings as
@pacarvalho
pacarvalho / template.yaml
Created February 19, 2021 16:06
SAM Template for Collaborative Filter Lambda
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
collaborative-filtering
SAM project for collaborative-filtering
Imports data from SQL and exports results to S3
Parameters:
RdsPassword:
@pacarvalho
pacarvalho / prediction_for_new_user.py
Created February 19, 2021 16:45
Collaborative Filtering Annex to Make Predictions for an Unseen User
(...)
print('Making predictions for an unseen user...')
items_to_rate = [trainset.to_raw_iid(i) for i in trainset.all_items()]
# We will consider this user to be ID 0 since that is not a valid ID for our real users
user_id = 0
# Make predictions
predictions = []
for iid in items_to_rate:
@pacarvalho
pacarvalho / child-process-promise.js
Last active April 17, 2022 15:03
Spawn Promise Wrapped Child Process in NodeJS for AWS Lambda
const childProcess = require('child_process'),
spawnPromise = function (command, argsarray, envOptions) {
return new Promise((resolve, reject) => {
console.log('executing', command, argsarray.join(' '));
const childProc = childProcess.spawn(command, argsarray, envOptions || {env: process.env, cwd: process.cwd()})
const resultBuffers = [];
childProc.stdout.on('data', buffer => {
console.log(buffer.toString());
@pacarvalho
pacarvalho / s3-util.js
Created April 17, 2022 14:37
Utility Function to Download Files from S3
/*global module, require, Promise, console */
const aws = require("aws-sdk");
const fs = require("fs");
const s3 = new aws.S3();
const downloadFileFromS3 = function (bucket, fileKey, filePath) {
"use strict";
console.log("Downloading", bucket, fileKey, filePath);
return new Promise(function (resolve, reject) {
@pacarvalho
pacarvalho / index.js
Created April 17, 2022 14:48
NodeJS Lambda for Processing Video from S3 Event
const s3Util = require("./s3-util");
const childProcessPromise = require("./child-process-promise");
const path = require("path");
const os = require("os");
const fs = require("fs");
const https = require("https");
const OUTPUT_BUCKET = process.env.OUTPUT_BUCKET;
const VIDEO_MIME_TYPE = process.env.VIDEO_MIME_TYPE;
const LAMBDA_SHARED_SECRET = process.env.LAMBDA_SHARED_SECRET;
@pacarvalho
pacarvalho / template.yaml
Last active April 17, 2022 14:55
AWS SAM Template for Video Processing Lambda
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
SAM project for Medium article about uploading and processing videos
Parameters:
EnvironmentValue:
AllowedValues:
- "staging"
- "production"
@pacarvalho
pacarvalho / videos_controller.rb
Created April 17, 2022 14:57
Medium Article - Custom Backend VideosController
class Api::VideosController < Api::ApplicationController
skip_before_action :authenticate_user, only: %i[derivative]
# POST /api/videos
def create
@video = current_user.videos.new
if @video.save
render json: VideoSerializer.render(@video, view: :with_upload_details), status: :ok
else
render json: @video.errors, status: :bad_request
end