Skip to content

Instantly share code, notes, and snippets.

View bohdanszymanik's full-sized avatar

Bohdan Szymanik bohdanszymanik

1 follower · 7 following
  • Wellington, New Zealand
View GitHub Profile
@bohdanszymanik
bohdanszymanik / kfoundry2michlet.fsx
Created May 29, 2012 10:36
MichLet file generator from Kayak Foundry export
open System
open System.IO
let cSs = File.ReadAllLines(@"C:\Extras\kfoundry1.6.4\520x50a.txt")
(*
Raw form data exported from kfoundry with baseline at the waterline looks like this:
Form Position, X (mm), Y (mm)
...
Form at 52 cm, 60, 39.3333
@bohdanszymanik
bohdanszymanik / IncidentAnalysis.R
Last active December 18, 2015 12:09
Example usage of R data.table in an incident analysis
# data was closed incidents in a tab separated txt file that looked like this
# week category P1 P2 P3
# sucked up into R it formed a data frame like so
# 1 28/05/2012 some category1 NA NA 1
# 2 28/05/2012 some category2 NA 2 1
# 3 28/05/2012 some category3 1 NA 1
# 4 28/05/2012 some category4 NA NA 2
# as it happens, this data was manually collected from excel spreadsheets
# unfortunately the spreadsheets were for differing week ranges and there was some overlap with
@bohdanszymanik
bohdanszymanik / RTweetsAnalysis.R
Last active December 19, 2015 12:38
R Tweets Example
# download tweets using twitter search api, combine text and count +ve -ve words to get sentiment
# minor modifications to the code taken from
# http://jeffreybreen.wordpress.com/2011/07/04/twitter-text-mining-r-slides/
library(RCurl)
library(twitteR)
library(ROAuth)
requestURL <- "https://api.twitter.com/oauth/request_token"
accessURL = "http://api.twitter.com/oauth/access_token"
@bohdanszymanik
bohdanszymanik / azureTableScript.fsx
Created July 11, 2013 10:45
WindowsAzureTableStorage basic F# script sample
#r @"C:\wd\AzureTxnUploader\packages\WindowsAzure.Storage.2.0.6.0\lib\net40\Microsoft.WindowsAzure.Storage.dll"
#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.OData.5.2.0\lib\net40\Microsoft.Data.OData.dll"
#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.Edm.5.2.0\lib\net40\Microsoft.Data.Edm.dll"
#r @"C:\wd\AzureTxnUploader\packages\System.Spatial.5.2.0\lib\net40\System.Spatial.dll"
#r "Microsoft.WindowsAzure.ServiceRuntime"
#r "System.Data.Services.Client"
#r "System.Linq"
open System
@bohdanszymanik
bohdanszymanik / txnSimulator.fsx
Created July 16, 2013 04:41
Sample using both Windows Azure Table Storage and a bit of simulation with MathNet.Numerics.Statistics
//
// We're going to simulate some financial transactions and experiement with storing them in azure table storage
//
//#r "..\packages\Fog.0.1.3.1\Lib\Net40\Fog.dll"
//#r "..\packages\FogMyBuild\Debug\Fog.dll"
//#r "Microsoft.WindowsAzure.Diagnostics"
#r @"C:\wd\AzureTxnUploader\packages\WindowsAzure.Storage.2.0.6.0\lib\net40\Microsoft.WindowsAzure.Storage.dll"
//#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.Edm.5.5.0\lib\net40\Microsoft.Data.Edm.dll"
@bohdanszymanik
bohdanszymanik / readAzureTableStorage.fsx
Created July 23, 2013 07:25
F# script example retrieving data from Azure Table Storage - use case is an imaginary store of transactions in Json encoded document format indexed with a partitionkey of year-customer and rowkey of month-account
#r @"C:\wd\AzureTxnUploader\packages\WindowsAzure.Storage.2.0.6.0\lib\net40\Microsoft.WindowsAzure.Storage.dll"
#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.OData.5.2.0\lib\net40\Microsoft.Data.OData.dll"
#r @"C:\wd\AzureTxnUploader\packages\Microsoft.Data.Edm.5.2.0\lib\net40\Microsoft.Data.Edm.dll"
#r @"C:\wd\AzureTxnUploader\packages\System.Spatial.5.2.0\lib\net40\System.Spatial.dll"
#r "Microsoft.WindowsAzure.ServiceRuntime"
open System
open Microsoft.WindowsAzure
open Microsoft.WindowsAzure.Storage
@bohdanszymanik
bohdanszymanik / characterRecognition.fsx
Last active December 28, 2015 04:29
Kaggle numeric character recognition in F# using: 1. CsvFile typeprovider to get the data 2. vectors with cosine similarity to determine k nearest neighbours and 3. the ability to display characters with wpf using wpf Rectangles laid out onto a canvas. Based on the machine learning example from Mathias Brandewinder and the coding dojo here: http…
open System
#r @"../packages/FSharp.Data.1.1.10/lib/net40/FSharp.Data.dll"
#r @"c:\wd\MathDemo\packages\MathNet.Numerics.2.6.2\lib\net40\MathNet.Numerics.dll"
#r "../packages/MathNet.Numerics.FSharp.2.6.0/lib/net40/MathNet.Numerics.FSharp.dll"
open MathNet.Numerics.LinearAlgebra
open MathNet.Numerics.LinearAlgebra.Double
@bohdanszymanik
bohdanszymanik / zipLogReader.fsx
Created November 25, 2013 08:58
Work mate's (@LukeGumbley) suggestion - read log files in compressed, very useful for large log files - big data stuff. The reduction in IO makes parsing lines substantially quicker. The test.zip file used for the code below held 100 log files totalling 500MB raw and 16MB zipped and all lines were processed in a just a few seconds on a laptop.
#r "System.IO"
#r "System.IO.Compression"
#r "System.IO.Compression.FileSystem"
open System.IO
open System.IO.Compression
// open up streams from there
for entry in Compression.ZipFile.OpenRead(@"c:/temp/test.zip").Entries do
printfn "%s" entry.FullName
@bohdanszymanik
bohdanszymanik / sentiment.sql
Last active October 14, 2020 01:22
Sentiment analysis of text data with a plain old t-sql approach using tables for stop words, score words, custom phrases, and messages with trending over time. (Proved to be remarkably good in our work environment for highlighting low message scores on emails at times this was expected to occur.)
-- T-SQL based sentiment analysis on text following approach in http://dl.dropboxusercontent.com/u/4839225/twitter%20text%20mining%20with%20R.pdf
-- with trending over time
-- create scoreword table and populate from a list of word scores such as the opinion lexicon listed in http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html
CREATE TABLE [ScoreWords](
[word] [varchar](50) NOT NULL,
[score] [int] NOT NULL
) ON [PRIMARY]
-- create a table of stopwords and populate them with a suitable list eg check http://en.wikipedia.org/wiki/Stop_words
@bohdanszymanik
bohdanszymanik / SCOMDGML.xslt
Created December 4, 2013 02:42
Take a SCOM management pack and transform it to DGML to display as a graph inside visual studio. Helps to visualise your system definition models.
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://schemas.microsoft.com/vs/2009/dgml" version="1.0">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="/">
<DirectedGraph>
<Nodes>
<xsl:apply-templates select="ManagementPack/TypeDefinitions/EntityTypes/ClassTypes/ClassType" />
</Nodes>
<Links>
<xsl:apply-templates select="ManagementPack/TypeDefinitions/EntityTypes/RelationshipTypes/RelationshipType" />