Skip to content

Instantly share code, notes, and snippets.

View ilyakava's full-sized avatar

Ilya Kavalerov ilyakava

View GitHub Profile
require 'csv'
require 'ruby-progressbar'
past_results = {}
filename = '/Users/artsyinc/Downloads/OpenContentGRI_out.csv'
length = %x(wc -l #{filename}).to_i
CSV.open('../new_getty2.csv', 'w') do |w_line|
pb = ProgressBar.create(:title => 'slugify artists', :total => length)
w_line << ['FileName','ArtworkCreator','ArtworkDateCreated','ArtworkTitle','Description','Title','Type','FileSize', 'slug_guess_1', 'slug_guess_2']
CSV.foreach(filename) do |r_line|
artist = r_line[1]
@ilyakava
ilyakava / gist:9531867
Created March 13, 2014 16:33
Matplotlib pip error message
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h:76:1: error: duplicate interface definition for class 'NSObject'
@interface NSObject <NSObject> {
^
@ilyakava
ilyakava / gm4java_mult_im_sing_proc.scala
Last active August 29, 2015 14:01
gm4java - trying to get multiple images in 1 gm process
// Failed attempts to process several images within a single gm process
// some of these are silly, but I was desperate
val config = new GMConnectionPoolConfig()
val service = new PooledGMService(config)
val command = new GMBatchCommand(service, "convert")
// Q1. can we make GMBatchCommand run an operation with multiple images in it?
val a = Files.createTempFile("thumbnail", ".jpg").toString
val aa = Files.createTempFile("thumbnail", ".jpg").toString
library(pixmap)
library(ppls)
# use the working directory of where-ever you are: WD. Set this yourself
# Ex. for me: WD <- "/Users/artsyinc/Downloads/w4240"
data <- read.csv(paste(WD, "/image_matrix2.csv", sep = ""), header = F)
image.matrix <- matrix(unlist(data), nrow = nrow(data), ncol = ncol(data))
# the matrix is features x trials
# each feature is a pixel in an image, each feature is an image of a person
mean.face.v <- as.vector(apply(image.matrix, 2, mean))

Setup for heroku for snowplow scala apps

  1. obfuscate access keys in conf as: ${AWS_ACCESS_KEY_ID}
    • if you get a error like adf, make sure resolve() the config file like: ConfigFactory.parseFile(new File("conf/application.conf")).resolve()
  2. add addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "0.8.0-M1") to project/plugins.sbt and .settings(com.typesafe.sbt.SbtNativePackager.packageArchetype.java_application: _*) to wherever val project is defined (project/ScalaCollectorBuild.scala)
    • this provides the sbt stage task that heroku runs to compile
    • be careful to skip every other line in plugins.sbt
  3. add java.runtime.version=1.7 to system.properties in repo root
  4. add worker: ./target/universal/stage/bin/name-of-my-app --config myconf.conf to Procfile in repo root, check name of executable after sbt compile stage
  • don't use sh here, heroku runs Ubuntu with dash not bash
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.7</source>
<target>1.7</target>
<encoding>UTF-8</encoding>
</configuration>
#!/usr/bin/env ruby
# About:
# there are 2 common conversions that I find myself doing:
# markdown -> html for online blog posts with mathjax
# markdown -> pdf for pdfs via pandoc
# unfortunately, if the markdown contains LaTeX, then there are different ways to escape the
# delimiters, as well as the matrix new row symbol. http://docs.mathjax.org/en/latest/tex.html#tex-and-latex-math-delimiters
# usage: `tex2jax_delim true infile.md outfile.md`
@ilyakava
ilyakava / square_rejected.txt
Created September 27, 2014 17:08
Images where a rectangle was found 09/27/2014, but no rectangle were satisfactory
07-2.tif
11-109.tif
11-78-target.tif
11.78-target.tif
13-441.tif
13-443.tif
1983-147-v01-target.tif
1983-94-v01.tif
1985-586.tif
1987-149-v01.tif
@ilyakava
ilyakava / squareless.txt
Created September 27, 2014 17:09
Images where no rectangle was found at all 09/27/2014
04-18.tif
08-17-v01.tif
10-198-v01.tif
12-13.tif
17-53.tif
19-113ab-v01.tif
1983-143.tif
1983-207C.tif
1983-264-2.tif
1983-65.tif
@ilyakava
ilyakava / squareless_09_28_2014.txt
Created September 28, 2014 17:14
No squares found on final iteration of deframing. 5% of all.
04-18.tif
08-17-v01.tif
10-198-v01.tif
12-13.tif
13-442.tif
17-52-v01.tif
17-53.tif
19-113ab-v01.tif
1983-143.tif
1983-207C.tif