phillyrb-2010-12-14-notes.txt

trotter spoke about chloe
brian spoke about ruby shadow tables
matt showed collavoce

Houdini:

easy mechanical turk integrtion

Jameson used MT for GreenKonnect, Chris considered a StackOverflow type
thing(?), ShareUrMeal uses Houdini to verify meal photos are of meals

Mechanical turk is a marketplace for work to get simple tasks done in realtime,
Houdini sits on top an provides qualified resutls, 500K workers, accessible via
API, artificial artificial intelligence

use cases: moderation, categorization, ads, audio transcription/video
captioning, claenup/proofreading, OCR, research/contaxt info/document search

Site called feedback army may be built on turk, lots of A/B testing services

turk overview: workers and requestors; HIT's; Questions; Assignments;
QUalifications (of workers, or via qualifying tasks);Notifications (from
amazon, when someones completed, accepted, etc);Blocking Workers (banning
particular users from completing your tasks, only global for account, not
specific to projects)

Examples of worker view, including details on payment, number available, who
requestor is, qualification test status/present

Creating HITs via WYSISYG or APi, WYSI - Easy, good for testing, no intration;
API - more flexible, complicated, programmtic integration

Example WYSIQYG screenshot, template name, title, description, keywords, time
alloted, HIT expiration, qulaifications, worker payment, number o assignmetns
per hit, automatic approval time, hit approval rate of X%.

HIT design layout with template variables.

API - HIT Questions, XML or External Question (link to outside pagem use JS,
whatever you want).  ShareUrMEal example HIT w/ instructions, picture,
selection, stuffed into an iFrame on Amazon

Turk API libraries: ruby-aws (rubyforge/projects/ruby-aws, rTurk
github/mdp/rturk, Turkee github/aantix/turkee

Houdini - simple, qualified results, automatic, trusted workers

Housini APIs - Generic & Optimized, Gen: Simple, Exact, Multi-answer;
Optimized: Image Moderation, Ad Moderation, Data Reconcilaition, Data
Extraction, ...

Simple API - one answer per task; cheap; image moderation, low profile content
(?)

Example simple API psuedocode: template.render(houdini.erb image: => url)
HTTP.post :identifier, :price, :title, :for,_html, ...

Exact API & example (missed) ASK MUTLIPLE TURKERS SAME QUESTION, ALL ANSWERS
RETURNED

Image Moeration API, no need ot build form_html, just send image url,
optionally send cusotm...notes(?)

Demo time!...  HotOrNot for LOLCats, CatOrNot...  defined moderate_image methof
on Image class, define postback_url, instructions, possible answers, api key,
image_url

Takes a min or so to get response back from Amazon, but showed up in sinatra
log eventually (twice as he resubmitted)

Admin Console: Review Work, perform "trusted" tasks, worker insights: avg
completion time, # of workers,; remove bad performing workers from specific
task groups

getting started, no charge for now: get an api key (houdini/sign-up), tunnlr

pricing example, $20/200 images/month, $99/1500 images

Other solutions, crowdflower, microtask, crowdcloud

Houdini docs: houdini/documentation, mturk docs