Skip to content

Instantly share code, notes, and snippets.

View emirkin's full-sized avatar

Eugene Mirkin emirkin

View GitHub Profile
@emirkin
emirkin / gist:3194028
Created July 28, 2012 17:07
Basic example for Android
BobikClient bobik = new BobikClient("YOUR_AUTH_KEY");
JSONObject request = new JSONObject();
for (String url : new String[]{"amazon.com", "google.com"})
request.accumulate("urls", url);
for (String query : new String[]{"//a/@href", "return $('.logo').length"})
request.accumulate("queries", query);
Job job = bobik.scrape(request, new JobListener() {
public void onSuccess(JSONObject scraped_data) {
@emirkin
emirkin / gist:3041523
Created July 3, 2012 18:17
Code to Scrape Drugstores
package com.example;
// Download Bobik SDK from http://usebobik.com/sdk
import android.util.Log;
import bobik.BobikClient;
import bobik.BobikHelper;
import bobik.Job;
import bobik.JobListener;
import org.json.JSONException;
@emirkin
emirkin / gist:3035521
Created July 2, 2012 20:30
Scraping Online Drug Stores
{
"cvs" => {
"Image" => "//div[@class='productSection1']//img[1]/@src",
"Title" => "//h4[@class='productBrand']//span[1]",
"Size" => "//span[@class='prodWeight']/preceding-sibling::text()[1]",
"Price" => "//*[@class='productPrice']",
"Link" => "//div[@class='productSection1']//img[1]/ancestor::a/@href"
},
"walgreens" => {
@emirkin
emirkin / gist:3031423
Created July 2, 2012 06:22
The gist of Pair Wise
package com.example;
import android.util.Log;
// Download Bobik SDK from http://usebobik.com/sdk
import bobik.BobikClient;
import bobik.Job;
import bobik.JobListener;
import org.json.JSONArray;
@emirkin
emirkin / gist:2859254
Created June 2, 2012 17:23
Scraping restaurants in Javascript
/*
* This example shows how to collect restaurant information and menus on the fly.
*/
// Instantiate Bobik client from Bobik SDK available at http://usebobik.com/sdk.
// You're welcome to link directly to the JS file. However we make no guarantees about keeping the link unchanged.
// Thus, you should do it only when you have a quick and immediate access to where this url is used (e.g. during development)
var bobik = new Bobik("YOUR_AUTH_TOKEN");
@emirkin
emirkin / gist:2728019
Created May 18, 2012 23:02
How to use Bobik in Javascript
/*
* Demonstrates how to scrape using different query types
*/
var urls = ['amazon.com', 'zynga.com', 'http://finance.google.com/', 'http://shopping.yahoo.com']
var queries = ["//th", "//img/@src", "return document.title", "return $('script').length", "#logo", ".logo"]
// Instantiate Bobik client from Bobik SDK available at http://usebobik.com/sdk.
// You're welcome to link directly to the JS file. However we make no guarantees about keeping the link unchanged.
@emirkin
emirkin / gist:2705804
Created May 15, 2012 22:57
How to use Bobik in Ruby
# Demonstrates synchronous scraping using Bobik
# To make this example asynchronous, simply change `true` to `false` in calling `scrape`.
client = Bobik::Client.new(:auth_token => YOUR_AUTH_TOKEN, :timeout_ms => 60000)
sample_data = {
urls: ['amazon.com', 'zynga.com', 'http://finance.yahoo.com/'],
queries: ["//th", "//img/@src", "return document.title", "return $('script').length"]
}