Socrata to AGOL

Instructions

1. Make sure you have ruby on your machine
2. git clone https://github.com/ajturner/arcgis-ruby.git
3. from the directory that repository was cloned into -> gem build arcgis.gemspec
4. gem install arcgis
5. edit the above file
  - replace username and pass
  - replace group with one you control
  - replace koop link
  - create a new key and register as a provider with koop
  - replace the URL variable with the socrata instance url
6. ruby registerSocrata.rb

registerSocrata.rb

%w{arcgis uri open-uri json}.each {|g| require g}

@username = 'user'
@password = 'pass'
@ago = "http://www.arcgis.com/sharing/rest/"
group = "b4bc0f9a091043b0b215c0db2c2f900f"
 
koop = "http://koop.dc.esri.com"
url = "http://finances.worldbank.org"
key = "worldbank_finances"
catalog = "#{url}/data.json"
 
 
response = open(catalog)
datasets = JSON.parse(response.read)['dataset']
 
# Register in ArcGIS Online
@online = Arcgis::Online.new(:host => @ago)
@online.login(:username => @username, :password => @password)
 
 
datasets.each do |dataset|
  unless dataset['identifier'].nil? || dataset['identifier'].length == 0
    puts "Adding #{dataset['identifier']}"
    # Create an item
    begin
    response = @online.item_add( 
                  :title => dataset['title'],
                  :description => dataset['description'],
                  :licenseInfo => dataset['license'],
                  :type => "Feature Service",
                  :folder => "MD_Socrata",
		              :url => "#{koop}/socrata/#{key}/#{dataset['identifier'].split('/').last}/FeatureServer/0",
                  :tags  => (dataset['keyword'].join(',') rescue nil)
                  )
    rescue
      puts $!
      redo
    end
    puts response

    begin
      response = @online.item_share(
          :id => response["id"], 
          :everyone => true, 
          :groups => group)
      puts response
    rescue
      puts $!
      redo
    end

  end
end

@dmfenton Thanks for this. I have started converting to python as it is really easy. One thing I am noticing though is that datasets like this:
https://data.wa.gov/Procurements-and-Contracts/OCIO-IT-Project-Oversight-Leadership-Details/3uf4-3kn2
that are tabular only and don't have geometry show up as layers in the JSON when they should show up as tables. I am wondering if this is something that we need to resolve in the socrata plugin for koop?

http://koop.dc.esri.com/socrata/wastate/3uf4-3kn2/FeatureServer
layers: [
{
id: 0,
name: "3uf4-3kn2",
parentLayerId: -1,
defaultVisibility: true,
subLayerIds: null,
minScale: 99999.99,
maxScale: 0
}
],
tables: [ ],

I don't think it causes any problems downstream in the Open Data app or AGOL. Still, I agree that it's something that should be fixed.

@sirws The problem you mentioned has been fixed and that fix has been deployed.

Awesome!