Skip to content

Instantly share code, notes, and snippets.

@jvilledieu
jvilledieu / Call_records_data_import
Last active November 12, 2018 17:53
Data importation script for call records analysis article
//Setup initial constraints
CREATE CONSTRAINT ON (a:PERSON) assert a.number is unique;
CREATE CONSTRAINT ON (b:CALL) assert b.id is unique;
CREATE CONSTRAINT ON (c:LOCATION) assert c.cell_tower is unique;
CREATE CONSTRAINT ON (d:STATE) assert d.name is unique;
CREATE CONSTRAINT ON (e:CITY) assert e.name is unique;
//Create the appropriate nodes
USING PERIODIC COMMIT 1000
LOAD CSV WITH HEADERS FROM "file:c:/Users/Jean/Downloads/call_records_dummy.csv" AS line
@jvilledieu
jvilledieu / import_trump_world.cql
Created January 16, 2017 09:03
Importing the Buzzfeed Trump World data into Neo4j (Based on Michael Hunger's work)
//Creating unique constraint
CREATE CONSTRAINT ON (o:Organization) ASSERT o.name IS UNIQUE;
//Importing the organizations and their connections
WITH 'https://docs.google.com/spreadsheets/u/1/d/1Z5Vo5pbvxKJ5XpfALZXvCzW26Cl4we3OaN73K9Ae5Ss/export?format=tsv&id=1Z5Vo5pbvxKJ5XpfALZXvCzW26Cl4we3OaN73K9Ae5Ss&gid=634968401' AS url, ['LOAN','LOBBIED','SALE','SUPPLIER','SHAREHOLDER','LICENSES','AFFILIATED','TIES','NEGOTIATION','INVOLVED','PARTNER'] as terms
load csv with headers from url as row fieldterminator '\t'
MERGE (a:Organization {name:row.`Organization A`})
MERGE (b:Organization {name:row.`Organization B`})
CREATE (a)-[:IS_CONNECTED_TO {type:row.Connection}]->(b);
@jvilledieu
jvilledieu / Paradise_enhanced.cql
Last active August 22, 2018 09:55
Queries to extend the paradise papers dataset with pagrank, community detection, geocoding, etc
//Check schema
CALL db.schema()
//Identify countries with limited # of adresses
MATCH (a:Address)
WITH a, a.countries as countries
RETURN countries, count(a) as score
ORDER BY score ASC
LIMIT 50
We can't make this file beautiful and searchable because it's too large.
company_permalink,company_name,company_category_list,company_market,company_country_code,company_state_code,company_region,company_city,acquirer_permalink,acquirer_name,acquirer_category_list,acquirer_market,acquirer_country_code,acquirer_state_code,acquirer_region,acquirer_city,acquired_at,acquired_month,acquired_quarter,acquired_year, price_amount ,price_currency_code
/organization/waywire,#waywire,|Entertainment|Politics|Social Media|News|,News,USA,NY,New York City,New York,/organization/magnify,Waywire Networks,|Curated Web|Video|,Curated Web,USA,NY,New York City,New York,17/10/2013,2013-10,2013-Q4,2013,,USD
/organization/fluff-friends,(fluff)Friends,,,,,,,/organization/social-gaming-network,SGN (Social Gaming Network),|Social Media|Games|,Games,USA,CA,Los Angeles,Beverly Hills,16/09/2008,2008-09,2008-Q3,2008,,USD
/organization/red,(RED),|Nonprofits|,Nonprofits,USA,NY,New York City,New York,/organization/nationstar-mortgage-holdings,Nationstar Mortgage Holdings,|Finance|,Finance,USA,TX,Dallas,Lewisville,0
//Creating companies
CALL apoc.load.json($url) YIELD value
WITH distinct value.home_domain as domain, value.home_name as name
merge (b:Business {domain:domain}) ON CREATE SET b.name = name;
//Creating more companies
call apoc.load.json($url) yield value
with distinct value.link_domain as domain, value.link_name as name
merge (b:Business {domain:domain}) ON CREATE SET b.name = name;
@jvilledieu
jvilledieu / import_all_transfers.cql
Created July 27, 2015 14:18
Jesus Barrasa script to import transfers
//Clean up the db
MATCH (n)
OPTIONAL MATCH (n)-[r]-()
DELETE n,r;
//Load Players
USING PERIODIC COMMIT
LOAD CSV FROM "file:c:/all_transfers.csv" AS row
MERGE (:Player {player_name: row[1], player_url: row[2], player_position: row[9]});
@jvilledieu
jvilledieu / transfer_early-summer_2015.csv
Created July 27, 2015 14:19
Data Jesus Barrasa scrapped on TransferMarkt about the early 2015 transfers
We can't make this file beautiful and searchable because it's too large.
"90/91","Aldair","/aldair/profil/spieler/4151","SL Benfica","Benfica","Portugal","AS Roma","AS Roma","Italy","Centre Back","24","6,30 Mill. £","6,30",1
"90/91","Thomas Häßler","/thomas-hassler/profil/spieler/553","1. FC Köln","1. FC Köln","Germany","Juventus FC","Juventus","Italy","Attacking Midfield","24","5,46 Mill. £","5,46",2
"90/91","Roberto Baggio","/roberto-baggio/profil/spieler/4153","AC Fiorentina","Fiorentina","Italy","Juventus FC","Juventus","Italy","Secondary Striker","23","5,43 Mill. £","5,43",3
"90/91","Karl-Heinz Riedle","/karl-heinz-riedle/profil/spieler/13806","SV Werder Bremen","Werder Bremen","Germany","SS Lazio","Lazio","Italy","Centre Forward","24","5,25 Mill. £","5,25",4
"90/91","Henrik Larsen","/henrik-larsen/profil/spieler/101330","Lyngby BK","Lyngby BK","Denmark","AC Pisa 1909","AC Pisa","Italy","Attacking Midfield","24","4,69 Mill. £","4,69",5
"90/91","Gheorghe Hagi","/gheorghe-hagi/profil/spieler/7939","Steaua Bucharest","Steaua","Romania","Real Madrid","Real Madrid","Spain","Attack
We can make this file beautiful and searchable if this error is corrected: No commas found in this CSV file in line 0.
Voter;Country;Name;Position;Nominee
Captain;Afghanistan;Amiri Islam;First_(5pt);Messi Lionel
Captain;Afghanistan;Amiri Islam;Second_(3pt);Cristiano Ronaldo
Captain;Afghanistan;Amiri Islam;Third_(1pt);Ibrahimovic Zlatan
Captain;Albania;Cana Lorik;First_(5pt);Cristiano Ronaldo
Captain;Albania;Cana Lorik;Second_(3pt);Robben Arjen
Captain;Albania;Cana Lorik;Third_(1pt);Mueller Thomas
Captain;Algeria;Bougherra Madjid;First_(5pt);Cristiano Ronaldo
Captain;Algeria;Bougherra Madjid;Second_(3pt);Robben Arjen
Captain;Algeria;Bougherra Madjid;Third_(1pt);Benzema Karim
We can't make this file beautiful and searchable because it's too large.
company_permalink,company_name,company_category_list,company_market,company_country_code,company_state_code,company_region,company_city,investor_permalink,investor_name,investor_category_list,investor_market,investor_country_code,investor_state_code,investor_region,investor_city,funding_round_permalink,funding_round_type,funding_round_code,funded_at,funded_month,funded_quarter,funded_year, raised_amount_usd
/organization/newsela,Newsela,|Education|,Education,USA,NY,New York City,New York,,,,,,,,,/funding-round/410430eccb35a53d483cdd0dc9f2fc8d,venture,A,24/09/2014,2014-09,2014-Q3,2014, 5 956 174
/organization/virtus-data-centres,VIRTUS Data Centres,|Software|,Software,GBR,,London,London,,,,,,,,,/funding-round/f7af4ec8bf735c375262ea349a028c8d,private_equity,,17/09/2014,2014-09,2014-Q3,2014, 81 216 295
/organization/chobolabs,CHOBOLABS,|Mobile Games|,Mobile Games,USA,CA,SF Bay Area,Palo Alto,,,,,,,,,/funding-round/bb93fc583b2ce8af17b1ca14c8a8151d,seed,,16/09/2014,2014-09,2014-Q3,2014, 1 300 000
/organization
@jvilledieu
jvilledieu / facebook.cql
Last active August 14, 2017 05:43
Import friends list scrapped from Facebook into Neo4j.
//-----------------------
//Import people
//-----------------------
CREATE CONSTRAINT ON (a:PEOPLE) ASSERT a.id IS UNIQUE;
USING PERIODIC COMMIT 2000
LOAD CSV WITH HEADERS FROM "file:c:/my-facebook-network.csv" AS line
FIELDTERMINATOR ','
WITH line