rvanbruggen

Network Analysis of Shakespeare's plays

What do you do when a new colleague starts to talk to you about how they would love to experiment with getting a dataset about Romeo & Juliet into a graph? Yes, that's right, you get your graph boots on, and you start looking out for a great dataset that you could play around with. And as usual, one things leads to another (it's all connected, remember!), and you end up with this incredible experiment that twists, turns and meanders into something fascinating. That's what happened here too.

Finding a Data source

That was so easy. I very quickly located a Dataset on Kaggle that I thought would be really interesting. It's a comma-separated file, about 110k lines long and 10MB in size, that holds all the lines that Shakespeare wr

rvanbruggen

Quickly making sense of the news with Neo4j, APOC and Google's NLP engine

The setup

Search for articles on https://eventregistry.org/intelligence?type=articles

Download as .csv file

Or put it into a google sheet: https://docs.google.com/spreadsheets/d/1G1Dfh-6Ue2nK17CGoEMiNfIjhYO05h9EJaRX9sDUxJI/edit?usp=sharing, which you can download as CSV from https://docs.google.com/spreadsheets/d/1G1Dfh-6Ue2nK17CGoEMiNfIjhYO05h9EJaRX9sDUxJI/gviz/tq?tqx=out:csv&sheet=news

rvanbruggen

Blogposts on http://blog.bruggen.com:

Part 1/3: http://blog.bruggen.com/2021/03/part-13-wikipedia-clickstream-analysis.html

Part 2/3: http://blog.bruggen.com/2021/03/part-23-wikipedia-clickstream-analysis.html

Part 3/3: http://blog.bruggen.com/2021/03/part-33-wikipedia-clickstream-analysis.html

rvanbruggen

import spotipy
from neo4j import GraphDatabase
from spotipy.oauth2 import SpotifyClientCredentials, SpotifyOAuth

# ------------------------------------ Configuration parameters ------------------------------------ #
user_id = "<<YOUR SPOTIFY USER ID>>"                      # Spotify user ID.
client = "<<YOUR SPOTIFY CLIENT ID>>"                     # Spotify client ID.
secret = "<<YOUR SPOTIFY CLIENT SECRET>>"                 # Spotify client secret.
playlist_uri = "spotify:playlist:1eCqsRrwBAFc2lf5ZLGa5m"  # public playlist with songs to be sorted.
neo4j_url = "neo4j://localhost:7687"                      # bolt url of the neo4j database.

rvanbruggen

How to analyse my crazy 2020 calendar in Neo4j

• First create ics file: https://support.google.com/calendar/answer/37111?hl=en

• Convert ics to based on csv https://openicsfile.com/csv-convert.html

• Put that .csv file into the import directory of your Neo4j installation.

rvanbruggen

import spotipy
from neo4j import GraphDatabase
from spotipy.oauth2 import SpotifyClientCredentials, SpotifyOAuth

# ------------------------------------ Configuration parameters ------------------------------------ #
user_id = "YOUR USER_ID"                                  # Spotify user ID.
client = "YOUR CLIENT"                                    # Spotify client ID.
secret = "YOUR SECRET"                                    # Spotify client secret.
# playlist_uri = "spotify:playlist:1eCqsRrwBAFc2lf5ZLGa5m"  # LONG original public playlist with songs to be sorted.
playlist_uri = "spotify:playlist:1BTunw40NV9HgFpLXQ7hpm"  # SHORT original public playlist with songs to be sorted.

rvanbruggen

Using Zeppelin with Neo4j to analyse the FinCEN Files

Last week, we got another great and widely publicised case of Graph Databases' usefullness throw our way. The ICIJ published their FinCEN Files research, and on top of allowing you to explore the data on their website they also published an anonymised subset of the data as a series of CSV/JSON files. My friends and colleagues Michael Hunger, Will Lyon and the rest of the team, helped with the process of making this subset available as a Neo4j database (see this github repo), and there's even a super easy FinCEN Files Neo4j Sandbox that you can spin up in no time for some investigation fun.

So of course I had to take this data for a spin myself - it seems real

rvanbruggen

ICIJ FinCEN Files in Neo4j

Before you go any further in exploring the data, take a look at these documents to give you more context

• the ICIJ Investigation ** an explanation of what Suspicious Activity Reports (SARs) actually are
• Michael Hunger's work with the data
• the raw data to download

rvanbruggen

Exponential growth in Neo4j

With the current surges of the Covid-19 Pandemic globally, there is a huge amount of debate raging in our societies - everywhere. It’s almost as if the duality between left and right that has been dividing many political spectra in the past few years, is now also translating itself into a duality that is all about more freedom for the individual (and potentially - a higher spread of the SARS-CoV-2 virus), versus more restrictions for the individual. It’s such a difficult debate - with no clear definitive outcome that I know of. There’s just too many uncertainties and variations in the pandemic - I personally don’t see how you can make generic statements about it very easily.

One thing I do know though, is that very smart and loveable people, in my own social and professional circle and beyond, seem to be confused by some of the data. Very often, they make seemingly rational arguments about the numbers that are seeing - but ig

rvanbruggen

//Look at schema
call db.schema.visualization

//what is connected and how
match (n)-[r]->(m)
return distinct labels(n), type(r), labels(m)

//how many nodes of each label
match (n)
return labels(n), count(n)