vecna

openapi: 3.0.3
info:
  title: Makhno API design 
  description: |-
   The endpoint sequence described here is part of the Makhno suite. The purpose is to define a protocol by which a client can request reachability checks from our infrastructure.
   
   ## There are some reserved words that we use in this document:
   
   `target URL`: this is the URL of a social media platform that we want to monitor. the subjects covered in this framework, are, for the most part. `target URLs`. they can be youtube videos or facebook posts. we are talking about a social media platform URL (not all of them are supported, of course, and different versions prograd in supporting more and more of them)
   

vecna

vecna

[{
  "url":"https://accounts.google.com/",
  "name":"login",
  "delay": 10000
}, {
  "url":"https://www.youtube.com/",
  "name":"home",
  "delay": 3000
}, {
  "url":"https://www.youtube.com/results?search_query=election+results",

vecna

[
  "https://www.pornhub.com",
  "https://www.pornhub.com",
  "https://www.pornhub.com",
  "https://www.pornhub.com",
  "https://www.pornhub.com",
  "https://www.pornhub.com",
  "https://www.pornhub.com",
  "https://www.pornhub.com",

vecna

[ "https://www.pornhub.com/",
  "https://www.pornhub.com/video?c=722",
  "https://www.pornhub.com/",
  "https://www.pornhub.com/",
  "https://www.pornhub.com/video?c=722",
  "https://www.pornhub.com/",
  "https://www.pornhub.com/",
  "https://www.pornhub.com/video?c=722",
  "https://www.pornhub.com/",
  "https://www.pornhub.com/",

vecna

Claudio Agosti is a technologist working in privacy, security, and software engineering for 20 years. Since 2016 is leading Tracking Exposed, a small team of technologists, digital activists, and researchers, to analyze platforms personalization algorithms. Hia goal is enable individuals to control the Automated Decision Making that affects them.

Before he was in TacticalTech (working on https://trackography.org/) in GlobaLeaks, and in the previous decade working as consultant in the digital security industry.

As additional reference:

• An article/profile about Claudio and Tracking Exposed.
• My poorly maintained linkedin page: https://www.linkedin.com/in/vecna/
• A few of my academic publications

vecna

Assuming there is a directory in which files get copied, the scripts above would

• check the format first with the expected one (which can be the format of fbcrawl or webXray, at the moment)
• perform mongoimport of the CSV

We need files with this file format: nameofwhatitis_date.csv

• the script removed everything from the 1st underscore (_) till the end
• the nameofwhatitis is used as column name, and would be used to identify the data collected.
• for fbcrawl, it should be the name of the page
• for webXray, it should be the name of the country analyzed, or the name of the site analyzed

vecna

[                                                                       
{                                                                       
        "savingTime" : "1980-05-13T00:01:10",                           
        "id" : "69696969696969aaaac0776da461e87a51e00003",              
        "userId" : 1,                                                   
        "impressionId" : "aaaaaaaaaaaaf4ef8d76216284505d236f62bdae",    
        "timelineId" : "11111159bca8a7663de2fa9b4d101c16b387ed18",      
        "html": ""                                                      
},                                                                      
{                                                                       

vecna

[{
        "visibility" : "public",
        "timelineId" : "11111159bca8a7663de2fa9b4d101c16b387ed18",
        "id" : "aaaaaaaaaaaaf4ef8d76216284505d236f62bdae",
        "userId" : 1,
        "impressionOrder" : 1,
        "impressionTime" : "1980-05-13T00:01:00Z",
        "htmlId" : "69696969696969aaaac0776da461e87a51e00003"
},
{

vecna

[{
        "startTime" : "1980-05-13T00:00:00Z",
        "userId" : 1,
        "geoip" : "GB",
        "id" : "11111159bca8a7663de2fa9b4d101c16b387ed18"
},
{
        "startTime" : "1980-05-13T00:00:00Z",
        "userId" : 2,
        "geoip" : "IT",