mdamien

List all the broken links on your website

Requirements:

python3 and scrapy (pip install scrapy)

Usage

• scrapy runspider -o items.csv -a site="https://yoursite.org" 1spider.py
• python3 2format_results.py

mdamien

cfhdojbkjhnklbpkdaibdccddilifddb Adblock Plus 10000000
gighmmpiobklfepjocnamgkkbiglidom AdBlock 10000000
efaidnbmnnnibpcajpcglclefindmkaj Adobe Acrobat 10000000
gomekmidlodglbbmalcneegieacbdmki Avast Online Security 10000000
eofcbnmajmjmplflapaojjnihcjkigck Avast SafePrice 10000000
chfdnecihphmhljaaejmgoiahnihplgn AVG Web TuneUp 10000000
flliilndjeohchalpbbcdekjklbdgfkk Avira Browser Safety 10000000
gpdjojdkbbmdfjfahjcgigfpmkopogic Pin It Button 10000000
lifbcibllhkdhoafpjfnlhfpfgnpldfl Skype 10000000
mallpejgeafdahhflmliiahjdpgbegpk FromDocToPDF 9298905

mdamien

[].slice.call(document.querySelectorAll('.replies-to .stream-items:first-child .ProfileTweet-action--favorite .ProfileTweet-actionButton .ProfileTweet-actionCountForPresentation'))
  .map(x => parseInt(x.textContent))
  .filter(x=>x)
  .sort((a, b) => b - a);

// to paste into the dev console when viewing a tweet replies
// output is the sorted list of number of likes for each reply

mdamien

{"A Discriminative Latent Variable Model for Online Clustering":["Rajhans Samdani"],"Affinity Weighted Embedding":["Ron Weiss","Hector Yee"],"Applications of Maximum Entropy Rankers to Problems in Spoken Language Processing":["Richard Sproat","Keith Hall"],"Asynchronous Stochastic Optimization for Sequence Training of Deep Neural\n Networks":["Georg Heigold","Erik McDermott","Vincent Vanhoucke","Andrew Senior","Michiel Bacchiani"],"Autoregressive Product of Multi-frame Predictions Can Improve the Accuracy of\n Hybrid Models":["Vincent Vanhoucke"],"Bayesian Sampling Using Stochastic Gradient Thermostats":[],"Bridging Text and Knowledge with Frames":["Srini Narayanan"],"Cicada: Predictive Guarantees for Cloud Network Bandwidth":[],"Corporate learning at scale: Lessons from a large online course at Google":["Mehryar Mohri","Afshin Rostamizadeh","Umar Syed"],"DaMN – Discriminative and Mutually Nearest: Exploiting Pairwise Category\n Proximity for Video Action Recognition":["Rahul\n Sukthankar"],"Deep Convolutiona

mdamien

Log.d(TAG, "API CALL: POST "+url);
OutputStream output = null;
try {
	String query = "furnail_name=hell&hi=2";//use URLEncode here
	URLConnection connection = new URL("https://your-url.com/action/save").openConnection();
	String authString = "Basic " + Base64.encodeToString((username + ":" + password).getBytes(),Base64.NO_WRAP);
	connection.setRequestProperty("Authorization", authString); 
	connection.setDoOutput(true);
	connection.setRequestProperty("Accept-Charset", "UTF-8");
	connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + "UTF-8");

mdamien

tweets.csv comes from `twint -u dam_io --csv -o tweets.csv`

mdamien

document.querySelectorAll('.list-article-consommation').forEach(article => {
  count = 1
  article.querySelectorAll('.content p').forEach(el => {
   if (!el.innerText.trim()) return
   el.innerHTML = '<i>('+ count +')</i> ' + el.innerHTML
   count += 1 
  })
})

mdamien

s.primas@senat.fr
c.morin-desailly@senat.fr
j.durain@senat.fr
a.richard@senat.fr
c.deroche@senat.fr
h.marseille@senat.fr
m.antiste@senat.fr
g.patient@senat.fr
r.delpicchia@senat.fr
p.schillinger@senat.fr

mdamien

var links = document.querySelectorAll('.mediaIndex a')
var player = document.querySelector('#html5_player')

var results = []

var i = 0;
function next() {
	if (links[i]) {
		links[i].click()
		setTimeout(() => {

mdamien

import glob
import os
import sys
from pathlib import Path

from lys import L, raw, render
from mwclient import Site

from tlfp.tools.common import open_json