This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env bash | |
# sudo ln -s "$(pwd)/csv2datasette" /usr/bin/csv2datasette | |
# csv2datasette is meant to explore CSV data. It is not meant to create a sustainable DB. | |
# csv2datasette is a bash script which open CSV files directly in Datasette. It offers | |
# a number of options for reading and exploring CSV files, such as --stats, inspired by WTFCsv. | |
# | |
# `--stats` option includes, for each column: the column name, the number of unique values, | |
# the number of filled rows, the number of missing values, the mininmum value, the maximum value, | |
# the average, the sum, the shortest string, the longest string, the number of numeric values, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(function() { | |
'use strict'; | |
console.log("DS: start"); | |
// Part I. -- Add "CSV without limit" link | |
// document.getElementsByClassName("export-links")? | |
if (document.getElementsByClassName('show-hide-sql')[0] && document.getElementById('sql-editor') !== null) { | |
displayLinkCSVWithoutLimit(); | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Add "count" button aside the "Custom SQL query returning XX rows" title | |
// Clicking on the button counts and displays total number of rows | |
// Probably does not play well with complex queries (eg. sub-queries) | |
// Test if we're located in a query's page | |
const hide = document.getElementsByClassName('show-hide-sql')[0] || false; | |
if (hide) { | |
var sql_request = document.getElementById('sql-editor').value; | |
// Remove request's comments |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# get total requests by status code | |
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | |
# get top requesters by IP | |
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head | awk -v OFS='\t' '{"host " $2 | getline ip; print $0, ip}' | |
# get top requesters by user agent | |
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head | |
# get top requests by URL |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CREATE TABLE IF NOT EXISTS "all" ( | |
[code] TEXT, | |
[url] TEXT, | |
[creator] TEXT, | |
[created_t] INTEGER, | |
[created_datetime] TEXT, | |
[last_modified_t] INTEGER, | |
[last_modified_datetime] TEXT, | |
[product_name] TEXT, | |
[abbreviated_product_name] TEXT, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
code | |
creator | |
created_t | |
last_modified_t | |
product_name | |
abbreviated_product_name | |
generic_name | |
quantity | |
packaging | |
packaging_text |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
CLEAR='\033[0m' | |
RED='\033[0;31m' | |
function usage() { | |
if [ -n "$1" ]; then | |
echo -e "${RED}👉 $1${CLEAR}\n"; | |
fi | |
echo "Usage: $0 [-n number-of-people] [-s section-id] [-c cache-file]" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Install useful tools | |
# TODO: compléter | |
sudo apt install curl git htop iftop lynx mc recode -y | |
# TODO: csvkit | |
# TODO: docker | |
# TODO: VSCODE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys # to allow inputs from command line | |
import simplejson as json # to read json data; simplejson is easier to read nested data | |
import urllib.request # to make http request | |
# What's the issue? | |
# 1. A lot of data quality issues are not solved or owned. | |
# 2. The current lists are not efficient because they are mostly unknown by the community and we have to look at it to see what have to be done. | |
# The goal of this tool is to push quality errors to some volonteers. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// ==UserScript== | |
// @name Folksonomy Engine user script | |
// @description Add Folksonomy Engine UI to Open Food Facts web pages. | |
// @namespace openfoodfacts.org | |
// @version 2021-05-10T13:18 | |
// @include https://*.openfoodfacts.org/* | |
// @include https://*.openproductsfacts.org/* | |
// @include https://*.openbeautyfacts.org/* | |
// @include https://*.openpetfoodfacts.org/* | |
// @include https://*.pro.openfoodfacts.org/* |
NewerOlder