Skip to content

Instantly share code, notes, and snippets.

@404mike
404mike / gist:17bf6470428cf01d9cee
Created June 24, 2014 12:05
Remove white space and change file name to lowercase
#!/bin/bash
ls | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | tr '[A-Z]' '[a-z]' | sed 's/_-_/_/g'`
done
+------------------------------------------------------------------+--------------------+
| Publisher | Number of articles |
+------------------------------------------------------------------+--------------------+
| Evening Express | 3343 |
| Weekly Mail | 286 |
| The Cardiff Times | 193 |
| Flintshire Observer Mining Journal... | 141 |
| South Wales Daily News | 131 |
| Rhyl Record and Advertiser | 111 |
| The Aberystwith Observer | 107 |
+------------------------------------------------------------------+--------------------+
| Publisher | Number of articles |
+------------------------------------------------------------------+--------------------+
| Evening Express | 3343 |
| Weekly Mail | 286 |
| The Cardiff Times | 193 |
| Flintshire Observer Mining Journal... | 141 |
| South Wales Daily News | 131 |
| Rhyl Record and Advertiser | 111 |
| The Aberystwith Observer | 107 |
# For more options and information see
# http://www.raspberrypi.org/documentat...
# Some settings may impact device functionality. See link above for details
# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1
# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan
disable_overscan=1
@404mike
404mike / ingest.php
Last active July 31, 2018 10:50
ingest.php
<?php
/**
* Transform the json data to solr xml
* clean the data
* ingest into solr
*/
class Ingest {
@404mike
404mike / Import_all_data_to_solr.php
Created July 31, 2018 10:49
Import_all_data_to_solr
<?php
/**
* Ingest script to import all the data in the complete directory into solr
*/
include('ingest.php');
class Import_all_data_to_solr {
import json
import os
from os.path import exists
def loopJsonFiles():
directory = os.fsencode('data')
for file in os.listdir(directory):
filename = os.fsdecode(file)
if filename.endswith(".json"):