Skip to content

Instantly share code, notes, and snippets.

Avatar

Joost van Veen joostvanveen

View GitHub Profile
@joostvanveen
joostvanveen / top-20-user-agents-and-ips-from-directadmin-logs.sh
Created Feb 17, 2019
Loop though all rotated access logs and get the top 20 list of User AgentsThis can be used to identify bots
View top-20-user-agents-and-ips-from-directadmin-logs.sh
# Loop though all rotated Directadmin access logs and get the top 20 list of User Agents
# This can be used to identify bots
zcat /home/USER/domains/DOMAIN/logs/LOGNAME.tar.gz* | awk -F\" '{print $6}' | sort | uniq -c | sort -nr | head -20
# Get top 20 IP addresses
zcat /home/USER/domains/DOMAIN/logs/Feb-2019.tar.gz | awk -F\" '{print $1}' | sort | uniq -c | sort -nr | head -20
# Get top 20 most visited URLs
zcat /home/USER/domains/DOMAIN/logs/Feb-2019.tar.gz | awk -F\" '{print $4}' | sort | uniq -c | sort -nr | head -20
@joostvanveen
joostvanveen / .htaccess
Last active Apr 1, 2020
Force trailing slash for SEO purposes using htaccess 301 redirects (mod_rewrite) - but on GET requests only, to avoid losing POST data on a POST request to a URI without a trailing slash.
View .htaccess
# Force trailing slash for SEO purposes
RewriteEngine On
# For GET and HEAD requests only. We do not want to redirect posted forms and such, or we'll lose all POST data!
RewriteCond %{REQUEST_METHOD} ^(GET|HEAD)$
# Not for actual files. We do not want to redirect urls like test.jpg to test.jpg/
RewriteCond %{REQUEST_FILENAME} !-f
# Redirect to trailing slash if no slash is present in URI
@joostvanveen
joostvanveen / convert_to_utf8.php
Created Dec 31, 2012
Check to see if a string is UTF-8 and convert it if it's not. Accidentally posted this as anonymous before :(
View convert_to_utf8.php
<?php
function getUtf8String($string) {
if ( !isUtf8($string) )
return utf8_encode($string);
return $string;
}
function isUtf8($string) {
if ( function_exists("mb_check_encoding") ) {
return mb_check_encoding($string, 'UTF8');
@joostvanveen
joostvanveen / convert_windows_cp1252_to_latin1.php
Created Dec 31, 2012
Convert a Windows CP1252 string to Latin1. Converts the typical Windows (Word) crappy display characters like curly quotation marks, triple dots, etc. Windows-1252 or CP-1252 is a character encoding of the Latin alphabet, used by default in the legacy components of Microsoft Windows in English and some other Western languages. It is one version …
View convert_windows_cp1252_to_latin1.php
<?php
function transcribe_cp1252_to_latin1($cp1252) {
return strtr(
$cp1252,
array(
"\x80" => "e", "\x81" => " ", "\x82" => "'", "\x83" => 'f',
"\x84" => '"', "\x85" => "…", "\x86" => "+", "\x87" => "#",
"\x88" => "^", "\x89" => "0/00", "\x8A" => "S", "\x8B" => "<",
"\x8C" => "OE", "\x8D" => " ", "\x8E" => "Z", "\x8F" => " ",
"\x90" => " ", "\x91" => "`", "\x92" => "'", "\x93" => '"',
@joostvanveen
joostvanveen / csplit.sh
Created Mar 9, 2018
Split large SQL files into separate files for each table
View csplit.sh
## Split large SQL files into separate files for each table if every tabel starts with a 'DROP TABLE IF EXISTS' statement
csplit -k $PWD/filename.sql '/^DROP TABLE IF EXISTS .*/' '{900}'
## Split large SQL files into separate files for each table if every tabel starts with a 'CREATE TABLE' statement
csplit -k $PWD/filename.sql '/^CREATE TABLE .*/' '{900}'
@joostvanveen
joostvanveen / clean_magento_product_images.sh
Created Apr 25, 2018
Clean old product images cache and orphaned product images for Magento 1
View clean_magento_product_images.sh
# Delete all cached images older than 365 days
find ~/public/media/catalog/product/cache/* -type f -atime +365 -exec rm {} \;
# Remove product images that are not present in the database
magerun media:images:removeorphans
@joostvanveen
joostvanveen / textile_filter.php
Created Mar 3, 2013
A pre_replace filter that strip out all characters that are NOT letters or numbers or Textile Markup special characters. You can use this to filter user input. If you have any improvements, let me know!
View textile_filter.php
<?php
/**
* Filter input based on a whitelist. This filter strips out all characters that
* are NOT:
* - letters
* - numbers
* - Textile Markup special characters.
*
* Textile markup special characters are:
@joostvanveen
joostvanveen / mysql_alter_charset_and_collation_to_utf8.sql
Created May 29, 2019
Alter the charset and collation to UTF8 for a database and a table. Also inspects the default charset and collation for the entire database.
View mysql_alter_charset_and_collation_to_utf8.sql
-- Inspect the default charset and collation for your database
SELECT default_character_set_name FROM information_schema.schemata WHERE schema_name = 'MYDATABASE_NAME';
-- Alter the default charset and collation for your database
-- CAREFUL: when you change the charset for your database, make sure the data in your tables matches that character set.
-- If it is not, follow these steps: http://www.alphadevx.com/a/420-Converting-a-MySQL-database-from-latin1-to-utf8
ALTER DATABASE MYDATABASE_NAME CHARACTER SET utf8 COLLATE utf8_general_ci;
-- Alter the default charset and collation for your database
ALTER TABLE MYTABLE_NAME CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
@joostvanveen
joostvanveen / .gitignore
Last active Apr 10, 2019
Default gitignore entrances for IDEs, OS specific hidden files, tarballs and compressed files. Use this for new projects.
View .gitignore
############ Project specific rules ###############
# Project specific rules go here...
# To ignore a folder, add asterix -> folder/*
# This way, unignored files like .keep and .gitkeep will still be added to git
################# Global rules ####################
## Never ignore .gitignore
!.gitignore
## Dependencies
@joostvanveen
joostvanveen / delete_products_and_categories_from_magento.sql
Created Mar 25, 2016
Delete all products and categories from Magento 1.9.2
View delete_products_and_categories_from_magento.sql
-- Foreign key check will cascade-delete all related data
-- Delete all products
DELETE FROM catalog_product_entity;
-- Reset the autoincrement for products
ALTER TABLE catalog_product_entity AUTO_INCREMENT = 1;
-- Delete all categories
DELETE FROM catalog_category_entity WHERE `entity_id` > 2;