abmmhasan

#Spider Websites with Wget – 20 Practical Examples

Wget is extremely powerful, but like with most other command line programs, the plethora of options it supports can be intimidating to new users. Thus what we have here are a collection of wget commands that you can use to accomplish common tasks from downloading single files to mirroring entire websites. It will help if you can read through the wget manual but for the busy souls, these commands are ready to execute.

1. Download a single file from the Internet
wget http://example.com/file.iso

2. Download a file but save it locally under a different name
wget ‐‐output-document=filename.html example.com

abmmhasan

-- Import(FROM) / Export(TO) CSV file from/into a table
-- Ref: https://www.postgresql.org/docs/current/sql-copy.html
-- If processing all columns, no need column specification
-- If CSV file don't include header, remove 'HEADER' from below query
-- For Export, can also specify Query, instead of Table & Column name
COPY table_name (column_1, column_2, column_3, column_5) 
[FROM/TO] 'csv_file_location' DELIMITER ',' CSV HEADER QUOTE '"' ESCAPE '"'

-- Dump database on remote host to file
-- Ref: https://www.postgresql.org/docs/current/app-pgdump.html

thefranke

The RSS Endpoint List

Please refer to this blogpost to get an overview.

Replace *-INSTANCE with one of the public instances listed in the scrapers section. Replace CAPITALIZED words with their corresponding identifiers on the website.

Social Media

Twitter

anvk

--- PSQL queries which also duplicated from https://github.com/anvk/AwesomePSQLList/blob/master/README.md
--- some of them taken from https://www.slideshare.net/alexeylesovsky/deep-dive-into-postgresql-statistics-54594192

-- I'm not an expert in PSQL. Just a developer who is trying to accumulate useful stat queries which could potentially explain problems in your Postgres DB.

------------
-- Basics --
------------

-- Get indexes of tables

ssaadh

#Spider Websites with Wget – 20 Practical Examples

Wget is extremely powerful, but like with most other command line programs, the plethora of options it supports can be intimidating to new users. Thus what we have here are a collection of wget commands that you can use to accomplish common tasks from downloading single files to mirroring entire websites. It will help if you can read through the wget manual but for the busy souls, these commands are ready to execute.

1. Download a single file from the Internet
wget http://example.com/file.iso

2. Download a file but save it locally under a different name
wget ‐‐output-document=filename.html example.com

ryanwinchester

<?php


/**
* Check if a given ip is in a network.
*
* @see https://gist.github.com/ryanwinchester/578c5b50647df3541794
*
* @param  string $ip     IP to check in IPV4 format eg. 127.0.0.1
* @param  string $range  IP/CIDR netmask eg. 127.0.0.0/24, also 127.0.0.1 is accepted and /32 assumed

samarpanda

<?php

// from http://php.net/manual/en/function.filesize.php
function formatBytes($bytes, $precision = 2) { 
    $units = array('B', 'KB', 'MB', 'GB', 'TB'); 
   
    $bytes = max($bytes, 0); 
    $pow = floor(($bytes ? log($bytes) : 0) / log(1024)); 
    $pow = min($pow, count($units) - 1); 
   

zziuni

# source : http://code.google.com/p/natvpn/source/browse/trunk/stun_server_list
# A list of available STUN server.

stun.l.google.com:19302
stun1.l.google.com:19302
stun2.l.google.com:19302
stun3.l.google.com:19302
stun4.l.google.com:19302
stun01.sipphone.com
stun.ekiga.net