Skip to content

Instantly share code, notes, and snippets.

View igolden's full-sized avatar
🎯
Focusing

Ian Golden igolden

🎯
Focusing
View GitHub Profile

Using CasperJS to scrape an infinite scroll page

This is a short script to scrape an infinite scroll page and write the resulting HTML to a file

You'll need to install CasperJS; on a Mac that will involve brew install casperjs --devel (if you don't have Homebrew, you can read how to do so here)

To run the script, you can use casperjs scrape.js --ssl-protocol=any from the command line.

/** @jsx React.DOM */
var SVGComponent = React.createClass({
render: function() {
return this.transferPropsTo(
<svg>{this.props.children}</svg>
);
}
});
@denji
denji / nginx-tuning.md
Last active May 21, 2024 18:18
NGINX tuning for best performance

Moved to git repository: https://github.com/denji/nginx-tuning

NGINX Tuning For Best Performance

For this configuration you can use web server you like, i decided, because i work mostly with it to use nginx.

Generally, properly configured nginx can handle up to 400K to 500K requests per second (clustered), most what i saw is 50K to 80K (non-clustered) requests per second and 30% CPU load, course, this was 2 x Intel Xeon with HyperThreading enabled, but it can work without problem on slower machines.

You must understand that this config is used in testing environment and not in production so you will need to find a way to implement most of those features best possible for your servers.

@datenimperator
datenimperator / sqlite.rb
Created November 22, 2013 16:19
Speed up your Rails sqlite database for large dataset. This should be a Rails initializer, goes into config/initializers.
if ::ActiveRecord::Base.connection_config[:adapter] == 'sqlite3'
if c = ::ActiveRecord::Base.connection
# see http://www.sqlite.org/pragma.html for details
# Page size of the database. The page size must be a power of two between 512 and 65536 inclusive
c.execute 'PRAGMA main.page_size=4096;'
# Suggested maximum number of database disk pages that SQLite will hold in memory at once per open database file
c.execute 'PRAGMA main.cache_size=10000;'
@schickling
schickling / Rakefile
Last active January 31, 2024 23:00
Activerecord without Rails
require "active_record"
namespace :db do
db_config = YAML::load(File.open('config/database.yml'))
db_config_admin = db_config.merge({'database' => 'postgres', 'schema_search_path' => 'public'})
desc "Create the database"
task :create do
ActiveRecord::Base.establish_connection(db_config_admin)
@nicolashery
nicolashery / environment-variables-jekyll-templates.md
Last active January 22, 2023 15:56
Make environment variables available in Jekyll Liquid templates

Environment variables in Jekyll templates

This is one way to pass some data (API tokens, etc.) to your Jekyll templates without putting it in your _config.yml file (which is likely to be committed in your GitHub repository).

Copy the environment_variables.rb plugin to your _plugins folder, and add any environment variable you wish to have available on the site.config object.

In a Liquid template, that information will be available through the site object. For example, _layouts/default.html could contain: