Skip to content

Instantly share code, notes, and snippets.

View Jeffallan's full-sized avatar
🏠
Working from home

Jeffallan

🏠
Working from home
View GitHub Profile
@tomschlick
tomschlick / Environments to bypass .htpasswd protection
Created November 8, 2009 08:30
This allows you to set certain urls/domains that are allowed to bypass a htpasswd protection layer, which is very useful for multiple environment setups (developement, staging, production)
#allows a single uri through the .htaccess password protection
SetEnvIf Request_URI "/testing_uri$" test_uri
#allows everything if its on a certain host
SetEnvIf HOST "^testing.yoursite.com" testing_url
SetEnvIf HOST "^yoursite.com" live_url
Order Deny,Allow
AuthName "Restricted Area"
AuthType Basic
@abarringer
abarringer / gist:846621
Created February 27, 2011 22:24
Convert XML to SQLite
require 'rubygems'
require 'sqlite3'
require 'rexml/document'
include REXML
xml =<<XML
<records>
<entry><name>abc</name><qty>3</qty></entry>
<entry><name>def</name><qty>5</qty></entry>
<entry><name>ghi</name><qty>2</qty></entry>
@chrisguitarguy
chrisguitarguy / xml.py
Created October 21, 2011 21:18
Parse an XML sitemap with Python, requests and BeautifulSoup
from __future__ import with_statement # we'll use this later, has to be here
from argparse import ArgumentParser
import requests
from BeautifulSoup import BeautifulStoneSoup as Soup
def parse_sitemap(url):
resp = requests.get(url)
# we didn't get a valid response, bail
@catawbasam
catawbasam / pandas_dbms.py
Last active May 26, 2024 05:32
Python PANDAS : load and save Dataframes to sqlite, MySQL, Oracle, Postgres
# -*- coding: utf-8 -*-
"""
LICENSE: BSD (same as pandas)
example use of pandas with oracle mysql postgresql sqlite
- updated 9/18/2012 with better column name handling; couple of bug fixes.
- used ~20 times for various ETL jobs. Mostly MySQL, but some Oracle.
to do:
save/restore index (how to check table existence? just do select count(*)?),
finish odbc,
@zenideas
zenideas / existing code to git repo
Created October 1, 2012 09:52
Adding existing source to remote git repo
If you've got local source code you want to add to a new remote new git repository without 'cloning' the remote first, do the following (I often do this - you create your remote empty repository in bitbucket/github, then push up your source)
1. Create the remote repository, and get the URL such as git://github.com/youruser/somename.git
2. If your local GIT repo is already set up, skips steps 2 and 3
3. Locally, at the root directory of your source, git init
4. Locally, add and commit what you want in your initial repo (for everything,
git add .
@abn
abn / mongodb-array-to-doc
Last active January 30, 2018 09:01
Mongodb: convert a an array into a sub-document. Eg: { "field" : ["v1", "v2", ...], 'date' : $DATE } to { "field" : { "v1" : $DATE, "v2" : $DATE, ... } , 'date' : $DATE}
db.collection.find({ field : { $exists : true } }).forEach( function (x) {
var temp = {}
var count = 0
for (var i in x.field) {
if (typeof x.field[i] == "string") {
// make sure index is a string
temp[x.field[i]] = x.date;
count++;
}
@tzermias
tzermias / pipelines.py
Last active March 14, 2024 23:35
Scrapy MySQL pipeline. Just a mirror to the asynchronous MySQL pipeline. Copy-paste it directly to pipelines.py. Database credentials are stored in settings.py. Based on http://snipplr.com/view/66986/
import MySQLdb.cursors
from twisted.enterprise import adbapi
from scrapy.xlib.pydispatch import dispatcher
from scrapy import signals
from scrapy.utils.project import get_project_settings
from scrapy import log
SETTINGS = get_project_settings()
@mobilemind
mobilemind / git-tag-delete-local-and-remote.sh
Last active July 2, 2024 00:02
how to delete a git tag locally and remote
# delete local tag '12345'
git tag -d 12345
# delete remote tag '12345' (eg, GitHub version too)
git push origin :refs/tags/12345
# alternative approach
git push --delete origin tagName
git tag -d tagName
'Put this function in new/existing MS-Access module.
'
' Version History:
' 2012-03-16 - First version http://en.latindevelopers.com/ivancp/2012/ms-access-to-mysql-with-relationships/
'
' 2014-02-09 - Seamus Casey
' a modification to Ivan's handy Access to MySQL relationship/constraint generator
'
' changes include:
' 1) skip Access system tables (TableDefAttributeEnum.dbSystemObjec)
@mywarr
mywarr / AccessDump.py
Last active April 18, 2024 19:14
use mdbtools to convert .mdb to .sqlite and .csv
#!/usr/bin/env python
#
# AccessDump.py
# A simple script to dump the contents of a Microsoft Access Database.
# It depends upon the mdbtools suite:
# http://sourceforge.net/projects/mdbtools/
import sys, subprocess, os
DATABASE = sys.argv[1]