Skip to content

Instantly share code, notes, and snippets.

@saidimu
saidimu / gist:1024207
Created June 14, 2011 02:36
Generating URLs to crawl from outside a Scrapy spider
from scrapy import log
from scrapy.item import Item
from scrapy.http import Request
from scrapy.contrib.spiders import XMLFeedSpider
def NextURL():
"""
Generate a list of URLs to crawl. You can query a database or come up with some other means
Note that if you generate URLs to crawl from a scraped URL then you're better of using a
@mayoff
mayoff / gist:1138816
Last active December 8, 2023 22:00
AppleScript to make Google Chrome open/reload a URL
(* To the extent possible under law, Rob Mayoff has waived all copyright and related or neighboring rights to “AppleScript to make Google Chrome open/reload a URL”. This work is published from: United States. https://creativecommons.org/publicdomain/zero/1.0/ *)
tell application "Google Chrome"
activate
set theUrl to "http://tycho.usno.navy.mil/cgi-bin/timer.pl"
if (count every window) = 0 then
make new window
end if
set found to false
@cshoe
cshoe / logging.py
Created May 23, 2012 20:55
Python dictConfig Logging
DEV_LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'basic': {
'format': '%(asctime)-6s: %(name)s - %(levelname)s - %(message)s',
}
},
'handlers': {
'console': {
@theaboutbox
theaboutbox / .ackrc
Created September 20, 2012 21:49
My .ackrc
--type-add=css=.sass,.less,.scss
--type-add=ruby=.rake,.rsel,.builder,.thor
--type-add=html=.haml,.html.erb,.html.haml
--type-add=js=.js.erb,.coffee
--type-set=cucumber=.feature
--type-set=c=.c,.cpp,.ino,.pde,.h
--ignore-dir=vendor
--ignore-dir=log
--ignore-dir=tmp
--ignore-dir=doc
@gordonmeyer
gordonmeyer / GetTitle.py
Created January 2, 2013 23:59
This script is for use with Keyboard Maestro, as described at: http://www.gordonmeyer.com/2013/01/keyboard-maestro-make-link-with-title-from-clipboard.html It outputs the title of a webpage. The URL is passed via a Keyboard Maestro variable. This is a rather crude, but reliable, method of getting the title and does not require the installation o…
#! /usr/bin/python
"""
brute force get title of web page
usage: getTitle.py url
Gordon Meyer January 1, 2013
www.gordonmeyer.com
"""
# modules used
import urllib2, urlparse, string, os
@iambibhas
iambibhas / django-image-save.py
Created February 27, 2013 21:30
Programatically save ImageField in Django
import requests
from django.core.files import File
from django.core.files.temp import NamedTemporaryFile
def save_image_from_url(model, url):
r = requests.get(url)
img_temp = NamedTemporaryFile(delete=True)
@vitorgalvao
vitorgalvao / Get Title and URL.applescript
Last active June 14, 2024 05:12
AppleScript and JavaScript for Automation to get frontmost tab’s url and title of various browsers.
-- AppleScript --
-- This example is meant as a simple starting point to show how to get the information in the simplest available way.
-- Keep in mind that when asking for a `return` after another, only the first one will be output.
-- This method is as good as its JXA counterpart.
-- Webkit variants include "Safari", "Webkit", "Orion".
-- Specific editions are valid, including "Safari Technology Preview".
-- "Safari" Example:
tell application "Safari" to return name of front document
@floer32
floer32 / centos_python_env_setup
Last active May 2, 2022 03:47 — forked from stantonk/doit
CentOS 6: Install Python 2.7.4, pip, virtualenv, and virtualenvwrapper on CentOS (plus some bonus items at the end if you want). You should probably run with `sudo`.
#!/bin/bash
# Source: http://toomuchdata.com/2012/06/25/how-to-install-python-2-7-3-on-centos-6-2/
# Install stuff #
#################
# Install development tools and some misc. necessary packages
yum -y groupinstall "Development tools"
yum -y install zlib-devel # gen'l reqs
@natewalck
natewalck / example.py
Created May 21, 2013 15:43
Examples for Practical Python for Mac Admins
# Numbers
number1 = 7
number2 = .256
number3 = -1024
number1 + number2 + number3
number1 < number2
number2 == number3
# Strings
@adamjohnson
adamjohnson / publickey-git-error.markdown
Last active June 20, 2024 05:57
Fix "Permission denied (publickey)" error when pushing with Git

"Help, I keep getting a 'Permission Denied (publickey)' error when I push!"

This means, on your local machine, you haven't made any SSH keys. Not to worry. Here's how to fix:

  1. Open git bash (Use the Windows search. To find it, type "git bash") or the Mac Terminal. Pro Tip: You can use any *nix based command prompt (but not the default Windows Command Prompt!)
  2. Type cd ~/.ssh. This will take you to the root directory for Git (Likely C:\Users\[YOUR-USER-NAME]\.ssh\ on Windows)
  3. Within the .ssh folder, there should be these two files: id_rsa and id_rsa.pub. These are the files that tell your computer how to communicate with GitHub, BitBucket, or any other Git based service. Type ls to see a directory listing. If those two files don't show up, proceed to the next step. NOTE: Your SSH keys must be named id_rsa and id_rsa.pub in order for Git, GitHub, and BitBucket to recognize them by default.
  4. To create the SSH keys, type ssh-keygen -t rsa -C "your_email@example.com". Th