Skip to content

Instantly share code, notes, and snippets.

View dovy's full-sized avatar

Dovy Paukstys dovy

158 followers · 10 following
View GitHub Profile
@ololobus
ololobus / Spark+ipython_on_MacOS.md
Last active November 22, 2022 22:24
Apache Spark installation + ipython/jupyter notebook integration guide for macOS

Apache Spark installation + ipython/jupyter notebook integration guide for macOS

Tested with Apache Spark 2.1.0, Python 2.7.13 and Java 1.8.0_112

For older versions of Spark and ipython, please, see also previous version of text.

Install Java Development Kit

@willprice
willprice / .travis.yml
Last active August 15, 2023 17:12
How to set up TravisCI for projects that push back to github
# Ruby is our language as asciidoctor is a ruby gem.
lang: ruby
before_install:
- sudo apt-get install pandoc
- gem install asciidoctor
script:
- make
after_success:
- .travis/push.sh
env:
@denji
denji / README.md
Last active April 26, 2024 18:09 — forked from istepanov/gist:3950977
Remove/Backup – settings & cli for macOS (OS X) – DataGrip, AppCode, CLion, Gogland, IntelliJ, PhpStorm, PyCharm, Rider, RubyMine, WebStorm
@timtadh
timtadh / upsert.py
Created December 5, 2013 19:14
How to compile an INSERT ... ON DUPLICATE KEY UPDATE with SQL Alchemy with support for a bulk insert.
#!/usr/bin/env python
# Copyright (c) 2012, Tim Henderson
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# - Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
@mathewbyrne
mathewbyrne / CsvResponse.php
Created March 21, 2013 22:54
A small Symfony 2 class for returning a response as a CSV file. Based on the Symfony JsonResponse class.
<?php
namespace Jb\AdminBundle\Http;
use Symfony\Component\HttpFoundation\Response;
class CsvResponse extends Response
{
protected $data;
@shantanuo
shantanuo / mysql_to_big_query.sh
Last active September 14, 2022 07:12
Copy MySQL table to big query. If you need to copy all tables, use the loop given at the end. Exit with error code 3 if blob or text columns are found. The csv files are first copied to google cloud before being imported to big query.
#!/bin/sh
TABLE_SCHEMA=$1
TABLE_NAME=$2
mytime=`date '+%y%m%d%H%M'`
hostname=`hostname | tr 'A-Z' 'a-z'`
file_prefix="trimax$TABLE_NAME$mytime$TABLE_SCHEMA"
bucket_name=$file_prefix
splitat="4000000000"
bulkfiles=200
@simonw
simonw / gist:104413
Created April 30, 2009 11:19
Turn a BeautifulSoup form in to a dict of fields and default values - useful for screen scraping forms and then resubmitting them
def extract_form_fields(self, soup):
"Turn a BeautifulSoup form in to a dict of fields and default values"
fields = {}
for input in soup.findAll('input'):
# ignore submit/image with no name attribute
if input['type'] in ('submit', 'image') and not input.has_key('name'):
continue
# single element nome/value fields
if input['type'] in ('text', 'hidden', 'password', 'submit', 'image'):