Skip to content

Instantly share code, notes, and snippets.

View mapcloud's full-sized avatar

Aquabiota Solutions AB mapcloud

View GitHub Profile
@mapcloud
mapcloud / unique_list_in_list_of_dictionaries.py
Last active April 29, 2017 11:45
Python: Creates an unique list for a given list of dictionaries. Useful to remove duplicates of dictionaries within a list.
# Credits to: http://stackoverflow.com/questions/11741876/getting-unique-values-from-a-list-of-dict
import ast
def unique_list_in_list_of_dictionaries(content_list: list())->list:
"""
content_list = [{'A':1, 'B':1, 'C':1},
{'A':1, 'B':1, 'C':1},
{'A':2, 'B':2, 'C':2}
{'A':3, 'B':2, 'C':1}
{'A':3, 'B':2, 'C':1}]
:returns:
@mapcloud
mapcloud / custom_gremlin_step_pyorient.py
Created June 8, 2017 09:01 — forked from lebedov/custom_gremlin_step_pyorient.py
Define and use custom step in Gremlin via pyorient.
#!/usr/bin/env python
"""
Define and use custom step in Gremlin via pyorient.
"""
from pyorient.ogm import Graph, Config
from pyorient.ogm.declarative import declarative_node, declarative_relationship
from pyorient.ogm.property import String
@mapcloud
mapcloud / bash
Created July 10, 2017 15:51
How to split a very large text file into smaller files
awk '{outfile=sprintf("fileName%02d.tsv",NR/100000+1);print > outfile}' fileName_of_really_big_textfile.tsv
@mapcloud
mapcloud / docker-housekeeping.md
Last active December 18, 2017 13:48
How to Delete All <none>, Untagged and dangling Docker Containers and Images?

Transfer docker image to non-internet-connected machine

Create a backup that can then be used with docker load

$ docker save busybox > busybox.tar

$ ls -sh busybox.tar

2.7M busybox.tar

@mapcloud
mapcloud / compassbearing.py
Created July 27, 2017 12:50 — forked from jeromer/compassbearing.py
compass bearing between two points in Python
def calculate_initial_compass_bearing(pointA, pointB):
"""
Calculates the bearing between two points.
The formulae used is the following:
θ = atan2(sin(Δlong).cos(lat2),
cos(lat1).sin(lat2) − sin(lat1).cos(lat2).cos(Δlong))
:Parameters:
- `pointA: The tuple representing the latitude/longitude for the
@mapcloud
mapcloud / pandas_to_geojson.py
Created July 28, 2017 13:27
Pandas to GeoJSON (Multiples points + features) with Python
# Real source heroes here: https://gis.stackexchange.com/questions/220997/pandas-to-geojson-multiples-points-features-with-python?rq=1
import json
import simplekml
import geojson
import pandas as pd
def data2kml(df):
kml = simplekml.Kml()
df.apply(lambda X: kml.newpoint(
name=X["name"],
@mapcloud
mapcloud / find_empty_string_values_in_df.py
Created July 28, 2017 13:29
Finding values which are empty strings with pandas and applymap
np.where(df.applymap(lambda x: x == ''))
foo bar baz
a 1
b 2
c
4
e 5
@mapcloud
mapcloud / pandas_df_rows_into_columns_by_category.py
Created October 9, 2017 15:09
pandas data frame rows into columns based by category. Kind of transposing the data.
# Source: https://stackoverflow.com/questions/39635993/how-to-convert-pandas-dataframe-rows-into-columns-based-on-category
# convert the module variables into columns and group by the id. So something like:
# Example
ls = [{'count':5, 'module':'payroll', 'id':2}, {'count': 53, 'module': 'general','id':2}, {'id': 5,'count': 35, 'module': 'tax'}, ]
df = pd.DataFrame.from_dict(ls)
# Solution
# You can use groupby by columns which first create new index and last column. then need aggreagate some way - I use mean, then convert one column DataFrame to Series by DataFrame.squeeze (then is not necessary remove top level of Multiindex in columns) and reshape by unstack. Last add_suffix to column name
df = df.groupby(['id','module']).mean().squeeze().unstack().add_suffix('_count')
@mapcloud
mapcloud / vim_add_to_end_of_each_line.md
Last active February 18, 2018 10:11
vim time savers

VIM

:%norm A*

Description

%       = for every line
norm    = type the following commands
A*      = Append `*` the end of current line