Skip to content

Instantly share code, notes, and snippets.

@mze3e
mze3e / google-bigquery-de-duplication.sql
Created August 7, 2021 17:42
Google BigQuery De Duplication
CREATE OR REPLACE TABLE `amazon_products.amazon_products`
AS
Select asin, brand, title, feature, description, price, image, also_buy, also_view, rank, category, similar_item, tech1, tech2, fit, main_cat, details from (
SELECT asin, brand, title, feature, description, price, image, also_buy, also_view, rank, category, similar_item, tech1, tech2, fit, main_cat, details,
ROW_NUMBER() OVER ( PARTITION BY asin ORDER BY asin ) AS rownum
FROM `amazon_products.amazon_products_latest`) A
where A.rownum=1
order by asin
@mze3e
mze3e / python-kafka-producer.py
Created April 14, 2021 13:33
Python Kafka Producer Examples
#Example 1
#Project: dino Author: thenetcircle File: kafka.py License: Apache License 2.0
def __init__(self, env, is_external_queue: bool):
super().__init__(env, is_external_queue, queue_type='kafka', logger=logger)
eq_host = env.config.get(ConfigKeys.HOST, domain=self.domain_key, default=None)
eq_queue = env.config.get(ConfigKeys.QUEUE, domain=self.domain_key, default=None)
if eq_host is None or len(eq_host) == 0 or (type(eq_host) == str and len(eq_host.strip()) == 0):
logging.warning('blank external host specified, not setting up external publishing')
@mze3e
mze3e / python-kafka-consumer.py
Created April 14, 2021 13:28
Python Kafka Consumer Examples
#https://www.programcreek.com/python/example/98440/kafka.KafkaConsumer
##Project: scrapy-cluster Author: istresearch File: kafka_monitor.py License: MIT License
def _create_consumer(self):
"""Tries to establing the Kafka consumer connection"""
try:
brokers = self.settings['KAFKA_HOSTS']
self.logger.debug("Creating new kafka consumer using brokers: " +
str(brokers) + ' and topic ' +
self.settings['KAFKA_INCOMING_TOPIC'])
@mze3e
mze3e / Docker CheatSheet
Last active April 2, 2021 21:41
Docker CheatSheet
#Docker Cheatsheet
##Build
Build an image from the Dockerfile in the current directory and tag the image
docker build -t <IMAGE_NAME>:1.0 .
List all images that are locally stored with the Docker Engine
docker image ls
Delete an image from the local image store
@mze3e
mze3e / gist:769fcfc9a57e8be29cd90d4fcce2866f
Created March 21, 2021 09:10
How to check linux flavor and version
For checking kernel version:
uname -a
For checking Linux flavor and version:
cat /proc/version
Once you know that you are running Red Hat for example, you can get to the point with:
cat /etc/redhat-release
Or on Debian:
@mze3e
mze3e / mongodb_cheat_sheet.md
Created September 3, 2020 21:33 — forked from bradtraversy/mongodb_cheat_sheet.md
MongoDB Cheat Sheet

MongoDB Cheat Sheet

Show All Databases

show dbs

Show Current Database

@mze3e
mze3e / rest-api-cheatsheet.md
Last active March 29, 2020 22:19 — forked from jahe/rest-api-cheatsheet.md
REST API Cheatsheet

REST API Cheatsheet

(MOVED TO https://github.com/mze3e/api-cheat-sheet)

General

  1. Build the API with consumers in mind--as a product in its own right.

    • Not for a specific UI.
    • Embrace flexibility / tunability of each endpoint (see #5, 6 & 7).
    • Eat your own dogfood, even if you have to mockup an example UI.
  2. Date/Time/Timestamp:: There is a standard: Use it: ISO 8601 in UTC: { "createdTimestamp": "2017-11-15T18:10:24.343Z" }

# A little Meteor CheatSheet about Iron-Router. (updated on a weekly basis)
# Check our Studio: https://gentlenode.com/
# Iron Router > Configuration
Router.configure
layoutTemplate: 'Main'
loadingTemplate: 'Loading'
@mze3e
mze3e / proxy.js
Last active August 29, 2015 14:03 — forked from drorm/proxy.js
Proxy for multiple subdomains
//This script proxies multiple subdomains for a Meteor Application. More to edit.
"use strict";
var httpProxy = require('http-proxy'),
http = require('http'),
accesslog = require('access-log'),
addresses;
var LISTENPORT = 80; //port we're listening to the outside world
METEOR CORE:
Anywhere: Meteor.isClient
Anywhere: Meteor.isServer
Anywhere: Meteor.startup(func)
Anywhere: Meteor.absoluteUrl([path], [options])
Anywhere: Meteor.settings
Anywhere: Meteor.release