Skip to content

Instantly share code, notes, and snippets.

@mze3e
mze3e / google-bigquery-de-duplication.sql
Created Aug 7, 2021
Google BigQuery De Duplication
View google-bigquery-de-duplication.sql
CREATE OR REPLACE TABLE `amazon_products.amazon_products`
AS
Select asin, brand, title, feature, description, price, image, also_buy, also_view, rank, category, similar_item, tech1, tech2, fit, main_cat, details from (
SELECT asin, brand, title, feature, description, price, image, also_buy, also_view, rank, category, similar_item, tech1, tech2, fit, main_cat, details,
ROW_NUMBER() OVER ( PARTITION BY asin ORDER BY asin ) AS rownum
FROM `amazon_products.amazon_products_latest`) A
where A.rownum=1
order by asin
@mze3e
mze3e / python-kafka-producer.py
Created Apr 14, 2021
Python Kafka Producer Examples
View python-kafka-producer.py
#Example 1
#Project: dino Author: thenetcircle File: kafka.py License: Apache License 2.0
def __init__(self, env, is_external_queue: bool):
super().__init__(env, is_external_queue, queue_type='kafka', logger=logger)
eq_host = env.config.get(ConfigKeys.HOST, domain=self.domain_key, default=None)
eq_queue = env.config.get(ConfigKeys.QUEUE, domain=self.domain_key, default=None)
if eq_host is None or len(eq_host) == 0 or (type(eq_host) == str and len(eq_host.strip()) == 0):
logging.warning('blank external host specified, not setting up external publishing')
@mze3e
mze3e / python-kafka-consumer.py
Created Apr 14, 2021
Python Kafka Consumer Examples
View python-kafka-consumer.py
#https://www.programcreek.com/python/example/98440/kafka.KafkaConsumer
##Project: scrapy-cluster Author: istresearch File: kafka_monitor.py License: MIT License
def _create_consumer(self):
"""Tries to establing the Kafka consumer connection"""
try:
brokers = self.settings['KAFKA_HOSTS']
self.logger.debug("Creating new kafka consumer using brokers: " +
str(brokers) + ' and topic ' +
self.settings['KAFKA_INCOMING_TOPIC'])
@mze3e
mze3e / Docker CheatSheet
Last active Apr 2, 2021
Docker CheatSheet
View Docker CheatSheet
#Docker Cheatsheet
##Build
Build an image from the Dockerfile in the current directory and tag the image
docker build -t <IMAGE_NAME>:1.0 .
List all images that are locally stored with the Docker Engine
docker image ls
Delete an image from the local image store
@mze3e
mze3e / gist:769fcfc9a57e8be29cd90d4fcce2866f
Created Mar 21, 2021
How to check linux flavor and version
View gist:769fcfc9a57e8be29cd90d4fcce2866f
For checking kernel version:
uname -a
For checking Linux flavor and version:
cat /proc/version
Once you know that you are running Red Hat for example, you can get to the point with:
cat /etc/redhat-release
Or on Debian:
View mongodb_cheat_sheet.md

MongoDB Cheat Sheet

Show All Databases

show dbs

Show Current Database

@mze3e
mze3e / rest-api-cheatsheet.md
Last active Mar 29, 2020 — forked from jahe/rest-api-cheatsheet.md
REST API Cheatsheet
View rest-api-cheatsheet.md

REST API Cheatsheet

(MOVED TO https://github.com/mze3e/api-cheat-sheet)

General

  1. Build the API with consumers in mind--as a product in its own right.

    • Not for a specific UI.
    • Embrace flexibility / tunability of each endpoint (see #5, 6 & 7).
    • Eat your own dogfood, even if you have to mockup an example UI.
  2. Date/Time/Timestamp:: There is a standard: Use it: ISO 8601 in UTC: { "createdTimestamp": "2017-11-15T18:10:24.343Z" }

View iron-router-cheatsheet.coffee
# A little Meteor CheatSheet about Iron-Router. (updated on a weekly basis)
# Check our Studio: https://gentlenode.com/
# Iron Router > Configuration
Router.configure
layoutTemplate: 'Main'
loadingTemplate: 'Loading'
@mze3e
mze3e / proxy.js
Last active Aug 29, 2015 — forked from drorm/proxy.js
Proxy for multiple subdomains
View proxy.js
//This script proxies multiple subdomains for a Meteor Application. More to edit.
"use strict";
var httpProxy = require('http-proxy'),
http = require('http'),
accesslog = require('access-log'),
addresses;
var LISTENPORT = 80; //port we're listening to the outside world
View meteor.cheatsheet.coffee
METEOR CORE:
Anywhere: Meteor.isClient
Anywhere: Meteor.isServer
Anywhere: Meteor.startup(func)
Anywhere: Meteor.absoluteUrl([path], [options])
Anywhere: Meteor.settings
Anywhere: Meteor.release