jezhumble

# Copyright 2019 Google LLC.
# SPDX-License-Identifier: Apache-2.0
# This snippet shows you how to use Blob.generate_signed_url() from within compute engine / cloud functions
# as described here: https://cloud.google.com/functions/docs/writing/http#uploading_files_via_cloud_storage
# (without needing access to a private key)
# Note: as described in that page, you need to run your function with a service account
# with the permission roles/iam.serviceAccountTokenCreator

import os, google.auth
from google.auth.transport import requests

juanpabloaj

import os
import logging

LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
logging.basicConfig(level=LOGLEVEL)

giampaolotrapasso

List of links from Designing Event-Driven Systems by Ben Stopford

Designing Event-Driven Systems book by Ben Stopford contains a lot of useful links to papers, books, documentation and definitions related to event driven design and Kafka. I just extracted them as reference for the future and added some groups to show them better.

You can read the book for free

• A Crystal Ball to Prioritise Technical Debt in Monoliths or Microservices: Adam Tornhill's Thoughts
• Brewer's CAP Theorem <= :julianbrowne

frankyaorenjie

FROM python:3.6

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
ENV GOOGLE_APPLICATION_CREDENTIALS /service/scripts/application_default_credentials.json

COPY . /service/scripts
WORKDIR /service/scripts

RUN pip install pipenv

ctf0

<?php

// use Illuminate\Support\Collection;
// use Illuminate\Pagination\Paginator;
// use Illuminate\Pagination\LengthAwarePaginator;

/**
  * Gera a paginação dos itens de um array ou collection.
  *
  * @param array|Collection      $items

BrikerMan

from gensim.models import KeyedVectors

# Load gensim word2vec
w2v_path = '<Gensim File Path>'
w2v = KeyedVectors.load_word2vec_format(w2v_path)

import io

# Vector file, `\t` seperated the vectors and `\n` seperate the words
"""

uhho

def s3_to_pandas(client, bucket, key, header=None):

    # get key using boto3 client
    obj = client.get_object(Bucket=bucket, Key=key)
    gz = gzip.GzipFile(fileobj=obj['Body'])
    
    # load stream directly to DF
    return pd.read_csv(gz, header=header, dtype=str)

def s3_to_pandas_with_processing(client, bucket, key, header=None):

revotu

from bs4 import BeautifulSoup

# remove all attributes
def _remove_all_attrs(soup):
    for tag in soup.find_all(True): 
        tag.attrs = {}
    return soup

# remove all attributes except some tags
def _remove_all_attrs_except(soup):

aparrish

graphadvantage

##Neo4j GraphGist - Enterprise Architectures: Real-time Graph Updates using Kafka Messaging

Neo4j Use Case: Low Latency Graph Analytics & OLTP - Update 1M Nodes in 90 secs with Kafka and Neo4j Bolt

Introduction

A recent Neo4j whitepaper describes how Monsanto is performing real-time updates on a 600M node Neo4j graph using Kafka to consume data extracted from a large Oracle Exadata instance.

This modern data architecture combines a fast, scalable messaging platform (Kafka) for low latency data provisioning and an enterprise graph database (Neo4j) for high performance, in-memory analytics & OLTP - creating new and powerful real-time graph analytics capabilities for your enterprise applications.