liviaerxin

FastAPI and Uvicorn Logging

When running FastAPI app, all the logs in console are from Uvicorn and they do not have timestamp and other useful information. As Uvicorn applies python logging module, we can override Uvicorn logging formatter by applying a new logging configuration.

Meanwhile, it's able to unify the your endpoints logging with the Uvicorn logging by configuring all of them in the config file log_conf.yaml.

Before overriding:

uvicorn main:app --reload

ledmaster

enderim

// To be able to easily print RDD content, you can either create a function inside the shell or an implicit class
// Ref to my answer: http://stackoverflow.com/a/41317574/1095213

def p(rdd: org.apache.spark.rdd.RDD[_]) = rdd.foreach(println) // Option 1

implicit class Printer(rdd: org.apache.spark.rdd.RDD[_]) { // Option 2
    def print = rdd.foreach(println)
}

// Example

bernhardschaefer

#!/bin/bash

# Minimum TODOs on a per job basis:
# 1. define name, application jar path, main class, queue and log4j-yarn.properties path
# 2. remove properties not applicable to your Spark version (Spark 1.x vs. Spark 2.x)
# 3. tweak num_executors, executor_memory (+ overhead), and backpressure settings

# the two most important settings:
num_executors=6
executor_memory=3g

roylee0704

docker logs nginx 2>&1 | grep "127." 

# ref: http://stackoverflow.com/questions/34724980/finding-a-string-in-docker-logs-of-container

dukehoops-zz

def next_task_id():
    return 'branch_upload'  # branch_skip_upload or branch_upload


decide_upload = BranchPythonOperator(python_callable=next_task_id, task_id='decide_upload', dag=main_dag)
parent >> decide_upload

branch_skip_upload = DummyOperator(task_id='branch_skip_upload', dag=main_dag)
branch_upload = DummyOperator(task_id='branch_upload', dag=main_dag)

wangruohui

In this article, I will share some of my experience on installing NVIDIA driver and CUDA on Linux OS. Here I mainly use Ubuntu as example. Comments for CentOS/Fedora are also provided as much as I can.

Table of Contents

• Install NVIDIA Graphics Driver via apt-get
• Install NVIDIA Graphics Driver via runfile
  • Remove Previous Installations (Important)
  • Download the Driver
  • Install Dependencies
  • Creat Blacklist for Nouveau Driver

• Stop lightdm/gdm/kdm

boxysean

# airflow/plugins/slack.py

import logging

from airflow.operators.python_operator import PythonOperator
from airflow.plugins_manager import AirflowPlugin
from slackclient import SlackClient
from titan.utils import config

athoune

import os
import sys
import luigi
from raven import Client

client = Client()

# Define your Luigi tasks here

@luigi.Task.event_handler(luigi.Event.FAILURE)

wolfdancer

##Google Interview Questions: Product Marketing Manager

• Why do you want to join Google? -- Because I want to create tools for others to learn, for free. I didn't have a lot of money when growing up so I didn't get access to the same books, computers and resources that others had which caused money, I want to help ensure that others can learn on the same playing field regardless of their families wealth status or location.
• What do you know about Google’s product and technology? -- A lot actually, I am a beta tester for numerous products, I use most of the Google tools such as: Search, Gmaill, Drive, Reader, Calendar, G+, YouTube, Web Master Tools, Keyword tools, Analytics etc.
• If you are Product Manager for Google’s Adwords, how do you plan to market this?
• What would you say during an AdWords or AdSense product seminar?
• Who are Google’s competitors, and how does Google compete with them? -- Google competes on numerous fields: --- Search: Baidu, Bing, Duck Duck Go