Skip to content

Instantly share code, notes, and snippets.

View ay27's full-sized avatar
:octocat:

Jinmian Ye ay27

:octocat:
  • Tencent
  • Shenzhen, China
View GitHub Profile
@alsrgv
alsrgv / tensorflow_mnist_estimator.py
Last active February 8, 2023 10:05
Horovod with Estimator API
# Copyright 2017 Uber Technologies, Inc. All Rights Reserved.
# Copyright 2016 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
@gyglim
gyglim / tensorboard_logging.py
Last active August 23, 2023 21:29
Logging to tensorboard without tensorflow operations. Uses manually generated summaries instead of summary ops
"""Simple example on how to log scalars and images to tensorboard without tensor ops.
License: BSD License 2.0
"""
__author__ = "Michael Gygli"
import tensorflow as tf
from StringIO import StringIO
import matplotlib.pyplot as plt
import numpy as np
@yaroslavvb
yaroslavvb / simple_barrier.py
Created December 16, 2016 06:03
Example of using shared counters to implement Barrier primitive
"""Example of barrier implementation using TensorFlow shared variables.
All workers synchronize on barrier, copy global parameters to local versions
and increment global parameter variable asynchronously. Should see something
like this:
bash> killall python
bash> python simple_barrier.py --num_workers=4
worker 0, local_param 4 global_param 5
worker 2, local_param 4 global_param 7
@yaroslavvb
yaroslavvb / sharded_ps_benchmark.py
Last active December 27, 2022 06:25
Example of local cluster with multiple workers/training loops sharded parameter server
#!/usr/bin/env python
# Benchmark transferring data, part of troubleshooting https://github.com/tensorflow/tensorflow/issues/6116
#
# Take a independent workers communicating with b parameter shards
# Each worker tries to add to variables stored on parameter server as fast as
# possible.
#
# macbook
# ps=1: 1.6 GB/s
# ps=2: 2.6 GB/s
@obskyr
obskyr / stream_response.py
Last active June 14, 2024 11:25
How to stream a requests response as a file-like object.
# -*- coding: utf-8 -*-
import requests
from io import BytesIO, SEEK_SET, SEEK_END
class ResponseStream(object):
def __init__(self, request_iterator):
self._bytes = BytesIO()
self._iterator = request_iterator

tmux cheatsheet

As configured in my dotfiles.

start new:

tmux

start new with session name: