Skip to content

Instantly share code, notes, and snippets.

View keisukefukuda's full-sized avatar

Keisuke Fukuda keisukefukuda

View GitHub Profile
2021/05/01
2021/05/02
2021/05/03
2021/05/04
2021/05/05
2021/05/06
2021/05/09
2021/05/10
2021/05/11
2021/05/12
#include <iostream>
#include <string>
#include <cstring>
#include <cstdlib>
#include <libgen.h>
using namespace std;
char* my_basename(char* str) {

UCX MacOS port is under construction. So far, only ucx_info is built Other conponents (gtests, examples, tools/perf, toos/profile) are future work.

# First, clone UCX repository
$ git clone https://github.com/openucx/ucx.git
$ cd ucx

# Fetch a set of patches from our fork
$ BRANCH_NAME="mac-os-build"
Follow https://github.com/hiroyuki-sato/ucx/issues/34
$ git clean -xfd
$ ./autogen.sh && ./configure --disable-numa --with-progress64=./progress64
/Applications/Xcode.app/Contents/Developer/usr/bin/make all-recursive
Making all in src/ucm
Making all in .
CC event/libucm_la-event.lo
In file included from event/event.c:21:
# learners.py
def update(self, transitions):
# Override DQN.update() and update_from_episodes() to kick
# allreduce communication to synchronize cumulative steps among
# all learners
# Note: Overriding update()/update_from_episodes() is a quick hack to
# implement synchronizing cumulative_steps.
# PFRL doesn't have a mechanism to hook events in a learner loop.
# (`step_hook` in pfrl.experiments.train_agent() works well, but
#!/bin/bash
set -e
set -u
onerror()
{
status=$?
script=$0
line=$1
shift
import chainermn
from chainer.datasets import get_cifar10
import pickle
comm = chainermn.create_communicator('pure_nccl')
if comm.rank == 0:
train, test = get_cifar10()
else:
train, test = None, None
@keisukefukuda
keisukefukuda / a_pdfmark.py
Last active September 17, 2019 02:39
Generate pdfmarks (TOC) file, to feed to gs command
# LICENSE: MIT License
# How to use:
# $ pip install PyYAML fire pdfrw
# $ python pdfmark.py --infile=pdfmarks.yaml --outfile=pdfmarks
# $ gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=out.pdf in.pdf
#
import sys
import pprint # NOQA
import yaml
use std::ops;
#[derive(Debug,Copy,Clone)]
struct Vec3 {
e: [f32; 3]
}
#[allow(dead_code)]
impl Vec3 {
pub fn new(e1: f32, e2: f32, e3: f32) -> Vec3 {
@keisukefukuda
keisukefukuda / 0 README.md
Last active August 4, 2019 15:06
How to run ChainerMN application on TSUBAME3

TSUBAME3上でのChainerMNアプリケーションの実行方法の説明

ChainerMNプログラムを分散環境で動かすには、動かしたいプログラムとデータに加えて、2つのファイルを用意します。

  • 動かしたいアプリケーションを起動するジョブスクリプト(ここでは例として以下のjob_mnist.sh)
  • 補助ファイル run.sh

ジョブスクリプトの内容は、走らせたいジョブの処理の内容に従って細かく変更を加えていきます。 補助ファイルは、最初に作成したあとは基本的に変更の必要はありません