Skip to content

Instantly share code, notes, and snippets.

Avatar

Keisuke Fukuda keisukefukuda

View GitHub Profile
View basename.cc
#include <iostream>
#include <string>
#include <cstring>
#include <cstdlib>
#include <libgen.h>
using namespace std;
char* my_basename(char* str) {
View how_to_build_ucx_on_macox.md

UCX MacOS port is under construction. So far, only ucx_info is built Other conponents (gtests, examples, tools/perf, toos/profile) are future work.

# First, clone UCX repository
$ git clone https://github.com/openucx/ucx.git
$ cd ucx

# Fetch a set of patches from our fork
$ BRANCH_NAME="mac-os-build"
View building-ucx.txt
Follow https://github.com/hiroyuki-sato/ucx/issues/34
$ git clean -xfd
$ ./autogen.sh && ./configure --disable-numa --with-progress64=./progress64
/Applications/Xcode.app/Contents/Developer/usr/bin/make all-recursive
Making all in src/ucm
Making all in .
CC event/libucm_la-event.lo
In file included from event/event.c:21:
View tmp.py
# learners.py
def update(self, transitions):
# Override DQN.update() and update_from_episodes() to kick
# allreduce communication to synchronize cumulative steps among
# all learners
# Note: Overriding update()/update_from_episodes() is a quick hack to
# implement synchronizing cumulative_steps.
# PFRL doesn't have a mechanism to hook events in a learner loop.
# (`step_hook` in pfrl.experiments.train_agent() works well, but
View install-opencv.sh
#!/bin/bash
set -e
set -u
onerror()
{
status=$?
script=$0
line=$1
shift
View scatter_dataset_test.py
import chainermn
from chainer.datasets import get_cifar10
import pickle
comm = chainermn.create_communicator('pure_nccl')
if comm.rank == 0:
train, test = get_cifar10()
else:
train, test = None, None
@keisukefukuda
keisukefukuda / a_pdfmark.py
Last active Sep 17, 2019
Generate pdfmarks (TOC) file, to feed to gs command
View a_pdfmark.py
# LICENSE: MIT License
# How to use:
# $ pip install PyYAML fire pdfrw
# $ python pdfmark.py --infile=pdfmarks.yaml --outfile=pdfmarks
# $ gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=out.pdf in.pdf
#
import sys
import pprint # NOQA
import yaml
View main.rs
use std::ops;
#[derive(Debug,Copy,Clone)]
struct Vec3 {
e: [f32; 3]
}
#[allow(dead_code)]
impl Vec3 {
pub fn new(e1: f32, e2: f32, e3: f32) -> Vec3 {
@keisukefukuda
keisukefukuda / 0 README.md
Last active Aug 4, 2019
How to run ChainerMN application on TSUBAME3
View 0 README.md

TSUBAME3上でのChainerMNアプリケーションの実行方法の説明

ChainerMNプログラムを分散環境で動かすには、動かしたいプログラムとデータに加えて、2つのファイルを用意します。

  • 動かしたいアプリケーションを起動するジョブスクリプト(ここでは例として以下のjob_mnist.sh)
  • 補助ファイル run.sh

ジョブスクリプトの内容は、走らせたいジョブの処理の内容に従って細かく変更を加えていきます。 補助ファイルは、最初に作成したあとは基本的に変更の必要はありません

View bcast_test.py
import hashlib
import sys
import numpy as np
from mpi4py import MPI
comm = MPI.COMM_WORLD
UINT32_MAX = 4294967295
INT32_MAX = 2147483647