rom1504

Steps:

1. wget https://gist.github.com/rom1504/67ada3dedbecc113ae2dbdfd9c642d83/raw/865fb35e00f21330b5b82aeb7c31941b6c18f649/spark_on_slurm.sh
2. wget https://gist.github.com/rom1504/67ada3dedbecc113ae2dbdfd9c642d83/raw/865fb35e00f21330b5b82aeb7c31941b6c18f649/worker_spark_on_slurm.sh
3. wget https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz && tar xf spark-3.3.1-bin-hadoop3.tgz
4. sbatch spark_on_slurm.sh
5. build a venv, install pyspark, then run something like this:

(you can get https://huggingface.co/datasets/laion/laion-coco/resolve/main/part-00000-2256f782-126f-4dc6-b9c6-e6757637749d-c000.snappy.parquet as an example parquet)

nateraw

"""
# Data
wget -nc --no-check-certificate https://www.crcv.ucf.edu/data/UCF101/UCF101.rar
unrar x UCF101.rar

# Annotations
wget -nc --no-check-certificate https://www.crcv.ucf.edu/data/UCF101/UCF101TrainTestSplits-RecognitionTask.zip
unzip UCF101TrainTestSplits-RecognitionTask.zip

Probably some extra unnecessary imports here

nadavrot

High-Performance Matrix Multiplication

This is a short post that explains how to write a high-performance matrix multiplication program on modern processors. In this tutorial I will use a single core of the Skylake-client CPU with AVX2, but the principles in this post also apply to other processors with different instruction sets (such as AVX512).

Intro

Matrix multiplication is a mathematical operation that defines the product of

Willian-Zhang

Tensorflow 1.7 with CUDA on macOS High Sierra 10.13.4 for eGPU

Largely based on the Tensorflow 1.6 gist, and Tensorflow 1.7 gist for xcode, this should hopefully simplify things a bit.

Requirements

• NVIDIA Web-Drivers 387.10.10.10.30.103 for 10.13.4
• CUDA-Drivers 387.178
• CUDA 9.1 Toolkit

alsrgv

# Copyright 2018 Uber Technologies, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,

jganzabal

This configuration worked for me, hope it helps

It is based on: https://becominghuman.ai/deep-learning-gaming-build-with-nvidia-titan-xp-and-macbook-pro-with-thunderbolt2-5ceee7167f8b

and on: https://stackoverflow.com/questions/44744737/tensorflow-mac-os-gpu-support

Hardware

TomRichter

<?xml version="1.0"?>
<TriggernometryExport Version="1">
  <ExportedTrigger Enabled="true" Name="On Wipe" Id="d2f2668d-dfd5-456d-a404-1d2b5cdd18cd" RegularExpression="(wipeout|0038:end|21:([0-9,a-f,A-F]{8}):40000010)" DebugLevel="Inherit" PrevActions="Keep" PrevActionsRefire="Allow" Scheduling="FromFire" PeriodRefire="Allow" RefirePeriodExpression="0">
    <Actions>
      <Action DiscordTts="false" OrderNumber="1" AuraImageMode="Normal" TextAuraAlignment="MiddleCenter" TextAuraFontSize="8.25" TextAuraEffect="None" TextAuraUseOutline="false" Enabled="true" ActionType="EndEncounter" ExecutionDelayExpression="0" Asynchronous="true" DebugLevel="Inherit" RefireInterrupt="false" RefireRequeue="true" SystemBeepFreqExpression="1000" SystemBeepLengthExpression="100" PlaySoundVolumeExpression="100" PlaySoundExclusive="true" PlaySoundMyself="false" PlaySpeechMyself="false" UseTTSVolumeExpression="100" UseTTSRateExpression="0" UseTTSExclusive="true" LaunchProcessWindowStyle="Normal" ExecScriptType="CSharp" MessageBoxIcon

delta2323

The goal of this document is to explain the basic mechanism of the dataset in Chainer and how to prepare customized dataset that can work with Trainer.

See the official document for the full detail of the specifications.

Intarface of dataset

In order to make the dataset be able to work with the Trainer, it must have two methods:

• __getitem__, which is used for indexing like dataset[i] or slicing like dataset[i:j]
• __len__ , which enables us to feed it to len()

kmhofmann

Building TensorFlow from source (TF 2.3.0, Ubuntu 20.04)

Why build from source?

The official instructions on installing TensorFlow are here: https://www.tensorflow.org/install. If you want to install TensorFlow just using pip, you are running a supported Ubuntu LTS distribution, and you're happy to install the respective tested CUDA versions (which often are outdated), by all means go ahead. A good alternative may be to run a Docker image.

I am usually unhappy with installing what in effect are pre-built binaries. These binaries are often not compatible with the Ubuntu version I am running, the CUDA version that I have installed, and so on. Furthermore, they may be slower than binaries optimized for the target architecture, since certain instructions are not being used (e.g. AVX2, FMA).

So installing TensorFlow from source becomes a necessity. The official instructions on building TensorFlow from source are here: ht

rajkrrsingh

$wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.bz2
$tar xvf protobuf-2.5.0.tar.bz2
$cd protobuf-2.5.0
$./configure CC=clang CXX=clang++ CXXFLAGS='-std=c++11 -stdlib=libc++ -O3 -g' LDFLAGS='-stdlib=libc++' LIBS="-lc++ -lc++abi"
$make -j 4 
$sudo make install
$protoc --version