mattiasarro

› ./configure
Please specify the location of python. [Default is /Users/m/code/3rd/conda/envs/p3gpu/bin/python]:
Found possible Python library paths:
  /Users/m/code/3rd/conda/envs/p3gpu/lib/python3.6/site-packages
  /Users/m/code/3rd/spark-2.1.0-bin-hadoop2.7//python
Please input the desired Python library path to use.  Default is [/Users/m/code/3rd/conda/envs/p3gpu/lib/python3.6/site-packages]

Using python library path: /Users/m/code/3rd/conda/envs/p3gpu/lib/python3.6/site-packages
Do you wish to build TensorFlow with MKL support? [y/N] N
No MKL support will be enabled for TensorFlow

mattiasarro

INFO: From Compiling tensorflow/core/kernels/fused_batch_norm_op.cu.cc:
./tensorflow/core/lib/core/status.h(32): warning: attribute "warn_unused_result" does not apply here

./tensorflow/core/framework/variant.h(343): error: constexpr constructor calls non-constexpr function "std::__1::unique_ptr<_Tp, _Dp>::unique_ptr() [with _Tp=tensorflow::Variant::ValueInterface, _Dp=std::__1::default_delete<tensorflow::Variant::ValueInterface>]"

1 error detected in the compilation of "/var/folders/_8/__mwc41n7kq1__sy9mdkz3c80000gn/T//tmpxft_0000aeb2_00000000-7_fused_batch_norm_op.cu.cpp1.ii".
ERROR: /Users/m/code/3rd/tensorflow_1_4/tensorflow/core/kernels/BUILD:3230:1: output 'tensorflow/core/kernels/_objs/fused_batch_norm_util_gpu/tensorflow/core/kernels/fused_batch_norm_op.cu.pic.o' was not created.
ERROR: /Users/m/code/3rd/tensorflow_1_4/tensorflow/core/kernels/BUILD:3230:1: not all outputs were created or valid.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the 

mattiasarro

Tensorflow 1.6 on macOS High Sierra 10.13.3 with GPU Acceleration (without disabling SIP)

This gist (based on a blog post at byai.io) documents how to set up TensorFlow 1.6 with (e)GPU support without the need to disable SIP. Following the original gist got me a saystem in which training TF on eGPU was successful, but there were various visual glitches due to the newer / less stable version of the driver.

As pointed out by ronchigram, many people are having issues with newer NVIDIA drivers, so it's worth using the nvidia-update script by Benjamin Dobell that installs the latest stable NVIDIA web driver, and if necessary patches it to run on your system. We also don't need to disable SIP when using nvidia-update.

I have als

mattiasarro

# Taken from https://johanwind.github.io/2023/03/23/rwkv_details.html. 
# I've added additional comments restructured it a tiny bit, which makes it clearer for me.

import numpy as np
from torch import load as torch_load  # Only for loading the model weights
from tokenizers import Tokenizer

exp = np.exp
layer_norm = lambda x, w, b : (x - np.mean(x)) / np.std(x) * w + b
sigmoid = lambda x : 1/(1 + exp(-x))