In this channel I introduce Rust for people who know C++ already. The videos are supposed to be short and cover only a single topic. Each video comes with a github gist, which contains all the code and commands for copy/paste, such that you can easily try out things. Furthermore, there are links to the excellent Rust documentation.
This is still work in progress. As you can see in the table below,
This worked on 14/May/23. The instructions will probably require updating in the future.
llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)
Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.
It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.
08737ef720f0510c7ec2aa84d7f70c691073c35d.
| Service | SSL | status | Response Type | Allowed methods | Allowed headers |
|---|
I recently happened upon a very interesting implementation of popen() (different API, same idea) called popen-noshell using clone(2), and so I opened an issue requesting use of vfork(2) or posix_spawn() for portability. It turns out that on Linux there's an important advantage to using clone(2). I think I should capture the things I wrote there in a better place. A gist, a blog, whatever.
This is not a paper. I assume reader familiarity with
fork()in particular and Unix in general, though, of course, I link to relevant wiki pages, so if the unfamiliar reader is willing to go down the rabbit hole, they should be able to come ou