rain-1

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

• Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.

rain-1

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

rain-1

On Llamafile not making sense

The LLamafile project doesn't make sense.

The claim is that it is "bringing LLMs to the people", but you could already run an LLM - which is a large binary file containing lots of floating point numbers - by using llama.cpp.

Llamafile joins a compiled binary program to run LLMs with a weights binary into a single file. This isn't a useful goal. you could simply distribute a zip containing an .exe and a weights file together. Or better still: Decouple the program that runs these chatbots from the chatbot weights.

Imagine if PNG files were also an executable that could pop open a window that displays a PNG on your computer. There is a reason we don't do this: It's not good engineering.

rain-1

Solving crosswords with GPT

This is my research report. I've included a lot of the code and chat interactions for people to read through if interested. I worked on this crossword https://www.theguardian.com/crosswords/quick/16553

I had a vision for a GPT powered crossword solver. My idea is that it would do a tree search over GPT generated guesses that would include the knowns so far, like:

I didn't end up doing that because ChatGPT and GPT-4 are terrible at questions involving the length of words, or guessing words that contain specific letters at specific locations. It can sometimes do them but usually fails. I think this is because it's token based. I am curious whether a character based LLM would be better at such tasks.

rain-1

The reverse turing test

I asked GPT-4 to come up with 10 questions to determine if the answerer was AI or human.

I provided my own answers for these questions and I also asked ChatGPT to answer them.

The result is that GPT-4 was able to correctly differentiate between AI and Human.

rain-1

Raspberry Pi, Static HTTPS site with Docker and Nginx

This tutorial is dated Oct 2021, if it's much further on than that this information might be out of date.

This is a guide on setting up a static HTTPS website on your raspberry pi using docker and nginx. The aim is to have this running on the raspberry pi and to be able to access it from a host computer on the same local network. You should already be able to ssh into your pi from your host computer and have raspberry pi OS set up.

Find your raspberry pi

rain-1

WannaCry|WannaDecrypt0r NSA-Cyberweapon-Powered Ransomware Worm

• Virus Name: WannaCrypt, WannaCry, WanaCrypt0r, WCrypt, WCRY
• Vector: All Windows versions before Windows 10 are vulnerable if not patched for MS-17-010. It uses EternalBlue MS17-010 to propagate.
• Ransom: between $300 to $600. There is code to 'rm' (delete) files in the virus. Seems to reset if the virus crashes.
• Backdooring: The worm loops through every RDP session on a system to run the ransomware as that user. It also installs the DOUBLEPULSAR backdoor. It corrupts shadow volumes to make recovery harder. (source: malwarebytes)
• Kill switch: If the website www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com is up the virus exits instead of infecting the host. (source: malwarebytes). This domain has been sinkholed, stopping the spread of the worm. Will not work if proxied (source).

update: A minor variant of the viru

rain-1

Could an LLM end up being the core part of a dangerous computer worm?

How would we neutralize such a thing if this happened?

Some virus and worm background

• https://en.wikipedia.org/wiki/Timeline_of_computer_viruses_and_worms

There is a hilarious story from https://users.cs.utah.edu/~elb/folklore/xerox.txt about an early computer virus called robin hood and friar tuck. This was basically just two programs running on a UNIX system that would look out for each other and reboot the other process if it was killed. It's interesting to note that since computer programs run thousands of times faster than humans, a human can't type kill -9 robinhood then type kill -9 friartuck in time. The computer is faster so it always wins if you try this. To defeat this you need to take a different approach than speed.

rain-1

Why is IRC distributed across multiple servers?

I have been wondering for a long time why IRC networks have multiple servers. Wouldn't it be simpler just to use a single server?

One of the problems of having multiple servers is that netsplits can occur. Anybody who has been on IRC for a while will have witnessed one. Hundreds of people suddenly ripped out of the chat. This can also screw up channel and user modes, and 'some people' have been known to wait for netsplits in order to takeover channels or enter password protected channels.

So lets compare situation (A) a single IRC server everyone connects to with the current setup people use (B) multiple servers. Let's say you run an IRC network with u = 40,000 users and n = 20 server nodes that people connect to via round robin DNS (meaning that when people resolve the DNS it gives them a random server from the set of 20 to connect to). These are vaguely realistic numbers modelled after libera.chat.

So in (B) you have roughly u/n = 2000 clients connected

rain-1

• There is a specification for a tail call instruction in wasm. github.com/WebAssembly/tail-call https://v8.dev/blog/wasm-tail-call
• Browsers, e.g. Chrome, implement this. https://webassembly.org/roadmap/
• At least one scheme implementation called Guile-Hoot targets this https://gitlab.com/spritely/guile-hoot ABI spritely.institute/Status

This spec seems to have gotten in thanks to work by apignotti, https://hn.algolia.com/?q=WebAssembly+tail+calls

There is also a very interesting project for generalized effect handlers that may build on top of this platform https://wasmfx.dev/community/

Great news for schemers with web browsers.