Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.
![Screenshot 2023-12-18 at 10 40 27 PM](https://private-user-images.githubusercontent.com/3837836/291468646-4c30ad72-76ee-4939-a5fb-16b570d38cf2.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjIyMjk0OTEsIm5iZiI6MTcyMjIyOTE5MSwicGF0aCI6Ii8zODM3ODM2LzI5MTQ2ODY0Ni00YzMwYWQ3Mi03NmVlLTQ5MzktYTVmYi0xNmI1NzBkMzhjZjIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcyOSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MjlUMDQ1OTUxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MWVkZDc5MDM1MDIwY2I5MDMxMDM0ODJiMzkyNDllZjMwNDdjMWU0NGNjODk0ZWYxNDljZDdkMmRiZDhkMDE5NCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.D5saVPKFDns8x6sfOgjKcThqyrMyy1UpWBYSIK1rD40)
You will need the following two files, be sure to change the IP address in both sections, the `hostfile_contents` | |
as well as the ansible inventory section to reflect your configuration | |
Rest of the steps should run parallely across the different machines, for any updates to this script, | |
check this thread: https://github.com/ggerganov/llama.cpp/issues/2164#issuecomment-1639640846 | |
If you want an automated way to do this in the future and would like an opionated setup procedure, | |
check this repo: https://github.com/theycallmeloki/edith-cli |
Andy Thomason is a Senior Programmer at Genomics PLC. He has been witing graphics systems, games and compilers since the '70s and specialises in code performance.
UPDATE (March 2020, thanks @ic): I don't know the exact AMI version but yum install docker
now works on the latest Amazon Linux 2. The instructions below may still be relevant depending on the vintage AMI you are using.
Amazon changed the install in Linux 2. One no-longer using 'yum' See: https://aws.amazon.com/amazon-linux-2/release-notes/
sudo amazon-linux-extras install docker
sudo service docker start
/***************************************************************************** | |
* QuantCup 1: Price-Time Matching Engine | |
* | |
* Submitted by: voyager | |
* | |
* Design Overview: | |
* In this implementation, the limit order book is represented using | |
* a flat linear array (pricePoints), indexed by the numeric price value. | |
* Each entry in this array corresponds to a specific price point and holds | |
* an instance of struct pricePoint. This data structure maintains a list |
""" | |
1-way UDP to TCP relay. | |
Test with netcat | |
1) Run TCP server: | |
nc -l 999 | |
2) Run UDP proxy: | |
python udpproxy.py | |
3) Run UDP client: | |
nc -u 127.0.0.1 8888 |
I was talking to a coworker recently about general techniques that almost always form the core of any effort to write very fast, down-to-the-metal hot path code on the JVM, and they pointed out that there really isn't a particularly good place to go for this information. It occurred to me that, really, I had more or less picked up all of it by word of mouth and experience, and there just aren't any good reference sources on the topic. So… here's my word of mouth.
This is by no means a comprehensive gist. It's also important to understand that the techniques that I outline in here are not 100% absolute either. Performance on the JVM is an incredibly complicated subject, and while there are rules that almost always hold true, the "almost" remains very salient. Also, for many or even most applications, there will be other techniques that I'm not mentioning which will have a greater impact. JMH, Java Flight Recorder, and a good profiler are your very best friend! Mea
I screwed up using git ("git checkout --" on the wrong file) and managed to delete the code I had just written... but it was still running in a process in a docker container. Here's how I got it back, using https://pypi.python.org/pypi/pyrasite/ and https://pypi.python.org/pypi/uncompyle6
apt-get update && apt-get install gdb
"""UDP proxy server.""" | |
import asyncio | |
class ProxyDatagramProtocol(asyncio.DatagramProtocol): | |
def __init__(self, remote_address): | |
self.remote_address = remote_address | |
self.remotes = {} |
Principles of Adult Behavior