ianschenck

GLM-5.3-Flash on SGLang — 8× AMD MI325X

One line: zai-org/GLM-5.3-Flash (FP8, 320B/18B-active MoE) served by SGLang on 8× Instinct MI325X in Docker, tensor-parallel 8, EAGLE speculative decoding on the NextN layer, at ~5–6× the upstream AMD recipe (~110–123 tok/s single-stream vs 18.9) after five local patch families — all reproduced below.

1. Hardware

GPUs	8× AMD Instinct MI325X (gfx942), 256 GB HBM each, 304 CUs, 64 KB LDS/workgroup
Host	AMD EPYC, 256 logical cores, 3 TiB RAM

ianschenck

config dnsmasq
	option domainneeded '1'
	option boguspriv '1'
	option filterwin2k '0'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'

ianschenck

import interface


class IFoo(interface.Interface):

    def foo(self):
        """foo this object."""


class IBar(interface.Interface):

ianschenck

# Many very old fcgi containers had a trick probably inherited from old 
# inetd, where a socket is syscall dup2'd onto fd 0 of a subprocess. 
# The parent creates and binds the socket while the children then just 
# listen on it.
#
# Something to note is that supervisord provides this facility with its
# [fcgi-program] configuration section. The benefit here is simple inet
# or fcgi style graceful restarts. Anyhow, here is how you grab a
# socket that is fd 0 and start your twisted wsgi container on it. The
# wsgi application here is `application`.

ianschenck

if __name__ == "__main__":
    reactor_args = {}
    
    def run_twisted_wsgi():
        from twisted.internet import reactor
        from twisted.web.server import Site
        from twisted.web.wsgi import WSGIResource

        resource = WSGIResource(reactor, reactor.getThreadPool(), app)
        site = Site(resource)

ianschenck

class Pool(object):

    def __init__(self, factory, size, finalizer=None):
        self._factory = factory
        self._finalizer = finalizer or lambda x: None
        self._pool = list()
        self._pool_sema = threading.Semaphore()
        for _ in range(size):
            self.put(self._factory())

ianschenck

go install path/to/main -ldflags "-X path/to/main.commit $(git rev-parse HEAD)"

ianschenck

(defun go-build ()
  "Build a go package"
  (interactive)
  (compile "go build"))
(defun go-test ()
  "Test a go packages"
  (interactive)
  (compile "go test -v"))
(defun go-vet ()
  "Vet a go package"

ianschenck

#!/bin/bash
set -o errexit -o nounset -o pipefail
function -h {
cat <<USAGE
 USAGE: mesos-docker-setup

  Installs tools for running Docker under Mesos with Marathon.

USAGE
}; function --help { -h ;}

ianschenck

"""
Deploy Ceph.
"""

import base64
import ConfigParser
import os
import StringIO
import struct
import time