The System
One Proxmox host, a handful of inference machines, and four repositories that build and run the whole thing. All of it is living at Z-Space.
Two layers
The infrastructure layer is the AI service itself: a gateway sitting in front of the inference machines. Never a single-user setup and never a direct line to a model. Everything goes through the gateway, which is what makes accounts, keys and quotas mean anything.
The application layer is what gets built on top. Chat, and eventually the shared knowledge base and the note-taking tools, are ordinary clients of the gateway: they get no special access and they never go around it. Keeping the two apart is what lets either one be replaced without touching the other.
The host
One machine, Heron, runs Proxmox and holds every service except inference. Each service is its own LXC container running natively under systemd, with no Docker anywhere in the stack. They talk to each other over an internal bridge that has no route in from the world.
| Role | What runs it |
|---|---|
| Reverse proxy | Caddy. Terminates TLS, and is the public edge. |
| Session broker | Corliss, this app. Sign-in, keys, the console. |
| Registry | HappyView, serving the membership lexicons |
| Gateway | LiteLLM. Every inference request goes through it. |
| Chat | Open WebUI |
| Sync relay | scn-sync-relay, the automerge server behind shared notes |
| Database | Postgres, shared by the services above |
| Sessions | Redis |
| Object store | Garage, S3-compatible |
The machines that think
Inference runs on separate hardware: a heterogeneous set of new, used and donated parts, some of it on loan from members. None of these machines is publicly exposed, and none runs anything but llama.cpp's server. The gateway is the only thing that talks to them.
| Node | Hardware |
|---|---|
| gx10 | ASUS Ascent GX10 (NVIDIA GB10), 128 GB unified memory. On loan, and currently the primary node. |
| orca | 2× RTX 3090, 48 GB VRAM, with one model pinned across both cards |
| salmon | 2× GTX 1080 Ti, 22 GB VRAM |
The fleet changes. Which models are actually being served, and by which node, is answered live on the API page.
Which models
Open-weight models only, with a standing preference for ones released from jurisdictions with meaningful transparency law: France, Canada, Switzerland. What each node serves is set at the gateway and changes as better weights land.
The code
Four repositories, all public. One of the project's aims is that another community could stand up a cluster like this one, and that is not possible if the recipe is private.
| Repository | What it is |
|---|---|
| zai-ops | Ansible and Proxmox. Builds a cluster from a bare machine and operates it. |
| Corliss | Django. The session broker, the member surface, and this page. |
| scn-member-registry | ATProto lexicons and the code that serves them. The record of who is a member. |
| scn-sync-relay | Rust. The sync server behind shared notes. |
The repository is still called zai-ops
after the project's earlier name. Renaming it would break the one URL that
bootstraps a new cluster, which was not worth the churn.
Read on: what this is, and the team that builds it.