AI

Backend structure for work that runs unattended — queues, workers, and machines that hand jobs to each other.

The model call is the cheap part. The rest is scheduling, isolation, routing and verification. The bias throughout is toward mechanisms you can inspect with ordinary tools: the filesystem instead of a broker, a directory name instead of a status column, an exit code instead of a model's opinion that it succeeded.

a queue where the directory is the status

A job is a file. A worker claims it by moving it, then moves it again to record the outcome. No server, no daemon; the move is atomic, so two workers cannot take one job. A job that keeps failing parks — it neither blocks the queue nor disappears.

 pending --claim--> claimed --> done
                       |
                       +------> failed --retry--> parked

 job file    declares: input, verify command, dependencies
 state       is the directory it currently sits in
 record      one result file + one log per job, kept after the run

one process per job

Each job gets its own process and its own timeout. Batches of scenes in one shared interpreter do not survive: one hangs or takes the interpreter down without raising anything catchable, and the run dies partway with no sign of which input killed it. Isolated, that costs one job and a log entry. The price is process startup per job.

routing work to the machine that can do it

Two machines over a private network, split on purpose. Work needing more model than the laptop has crosses to the desktop as one of two commands. The split is a safety boundary, not a convenience.

 laptop                     desktop
 +--------+  think ------>  no tools bound   (answers only, cannot touch files)
 | orch.  |  act   ------>  executor         (emits a plan; acts only on commit)
 +--------+
     far side unreachable --> exit non-zero, never a silent local fallback

agents as workers

Model tiers are workers; a controller hands them tasks. Every task declares its dependencies and a command that proves it worked — the exit status decides, not the worker's report. Roles are pinned where work is dispatched, not requested in a prompt.

 task        depends_on: [...]   verify: <command>   tier: cheap | strong
 dispatch    one controller at a time, held by a lock
 failure     dead worker relaunched once, then the task parks and the loop continues
 recovery    a crashed run is cleaned up on next start, not by hand
 unattended  batches scheduled against a hardware wake timer: wake, drain, sleep

an agent that looks at what it just did

For game and interface work the thing to check is not a number, it is whether the screen looks right. So the loop closes visually, and checking the fix is a separate step from making it.

 build --> run --> capture frames --> model reads the frames
   ^                                          |
   |                                    proposed fix
   |                                          |
   +---------------- apply -------------------+
              then re-check, as its own step

 also: resumable sessions, non-blocking checkpoints, cheap/strong tiers per step

The oldest piece here and the least tidy — it depends on a shared component kept outside the project, so it is a design worth describing rather than something to pick up and run.

a shader environment built to be described

One renderer, two front ends: a hot-reloading window for a person, and a headless capture that writes frames at chosen times. Same shader, same uniforms, same pixels — so a model can be handed the actual output instead of a description of it. That closed the round-trip cost of asking whether a visual change landed.

 renderer    shared; windowed and headless differ only in the window layer
 capture     frames at given times and size --> images a model reads back
 overlay     separate binary: two windowing libraries cannot co-link on Windows
 chat        two append-only files, one per direction; single writer each
 model call  none in the app -- it shells out to a local CLI

controlling one machine from another

A pair of programs that let a laptop drive a desktop as if sitting at it: one side captures the screen, encodes it and sends it; the other displays it and sends keyboard, mouse and clipboard back. Go, the operating system's own capture path, hardware H.264 — a remote screen is only useful if it keeps up.

 transport   encrypted; client pins the host by certificate fingerprint and
             aborts before any data moves if it does not match
 session     long random token, constant-time compare, one controller at a time,
             idle disconnect, rate-limited input, address allow-list
 honesty     what is not done yet (session-token rotation) is written down as such

tools and skills

AreaWhat it is used for here
GoThe remote-control pair, the render farm's job handling, and the agent services. Chosen for single-binary deployment and for concurrency that is easy to reason about.
PythonEverything that runs inside the 3D application headlessly — rendering jobs, scene probing, thumbnail capture.
PowerShellThe orchestration layer: queue workers, job submission, status and recovery, scheduling.
C / cgoBridging Go to the platform capture, video-encoding and OpenGL libraries.
GLSL / OpenGLThe shader environment: one renderer driving both a live window and headless frame capture.
Video encodingReal-time H.264 compression of captured frames, plus measuring the codec path rather than assuming it.
Network securityModern TLS, certificate pinning, constant-time token comparison, address allow-listing, rate limiting, single-session enforcement.
Local model runtimesRunning quantised models on consumer hardware and serving them behind one endpoint.
Agent toolingDefining typed operations a model may call, and the protocol that carries them into a running application.
Vision checkingReading a rendered screen back with a model to judge whether a change did what it was supposed to. Using vision models, not training them.
Queues and schedulingAtomic claiming, dependency ordering, retry and parking policy, crash recovery, hardware wake timers.
TestingQueue invariants and codec behaviour covered by tests that run without the heavy software installed.
Version controlDistributed for source; a centralised depot with exclusive checkout for binary art.

how this is put together

Four rules run through all of it. Status is where a file lives, not a variable someone keeps updated. Verification is an exit code, not a claim. Failure is bounded to one job. Recovery happens on the next start, not by hand. None of that is specific to models — it is ordinary batch-system design — but it is what makes a model something you can put in a pipeline rather than sit and watch.

Scope, plainly: systems and orchestration around models — scheduling, routing, isolation, verification. Not model training, and no computer-vision work.

← back