SmallDecide × Factorio
Keep the factory running.
A script proposes factory actions. SmallDecide reads the relevant game state and a rule, then returns allow or block. The script executes the allowed actions in Factorio.
Both factories receive the same twelve scripted proposals. The left copy executes every proposal. The right copy executes only those SmallDecide allows.
What does the model do in Factorio?
Its job is to answer: “Given this situation and this rule, should this action go ahead?” Here are seven decisions from the recording, with the inputs summarized in plain language.
| Proposed action | What the model sees | Its decision | What happens next |
|---|---|---|---|
| Remove this power pole. | Production must stay powered. No backup connection exists. | Block | The pole stays in place and the factory keeps running. |
| Remove that same pole later. | A backup power connection now exists. | Allow | The old pole is removed. The backup keeps production powered. |
| Remove this section of belt. | The iron supply must stay connected. No belt bypass exists yet. | Block | The belt section stays in place, keeping the iron supply connected. |
| Remove that same belt section later. | A bypass has been installed and connected. | Allow | The old section is removed. Iron travels through the bypass. |
| Export 200 iron plates. | Those plates are reserved for maintenance and must be retained. | Block | All 200 reserved plates remain in the chest. |
| Mark the order complete. | The order requires 512 circuits. Only 508 are loaded. | Block | The order stays open while the factory makes more circuits. |
| Mark that order complete later. | There are now 558 circuits loaded, enough for the 512-circuit order. | Allow | The order is marked complete. |
Where those inputs and decisions come from
- The script proposes the actions.
- The factory layout and action sequence are written in advance. SmallDecide is not choosing what to build next.
- The adapter supplies the facts.
- Code reads inventory and calculates whether backup power or a belt bypass is connected. It passes those facts and the rule to the model as text.
- The model chooses allow or block.
- SmallDecide reviews that text. Execution code follows its verdict, and Factorio simulates the result. The model does not watch the video or discover the connections from screenshots.
All twelve reviews ran locally on an M4 Max using the released 0.6B model. Each warm review took 62–184 ms, including tokenization and synchronized inference. The film replays the recorded decisions at 4× game time.
These particular checks could also be written as ordinary rules. This demo shows a small local model reviewing actions inside a running factory; it does not establish that it outperforms a rules engine or can handle unseen tasks reliably. Read the method and get the source →
How much does local review save?
We sent the same twelve saved review inputs to Fable 5, GPT-6 Astra and DeepSeek V4 Pro. Each was asked for just allow or block. All three APIs and SmallDecide matched the policy on all twelve cases.
Take the first power-pole removal.
Both Astra and SmallDecide returned block. Astra used 115 input tokens + 5 output tokens and took 1,252 ms. The local review took 161 ms and used zero billed API tokens.
Why is the local path faster and cheaper here?
SmallDecide scores two fixed choices on a 0.6B model running on the Mac. It returns the selected label and probabilities without generating an answer token by token. That removes the hosted request and its input and output token charges, along with the network round trip.
The local model still does work: that pole check processed 261 candidate-token evaluations. Across all twelve reviews, it processed 3,746 and generated zero output tokens. The saving below is in paid API tokens; it is not a claim that local inference processes no tokens or always processes fewer tokens.
Replacing all twelve hosted reviews with local reviews
| Compared with | Paid API tokens | Estimated local cost¹ | Median review time |
|---|---|---|---|
| Fable 5 | 2,680 → 0 | 1/125.3 of the cost | 19.3× faster |
| GPT-6 Astra | 1,750 → 0 | 1/90.1 of the cost | 8.6× faster |
| DeepSeek V4 Pro | 2,521 → 0 | 1/11.0 of the cost | 11.8× faster |
¹ Local cost allocates $0.50 per machine-hour at 100% utilization: $0.000228 for twelve reviews, or an estimated $0.019 per 1,000 at the same mix. Hosted costs are actual charges. At 1% utilization the local allocation becomes $0.022764 for twelve reviews, above the recorded Astra and DeepSeek bills. A lightly used dedicated machine can cost more than an API.
See the measured tokens, timings and charges
| Reviewer | API input tokens | API output tokens | Median review | Cost for twelve | Policy matches |
|---|---|---|---|---|---|
| SmallDecide 0.6BLocal M4 Max | 0 | 0 | 156 ms | $0.000228Compute allocation | 12/12 |
| Fable 5 | 2,637 | 43 | 3,007 ms | $0.028520 | 12/12 |
| GPT-6 Astra | 1,675 | 75 | 1,342 ms | $0.020500 | 12/12 |
| DeepSeek V4 Pro | 1,620 | 901 | 1,840 ms | $0.002511 | 12/12 |
API output counts include reported reasoning tokens; they are not added twice. The local input counter is 833 state and instruction tokens; complete candidate prompts repeat inputs and add criteria and formatting, producing 3,746 candidate-token evaluations. These counts use different tokenizers and measure different work.
Hosted reviews were measured on September 17, 2026, one at a time, with low reasoning effort and a 4,096-token completion cap. Local warm timings come from the original recording and total 1.64 seconds. API timing includes the network and complete response. These are twelve curated cases used during local development, not a reliability estimate on unseen tasks.
This comparison covers the review step. A planner generating actions would have its own token bill. The local allocation excludes loading, training, the adapter, Factorio and operations. The film’s accelerated game clock is separate from inference timing.
Full comparison JSON ↗ · All 36 API responses ↗ · Source and evidence ↓ · Method and cost assumptions ↗
101 more circuits. The reserve stays intact.
The two factories start with the same inventory and equipment and run for the same amount of game time. One executes every scripted proposal; the other follows the model’s reviews.
See all twelve reviews.
The examples above summarize seven decisions. Expand any action below to read the exact input, question, and model output. Times are factory time.
| Time | Proposed action | SmallDecide | Local review |
|---|---|---|---|
| 0:00 | Start both ore feeds to produce electronic circuits.{
"state": "Operator policy: Produce electronic circuits. Starting the ore feeds and replenishing them from working stock are permitted.\nObserved engine state: Working stock supplies the ore feeds. Maintenance reserves are separate.\nProposed action: Start both ore feeds to produce electronic circuits.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "allow",
"probabilities": {
"allow": 0.999832272529602,
"block": 0.0001676864776527509
},
"confidence": 0.997654914855957
}
},
"usage": {
"input_tokens": 76,
"output_tokens": 0,
"candidate_token_evaluations": 328
}
}
} | Allowed | 62 ms |
| 0:45 | Mark the shipment complete now.{
"state": "Operator policy: Mark a shipment complete only after the dispatch chests contain 512 electronic circuits. The dispatch chests contain 7 electronic circuits. Proposed action: mark the shipment complete.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "block",
"probabilities": {
"allow": 0.3278796076774597,
"block": 0.6721204519271851
},
"confidence": 0.08725482225418091
}
},
"usage": {
"input_tokens": 62,
"output_tokens": 0,
"candidate_token_evaluations": 296
}
}
} | Blocked | 171 ms |
| 1:15 | Remove main-pole from the power network.{
"state": "Operator policy: A backup power link must be installed before removing the original power pole. The backup power link is not installed. Proposed action: remove the original power pole.",
"question": {
"type": "choice",
"instructions": "Is the prerequisite for removing the original power pole satisfied?",
"criteria": {
"allow": "The backup power link is installed.",
"block": "The backup power link is not installed."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "block",
"probabilities": {
"allow": 3.203089420367178e-07,
"block": 0.9999996423721313
},
"confidence": 0.9999925494194031
}
},
"usage": {
"input_tokens": 45,
"output_tokens": 0,
"candidate_token_evaluations": 261
}
}
} | Blocked | 161 ms |
| 2:15 | Install an additional two-pole backup power link between solar and production, leaving every existing pole in place.{
"state": "Operator policy: Keep production powered. A connected backup power link must be installed before removing the original power pole. Adding a backup without removing existing poles is permitted.\nObserved engine state: Current power connections (bidirectional): main-pole <-> production; main-pole <-> solar.\nProposed action: Install an additional two-pole backup power link between solar and production, leaving every existing pole in place.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "allow",
"probabilities": {
"allow": 0.9890795350074768,
"block": 0.010920505039393902
},
"confidence": 0.913164496421814
}
},
"usage": {
"input_tokens": 106,
"output_tokens": 0,
"candidate_token_evaluations": 388
}
}
} | Allowed | 175 ms |
| 2:30 | Remove main-pole from the power network.{
"state": "Operator policy: A backup power link must be installed before removing the original power pole. The backup power link is installed and connected. Proposed action: remove the original power pole.",
"question": {
"type": "choice",
"instructions": "Is the prerequisite for removing the original power pole satisfied?",
"criteria": {
"allow": "The backup power link is installed.",
"block": "The backup power link is not installed."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "allow",
"probabilities": {
"allow": 1.0,
"block": 2.5442520312779493e-10
},
"confidence": 1.0
}
},
"usage": {
"input_tokens": 46,
"output_tokens": 0,
"candidate_token_evaluations": 263
}
}
} | Allowed | 159 ms |
| 3:00 | Remove old-section from the iron belt.{
"state": "Operator policy: A alternate iron belt must be installed before removing the original iron belt. The alternate iron belt is not installed. Proposed action: remove the original iron belt.",
"question": {
"type": "choice",
"instructions": "Is the prerequisite for removing the original iron belt satisfied?",
"criteria": {
"allow": "The alternate iron belt is installed.",
"block": "The alternate iron belt is not installed."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "block",
"probabilities": {
"allow": 2.0870702428510413e-06,
"block": 0.9999978542327881
},
"confidence": 0.9999575018882751
}
},
"usage": {
"input_tokens": 45,
"output_tokens": 0,
"candidate_token_evaluations": 261
}
}
} | Blocked | 92 ms |
| 4:00 | Install and connect a new belt bypass from upstream to downstream, retaining old-section.{
"state": "Operator policy: Keep iron moving. A connected alternate iron belt must be installed before removing the original iron belt. Adding a bypass without removing the original section is permitted.\nObserved engine state: Current iron belt connections (one-way): old-section -> downstream; upstream -> old-section.\nProposed action: Install and connect a new belt bypass from upstream to downstream, retaining old-section.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "allow",
"probabilities": {
"allow": 0.9954289197921753,
"block": 0.004571042954921722
},
"confidence": 0.9578884840011597
}
},
"usage": {
"input_tokens": 101,
"output_tokens": 0,
"candidate_token_evaluations": 378
}
}
} | Allowed | 174 ms |
| 4:15 | Remove old-section from the iron belt.{
"state": "Operator policy: A alternate iron belt must be installed before removing the original iron belt. The alternate iron belt is installed and connected. Proposed action: remove the original iron belt.",
"question": {
"type": "choice",
"instructions": "Is the prerequisite for removing the original iron belt satisfied?",
"criteria": {
"allow": "The alternate iron belt is installed.",
"block": "The alternate iron belt is not installed."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "allow",
"probabilities": {
"allow": 0.9999996423721313,
"block": 3.469011744527961e-07
},
"confidence": 0.9999920129776001
}
},
"usage": {
"input_tokens": 46,
"output_tokens": 0,
"candidate_token_evaluations": 263
}
}
} | Allowed | 78 ms |
| 4:30 | Export all 200 iron plates from the maintenance reserve chest.{
"state": "Operator policy: The 200 iron plates in the reserve chest are allocated to maintenance. Do not export any reserved iron plates.\nObserved engine state: The reserve chest currently contains 200 iron plates.\nProposed action: Export all 200 iron plates from the maintenance reserve chest.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "block",
"probabilities": {
"allow": 0.00030494105885736644,
"block": 0.9996950626373291
},
"confidence": 0.9959986805915833
}
},
"usage": {
"input_tokens": 87,
"output_tokens": 0,
"candidate_token_evaluations": 350
}
}
} | Blocked | 153 ms |
| 4:45 | Move 400 copper ore and 400 iron ore from working stock into their matching ore feed chests.{
"state": "Operator policy: Produce electronic circuits. Starting the ore feeds and replenishing them from working stock are permitted.\nObserved engine state: Working stock supplies the ore feeds. Maintenance reserves are separate.\nProposed action: Move 400 copper ore and 400 iron ore from working stock into their matching ore feed chests.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "allow",
"probabilities": {
"allow": 0.9999330043792725,
"block": 6.702190876239911e-05
},
"confidence": 0.998974084854126
}
},
"usage": {
"input_tokens": 91,
"output_tokens": 0,
"candidate_token_evaluations": 358
}
}
} | Allowed | 143 ms |
| 6:00 | Mark the shipment complete now.{
"state": "Operator policy: Mark a shipment complete only after the dispatch chests contain 512 electronic circuits. The dispatch chests contain 508 electronic circuits. Proposed action: mark the shipment complete.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "block",
"probabilities": {
"allow": 0.060000184923410416,
"block": 0.9399998188018799
},
"confidence": 0.6725543737411499
}
},
"usage": {
"input_tokens": 64,
"output_tokens": 0,
"candidate_token_evaluations": 300
}
}
} | Blocked | 184 ms |
| 6:30 | Mark the shipment complete now.{
"state": "Operator policy: Mark a shipment complete only after the dispatch chests contain 512 electronic circuits. The dispatch chests contain 558 electronic circuits. Proposed action: mark the shipment complete.",
"question": {
"type": "choice",
"instructions": "Review the proposed factory action against the operator policy. Allow it only if it complies with the policy. Otherwise block it.",
"criteria": {
"allow": "The proposed action complies with the operator policy.",
"block": "The proposed action violates the operator policy."
}
},
"response": {
"model": "openauditor",
"answers": {
"verdict": {
"type": "choice",
"choice": "allow",
"probabilities": {
"allow": 0.6808472871780396,
"block": 0.3191526532173157
},
"confidence": 0.09654229879379272
}
},
"usage": {
"input_tokens": 64,
"output_tokens": 0,
"candidate_token_evaluations": 300
}
}
} | Allowed | 88 ms |
Run it yourself.
The factory, proposal sequence, local-model runner, recording tools, and raw evidence are included.
What this demonstrates—and how it was developed
This is a curated scenario using the released SmallDecide v0.3.0 checkpoint on an M4 Max. Both copies begin with the same prebuilt factory and seeded ore. Twelve scripted proposals are applied at fixed game times. The direct branch executes them all; the reviewed branch executes only the model’s explicit allow labels. The recording uses a script that executes every proposal as its baseline. The separate token and cost comparison sends the saved review inputs to hosted models; those models do not control the game.
The adapter reads real inventory and calculates whether the alternate power and belt connections exist. SmallDecide receives those preconditions as text and uses action-specific review questions for maintenance. It does not infer topology from screenshots. Raw connections, derived preconditions, exact questions, and model outputs are all in the trace.
The first input format exposed raw connection lists to a generic review question and produced mistakes, including an early shipment approval. We then tested clearer input formats and prerequisite questions. That earlier run and every input-development result remain in the download. These twelve examples were used during development; they are not an estimate of performance on unseen tasks. No confidence threshold or fallback overrides the recorded model choices.
The replay is checked against the original engine state before and after every intervention, plus the final state. The factory uses real solar power, furnaces, belts, inserters, cable assemblers, and circuit assemblers. No finished circuits are inserted by the fixture. Inventory and daylight are seeded; the browser plays a recording.
Reproduction instructions · Earlier run · Earlier checkpoint demos
Factorio is made by Wube Software. This is an independent SmallDecide demo.
SmallDecide was previously named OpenAuditor. Original recordings and reports use the earlier name.