smalldecideGet the code
Documentation / API

Three question types.
One interface.

Define the question and its answer space. The model scores the candidates and returns structured probabilities.

Review a state

Python
from smalldecide import SmallDecide, Noul

model = SmallDecide.from_pretrained("artifacts/model", device="cpu")
result = model.review(
    {"message": "Please refund the duplicate charge."},
    {"refund": Noul("Is a refund requested?")},
)
response = result.to_dict()

state is any JSON value. questions is a nonempty mapping of your keys to typed question objects, or their JSON equivalents. Questions share the state, but each candidate is encoded independently. Include the complete comparison context in the state when the decision depends on other options.

The loader defaults to device="cpu". The optional model="smalldecide" argument to review() labels the response; it does not switch checkpoints. HTTP and CLI JSON requests accept the same optional model field.

Choice · select a candidate

Provide at least two distinct, nonempty string keys and descriptions. The highest-probability key becomes choice.

Python
Choice("Which team handles this?", {
    "billing": "Charges and refunds",
    "technical": "Software faults",
})
Equivalent JSON
{
  "type": "choice",
  "instructions": "Which team handles this?",
  "criteria": {
    "billing": "Charges and refunds",
    "technical": "Software faults"
  }
}

Score · use an ordered rubric

Provide at least two levels, lowest first. The answer includes a distribution over indices and their expected value. With three levels the score ranges from 0 to 2.

Python
Score("Rate urgency.", ["Routine", "Time-sensitive", "Emergency"])

In JSON, use "type": "score" and an array for criteria.

Noul · estimate P(true)

A yes/no question returns noul, the estimated probability that the statement is true. Optional criteria describe the two outcomes.

Python
Noul("Is a refund requested?")

Noul("Does the message describe a software fault?", {
    "true": "A bug, outage, or integration failure is described",
    "false": "No software fault is described",
})

In JSON, use "type": "noul". The aliases "binary" and "boolean" are accepted. Criteria keys must be "true" or "false".

Response and usage

result.to_dict() returns an object with model, answers, and usage. Each answer retains the question’s key and uses its canonical type (choice, score, or noul); binary and boolean inputs return noul. Download a complete recorded request and response from v0.3.0.

FieldMeaning
choiceSelected candidate key.
probabilitiesCandidate probabilities summing to one, for Choice and Score.
scoreExpected zero-based rubric index.
legendScore index to rubric description.
noulEstimated probability that the statement is true.
confidenceDistribution concentration for Choice and Score. It is not the probability of being correct.
usage.input_tokensState and question instruction tokens, counted once.
usage.output_tokensZero. The model does not generate text.
usage.candidate_token_evaluationsTokens across all candidate prompts, including repeated state. A workload count, not a bill or a direct cost measurement.

Local HTTP transport

Start smalldecide playground --model artifacts/model --device cpu, then send a request to the local server:

Terminal
curl http://127.0.0.1:8080/v1/review \
  -H 'Content-Type: application/json' \
  -d '{"state":{"message":"Refund the duplicate charge"},
       "questions":{"refund":{"type":"noul",
       "instructions":"Is a refund requested?"}}}'

The transport uses the same request and response shape as Python. GET /health returns a basic health response. The local server has no authentication and serializes inference requests.

Calibration and confidence

A temperature per question type was fitted on validation groups separate from model selection and testing. This does not guarantee calibrated probabilities on new inputs. Some measured errors are extremely overconfident.

Confidence means concentration. For K candidates with probabilities p, confidence = 1 − H(p) / log K. A value near one means almost all probability mass is assigned to one candidate. It does not establish that the answer is accurate. The published routing gate instead uses the largest value in probabilities, with threshold 0.8502035140991211. Substituting the confidence field changes the gate. For a new domain, measure calibration and missed failures on representative data and select thresholds on a separate validation set.

Limits and errors

  • The exported checkpoint allows up to 16,384 tokens per candidate. Over-length prompts raise an error; they are not silently truncated.
  • There is no API-level cap on candidate or question count. Runtime and memory constrain large requests; candidates are evaluated in batches.
  • The local HTTP request body limit is 16 MiB.
  • Malformed questions, missing fields, or over-length inputs raise validation errors. The HTTP handler returns 422 with an error message for these cases.

Choice requires two or more options, Score requires two or more levels, and every question needs instructions. A review must contain at least one question.