Reading the score

The four bands

Every response carries the ceiling that was applied to the text you sent. Read it, then sort the score into one of four bands. Three of them are a request for a human; one of them is not.

const { spam_probability: score, max_confidence: ceiling } = await predict(text);

// Yours, not ours. A deterministic tell you can defend to the person whose
// message you held. We do not publish a list: what corroborates depends on
// what you host, and a list from us would become a contract we do not test.
const corroborated = yourOwnSignal(text);

if (score > ceiling) {
  // CONFIRMED. Nothing the model can do on its own reaches here: the
  // calibrated score is clamped to the ceiling before anything else runs.
  // The only thing above it is a human-confirmed spam report on this exact
  // text, from your own account. Act on this band unconditionally.
  autoAct();
} else if (score >= ceiling) {
  // CENSORED. "At least this", not "this". The model is as confident as it
  // is permitted to be on its own word and the true value could be anywhere
  // above. Unranked: a 419 letter and an invoice arrive identical. Do not
  // measure the gap to a threshold. You may act here too, but only on a
  // corroborating signal you own -- otherwise queue it.
  corroborated ? autoAct() : review();
} else if (score >= 0.5) {
  // GRADED. The only band where the number ranks, and so the only band where
  // arithmetic on it means anything. Sort your queue by it, blend it, weight
  // it -- all legitimate here.
  review();
} else {
  accept();
}

Four bands, not two. Collapsing the last two throws away the only distinction that tells you whether a high score is evidence or an artefact of the clamp — confidence ceiling has the measurements behind that.

Where a given number came from

Four things can set the value you receive, and two of them are not the classifier. When a score looks wrong, this is the table to read:

One inference is sound; its converse is not

Sound: a score above the ceiling means a human-confirmed spam report exists for that exact text on your account. The clamp is applied before corrections are folded in, so nothing else can get there.

Not sound: a score at or below the ceiling does not mean the text is uncorrected. A confirmed spam report clears the ceiling only when the model already read the text at roughly 0.59 or above; report something the model scored 0.50 and the corrected score lands near 0.80 — still under the ceiling, still labelled medium, and indistinguishable from an uncorrected graded score. A confirmed ham report lands below the ceiling by construction.

So the comparison tells you when you may auto-act. It does not tell you that everything underneath is the model talking unaided. If you need that distinction, keep your own record of what you reported — POST /v1/feedback returns a report_count you can store alongside the decision.

likelihood is a label, not a routing field

The response also carries a coarse bucket. It is there so a dashboard, an audit log, or an admin column can show something readable without picking a threshold. It is the wrong field to branch on, for two reasons:

• high is unreachable from the model alone. It starts at the block band, which sits above the ceiling by construction. On an integration with no feedback loop the high bucket never fires, and a branch keyed to it is not mis-tuned — it is inert, which is much harder to notice than wrong.
• medium spans two different things. It runs from 0.50 to the block band, so it covers the graded band and the censored ceiling together. A medium that ranks and a medium that cannot be ranked at all arrive under the same label.

Log it, display it, tag your records with it. Route on spam_probability against max_confidence.

Choosing thresholds

Your review threshold is yours. Start at 0.50, then validate it against your own traffic and your own tolerance for false positives. The calibration was fitted on short user-generated and form text — comments, contact and signup forms, account requests — so that is the population the number is meaningful over. Pointing the endpoint somewhere else, such as inbound email, is a reasonable thing to do and a decision you own.

Your auto-act threshold should not be a constant at all. Any number you choose is wrong in one of two ways. Below the ceiling, you are auto-acting on the model's unaided word — the thing we measured and withdrew, after two production sites enforcing at the block band refused 16 of 38 ordinary transactional requests in one four-hour window. At or above the ceiling, the branch cannot fire from the model at all. The honest rule has no constant in it: score > max_confidence.

The part most migrations miss. Replacing a hardcoded 0.85 with > max_confidence does not by itself make the branch reachable. Nothing exceeds the ceiling until a confirmed spam correction exists for that text, so an integration that never reports anything back has swapped one inert branch for another. If you want an automatic path at all, you have to close the loop — POST /v1/feedback is what makes that band reachable, and it does not consume your prediction quota.

Two worked patterns

Same three comparisons, opposite failure postures. Which way to fail is not a style choice; it follows from which of the two errors costs more in your medium.

Public broadcast: fail closed

# Public broadcast -- fail CLOSED. An unclassified comment published to
# every reader of a thread is worse than one comment held for a moderator.
async def process_new_comment(comment, site) -> Comment:
    if not site.spam_filter:
        comment.status = (
            CommentStatus.APPROVED if site.auto_approve else CommentStatus.PENDING
        )
        return comment

    result = await predict(comment.body)          # None on timeout/5xx
    if result is None:
        comment.status = CommentStatus.PENDING     # fail closed
        return comment

    score, ceiling = result["spam_probability"], result["max_confidence"]
    comment.spam_score = score

    if score > ceiling:
        # Confirmed spam for this exact text. Safe to auto-hide.
        comment.status = CommentStatus.SPAM
    elif score >= settings.SIFTFY_REVIEW_THRESHOLD:   # yours to pick; 0.50 to start
        comment.status = CommentStatus.PENDING
    elif site.auto_approve:
        comment.status = CommentStatus.APPROVED
    else:
        comment.status = CommentStatus.PENDING
    return comment


# Close the loop. Without this the `score > ceiling` branch above can never
# fire, because nothing else in the system exceeds the ceiling.
async def on_moderator_decision(comment, verdict: str) -> None:
    await siftfy_feedback(comment.body, "spam" if verdict == "spam" else "ham")

Private inbox: fail open

# Private inbox -- fail OPEN. One borderline message delivered beats
# legitimate mail disappearing during a transient outage.
async def classify_message(user_id: str, msg_id: str) -> SpamDecision:
    msg = await repository.get_message(user_id, msg_id)
    if msg.get("folder") != "inbox":
        return SpamDecision(verdict="skipped")

    if (pref := await preference_action(user_id, msg)) == "block":
        await repository.update_message_folder(user_id, msg_id, "spam")
        return SpamDecision(verdict="blocked", moved_to_spam=True)
    if pref == "allow":
        return SpamDecision(verdict="allowlisted")

    try:
        payload = await predict(build_detector_text(msg)[:20_000])
    except Exception as exc:
        await repository.update_message_spam_metadata(
            user_id, msg_id, verdict="unavailable", error=str(exc)[:300]
        )
        return SpamDecision(verdict="unavailable", error=str(exc))  # fail open

    score = float(payload["spam_probability"])
    ceiling = float(payload["max_confidence"])

    if score > ceiling:
        verdict = "spam"                  # confirmed; move it
    elif score >= settings.spam_suspicious_threshold:
        verdict = "suspicious"            # annotate, keep in Inbox
    else:
        verdict = "clean"

    await repository.update_message_spam_metadata(
        user_id, msg_id, verdict=verdict, probability=score,
        # The censored band and the graded band are both "suspicious", so the
        # verdict does not separate them -- but only one of them may be sorted
        # by. Persist that, and the Suspicious view can order the graded ones
        # and leave the clamped ones unranked instead of implying a ranking
        # the number does not carry.
        ranked=score < ceiling,
        model=f"siftfy:{payload.get('likelihood', '')}",   # label, not routing
    )
    if verdict == "spam":
        await repository.update_message_folder(user_id, msg_id, "spam")
    return SpamDecision(verdict=verdict, probability=score)


# "Report spam" / "Not spam" is the loop. Keep the local sender preference --
# it is a faster, cheaper short-circuit -- and report the text as well.
async def on_user_report(user_id: str, msg, label: str) -> None:
    await write_sender_preference(user_id, msg, label)
    await siftfy_feedback(build_detector_text(msg)[:20_000], label)

A comment is republished to everyone who reads the thread, so an unclassified one is better held than shown. A message is addressed to one person who is waiting for it, so losing legitimate mail to a transient outage costs more than delivering one borderline message. Both of these are ours; see EchoThread and Emcognito WebMail for the fuller write-ups.

Replacing a hardcoded threshold

Then delete the constant rather than moving it. A threshold that lives in an environment variable still has to be right, still goes stale when the ceiling rises, and now goes stale in a place nobody reviews.

Getting a better result

• Send the text the score was fitted on, or accept that you are extrapolating. Short user-generated and form text is the calibrated population.
• Send enough of it. For mail, subject plus sender plus body carries the signal far better than the body alone, which is what our own webmail found.
• Rank rather than threshold where you can. What the model does reliably is order a corpus. A moderation queue sorted by score gets more out of it than any single cut.
• Close the loop. Corrections take effect on the next prediction, are scoped to your account, and are the only way to reach the band you may act on unconditionally.
• Never cache max_confidence. It is per-language and deliberately temporary. A copy in your code goes stale without failing, and it fails looking like a model change rather than a constant change.

Where to next

• POST /v1/predict → the request and response reference, and the measurements behind the ceiling.
• POST /v1/feedback → report a correction, and what it does to the next score.
• Content Gate → the same scoring rules as a gate decision, for form submissions.