Cite - judge documents with Jev, not a generative LLM

News Announcing
, ,
1

Hi everyone!

I’ve just released Cite 0.1.0. It checks documents against rules you write, and every answer points to the exact passages it came from.

The problem

You have a document and rules for what to look for in it:

  • Which lines of this log are failures, and of what kind?
  • Where are the sponsor reads in this video’s captions?
  • Which allergens does this recipe contain?
  • Did the support agent verify the customer before the refund?

You want answers you can check against the text, not a model’s summary.

The solution

Declare what to look for in a Spark DSL, hand over the passages you already have, and Cite returns findings that cite those passages exactly as you gave them:

defmodule Riddles do
  use Cite.Policy

  concern :riddle do
    detect do
      question "Does {passage} pose a riddle?"
      yes "A question asked to be puzzled over, whether or not it has an answer."
      no "A plain question, a statement, or a remark."
    end
  end
end

source = Cite.source(["Why is a raven like a writing-desk?", "Take some more tea."])
Cite.judge(client, source, Riddles)
# one :riddle finding, citing "Why is a raven like a writing-desk?"

The model is Jev, TypeSafe’s decision model. It answers with a probability, a level or a choice, never with prose, so nothing is generated and nothing can be misquoted.

How it works

  • Two fixed rounds: a screen of every passage, then one judgment per finding with all of its passages in view.
  • Gathering is plain Elixir: fixed rules, no model calls between the rounds.
  • Verdicts: :holds, :review (a person decides) or :fails. A failed request never counts as a “no”.
  • Policy mistakes are compile errors.

Tutorials

Each policy was built on a real public dataset, tuned on one split, and checked on held-out data against a keyword baseline.

Notes

  • It needs a TypeSafe API key. Providers are a behaviour, so Cite isn’t tied to Jev. I’m not affiliated with TypeSafe.
  • By design, it asks nothing about the whole document and cites only passages you gave it.

I’d especially like to hear:

  • Whether the DSL reads well, and what you’d rename;
  • Which model provider you’d want next;
  • What you’d point it at.

Hex: cite | Hex
Docs: Cite v0.1.0 — Documentation

6 Likes