ReqLLM - Composable LLM client built on Req

News Announcing
, , ,
13

Congratulations on launching ReqLLM! I really like the wide support of many providers and the composable design.

One quick question: you mentioned cost tracking, but I can’t find how to enable it - e.g. the following script only prints token usage, but not cost, even after running mix req_llm.model_sync

alias ReqLLM.Response

ReqLLM.put_key(:google_api_key, System.fetch_env!("GOOGLE_API_KEY"))
ReqLLM.put_key(:openai_api_key, System.fetch_env!("OPENAI_API_KEY"))
ReqLLM.put_key(:xai_api_key, System.fetch_env!("XAI_API_KEY"))

models = ["openai:gpt-5-nano", "xai:grok-4-fast-non-reasoning", "google:gemini-2.5-flash"]

models
|> Enum.each(fn model ->
  {:ok, response} = ReqLLM.generate_text(model, "What model are you?")

  text = Response.text(response)
  usage = Response.usage(response)
  IO.puts("#{model}: #{text}\n(#{inspect(usage)}\n\n")
end)

prints only token usage:

openai:gpt-5-nano: I’m ChatGPT, a large language model created by OpenAI. I’m based on the GPT-4 architecture (the GPT-4 family of models). If you want, I can share more about my capabilities or limitations.
(%{input_tokens: 11, output_tokens: 824, total_tokens: 835}


xai:grok-4-fast-non-reasoning: I'm Grok, built by xAI.
(%{input_tokens: 132, output_tokens: 9, total_tokens: 142, live_search_sources: 0}


google:gemini-2.5-flash: I am a large language model, trained by Google.
(%{input_tokens: 6, output_tokens: 11, total_tokens: 17, cached_content_token_count: 0, total_token_count: 42}
1 Like