Inference API

Call open models through an OpenAI-compatible API. Any OpenAI SDK or tool works: change the base URL and use a Flint API key. Usage is billed per token from your balance.

Base URL and key

https://flint.monster/api/v1

Create a key under Inference → Create API key (or Settings → API keys). Keys start with pk_live_ and are shown once, so store yours somewhere safe. Send it as a bearer token:

Authorization: Bearer pk_live_…

Models

The Models list on the Inference page shows what's available now, with live status and prices. At the time of writing:

List them from code with GET /models.

Chat completions

curl https://flint.monster/api/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Explain GPUs in one sentence."}],
    "max_tokens": 100
  }'

from openai import OpenAI

client = OpenAI(base_url="https://flint.monster/api/v1", api_key="pk_live_…")

reply = client.chat.completions.create(
    model="llama-3.1-8b-instruct",
    messages=[{"role": "user", "content": "Explain GPUs in one sentence."}],
)
print(reply.choices[0].message.content)
print(reply.usage)  # prompt_tokens, completion_tokens

Streaming

Set stream: true to receive tokens as they're generated (server-sent events). The last chunk before data: [DONE] carries usage.

stream = client.chat.completions.create(
    model="qwen2.5-1.5b-instruct",
    messages=[{"role": "user", "content": "Count to ten."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)

Completions

POST /completions takes a prompt instead of messages, for the same chat models.

Embeddings

curl https://flint.monster/api/v1/embeddings \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "nomic-embed-text-v1.5", "input": ["search_query: rent a GPU"]}'

emb = client.embeddings.create(
    model="nomic-embed-text-v1.5",
    input=["search_document: GPUs by the minute", "search_query: rent a GPU"],
)
print(len(emb.data[0].embedding))  # 768

Nomic Embed works best with a task prefix: search_query: for queries and search_document: for the text you index.

Billing

Each request is metered from the usage the model returns: input tokens at the model's input price, output tokens at its output price. Charges are taken from your balance every few minutes in whole cents, so very small usage accumulates until it reaches a cent. Inference → Usage this month shows requests, tokens and cost per model.

Errors

Errors use the OpenAI format: {"error": {"message": …, "type": …, "code": …}}.

The OpenAI SDKs retry 429 and 503 automatically.

Limits

• Context length per request is listed per model above.
• Requests are limited per key; ask [email protected] if you need more.