Kay-X Voice API

Speech in, structured data out, speech back. Integration reference for the voice intent endpoints.

Contents

1. The model

Kay-X answers exactly one question: did I understand the caller? It never decides whether you can act — stock levels, balances and safety checks are yours. The boundary is fixed, and every endpoint sits on one side of it.

The whole flow, once

There is one endpoint you send voice to, and three possible answers. Everything else is detail.

next.endpoint already contains the right path, so you can route on it directly rather than writing the branch yourself.

2. Authentication

Runtime endpoints accept either. Never put an API key in browser JavaScript — it has no origin restriction and spends credits. Browser → your backend → Kay-X.

3. Two integration styles

Pick one before you write anything. They are not mixed.

4. POST/api/v1/voice/intent

multipart/form-data. This is the only endpoint that takes audio.

Text variant, for testing

POST/api/v1/voice/intent/text — same pipeline without transcription. Cheaper, instant, and it does not write to the training pipeline, so test data stays out.

{ "business_id": "clinic", "session_id": "t1",
  "text": "Take Amoxicillin three times a day for five days" }

5. The response, field by field

{
  "business_id":   "clinic",
  "message_id":    "wa-0001",
  "session_id":    "whatsapp-23276123456",
  "audio_ref":     "gs://…",
  "transcript_raw":"Take Amoxicillin three times a day for five days",

  "intent":  { "name": "new_prescription", "confidence": 0.95 },

  "slots": {
    "medicine_name": { "name":"medicine_name", "value":"Amoxicillin",
                       "raw_text":"Amoxicillin", "confidence":1.0 },
    "duration_days": { "name":"duration_days", "value":5,
                       "raw_text":"five days",  "confidence":0.95 }
  },

  "needs_clarification": false,
  "resolved": true,
  "final_values": { "medicine_name":"Amoxicillin", "duration_days":5 },

  "next": { "action": "done" },

  "normalizer_candidates": [ … ]
}

6. next — what to do

"next": {
  "action":   "ask",
  "say":      "Fo Ciprofloxacin, omoch tem pa dey?",
  "say_en":   "For Ciprofloxacin, how many times per day?",
  "audio":    "UklGRj…",
  "endpoint": "POST /api/v1/voice/intent (same session_id)",
  "item":     1,
  "missing":  ["frequency", "duration_days"]
}

say is Krio and is meant to be spoken. say_en is English and is for your logs — never show it to a caller. audio is say already synthesized as base64 WAV, present when the business has speak clarifications enabled. Recovery wording repeats, so after the first time it is served from cache and costs nothing.

const { next, final_values } = await postVoiceNote(audio);

switch (next.action) {
  case 'done':    return fillForm(final_values);
  case 'ask':
  case 'confirm': await play(next.audio) ?? speak(next.say);
                  return waitForReply(next.action);
}

7. POST/api/v1/voice/resolve

Where a confirm goes. Message style only. Free — it costs no credits.

Confirm

{ "message_id": "wa-0001", "reply": "yes" }

Affirmative: yes y yeah yep confirm ok okay correct sure fine. Negative: no n nope cancel wrong reject stop.

Correct a value

{ "message_id": "wa-0001", "correction": { "duration_days": 7 } }

Correct one entry of a repeating group

{ "message_id": "wa-0001", "item": 1,
  "correction": { "frequency": "2x daily", "duration_days": 7 } }

Only that entry changes. Fixing the third medicine never makes the caller re-dictate the first two.

Response

{ "message_id":"wa-0001", "status":"resolved",
  "intent":"new_prescription",
  "final_values": { "medications": [ … ] } }

A rejection returns "status":"rejected" and no final_values — do not act on it. 409 means already resolved; treat as terminal.

8. POST/api/v1/voice/reject

Context style only. For when the caller was understood perfectly and your validation refuses the value — an unregistered wallet, a meter that will not verify, a drug you do not stock.

This is a different message from “I did not hear you”. A caller who gave a clear answer needs to know it was understood and still will not work. Without this endpoint you would have to reach into the context token you are meant to treat as opaque, just to undo one field.

{
  "context": "<token from the previous turn>",
  "slot":    "recipient_phone",
  "value":   "+23276123456",
  "reason":  "not registered on Orange Money",
  "message": "Da nomba de no de na Orange Money. Duya gi wan oda nomba."
}

Returns a turn state whose say holds the question to ask next, with audio when spoken recovery is on. If you omit message, the slot's configured reject_prompt is used; if that is also empty, the caller simply hears the slot's normal question again.

9. POST/api/v1/voice/respond

The output half. Turns your own event label plus values into spoken Krio. Nothing to do with understanding — you call it when you have acted.

{
  "businessId": "clinic",
  "intent":     "prescription_saved",
  "slots":      { "medicine": "Amoxicillin", "amount": "500" },
  "sessionId":  "whatsapp-23276123456",
  "speakerId":  "jojo"
}

{
  "text":       "Yu prescripshon don rich. Amoxicillin faiv hondred leones.",
  "audio":      "UklGRj…",
  "templateId": "…", "variantId": "…",
  "cacheHit":   false,
  "requiresQa": false
}

What actually happens

1. Your intent label is looked up in this business's bindings to find a template. No binding → 404.
2. One variant of that template is chosen by the binding's policy (weighted_random by default), so the assistant does not repeat itself word for word.
3. Your slots are rendered into the template: "500" becomes faiv hondred leones, a phone number becomes digits one at a time.
4. The result is synthesized to speech, or served from cache.
5. The assembly is logged for QA.

Free text, no template

POST/api/v1/voice/respond/text — raw Krio straight to speech. Max 30 words / 300 characters. Never cached, always flagged for QA. Use it for one-offs; use templates for anything repeated.

10. Repeating groups

When one utterance names several things — “Amoxicillin three times a day, and Paracetamol twice” — the response gains items, one entry per thing, each judged on its own.

"items": [
  { "index":0, "label":"Amoxicillin",
    "needs_clarification": false,
    "final_values": { "medicine_name":"Amoxicillin", "duration_days":5 } },

  { "index":1, "label":"Ciprofloxacin",
    "needs_clarification": true,
    "missing_slots": ["frequency","duration_days"] }
],
"next": { "action":"ask", "item":1,
          "say":"Fo Ciprofloxacin, omoch tem pa dey?" }

Amoxicillin passed and stays passed. Only the incomplete entry is asked about, by name. One entry per turn — “for Amoxicillin how many days and for Paracetamol how many times” is not answerable out loud.

When everything resolves, the rows appear under the group's own name:

"final_values": {
  "medications": [
    { "medicine_name":"Amoxicillin", "frequency":"3x daily", "duration_days":5 },
    { "medicine_name":"Paracetamol", "frequency":"2x daily", "duration_days":3 }
  ]
}

One field to prefill a form from. One entry or ten, same shape — no branching.

11. Follow-up answers: fold, don't merge

Read this section even if you skip the rest. It is where integrations quietly corrupt data.

Every voice note is its own round trip. We hold no conversation state — the session_id groups turns for context, but each response describes that utterance, not your accumulated form. Statelessness is deliberate: it means you can retry, replay, load-balance and scale without us holding a lock on your conversation. The cost is that stitching turns together is yours.

What it buys the caller

The whole point is that nobody re-dictates. A clinician forced to repeat an entire prescription because one name did not come through has been handed something worse than a pen.

Turn 2 is one word, and row #1 is never re-heard or re-written. That only holds if you fold.

Why merging by index breaks

When the missing field is the one that identifies an entry — medicine_name — there is no label to hang the reply on. So the answer comes back one of three ways:

The rule

Remember what you asked. Recognise a reply that carries only the answer. Fold it into the row that was waiting.

1. Remember the question — on every ask:

pending = { item: next.item, missing: next.missing };

2. Classify each returned item. It is an answer fragment when all three hold:

function isAnswerFragment(item, pending, target) {
  const filled = FIELDS.filter((f) => isFilled(item.slots?.[f]?.value));
  if (filled.length === 0) return false;

  // A different medicine is a different medicine, not an answer.
  if (isFilled(item.slots?.medicine_name?.value) &&
      isFilled(target.medicine_name) &&
      !sameName(item.slots.medicine_name.value, target.medicine_name)) return false;

  // Everything it carries is either what we asked for, or a gap in the row.
  return filled.every((f) => pending.missing.includes(f) || !isFilled(target[f]));
}

That middle guard earns its keep: it stops "and also add Paracetamol" being swallowed into the row you were asking about.

3. Fold fragments, merge the rest.

const fragments = items.filter((i) => isAnswerFragment(i, pending, rows[pending.item]));
const rest      = items.filter((i) => !fragments.includes(i));

let next = mergeByIndex(rows, rest);              // real medicines, as usual
next = foldInto(next, pending.item, union(fragments));

When folding: fill gaps, overwrite only fields you actually asked about, and never overwrite a value the user typed by hand.

4. Recompute completeness yourself. After a fold, the response's needs_clarification, missing_slots and next.item describe our item list, which no longer lines up with yours.

rows = rows.map((row) => {
  const missing = REQUIRED.filter((f) => !isFilled(row[f]));
  return { ...row, missing, needsClarification: missing.length > 0 };
});

If every row is complete you are done — enable Save even though the response said ask. If something is still open, re-point the question at your first incomplete row.

5. Do not trust final_values after a fold. It will hold a different number of entries than your rows, and pairing them off by position writes the wrong medicine.

const base = final_values.medications.length === rows.length
  ? final_values.medications   // aligned — use it verbatim
  : null;                      // folded — the form is the source of truth

Test these four

1. Two medicines, the second nameless. Answer with the name only. → two rows, row 1 byte-identical, row 2 completed.
2. Same, but the answer arrives as a third item. → still two rows.
3. Answer with a different, complete medicine. → three rows; it was not folded.
4. The answer arrives in place. → no fold, nothing changed.

Test 1 is the one that matters. If answering about the second medicine disturbs the first, the fold is wrong — and everything else can look fine while that is still broken.

12. Errors

Every error is {"error": "..."}.

404 doubles as “not yours” deliberately, so a caller cannot probe which ids exist. Do not write copy that distinguishes them.

13. Checklist

• Send message_id on every /voice/intent, or retries double-charge.
• Reuse session_id across turns, or context is lost.
• Branch on next.action, and treat ask and confirm differently.
• Speak next.say. Log next.say_en. Never the reverse.
• Fill forms from final_values, not slots.
• Fold follow-up answers into the row that was asked about (§11). Merging by index corrupts the form in two of three cases.
• Recompute completeness locally after a fold — the response describes our item list, not yours.
• Check final_values length against your row count before saving. After a fold they no longer line up, and pairing by position writes the wrong entry.
• Never let a fold overwrite a value the user typed by hand.
• Ignore normalizer_candidates — strip it server-side so it cannot reach a UI by accident.
• Send plain values to /voice/respond — "500", not "faiv hondred".
• Handle 402 globally with a top-up prompt.
• Keep API keys server-side.
• Casing differs by half: the intent layer is snake_case, the response layer is camelCase. Check the examples per endpoint.