# RelayKin Gateway: Integration Guide

This guide is for a developer or AI agent connecting an application to WhatsApp, Telegram or SMS through the RelayKin Gateway. One API key and secret can reach WhatsApp, Telegram and SMS accounts through the same endpoints and webhooks (see [Telegram accounts](#telegram-accounts) and [SMS accounts](#sms-accounts) for the differences). With it your application can:

- **Send** WhatsApp and Telegram messages: text, image, video, audio, **voice notes**, documents, stickers and locations.
- **Receive** incoming messages and status events, pushed to your webhook (signed) or fetched by polling.
- **Download** media that contacts send: images, voice notes, documents and so on.
- **Voice:** send text that arrives as a voice note, and get incoming voice notes as text (when enabled on the portal).
- **AI:** generate text with `POST /v1/ai/respond` on the ChatGPT account connected to the portal (when connected).

The machine-readable contract is in [`openapi.yaml`](openapi.yaml). This file explains how to use it.

## Try the API in your browser

The portal serves an interactive reference (Swagger UI) at `https://<PORTAL_DOMAIN>/api-docs`. It lists every endpoint with its parameters, examples and errors, and lets you send real requests:

1. Press **Authorize** and enter your API key (`X-API-Key`) and API secret (`X-API-Secret`).
2. Open an endpoint and press **Try it out**.

- Use a **test key** (see [Sandbox (test keys)](#sandbox-test-keys)) there. With a live key, requests send real messages and change real data.
- If your key has an IP allowlist, it must allow the IP address of the browser you use, or every request returns `403 ip_not_allowed`.
- The portal does not store anything you enter on that page.

---

## 1. What you need

The portal admin creates a key under **Portal → Integrations → New API key** and gives you:

| Value | Example | Use |
|---|---|---|
| Base URL | `https://app.relaykin.ai/v1` | Prefix for every API call |
| API key | `kwa_3f9c1a...` | `X-API-Key` header (public identifier) |
| API secret | `kws_Jx8...` | `X-API-Secret` header (**keep secret**, shown only once) |
| Webhook signing secret | `whsec_9Qa...` | Verifies webhooks the gateway sends you |

Store them as environment variables, never in source code:

```sh
RELAYKIN_BASE_URL=https://app.relaykin.ai/v1
RELAYKIN_API_KEY=kwa_...
RELAYKIN_API_SECRET=kws_...
RELAYKIN_WEBHOOK_SECRET=whsec_...
```

The admin also chooses which accounts (WhatsApp, Telegram or SMS) the key may use, an optional webhook URL, an optional event filter, and an optional IP allowlist.

> **Production base URL:** `https://app.relaykin.ai/v1` (valid TLS certificate, served through Cloudflare).
> The previous addresses, `https://relaykin.signalkin.ai/v1` and `https://kinwa.signalkin.ai/v1`, still work for existing integrations and webhooks (only browser pages there redirect). Move to the new one when convenient.
> When the admin sets an IP allowlist on your key, list the IP your server uses to call out, **IPv6 included** if it
> has one: the gateway sees the address Cloudflare received your request from.

---

## 2. Authentication

Send both headers on **every** request, over HTTPS only:

```
X-API-Key: kwa_...
X-API-Secret: kws_...
```

- A missing or wrong key or secret returns `401 unauthorized`.
- A request from an IP that isn't on the key's allowlist returns `403 ip_not_allowed`.
- Using an account the key isn't allowed to use returns `403 account_not_allowed`.
- A call the key's permission does not cover returns `403 not_permitted`. A `full` key can call everything. A `read` key can call every `GET` route except `GET /v1/webhook`, plus `POST /v1/ask`, `POST /v1/search` and `POST /v1/mcp`. A `send` key can call only `POST /v1/messages`, `GET /v1/accounts`, `GET /v1/accounts/{account_id}`, `GET` and `DELETE /v1/queue/{queue_id}`, `GET /v1/accounts/{account_id}/messages/{message_id}`, `POST /v1/content-check` and `POST /v1/mcp`. A send key cannot poll `GET /v1/events`, so a system that needs incoming events should use a webhook (the admin sets it on the key) or a `read` or `full` key.
- An expired key returns `401 key_expired`. While the portal's owner has suspended API access, every call returns `503 suspended`.
- If the secret leaks, the admin can rotate it with **New secret**; the old secret stops working immediately.

**Rate limit:** 20 requests per second per key, with bursts up to 40. Beyond that the gateway returns `429 rate_limited` with a `Retry-After` header.

---

## 3. Quick start

```sh
# 1. Which accounts can I use? Each has "channel": "whatsapp", "telegram" or "sms".
curl -s "$RELAYKIN_BASE_URL/accounts" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET"
# → {"data":[{"id":1,"name":"Sales MY","phone":"60123456789","status":"connected","connected":true,"last_seen":"...","channel":"whatsapp"}]}

# 2. Send a text from account 1
curl -s "$RELAYKIN_BASE_URL/messages" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET" \
  -H "Content-Type: application/json" \
  -d '{"account_id":1,"to":"60198765432","type":"text","text":"Hello from our app 👋"}'
# → {"data":{"message_id":"3EB0C1...","status":"...","account_id":1,"to":"60198765432","type":"text"}}
```

Only send from accounts with `"connected": true`. Otherwise the call returns `409 account_not_connected`.

---

## 4. Sending messages: `POST /v1/messages`

All message types share one endpoint. The JSON body:

| Field | Type | Required | Notes |
|---|---|---|---|
| `account_id` | integer | yes | The WhatsApp account (sender), from `GET /v1/accounts` |
| `to` | string | yes | International number with country code (`60198765432`; `+`, spaces and dashes are allowed), **or** a JID: `...@s.whatsapp.net`, a group `...@g.us`, or `...@lid`. To reply, pass the incoming event's `data.chat_id`. |
| `type` | string | yes | `text`, `image`, `video`, `audio`, `voice`, `document`, `sticker`, or `location` |
| `text` | string | for `text`; optional for `voice` | Up to 65,536 bytes. For `voice` without media: the text read aloud (text-to-voice), up to 4,096 characters |
| `media_url` | string | media types* | Public `http(s)` URL; the account's server downloads it |
| `media_base64` | string | media types* | Base64 file content; a `data:...;base64,` prefix is allowed |
| `filename` | string | no | Filename for uploads (shown for documents) |
| `caption` | string | no | For `image`, `video`, `document` |
| `reply_to` | string | no | Message ID to quote (reply to) |
| `view_once` | boolean | no | For `image` and `video` |
| `latitude`, `longitude` | number | for `location` | Decimal degrees |
| `tts_voice` | string | no | Text-to-voice only: voice name (provider voices such as `alloy`, `nova`, `shimmer`, `echo`, `onyx`). Default is set by the admin |
| `tts_instructions` | string | no | Text-to-voice only: how to speak, e.g. `"Warm, friendly, Malaysian English accent"` |
| `template_id` | integer | no | Send a saved message template instead of `text`: leave `text` out and use `type: "text"` (or omit `type`). The template id is shown under Settings → Message templates. |
| `variables` | object | with `template_id` | Values for the template's `{{variables}}`, e.g. `{"name":"Aina","order":"A12"}`. Every variable the template uses is required, except `phone`, which defaults to the recipient. Unknown names are refused. |
| `translate_to` | string | no | Translate `text` (or `caption`) with AI before sending: `"auto"` uses the chat's language (see 6, Translation), or give a language code such as `"ms"`. With `"auto"` and no known language, the text is sent unchanged. Needs the ChatGPT connection; metered as AI use. |

\* Media types need **exactly one** of `media_url` or `media_base64` (except `voice` sent with `text`; see below). The request body can be up to 32 MB, which is about 24 MB of file data after base64 encoding. WhatsApp's own size limits still apply.

### Examples

```jsonc
// Text, quoting a received message
{"account_id":1,"to":"60198765432","type":"text","text":"Thanks, noted!","reply_to":"3EB0ABC123"}

// Image by URL with caption
{"account_id":1,"to":"60198765432","type":"image","media_url":"https://cdn.example.com/promo.jpg","caption":"Our new menu"}

// Document upload
{"account_id":1,"to":"60198765432","type":"document","media_base64":"JVBERi0xLjcK...","filename":"invoice-1042.pdf","caption":"Invoice #1042"}

// Location
{"account_id":1,"to":"60198765432","type":"location","latitude":3.1390,"longitude":101.6869}

// Group message
{"account_id":1,"to":"120363025555555555@g.us","type":"text","text":"Hello team"}

// A saved template: "Hi {{name}}, order {{order}} is on its way."
{"account_id":1,"to":"60198765432","template_id":7,"variables":{"name":"Aina","order":"A12"}}
```

**Templates:** a key uses only its own organisation's templates (a key of the portal's owner, the owner's). Another organisation's template, or one that does not exist, returns `404 template_not_found`. A missing or unknown variable, or a filled-in message too long for the account's channel (4,096 characters on WhatsApp and Telegram, 1,600 on SMS), returns `400 invalid_request`.

### Voice notes (`type: "voice"`)

`voice` sends a real WhatsApp **voice note**: it plays inline with a waveform and shows the blue mic when played.

- Send any common audio format, such as MP3, WAV, M4A/AAC or OGG, as `media_url` or `media_base64`.
- The gateway converts it automatically to the format WhatsApp requires (OGG/Opus, mono, 48 kHz).
- This works well for text-to-speech output: generate the audio, base64 it, and send it.

```jsonc
{"account_id":1,"to":"60198765432","type":"voice","media_base64":"<base64 of tts.mp3>","filename":"reply.mp3"}
```

`audio` sends the file as a regular audio attachment instead, shown as a music file rather than a voice note.

#### Text-to-voice

Send `type: "voice"` with `text` and **no** media, and the gateway generates the speech for you and sends it as a voice note:

```jsonc
{"account_id":1,"to":"60198765432","type":"voice","text":"Hi Ali, your order has shipped and arrives tomorrow.","tts_voice":"nova"}
```

- Up to 4,096 characters per voice note.
- Takes a few seconds longer than a normal send (the audio is generated first).
- `400 tts_disabled` means text-to-voice isn't enabled on this portal; generate the audio yourself and send `media_base64`.
- `502 tts_failed` means the speech provider failed; retry with backoff like a `502`.

### Response

```json
{"data":{"message_id":"3EB0C127D7BACC83D6A1","status":"...","account_id":1,"to":"60198765432","type":"voice"}}
```

Keep `message_id`: delivery and read receipts (`message.ack` events) and replies refer to it.

With `translate_to`, the response also has `translation`: `{"text":"what was sent","language":"ms"}`, plus `"skipped": true` when the text was already in that language and went out unchanged. It is left out when nothing was translated.

---

## 5. Receiving messages

There are two ways to receive events, and you can use both:

| | Webhook (push) | Polling (pull) |
|---|---|---|
| Setup | Admin sets a **webhook URL** on the key | Nothing extra |
| Latency | About 1 second | Your poll interval |
| Reliability | Retried for ~23 hours if your endpoint is down | Events kept for **7 days** |
| Best for | Production | Development, firewalled apps, backfill after an outage |

### 5.1 Event envelope (same for webhook and polling)

```json
{
  "id": 1842,
  "event": "message",
  "account_id": 1,
  "account": {"id":1,"name":"Sales MY","phone":"60123456789","status":"connected","connected":true,"last_seen":"2026-11-05T09:40:00Z"},
  "created_at": "2026-11-05T09:41:12.512Z",
  "data": {
    "id": "3EB0C127D7BACC83D6A1",
    "chat_id": "60198765432@s.whatsapp.net",
    "from": "60198765432@s.whatsapp.net",
    "from_name": "Ali",
    "sender_display_name": "Ali",
    "timestamp": "2026-11-05T09:41:12Z",
    "is_from_me": false,
    "body": "Hi, is the shop open today?"
  }
}
```

- `id` is unique and increasing. Use it to **deduplicate**, because a webhook can arrive more than once, and as the polling cursor.
- An incoming one-to-one `message` can carry `language`: the contact's language (ISO 639-1, such as `ms`) as known when the message arrived, either set by staff or detected from earlier messages, or shown by the message's script (Chinese, Thai, Arabic and so on). It is left out when not known yet, and never set on your own messages or groups.
- `event` is the event type (see 5.6).
- `data` is the WhatsApp event, passed through as-is.

### 5.2 Media in incoming messages

When a contact sends media, `data` contains the media field (`image`, `video`, `audio`, `document`, `sticker` or `video_note`) plus two fields added by the gateway:

```json
"data": {
  "id": "3EB0...", "chat_id": "60198765432@s.whatsapp.net", "from": "60198765432@s.whatsapp.net",
  "audio": "statics/media/1752404905-b939.ogg",
  "media_info": {"mime_type": "audio/ogg; codecs=opus", "seconds": 7, "ptt": true, "file_length": 12034},
  "media_type": "audio",
  "media_url": "https://app.relaykin.ai/v1/media/1/1752404905-b939.ogg"
}
```

- `media_info.ptt: true` means it's a **voice note**.
- Download the file with `GET media_url`, **with the same `X-API-Key` and `X-API-Secret` headers**. The response is the raw file with its `Content-Type`.
- Ignore the raw `statics/...` path; it's internal to the account's server.
- Download promptly. Media files are **deleted from the account's server after 30 days** (the admin can change this); after that `GET media_url` returns `404 media_not_found`. Save your own copy of anything you need to keep.
- Captions appear in `data.body`.

### 5.3 Voice note transcription

When enabled on the portal, every incoming voice note is transcribed automatically. You receive the normal `message` event first (with `media_info.ptt: true`), and a few seconds later a **separate** event with the text:

```json
{
  "id": 1843,
  "event": "message.transcribed",
  "account_id": 1,
  "data": {
    "message_id": "3EB0C127D7BACC83D6A1",
    "chat_id": "60198765432@s.whatsapp.net",
    "from": "60198765432@s.whatsapp.net",
    "from_name": "Ali",
    "timestamp": "2026-11-05T09:41:12Z",
    "media_url": "https://app.relaykin.ai/v1/media/1/1752404905-b939.ogg",
    "seconds": 7,
    "text": "Hi, is the shop open today?"
  }
}
```

- `data.message_id` is the `data.id` of the original `message` event; use it to match the two.
- If transcription fails you get `message.transcription_failed` with the same fields plus `error` instead of `text`. Fall back to downloading the audio, or ask the contact to type.
- Not transcribed: audio files that are not voice notes, voice notes the account sent itself (`is_from_me`), and voice notes longer than the admin's limit (default 10 minutes). Those produce no transcription event.
- **Simple bot pattern:** for `message` events with `media_info.ptt: true`, wait for the matching `message.transcribed` and treat its `text` like a typed message.

### 5.4 Webhooks

The gateway sends a `POST` to your webhook URL with the envelope as the JSON body and these headers:

```
Content-Type: application/json
X-RelayKin-Event: message
X-RelayKin-Event-Id: 1842
X-RelayKin-Signature: t=1793870472,v1=5f2c9a...e1
```

Each webhook also carries the same three values under the previous names, `X-KinWA-Event`, `X-KinWA-Event-Id` and `X-KinWA-Signature`, so integrations built before the rename keep working. New integrations should read the `X-RelayKin-…` names.

**Your endpoint must:**
1. **Verify the signature** before trusting the body (see below).
2. Return any **2xx within 15 seconds**. Do slow work, such as AI replies, asynchronously after acknowledging.
3. **Deduplicate** on `X-RelayKin-Event-Id` (or the envelope `id`).

**Retries:** after a non-2xx response or a timeout, the gateway retries after 30s, 2m, 10m, 30m, 1h, 3h, 6h and 12h, then gives up. You can still fetch the event by polling for 7 days.

**Delivery log and resends:** the portal's Integrations page has a delivery log for each key. It shows every delivery, the HTTP status and the start of your endpoint's answer, and how long it took. Staff can resend one delivery, or every failed delivery in a date range. A resend:

- is signed afresh, with the time it is sent in `X-RelayKin-Signature`, so your 5-minute check still passes;
- carries the same `X-RelayKin-Event-Id` and envelope `id` as the first try, so deduplicating on it drops a resend you already handled;
- adds `X-RelayKin-Replay: 1` and `X-RelayKin-Replay-Of: <delivery id>`, the portal's id of the delivery being repeated.

```
X-RelayKin-Event: message
X-RelayKin-Event-Id: 1842
X-RelayKin-Signature: t=1793956872,v1=9a0d...c4
X-RelayKin-Replay: 1
X-RelayKin-Replay-Of: 5531
```

Only events still kept (7 days by default) can be resent.

#### Webhook health, pausing and probes

The portal scores your webhook every minute from its last hour of deliveries: how many succeeded, how many failed in a row, how long your endpoint took to answer, and how often it did not answer at all. The score (0 to 100) and its state show next to the key and in the delivery log:

| State | When |
|---|---|
| `healthy` | Most deliveries succeed (also shown before there are any) |
| `degraded` | 2 or more failures in a row, or a score under 90 over at least 5 deliveries |
| `failing` | 5 or more failures in a row, or under half succeeding over at least 10 deliveries |

When a webhook becomes degraded or failing, the portal's admins and, for a customer's key, that customer's managers are told through their alert channels (quiet hours apply).

If **every** delivery has failed for an hour (the portal's owner can change this), deliveries are **paused**:

- New events are not retried against your endpoint. They wait as `paused` deliveries, kept like the rest of the delivery log (up to 10,000 per key by default; the oldest are dropped beyond that, and events older than 7 days can no longer be sent). Polling `GET /v1/events` keeps working throughout.
- Every 10 minutes the portal sends a **probe**: an envelope with `"event": "probe"`, `"id": 0` and `"data": {"probe": true, ...}`, signed like any webhook, with the header `X-RelayKin-Probe: 1`. Answer it with a 2xx like any other event; don't process it as a message.
- After 2 successful probes in a row, deliveries resume by themselves. Staff can also resume from the key's delivery log. Staff can pause by hand too; a webhook paused by a person is not probed and stays paused until someone resumes it. Changing the webhook URL resumes.
- On resume, the events that waited are sent **in order, oldest first, one at a time**; events arriving meanwhile queue behind them. Each carries `X-RelayKin-Replay: 1` and `X-RelayKin-Replay-Of: <delivery id>` (its own delivery id), and its original event id, so deduplication works as for resends.

#### Signature verification

`v1` is the hex HMAC-SHA256 of `"<t>.<raw request body>"`, using the webhook signing secret. Compute it over the **raw bytes** of the body, before any JSON parsing. Reject requests whose `t` is more than 5 minutes old, to block replays.

**Node.js (Express):**
```js
import crypto from "node:crypto";
import express from "express";

const app = express();
app.post("/whatsapp/webhook", express.raw({ type: "application/json" }), (req, res) => {
  const header = req.get("X-RelayKin-Signature") || "";
  const parts = Object.fromEntries(header.split(",").map(p => p.split("=")));
  const t = Number(parts.t);
  const expected = crypto.createHmac("sha256", process.env.RELAYKIN_WEBHOOK_SECRET)
    .update(`${t}.`).update(req.body).digest("hex");
  const ok = parts.v1 && parts.v1.length === expected.length &&
    crypto.timingSafeEqual(Buffer.from(parts.v1), Buffer.from(expected)) &&
    Math.abs(Date.now() / 1000 - t) < 300;
  if (!ok) return res.sendStatus(401);

  const evt = JSON.parse(req.body);
  res.sendStatus(200);              // acknowledge fast
  handleEvent(evt).catch(console.error); // then process (dedupe on evt.id)
});
```

**Python (FastAPI):**
```python
import hmac, hashlib, json, os, time
from fastapi import FastAPI, Request, Response, BackgroundTasks

app = FastAPI()
SECRET = os.environ["RELAYKIN_WEBHOOK_SECRET"].encode()

@app.post("/whatsapp/webhook")
async def webhook(request: Request, background: BackgroundTasks):
    raw = await request.body()
    parts = dict(p.split("=", 1) for p in request.headers.get("X-RelayKin-Signature", "").split(",") if "=" in p)
    t = int(parts.get("t", "0"))
    expected = hmac.new(SECRET, f"{t}.".encode() + raw, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(parts.get("v1", ""), expected) or abs(time.time() - t) > 300:
        return Response(status_code=401)
    background.add_task(handle_event, json.loads(raw))  # dedupe on event["id"]
    return Response(status_code=200)
```

To test, the admin can press **Test webhook** in the portal, or you can call `POST /v1/webhook/test`. Either one sends an event with `"event":"test"` and `"id":0`.

#### Changing the webhook later

You don't need a webhook on day one. Start with polling, and set, change or remove the webhook at any time with your own key:

```sh
curl -s -X PUT "$RELAYKIN_BASE_URL/webhook" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET" -H "Content-Type: application/json" \
  -d '{"url":"https://your-app.example.com/whatsapp/webhook"}'
# → {"data":{"url":"https://your-app.example.com/whatsapp/webhook","polling_only":false}}
```

- `GET /v1/webhook` shows the current URL.
- The URL must be `https` and resolve to a public address, otherwise you get `400 invalid_webhook`. Redirects aren't followed.
- `{"url":""}` removes the webhook. Undelivered events are cancelled but stay available through polling.
- Changing the URL sends undelivered events to the new one straight away. Events from before a webhook existed are not sent; fetch those by polling.
- The signing secret stays the same. Then call `POST /v1/webhook/test` to confirm it works.

### 5.5 Polling: `GET /v1/events`

```sh
curl -s "$RELAYKIN_BASE_URL/events?after=1842&limit=100" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET"
# → {"data":[ <envelope>, ... ], "next_after": 1907}
```

- `after`: return events with `id` greater than this. Start at `0`, then use `next_after` from the previous response, and persist it.
- `limit`: 1–500 (default 100). `account_id` (optional) filters to one account.
- Poll every 2–10 seconds. When `data` is empty, keep the same cursor.

### 5.6 Event types

Most events come from WhatsApp; the rest are added by the gateway. The key's event filter can limit which ones you receive. New event types may be added, so ignore any you don't recognize.

| Event | Meaning |
|---|---|
| `message` | Incoming (or own, `is_from_me: true`) message of any type |
| `message.ack` | Delivery/read receipt for a sent message |
| `message.reaction` | Emoji reaction added or removed |
| `message.edited` / `message.revoked` / `message.deleted` | Message edited, deleted for everyone, or deleted for me |
| `chat_presence` | Contact is typing or recording |
| `group.participants` / `group.info` / `group.joined` | Group membership and name changes |
| `message.transcribed` / `message.transcription_failed` | Text of an incoming voice note (see 5.3); added by the gateway |
| `message.tagged` | AI labels for a message: `tags.intent`, `tags.language`, `tags.urgency`. Sent separately, a few seconds after the `message` event (see 6) |
| `reply.suggested` | The AI drafted a reply for approval: `draft_id`, `draft`, `chat_id`, `contact`. While the account runs a live A/B test of its AI setup, also `ai_variant`: `control` or `variant`, the group the contact is in (left out otherwise) |
| `conversation.handoff` / `conversation.resumed` | A chat moved to a person, with `reason`, or back to the AI. A group handed over by the group assistant has `group: true` |
| `group.ai_replied` | The group assistant answered in a group where it is turned on: `chat_id`, `message_id`, `reply_to` (the message it quoted), `trigger` (`mention`, `trigger word` or `reply`) and `text`. Group `message` events can carry `mentioned_jids`, `mentions_me`, `replied_to_id`, `replied_to_participant` and `replied_to_me` (see the webhook payload documentation) |
| `conversation.extracted` | Structured data the AI filled in from a chat changed, or staff corrected it: `schema_id`, `schema`, `status` (`complete` or `incomplete`), `fields`, `evidence` (field → message ids), `changed`, `missing`, `corrected`, `trigger`, `source` (`ai` or `staff`). See "Structured data from chats" in section 6 |
| `contact.memory_updated` | The AI's memory of a contact changed (the AI updated it after the chat went quiet or was closed, or staff edited it): `chat_id`, `contact`, `person_id`, `summary`, `last_outcome`, `facts` (each `id`, `kind` (`fact`, `preference` or `issue`), `text`, `evidence` (message ids), `pinned`, `source` (`ai` or `staff`), `account_id`), `source` (`ai` or `staff`) and, from the AI, `trigger`. See "Customer memory" in section 6 |
| `message.voice_reply` | An AI reply to a voice note: `text`, `message_id`, `chat_id`, `source` (`ai` or `draft`), `sent_as` (`voice`, or `text` with `fallback_reason`). See "Voice-note replies" in section 6 |
| `ai.action` | The reply AI called (or asked to call) one of the business's actions, or a person decided on one: `call_id`, `action`, `tool`, `chat_id`, `status`, `input`, `http_status`, `latency_ms`, `error`. See "AI actions" in section 6 |
| `flow.started` / `flow.step` / `flow.completed` / `flow.exited` | A chat entered a conversation flow, finished a step (`step_id`, `outcome`, `field`, `value`), completed it with its `fields`, or left it early with a `reason`. See "Conversation flows" in section 6 |
| `ai.poll_answered` | The customer answered a clarifying question the reply AI asked with options (a WhatsApp poll vote, or a number typed in reply to the numbered list sent on Telegram and SMS): `chat_id`, `contact`, `message_id` (the vote or reply), `poll_message_id`, `question`, `options`, `answer` (the chosen options) and `native` (`true` for a WhatsApp poll). The original `message` event (a vote reads `Poll vote: …` with a `poll` object) is delivered as usual |
| `message.screened` | The spam check flagged a new contact's first messages: `chat_id`, `contact`, `verdict` (`spam`, `scam` or `abusive`), `score` (0–100, from the fixed rules), `reasons`, `source` (`rules` or `ai`), `action` (`tag` or `review`) and `screening_id`. A flagged contact gets no automatic reply |
| `contact.screening_decided` | A person reviewed a screened contact: `chat_id`, `contact`, `decision` (`genuine` or `spam`), `blocked` (added to the opt-out list), `verdict` and `screening_id` |
| `message.media_understood` | The AI read an incoming photo or document: `message_id`, `status` (`ok`, `skipped`, `failed`), `kind`, `description`, `text` (ID numbers masked), or a `reason`. See "Photos and documents" in section 6 |
| `message.queue_sent` / `message.queue_failed` | A held message went out (`queue_id`, `message_id`) or was given up on (`queue_id`, `code`, `error`) |
| `conversation.closed` / `conversation.rated` | Staff closed a chat (`rating_requested`), and the customer's 1 to 5 `rating`. When the account writes wrap-up notes, `conversation.closed` also has `wrap_up`: `{"issue":"...","outcome":"...","follow_up":"...","created_at":"..."}` (`follow_up` is `""` when none). It is left out when the AI could not write one. With wrap-up notes on, `conversation.closed` is sent once the note is written (up to about 45 seconds after the close) |
| `followup.sent` | A scheduled follow-up went out: `follow_up_id`, `chat_id`, `contact`, `message_id`, `text`, `language`, `reason`, `source` (`wrapup`, `agent` or `api`), `mode` (`draft` or `auto`), `approved_by`, `due_at`, `sent_at`. See "Follow-ups" in section 6 |
| `message.classified` | An incoming message's `urgency` (`urgent` or `normal`) and `mood` (`angry`, `neutral` or `happy`), with `language`, `message_id`, `chat_id` and `contact`. Sent a few seconds after the `message` event when the account classifies messages |
| `contact.consent` | Marketing consent was recorded: `status` (`granted` or `withdrawn`), `method` and `source` |
| `contact.opted_out` / `contact.opted_in` | A contact opted out or back in; see section 7 for what happens to sends |
| `call.offer` | Someone called the account. `data.auto_rejected: true` means it was declined automatically (see section 8) |
| `device.logged_out` / `device.temporary_ban` / `device.reachout_timelock` | Account problems; stop sending and alert a human |

The full field reference for `data` is in the RelayKin webhook payload documentation (`docs/webhook-payload.md` in this repository).

**Avoid reply loops:** ignore `message` events where `data.is_from_me` is `true`. These include messages your own app sent.

---

## 6. Other operations

| Method & path | Body | Purpose |
|---|---|---|
| `GET /v1/accounts/{account_id}` | | One account's status |
| `GET /v1/accounts/{account_id}/chats/{chat}/messages?limit=50&offset=0&search=` | | Stored chat history (`chat` = phone or JID) |
| `POST /v1/accounts/{account_id}/messages/{message_id}/read` | `{"chat":"60198765432"}` | Mark a received message as read (blue ticks) |
| `POST /v1/accounts/{account_id}/messages/{message_id}/reaction` | `{"chat":"60198765432","emoji":"👍"}` | React; `""` removes the reaction |
| `POST /v1/accounts/{account_id}/typing` | `{"chat":"60198765432","state":"typing"}` | Show "typing…"; `"paused"` stops it |
| `GET /v1/media/{account_id}/{file}` | | Download incoming media (use `data.media_url`) |
| `POST /v1/webhook/test` | | Send a test event to your webhook |
| `GET /v1/webhook` | | Show your webhook URL |
| `PUT /v1/webhook` | `{"url":"https://..."}` | Set, change or remove (`""`) your webhook (see 5.4) |
| `POST /v1/ai/respond` | `{"input":"...","instructions":"...","model":"..."}` | Generate text (see below) |
| `POST /v1/translate` | `{"text":"...","to":"ms"}` | Translate text (see Translation below) |
| `GET /v1/accounts/{account_id}/chats/{chat}/language` | | A chat's language: `language` (in use), `detected`, `override` |
| `PUT /v1/accounts/{account_id}/chats/{chat}/language` | `{"language":"ms"}` | Set the chat's language; `""` goes back to the detected one |

### Ask your inbox: `POST /v1/ask`

Ask a question in plain language about the conversations of the key's accounts (full and read keys; test keys cannot):

```bash
curl -s -X POST https://<PORTAL_DOMAIN>/v1/ask -H "X-API-Key: $KEY" -H "X-API-Secret: $SECRET" \
  -H "Content-Type: application/json" -d '{"question":"Who asked about refunds last week?"}'
```

`account_ids` (optional) narrows the search to some of the key's accounts; naming another account is `403 account_not_allowed`. The answer is `data.answer`, with `[C1]`-style marks matching `data.citations[]`: each has `account_id`, `chat` (the chat id), `message_ids` (messages that support the answer, readable with `GET /v1/accounts/{account_id}/chats/{chat}/messages`), a short `snippet` and `inbox_url`, which opens the chat in the portal's Inbox. `data.plan` shows how the question was read (date range, accounts, channels, labels, moods, keywords) and `data.stats` how much was searched; `partial: true` means a limit or the time budget was reached.

It is bounded: the portal searches what it already keeps (stored events, transcripts, wrap-up notes, extracted fields), then, if that found too few chats, the newest chats of up to 5 accounts on their servers; it reads at most 8 chats and 240 messages, within about 40 seconds, then answers. Two AI calls are made per question, metered like other AI use (source `ask`), with the same spend limits and errors as `POST /v1/ai/respond`.

### Chat search: `POST /v1/search`

Find chats by what they are about, the same search as the Inbox's plain-language search (full and read keys; send and test keys cannot):

```bash
curl -s -X POST https://<PORTAL_DOMAIN>/v1/search -H "X-API-Key: $KEY" -H "X-API-Secret: $SECRET" \
  -H "Content-Type: application/json" -d '{"query":"customers who complained about late delivery this week"}'
# → {"data":{"query":"...","plan":{"from":"...","to":"...","keywords":["late","delivery","lambat",...],...},
#     "results":[{"account_id":1,"account_name":"Sales MY","channel":"whatsapp","chat":"60198765432@s.whatsapp.net","chat_ref":"...",
#     "display":"60198765432","snippet":"my parcel is late again","mood":"angry","matches":3,"kinds":["message","wrapup"],"last_at":"...","inbox_url":"..."}],
#     "more":false,"elapsed_ms":640}}
```

One AI call reads the text as a date range, accounts, channels, labels, moods and search words (with variants and translations); the portal then searches what it keeps (message text, voice transcripts, photo notes, wrap-up notes and moods) and returns at most 30 chats, best first. Nothing is read from the accounts' servers and no answer is written; use `read_messages` or `GET /v1/accounts/{account_id}/chats/{chat}/messages` to open a result. The search is limited to the key's accounts in the database itself, so `account_ids` (optional) can only narrow it; naming another account is `403 account_not_allowed`. Each key may search **30 times an hour** (`429 search_limit_reached`, `Retry-After: 600`). The AI call is metered like other AI use (source `chat_search`); without an AI the text's own words are searched (`plan.guessed: true`).

### Knowledge gaps: `/v1/accounts/{account_id}/kb/gaps`

Once a week the portal groups the questions an account's AI could not answer (it handed the chat over, a rule refused the reply, no knowledge matched, or it said it did not know) into gaps, each with a drafted knowledge-base entry. Example questions are masked.

| Method & path | Keys | Purpose |
|---|---|---|
| `GET /v1/accounts/{account_id}/kb/gaps?all=1` | full, read | Open gaps (`all=1` adds approved and dismissed ones): `data[]` with `id`, `theme`, `frequency`, `signals`, `samples`, `question`, `answer`, `status`; `runs[]` and `running` |
| `POST /v1/accounts/{account_id}/kb/gaps/{gap}/approve` | full | Add the entry to the account's "Answers to knowledge gaps" FAQ source. Body `{"question":"...","answer":"..."}` changes the draft; no body approves it as drafted. Blanks such as `[opening hours]` must be filled in first (`400`) |
| `POST /v1/accounts/{account_id}/kb/gaps/{gap}/dismiss` | full | Mark the gap as not worth an entry |

A gap of another account is `404 gap_not_found`; one already decided is `409 gap_decided`. Decisions are recorded as made by "API key <name>".

### Text AI: `POST /v1/ai/respond`

Runs one text model call on the ChatGPT account the admin connected in the portal. `input` is required (up to 32,000 characters); `instructions` (the system prompt) and `model` are optional.

```sh
curl -s "$RELAYKIN_BASE_URL/ai/respond" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET" -H "Content-Type: application/json" \
  -d '{"instructions":"Reply as a friendly shop assistant, in one or two sentences.","input":"Are you open today?"}'
# → {"data":{"text":"Yes, we are open until 9pm!","model":"...","input_tokens":31,"output_tokens":12}}
```

- Text only: speech goes through `type: "voice"` and `message.transcribed`.
- This does not send anything to WhatsApp. To reply to a contact, pass `text` to `POST /v1/messages`.
- Each call is independent: send any conversation history you need inside `input`.
- Allow up to 90 seconds. There is no fallback provider, so handle the errors in section 7.

---

### Translation: `POST /v1/translate`

Translates text with the account's AI provider. `text` is required (up to 8,000 characters). `to` is a language code (`en`, `ms`, `zh`, `zh-tw`...), or `"auto"` for the language of a chat, which then needs `account_id` and `chat` (a phone number or JID). With `account_id`, translations are cached per account, text and language, so translating the same text again does not use the AI; without it nothing is cached. Calls are metered as AI use (source `translate`).

```sh
curl -s "$RELAYKIN_BASE_URL/translate" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET" -H "Content-Type: application/json" \
  -d '{"text":"Your order ships tomorrow","to":"auto","account_id":1,"chat":"60198765432"}'
# → {"data":{"text":"Pesanan anda dihantar esok","language":"ms","source_language":"en","cached":false}}
```

- `409 language_unknown`: `to` is `"auto"` and the chat's language is not known yet.
- A chat's language is detected from the contact's messages when the account has Inbox translation or message classification on, and staff can set it in the Inbox. AI auto-replies are written in it.
- To send translated, use `translate_to` on `POST /v1/messages` instead of translating and sending in two steps.

### Chat summary: `POST /v1/accounts/{account_id}/chats/{chat}/summary`

Summarizes the latest messages with the connected ChatGPT account. The body is optional: `{"limit": 100}` (1 to 200).

```json
{"data":{"summary":"- Wants a refund for order 1042\n- Refund agreed\n- Waiting on bank details","messages":37}}
```

### Chat history extras

`GET /v1/accounts/{account_id}/chats/{chat}/messages` now also returns `transcript` on transcribed voice notes and `tags` on tagged messages, and `media_text` on photos and documents the AI read (or `media_status` saying why it did not; see "Photos and documents" below).

### Group details: `GET /v1/accounts/{account_id}/groups/{group}`

Returns a group's details, including its name. `{group}` is the group JID, with or without `@g.us`. The `message` event doesn't include the group name, so call this when you need it, and cache the result.

### Handing a chat to a person: `PUT /v1/accounts/{account_id}/chats/{chat}/handoff`

Stops the AI in a chat, or gives the chat back. Only relevant when the account has AI replies on.

```json
{"status":"human","reason":"VIP customer"}
```

`status` is `human` or `ai`.

### AI drafts

When an account is set to suggest replies, `GET /v1/drafts` lists open drafts for the accounts your key may use.

- `POST /v1/drafts/{draft_id}/send` sends one. An optional `{"text":"..."}` replaces the draft's text.
- `POST /v1/drafts/{draft_id}/dismiss` discards one.

A draft that was already handled returns `409 draft_closed`.

### Structured data from chats: `GET /v1/accounts/{account_id}/chats/{chat}/extracted`

An admin or the customer's manager can define extraction schemas in the portal (Insights tab), for one account or for every account of a customer: named fields with a type (`text`, `number`, `date`, `phone`, `email`, `enum`, `boolean`), a description and a required flag, and when they run (after the customer's messages once they pause, when the chat goes to a person, when it is closed). The AI reads the chat's latest messages and fills the fields in. Each value is checked against its type and must name the messages it came from; an invalid answer is retried once, and if it is still invalid the valid fields are kept and the data is marked `incomplete`. Staff can correct a value in the Inbox; a corrected field is no longer changed by the AI.

```json
{"chat_id":"60198765432@s.whatsapp.net","data":[{
  "schema_id":4,"schema":"Order","status":"complete","trigger":"inbound","updated_at":"2026-11-10T10:03:00Z",
  "field_definitions":[{"name":"quantity","type":"number","required":true},{"name":"email","type":"email","required":true}],
  "fields":{"quantity":3,"email":"ali@example.com"},
  "evidence":{"quantity":["3EB0C127D7BACC83D6A1"],"email":["3EB0C127D7BACC83D6A3"]},
  "corrected":[],"missing":[]}]}
```

- `status` is `pending` for a schema that has not run on the chat yet. Dates are `YYYY-MM-DD`; numbers are JSON numbers.
- Add `?history=true` for each schema's changes, newest first (`field`, `old`, `new`, `evidence`, `source`, `actor`, `created_at`).
- Every change also arrives as a `conversation.extracted` event. The account need not be connected to read stored data.

### Customer memory: `GET /v1/accounts/{account_id}/contacts/{contact}/memory`

With **Remember each customer** on in an account's AI settings (off by default), the AI keeps a short profile of each contact: a `summary`, how the last conversation ended (`last_outcome`), and `facts` of three kinds (`preference`, `fact`, `issue` for problems still open), each naming the messages (`evidence`) it came from. It is updated in the background about 10 minutes after the chat goes quiet, and when it is closed, at most 6 times a day per person. Staff can edit, pin or delete facts in the Inbox; the AI never changes what staff wrote, edited or pinned, and never adds a deleted fact again from the same messages. `fields` adds the data extracted from the contact's chats (see above). `contact` is a phone number, a Telegram user id or a chat id.

```json
{"contact":"60198765432","chat_id":"60198765432@s.whatsapp.net","data":{
  "enabled":true,"person_id":17,"summary":"Runs a bakery in Ipoh and orders flour monthly.","summary_by":"ai",
  "last_outcome":"Order 1042 confirmed for Friday delivery.","updated_at":"2026-11-10T10:20:00Z",
  "facts":[{"id":5,"person_id":17,"kind":"preference","text":"Prefers delivery after 3pm.","account_id":1,"contact":"60198765432",
    "evidence":["3EB0C127D7BACC83D6A1"],"pinned":false,"source":"ai","created_at":"...","updated_at":"..."}],
  "fields":[{"account_id":1,"schema":"Order","field":"email","type":"email","value":"ali@example.com"}]}}
```

- `person_id` is `0` while nothing is remembered or linked. The memory is shared by the person's linked contacts (below).
- The reply AI, suggested replies and wrap-up notes are given it (up to about 1,500 characters, marked as background, never as instructions), with the person's latest messages on their other linked channels (up to about 2,000 characters).
- Changes arrive as `contact.memory_updated`. AI calls that update it count as AI usage with source `memory`.
- **Privacy:** a contact's export holds the memory; erasing a contact erases the whole shared memory and unlinks the contact; an opt-out wipes it, and an opted-out contact is not remembered. It is kept as long as the message log.

### One person across channels: `GET /v1/accounts/{account_id}/contacts/{contact}/person` and `GET /v1/people/{person_id}`

Staff can link the same person's contacts across a customer's accounts (several WhatsApp numbers, SMS, Telegram) in the Inbox: the portal suggests contacts with the same phone number (as their number, through a WhatsApp LID whose number is known, or in extracted data) or the same extracted email, and staff confirm or reject each, or link by hand. Links never cross customers.

- `.../contacts/{contact}/person` returns `{"person": {"id":17,"contacts":[...]}, "suggestions":[...]}` (`person` is `null` when the contact is not linked); each contact has `account_id`, `account_name`, `channel`, `contact`, `chat_id`, `how` (`own`, `manual`, `suggestion`), `linked_by`, `created_at`; each suggestion has `account_id`, `channel`, `contact`, `chat_id`, `name` and `reasons`.
- `GET /v1/people/{person_id}` returns the person's contacts on your key's accounts and their shared `memory`; `404 person_not_found` when none is yours.
- Only contacts on accounts your key may use are listed.
- **Opt-outs stay per channel:** a contact who sends STOP on SMS is still reachable on WhatsApp unless they opt out there too. The shared memory is wiped by an opt-out on any linked channel.

### Follow-ups: `/v1/accounts/{account_id}/follow-ups`

A follow-up is a message the account sends a contact later. Staff schedule them in the Inbox, and an account with follow-ups on schedules one from each wrap-up note that names one ("confirm delivery Friday"). You can schedule them too:

```bash
curl -X POST "$BASE/v1/accounts/1/follow-ups" -H "X-API-Key: $KEY" -H "X-API-Secret: $SECRET" -H "Content-Type: application/json" \
  -d '{"to":"60198765432","reason":"Confirm the delivery arrived","when":"Friday afternoon"}'
# → 201 {"data":{"id":12,"chat_id":"60198765432@s.whatsapp.net","contact":"60198765432","source":"api","reason":"Confirm the delivery arrived",
#        "due_at":"2026-11-13T07:00:00Z","message":"Hai! Adakah barang anda sudah sampai?","language":"ms","mode":"draft","status":"pending",...}}
```

- `to` (or `chat`) and `reason` or `message` are required. Without `due_at` (RFC 3339) the AI reads `when` (or the reason) into a time in the portal's time zone; a time it cannot read becomes the next day at 10:00, with a `due_note`. Without `message` the AI drafts one in the contact's language. A `due_at` in the past or more than 180 days away is refused with `400 invalid_request`.
- `best_time_within_hours` (0 to 168, default 0 = off) moves the time to the hour the contact usually writes or reads messages (from the last 90 days of the message log; else the number's busiest hour) within that many hours after it. Only times outside the quiet hours and inside the contact-initiated window are candidates; with no history it stays. The answer's `due_at` is the predicted send time and `due_note` says why it moved.
- `mode` is the account's: `draft` (the default: a person approves it in the Inbox, `status` `pending` until then) or `auto` (`approved` at once). Statuses: `pending`, `approved`, `sending`, `sent`, `cancelled`, `blocked`, `failed`.
- `GET` lists waiting follow-ups, soonest first; `?status=all` lists every kind, newest first; `?chat=` narrows to one contact. `DELETE /v1/accounts/{account_id}/follow-ups/{follow_up_id}` cancels one (`409 follow_up_closed` when it was already sent or cancelled).
- A follow-up is cancelled when the contact writes first, and one from a wrap-up note when the chat is closed again. Each goes once, even with two portal processes.
- **Contact-initiated window.** These accounts use personal WhatsApp, so no platform rule limits when a business writes first; RelayKin applies its own to keep numbers safe. A follow-up goes only within 24 hours of the contact's last message (the account can change the hours or turn the rule off), or else to a contact with recorded marketing consent. Otherwise it ends `blocked`. It never goes during the account's quiet hours (21:00 to 09:00 in the portal's time zone by default); it waits until they end. The sending guard (limits, opt-outs, ban-risk policy, quotas, credit) applies as to any send: an opted-out contact's follow-up is `cancelled`.
- When one goes out, a `followup.sent` event follows. Its message is in the message log with source `followup`.

### Voice-note replies

When an account has voice replies on, an AI reply to a contact who sent a voice note (an automatic reply, or a draft a person approved) goes back as a voice note, read aloud in the reply's language. The text goes out in `message.voice_reply` and shows as the voice note's `transcript` in the chat history. A reply longer than the account's limit (600 characters by default), a failed speech call, or an SMS account sends the text instead, with `sent_as: "text"` and a `fallback_reason` (`too_long`, `voice_failed`, `sms`, `speech_disabled`). Telegram accounts get a Telegram voice message.

### AI actions (`ai.action`)

An admin or the customer's manager can give the reply AI actions on the business's own systems (account settings → AI actions): an HTTPS endpoint (method, URL template such as `https://shop.example.com/orders/{order_id}`, headers stored encrypted, a JSON Schema for the input, a timeout of at most 10 seconds and a response size cap), or an MCP server (Streamable HTTP, optional bearer token) whose tools are offered as `<name>__<tool>`. Each action is `auto` (the AI may call it), `approve` (each call waits in the Inbox until a person approves, edits or rejects it, and the reply then carries on with the result) or `off`. Actions apply to one account or to every account of a customer.

- What your endpoint receives: GET and DELETE send input fields not used in the URL as query parameters; POST, PUT and PATCH send the input as a JSON body. `{contact}`, `{chat_id}` and `{account_id}` in the URL are filled in by RelayKin, never by the AI. Requests carry `X-RelayKin-Account`, `X-RelayKin-Chat` and an `Idempotency-Key` (`relaykin-action-<call id>`). Redirects are not followed, and only public addresses are called.
- The AI sees each action's name, description and input schema, never the URL, headers or token. Inputs are checked against the schema before any call; at most 5 calls and 30 seconds of waiting per reply. Personal details the AI sees as placeholders are put back before your endpoint is called. The final reply still passes the account's AI rules.
- Only the Claude API and the OpenAI API can call actions; on the ChatGPT subscription the AI answers without them.
- **Limited to the chat's contact:** an action can name the input fields that carry the customer (a phone number, the chat id, or the customer record in RelayKin's customer memory). RelayKin fills them from the conversation, and a different value from the AI (another customer's phone) is replaced with the chat's, or the call is refused, as the action says; your endpoint only ever receives the chat's own contact in those fields. An action can also say where your answer names the customer (a JSON path such as `customer.phone`): an answer that names anyone else, or no one, is withheld from the AI (`status: "dropped"`) and the chat goes to a person.
- Every call (and every decision on one) is sent as `ai.action`:

```json
{"call_id":812,"action":"lookup_order","tool":"lookup_order","chat_id":"60198765432@s.whatsapp.net","contact":"60198765432","message_id":"3EB0C1...",
 "status":"ok","risk":"auto","input":{"order_id":"A100"},"http_status":200,"latency_ms":184,"error":"","source":"assist"}
```

`status` is `ok`, `error` (the endpoint failed, timed out or answered 4xx/5xx), `invalid` (the AI's input did not match the schema; nothing was sent), `refused` (a private address, the per-reply limit, or another customer's details for an action limited to the chat's contact), `dropped` (the answer was about another contact and was withheld), `pending` (waiting for approval), `rejected` or `expired` (nobody decided within 24 hours); decided calls add `decided_by`. `source` is `assist` for live chats, `flow` for a conversation flow's action step, and `test` for a test call from the settings page. AI tests (evals) never call your endpoints: each test case can carry `action_stubs`, fixed results per action.

### Conversation flows: `POST /v1/accounts/{account_id}/chats/{chat}/flows/{flow}/start`

An admin or the customer's manager builds flows in the portal (account settings → Conversation flows): step-by-step journeys such as booking, lead qualifying or order status. A flow starts on a keyword or pattern, a new contact's first message, an intent the AI recognises, a reply to a campaign, or this endpoint. Its steps send messages, ask questions (the answer is checked as text, a number, a date, a phone number, an email address or one of a list, and kept in a named field), branch on fields, call an AI action, let the AI answer from the knowledge base, add labels, hand the chat to a person, wait, and end. While a chat is in a flow, the flow answers instead of keyword auto-replies and AI replies; `STOP` always ends it, and a person replying from the Inbox pauses it.

```sh
curl -X POST "https://<PORTAL>/v1/accounts/1/chats/60198765432/flows/7/start" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET" \
  -H "Content-Type: application/json" -d '{"fields":{"order_id":"A100"}}'
# → 201 {"data":{"id":55,"flow_id":7,"version":3,"chat_id":"60198765432@s.whatsapp.net","status":"active","step_id":"ask_date","fields":{"order_id":"A100"},...}}
```

- The body is optional. `fields` (text, numbers or true/false; names `a-z`, `0-9`, `_`) are filled in before the first step, for `{{field}}` placeholders and branches. The flow runs at once and the response shows where it stopped (a question waiting for an answer, or `completed`).
- A chat already in a flow answers `409 flow_active`; send `"replace": true` to end that flow (`flow.exited` with reason `replaced`) and start this one. A flow that is turned off answers `409 flow_disabled`; another account's flow `404 flow_not_found`. Flows run in one-to-one chats only. Test keys cannot start flows.
- `GET /v1/accounts/{account_id}/chats/{chat}/flow` returns the flow the chat is in (`flow`, `version`, `status` `active` or `paused`, `step_id`, `step_type`, `fields`), or `null`.
- Choices go out as numbered text ("1. Morning"); the customer replies with the number or the option's text, and the AI reads free-text answers to choice and date questions ("late afternoon", "next Friday") unless the step turns that off. Dates are kept as `YYYY-MM-DD`, numbers as JSON numbers, an action's JSON answer as an object.
- Each edit of a flow is a new version; a chat finishes on the version it started with (`version` in every event).
- When a flow ends, its fields are also copied into the chat's structured data, for schema fields of the same name (`conversation.extracted` with `source: "flow"`).

Events (all with `flow_id`, `flow`, `version`, `run_id`, `chat_id`, `contact`, `trigger`):

```json
{"flow_id":7,"flow":"Appointment booking","version":3,"run_id":55,"chat_id":"60198765432@s.whatsapp.net","contact":"60198765432","trigger":"keyword",
 "step_id":"date","step_type":"question","outcome":"answered","field":"date","value":"2026-12-24","fields":{"service":"Consultation","date":"2026-12-24"}}
```

- `flow.started`: a chat entered a flow. `trigger` is `keyword`, `regex`, `new_contact`, `intent`, `campaign_reply` or `api`.
- `flow.step`: a step finished (above). `outcome` is `sent`, `answered`, `invalid` (too many invalid answers, going to the step's fallback), `branched`, `ok` or `failed` (an action), `done` or `turns_used` (an AI step), `set`, `waited`, `timeout`, `handoff` or `end`. Question, action and set steps include `field` and `value`.
- `flow.completed`: the flow reached its end, with every collected `fields`.
- `flow.exited`: the flow ended early, with `step_id`, `fields` and `reason`: `opted_out`, `handoff` (a hand-off step, too many invalid answers, a failed action without a fallback, or the AI handing over), `timeout`, `ended_by_staff`, `replaced`, `missing_step` or `loop`.

### Photos and documents (`message.media_understood`)

When an account has "Read photos and documents" on (off by default), the AI describes each incoming photo (JPEG, PNG, GIF, WebP up to 5 MB) and reads each PDF (up to 10 MB and 20 pages) or text file a contact sends in a one-to-one chat, on WhatsApp and Telegram. What it read is kept with the message like a voice-note transcript: it shows in the Inbox, in `media_text` in the chat history, and in what the reply AI, the Inbox copilot and structured data extraction read. ID, passport and licence numbers, Malaysian IC numbers and card numbers are never kept in full (only the last 4 characters). A caption sent with a photo is answered after the photo is read, together with it. Needs the Claude API or the OpenAI API as the account's AI provider.

```json
{"message_id":"3EB0C1...","chat_id":"60198765432@s.whatsapp.net","from":"60198765432@s.whatsapp.net","from_name":"Ali","timestamp":"...",
 "media_type":"image","media_url":"https://portal.example.com/v1/media/1/....jpeg","status":"ok",
 "kind":"receipt","description":"A receipt from Kedai Ali for RM 45.90 dated 10 Nov 2026.","text":"TOTAL RM45.90","id_document":false}
```

`kind` is one of `receipt`, `invoice`, `product_photo`, `form`, `id_document`, `screenshot`, `document`, `photo`, `other`. When the file was not read, `status` is `skipped` (an unsupported type, over a limit, or a provider that cannot read files) or `failed`, with a `reason`, and there is no description. These texts are part of a contact's data export and erasure, and are deleted with transcripts by the retention period.

### Holding messages over a limit

Add `"queue": true` to `POST /v1/messages`, or ask an admin to turn it on for the account, and a message over a pace limit is held instead of refused. You get `202` instead of `200`:

```json
{"data":{"queue_id":41,"status":"queued","reason":"send_rate_limited","account_id":3,"to":"60123456789","type":"text"}}
```

- The portal sends it when the limit allows, and you receive `message.queue_sent` or `message.queue_failed`.
- A held message gives up after 72 hours, and always respects opt-outs.
- While the account's server is down, held messages wait and then go out in the order they were held. One the server took but never confirmed (it restarted mid-send) is reported with `message.queue_failed` and code `send_uncertain` rather than sent again.
- On Telegram, `telegram_limited` is held too, and sent once the limit ends. A send that Telegram itself refuses for flooding goes back in the queue for 15 minutes without using up an attempt. `account_banned` is never queued.
- `bulk_message_blocked` and `opted_out` are never queued. Neither is `media_base64` or text-to-voice, so use `media_url`.
- `GET /v1/queue/{queue_id}` shows its status. `DELETE /v1/queue/{queue_id}` cancels one still waiting.

### Contact labels and notes

`GET /v1/accounts/{account_id}/contacts/{contact}` returns a contact's record. `PUT` the same path with `{"labels":["VIP"],"notes":"Prefers mornings"}` to replace it. These are the same labels and notes staff see in the Inbox.

### Marketing consent

`GET /v1/accounts/{account_id}/contacts/{contact}/consent` returns `granted` (the latest decision) and the full `history`.

`POST` the same path with `{"status":"granted","source":"checkout form","note":"ticked the box"}` to record a decision. `source` is required. The history is never edited, so a later `withdrawn` entry replaces an earlier grant. A contact who sends STOP is recorded as withdrawn automatically.

Campaigns can be set to message only contacts with consent; it is checked as each message goes out.

#### `POST /v1/consent` and `GET /v1/consent`

The same records, with the account and contact in the body or query:

```sh
curl -s -X POST "$RELAYKIN_BASE_URL/consent" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET" -H "Content-Type: application/json" \
  -d '{"account_id":3,"contact":"60123456789","status":"granted","method":"form","source":"checkout page","evidence":"order 1042, box ticked 2026-11-10 14:02","note":""}'
# → 201 {"data":{"id":88,"contact":"60123456789","status":"granted","method":"form","source":"checkout page","evidence":"order 1042, ...","recorded_by":"api:Shop","created_at":"..."}}
```

| Field | |
|---|---|
| `account_id`, `contact` | Required. `contact` is a phone number, as for sending |
| `status` | Required: `granted` or `withdrawn` |
| `method` | How it was given: `form`, `api`, `campaign`, `import`, `agent`, `keyword` or `other`. Defaults to `api` |
| `source` | Where, in your words (up to 100 characters). Defaults to the method |
| `evidence` | What proves it, such as a form id, order number or recording reference (up to 1,000 characters) |
| `note` | Anything else (up to 1,000 characters) |

`GET /v1/consent?account_id=3&contact=60123456789` returns that contact's `records` (newest first) and `granted`. Without `contact` it returns the account's latest 200 records; pass `before=<id>` for older ones.

The portal records these for you, each with its method and evidence:

| What happened | `status` | `method` |
|---|---|---|
| The contact sent STOP, BERHENTI or UNSUBSCRIBE | `withdrawn` | `keyword`, with the message as evidence (and the campaign, if one just messaged them) |
| The contact sent START, MULA or SUBSCRIBE | `opted_in` | `keyword` |
| Staff or the API added or removed an opt-out | `withdrawn` / `opted_in` | `agent` / `api` |
| A contact import with `consent=yes` | `granted` | `import`, with the file name as evidence |

`opted_in` means the contact came off the opt-out list. It is not marketing consent, so a campaign that needs consent still leaves them out until a `granted` record is added. Staff can download any account's or contact's records as CSV from the account's settings.

### Opt-outs

Contacts who send only `STOP`, `BERHENTI` or `UNSUBSCRIBE` are opted out automatically. They opt back in by sending `START`, `MULA` or `SUBSCRIBE`.

- `GET /v1/accounts/{account_id}/opt-outs` lists them.
- `PUT /v1/accounts/{account_id}/opt-outs/{contact}` adds one.
- `DELETE /v1/accounts/{account_id}/opt-outs/{contact}` removes one. Only do this if the contact asked.

### Ban-risk guard and content checks

Each WhatsApp number gets a **ban-risk score** (0–100) from how it sends and how people respond: volume against the number's age (warm-up), share of messages to people who never wrote in, reply rate, opt-outs (`STOP`), failed and undelivered messages, bursts, identical text, links and link shorteners, night sending, and restrictions WhatsApp reported (temporary bans, reach-out timelocks, logouts). Under 40 is low, 40–69 medium, 70+ high. The portal's admins and the customer's managers are alerted when it turns high.

Each account has a **policy**, set in the portal:

| Policy | What a raised score does |
|---|---|
| Warn only (default) | Nothing is held back. Sends carry warnings (below) |
| Slow down | Keeps to the warm-up schedule (100 messages a day for the first 3 days, 300 to day 7, 600 to day 14, 1200 to day 30) and drops to a safe rate: 15/minute at medium risk, 6/minute and half the new-contact allowance at high risk. Over-limit sends get the usual `429` codes (queueable with `"queue": true`) |
| Block risky sends | As slow down, and at high risk refuses messages to contacts who never wrote in (`403 ban_risk_blocked`) and high-risk bulk text (`422 content_risk_blocked`). Replies to people who wrote in still go |

**Warnings on `POST /v1/messages`.** When the number's risk is medium or high, the response carries `X-RelayKin-Ban-Risk: high; score=78; policy=warn` and `data.warnings.ban_risk`. When the same text has gone to 10 or more contacts in the last hour, the API treats the sends as a batch and checks the text: a medium or high result adds `X-RelayKin-Content-Risk: high; score=55` and `data.warnings.content` (`level`, `score`, `reasons`, `suggested_rewrite`, `ai_reviewed`, `ai_pending`). The fixed spam rules answer at once; when AI is connected, the text is also reviewed by the model in the background and later sends of the batch use that review. Warnings never block a send unless the account's policy is **block**.

**Check before a batch: `POST /v1/content-check`**

```bash
curl -s -X POST https://<PORTAL_DOMAIN>/v1/content-check -H "X-API-Key: $KEY" -H "X-API-Secret: $SECRET" \
  -H "Content-Type: application/json" -d '{"account_id":1,"text":"Hi {{name}}, our sale starts today! Reply STOP to opt out."}'
# → {"data":{"level":"low","score":5,"reasons":[...],"suggested_rewrite":"...","ai_reviewed":true,"account_risk":{"level":"low","score":12,"policy":"warn"},"would_block":false}}
```

It looks for link shorteners, spammy or misleading wording, shouting, many exclamation marks or emoji, a missing opt-out line and very long text, and (live keys, when AI is connected) asks the model for its view and a safer rewrite. `marketing` (default `true`) makes a missing opt-out weigh more. Test keys get the fixed rules only. Send keys may call it.

Campaigns in the portal run the same check before they start; a high-risk message or number must be confirmed by the person starting it, and is refused under the block policy.

### Sandbox (test keys)

Build and test your integration without sending real messages. Ask for a **test key** (admins and customers' managers create one under **Integrations → New API key → Test key**). Test credentials have the same format with `test_` after the prefix:

```sh
RELAYKIN_API_KEY=kwa_test_...
RELAYKIN_API_SECRET=kws_test_...
```

Use them against the same base URL. With a test key:

- **Nothing is sent.** No message reaches WhatsApp, SMS or Telegram, and the account's own server is never contacted, so the account doesn't need to be connected.
- **Nothing is charged or counted.** No credit, no monthly limits, quotas or sending limits are used, and test messages never appear in the message log, usage, billing, data exports or reports. The account's real opt-outs and limits are not applied either; use the magic numbers below to see those answers.
- **Input is checked exactly like live.** The same validation and the same error answers (`400 invalid_request`, `403 account_not_allowed`, `404 template_not_found`, SMS text-only, Telegram chat rules, and so on). Templates are filled as for live sends. Text-to-voice is checked but no audio is generated.
- **Answers look real.** `POST /v1/messages` returns `200` with a 20-character `message_id` (starting `SBX`), `"status": "sent"` and `"sandbox": true`.
- **The lifecycle is simulated.** About every 2 seconds the message moves on, and each step arrives by webhook and polling like a live receipt: `message.ack` with `receipt_type` `sent`, then `delivered`, then `read` (SMS accounts stop at `delivered`, as SMS has no read receipts). Test events carry `"sandbox": true` in the envelope and in `data`, and test webhooks add the header `X-RelayKin-Sandbox: 1`.
- **Only your test key sees them.** Test events never reach live keys, and a test key never receives the account's real events.

**Magic recipient numbers** (by the last four digits of `to`, on any channel):

| `to` ends in | Result |
|---|---|
| `0001` | Accepted, `message.ack` `sent`, then `message.failed` with a `reason` |
| `0002` | Delivered, never read |
| `0003` | Sent, never delivered |
| `0004` | Refused at once: `403 opted_out` |
| `0005` | Refused at once: `502 whatsapp_error` (retry with backoff) |
| `0006` | Refused at once: `429 send_rate_limited` with `Retry-After: 30` |
| anything else | `sent` → `delivered` → `read` |

For example `60190000001` fails, and `60123456789` goes all the way to read.

**Simulating an incoming message:** `POST /v1/sandbox/inbound` (test keys only, with `full` permission) emits a `message` event as if a contact wrote to the account:

```sh
curl -X POST "$RELAYKIN_BASE_URL/sandbox/inbound" \
  -H "X-API-Key: $RELAYKIN_API_KEY" -H "X-API-Secret: $RELAYKIN_API_SECRET" \
  -H "Content-Type: application/json" \
  -d '{"account_id":1,"from":"60123456789","text":"Hi, is the shop open?"}'
# → 201 {"data":{"event_id":2051,"message_id":"SBX...","chat_id":"60123456789@s.whatsapp.net","sandbox":true}}
```

**What a test key can call:** `GET /v1/accounts`, `GET /v1/accounts/{id}`, `POST /v1/messages`, `GET /v1/accounts/{id}/messages/{message_id}` (its own simulated messages only), mark read, reactions, typing (checked and answered, nothing sent), `POST /v1/content-check` (fixed rules only, no AI), `GET /v1/events`, `GET`/`PUT /v1/webhook`, `POST /v1/webhook/test`, `POST /v1/sandbox/inbound` and `POST /v1/mcp` (each tool is limited to these routes). The key's permission still applies on top: for example a `send` test key cannot call `GET /v1/events`, and only a `full` test key can call `POST /v1/sandbox/inbound`. Anything that would read real conversations or change real data (chat history, media, contacts, consent, opt-outs, drafts, the send queue, AI) returns `403 sandbox_unsupported`. A live key calling `/v1/sandbox/inbound` gets the same.

Test webhook deliveries appear in the key's delivery log marked **test**, and the webhook's health is scored like a live one's.

### AI agents (MCP): `POST /v1/mcp`

Customers' own AI agents (Claude Desktop, Claude Code and other Model Context Protocol clients) can use the gateway as an MCP server:

- **Address:** `https://<PORTAL_DOMAIN>/v1/mcp`, Streamable HTTP transport, JSON-RPC 2.0. Every request is a `POST`; the answer is one JSON response (no event stream, no session). `GET` answers `405`.
- **Authentication:** `Authorization: Bearer <key_id>:<secret>` (the same key and secret; the usual `X-API-Key`/`X-API-Secret` headers also work). A missing, wrong, expired or revoked key gets `401` with `WWW-Authenticate: Bearer`. The IP allowlist and the 20 requests/second limit apply to each MCP request. Requests from web pages on other sites (an `Origin` header that is not the portal's) are refused.
- **Protocol:** `initialize` (versions 2025-11-25, 2025-06-18 and 2025-03-26; an unknown version gets 2025-11-25), `notifications/initialized`, `ping`, `tools/list`, `tools/call`, and `server/discover` (which lists those versions). Tool results come as text content holding JSON, plus `structuredContent` when the client sends `MCP-Protocol-Version: 2025-06-18` or later.
- **Scope:** each tool calls the matching `/v1` route through that route's own handler, with the key's accounts, permission (full, read or send) and sandbox rules. `tools/list` shows only the tools the key may use. Sends are `POST /v1/messages`: quotas, credit, dunning pause, consent, opt-outs, the sending guard and the ban-risk policy all apply, and refusals come back as a tool result with `isError: true` and the same `code`, `message` and `retry_after_seconds` as the REST API. Test keys stay sandboxed (simulated sends, no reading of real chats).

| Tool | Route | Keys |
|---|---|---|
| `list_accounts` | `GET /v1/accounts` | all |
| `list_chats` (`account_id`, `search`, `offset`) | `GET /v1/accounts/{id}/chats` | full, read |
| `read_messages` (`account_id`, `chat`, `limit`, `offset`) | `GET /v1/accounts/{id}/chats/{chat}/messages` | full, read |
| `search_messages` (`account_id`, `query`, `chat`?) | the same, with `search`; without `chat` it searches the 10 most recent chats | full, read |
| `send_message` (`account_id`, `to`, `type`, `text`, `media_url`, `caption`, `filename`, `reply_to`, `queue`) | `POST /v1/messages` | full, send, test |
| `mark_read` (`account_id`, `chat`, `message_id`) | `POST /v1/accounts/{id}/messages/{message_id}/read` | full, test |
| `get_contact` (`account_id`, `contact`) | `GET /v1/accounts/{id}/contacts/{contact}` | full, read |
| `get_opt_out_status` (`account_id`, `contact`) | `GET /v1/accounts/{id}/opt-outs/{contact}` | full, read |
| `get_delivery_status` (`account_id`, `message_id`) | `GET /v1/accounts/{id}/messages/{message_id}` | all (send keys: their own messages; test keys: simulated ones) |
| `check_content` (`account_id`, `text`) | `POST /v1/content-check` | full, send, test |
| `ask_inbox` (`question`, `account_ids`?) | `POST /v1/ask` | full, read |
| `search_chats` (`query`, `account_ids`?) | `POST /v1/search` (30 searches an hour per key) | full, read |
| `list_knowledge_gaps` (`account_id`, `all`?) | `GET /v1/accounts/{id}/kb/gaps` | full, read |
| `approve_knowledge_gap` (`account_id`, `gap_id`, `question`?, `answer`?) | `POST /v1/accounts/{id}/kb/gaps/{gap}/approve` | full |
| `dismiss_knowledge_gap` (`account_id`, `gap_id`) | `POST /v1/accounts/{id}/kb/gaps/{gap}/dismiss` | full |

**Claude configuration** (Claude Desktop `claude_desktop_config.json` or Claude Code `.mcp.json`; the API keys page shows it with your key id):

```json
{
  "mcpServers": {
    "relaykin": {
      "type": "http",
      "url": "https://<PORTAL_DOMAIN>/v1/mcp",
      "headers": { "Authorization": "Bearer kwa_your_key_id:<your secret>" }
    }
  }
}
```

or `claude mcp add --transport http relaykin https://<PORTAL_DOMAIN>/v1/mcp --header "Authorization: Bearer kwa_your_key_id:<your secret>"`. Keep the secret out of shared or committed files; give an agent a read-only or test key until you trust it to send.

The routes the tools added are ordinary REST routes too: `GET /v1/accounts/{account_id}/chats`, `GET /v1/accounts/{account_id}/opt-outs/{contact}` and `GET /v1/accounts/{account_id}/messages/{message_id}`.

## 7. Errors

Every error has the same shape:

```json
{"error":{"code":"account_not_connected","message":"account WhatsApp session is awaiting_pair; it must be connected"}}
```

| HTTP | `code` | What to do |
|---|---|---|
| 400 | `invalid_body`, `invalid_json`, `invalid_request`, `invalid_account`, `invalid_chat`, `invalid_contact`, `invalid_person`, `invalid_flow`, `invalid_file`, `tts_disabled`, `invalid_webhook` | Fix the request; don't retry as-is |
| 400 | `no_webhook` | `POST /v1/webhook/test`: the key has no webhook URL. Set one with `PUT /v1/webhook` |
| 400 | `unsupported_protocol_version` | `POST /v1/mcp`: the `MCP-Protocol-Version` header names a version the endpoint does not support |
| 401 | `unauthorized` | Check the key and secret |
| 401 | `key_expired` | The key passed its expiry date. Ask the admin for a new key |
| 403 | `ip_not_allowed`, `account_not_allowed` | Ask the admin to adjust the key |
| 403 | `not_permitted` | The key's permission (`read` or `send`) does not cover this call; see section 2. Ask the admin for a key with the right permission |
| 403 | `origin_not_allowed` | `POST /v1/mcp` was called from a web page on another site. Call it from a server or an MCP client |
| 403 | `sandbox_unsupported` | A test key called something only live keys can (or a live key called `/v1/sandbox/inbound`); see "Sandbox (test keys)" |
| 403 | `opted_out` | The contact opted out. Don't retry, and remove them from your campaigns |
| 403 | `account_banned` | Telegram has banned this account. Don't retry; alert a human |
| 403 | `ban_risk_blocked` | The number's ban risk is high and its policy blocks messages to new contacts. Don't retry; replies to people who wrote in still work. Alert a human |
| 422 | `content_risk_blocked` | The text is high-risk for a bulk send and the account's policy blocks it. Rewrite it (see `POST /v1/content-check`); retrying the same text won't help |
| 404 | `message_not_found` | `GET /v1/accounts/{id}/messages/{message_id}`: no such message from this account (or, for a send key, not sent by this key) |
| 404 | `account_not_found`, `media_not_found`, `draft_not_found`, `queue_not_found`, `template_not_found` | |
| 409 | `account_not_ready`, `account_not_connected` | The account is being set up or is offline; retry later and alert a human if it persists |
| 409 | `draft_closed`, `queue_closed` | Already sent, dismissed or cancelled |
| 409 | `chat_assigned` | Staff assigned this chat to someone else; only an admin or API key can send its draft |
| 429 | `rate_limited` | Wait for `Retry-After` seconds |
| 429 | `send_rate_limited`, `daily_limit_reached`, `new_contact_limit_reached` | The account's sending guard is protecting the number. Wait for `Retry-After` seconds, and slow down |
| 429 | `telegram_limited` | Telegram is limiting this account. Wait for `Retry-After` seconds; sending through a limit only makes it longer |
| 429 | `customer_limit_reached` | Your organisation has sent its messages for this month. Sending works again at the start of next month (UTC; see `Retry-After`) or when the portal's owner raises the limit. Don't retry before then |
| 429 | `customer_ai_limit_reached` | Your organisation has used its AI replies for this month (`POST /v1/ai/respond` and AI features). Same as above |
| 429 | `bulk_message_blocked` | The same text already went to too many contacts this hour. Personalize it; retrying the same text won't help |
| 402 | `credit_exhausted` | Your organisation's prepaid credit has run out, so sends are refused. Sending works again once the portal's owner tops it up. Don't retry until then |
| 402 | `customer_credit_exhausted` | The same, for `POST /v1/ai/respond` and AI features |
| 402 | `payment_overdue` | Your organisation's sending is paused because an invoice is long overdue. The message says which invoice and how to pay. Receiving messages, events and webhooks keep working. Sending works again as soon as the invoice is paid (or the portal's owner resumes it); don't retry until then |
| 402 | `testing_ended` | Your organisation's testing period ended (its end date passed or its message cap was reached), so sends are refused. Receiving keeps working. Sending works again once the portal's owner moves you to production; don't retry until then |
| 429 | `account_quota` | This account reached its daily or monthly cap. Wait for `Retry-After` seconds, which run to when the cap resets |
| 429 | `ai_spend_limit` | The portal's monthly AI spend limit was reached. Wait for `Retry-After` seconds |
| 400 | `ai_model_unsupported`, `ai_unsupported` | The connected AI refuses that model or request; omit `model` |
| 503 | `suspended` | The portal's owner has suspended API access. Every call is refused until it is lifted; alert a human |
| 503 | `ai_not_connected`, `ai_reconnect_required`, `ai_no_models`, `ai_no_key` | The admin must connect or reconnect the ChatGPT account; alert a human, don't retry in a loop |
| 504 | `ai_timeout` | The model was too slow; retry once or shorten the input |
| 502 | `webhook_failed` | `POST /v1/webhook/test`: your endpoint did not answer with a 2xx in time. The message says what went wrong |
| 502 | `whatsapp_error`, `account_unreachable`, `tts_failed`, `ai_failed` | WhatsApp, Telegram or the account's server failed; retry with backoff (for example 2s, 5s, 15s), at most 3 times |
| 502 | `send_uncertain` | The account's server took the message but its answer was lost (it restarted, or the connection dropped), so the message may have gone out. Don't retry blindly: check the message log or your events first |

`whatsapp_error` is the code for any error the messaging network or the account's server returns, Telegram included; the name is kept so existing integrations keep working. It carries that error's HTTP status: `404` when a phone number has no reachable Telegram account, `429` when Telegram is rate-limiting (wait and slow down), `502` for failures.

**Retrying sends:** sending isn't idempotent. A retry after a timeout can produce a duplicate message, so only retry on `429` (except `bulk_message_blocked`), `502` (except `send_uncertain`) or connection errors before a response arrives. Never retry on `400`, `401`, `402`, `403` or `409`.

---

## 8. Capabilities and limits

| Feature | Supported |
|---|---|
| Send/receive text, image, video, audio, document, sticker, location | ✅ |
| Send voice notes (any audio format, auto-converted) | ✅ `type: "voice"` |
| Text-to-voice: send text that arrives as a voice note | ✅ `type: "voice"` + `text` (when enabled) |
| Receive voice notes and download them | ✅ `media_info.ptt: true` + `media_url` |
| Voice note transcription (voice-to-text) | ✅ `message.transcribed` event (when enabled) |
| Replies (quote), reactions, read receipts, typing indicator | ✅ |
| Group messages | ✅ `to` = `...@g.us` |
| Generate text with AI | ✅ `POST /v1/ai/respond` (when a ChatGPT account is connected) |
| Learn that an incoming call is ringing | ✅ `call.offer` event |
| Decline calls automatically and reply with a message | ✅ Portal setting (on by default) |
| **Place a WhatsApp voice/video call** | ❌ Not supported |
| **Answer a call, or play audio into a call** | ❌ Not supported |

**Why calls don't work:** WhatsApp calling uses a separate real-time voice protocol. The library this system uses (whatsmeow, the version this repository pins) only receives call notifications and can reject calls; it can't start, answer or stream audio into calls. The practical substitute is to **send a voice note**. For example, generate speech with TTS and send it with `type: "voice"`. By default the portal **declines incoming calls automatically** and sends the caller a short text (at most once every 10 minutes per caller) asking them to message instead; you still receive `call.offer` with `auto_rejected: true`, and can follow up with your own voice note.

**Responsible use:** these are regular WhatsApp accounts (unofficial API). Mass unsolicited messages, identical bulk texts and sudden volume spikes get numbers banned. Message people who expect to hear from you, vary the content, and pace your sends. For high-volume marketing, use the official WhatsApp Business Platform.

---

### Telegram accounts

An account with `"channel": "telegram"` in `GET /v1/accounts` uses the same endpoints, webhooks, signatures, sending limits, opt-outs and queue as a WhatsApp account. Pick the account by `account_id` as usual. The differences:

**Recipients and IDs**

- **Chats** are `123@tg.user` (a person), `456@tg.group` or `c789@tg.group` (groups and supergroups). Keep these as opaque strings and reply to `data.chat_id`. The digits in `123@tg.user` are a Telegram ID, not a phone number.
- **Sending to a phone number** works in `to`. The portal looks up the person's Telegram chat and sends there. It returns `404 whatsapp_error` if the number has no Telegram account or the person's privacy settings hide it. Looking a number up can add it to the account's Telegram contacts; reading a chat never does.
- A WhatsApp address (`...@s.whatsapp.net`, `...@g.us`, `...@lid`) sent to a Telegram account is refused with `400 invalid_request`.
- **Message IDs** in supergroups look like `c789_55`. Treat every message ID as opaque.

**What works the same**

| Feature | Telegram |
|---|---|
| Send text, image, video, audio, document, sticker, location | ✅ |
| Voice notes (`type: "voice"`, any audio format, converted to OGG/Opus) | ✅ |
| Replies, reactions, read receipts, typing indicator | ✅ |
| Receive messages and download media (`media_url`) | ✅ |
| Events `message`, `message.ack`, `message.edited`, `message.deleted`, `device.logged_out` | ✅ |
| Contact labels, notes, consent, opt-outs, handoff | ✅ |
| Holding messages over a limit (`"queue": true`) | ✅, Telegram limits included |
| Campaigns | ❌ always refused for Telegram accounts |
| View-once media | ❌ `400` |
| Calls | ❌ |
| AI endpoints, AI drafts, transcription, text-to-voice | Only when the portal's owner has allowed AI for Telegram; otherwise they return no results |

**Limits and bans**

Telegram limits accounts that message many new people or send too fast, and bans accounts it considers spam. The portal checks each Telegram account's status regularly and acts on it before sending:

- **Limited:** sends are refused with `429 telegram_limited` and a `Retry-After` header, set to when the limit ends (15 minutes if Telegram doesn't say). With `"queue": true` (or the account's hold setting on), the message is held instead and sent once the limit ends.
- **Banned:** sends are refused with `403 account_banned`. Don't retry; alert a human.
- **Telegram refusing a send directly:** you get `429 whatsapp_error` with Telegram's wait time in the message. Back off and don't retry at once.

New Telegram accounts get the same warm-up limit on new contacts as WhatsApp numbers. Message people who expect to hear from you.

### SMS accounts

An account with `"channel": "sms"` sends and receives SMS through an Android phone with that number's SIM card. It uses the same endpoints, webhooks, signatures, sending limits, opt-outs and queue. The differences:

- **Chats** are `60123456789@sms`. Reply to `data.chat_id` as usual. Messages from a named sender such as a bank arrive as `MAYBANK@sms`; they can't be replied to (`400 invalid_request`).
- **Sending:** put an international number in `to` (`60123456789`, `+60 12-345 6789` or `60123456789@sms`). WhatsApp and Telegram addresses are refused with `400 invalid_request`.
- **Text only:** `type` must be `text`, at most 1,600 characters (long texts go out as several SMS parts, each charged by the mobile network). Any other type returns `400 invalid_request` "SMS accounts can only send text". Reactions, read receipts and typing are ignored.
- **Delivery:** `message.ack` arrives with `receipt_type` `sent` when the phone sent the SMS, and `delivered` when the recipient's phone got it (if the network reports delivery). A failed SMS arrives as `message.failed` with a `reason`.
- **Incoming:** a `message` event with `chat_id` and `from` as `<number>@sms`, `from_name` as `+<number>`, and the text in `body`. No media.
- **Campaigns:** run from the portal only, and slowly: about one SMS every 45 seconds, a daily cap, and no sending at night, to protect the SIM card.
- **Not available:** media, groups and calls.
- **Phone offline:** if the phone is off or offline, the message waits on the server and goes out when the phone checks in again. The send answers at once, and a `message.ack` follows once the phone has sent it. If the phone was unpaired, sends return `409 account_not_connected`.
- **Errors from the phone's service** use `whatsapp_error` with the HTTP status, like Telegram's.

## 9. Checklist for the integrating agent

1. Load the four `RELAYKIN_*` values from the environment.
2. Call `GET /v1/accounts`; pick an account with `connected: true`. Make the account ID configurable.
3. Implement `sendMessage({to, type, ...})` with `POST /v1/messages`. Handle the errors in section 7, and retry only on `429` and `502`.
4. Implement the webhook endpoint: raw body, signature check, 2xx within 15s, dedupe on the envelope `id`, async processing.
5. Ignore `is_from_me: true`. Reply to `data.chat_id`.
6. Download incoming media through `data.media_url` with the API headers, and store your own copy (server copies are deleted after 30 days).
   For voice notes, use the `message.transcribed` event's `text` when transcription is enabled.
7. Optionally poll `GET /v1/events` from a persisted cursor to backfill after downtime. This needs a `full` or `read` key: a `send` key cannot poll, so rely on the webhook or use a second, read-only key for polling.
8. Treat `device.logged_out`, `device.temporary_ban`, `account_banned` and a non-connected `account` status as alerts for a human.
9. For Telegram accounts, handle `429 telegram_limited` by waiting for `Retry-After` (or send with `"queue": true`), and treat chat and message IDs as opaque strings.
10. Test with the portal's **Test webhook** button, then with a real message to the account.
