IntakeBot — Conversational Form Intake: Requirements and Design

Conversational form intake engine. No dynamic forms are built or shown to the user. The bot collects information purely through conversation (chat or voice). Forms are defined in JSON and stored in a form repository; the Agent runs in intake mode when started with --FORMS. Data is written to an intake folder, timestamped and prefixed by form name, with partial data persisted as the LLM collects it.

1. Overview

1.1 Purpose

IntakeBot is a conversational form intake engine. The bot:

Talks the user through a form: introduces the form, asks for each field in order, and collects answers via conversation only (chat or voice).
Does not render or display any form UI — no on-screen fields, cards, or dynamic forms. The user never sees a “form”; they only converse with the bot.
When multiple forms exist, helps the user choose which form to fill using intent instructions and explicit options.
Persists data into an intake folder: one timestamped file per intake session, updated once per user turn (as fields are collected) so partial data is always on disk.

1.2 Design Principles

Conversation-only — No forms are built or shown. All collection happens via dialogue.
Form-depo + intake mode — Form definitions live in Agent/form-depo/. The Agent is started with --FORMS to run in intake mode (form intake only, no domain RAG/Q&A).
Form-agnostic — Any form that fits the JSON schema works. The schema includes intent/selection instructions so the LLM can decide which form the user wants when there are many.
Partial data always — Partial data is written once per user turn (after all record_field and any submit_form for that turn). We always have up-to-date partial data on disk, not only on completion.

1.3 Out of Scope (First Version)

Dynamic form UI, cards, or progress bars. Purely conversational.
Anonymous intake without a defined identity (e.g. session id) for the intake file.
Conditional fields, pattern validation, min/max. Deferred.

2. Form Repository (form-depo) and Intake Mode

2.1 Form-depo Folder

Location: When FORM_DEPO_PATH is unset or empty, use <project_root>/Agent/form-depo (domain-chatbot project root). Set the FORM_DEPO_PATH env var to override.
Content: One JSON file per form definition. File names can match form id (e.g. contact_form.json) or be arbitrary; the form’s id inside the JSON is the canonical identifier.
Loading: The API (service_formintake) loads forms from the resolved path: it scans *.json, validates each as a form schema. These are the only forms available for intake.

2.2 Intake Mode and `--FORMS`

Trigger: The Agent’s main script is run with --FORMS (e.g. python run_agent.py --FORMS or python -m Agent.main --FORMS). The main() in Agent/main.py reads this flag.
Behaviour when --FORMS is set:
- The Agent operates in intake mode.
- Forms are loaded by the API from the form-depo path: when FORM_DEPO_PATH is unset, <project_root>/Agent/form-depo; set FORM_DEPO_PATH to override.
- No domain RAG, no file_search, no domain Q&A. The bot’s only job is form intake: form selection (if multiple) and then field collection for the chosen form.
When --FORMS is not set: The Agent runs in the existing, normal mode (domain RAG, Pipecat, etc.). Intake and form-depo are not used.

2.3 Single Form vs Multiple Forms

Single form in form-depo: Bot can go straight to the form’s introduction and first field (optionally still mention which form it is).
Multiple forms in form-depo: Bot must first disambiguate:
- Use each form’s intent/selection instructions (and optionally keywords) so the LLM can interpret what the user wants.
- If the user’s intent is unclear or several forms could match, the bot presents options (e.g. using title or short_label per form) and asks the user to choose. Once chosen, load that form and start intake.

3. Form Schema (JSON)

3.1 Goals

Form identity, title, introduction, and fields.
Intent/selection instructions for the LLM to decide when this form is the one the user wants (and for disambiguation when there are multiple forms).
Multi-language for all user-facing strings.
Field types, required/optional, options for select.

3.2 Proposed Structure

{
  "id": "contact_form",
  "title": { "en": "Contact Form", "fa": "فرم تماس" },
  "short_label": { "en": "Contact", "fa": "تماس" },
  "intent_instructions": {
    "en": "Choose this form when the user wants to: send a message, get in touch, leave contact info, submit feedback, or ask a question.",
    "fa": "این فرم وقتی انتخاب شود که کاربر بخواهد: پیام بفرستد، تماس بگیرد، اطلاعات تماس بگذارد، بازخورد بدهد یا سؤال بپرسد."
  },
  "introduction": {
    "en": "I'll help you with that. I'll ask a few questions.",
    "fa": "کمک می‌کنم. چند سؤال می‌پرسم."
  },
  "fields": [
    {
      "id": "full_name",
      "type": "text",
      "required": true,
      "label": { "en": "Full name", "fa": "نام کامل" },
      "placeholder": { "en": "e.g. John Doe", "fa": "مثال: علی محمدی" }
    },
    {
      "id": "email",
      "type": "email",
      "required": true,
      "label": { "en": "Email", "fa": "ایمیل" }
    },
    {
      "id": "message",
      "type": "textarea",
      "required": false,
      "label": { "en": "Message", "fa": "پیام" }
    }
  ],
  "submit_label": { "en": "Submit", "fa": "ارسال" },
  "metadata": {
    "version": "1.0",
    "supported_languages": ["en", "fa"]
  }
}

3.3 Intent and Form Selection

intent_instructions (object, one key per language): Instructions for the LLM to decide if the user’s message matches this form. Used when:
- There is a single form (to confirm it’s the right one), or
- There are multiple forms (to shortlist and, if needed, to present options).
short_label (object, per language): Short name for the “Which form do you want?” list. If missing, fallback to title or id.
When multiple forms match or intent is ambiguous: the bot lists forms using short_label (or title) and asks the user to pick. The LLM then continues with the chosen form’s schema.

3.4 Field Types

Type	Description	Validation (v1)	Example in stored data
`text`	Short text	Non-empty if required	`"John"`
`email`	Email	Basic `@.*` if required	`"a@b.com"`
`phone`	Phone	Optional	`"+1234567890"`
`number`	Numeric	Optional	`42`
`date`	Date	Optional (e.g. ISO)	`"2025-01-25"`
`select`	Single choice	Value in `options`	`"opt_a"`
`textarea`	Longer text	Non-empty if required	`"Hello..."`
`boolean`	Yes/no	Normalize to bool	`true`

select example:

{
  "id": "country",
  "type": "select",
  "required": true,
  "label": { "en": "Country", "fa": "کشور" },
  "options": [
    { "value": "ca", "label": { "en": "Canada", "fa": "کانادا" } },
    { "value": "us", "label": { "en": "United States", "fa": "ایالات متحده" } }
  ]
}

3.5 Field Order and Required vs Optional

Order: fields array order = asking order. Bot asks the next required empty field first; optional fields follow or can be skipped.
Required: If required: true, the form is not considered complete until that field is in form_state with a non-null value.
Optional: If required: false, user may skip; we store form_state[field_id] = null (explicit skip). Downstream can distinguish "skipped" vs "never asked".

3.6 Localization

For title, short_label, intent_instructions, introduction, label, placeholder, submit_label, options[].label: use the user’s language with fallback to "en".

4. Intake Storage (intake folder, partial data)

4.1 Intake Folder

Location: When INTAKE_PATH is unset or empty, use <project_root>/Agent/intake. Set the INTAKE_PATH env var to override.
Role: All intake data is written here. No use of the /submissions API in intake mode; the intake folder is the persistence layer.

4.2 File Naming and Lifecycle

Filename: {form_name}_{session_start_timestamp}.json
- form_name = form’s id (e.g. contact_form).
- session_start_timestamp = set as soon as a form is selected (before the first record_field). Format: YYYYMMDD_HHmmss (e.g. 20250125_143022). Stable for the whole session.
One file per intake session. The file is created on form selection with an initial write (form_state: {}). The same file is then overwritten once per user turn (after all record_field and any submit_form for that turn) so that partial data is always on disk.

4.3 File Contents (Partial and Final)

Each write includes at least:

form_id: form’s id
started_at: ISO or same timestamp as in the filename
updated_at: time of this write
form_state: { "field_id": value } — partial while collecting, complete when submitted. Values can be null for skipped optional fields.
completed_at: set only when the form is submitted; omit otherwise.

Example (initial, on form selection):

{
  "form_id": "contact_form",
  "started_at": "2025-01-25T14:30:22",
  "updated_at": "2025-01-25T14:30:22",
  "form_state": {}
}

Example (partial, after two fields):

{
  "form_id": "contact_form",
  "started_at": "2025-01-25T14:30:22",
  "updated_at": "2025-01-25T14:30:45",
  "form_state": {
    "full_name": "John Doe",
    "email": "j@example.com"
  }
}

Example (final, after submit_form):

{
  "form_id": "contact_form",
  "started_at": "2025-01-25T14:30:22",
  "updated_at": "2025-01-25T14:31:10",
  "completed_at": "2025-01-25T14:31:10",
  "form_state": {
    "full_name": "John Doe",
    "email": "j@example.com",
    "message": "Hello, I have a question."
  }
}

4.4 When We Write

On form selection: As soon as the user has chosen a form (or the only form is chosen), create the intake file, set session_start_timestamp (format YYYYMMDD_HHmmss), and write the initial object: form_id, started_at, updated_at, form_state: {}, no completed_at. The response for that turn must include session_start_timestamp so the client can send it on /query/intake/continue.
On record_field: The handler only validates and updates form_state in memory; it does not write. The orchestrator in service_formintake does one write at end of the user turn (after all record_field and any submit_form for that turn). The file already exists from the form-selection write. If submit_form runs in that turn, its handler performs the final write (with completed_at) and no separate end-of-turn write is needed.
On submit_form: Same file; add completed_at and do a final write with the complete form_state.

5. User Stories and Requirements

5.1 Happy Path (Single Form)

#	Actor	Action	Expected outcome
U1	User	Starts a conversation (chat or voice) and wants to fill the only form in form-depo	Bot introduces the form with `introduction[language]` and asks for the first required field.
U2	User	Answers a field (e.g. "John Doe" for full_name)	Bot records it, writes partial data to `{form_id}_{timestamp}.json`, confirms briefly, asks for the next field.
U3	User	Provides all required (and any optional) fields	Bot calls `submit_form`, writes the final state with `completed_at` to the same file, and confirms.

5.2 Form Selection (Multiple Forms)

#	Actor	Action	Expected outcome
U4	User	Says something ambiguous (e.g. "I need to send something") and form-depo has several forms	Bot uses `intent_instructions` to shortlist; if still ambiguous, presents options using `short_label` (or `title`) and asks the user to choose.
U5	User	Picks a form (e.g. "Contact" or "Option 1")	Bot loads that form’s schema, creates the intake file `{form_id}_{session_start_timestamp}.json` with `form_state: {}`, includes `session_start_timestamp` in the response, says the form’s introduction, and asks for the first required field.
U6	User	Says something that clearly matches one form’s `intent_instructions`	Bot chooses that form, creates the intake file, includes `session_start_timestamp` in the response, gives the introduction, and asks for the first field.

5.3 Multi-Turn and Continue

#	Scenario	Requirement
U7	User pauses mid-form	Client stores `form_id`, `form_state`, and `session_start_timestamp`; on resume, sends all three so the bot continues from the next empty required field. The intake file for that session remains; we keep overwriting with partial writes when more fields are recorded.
U8	User corrects a previous answer	Bot overwrites via `record_field` with the same `field_id` and new value; on the next write, the intake file reflects the correction.
U9	User asks "what did I say for email?"	Bot reads from `form_state` and answers.

5.4 Multi-Lingual and Voice

#	Scenario	Requirement
U10	User speaks or types in Farsi	Bot responds in Farsi; uses `label["fa"]`, `introduction["fa"]`, `short_label["fa"]`, `intent_instructions["fa"]`.
U11	User uses voice (Pipecat/Agent)	In intake mode, the same Agent/Pipecat stack is used; STT/TTS and language selection stay as today. No form UI—conversation only.

5.5 Validation, Skip, and Errors

#	Scenario	Requirement
U12	Invalid value (e.g. email)	`record_field` validates; on failure, returns an error and re-asks; no write to the intake file for that value.
U13	User says "skip" for optional field	Bot calls `record_field(field_id, null)`; handler sets `form_state[field_id] = null`. Next write includes that key with `null`.
U14	User says "skip" for required field	Bot explains it is required and re-asks.
U15	`submit_form` but required fields missing	Tool returns `{ ok: false, missing: [...] }`; LLM asks for those; no `completed_at` write.

5.6 Abandonment and Edge Cases

#	Scenario	Requirement
U16	User says "cancel" or "never mind"	Bot acknowledges (natural language; no tool). Client clears `form_id`, `form_state`, and `session_start_timestamp`. The intake file stays as last partial write (no `completed_at`).
U17	User sends chitchat mid-form	Bot answers briefly and returns to the pending field.
U18	User gives multiple fields in one message	LLM may call `record_field` multiple times; each valid call updates `form_state`; one write at end of the turn (orchestrator).

6. Expected Bot Behaviour

This section describes what end users experience during intake. Internal prompt and orchestration details are not published in public documentation.

6.1 Role

The bot acts as a conversational form intake assistant. It collects answers through dialogue only—no on-screen form UI. In intake mode it uses your form definitions from the form repository, not general domain Q&A or knowledge-base search.

6.2 Form selection (multiple forms)

Uses each form's intent_instructions and labels to match the user's request
If one form clearly matches: introduces that form and begins field collection
If several could match: presents options (by short_label or title) and asks the user to choose
If none match: asks what the user needs and maps to a form when possible

6.3 Field collection

Delivers the form introduction in the user's language, then asks required fields in order
Accepts corrections, optional-field skips, and brief chitchat before returning to the pending question
Validates values (for example email format) and re-asks when input is invalid
Marks the intake complete only when all required fields are satisfied

6.4 Cancel and partial saves

If the user cancels, the bot acknowledges and the session ends; partial answers already collected remain in the intake record
Data is persisted incrementally during the conversation so partial intakes are not lost on disconnect

7. Intake Tools (Integrator Summary)

Intake mode exposes two primary tools to the conversation layer:

Tool	Purpose
`record_field`	Validate and store one field value (or skip an optional field)
`submit_form`	Mark the intake complete when all required fields are present

Handlers validate input against the form schema. The intake service persists data once per user turn after tool calls complete. For API contracts and self-hosted deployment, contact your Aventora administrator.

8. Intake Mode: Agent and Startup

8.1 Entrypoint

Start the Domain Assistant agent with the --FORMS flag to run in intake mode. In this mode:

Forms load from the configured form repository path (FORM_DEPO_PATH when set)
The agent handles form selection and field collection only (no domain RAG/Q&A)
Completed and partial intakes write to the configured intake folder (INTAKE_PATH when set)

8.2 When `--FORMS` Is Not Set

Agent runs in standard domain assistant mode (chatbot Q&A, voice, and related features). Intake mode is not active.

9. Request and Response (Intake Mode)

In intake mode the Agent typically handles conversations via WebSocket or Pipecat. The exact message shape can match the existing chat/voice protocol; the following is the logical contract for intake.

9.1 Form Selection Phase

Request: User message (text or STT).
Response: Either (a) a list of form options asking the user to choose, or (b) direct introduction to the chosen form and the first question. No form_schema needs to be sent by the client; forms come from form-depo.

9.2 Field Collection Phase

Request: User message. The server must know form_id, form_state, and session_start_timestamp (from the form-selection response or from the client on continue).
Response: answer, form_id, form_state; when the intake file was created on form selection, also session_start_timestamp (so the client can send it on every continue). When the form is submitted: form_submitted: true, form_data: form_state. The intake file is created on form selection and updated on record_field and on submit_form as described in §4.

9.3 Session and Continuation

The Agent (or client) must track per session: form_id, form_state, session_start_timestamp (required so we can read/write the same intake file). The client must send session_start_timestamp on every /query/intake/continue (it is returned in the response when the file is first created on form selection).

10. Data Layout (Summary)

Location	Contents
Form repository	JSON form definitions (one file per form)
Intake folder	Timestamped intake session files (`{form_id}_{timestamp}.json`) with partial and completed states

Configure paths with FORM_DEPO_PATH and INTAKE_PATH in your deployment environment.

11. Implementation Notes

Intake mode is implemented as a dedicated service separate from standard domain Q&A. Self-hosted operators should use the Domain Assistant deployment guide and contact Aventora support for upgrade paths. Internal file-level design documentation is available to implementation partners under NDA.

1. Overview​

1.1 Purpose​

1.2 Design Principles​

1.3 Out of Scope (First Version)​

2. Form Repository (form-depo) and Intake Mode​

2.1 Form-depo Folder​

2.2 Intake Mode and --FORMS​

2.3 Single Form vs Multiple Forms​

3. Form Schema (JSON)​

3.1 Goals​

3.2 Proposed Structure​

3.3 Intent and Form Selection​

3.4 Field Types​

3.5 Field Order and Required vs Optional​

3.6 Localization​

4. Intake Storage (intake folder, partial data)​

4.1 Intake Folder​

4.2 File Naming and Lifecycle​

4.3 File Contents (Partial and Final)​

4.4 When We Write​

5. User Stories and Requirements​

5.1 Happy Path (Single Form)​

5.2 Form Selection (Multiple Forms)​

5.3 Multi-Turn and Continue​

5.4 Multi-Lingual and Voice​

5.5 Validation, Skip, and Errors​

5.6 Abandonment and Edge Cases​

6. Expected Bot Behaviour​

6.1 Role​

6.2 Form selection (multiple forms)​

6.3 Field collection​

6.4 Cancel and partial saves​

7. Intake Tools (Integrator Summary)​

8. Intake Mode: Agent and Startup​

8.1 Entrypoint​

8.2 When --FORMS Is Not Set​

9. Request and Response (Intake Mode)​

9.1 Form Selection Phase​

9.2 Field Collection Phase​

9.3 Session and Continuation​

10. Data Layout (Summary)​

11. Implementation Notes​