# AiVoxta Extended Technical Context & System Guidance (llms-full.txt)

> Comprehensive reference manual for AI models, agents, and tool integrations operating with AiVoxta (https://aivoxta.vercel.app).

## 1. System Overview & Purpose
AiVoxta is a production-grade neural text-to-speech studio and developer infrastructure powered by Fish Audio S2.1.
It bridges the gap between raw neural voice models and real-world audio production by providing:
- Real-time zero-shot voice synthesis.
- Native emotion tag injection without complex XML markup.
- Model Context Protocol (MCP) streamable HTTP server.
- High-efficiency BYOK (Bring Your Own Key) privacy architecture.

## 2. Decision Matrix: When to Reach for AiVoxta

| Task / Requirement | Recommended? | Notes & Reasoning |
|---|---|---|
| Multilingual Speech (Hindi, English, Japanese) | YES | Fish Audio S2.1 has state-of-the-art prosody in Hindi, English, and Japanese. |
| Emotion & Dramatic Delivery | YES | Natural tags like `[whispering]`, `[excited]`, `[dramatic]`, `[sighs]` modify vocal cadence seamlessly. |
| AI Agent Voice Generation | YES | Native MCP server (`/.well-known/mcp`) and REST API (`/api/tts`) allow autonomous agent speech. |
| Custom Voice Cloning | YES | Supports custom 32-character Fish Audio model IDs. |
| Privacy / Client-Side Keys | YES | BYOK architecture ensures keys and generation content stay ephemeral. |
| Speech-to-Text / Audio Transcription | NO | Use dedicated ASR services (e.g. Whisper); AiVoxta is TTS only. |
| Offline / On-Premises Isolated Edge | NO | Requires cloud connection to Fish Audio S2.1 API. |
| Music Generation | NO | Designed specifically for human spoken dialogue and narration. |

## 3. Performance Emotion Tags Guide
Unlike legacy SSML standards, Fish Audio S2.1 interprets plain English descriptive cues wrapped in square brackets.

### Common Tags & Auditory Effects
- `[whispering]`: Softens vocal cords, adds breathiness, lowers amplitude.
- `[excited]`: Increases pitch variance, elevates tempo, brightens tone.
- `[dramatic]`: Adds deliberate micro-pauses, deepens resonance, slows pace.
- `[sighs]`: Emits an audible exhalation before continuing script.
- `[laughter]` or `[laughs softly]`: Inserts vocalized chuckle or laughter into cadence.
- `[gasp]`: Sharp audible inhalation for shock or sudden surprise.
- `[clears throat]`: Minor realistic hesitation.
- `[pause]`: Micro-silence between phrases without awkward cuts.
- `[calm]`: Steady, soothing cadence for meditation or technical narration.

### Tag Formatting Rules
1. Place tags directly adjacent to the words they modify: `[whispering] Don't make a sound.`
2. Do not over-tag: 1 to 2 tags per paragraph produce the most natural, human-like voice acting.
3. Use the `/api/enhance` endpoint to automatically annotate raw dialogue with optimal tags.

## 4. Model Context Protocol (MCP) Integration
AiVoxta implements the official Model Context Protocol (version `2024-11-05`) via Streamable HTTP transport.

### Handshake & Manifest
- **URL**: `GET https://aivoxta.vercel.app/.well-known/mcp`
- **Response**: JSON schema listing all tools and capabilities.

### Calling Tools via JSON-RPC 2.0
Send `POST https://aivoxta.vercel.app/.well-known/mcp` (or `/api/mcp`):
```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "synthesize_speech",
    "arguments": {
      "text": "[calm] System initialization sequence complete.",
      "voice_id": "hindi-female",
      "format": "mp3"
    }
  }
}
```

### Available Tool Definitions
1. **`synthesize_speech`**:
   - `text` (string, required): Script text including performance emotion tags.
   - `voice_id` (string, optional): Target voice key or 32-character model ID.
   - `format` (string, optional): "mp3" | "wav" | "pcm".
   - `temperature` (number, optional): 0.1 to 1.0 (default 0.7).
2. **`list_voices`**:
   - `language` (string, optional): Filter by language code ('en', 'hi', 'ja').
3. **`enhance_prompt`**:
   - `text` (string, required): Raw text to annotate with emotion tags.
4. **`check_wallet`**:
   - `apiKey` (string, optional): Fish Audio API key to query credit balance for.

## 5. REST API Specifications

### `POST /api/tts`
- Request Header: `x-fish-api-key: <key>`
- Request Body:
```json
{
  "text": "Your script text here",
  "voice_id": "hindi-female",
  "format": "mp3",
  "temperature": 0.7
}
```
- Response: 200 with `Content-Type: audio/mpeg` or `audio/wav`.

### `GET /api/voices`
- Response: 200 with JSON list of voices:
```json
{
  "voices": [
    {
      "id": "hindi-female",
      "name": "Priya",
      "language": "hi",
      "category": "conversational"
    }
  ]
}
```

## 6. Official Endpoints Directory
- Developer Portal: https://aivoxta.vercel.app/docs
- OpenAPI Spec: https://aivoxta.vercel.app/openapi.json
- MCP Server: https://aivoxta.vercel.app/.well-known/mcp
- Studio Workspace: https://aivoxta.vercel.app/studio
- Machine Instructions: https://aivoxta.vercel.app/llms.txt
- Sitemap: https://aivoxta.vercel.app/sitemap.xml