Trixy7/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Trixy is a professional voice assistant system written in Python 3.10+ with PyTorch. It operates in four modes:

• Server: Central hub managing satellites, plugins, events, and ML inference
• Client (Satellite): Lightweight device with wakeword/voice recognition, connects to server
• Standalone: All-in-one mode with full functionality without server
• Train: ML trainer for wakeword and voice recognition models

Running the Application

# Server mode
python3 main.py server [--debug] [--config config/server_config.json]

# Client/Satellite mode
python3 main.py client --host <server_ip> --port 2101 --room <room> --alias <name> [--debug]

# Standalone mode
python3 main.py standalone [--debug] [--config config/standalone_config.json]

# ML Training
python3 main.py train [wakeword|voice-recognition|all]

The --debug flag enables stdout output via print() instead of TUI logging.

Architecture

Service Container Pattern

All major components inherit from IService with:

• PRIORITY: ServicePriority enum (CRITICAL, CORE, NETWORK, MANAGER, PLUGIN, OPTIONAL)
• GROUP: ServiceGroup for categorization
• DEPENDENCIES: List of required service names
• Lifecycle hooks: on_pre_start(), on_post_start(), on_pre_stop(), on_post_stop(), on_dependency_ready()

Event-Driven System

Central pub/sub with priority levels (SYSTEM, HIGHEST, HIGH, NORMAL, LOW, LOWEST, MONITOR):

@TrixyEvent(["wakeword_received"])
def on_wakeword(self, event_name, event_data):
    pass

Events are cancellable and support async handlers.

Network Protocol (Trixy Protocol)

Custom binary protocol on 4 ports:

• Port 2101: Command socket (AES-256-GCM encrypted)
• Port 2102: Audio input stream (16KHz mono, satellite → server)
• Port 2103: Audio output stream (16KHz mono, server → satellite)
• Port 2104: Music stream (48KHz stereo)

Protocol structure: Magic TRXI + version + datetime + 32-bit flags + MD5 checksum + class name + serialized data

Hard-coded commands for efficiency: TRXINOOP, TRXIPING, TRXIPONG, TRXIPRNT, TRXYHELO

Satellite Management

• Satellites must be registered before connecting (60-second registration window)
• Per-satellite instances track: room ID, MAC address, alias, sockets, conversation reference
• Auto-reconnect with 5-second retry interval
• Registration files stored as JSON by MAC address in ./satellites/

Directory Structure

./
├── main.py                    # Entry point
├── config/                    # Configuration files (server/client/standalone/trainer)
├── trixy_core/                # Core framework
│   ├── service/               # Service container and IService base
│   ├── events/                # EventManager and event data classes
│   ├── network/               # Protocol, encryption, sockets, cmd/ messages
│   ├── satellite/             # SatelliteManager, registration, heartbeat
│   ├── config/                # ConfigManager with typed dataclasses
│   ├── wakeword/              # Wakeword detection service (client-side)
│   ├── plugins/               # Plugin system
│   ├── scheduler/             # Schedule triggers and actions
│   ├── conversation/          # Conversation state management
│   ├── tui/                   # Textual-based UI framework
│   └── utils/                 # pinfo(), pdebug(), perror(), pwarn()
├── plugins/                   # User plugins (each in own directory with main.py + config.json)
├── models/                    # ML models
│   ├── wakeword/{model}/      # model.pth, model.onnx, model.pt, metafile.json
│   └── voice_recognition/{model}/
├── trainer/
│   ├── data/{wakeword|voice_recognition}/raw/  # Raw training data
│   └── trainers/              # Trainer implementations
├── assets/{profile}/          # Audio files, profiles (with "default" fallback)
├── satellites/                # Registered satellite JSON files
├── certs/                     # SSL certificates
└── logs/                      # Application logs

Code Conventions

• Language: All comments and documentation in German
• Debug output: Use pinfo(), pdebug(), perror(), pwarn() from trixy_core.utils.debug
• Type hints: Use Python 3.10+ features throughout
• Config: JSON format with typed dataclasses, supports file watching and auto-reload

Plugin Development

Plugins extend TrixyPlugin and must have:

• main.py with plugin class
• config.json for configuration

Plugin class has access to: self.application, self.config, self.enabled, reload_config(), save_config()

ML Models

• Wakeword: 2 models ("custom", "system_command") - OpenWakeWord-based architecture
• Voice Recognition: Dynamic speaker count, speaker ID is array index in metadata
• Model files: Password-protected .pth (zip), also exported as .pt and .onnx
• Metadata in metafile.json includes training info, speaker names, architecture version

Audio specs: 16KHz, 16-bit PCM mono, Log-Mel-Spectrogram features

Wakeword Detection Workflow

1. Client detects wakeword → pause detection → buffer audio → notify server
2. Server waits 1 second for multi-satellite arbitration (highest volume wins)
3. Server creates conversation session → signals selected satellite
4. Satellite streams audio until 3 seconds silence or 60 seconds max
5. Server triggers raw_audio_input_received event → STT → NLP → intent handling

Project Memory System

Memory Files

Das Projekt nutzt ein strukturiertes Memory-System in docs/project_notes/:

Datei	Zweck
bugs.md	Bug-Log mit Ursachen und Lösungen
decisions.md	Architekturentscheidungen (ADRs) mit Kontext
key_facts.md	Projektkonfiguration und Referenzinformationen
issues.md	Work-Log abgeschlossener Arbeiten

Memory-Aware Protocols

Bei Bug-Fixes:

1. Problem analysieren und Ursache finden
2. Bug beheben und testen
3. Eintrag in docs/project_notes/bugs.md hinzufügen

Bei Architekturentscheidungen:

1. Entscheidung mit Kontext dokumentieren
2. Alternativen und Trade-offs auflisten
3. Eintrag in docs/project_notes/decisions.md hinzufügen

Bei abgeschlossenen Tickets/Features:

1. Arbeit abschließen und verifizieren
2. Eintrag in docs/project_notes/issues.md hinzufügen

Bei Konfigurationsänderungen:

1. docs/project_notes/key_facts.md aktualisieren