# Enhanced Early Stopping with min_epochs Parameter

## Overview

The Trixy ML training framework now includes enhanced early stopping functionality that supports a `min_epochs` parameter. This ensures that training continues for at least a specified number of epochs before early stopping can take effect, giving models sufficient time to learn initial patterns.

## Why min_epochs is Important

### Problem with Standard Early Stopping
Standard early stopping can terminate training too early, especially for:
- **Complex models** that need time to converge
- **Deep learning models** with slow initial learning phases  
- **Audio models** that need to learn spectral patterns
- **Speaker recognition** where embeddings need time to stabilize

### Benefits of min_epochs
- **Prevents premature termination** during initial learning phases
- **Ensures model stability** before considering stopping
- **Improves final model quality** by allowing sufficient training time
- **Maintains backward compatibility** with existing configurations

## Configuration

### TrainerConfig Parameters

```python
from trainer.base import TrainerConfig

config = TrainerConfig(
    num_epochs=100,           # Maximum number of epochs
    min_epochs=20,            # Minimum epochs before early stopping can trigger
    early_stopping_patience=10, # Epochs to wait for improvement after min_epochs
    # ... other parameters
)
```

### Parameter Relationships

- `min_epochs`: Minimum number of epochs that must complete before early stopping is considered
- `num_epochs`: Maximum number of epochs (training stops here regardless)
- `early_stopping_patience`: Number of epochs to wait for improvement **after** min_epochs
- `max_epochs`: Hard upper limit (for safety)

### Validation Rules

The configuration system validates these relationships:
- `min_epochs >= 0`
- `min_epochs <= num_epochs`
- `min_epochs <= max_epochs`
- `num_epochs <= max_epochs`
- `early_stopping_patience > 0`

## How Enhanced Early Stopping Works

### Training Flow

1. **Initial Phase (epochs 0 to min_epochs-1)**:
   - Training proceeds normally
   - Validation loss is tracked
   - Best model state is saved when validation improves
   - **Early stopping counter is NOT incremented** (even if validation worsens)
   - **Early stopping counter is reset to 0** if still below min_epochs

2. **Early Stopping Phase (epochs min_epochs and beyond)**:
   - Early stopping counter increments when validation doesn't improve
   - Training stops when counter reaches `early_stopping_patience`
   - Best model state from entire training is restored

### Code Example

```python
# Enhanced early stopping logic
if val_loss is not None:
    if val_loss < self.metrics.best_val_loss:
        self.best_model_state = self.model.state_dict().copy()
        self.early_stopping_counter = 0
    else:
        # Only increment counter after min_epochs
        if epoch >= self.config.min_epochs:
            self.early_stopping_counter += 1
            
            if self.early_stopping_counter >= self.config.early_stopping_patience:
                self.logger.info(f"Early stopping triggered after {epoch + 1} epochs "
                               f"(minimum {self.config.min_epochs} epochs completed)")
                break
        else:
            # Reset counter if we're still below min_epochs
            self.early_stopping_counter = 0
```

## Examples

### Wakeword Detection
```python
config = TrainerConfig(
    trainer_name='wakeword_trainer',
    num_epochs=100,
    min_epochs=20,              # Allow 20 epochs to learn wakeword patterns
    early_stopping_patience=15, # Then wait 15 epochs for improvement
    # With this config, earliest stopping is at epoch 35 (20 + 15)
)
```

### Voice Recognition
```python
config = TrainerConfig(
    trainer_name='voice_recognition_trainer', 
    num_epochs=200,
    min_epochs=30,              # Allow 30 epochs for embeddings to stabilize
    early_stopping_patience=20, # Then wait 20 epochs for improvement
    # With this config, earliest stopping is at epoch 50 (30 + 20)
)
```

### Conservative Training
```python
config = TrainerConfig(
    num_epochs=500,
    min_epochs=100,             # Ensure substantial training
    early_stopping_patience=50, # Very patient after min_epochs
    # With this config, earliest stopping is at epoch 150 (100 + 50)
)
```

## Training Scenarios

### Scenario 1: Normal Early Stopping
```
Epochs: 1-19   | min_epochs phase, validation worsening, counter stays 0
Epoch 20:      | min_epochs reached, validation still bad, counter = 1
Epochs 21-24:  | validation continues worsening, counter = 2,3,4,5
Epoch 25:      | counter = 6, reaches patience (5), training stops
Result:        | Training stops at epoch 25 (min_epochs + patience)
```

### Scenario 2: Improvement During min_epochs
```
Epochs: 1-10   | min_epochs phase, validation worsening, counter stays 0
Epoch 11:      | validation improves! counter reset to 0
Epochs 12-19:  | validation worsens again, but still in min_epochs, counter stays 0
Epoch 20:      | min_epochs reached, validation bad, counter = 1
Epochs 21-24:  | validation continues worsening, counter = 2,3,4,5
Epoch 25:      | counter = 6, reaches patience, training stops
Result:        | Training stops at epoch 25, but best model is from epoch 11
```

### Scenario 3: No Early Stopping
```
Epochs: 1-100  | Validation improves periodically, counter never reaches patience
Result:        | Training completes all 100 epochs normally
```

## Backward Compatibility

### Default Values
- Existing configurations without `min_epochs` get default value of 10
- This provides reasonable protection without changing behavior dramatically
- All validation still works as expected

### Migration
No code changes required for existing trainers:
```python
# Old code still works
config = TrainerConfig(
    num_epochs=100,
    early_stopping_patience=10
    # min_epochs automatically set to 10
)
```

## Best Practices

### Choosing min_epochs

1. **Model Complexity**: More complex models need higher min_epochs
   - Simple models: 5-15 epochs
   - Medium models: 15-30 epochs  
   - Complex models: 30+ epochs

2. **Dataset Size**: Smaller datasets may need more epochs to stabilize
   - Large datasets: Lower min_epochs acceptable
   - Small datasets: Higher min_epochs recommended

3. **Architecture Type**:
   - **CNN models**: 10-20 epochs for feature learning
   - **RNN/LSTM**: 20-40 epochs for sequence learning
   - **Transformer**: 30+ epochs for attention learning
   - **Speaker embeddings**: 25-50 epochs for embedding space

4. **Learning Rate**: Higher learning rates may converge faster
   - High LR: Lower min_epochs
   - Low LR: Higher min_epochs

### Recommended Configurations

```python
# Rapid prototyping
TrainerConfig(num_epochs=50, min_epochs=5, early_stopping_patience=5)

# Balanced training  
TrainerConfig(num_epochs=100, min_epochs=15, early_stopping_patience=10)

# Production model
TrainerConfig(num_epochs=200, min_epochs=30, early_stopping_patience=20)

# Research/experimental
TrainerConfig(num_epochs=500, min_epochs=50, early_stopping_patience=30)
```

## Implementation Details

### Files Modified
- `trainer/base.py`: Enhanced early stopping logic in training loop
- `trainer/config.py`: Added validation rules and default values
- `trainer/wakeword/trainer.py`: Updated default configurations
- `trainer/voice_recognition/trainer.py`: Updated default configurations

### Validation Added
- Range validation: `min_epochs` must be >= 0
- Relationship validation: `min_epochs` <= `num_epochs` <= `max_epochs`
- Cross-trainer validation in config manager

### Logging Enhanced
Early stopping messages now include min_epochs information:
```
Early stopping triggered after 35 epochs (minimum 20 epochs completed)
```

## Testing

The enhanced functionality has been tested for:
- ✅ Correct early stopping behavior after min_epochs
- ✅ Counter reset during min_epochs phase  
- ✅ Configuration validation edge cases
- ✅ Backward compatibility with existing configs
- ✅ Integration with both wakeword and voice recognition trainers

## Summary

The `min_epochs` parameter provides fine-grained control over early stopping behavior, ensuring models receive adequate training time while still preventing overfitting. This enhancement maintains full backward compatibility while providing powerful new training control capabilities.