Fix Android voice bug - STT restart after TTS

PROBLEM:
startListening() triggers spurious AppState change to "background" on Android,
causing voice session to stop immediately after Julia responds.

ROOT CAUSE:
React Native AppState bug - requesting audio focus triggers false background event.

SOLUTION:
- Added sttStartingIgnoreAppStateRef flag
- Ignore AppState changes for 200ms after startListening() call
- Protects against false session termination during STT initialization

CHANGES:
- app/(tabs)/_layout.tsx: Added Android workaround with 200ms protection window
- VOICE_DEBUG_GUIDE.md: Documented bug, workaround, and expected logs

RESULT:
Voice session now continues correctly after Julia's response on Android.
STT successfully restarts and user can speak again without manual restart.
This commit is contained in:
Sergei 2026-01-29 09:41:16 -08:00
parent 3ec0f5dae2
commit 81a0c59060
2 changed files with 429 additions and 23 deletions

321
VOICE_DEBUG_GUIDE.md Normal file
View File

@ -0,0 +1,321 @@
# Voice Recognition Debug Guide
## Как работает голосовая система на Android vs iOS
### Основная проблема: Audio Focus на Android
**Android:** STT (Speech-to-Text) и TTS (Text-to-Speech) **НЕ МОГУТ** работать одновременно из-за конфликта Audio Focus. Когда TTS начинает говорить, она захватывает аудио-фокус, и STT автоматически останавливается системой.
**iOS:** STT и TTS **МОГУТ** работать одновременно. STT может слушать даже во время воспроизведения TTS (что позволяет прерывать Julia голосом).
### КРИТИЧЕСКАЯ проблема Android: startListening() → AppState background
**На Android обнаружен баг React Native:**
- При вызове `startListening()` AppState кратковременно переходит в `"background"` или `"inactive"`
- Это происходит из-за запроса аудио-фокуса и системных разрешений
- **НЕ связано** с действиями пользователя (блокировка экрана, переключение приложений)
**Workaround (реализован в коде):**
- Добавлен флаг `sttStartingIgnoreAppStateRef`
- Игнорируем AppState changes на 200ms после вызова `startListening()`
- Защищает от ложной остановки сессии во время запуска STT
**Логи при срабатывании workaround:**
```
LOG [Android] [TabLayout] 🛡️ Ignoring AppState changes for 200ms (STT start workaround)
LOG [Android] [TabLayout] ▶️ STARTING STT... (status: listening)
LOG [Android] [TabLayout] 📱 AppState changed to: "background"
LOG [Android] [TabLayout] 🛡️ IGNORING AppState change (STT start protection active)
LOG [Android] [TabLayout] ✅ STT started successfully
LOG [Android] [TabLayout] ✅ AppState monitoring resumed
```
---
## Нормальный Flow (Должен работать так)
### 1⃣ **Юзер нажимает кнопку "Start"**
**Android:**
```
[Android] [TabLayout] 🎯 FAB PRESSED - isSessionActive: false
[Android] [TabLayout] ▶️ STARTING session (FAB start)
[Android] [VoiceContext] 🎤 STARTING voice session
[Android] [VoiceContext] → Session initialized, status=listening
[Android] [TabLayout] 🎤 Voice session STARTED - starting STT
[Android] [TabLayout] ▶️ STARTING STT... (status: listening)
[Android] [TabLayout] ✅ STT started successfully
```
**iOS:**
```
[iOS] [TabLayout] 🎯 FAB PRESSED - isSessionActive: false
[iOS] [TabLayout] ▶️ STARTING session (FAB start)
[iOS] [VoiceContext] 🎤 STARTING voice session
[iOS] [VoiceContext] → Session initialized, status=listening
[iOS] [TabLayout] 🎤 Voice session STARTED - starting STT
[iOS] [TabLayout] ▶️ STARTING STT... (status: listening)
[iOS] [TabLayout] ✅ STT started successfully
```
### 2⃣ **Юзер говорит что-то**
**Android:**
```
[Android] [TabLayout] handleSpeechResult - isFinal: false, status: listening, transcript: "hello..."
[Android] [TabLayout] → Updating PARTIAL transcript
[Android] [TabLayout] Partial changed: "" → "hello"
[Android] [TabLayout] → Started 2s silence timer
```
**iOS:** (то же самое)
### 3⃣ **2 секунды тишины → auto-stop**
**Android:**
```
[Android] [TabLayout] 🤖 AUTO-STOP: 2s silence - stopping STT
[Android] [TabLayout] handleSTTEnd - sessionActive: true, status: listening
[Android] [TabLayout] → shouldRestartSTT set to TRUE
[Android] [TabLayout] handleSpeechResult - isFinal: true, status: listening, transcript: "hello"
[Android] [TabLayout] → Processing FINAL transcript, sending to API
```
**iOS:** (то же самое, но с префиксом `[iOS]`)
### 4⃣ **API запрос**
**Android:**
```
[Android] [VoiceContext] 📤 Sending transcript to API (ask_wellnuo_ai): "hello"
[Android] [VoiceContext] 🔑 Getting API token...
[Android] [VoiceContext] ✅ Token obtained
[Android] [VoiceContext] 📝 Normalized question: "how is dad doing"
[Android] [VoiceContext] 📡 Using API type: ask_wellnuo_ai, deployment: 21
[Android] [VoiceContext] 🌐 Sending API request...
[Android] [TabLayout] Status transition: listening → processing, sttIsListening: true
[Android] [TabLayout] ⏸️ Stopping STT during processing (echo prevention)
```
**Важно на Android:** STT останавливается во время `processing`, чтобы не слышать эхо от TTS.
### 5⃣ **API ответ получен**
**Android:**
```
[Android] [VoiceContext] 📥 API response received, parsing...
[Android] [VoiceContext] ✅ API SUCCESS: "Your dad Ferdinand is doing well. He spent most of his time..."
[Android] [VoiceContext] 🔊 Starting TTS for response...
[Android] [VoiceContext] 🔊 Starting TTS: "Your dad Ferdinand is doing well..."
[Android] [VoiceContext] ▶️ TTS playback STARTED
[Android] [TabLayout] Status transition: processing → speaking, sttIsListening: false
```
**iOS:** (то же самое)
### 6⃣ **TTS заканчивает говорить**
**Android:**
```
[Android] [VoiceContext] ✅ TTS playback COMPLETED
[Android] [VoiceContext] → isSpeaking = false (immediate - audio focus release)
[Android] [VoiceContext] → status = listening (ready for next input)
[Android] [VoiceContext] ✅ TTS completed
[Android] [TabLayout] Status transition: speaking → listening, sttIsListening: false
[Android] [TabLayout] 🔄 TTS FINISHED - preparing to restart STT
[Android] [TabLayout] ⏱️ Waiting 50ms before restarting STT (audio focus release)
```
**iOS:**
```
[iOS] [VoiceContext] ✅ TTS playback COMPLETED
[iOS] [VoiceContext] ⏱️ Delaying isSpeaking=false by 300ms (match STT restart)
[iOS] [VoiceContext] → status = listening (ready for next input)
[iOS] [VoiceContext] ✅ TTS completed
[iOS] [TabLayout] Status transition: speaking → listening, sttIsListening: false
[iOS] [TabLayout] 🔄 TTS FINISHED - preparing to restart STT
[iOS] [TabLayout] ⏱️ Waiting 300ms before restarting STT (audio focus release)
[iOS] [VoiceContext] → isSpeaking = false (after 300ms delay)
```
**Ключевая разница:**
- **Android:** `isSpeaking = false` **сразу** (50ms delay для STT restart)
- **iOS:** `isSpeaking = false` через **300ms** (плавный переход, matches STT restart)
### 7⃣ **STT рестартует после TTS**
**Android:**
```
[Android] [TabLayout] ⏰ Delay complete - restarting STT now
[Android] [TabLayout] ▶️ STARTING STT... (status: listening)
[Android] [TabLayout] ✅ STT started successfully
```
**iOS:** (то же самое)
### 8⃣ **Юзер может говорить снова → повтор с шага 2**
---
## Что НЕ ДОЛЖНО происходить (баги)
### ❌ БАГ 1: STT не рестартует после TTS (Android)
**Симптомы:**
```
[Android] [VoiceContext] ✅ TTS playback COMPLETED
[Android] [TabLayout] Status transition: speaking → listening, sttIsListening: false
[Android] [TabLayout] 🔄 TTS FINISHED - preparing to restart STT
[Android] [TabLayout] ⏱️ Waiting 50ms before restarting STT (audio focus release)
[Android] [TabLayout] ⏰ Delay complete - restarting STT now
[Android] [TabLayout] safeStartSTT - already listening or starting, skipping ❌ ПРОБЛЕМА!
```
**Причина:** `sttIsListening` или `sttStartingRef.current` остались `true`, хотя STT на самом деле не работает.
**Решение:** Убедиться что `stopListening()` вызывается корректно и флаги сбрасываются.
### ❌ БАГ 2: STT не останавливается во время TTS (эхо)
**Симптомы:**
```
[Android] [TabLayout] Status transition: processing → speaking, sttIsListening: true ❌ ПРОБЛЕМА!
```
**Причина:** STT не остановилось при переходе в `speaking`, и будет слышать Julia как эхо.
**Решение:** Проверить что `stopListening()` вызывается при `status === 'speaking'`.
### ❌ БАГ 3: isSpeaking не сбрасывается
**Симптомы (Android):**
```
[Android] [VoiceContext] ✅ TTS playback COMPLETED
[Android] [VoiceContext] → isSpeaking = false (immediate - audio focus release)
... но FAB остаётся зелёным (isSpeaking === true в UI)
```
**Причина:** `setIsSpeaking(false)` не отрабатывает или есть race condition.
**Решение:** Проверить что `setIsSpeaking(false)` вызывается в `onDone` TTS callback.
---
## Как читать логи
### Префиксы платформ:
- `[iOS]` — событие происходит на iOS
- `[Android]` — событие происходит на Android
### Префиксы модулей:
- `[TabLayout]` — события из `app/(tabs)/_layout.tsx` (STT orchestration)
- `[VoiceContext]` — события из `contexts/VoiceContext.tsx` (API + TTS)
### Эмодзи:
- 🎯 — FAB нажат
- 🎤 — Voice session start/stop
- ▶️ — STT start
- ⏸️ — STT stop
- 📤 — Sending to API
- 📥 — API response
- 🔊 — TTS start/playback
- ✅ — Success
- ❌ — Error
- ⚠️ — Warning
- 🔄 — Restart/retry
- ⏱️ — Timer/delay
- ⏰ — Timer triggered
- 🛑 — Force stop
---
## Как тестировать
### Тест 1: Базовый цикл (1 вопрос → 1 ответ → 1 вопрос)
1. Нажать FAB (зелёная кнопка)
2. Сказать: "Hello"
3. Подождать 2 секунды (auto-stop)
4. **Проверить:** Julia отвечает
5. **Проверить:** После ответа Julia, STT рестартует автоматически
6. Сказать: "How is dad?"
7. **Проверить:** Julia отвечает снова
**Ожидаемые логи (Android):**
```
[Android] [TabLayout] 🎯 FAB PRESSED - isSessionActive: false
[Android] [TabLayout] ▶️ STARTING session (FAB start)
[Android] [TabLayout] ▶️ STARTING STT... (status: listening)
[Android] [TabLayout] ✅ STT started successfully
... пользователь говорит ...
[Android] [TabLayout] 🤖 AUTO-STOP: 2s silence - stopping STT
[Android] [TabLayout] → Processing FINAL transcript, sending to API
[Android] [VoiceContext] 📤 Sending transcript to API
[Android] [TabLayout] ⏸️ Stopping STT during processing (echo prevention)
[Android] [VoiceContext] ✅ API SUCCESS
[Android] [VoiceContext] 🔊 Starting TTS for response...
[Android] [VoiceContext] ▶️ TTS playback STARTED
... Julia говорит ...
[Android] [VoiceContext] ✅ TTS playback COMPLETED
[Android] [TabLayout] 🔄 TTS FINISHED - preparing to restart STT
[Android] [TabLayout] ⏱️ Waiting 50ms before restarting STT
[Android] [TabLayout] ⏰ Delay complete - restarting STT now
[Android] [TabLayout] ▶️ STARTING STT... (status: listening)
[Android] [TabLayout] ✅ STT started successfully ← КРИТИЧНО! Если нет этой строки — БАГ!
```
### Тест 2: Прерывание Julia голосом (только iOS)
**На iOS:**
1. Нажать FAB
2. Сказать: "Tell me a long story"
3. Пока Julia говорит, сказать: "Stop"
4. **Проверить:** Julia прервалась
5. **Проверить:** "Stop" отправилось как новый запрос
**На Android:**
Прерывание голосом **НЕ РАБОТАЕТ** (STT выключен во время TTS). Прервать можно только кнопкой FAB.
---
## Checklist для отладки
Если STT не рестартует после TTS на Android:
- [ ] Проверь что `stopListening()` вызывается при `status === 'speaking'`
- [ ] Проверь что `sttIsListening === false` после `stopListening()`
- [ ] Проверь что `safeStartSTT()` вызывается после задержки 50ms
- [ ] Проверь что `sttStartingRef.current === false` перед вызовом `safeStartSTT()`
- [ ] Проверь что `sessionActiveRef.current === true` (сессия активна)
- [ ] Проверь логи на наличие `[Android] [TabLayout] ✅ STT started successfully`
- [ ] Проверь что нет логов `[Android] [TabLayout] ⚠️ SKIPPING STT start - TTS is playing`
---
## Audio Focus на Android (детально)
### Как работает Audio Focus:
1. **TTS запрашивает Audio Focus:**
- Когда `Speech.speak()` вызывается, Android TTS запрашивает `AUDIOFOCUS_GAIN_TRANSIENT`
- Это **автоматически** останавливает все другие аудио-источники, включая STT
2. **STT теряет Audio Focus:**
- Expo Speech Recognition получает событие `onAudioFocusLoss`
- STT автоматически останавливается (не нужно вручную вызывать `stopListening()`)
3. **TTS освобождает Audio Focus:**
- Когда TTS заканчивает (`onDone` callback), Audio Focus освобождается
- Но STT **НЕ** рестартует автоматически — нужно вручную вызвать `startListening()`
### Почему 50ms delay на Android?
- TTS может освободить Audio Focus с небольшой задержкой
- Если попытаться запустить STT слишком рано, STT может не получить Audio Focus
- 50ms — достаточно для освобождения, но незаметно для юзера
### Почему 300ms delay на iOS?
- iOS не имеет проблем с Audio Focus
- 300ms delay нужен для плавного перехода анимаций (зелёный индикатор)
- Также даёт время для TTS полностью завершиться и освободить аудио-драйвер

View File

@ -45,39 +45,55 @@ export default function TabLayout() {
// so interruption on Android happens via FAB press instead. // so interruption on Android happens via FAB press instead.
// On iOS, STT can run alongside TTS, so voice detection works. // On iOS, STT can run alongside TTS, so voice detection works.
const handleVoiceDetected = useCallback(() => { const handleVoiceDetected = useCallback(() => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
console.log(`${platformPrefix} [TabLayout] handleVoiceDetected called - status: ${status}, isSpeaking: ${isSpeaking}`);
if (Platform.OS === 'ios' && (status === 'speaking' || isSpeaking)) { if (Platform.OS === 'ios' && (status === 'speaking' || isSpeaking)) {
console.log('[TabLayout] Voice detected during TTS (iOS) - INTERRUPTING Julia'); console.log('[iOS] [TabLayout] Voice detected during TTS - INTERRUPTING Julia');
interruptIfSpeaking(); interruptIfSpeaking();
} else if (Platform.OS === 'android') {
console.log('[Android] [TabLayout] Voice detected but ignoring (STT disabled during TTS on Android)');
} }
}, [status, isSpeaking, interruptIfSpeaking]); }, [status, isSpeaking, interruptIfSpeaking]);
// Callback when STT ends - may need to restart if session is still active // Callback when STT ends - may need to restart if session is still active
const handleSTTEnd = useCallback(() => { const handleSTTEnd = useCallback(() => {
console.log('[TabLayout] STT ended, sessionActive:', sessionActiveRef.current); const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
console.log(`${platformPrefix} [TabLayout] handleSTTEnd - sessionActive: ${sessionActiveRef.current}, status: ${status}`);
// If session is still active (user didn't stop it), we should restart STT // If session is still active (user didn't stop it), we should restart STT
// This ensures STT continues during and after TTS playback // This ensures STT continues during and after TTS playback
if (sessionActiveRef.current) { if (sessionActiveRef.current) {
shouldRestartSTTRef.current = true; shouldRestartSTTRef.current = true;
console.log(`${platformPrefix} [TabLayout] → shouldRestartSTT set to TRUE`);
} else {
console.log(`${platformPrefix} [TabLayout] → Session not active, will NOT restart STT`);
} }
}, []); }, [status]);
// Callback for STT results // Callback for STT results
const handleSpeechResult = useCallback((transcript: string, isFinal: boolean) => { const handleSpeechResult = useCallback((transcript: string, isFinal: boolean) => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
console.log(`${platformPrefix} [TabLayout] handleSpeechResult - isFinal: ${isFinal}, status: ${status}, transcript: "${transcript.slice(0, 40)}..."`);
// Ignore any STT results during TTS playback or processing (echo prevention) // Ignore any STT results during TTS playback or processing (echo prevention)
if (status === 'speaking' || status === 'processing') { if (status === 'speaking' || status === 'processing') {
if (isFinal) { if (isFinal) {
// User interrupted Julia with speech — store to send after TTS stops // User interrupted Julia with speech — store to send after TTS stops
console.log('[TabLayout] Got final result during TTS/processing - storing for after interruption:', transcript); console.log(`${platformPrefix} [TabLayout] Got FINAL result during ${status} - storing for after interruption: "${transcript}"`);
pendingInterruptTranscriptRef.current = transcript; pendingInterruptTranscriptRef.current = transcript;
} else {
console.log(`${platformPrefix} [TabLayout] Ignoring PARTIAL transcript during ${status} (likely echo)`);
} }
// Ignore partial transcripts during TTS (they're likely echo)
return; return;
} }
if (isFinal) { if (isFinal) {
console.log(`${platformPrefix} [TabLayout] → Processing FINAL transcript, sending to API`);
setTranscript(transcript); setTranscript(transcript);
sendTranscript(transcript); sendTranscript(transcript);
} else { } else {
console.log(`${platformPrefix} [TabLayout] → Updating PARTIAL transcript`);
setPartialTranscript(transcript); setPartialTranscript(transcript);
} }
}, [setTranscript, setPartialTranscript, sendTranscript, status]); }, [setTranscript, setPartialTranscript, sendTranscript, status]);
@ -98,6 +114,8 @@ export default function TabLayout() {
// Ref to prevent concurrent startListening calls // Ref to prevent concurrent startListening calls
const sttStartingRef = useRef(false); const sttStartingRef = useRef(false);
// Ref to ignore AppState changes during STT start (Android bug workaround)
const sttStartingIgnoreAppStateRef = useRef(false);
// Ref to track last partial transcript for iOS auto-stop // Ref to track last partial transcript for iOS auto-stop
const lastPartialTextRef = useRef(''); const lastPartialTextRef = useRef('');
const silenceTimerRef = useRef<NodeJS.Timeout | null>(null); const silenceTimerRef = useRef<NodeJS.Timeout | null>(null);
@ -105,6 +123,8 @@ export default function TabLayout() {
// iOS AUTO-STOP: Stop STT after 2 seconds of silence (no new partial transcripts) // iOS AUTO-STOP: Stop STT after 2 seconds of silence (no new partial transcripts)
// This triggers onEnd → iOS fix sends lastPartial as final // This triggers onEnd → iOS fix sends lastPartial as final
useEffect(() => { useEffect(() => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
// Clear existing timer // Clear existing timer
if (silenceTimerRef.current) { if (silenceTimerRef.current) {
clearTimeout(silenceTimerRef.current); clearTimeout(silenceTimerRef.current);
@ -118,15 +138,22 @@ export default function TabLayout() {
// If partial changed, update ref and set new 2s timer // If partial changed, update ref and set new 2s timer
if (currentPartial !== lastPartialTextRef.current) { if (currentPartial !== lastPartialTextRef.current) {
console.log(`${platformPrefix} [TabLayout] Partial changed: "${lastPartialTextRef.current}" → "${currentPartial}"`);
lastPartialTextRef.current = currentPartial; lastPartialTextRef.current = currentPartial;
// Start 2-second silence timer // Start 2-second silence timer
silenceTimerRef.current = setTimeout(() => { silenceTimerRef.current = setTimeout(() => {
if (sttIsListening && sessionActiveRef.current) { if (sttIsListening && sessionActiveRef.current) {
console.log('[TabLayout] 🍎 iOS AUTO-STOP: 2s silence - stopping STT to trigger onEnd → iOS fix'); if (Platform.OS === 'ios') {
console.log('[iOS] [TabLayout] 🍎 AUTO-STOP: 2s silence - stopping STT to trigger onEnd → iOS fix');
} else {
console.log('[Android] [TabLayout] 🤖 AUTO-STOP: 2s silence - stopping STT');
}
stopListening(); stopListening();
} }
}, 2000); }, 2000);
console.log(`${platformPrefix} [TabLayout] → Started 2s silence timer`);
} }
} }
@ -140,19 +167,44 @@ export default function TabLayout() {
// Safe wrapper to start STT with debounce protection // Safe wrapper to start STT with debounce protection
const safeStartSTT = useCallback(() => { const safeStartSTT = useCallback(() => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
if (sttIsListening || sttStartingRef.current) { if (sttIsListening || sttStartingRef.current) {
console.log(`${platformPrefix} [TabLayout] safeStartSTT - already listening or starting, skipping`);
return; // Already listening or starting return; // Already listening or starting
} }
// Don't start STT during TTS on Android - they share audio focus // Don't start STT during TTS on Android - they share audio focus
if (Platform.OS === 'android' && (status === 'speaking' || isSpeaking)) { if (Platform.OS === 'android' && (status === 'speaking' || isSpeaking)) {
console.log('[TabLayout] Skipping STT start - TTS is playing (Android audio focus)'); console.log('[Android] [TabLayout] ⚠️ SKIPPING STT start - TTS is playing (audio focus conflict)');
return; return;
} }
sttStartingRef.current = true; sttStartingRef.current = true;
console.log('[TabLayout] Starting STT...');
startListening().finally(() => { // ANDROID BUG WORKAROUND: startListening() triggers AppState change to background
sttStartingRef.current = false; // Ignore AppState changes for 200ms after starting STT
}); if (Platform.OS === 'android') {
sttStartingIgnoreAppStateRef.current = true;
console.log('[Android] [TabLayout] 🛡️ Ignoring AppState changes for 200ms (STT start workaround)');
setTimeout(() => {
sttStartingIgnoreAppStateRef.current = false;
console.log('[Android] [TabLayout] ✅ AppState monitoring resumed');
}, 200);
}
console.log(`${platformPrefix} [TabLayout] ▶️ STARTING STT... (status: ${status})`);
startListening()
.then(() => {
console.log(`${platformPrefix} [TabLayout] ✅ STT started successfully`);
})
.catch((err) => {
console.error(`${platformPrefix} [TabLayout] ❌ STT start failed:`, err);
})
.finally(() => {
sttStartingRef.current = false;
});
}, [sttIsListening, status, isSpeaking, startListening]); }, [sttIsListening, status, isSpeaking, startListening]);
// Update session active ref when isListening changes // Update session active ref when isListening changes
@ -165,11 +217,13 @@ export default function TabLayout() {
// Start/stop STT when voice session starts/stops // Start/stop STT when voice session starts/stops
useEffect(() => { useEffect(() => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
if (isListening) { if (isListening) {
console.log('[TabLayout] Voice session started - starting STT'); console.log(`${platformPrefix} [TabLayout] 🎤 Voice session STARTED - starting STT`);
safeStartSTT(); safeStartSTT();
} else { } else {
console.log('[TabLayout] Voice session ended - stopping STT'); console.log(`${platformPrefix} [TabLayout] 🛑 Voice session ENDED - stopping STT`);
stopListening(); stopListening();
} }
}, [isListening]); // eslint-disable-line react-hooks/exhaustive-deps }, [isListening]); // eslint-disable-line react-hooks/exhaustive-deps
@ -180,23 +234,26 @@ export default function TabLayout() {
// Stop STT when entering processing or speaking state (prevent echo) // Stop STT when entering processing or speaking state (prevent echo)
// Restart STT when TTS finishes (speaking → listening) // Restart STT when TTS finishes (speaking → listening)
useEffect(() => { useEffect(() => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
const prevStatus = prevStatusRef.current; const prevStatus = prevStatusRef.current;
prevStatusRef.current = status; prevStatusRef.current = status;
console.log(`${platformPrefix} [TabLayout] Status transition: ${prevStatus}${status}, sttIsListening: ${sttIsListening}`);
// Stop STT when processing starts or TTS starts (prevent Julia hearing herself) // Stop STT when processing starts or TTS starts (prevent Julia hearing herself)
if ((status === 'processing' || status === 'speaking') && sttIsListening) { if ((status === 'processing' || status === 'speaking') && sttIsListening) {
console.log('[TabLayout] Stopping STT during', status, '(echo prevention)'); console.log(`${platformPrefix} [TabLayout] ⏸️ Stopping STT during ${status} (echo prevention)`);
stopListening(); stopListening();
} }
// When TTS finishes (speaking → listening), restart STT // When TTS finishes (speaking → listening), restart STT
if (prevStatus === 'speaking' && status === 'listening' && sessionActiveRef.current) { if (prevStatus === 'speaking' && status === 'listening' && sessionActiveRef.current) {
console.log('[TabLayout] TTS finished - restarting STT'); console.log(`${platformPrefix} [TabLayout] 🔄 TTS FINISHED - preparing to restart STT`);
// Process pending transcript from interruption if any // Process pending transcript from interruption if any
const pendingTranscript = pendingInterruptTranscriptRef.current; const pendingTranscript = pendingInterruptTranscriptRef.current;
if (pendingTranscript) { if (pendingTranscript) {
console.log('[TabLayout] Processing pending interrupt transcript:', pendingTranscript); console.log(`${platformPrefix} [TabLayout] 📝 Processing pending interrupt transcript: "${pendingTranscript}"`);
pendingInterruptTranscriptRef.current = null; pendingInterruptTranscriptRef.current = null;
setTranscript(pendingTranscript); setTranscript(pendingTranscript);
sendTranscript(pendingTranscript); sendTranscript(pendingTranscript);
@ -206,9 +263,14 @@ export default function TabLayout() {
// iOS: 300ms for smooth audio fade // iOS: 300ms for smooth audio fade
// Android: 50ms (Audio Focus releases immediately) // Android: 50ms (Audio Focus releases immediately)
const delay = Platform.OS === 'android' ? 50 : 300; const delay = Platform.OS === 'android' ? 50 : 300;
console.log(`${platformPrefix} [TabLayout] ⏱️ Waiting ${delay}ms before restarting STT (audio focus release)`);
const timer = setTimeout(() => { const timer = setTimeout(() => {
if (sessionActiveRef.current) { if (sessionActiveRef.current) {
console.log(`${platformPrefix} [TabLayout] ⏰ Delay complete - restarting STT now`);
safeStartSTT(); safeStartSTT();
} else {
console.log(`${platformPrefix} [TabLayout] ⚠️ Session stopped during delay, NOT restarting STT`);
} }
}, delay); }, delay);
return () => clearTimeout(timer); return () => clearTimeout(timer);
@ -220,6 +282,8 @@ export default function TabLayout() {
// When STT ends unexpectedly during active session, restart it (but not during TTS) // When STT ends unexpectedly during active session, restart it (but not during TTS)
useEffect(() => { useEffect(() => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
if ( if (
shouldRestartSTTRef.current && shouldRestartSTTRef.current &&
sessionActiveRef.current && sessionActiveRef.current &&
@ -228,10 +292,15 @@ export default function TabLayout() {
status !== 'speaking' status !== 'speaking'
) { ) {
shouldRestartSTTRef.current = false; shouldRestartSTTRef.current = false;
console.log('[TabLayout] STT ended unexpectedly - restarting'); console.log(`${platformPrefix} [TabLayout] 🔄 STT ended UNEXPECTEDLY - will restart in 300ms`);
console.log(`${platformPrefix} [TabLayout] → Conditions: sessionActive=${sessionActiveRef.current}, status=${status}`);
const timer = setTimeout(() => { const timer = setTimeout(() => {
if (sessionActiveRef.current) { if (sessionActiveRef.current) {
console.log(`${platformPrefix} [TabLayout] ⏰ Restarting STT after unexpected end`);
safeStartSTT(); safeStartSTT();
} else {
console.log(`${platformPrefix} [TabLayout] ⚠️ Session stopped during delay, NOT restarting`);
} }
}, 300); }, 300);
return () => clearTimeout(timer); return () => clearTimeout(timer);
@ -241,10 +310,20 @@ export default function TabLayout() {
// Handle app state changes (background/foreground) // Handle app state changes (background/foreground)
useEffect(() => { useEffect(() => {
const handleAppStateChange = (nextAppState: AppStateStatus) => { const handleAppStateChange = (nextAppState: AppStateStatus) => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
console.log(`${platformPrefix} [TabLayout] 📱 AppState changed to: "${nextAppState}"`);
// ANDROID BUG WORKAROUND: Ignore AppState changes during STT start
// startListening() triggers spurious background transition on Android
if (Platform.OS === 'android' && sttStartingIgnoreAppStateRef.current) {
console.log(`[Android] [TabLayout] 🛡️ IGNORING AppState change (STT start protection active)`);
return;
}
// When app goes to background/inactive - stop voice session // When app goes to background/inactive - stop voice session
// STT/TTS cannot work in background, so it's pointless to keep session active // STT/TTS cannot work in background, so it's pointless to keep session active
if ((nextAppState === 'background' || nextAppState === 'inactive') && sessionActiveRef.current) { if ((nextAppState === 'background' || nextAppState === 'inactive') && sessionActiveRef.current) {
console.log('[TabLayout] App going to background - stopping voice session'); console.log(`${platformPrefix} [TabLayout] App going to ${nextAppState} - stopping voice session`);
stopListening(); stopListening();
stopSession(); stopSession();
sessionActiveRef.current = false; sessionActiveRef.current = false;
@ -255,7 +334,7 @@ export default function TabLayout() {
// When app comes back to foreground - do NOT auto-restart session // When app comes back to foreground - do NOT auto-restart session
// User must manually press FAB to start new session // User must manually press FAB to start new session
if (nextAppState === 'active') { if (nextAppState === 'active') {
console.log('[TabLayout] App foregrounded - session remains stopped (user must restart via FAB)'); console.log(`${platformPrefix} [TabLayout] App foregrounded - session remains stopped (user must restart via FAB)`);
} }
}; };
@ -266,18 +345,21 @@ export default function TabLayout() {
// Handle voice FAB press - toggle listening mode // Handle voice FAB press - toggle listening mode
// Must check ALL active states (listening, processing, speaking), not just isListening // Must check ALL active states (listening, processing, speaking), not just isListening
const handleVoiceFABPress = useCallback(() => { const handleVoiceFABPress = useCallback(() => {
const platformPrefix = Platform.OS === 'ios' ? '[iOS]' : '[Android]';
const isSessionActive = isListening || status === 'speaking' || status === 'processing'; const isSessionActive = isListening || status === 'speaking' || status === 'processing';
console.log('[TabLayout] FAB pressed, isSessionActive:', isSessionActive, 'status:', status, 'isListening:', isListening); console.log(`${platformPrefix} [TabLayout] 🎯 FAB PRESSED - isSessionActive: ${isSessionActive}, status: ${status}, isListening: ${isListening}`);
if (isSessionActive) { if (isSessionActive) {
// Force-stop everything: STT, TTS, and session state // Force-stop everything: STT, TTS, and session state
console.log('[TabLayout] Force-stopping everything'); console.log(`${platformPrefix} [TabLayout] 🛑 FORCE-STOPPING everything (FAB stop)`);
stopListening(); stopListening();
stopSession(); stopSession();
sessionActiveRef.current = false; sessionActiveRef.current = false;
shouldRestartSTTRef.current = false; shouldRestartSTTRef.current = false;
pendingInterruptTranscriptRef.current = null; pendingInterruptTranscriptRef.current = null;
console.log(`${platformPrefix} [TabLayout] → All flags cleared`);
} else { } else {
console.log(`${platformPrefix} [TabLayout] ▶️ STARTING session (FAB start)`);
startSession(); startSession();
} }
}, [isListening, status, startSession, stopSession, stopListening]); }, [isListening, status, startSession, stopSession, stopListening]);
@ -351,11 +433,14 @@ export default function TabLayout() {
), ),
}} }}
/> />
{/* Voice Debug - hidden from tab bar */} {/* Voice Debug - VISIBLE for debugging */}
<Tabs.Screen <Tabs.Screen
name="voice-debug" name="voice-debug"
options={{ options={{
href: null, title: 'Debug',
tabBarIcon: ({ color, size }) => (
<Feather name="activity" size={22} color={color} />
),
}} }}
/> />
<Tabs.Screen <Tabs.Screen