13 Commits

Author SHA1 Message Date
8a6d9fa420 Add voice error handling - speak errors aloud and continue session
Previously: API errors were silent, session stopped, user confused
Now: Julia speaks error message, adds to transcript, keeps listening

Changes:
- Catch block now speaks error via TTS: "Sorry, I encountered an error: [msg]"
- Error added to transcript (visible in chat history)
- Session continues in listening state (doesn't stop on error)
- User can try again immediately without restarting session

UX improvement: graceful error recovery instead of silent failure

contexts/VoiceContext.tsx:332-351
2026-01-28 20:07:51 -08:00
9422c32926 Extend speaking indicator duration to cover STT restart delay
User feedback: green speaker indicator was turning off too early,
creating an awkward visual gap during the 300ms audio focus delay.

Changes:
- Added 300ms delay to setIsSpeaking(false) in TTS onDone callback
- Keeps green "Speaking" indicator on until STT actually starts recording
- onError and onStopped still turn off immediately (user interruption)
- Smooth visual transition: TTS ends → 300ms delay → STT starts

UX improvement: eliminates the brief "nothing is happening" moment
between Julia finishing speech and microphone starting to listen.

contexts/VoiceContext.tsx:393
2026-01-28 20:05:38 -08:00
dccf05947c Add Voice API type to debug UI and logs
Changes:
- Show API function (voice_ask or ask_wellnuo_ai) in Debug tab
- Update console.log to include API type in transcript log
- Add "API Function" field in Status Card with blue color

Now Debug tab clearly shows which API function is being used, making it easy to verify the Profile settings are working correctly.
2026-01-28 19:53:53 -08:00
0881a9565d Fix updateVoiceApiType function and add API logging
Fixes error: "updateVoiceApiType is not a function"

Changes:
- Add voiceApiType state to VoiceContext
- Implement updateVoiceApiType callback
- Load saved voice API type from SecureStore on mount
- Use voiceApiType in sendTranscript (instead of hardcoded 'ask_wellnuo_ai')
- Add console.log showing which API type is being used
- Export voiceApiType and updateVoiceApiType in context provider

Now Voice API selector in Profile works correctly and logs show which API function (voice_ask or ask_wellnuo_ai) is being called.
2026-01-28 19:50:00 -08:00
05f872d067 fix: voice session improvements - FAB stop, echo prevention, chat TTS
- FAB button now correctly stops session during speaking/processing states
- Echo prevention: STT stopped during TTS playback, results ignored during speaking
- Chat TTS only speaks when voice session is active (no auto-speak for text chat)
- Session stop now aborts in-flight API requests and prevents race conditions
- STT restarts after TTS with 800ms delay for audio focus release
- Pending interrupt transcript processed after TTS completion
- ChatContext added for message persistence across tab navigation
- VoiceFAB redesigned with state-based animations
- console.error replaced with console.warn across voice pipeline
- no-speech STT errors silenced (normal silence behavior)
2026-01-27 22:59:55 -08:00
88d4afcdfd Display Julia's voice responses in chat
When user speaks via voice mode, both their question and Julia's
response are now shown in the text chat. This provides a unified
conversation history for both voice and text interactions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-27 16:41:14 -08:00
3c7a48df5b Integrate TTS interruption in VoiceFAB when voice detected
- Add onVoiceDetected callback to useSpeechRecognition hook
  - Triggered on first interim result (voice activity detected)
  - Uses voiceDetectedRef to ensure callback fires only once per session
  - Reset flag on session start/end

- Connect STT to VoiceContext in _layout.tsx
  - Use useSpeechRecognition with onVoiceDetected callback
  - Call interruptIfSpeaking() when voice detected during 'speaking' state
  - Forward STT results to VoiceContext (setTranscript, sendTranscript)
  - Start/stop STT based on isListening state

- Export interruptIfSpeaking from VoiceContext provider

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-27 16:34:07 -08:00
356205d8c0 Remove LiveKit SDK and related code
- Remove @livekit/react-native-expo-plugin from app.json
- Remove @config-plugins/react-native-webrtc plugin
- Delete utils/audioSession.ts (depended on LiveKit AudioSession)
- Update VoiceCallContext.tsx comments
- Update callManager.ts comments
- Update _layout.tsx TODO comment
- Remove LiveKit documentation files
- Add interruptIfSpeaking to VoiceContext for TTS interrupt

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-27 16:28:23 -08:00
caf47ead9c Add VoiceContext with API integration and TTS
- Create VoiceContext.tsx with sendTranscript() for WellNuo API calls
- Integrate expo-speech TTS for response playback
- Add VoiceProvider to app layout
- Flow: transcript → normalize → API → response → TTS → continue listening

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-27 16:16:50 -08:00
Sergei
aec300bd98 feat: Add floating bubble during voice calls
- Create VoiceCallContext for global voice call state management
- Add FloatingCallBubble component with drag support
- Add minimize button to voice call screen
- Show bubble when call is minimized, tap to return to call
- Button shows active call state with green color

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-24 20:39:27 -08:00
Sergei
a578ec8081 feat: Pass Debug tab deployment ID to voice calls
- Add debugDeploymentId to BeneficiaryContext for sharing between screens
- Sync Debug tab's deploymentId state with global context
- voice-call.tsx now prioritizes debugDeploymentId when starting calls
- Enables testing voice calls with specific deployment IDs from Debug screen
2026-01-24 00:05:47 -08:00
Sergei
dde0ecb9cd Add Julia AI voice agent with LiveKit integration
Voice AI Features:
- LiveKit Agents integration for real-time voice calls
- Julia AI agent (Python) deployed to LiveKit Cloud
- Token server for authentication
- Debug screen with voice call testing
- Voice call screen with full-screen UI

Agent Configuration:
- STT: Deepgram Nova-2
- LLM: OpenAI GPT-4o
- TTS: Deepgram Aura Asteria (female voice)
- Turn Detection: LiveKit Multilingual Model
- VAD: Silero
- Noise Cancellation: LiveKit BVC

Files added:
- julia-agent/ - Complete agent code and token server
- app/voice-call.tsx - Full-screen voice call UI
- services/livekitService.ts - LiveKit client service
- contexts/VoiceTranscriptContext.tsx - Transcript state
- polyfills/livekit-globals.ts - WebRTC polyfills

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-17 17:58:31 -08:00
Sergei
8bc9649146 WellNuo Lite v1.0.0 - simplified version for App Store review
- Removed voice input features
- Simplified profile page (only legal links and logout)
- Chat with AI context working
- Auto-select first beneficiary
- Dashboard WebView intact

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-24 17:13:13 -08:00