14 Commits

Author SHA1 Message Date
Sergei
5ecb5f9683 Fix Julia AI voice: use SINGLE_DEPLOYMENT_MODE for Lite
- livekitService.ts: send empty beneficiaryNamesDict in Lite mode
- agent.py: handle None beneficiary_names_dict correctly
- chat.tsx: align text chat with same SINGLE_DEPLOYMENT_MODE flag

This fixes Julia saying "I didn't get the name of beneficiary"
by letting WellNuo API use the default beneficiary for deployment_id.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-22 16:49:55 -08:00
Sergei
906213e620 Add beneficiary_names_dict support for voice assistant
- Voice agent now extracts deploymentId and beneficiaryNamesDict from
  participant metadata passed via LiveKit token
- WellNuoLLM class accepts dynamic deployment_id and beneficiary_names_dict
- API calls now include personalized beneficiary names for better responses
- Text chat already has this functionality (verified)
- Updated LiveKit agent deployed to cloud

Also includes:
- Speaker toggle button in voice call UI
- Keyboard controller integration for chat
- Various UI improvements
2026-01-20 14:41:33 -08:00
Sergei
e3192ead12 Voice call improvements: single call limit, hide debug tab, remove speaker toggle
Changes:
- Add CallManager singleton to ensure only 1 call per device at a time
- Hide Debug tab from production (href: null)
- Remove speaker/earpiece toggle button (always use speaker)
- Agent uses voice_ask API (fast ~1 sec latency)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-19 23:55:27 -08:00
Sergei
173c0a8262 Add session expired detection and auto-refresh in WebView
- Monitor page content for "session expired" patterns
- Send message to React Native when detected
- Auto-refresh token and reload WebView
- Add logging to refreshToken for debugging
2026-01-18 22:48:41 -08:00
Sergei
059bc29b6b WIP: LiveKit voice call integration with Julia AI agent
NOT TESTED ON REAL DEVICE - simulator only verification

Components:
- LiveKit Cloud agent deployment (julia-agent/julia-ai/)
- React Native LiveKit client (hooks/useLiveKitRoom.ts)
- Voice call screen with audio session management
- WellNuo voice_ask API integration in Python agent

Tech stack:
- LiveKit Cloud for agent hosting
- @livekit/react-native SDK
- Deepgram STT/TTS (via LiveKit Cloud)
- Silero VAD for voice activity detection

Known issues:
- Microphone permissions may need manual testing on real device
- LiveKit audio playback not verified on physical hardware
- Agent greeting audio not confirmed working end-to-end

Next steps:
- Test on physical iOS device
- Verify microphone capture works
- Confirm TTS audio playback
- Test full conversation loop
2026-01-18 20:16:25 -08:00
Sergei
dde0ecb9cd Add Julia AI voice agent with LiveKit integration
Voice AI Features:
- LiveKit Agents integration for real-time voice calls
- Julia AI agent (Python) deployed to LiveKit Cloud
- Token server for authentication
- Debug screen with voice call testing
- Voice call screen with full-screen UI

Agent Configuration:
- STT: Deepgram Nova-2
- LLM: OpenAI GPT-4o
- TTS: Deepgram Aura Asteria (female voice)
- Turn Detection: LiveKit Multilingual Model
- VAD: Silero
- Noise Cancellation: LiveKit BVC

Files added:
- julia-agent/ - Complete agent code and token server
- app/voice-call.tsx - Full-screen voice call UI
- services/livekitService.ts - LiveKit client service
- contexts/VoiceTranscriptContext.tsx - Transcript state
- polyfills/livekit-globals.ts - WebRTC polyfills

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-17 17:58:31 -08:00
Sergei
a2eb4e6882 Fix iOS audio and transcript streaming
- Add AVAudioSession configuration via @livekit/react-native
- Configure playAndRecord, defaultToSpeaker, voiceChat for iOS
- Fix transcript spam: update existing message until isFinal
- Remove unused Sherpa TTS service
- Add simulator build profile to eas.json
2026-01-16 13:56:29 -08:00
Sergei
c1380b55dd Add unified assistant with Ultravox voice AI
Chat screen now supports both:
- Text messaging (keyboard input)
- High-quality Ultravox voice calls (WebRTC)

Features:
- Voice call button in input bar (phone icon)
- Green status bar when call is active
- Transcripts from voice calls appear in chat history
- Voice badge on messages from voice conversation
- Mute button during calls
- Auto-end call when leaving screen

Background audio configured for iOS (audio, voip modes)
2026-01-16 12:20:17 -08:00
Sergei
da2c4bebc9 Integrate voice chat with TTS and speech recognition
App screens:
- chat.tsx: Voice-enabled chat with TTS responses
- debug.tsx: TTS debugging and testing screen
- index.tsx: Updated home with voice indicators
- _layout.tsx: Added TTS and error boundaries

Config:
- app.json: Microphone permissions for voice input
- package.json: Added Sherpa ONNX dependencies
- constants/theme.ts: Voice UI colors

Features:
- Voice input via speech recognition
- TTS voice output for chat responses
- Real-time voice activity indication
- Debug screen for TTS testing
- Error boundaries for stability

User experience:
- Hands-free chat interaction
- Visual feedback during voice processing
- Graceful error handling
2026-01-14 19:09:50 -08:00
Sergei
b2639dd540 Add Sherpa TTS voice synthesis system
Core TTS infrastructure:
- sherpaTTS.ts: Sherpa ONNX integration for offline TTS
- TTSErrorBoundary.tsx: Error boundary for TTS failures
- ErrorBoundary.tsx: Generic error boundary component
- VoiceIndicator.tsx: Visual indicator for voice activity
- useSpeechRecognition.ts: Speech-to-text hook
- DebugLogger.ts: Debug logging utility

Features:
- Offline voice synthesis (no internet needed)
- Multiple voices support
- Real-time voice activity indication
- Error recovery and fallback
- Debug logging for troubleshooting

Tech stack:
- Sherpa ONNX runtime
- React Native Audio
- Expo modules
2026-01-14 19:09:27 -08:00
Sergei
5fc7586f09 Revert Account Deletion feature - B2B model exception
Removed in-app account deletion implementation.

Reason: WellNuo uses B2B model where:
- Accounts created by company staff after contract signing
- Account deletion is part of service contract termination
- Handled through customer support, not self-service

Updated APPLE_REVIEW_RESPONSE.md with B2B exception explanation
for Apple Guidelines 5.1.1(v) and 3.1.1.
2026-01-12 12:01:23 -08:00
Sergei
7205c2ad1e Fix Apple Review issues: Login cleanup + Account Deletion
Apple App Store Review - Submission ID: 0992528e-4ce9-4167-9a1b-07f4334a8055

Fix #2: Login Screen Cleanup (Guideline 2.1)
- Removed non-functional "Create Account" button
- Removed "Forgot Password" button (not needed)
- Added actual logo image instead of text
- Streamlined login screen to focus on core functionality

Fix #3: Account Deletion Feature (Guideline 5.1.1(v))
- Added "Delete Account" button to Profile screen
- Implemented confirmation dialog with clear warning
- API integration: deleteAccount() method in api.ts
- Automatically logs out and clears data after deletion
- Redirects to login screen after successful deletion

Updated APPLE_REVIEW_RESPONSE.md with full documentation.
2026-01-12 11:55:31 -08:00
Sergei
8c0be34f65 Fix WebView session expiration with auto-refresh token
- Add refreshToken() method to automatically refresh expired tokens
- Add isTokenExpiringSoon() to check if token expires within 1 hour
- Store password in SecureStore for auto-refresh capability
- Add periodic token check every 30 minutes in WebView
- Intercept WebView navigation to login pages and refresh instead
- Re-inject fresh token into WebView localStorage after refresh
- Add metro.config.js to fix Metro bundler path resolution

This prevents the app from showing web login page when session expires.
2026-01-07 12:11:13 -08:00
Sergei
8bc9649146 WellNuo Lite v1.0.0 - simplified version for App Store review
- Removed voice input features
- Simplified profile page (only legal links and logout)
- Chat with AI context working
- Auto-select first beneficiary
- Dashboard WebView intact

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-24 17:13:13 -08:00