Changes:
- Add stopSession and isActive from VoiceContext to Profile
- Stop active voice session before saving new API type
- Prevents mid-session API type changes
Now when user switches between voice_ask and ask_wellnuo_ai in Profile settings, any active voice session will be stopped automatically.
Добавлено:
- Voice Debug tab - real-time логи STT/API/TTS/Timer
- iOS STT fix - отправка последнего partial как final при onEnd
- iOS auto-stop - автоматическая остановка STT после 2s тишины
- Voice API selector в Profile (voice_ask / ask_wellnuo_ai)
Исправлено:
- iOS никогда не отправлял isFinal:true - теперь отправляет через onEnd
- STT не останавливался после тишины - добавлен auto-stop таймер
- Profile Voice API selector восстановлен после rollback
Известные issues:
- TypeScript ошибки (setTimeout type) - не критично
- updateVoiceApiType отсутствует в VoiceContext - нужно добавить
Стабильная версия для тестирования на iPhone.
- FAB button now correctly stops session during speaking/processing states
- Echo prevention: STT stopped during TTS playback, results ignored during speaking
- Chat TTS only speaks when voice session is active (no auto-speak for text chat)
- Session stop now aborts in-flight API requests and prevents race conditions
- STT restarts after TTS with 800ms delay for audio focus release
- Pending interrupt transcript processed after TTS completion
- ChatContext added for message persistence across tab navigation
- VoiceFAB redesigned with state-based animations
- console.error replaced with console.warn across voice pipeline
- no-speech STT errors silenced (normal silence behavior)
- Enable STT listening during TTS playback to detect user interruption
- When voice detected during Julia's speech, immediately stop TTS
- Store interrupted transcript and process it after TTS stops
- Remove 'speaking' status check from STT watchdog to allow parallel STT+TTS
- Add pending transcript mechanism to handle race condition between
TTS stop and STT final result
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add watchdog mechanism to ensure STT continues when switching tabs:
- Monitor STT state every 500ms and restart if session active but STT stopped
- Handle AppState changes to restart STT when app returns to foreground
- Prevents session interruption due to native audio session changes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds logic to detect when voice status transitions from 'speaking' to
'listening' and automatically restarts the speech recognition. This
enables continuous voice conversation flow - after Julia speaks her
response, the app immediately starts listening for the next user input.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Integrate useTextToSpeech hook in chat screen
- Automatically speak assistant responses after API call
- Add volume icon button in header when TTS is active (tap to stop)
- Stop TTS when clearing chat history
- Use Russian language (ru-RU) for TTS output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When user speaks via voice mode, both their question and Julia's
response are now shown in the text chat. This provides a unified
conversation history for both voice and text interactions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add sessionActiveRef to track when voice session is active
- Add shouldRestartSTTRef to auto-restart STT after it ends
- STT now continues listening during TTS playback
- Voice detection callback checks both status and isSpeaking
- Final results during TTS are ignored (user interrupted)
- STT automatically restarts after ending if session is still active
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add onVoiceDetected callback to useSpeechRecognition hook
- Triggered on first interim result (voice activity detected)
- Uses voiceDetectedRef to ensure callback fires only once per session
- Reset flag on session start/end
- Connect STT to VoiceContext in _layout.tsx
- Use useSpeechRecognition with onVoiceDetected callback
- Call interruptIfSpeaking() when voice detected during 'speaking' state
- Forward STT results to VoiceContext (setTranscript, sendTranscript)
- Start/stop STT based on isListening state
- Export interruptIfSpeaking from VoiceContext provider
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- FAB now toggles between idle and listening states
- Green background (idle) → Red background (listening)
- Icon switches between mic-outline and mic
- Connects to VoiceContext for state management
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add floating action button for voice calls to the tab bar layout:
- Import and render VoiceFAB component
- Add placeholder handler for voice call initiation
- Wrap Tabs in View to properly position FAB overlay
- FAB automatically hides when a call is active (via VoiceCallContext)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This LiveKit hook was no longer used after switching to speech recognition.
Also removed the outdated comment referencing it in _layout.tsx.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Configure LiveKit Expo plugin with audioType: "media" in app.json
This forces speaker output on Android instead of earpiece
- Remove microphone icon from voice messages in chat
- Remove audio output picker button (no longer needed)
- Clean up audioSession.ts configuration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add Audio button during active calls to switch output
- Fallback to Speaker/Earpiece options when LiveKit API unavailable
- Speaker now works correctly on Android
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Android Audio:
- Use inCommunication mode with forceHandleAudioRouting
- Explicit selectAudioOutput('speaker') after session start
- Keeps echo cancellation while forcing speaker output
Profile Modal:
- Add KeyboardAvoidingView for deployment ID input
- Prevents modal buttons from being hidden by keyboard
Co-Authored-By: Claude <noreply@anthropic.com>
- Add deploymentName state to chat screen
- Load and display beneficiary name in initial welcome message
- Save deployment name to SecureStore when validating in profile
- End call and clear chat when deployment changes
- Fix text input not clearing after sending message
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Chat improvements:
- Add pulsing animation to voice call button during active call
- Log call start/end with duration to chat history
- End call automatically when deployment ID changes
- Reduce bottom padding (removed SafeArea bottom edge)
Julia Agent:
- Disable user interruption (min_interruption_duration=999)
- Agent now speaks without being interrupted
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add SafeAreaProvider wrapper in root layout for reliable insets on Android
- Add minimum 16px bottom padding on Android to prevent overlap with
gesture navigation or software buttons (Samsung, Pixel, etc.)
- Keep 10px minimum for other platforms
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add scrollToLatestMessage helper that respects sortNewestFirst setting:
- When newest first: scroll to top (offset 0)
- When oldest first: scroll to end
Applied to keyboard show, voice messages, and content size changes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add validateDeploymentId() method in api.ts that checks if ID exists
in user's deployments list
- Update profile.tsx to validate deployment ID before saving
- Show validation error message if ID is invalid
- Display deployment name alongside ID after validation
- Add loading state during validation
- Add deploymentId storage methods in api.ts (set/get/clear)
- Add Settings section in Profile with Deployment ID menu item
- Add modal dialog to edit deployment ID
- Update chat.tsx to use custom deployment ID from settings
- Priority: custom > currentBeneficiary > first beneficiary > fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add a sort button in the chat header that toggles between oldest-first
(default) and newest-first message ordering. The arrow icon indicates
the current sort direction.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add trash icon button in chat header that clears all messages
after confirmation. Resets chat to initial welcome message.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Voice transcripts from both user and Julia now appear in the chat
immediately during the call, not just after it ends. Each voice call
shows a "Voice Call" separator, and voice messages display with a
microphone icon indicator.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Create VoiceCallContext for global voice call state management
- Add FloatingCallBubble component with drag support
- Add minimize button to voice call screen
- Show bubble when call is minimized, tap to return to call
- Button shows active call state with green color
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Use useSafeAreaInsets() to dynamically calculate tab bar height and bottom
padding instead of hardcoded values. This ensures the tab bar properly
accounts for Android navigation buttons (Samsung) and iOS home indicator.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add voice call button in chat input area
- Implement LiveKit room connection with Julia AI agent
- Create full-screen voice call modal with:
- Visual avatar with speaking indicator
- Call duration timer
- Agent state display (listening/thinking/speaking)
- Hang up button
- Add real-time transcription tracking for voice calls
- Keep screen awake during active calls
- Integrate with existing VoiceTranscriptContext for history
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Deleted app/voice-call.tsx fullscreen modal and removed all navigation
references to it from chat.tsx, voice.tsx and _layout.tsx.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add debugDeploymentId to BeneficiaryContext for sharing between screens
- Sync Debug tab's deploymentId state with global context
- voice-call.tsx now prioritizes debugDeploymentId when starting calls
- Enables testing voice calls with specific deployment IDs from Debug screen
- livekitService.ts: send empty beneficiaryNamesDict in Lite mode
- agent.py: handle None beneficiary_names_dict correctly
- chat.tsx: align text chat with same SINGLE_DEPLOYMENT_MODE flag
This fixes Julia saying "I didn't get the name of beneficiary"
by letting WellNuo API use the default beneficiary for deployment_id.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix race condition where connect() was called before beneficiaryData loaded
- Add connectCalledRef to prevent duplicate connect calls
- Wait for beneficiaryData.deploymentId before initiating call
- Add 5s timeout fallback for edge cases (API failure/no beneficiaries)
- Hide Debug tab, show only Julia tab in navigation
- Add Android keyboard layout fix for password fields
- Bump version to 1.0.5
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Voice agent now extracts deploymentId and beneficiaryNamesDict from
participant metadata passed via LiveKit token
- WellNuoLLM class accepts dynamic deployment_id and beneficiary_names_dict
- API calls now include personalized beneficiary names for better responses
- Text chat already has this functionality (verified)
- Updated LiveKit agent deployed to cloud
Also includes:
- Speaker toggle button in voice call UI
- Keyboard controller integration for chat
- Various UI improvements
- Remove speaker button empty space (2-button centered layout)
- Remove "Asteria voice" text from voice call screen
- Fix chat input visibility with keyboard
- Add keyboard show listener for auto-scroll
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Add CallManager singleton to ensure only 1 call per device at a time
- Hide Debug tab from production (href: null)
- Remove speaker/earpiece toggle button (always use speaker)
- Agent uses voice_ask API (fast ~1 sec latency)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Changed audioStreamType from 'voiceCall' to 'music' on Android
- voiceCall stream defaults to earpiece
- music stream defaults to speaker
- Added Debug tab to test voice calls with detailed logs
- Added speaker/earpiece toggle button with proper stream switching
- Full Android AudioSession support for LiveKit voice calls
audioSession.ts:
- configureAudioForVoiceCall: uses music/media for speaker output
- setAudioOutput: switches between music (speaker) and voiceCall (earpiece)
- reconfigureAudioForPlayback: ensures speaker output on Android
debug.tsx:
- Added platform info display
- Added speaker toggle with logging
- Improved UI with control rows
As requested - same parameters, same response format.
Changed in both:
- chat.tsx (text chat)
- agent.py (voice agent)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Use anandk credentials (same as julia-agent)
- Add normalizeQuestion() function to transform questions
into format WellNuo API understands
- "how is my dad" → "how is dad doing"
- Remove user's SecureStore credentials
- Use cached token with auto-refresh on 401
This makes text chat return real Ferdinand sensor data
just like voice calls do.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Change clientId from '001' to 'MA_001'
- Change deployment_id to '21' (Ferdinand)
- Send raw question without context wrapping
- Same params as julia-agent/julia-ai/src/agent.py
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Monitor page content for "session expired" patterns
- Send message to React Native when detected
- Auto-refresh token and reload WebView
- Add logging to refreshToken for debugging
- Enable Chat tab (replace Debug) - text chat with Julia AI
- Add voice call button in chat header and input area
- Add speaker/earpiece toggle in voice-call screen
- setAudioOutput() function for switching audio output
NOT TESTED ON REAL DEVICE - simulator only verification
Components:
- LiveKit Cloud agent deployment (julia-agent/julia-ai/)
- React Native LiveKit client (hooks/useLiveKitRoom.ts)
- Voice call screen with audio session management
- WellNuo voice_ask API integration in Python agent
Tech stack:
- LiveKit Cloud for agent hosting
- @livekit/react-native SDK
- Deepgram STT/TTS (via LiveKit Cloud)
- Silero VAD for voice activity detection
Known issues:
- Microphone permissions may need manual testing on real device
- LiveKit audio playback not verified on physical hardware
- Agent greeting audio not confirmed working end-to-end
Next steps:
- Test on physical iOS device
- Verify microphone capture works
- Confirm TTS audio playback
- Test full conversation loop
