- Add Audio button during active calls to switch output
- Fallback to Speaker/Earpiece options when LiveKit API unavailable
- Speaker now works correctly on Android
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add AudioOutputDevice interface with id, name, type fields
- Add getAvailableAudioOutputs() to list available audio devices
- Add selectAudioOutput(deviceId) to switch to specific device
- Add mapDeviceType() helper for device type normalization
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Root cause: Audio from remote participant (Julia AI) was not playing
because room.startAudio() was never called after connecting.
This is REQUIRED by LiveKit WebRTC to enable audio playback.
The fix matches the working implementation in debug.tsx (Robert version).
Changes:
- Add room.startAudio() call after room.connect()
- Add canPlayAudio state tracking
- Add proper error handling for startAudio
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- audioStreamType: music (routes to SPEAKER by default)
- audioMode: normal (not inCommunication which uses earpiece)
- audioAttributesUsageType: media
- audioAttributesContentType: music
Previous voiceCall stream was routing to earpiece on Android devices.
Android Audio:
- Use inCommunication mode with forceHandleAudioRouting
- Explicit selectAudioOutput('speaker') after session start
- Keeps echo cancellation while forcing speaker output
Profile Modal:
- Add KeyboardAvoidingView for deployment ID input
- Prevents modal buttons from being hidden by keyboard
Co-Authored-By: Claude <noreply@anthropic.com>
- Add deploymentName state to chat screen
- Load and display beneficiary name in initial welcome message
- Save deployment name to SecureStore when validating in profile
- End call and clear chat when deployment changes
- Fix text input not clearing after sending message
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Chat improvements:
- Add pulsing animation to voice call button during active call
- Log call start/end with duration to chat history
- End call automatically when deployment ID changes
- Reduce bottom padding (removed SafeArea bottom edge)
Julia Agent:
- Disable user interruption (min_interruption_duration=999)
- Agent now speaks without being interrupted
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add SafeAreaProvider wrapper in root layout for reliable insets on Android
- Add minimum 16px bottom padding on Android to prevent overlap with
gesture navigation or software buttons (Samsung, Pixel, etc.)
- Keep 10px minimum for other platforms
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add scrollToLatestMessage helper that respects sortNewestFirst setting:
- When newest first: scroll to top (offset 0)
- When oldest first: scroll to end
Applied to keyboard show, voice messages, and content size changes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Account for tab bar height (60px) when calculating maximum Y position
for the draggable bubble. Previously the bubble could be dragged over
the tab bar navigation, now it stays above it.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Button.tsx: Replace `condition && style` with ternary operators
to ensure array elements are always ViewStyle/TextStyle (not false)
- ultravoxService.ts: Add 'in' type guard before accessing optional
'location' property on alert objects
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add validateDeploymentId() method in api.ts that checks if ID exists
in user's deployments list
- Update profile.tsx to validate deployment ID before saving
- Show validation error message if ID is invalid
- Display deployment name alongside ID after validation
- Add loading state during validation
- Add deploymentId storage methods in api.ts (set/get/clear)
- Add Settings section in Profile with Deployment ID menu item
- Add modal dialog to edit deployment ID
- Update chat.tsx to use custom deployment ID from settings
- Priority: custom > currentBeneficiary > first beneficiary > fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add a sort button in the chat header that toggles between oldest-first
(default) and newest-first message ordering. The arrow icon indicates
the current sort direction.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add trash icon button in chat header that clears all messages
after confirmation. Resets chat to initial welcome message.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Voice transcripts from both user and Julia now appear in the chat
immediately during the call, not just after it ends. Each voice call
shows a "Voice Call" separator, and voice messages display with a
microphone icon indicator.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changed tap behavior on the floating call bubble to end the call
instead of maximizing it. Removed the separate small end call button
since it's no longer needed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Create VoiceCallContext for global voice call state management
- Add FloatingCallBubble component with drag support
- Add minimize button to voice call screen
- Show bubble when call is minimized, tap to return to call
- Button shows active call state with green color
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Use useSafeAreaInsets() to dynamically calculate tab bar height and bottom
padding instead of hardcoded values. This ensures the tab bar properly
accounts for Android navigation buttons (Samsung) and iOS home indicator.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add voice call button in chat input area
- Implement LiveKit room connection with Julia AI agent
- Create full-screen voice call modal with:
- Visual avatar with speaking indicator
- Call duration timer
- Agent state display (listening/thinking/speaking)
- Hang up button
- Add real-time transcription tracking for voice calls
- Keep screen awake during active calls
- Integrate with existing VoiceTranscriptContext for history
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Deleted app/voice-call.tsx fullscreen modal and removed all navigation
references to it from chat.tsx, voice.tsx and _layout.tsx.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add debugDeploymentId to BeneficiaryContext for sharing between screens
- Sync Debug tab's deploymentId state with global context
- voice-call.tsx now prioritizes debugDeploymentId when starting calls
- Enables testing voice calls with specific deployment IDs from Debug screen
- livekitService.ts: send empty beneficiaryNamesDict in Lite mode
- agent.py: handle None beneficiary_names_dict correctly
- chat.tsx: align text chat with same SINGLE_DEPLOYMENT_MODE flag
This fixes Julia saying "I didn't get the name of beneficiary"
by letting WellNuo API use the default beneficiary for deployment_id.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The package was removed in commit d9fff44 but the plugin
entry was left in app.json, causing EAS Build to fail with
'Unknown error in Read app config' phase.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
If primary audio config fails (OSStatus -50), automatically try:
1. videoChat mode (speaker default)
2. voiceChat mode (earpiece default)
3. minimal config (most compatible)
Also make speaker output setting non-critical - call will work
even if output can't be changed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove expo-speech (TTS) - not used
- Remove expo-speech-recognition (STT) - not used
- Delete dead code: hooks/useSpeechRecognition.ts
These packages add native audio modules that can conflict with
LiveKit's AudioSession management on iOS.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove 'defaultToSpeaker' from audioCategoryOptions (conflicts with some iOS versions)
- Use 'videoChat' mode instead of 'voiceChat' for speaker output
- Reorder operations: set config first, start session, then configure output
- Keep audioCategoryOptions minimal to avoid param conflicts
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The agent was reading participant metadata immediately after connecting,
but the mobile user might not have joined yet or metadata wasn't synced.
Changes:
- Replace extract_beneficiary_data() with wait_for_participant_with_metadata()
- Wait up to 10 seconds for participant with deploymentId
- Log waiting status every 2 seconds for debugging
- Fallback gracefully if timeout reached
This fixes "I couldn't get that information right now" error in voice calls.
- Fix race condition where connect() was called before beneficiaryData loaded
- Add connectCalledRef to prevent duplicate connect calls
- Wait for beneficiaryData.deploymentId before initiating call
- Add 5s timeout fallback for edge cases (API failure/no beneficiaries)
- Hide Debug tab, show only Julia tab in navigation
- Add Android keyboard layout fix for password fields
- Bump version to 1.0.5
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Voice agent now extracts deploymentId and beneficiaryNamesDict from
participant metadata passed via LiveKit token
- WellNuoLLM class accepts dynamic deployment_id and beneficiary_names_dict
- API calls now include personalized beneficiary names for better responses
- Text chat already has this functionality (verified)
- Updated LiveKit agent deployed to cloud
Also includes:
- Speaker toggle button in voice call UI
- Keyboard controller integration for chat
- Various UI improvements
- Remove speaker button empty space (2-button centered layout)
- Remove "Asteria voice" text from voice call screen
- Fix chat input visibility with keyboard
- Add keyboard show listener for auto-scroll
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Remove allowBluetoothA2DP from audioCategoryOptions. This option is
incompatible with playAndRecord category on some iOS versions.
The allowBluetooth (HFP profile) is sufficient for voice calls.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Add CallManager singleton to ensure only 1 call per device at a time
- Hide Debug tab from production (href: null)
- Remove speaker/earpiece toggle button (always use speaker)
- Agent uses voice_ask API (fast ~1 sec latency)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Changed audioStreamType from 'voiceCall' to 'music' on Android
- voiceCall stream defaults to earpiece
- music stream defaults to speaker
- Added Debug tab to test voice calls with detailed logs
- Added speaker/earpiece toggle button with proper stream switching
- Full Android AudioSession support for LiveKit voice calls
audioSession.ts:
- configureAudioForVoiceCall: uses music/media for speaker output
- setAudioOutput: switches between music (speaker) and voiceCall (earpiece)
- reconfigureAudioForPlayback: ensures speaker output on Android
debug.tsx:
- Added platform info display
- Added speaker toggle with logging
- Improved UI with control rows
As requested - same parameters, same response format.
Changed in both:
- chat.tsx (text chat)
- agent.py (voice agent)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Use anandk credentials (same as julia-agent)
- Add normalizeQuestion() function to transform questions
into format WellNuo API understands
- "how is my dad" → "how is dad doing"
- Remove user's SecureStore credentials
- Use cached token with auto-refresh on 401
This makes text chat return real Ferdinand sensor data
just like voice calls do.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Change clientId from '001' to 'MA_001'
- Change deployment_id to '21' (Ferdinand)
- Send raw question without context wrapping
- Same params as julia-agent/julia-ai/src/agent.py
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Monitor page content for "session expired" patterns
- Send message to React Native when detected
- Auto-refresh token and reload WebView
- Add logging to refreshToken for debugging
- Enable Chat tab (replace Debug) - text chat with Julia AI
- Add voice call button in chat header and input area
- Add speaker/earpiece toggle in voice-call screen
- setAudioOutput() function for switching audio output
WellNuo API only returns real sensor data for exact phrase
"how is dad doing". This function maps various user questions
(how is Ferdinand, how's dad, is he okay, etc.) to that phrase.
- Added keyword detection for status, subject, sleep, activity
- All variations now return Ferdinand's real data
This enables WellNuo voice_ask API to return real sensor data
about Ferdinand (deployment_id=21) instead of generic responses.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- chat() must be synchronous, returning LLMStream directly
- LLMStream.__init__() takes 4 params: llm, chat_ctx, tools, conn_options
- _run() emits chunks via self._event_ch.send_nowait()
- Added model and provider properties required by LLM base class
Tested: STT (Deepgram Nova-2), TTS (Deepgram Aura Asteria), VAD (Silero)
all working. WellNuo voice_ask API integration confirmed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
NOT TESTED ON REAL DEVICE - simulator only verification
Components:
- LiveKit Cloud agent deployment (julia-agent/julia-ai/)
- React Native LiveKit client (hooks/useLiveKitRoom.ts)
- Voice call screen with audio session management
- WellNuo voice_ask API integration in Python agent
Tech stack:
- LiveKit Cloud for agent hosting
- @livekit/react-native SDK
- Deepgram STT/TTS (via LiveKit Cloud)
- Silero VAD for voice activity detection
Known issues:
- Microphone permissions may need manual testing on real device
- LiveKit audio playback not verified on physical hardware
- Agent greeting audio not confirmed working end-to-end
Next steps:
- Test on physical iOS device
- Verify microphone capture works
- Confirm TTS audio playback
- Test full conversation loop
