Changes:
- Updated livekitService.ts to use remote token server
- Julia-robust agent with disabled interruptions
- Added discard_audio_if_uninterruptible=True
- Added min_interruption_duration=2.0
- Token server configured for julia-robust agent
- livekitService.ts: send empty beneficiaryNamesDict in Lite mode
- agent.py: handle None beneficiary_names_dict correctly
- chat.tsx: align text chat with same SINGLE_DEPLOYMENT_MODE flag
This fixes Julia saying "I didn't get the name of beneficiary"
by letting WellNuo API use the default beneficiary for deployment_id.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The agent was reading participant metadata immediately after connecting,
but the mobile user might not have joined yet or metadata wasn't synced.
Changes:
- Replace extract_beneficiary_data() with wait_for_participant_with_metadata()
- Wait up to 10 seconds for participant with deploymentId
- Log waiting status every 2 seconds for debugging
- Fallback gracefully if timeout reached
This fixes "I couldn't get that information right now" error in voice calls.
- Voice agent now extracts deploymentId and beneficiaryNamesDict from
participant metadata passed via LiveKit token
- WellNuoLLM class accepts dynamic deployment_id and beneficiary_names_dict
- API calls now include personalized beneficiary names for better responses
- Text chat already has this functionality (verified)
- Updated LiveKit agent deployed to cloud
Also includes:
- Speaker toggle button in voice call UI
- Keyboard controller integration for chat
- Various UI improvements
- Remove speaker button empty space (2-button centered layout)
- Remove "Asteria voice" text from voice call screen
- Fix chat input visibility with keyboard
- Add keyboard show listener for auto-scroll
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Add CallManager singleton to ensure only 1 call per device at a time
- Hide Debug tab from production (href: null)
- Remove speaker/earpiece toggle button (always use speaker)
- Agent uses voice_ask API (fast ~1 sec latency)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
As requested - same parameters, same response format.
Changed in both:
- chat.tsx (text chat)
- agent.py (voice agent)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
WellNuo API only returns real sensor data for exact phrase
"how is dad doing". This function maps various user questions
(how is Ferdinand, how's dad, is he okay, etc.) to that phrase.
- Added keyword detection for status, subject, sleep, activity
- All variations now return Ferdinand's real data
This enables WellNuo voice_ask API to return real sensor data
about Ferdinand (deployment_id=21) instead of generic responses.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- chat() must be synchronous, returning LLMStream directly
- LLMStream.__init__() takes 4 params: llm, chat_ctx, tools, conn_options
- _run() emits chunks via self._event_ch.send_nowait()
- Added model and provider properties required by LLM base class
Tested: STT (Deepgram Nova-2), TTS (Deepgram Aura Asteria), VAD (Silero)
all working. WellNuo voice_ask API integration confirmed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
NOT TESTED ON REAL DEVICE - simulator only verification
Components:
- LiveKit Cloud agent deployment (julia-agent/julia-ai/)
- React Native LiveKit client (hooks/useLiveKitRoom.ts)
- Voice call screen with audio session management
- WellNuo voice_ask API integration in Python agent
Tech stack:
- LiveKit Cloud for agent hosting
- @livekit/react-native SDK
- Deepgram STT/TTS (via LiveKit Cloud)
- Silero VAD for voice activity detection
Known issues:
- Microphone permissions may need manual testing on real device
- LiveKit audio playback not verified on physical hardware
- Agent greeting audio not confirmed working end-to-end
Next steps:
- Test on physical iOS device
- Verify microphone capture works
- Confirm TTS audio playback
- Test full conversation loop
