- Add App Store build credentials and build-ios.sh script - Document Julia AI voice agent architecture - Add detailed instructions for editing agent behavior - Document token server deployment
273 lines
6.1 KiB
Markdown
273 lines
6.1 KiB
Markdown
# WellNuo Lite
|
|
|
|
Voice AI companion for elderly care monitoring.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
npm install
|
|
expo-sim 8081 # Запуск на симуляторе
|
|
```
|
|
|
|
---
|
|
|
|
## App Store Build & Submit
|
|
|
|
### Credentials
|
|
|
|
| Parameter | Value |
|
|
|-----------|-------|
|
|
| Bundle ID | `com.wellnuo.BluetoothScanner` |
|
|
| Apple Team ID | `UHLZD54ULZ` |
|
|
| ASC App ID | `6756594148` |
|
|
| ASC API Key ID | `GA9C2GRPHS` |
|
|
| ASC Issuer ID | `dcac5647-0710-4764-affd-2d3270bf49d4` |
|
|
| API Key File | `AuthKey_GA9C2GRPHS.p8` |
|
|
|
|
### Build iOS for TestFlight
|
|
|
|
```bash
|
|
./build-ios.sh production
|
|
```
|
|
|
|
Или вручную:
|
|
|
|
```bash
|
|
export EXPO_APPLE_TEAM_ID="UHLZD54ULZ"
|
|
export EXPO_APPLE_TEAM_TYPE="INDIVIDUAL"
|
|
export EXPO_ASC_KEY_ID="GA9C2GRPHS"
|
|
export EXPO_ASC_ISSUER_ID="dcac5647-0710-4764-affd-2d3270bf49d4"
|
|
export EXPO_ASC_API_KEY_PATH="$(pwd)/AuthKey_GA9C2GRPHS.p8"
|
|
|
|
npx eas-cli build --platform ios --profile production
|
|
```
|
|
|
|
### Submit to TestFlight
|
|
|
|
После завершения сборки:
|
|
|
|
```bash
|
|
eas submit --platform ios --latest
|
|
```
|
|
|
|
### Check Build Status
|
|
|
|
```bash
|
|
eas build:list --platform ios --limit 5
|
|
```
|
|
|
|
---
|
|
|
|
## Julia AI Voice Agent
|
|
|
|
### Architecture
|
|
|
|
```
|
|
Mobile App (LiveKit SDK)
|
|
↓
|
|
LiveKit Cloud
|
|
↓
|
|
Julia AI Agent (CA_zKZCpX36ZT5C)
|
|
├── STT: Deepgram Nova-2
|
|
├── LLM: OpenAI GPT-4o
|
|
└── TTS: Deepgram Aura Asteria
|
|
```
|
|
|
|
### Quick Reference
|
|
|
|
| Item | Value |
|
|
|------|-------|
|
|
| Agent ID | `CA_zKZCpX36ZT5C` |
|
|
| Agent Name | `julia-ai` |
|
|
| LiveKit Cloud | `live-kit-demo-70txlh6a.livekit.cloud` |
|
|
| Token Server URL | `https://wellnuo.smartlaunchhub.com/julia/token` |
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
julia-agent/
|
|
├── julia-ai/ # LiveKit Agent (Python)
|
|
│ ├── src/agent.py # Main agent code
|
|
│ ├── livekit.toml # Agent deployment config
|
|
│ ├── .env.local # API keys (OPENAI, DEEPGRAM)
|
|
│ ├── pyproject.toml # Python dependencies
|
|
│ └── Dockerfile # Deployment container
|
|
└── token-server/ # Token Server (Node.js)
|
|
├── server.js # Express server
|
|
└── .env # LiveKit credentials
|
|
```
|
|
|
|
### How It Works
|
|
|
|
1. **Mobile app** requests token from Token Server
|
|
2. **Token Server** generates JWT with room + agent config
|
|
3. **Mobile app** connects to LiveKit Cloud with token
|
|
4. **LiveKit Cloud** spawns Julia AI agent in the room
|
|
5. **Agent** handles voice conversation using STT → LLM → TTS pipeline
|
|
|
|
### Editing Agent Behavior
|
|
|
|
Agent configuration is in `julia-agent/julia-ai/src/agent.py`:
|
|
|
|
**Change AI personality (system prompt):**
|
|
```python
|
|
# In build_system_prompt() function - line 77
|
|
return f"""You are Julia, a compassionate AI care assistant...
|
|
```
|
|
|
|
**Change voice model:**
|
|
```python
|
|
# Line 186 - TTS model
|
|
tts=deepgram.TTS(model="aura-asteria-en"), # Female voice
|
|
# Options: aura-asteria-en, aura-luna-en, aura-stella-en
|
|
```
|
|
|
|
**Change LLM model:**
|
|
```python
|
|
# Line 181-184 - LLM model
|
|
llm=openai.LLM(
|
|
model="gpt-4o", # or "gpt-4o-mini" for faster/cheaper
|
|
api_key=os.getenv("OPENAI_API_KEY"),
|
|
),
|
|
```
|
|
|
|
**Change STT model:**
|
|
```python
|
|
# Line 179 - Speech-to-text
|
|
stt=deepgram.STT(model="nova-2"), # Most accurate
|
|
```
|
|
|
|
**Change initial greeting:**
|
|
```python
|
|
# Line 212-214 - First message when user connects
|
|
await session.generate_reply(
|
|
instructions="Greet the user warmly as Julia..."
|
|
)
|
|
```
|
|
|
|
### Deploy Agent Changes
|
|
|
|
```bash
|
|
cd julia-agent/julia-ai
|
|
|
|
# Install LiveKit CLI (if not installed)
|
|
brew install livekit/tap/lk
|
|
|
|
# Deploy to LiveKit Cloud
|
|
lk agent deploy
|
|
```
|
|
|
|
The agent will be rebuilt and deployed automatically. Takes ~2-3 minutes.
|
|
|
|
### Token Server
|
|
|
|
Generates LiveKit tokens for mobile app connections.
|
|
|
|
**Location:** `root@91.98.205.156:/var/www/julia-token-server/`
|
|
**PM2 process:** `julia-token-server`
|
|
**URL:** `https://wellnuo.smartlaunchhub.com/julia/token`
|
|
|
|
**API:**
|
|
```bash
|
|
POST /token
|
|
Body: { "userId": "user-123" }
|
|
Response: { "token": "...", "roomName": "...", "wsUrl": "wss://..." }
|
|
```
|
|
|
|
**Update token server:**
|
|
```bash
|
|
# Copy changes to server
|
|
scp julia-agent/token-server/server.js root@91.98.205.156:/var/www/julia-token-server/
|
|
|
|
# Restart
|
|
ssh root@91.98.205.156 "pm2 restart julia-token-server"
|
|
```
|
|
|
|
### Environment Variables
|
|
|
|
**Agent (.env.local):**
|
|
```env
|
|
OPENAI_API_KEY=sk-...
|
|
DEEPGRAM_API_KEY=...
|
|
LIVEKIT_URL=wss://live-kit-demo-70txlh6a.livekit.cloud
|
|
LIVEKIT_API_KEY=...
|
|
LIVEKIT_API_SECRET=...
|
|
```
|
|
|
|
**Token Server (.env):**
|
|
```env
|
|
LIVEKIT_API_KEY=APIEivUcPW3WSrV
|
|
LIVEKIT_API_SECRET=...
|
|
PORT=3001
|
|
```
|
|
|
|
### Testing Locally
|
|
|
|
```bash
|
|
cd julia-agent/julia-ai
|
|
|
|
# Install dependencies
|
|
uv sync
|
|
|
|
# Download required models (first time)
|
|
uv run python src/agent.py download-files
|
|
|
|
# Test in console
|
|
uv run python src/agent.py console
|
|
|
|
# Run for frontend connection
|
|
uv run python src/agent.py dev
|
|
```
|
|
|
|
### LiveKit Dashboard
|
|
|
|
Monitor agent sessions and logs: https://cloud.livekit.io
|
|
|
|
---
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
WellNuoLite/
|
|
├── app/ # Expo Router screens
|
|
│ ├── (auth)/ # Auth flow screens
|
|
│ └── (tabs)/ # Main app tabs
|
|
├── components/ # Reusable components
|
|
├── services/ # API, navigation
|
|
├── contexts/ # React contexts
|
|
├── julia-agent/ # LiveKit AI agent
|
|
│ └── julia-ai/
|
|
│ └── src/agent.py # Main agent code
|
|
├── AuthKey_GA9C2GRPHS.p8 # ASC API Key
|
|
├── build-ios.sh # iOS build script
|
|
├── eas.json # EAS configuration
|
|
└── app.json # Expo configuration
|
|
```
|
|
|
|
---
|
|
|
|
## Links
|
|
|
|
| Resource | URL |
|
|
|----------|-----|
|
|
| App Store Connect | https://appstoreconnect.apple.com |
|
|
| EAS Dashboard | https://expo.dev/accounts/serter20692/projects/WellNuo |
|
|
| LiveKit Dashboard | https://cloud.livekit.io |
|
|
| Privacy Policy | https://wellnuo.smartlaunchhub.com/privacy.html |
|
|
| Support | https://wellnuo.smartlaunchhub.com/support.html |
|
|
|
|
---
|
|
|
|
## Full App Store Instructions
|
|
|
|
See: `/Users/sergei/Desktop/WellNuo/INSTRUCTIONS_APP_STORE.md`
|
|
|
|
Contains:
|
|
- Screenshots requirements
|
|
- App Store texts (copy-paste ready)
|
|
- Privacy labels
|
|
- Review notes
|
|
- Checklist before submission
|