Archived
Compare commits
13
Commits
main
..
59d8182b6c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
59d8182b6c | ||
|
|
b7cec80a99 | ||
|
|
5577162594 | ||
|
|
f6722a223f | ||
|
|
afe37ad5f4 | ||
|
|
f043d6b078 | ||
|
|
fa21c81787 | ||
|
|
c76afbd11e | ||
|
|
ea9ec75631 | ||
|
|
c5db94658d | ||
|
|
ed329213de | ||
|
|
3620a2387b | ||
|
|
24f95639dc |
No files matched your search
@@ -1,3 +1,5 @@
|
||||
# AGENTS.md
|
||||
|
||||
- On startup, read `docs/PROJECT_STATUS.md` before planning or changing this project. It is the current handoff report and records decisions, completed work, verification, deployment dependencies, and remaining tasks.
|
||||
- Keep `docs/PROJECT_STATUS.md` updated as work progresses, especially after implementation, testing, deployment, or changes to blockers and user decisions. Distinguish verified results from assumptions, and update it before handing off or committing material changes.
|
||||
- If using XcodeBuildMCP, use the installed XcodeBuildMCP skill before calling XcodeBuildMCP tools.
|
||||
@@ -1,19 +1,19 @@
|
||||
# Voice — iOS client for Hope Voice
|
||||
|
||||
Native SwiftUI iOS app (iOS 17+) that talks to the [Hope Voice](https://github.com/) agent:
|
||||
voice/text conversation, transcript history, voice & model pickers, media player, and a
|
||||
background-tasks panel. Mirrors the core web UI of `hope-voice-api`; no server-side changes.
|
||||
voice/text conversation, transcript history, voice & model pickers, and a background-tasks
|
||||
panel. Uses `hope-voice-api`.
|
||||
|
||||
> **Status:** Core functionality is implemented and verified in simulator to the extent the
|
||||
> environment allows. Full device qualification (mic capture, audible media playback,
|
||||
> interruption behavior) and the Hope visual redesign (`ui-ux-spec.md`) remain pending —
|
||||
> environment allows. Full device qualification (mic capture, interruption behavior)
|
||||
> remains pending. The native Talk/Activity redesign is implemented —
|
||||
> see [PLAN.md](PLAN.md) and [docs/DEVICE_TESTING.md](docs/DEVICE_TESTING.md).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Xcode 16+ (Swift 6), iOS 17 simulator or a real iPhone on the LAN.
|
||||
- [xcodegen](https://github.com/yonaskolb/XcodeGen) (`brew install xcodegen`).
|
||||
- The `voice` container running on Unraid (LiveKit + token server + media API).
|
||||
- The `voice` container running on Unraid (LiveKit + token server).
|
||||
- Mic: the simulator mic is unreliable in this environment (CoreAudio `-4010`, 0 input
|
||||
channels) — voice mode needs a real device. If the simulator was wiped, grant permission
|
||||
first: `xcrun simctl privacy booted grant microphone dev.shane.voice`.
|
||||
@@ -33,52 +33,47 @@ To run in the simulator, use Xcode or `xcrun simctl` with the built app.
|
||||
|
||||
Default base URL is `https://192.168.86.2:9178` (see `Sources/App/SettingsStore.swift`).
|
||||
Change it at runtime in **Settings → Server URL** (the app disconnects and recreates the
|
||||
session when idle). The app derives LiveKit signaling and media endpoints from this base URL.
|
||||
session when idle). The app derives the LiveKit signaling endpoint from this base URL.
|
||||
|
||||
### TLS cert pinning
|
||||
|
||||
The deployment uses a self-signed cert that is **regenerated at container start**, so the
|
||||
pinned SHA-256 in `Sources/Networking/TLSPinning.swift` breaks on every cert rotation.
|
||||
Re-export after a server restart:
|
||||
The deployment uses a persistent self-signed cert. The entrypoint regenerates it if
|
||||
missing or issued for a different IP. The pinned SHA-256 in
|
||||
`Sources/Networking/TLSPinning.swift` must match after any actual cert rotation:
|
||||
|
||||
```sh
|
||||
openssl s_client -connect 192.168.86.2:9178 </dev/null 2>/dev/null \
|
||||
| openssl x509 -outform der | openssl dgst -sha256
|
||||
```
|
||||
|
||||
Paste the new hash into `TLSPinning.pinnedCertSHA256Hex`. Until you do, token/media calls
|
||||
Paste the new hash into `TLSPinning.pinnedCertSHA256Hex`. Until you do, token calls
|
||||
fail with a pin mismatch (not a network error).
|
||||
|
||||
## Features
|
||||
|
||||
- **Conversation** — LiveKit room with the Hope agent; voice mode (mic + TTS) and text mode
|
||||
(composer), toggleable and persisted. State machine (idle/connecting/listening/thinking/
|
||||
speaking) driven by audio levels, orb animation, mic meter, barge-in hint.
|
||||
speaking) driven by audio levels, with a compact status indicator and mic meter.
|
||||
- **Transcripts** — partials updated in place, finals with timestamps, delivery-marker
|
||||
stripping, last-50 history persisted to UserDefaults, clear button.
|
||||
- **Voice & model pickers** — 13 Azure TTS voices; LLM models from the agent's `models`
|
||||
message. Both persist and sync on join (`set_voice`/`set_model`).
|
||||
- **Media player** — now-playing card (art/title/channel/status), transport (play/pause,
|
||||
±10 s, stop), seek + volume sliders, search with "in library" tags, agent-offered choices,
|
||||
library list (play/remove), add-stream form. AVPlayer-backed with a pinned-TLS Range proxy;
|
||||
1.5 s `/state` polling and full web-parity reconciliation/reporting.
|
||||
- **Background tasks panel** — live `task_event` cards (steps, result, error) with an active
|
||||
count badge in the toolbar.
|
||||
- **Activity tab** — live `task_event` updates (steps, result, error) with an active
|
||||
count badge on the tab.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- **Simulator mic**: CoreAudio returns `-4010` (no usable input source) in this environment;
|
||||
voice capture must be verified on a real device. Text mode works in the simulator.
|
||||
- **webm media**: iOS AVPlayer cannot decode webm/opus (`-11828`). Library tracks from yt-dlp
|
||||
are webm, so audible playback requires the server to serve m4a/aac/mp3. Everything up to
|
||||
decode (pinning, Range proxy, reconciliation, UI) is verified; no client change needed once
|
||||
a supported format exists.
|
||||
- **Cert pin rotation**: see above — re-export the pin after each server cert regeneration.
|
||||
- **LiveKit signaling** uses plain `ws://192.168.86.2:7880` (not wss) because the Swift SDK
|
||||
cannot trust the self-signed cert on its WebSocket; token + media still go over pinned TLS.
|
||||
cannot trust the self-signed cert on its WebSocket; token calls still go over pinned TLS.
|
||||
|
||||
## Docs
|
||||
|
||||
- [docs/PROJECT_STATUS.md](docs/PROJECT_STATUS.md) — current handoff, decisions, and remaining work; read first.
|
||||
- [docs/VERIFICATION.md](docs/VERIFICATION.md) — test evidence and remaining qualification.
|
||||
- [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) — live NAS configuration and rollback.
|
||||
- [PLAN.md](PLAN.md) — phase-by-phase progress, verified facts, open items.
|
||||
- [docs/DEVICE_TESTING.md](docs/DEVICE_TESTING.md) — real-device qualification checklist.
|
||||
- [UPDATE_PATH.md](UPDATE_PATH.md) — original plan & protocol reference (read-only).
|
||||
@@ -0,0 +1,119 @@
|
||||
import SwiftUI
|
||||
import UIKit
|
||||
|
||||
/// Nearlight design tokens (design spec: local://ui-design-spec.md).
|
||||
///
|
||||
/// Views reference semantic roles; the primitive hex values live only here.
|
||||
/// All custom surfaces are opaque; native system materials are untouched.
|
||||
enum Nearlight {
|
||||
// MARK: Palette (light / dark variants)
|
||||
|
||||
/// Screen background.
|
||||
static let ground = Color(light: UIColor(red: 0.957, green: 0.949, blue: 0.965, alpha: 1),
|
||||
dark: UIColor(red: 0.110, green: 0.098, blue: 0.125, alpha: 1))
|
||||
/// Controls, grouped rows, dock.
|
||||
static let surface = Color(light: .white,
|
||||
dark: UIColor(red: 0.157, green: 0.137, blue: 0.180, alpha: 1))
|
||||
/// Primary text.
|
||||
static let ink = Color(light: UIColor(red: 0.161, green: 0.137, blue: 0.184, alpha: 1),
|
||||
dark: UIColor(red: 0.965, green: 0.937, blue: 0.969, alpha: 1))
|
||||
/// Supporting text, inactive icons.
|
||||
static let quiet = Color(light: UIColor(red: 0.396, green: 0.365, blue: 0.424, alpha: 1),
|
||||
dark: UIColor(red: 0.749, green: 0.698, blue: 0.776, alpha: 1))
|
||||
/// Action tint, selection, Lightwell.
|
||||
static let rosewood = Color(light: UIColor(red: 0.529, green: 0.278, blue: 0.380, alpha: 1),
|
||||
dark: UIColor(red: 0.890, green: 0.635, blue: 0.737, alpha: 1))
|
||||
/// Errors and destructive actions.
|
||||
static let alert = Color(light: UIColor(red: 0.675, green: 0.208, blue: 0.282, alpha: 1),
|
||||
dark: UIColor(red: 1.0, green: 0.604, blue: 0.675, alpha: 1))
|
||||
|
||||
// MARK: Derived roles
|
||||
|
||||
/// Decorative separator: Ink at 14% over its parent.
|
||||
static var separator: Color { ink.opacity(0.14) }
|
||||
/// Selected wash: Rosewood at 10% over Surface.
|
||||
static var selectedWash: Color { rosewood.opacity(0.10) }
|
||||
/// Error wash: Alert at 8% over Surface.
|
||||
static var errorWash: Color { alert.opacity(0.08) }
|
||||
/// Primary-action label: Surface in light, Ground in dark.
|
||||
static var onRosewood: Color {
|
||||
UITraitCollection.current.userInterfaceStyle == .dark ? ground : surface
|
||||
}
|
||||
|
||||
// MARK: Spacing scale
|
||||
|
||||
static let s4: CGFloat = 4
|
||||
static let s8: CGFloat = 8
|
||||
static let s12: CGFloat = 12
|
||||
static let s16: CGFloat = 16
|
||||
static let s20: CGFloat = 20
|
||||
static let s24: CGFloat = 24
|
||||
static let s32: CGFloat = 32
|
||||
static let s48: CGFloat = 48
|
||||
|
||||
// MARK: Radius scale
|
||||
|
||||
static let radiusSegment: CGFloat = 8
|
||||
static let radiusControl: CGFloat = 12
|
||||
static let radiusDock: CGFloat = 16
|
||||
|
||||
// MARK: Motion tokens
|
||||
|
||||
/// quick = 0.12s easeOut.
|
||||
static let quick: Animation = .easeOut(duration: 0.12)
|
||||
/// standard = 0.22s timingCurve(0.22, 1, 0.36, 1).
|
||||
static let standard: Animation = .timingCurve(0.22, 1, 0.36, 1, duration: 0.22)
|
||||
/// settle = 0.32s timingCurve(0.22, 1, 0.36, 1).
|
||||
static let settle: Animation = .timingCurve(0.22, 1, 0.36, 1, duration: 0.32)
|
||||
/// cycle = 2.4s easeInOut (Lightwell pulses).
|
||||
static let cycle: Animation = .easeInOut(duration: 2.4)
|
||||
}
|
||||
|
||||
extension Color {
|
||||
/// Builds an adaptive color from explicit light/dark UIColors without
|
||||
/// requiring an asset catalog.
|
||||
init(light: UIColor, dark: UIColor) {
|
||||
self.init(uiColor: UIColor { traits in
|
||||
traits.userInterfaceStyle == .dark ? dark : light
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Shared typography roles from the spec. SF Pro default design except the
|
||||
/// rounded welcome heading and wordmark.
|
||||
enum NearlightFont {
|
||||
/// Welcome heading: 32 medium, rounded, largeTitle metrics.
|
||||
static func welcome(_ sizeCategory: DynamicTypeSize) -> Font {
|
||||
scaled(.largeTitle, base: 32, weight: .medium, design: .rounded, sizeCategory: sizeCategory)
|
||||
}
|
||||
/// "Hope" wordmark: 20 semibold, rounded, title3 metrics.
|
||||
static func wordmark(_ sizeCategory: DynamicTypeSize) -> Font {
|
||||
scaled(.title3, base: 20, weight: .semibold, design: .rounded, sizeCategory: sizeCategory)
|
||||
}
|
||||
/// Section/detail heading: 22 semibold, title2 metrics.
|
||||
static func heading(_ sizeCategory: DynamicTypeSize) -> Font {
|
||||
scaled(.title2, base: 22, weight: .semibold, sizeCategory: sizeCategory)
|
||||
}
|
||||
/// Control/row label: 17 semibold, body metrics.
|
||||
static func control(_ sizeCategory: DynamicTypeSize) -> Font {
|
||||
scaled(.body, base: 17, weight: .semibold, sizeCategory: sizeCategory)
|
||||
}
|
||||
/// Status/support: 15 regular, subheadline metrics.
|
||||
static func support(_ sizeCategory: DynamicTypeSize) -> Font {
|
||||
scaled(.subheadline, base: 15, weight: .regular, sizeCategory: sizeCategory)
|
||||
}
|
||||
/// Speaker/timestamp: 13 medium, footnote metrics.
|
||||
static func metadata(_ sizeCategory: DynamicTypeSize) -> Font {
|
||||
scaled(.footnote, base: 13, weight: .medium, sizeCategory: sizeCategory)
|
||||
}
|
||||
|
||||
/// Scales a base size with the given text style via UIFontMetrics and
|
||||
/// returns a SwiftUI font with the requested weight/design.
|
||||
private static func scaled(_ textStyle: UIFont.TextStyle, base: CGFloat, weight: Font.Weight,
|
||||
design: Font.Design = .default, sizeCategory: DynamicTypeSize) -> Font {
|
||||
// UIFontMetrics scales the base size against the CURRENT Dynamic Type
|
||||
// setting at render time (no explicit traits: inherits environment).
|
||||
let scaledSize = UIFontMetrics(forTextStyle: textStyle).scaledValue(for: base)
|
||||
return .system(size: scaledSize, weight: weight, design: design)
|
||||
}
|
||||
}
|
||||
@@ -3,19 +3,16 @@ import Combine
|
||||
|
||||
final class SettingsStore: ObservableObject {
|
||||
@Published var serverURL: String {
|
||||
didSet { UserDefaults.standard.set(serverURL, forKey: Keys.serverURL) }
|
||||
didSet { AppPreferences.store.set(serverURL, forKey: Keys.serverURL) }
|
||||
}
|
||||
@Published var voiceID: String {
|
||||
didSet { UserDefaults.standard.set(voiceID, forKey: Keys.voiceID) }
|
||||
didSet { AppPreferences.store.set(voiceID, forKey: Keys.voiceID) }
|
||||
}
|
||||
@Published var modelID: String? {
|
||||
didSet { UserDefaults.standard.set(modelID, forKey: Keys.modelID) }
|
||||
didSet { AppPreferences.store.set(modelID, forKey: Keys.modelID) }
|
||||
}
|
||||
@Published var textMode: Bool {
|
||||
didSet { UserDefaults.standard.set(textMode, forKey: Keys.textMode) }
|
||||
}
|
||||
@Published var volume: Double {
|
||||
didSet { UserDefaults.standard.set(volume, forKey: Keys.volume) }
|
||||
didSet { AppPreferences.store.set(textMode, forKey: Keys.textMode) }
|
||||
}
|
||||
|
||||
private enum Keys {
|
||||
@@ -23,13 +20,12 @@ final class SettingsStore: ObservableObject {
|
||||
static let voiceID = "voiceID"
|
||||
static let modelID = "modelID"
|
||||
static let textMode = "textMode"
|
||||
static let volume = "volume"
|
||||
}
|
||||
|
||||
static let defaultServerURL = "https://192.168.86.2:9178"
|
||||
|
||||
init() {
|
||||
let d = UserDefaults.standard
|
||||
let d = AppPreferences.store
|
||||
// Sanitize legacy values: earlier versions wrote the field to storage on
|
||||
// every keystroke, so an invalid half-typed URL could be persisted and
|
||||
// shipped to the network layer. Fall back to the documented default and
|
||||
@@ -47,10 +43,32 @@ final class SettingsStore: ObservableObject {
|
||||
voiceID = Voices.all.contains { $0.id == storedVoice } ? (storedVoice ?? "") : Voices.defaultID
|
||||
modelID = d.string(forKey: Keys.modelID)
|
||||
textMode = d.bool(forKey: Keys.textMode)
|
||||
volume = d.object(forKey: Keys.volume) as? Double ?? 0.8
|
||||
}
|
||||
|
||||
var trimmedServerURL: String {
|
||||
serverURL.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
}
|
||||
|
||||
enum AppPreferences {
|
||||
static let store: UserDefaults = {
|
||||
#if DEBUG
|
||||
if ProcessInfo.processInfo.arguments.contains("--ui-testing") {
|
||||
let suite = "dev.shane.voice.ui-testing"
|
||||
let defaults = UserDefaults(suiteName: suite)!
|
||||
if ProcessInfo.processInfo.arguments.contains("--reset-ui-testing") {
|
||||
defaults.removePersistentDomain(forName: suite)
|
||||
// Deterministic offline errors; UI tests must not contact the NAS.
|
||||
defaults.set("https://127.0.0.1:1", forKey: "serverURL")
|
||||
}
|
||||
if ProcessInfo.processInfo.arguments.contains("--live-testing") {
|
||||
defaults.set(SettingsStore.defaultServerURL, forKey: "serverURL")
|
||||
defaults.set(true, forKey: "textMode")
|
||||
defaults.set("qwen2.5-vl-3b", forKey: "modelID")
|
||||
}
|
||||
return defaults
|
||||
}
|
||||
#endif
|
||||
return .standard
|
||||
}()
|
||||
}
|
||||
@@ -12,7 +12,7 @@ struct VoiceApp: App {
|
||||
var body: some Scene {
|
||||
WindowGroup {
|
||||
ConversationView(settings: settings)
|
||||
.preferredColorScheme(.dark)
|
||||
.tint(Nearlight.rosewood)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,343 +1,613 @@
|
||||
import AVFoundation
|
||||
import SwiftUI
|
||||
import Combine
|
||||
|
||||
/// Owns one session across all tabs. Changing tabs never ends a conversation.
|
||||
struct ConversationView: View {
|
||||
@ObservedObject var settings: SettingsStore
|
||||
@State private var sessionID = UUID()
|
||||
@State private var session: VoiceSession
|
||||
/// The exact server URL the current session was constructed with. Used to
|
||||
/// avoid redundant session replacement when the applied URL hasn't actually
|
||||
/// changed (e.g. re-applying the same value from Settings).
|
||||
@State private var activeServerURL: String
|
||||
/// Identity guard: prevents a second `.onChange` from starting a concurrent
|
||||
/// session replacement while the first is still disconnecting the old one.
|
||||
@State private var isReplacingSession = false
|
||||
/// Latest accepted server URL that has not yet been applied to a session.
|
||||
/// Coalesced: rapid Applies during an active replacement retain the newest
|
||||
/// value rather than dropping it. A single retained worker loop drains
|
||||
/// arbitrary pending URLs until none remain (Iteration 08 Item 2).
|
||||
@State private var pendingServerURL: String?
|
||||
@State private var composerText = ""
|
||||
@State private var connectError: String?
|
||||
@State private var tick = 0
|
||||
@State private var isReplacingSession = false
|
||||
@State private var selectedTab = 0
|
||||
|
||||
init(settings: SettingsStore) {
|
||||
self.settings = settings
|
||||
let initialSession = Self.makeSession(settings: settings)
|
||||
_session = State(initialValue: initialSession)
|
||||
_activeServerURL = State(initialValue: settings.trimmedServerURL)
|
||||
let url = settings.trimmedServerURL
|
||||
_activeServerURL = State(initialValue: url)
|
||||
_session = State(initialValue: Self.makeSession(url, settings: settings))
|
||||
}
|
||||
|
||||
/// Creates a session from the configured server URL. `SettingsStore.init`
|
||||
/// guarantees the stored value is always valid (legacy invalid values are
|
||||
/// sanitized to the documented default), so the failure branch is a safety
|
||||
/// net only.
|
||||
private static func makeSession(settings: SettingsStore) -> VoiceSession {
|
||||
let config = ServerConfig.parse(settings.trimmedServerURL)
|
||||
switch config {
|
||||
case .success(let value):
|
||||
return VoiceSession(config: value, tokenClient: TokenClient(config: value), settings: settings)
|
||||
case .failure:
|
||||
// Defensive: should be unreachable because SettingsStore sanitizes.
|
||||
let fallback = ServerConfig.parse(SettingsStore.defaultServerURL)
|
||||
if case .success(let value) = fallback {
|
||||
settings.serverURL = SettingsStore.defaultServerURL
|
||||
return VoiceSession(config: value, tokenClient: TokenClient(config: value), settings: settings)
|
||||
}
|
||||
fatalError("Unreachable: default server URL is valid")
|
||||
private static func makeSession(_ url: String, settings: SettingsStore) -> VoiceSession {
|
||||
guard case .success(let config) = ServerConfig.parse(url) else {
|
||||
preconditionFailure("Stored server URL failed validation; init sanitizes to a valid value.")
|
||||
}
|
||||
let session = VoiceSession(config: config, tokenClient: TokenClient(config: config), settings: settings)
|
||||
#if DEBUG
|
||||
if ProcessInfo.processInfo.arguments.contains("--ui-fixtures") {
|
||||
session.transcriptHandler.receiveDataTranscript(role: .user, text: "Help me plan a quiet weekend.")
|
||||
session.transcriptHandler.receiveDataTranscript(role: .agent, text: "Let’s leave some room to breathe. A walk, a good book, and time for your favorite music would be a lovely start.")
|
||||
session.taskStore.injectSampleEvents()
|
||||
}
|
||||
if ProcessInfo.processInfo.arguments.contains("--ui-long-transcript") {
|
||||
for index in 1...30 {
|
||||
let role: TranscriptEvent.Role = index.isMultiple(of: 2) ? .agent : .user
|
||||
session.transcriptHandler.receiveDataTranscript(
|
||||
role: role,
|
||||
text: "Earlier message \(index): a deliberately long restored transcript row for reading-position verification."
|
||||
)
|
||||
}
|
||||
}
|
||||
#endif
|
||||
return session
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
NavigationStack {
|
||||
VStack(spacing: 0) {
|
||||
statusHeader
|
||||
TranscriptListView(
|
||||
events: session.transcriptHandler.events,
|
||||
onClear: {
|
||||
// Atomically clear the canonical visible collection,
|
||||
// dedupe state, and persistence so Clear is truthful.
|
||||
session.transcriptHandler.clearAll()
|
||||
}
|
||||
)
|
||||
composer
|
||||
TabView(selection: $selectedTab) {
|
||||
NavigationStack {
|
||||
ConversationContent(settings: settings, session: session)
|
||||
.toolbar { settingsLink }
|
||||
}
|
||||
.navigationTitle("Hope Voice")
|
||||
.toolbar { toolbarContent }
|
||||
.id(sessionID)
|
||||
.onAppear {
|
||||
if case .failure(let error) = ServerConfig.parse(settings.trimmedServerURL) {
|
||||
connectError = error.localizedDescription
|
||||
}
|
||||
}
|
||||
.onChange(of: settings.textMode) { _, newValue in
|
||||
guard session.isReady else { return }
|
||||
Task { await session.applyTextMode(newValue) }
|
||||
}
|
||||
}
|
||||
.onReceive(session.objectWillChange.receive(on: RunLoop.main)) { _ in
|
||||
tick &+= 1
|
||||
}
|
||||
.onChange(of: settings.serverURL) { _, newValue in
|
||||
// The URL field now uses explicit Apply semantics (SettingsView),
|
||||
// so this fires only when the user has applied a value that passed
|
||||
// validation. Coalesce rapid accepted values: every accepted Apply
|
||||
// writes `pendingServerURL` (latest wins). A single retained worker
|
||||
// loop drains arbitrary pending URLs until none remain, covering
|
||||
// A→B→C semantics where C arrives while B is disconnecting
|
||||
// (Iteration 08 Item 2).
|
||||
let trimmed = newValue.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !trimmed.isEmpty, case .success = ServerConfig.parse(trimmed) else { return }
|
||||
.tabItem { Label("Talk", systemImage: "bubble.left.and.bubble.right") }
|
||||
.tag(0)
|
||||
|
||||
pendingServerURL = trimmed
|
||||
|
||||
// Start the worker if not already active. The loop consumes
|
||||
// pending values until empty, so a single Task handles any number
|
||||
// of rapid Applies without nested tasks.
|
||||
guard !isReplacingSession else { return }
|
||||
isReplacingSession = true
|
||||
Task { @MainActor [settings] in
|
||||
while let url = self.pendingServerURL {
|
||||
self.pendingServerURL = nil
|
||||
guard url != self.activeServerURL else { continue }
|
||||
let oldSession = self.session
|
||||
await oldSession.disconnect()
|
||||
self.activeServerURL = url
|
||||
self.sessionID = UUID()
|
||||
// Construct from the captured validated config, not by
|
||||
// rereading settings.serverURL (which may have changed
|
||||
// during disconnect).
|
||||
switch ServerConfig.parse(url) {
|
||||
case .success(let config):
|
||||
self.session = VoiceSession(config: config, tokenClient: TokenClient(config: config), settings: settings)
|
||||
case .failure:
|
||||
let fallback = ServerConfig.parse(SettingsStore.defaultServerURL)
|
||||
if case .success(let value) = fallback {
|
||||
self.session = VoiceSession(config: value, tokenClient: TokenClient(config: value), settings: settings)
|
||||
} else {
|
||||
fatalError("Unreachable: default server URL is valid")
|
||||
}
|
||||
}
|
||||
}
|
||||
self.isReplacingSession = false
|
||||
}
|
||||
ActivityTab(taskStore: session.taskStore, settings: settings, session: session)
|
||||
.tag(1)
|
||||
}
|
||||
.tint(Nearlight.rosewood)
|
||||
.onChange(of: settings.textMode) { _, enabled in
|
||||
Task { await session.applyTextMode(enabled) }
|
||||
}
|
||||
.onChange(of: settings.serverURL) { _, value in replaceServer(value) }
|
||||
}
|
||||
|
||||
private var statusHeader: some View {
|
||||
VStack(spacing: 12) {
|
||||
OrbView(state: orbState)
|
||||
Text(statusText)
|
||||
.font(.headline)
|
||||
if session.state == .ready, session.stateModel.isThinkingChipVisible {
|
||||
Label("Thinking…", systemImage: "ellipsis")
|
||||
.font(.caption)
|
||||
.padding(.horizontal, 10)
|
||||
.padding(.vertical, 4)
|
||||
.background(.orange.opacity(0.2), in: Capsule())
|
||||
}
|
||||
if session.state == .ready, !settings.textMode {
|
||||
micMeter
|
||||
}
|
||||
if session.state == .ready, settings.textMode {
|
||||
Text("Barge in anytime — type to reply")
|
||||
.font(.caption)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
if let agent = session.agent {
|
||||
Text("Agent: \(agent.name ?? agent.identity)")
|
||||
.font(.caption)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
if session.state == .ready {
|
||||
Text("\(Voices.display(for: settings.voiceID) ?? settings.voiceID) · \(session.selectedModel ?? settings.modelID ?? "—")")
|
||||
.font(.caption)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
if session.state == .ready {
|
||||
HStack(spacing: 16) {
|
||||
// The mic toggle is disabled in text mode to enforce the
|
||||
// privacy invariant at the UI level as well (Finding 4).
|
||||
Toggle(isOn: Binding(
|
||||
get: { !session.isMuted },
|
||||
set: { newValue in Task { await session.setMuted(!newValue) } }
|
||||
)) {
|
||||
Label(session.isMuted ? "Muted" : "Mic on", systemImage: session.isMuted ? "mic.slash" : "mic")
|
||||
}
|
||||
.toggleStyle(.button)
|
||||
.disabled(settings.textMode)
|
||||
Toggle("Text mode", isOn: textModeBinding)
|
||||
.toggleStyle(.button)
|
||||
}
|
||||
}
|
||||
if session.state == .ready, session.isReconnecting {
|
||||
Label("Connection lost — reconnecting…", systemImage: "arrow.triangle.2.circlepath")
|
||||
.font(.footnote)
|
||||
.foregroundStyle(.orange)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
if let micError = session.micError {
|
||||
Label(micError, systemImage: "exclamationmark.triangle")
|
||||
.font(.footnote)
|
||||
.foregroundStyle(.orange)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
if let connectError {
|
||||
Label(connectError, systemImage: "exclamationmark.triangle")
|
||||
.font(.footnote)
|
||||
.foregroundStyle(.red)
|
||||
}
|
||||
}
|
||||
.padding()
|
||||
}
|
||||
|
||||
private var micMeter: some View {
|
||||
GeometryReader { geo in
|
||||
ZStack(alignment: .leading) {
|
||||
Capsule()
|
||||
.fill(Color(.systemFill))
|
||||
Capsule()
|
||||
.fill(Color.green.gradient)
|
||||
.frame(width: min(geo.size.width, geo.size.width * session.stateModel.micLevel * 2.5))
|
||||
}
|
||||
}
|
||||
.frame(height: 4)
|
||||
}
|
||||
|
||||
private var textModeBinding: Binding<Bool> {
|
||||
$settings.textMode
|
||||
}
|
||||
|
||||
private var composer: some View {
|
||||
HStack(spacing: 8) {
|
||||
switch session.state {
|
||||
case .idle, .failed:
|
||||
Button(action: startSession) {
|
||||
Label("Start", systemImage: "phone.fill")
|
||||
}
|
||||
.buttonStyle(.borderedProminent)
|
||||
case .connecting, .waitingAgent, .tearingDown:
|
||||
ProgressView()
|
||||
case .ready:
|
||||
TextField(settings.textMode ? "Type a message…" : "Voice mode active",
|
||||
text: $composerText,
|
||||
axis: .vertical)
|
||||
.textFieldStyle(.roundedBorder)
|
||||
.lineLimit(1...4)
|
||||
.disabled(!settings.textMode)
|
||||
Button(action: sendComposer) {
|
||||
Image(systemName: "arrow.up.circle.fill")
|
||||
.font(.title2)
|
||||
}
|
||||
.accessibilityLabel("Send message")
|
||||
.disabled(!settings.textMode || composerText.isEmpty)
|
||||
Button(role: .destructive, action: endSession) {
|
||||
Image(systemName: "phone.down.fill")
|
||||
}
|
||||
.accessibilityLabel("End conversation")
|
||||
}
|
||||
}
|
||||
.padding()
|
||||
}
|
||||
|
||||
@ToolbarContentBuilder
|
||||
private var toolbarContent: some ToolbarContent {
|
||||
ToolbarItem(placement: .topBarTrailing) {
|
||||
NavigationLink {
|
||||
TaskPanelView(taskStore: session.taskStore)
|
||||
} label: {
|
||||
Image(systemName: "tray.full")
|
||||
.overlay(alignment: .topTrailing) {
|
||||
if session.taskStore.activeCount > 0 {
|
||||
Text("\(session.taskStore.activeCount)")
|
||||
.font(.caption2.weight(.bold))
|
||||
.foregroundStyle(.white)
|
||||
.padding(3)
|
||||
.background(Color.red, in: Circle())
|
||||
.offset(x: 6, y: -6)
|
||||
}
|
||||
}
|
||||
}
|
||||
.accessibilityLabel("Background tasks")
|
||||
}
|
||||
ToolbarItem(placement: .topBarTrailing) {
|
||||
NavigationLink {
|
||||
MediaPanelView(playerController: session.mediaController, settings: settings)
|
||||
} label: {
|
||||
Image(systemName: "music.note")
|
||||
}
|
||||
.accessibilityLabel("Media player")
|
||||
}
|
||||
@ToolbarContentBuilder private var settingsLink: some ToolbarContent {
|
||||
ToolbarItem(placement: .topBarTrailing) {
|
||||
NavigationLink {
|
||||
SettingsView(settings: settings, session: session)
|
||||
} label: {
|
||||
Image(systemName: "gearshape")
|
||||
Image(systemName: "gearshape").frame(minWidth: 44, minHeight: 44)
|
||||
}
|
||||
.accessibilityLabel("Settings")
|
||||
}
|
||||
}
|
||||
|
||||
private var orbState: OrbView.OrbState {
|
||||
switch session.state {
|
||||
case .idle: return .idle
|
||||
case .connecting, .waitingAgent, .tearingDown: return .connecting
|
||||
case .ready:
|
||||
guard !settings.textMode else { return .idle }
|
||||
switch session.stateModel.phase {
|
||||
case .idle: return .idle
|
||||
case .connecting: return .connecting
|
||||
case .listening: return .listening
|
||||
case .thinking: return .thinking
|
||||
case .speaking: return .speaking
|
||||
private func replaceServer(_ value: String) {
|
||||
let url = value.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard case .success = ServerConfig.parse(url) else { return }
|
||||
pendingServerURL = url
|
||||
guard !isReplacingSession else { return }
|
||||
isReplacingSession = true
|
||||
Task { @MainActor in
|
||||
// Coalesce rapid Apply actions; construct from the captured URL.
|
||||
while let next = pendingServerURL {
|
||||
pendingServerURL = nil
|
||||
guard next != activeServerURL else { continue }
|
||||
await session.disconnect()
|
||||
activeServerURL = next
|
||||
session = Self.makeSession(next, settings: settings)
|
||||
}
|
||||
case .failed: return .failed
|
||||
isReplacingSession = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private struct ActivityTab: View {
|
||||
@ObservedObject var taskStore: TaskStore
|
||||
@ObservedObject var settings: SettingsStore
|
||||
@ObservedObject var session: VoiceSession
|
||||
|
||||
var body: some View {
|
||||
NavigationStack {
|
||||
TaskPanelView(taskStore: taskStore)
|
||||
.toolbar {
|
||||
ToolbarItem(placement: .topBarTrailing) {
|
||||
NavigationLink {
|
||||
SettingsView(settings: settings, session: session)
|
||||
} label: {
|
||||
Image(systemName: "gearshape").frame(minWidth: 44, minHeight: 44)
|
||||
}
|
||||
.accessibilityLabel("Settings")
|
||||
}
|
||||
}
|
||||
}
|
||||
.tabItem { Label("Activity", systemImage: "checklist") }
|
||||
.badge(taskStore.activeCount)
|
||||
}
|
||||
}
|
||||
|
||||
/// Maps session + conversation models onto Lightwell presentation phases.
|
||||
@MainActor
|
||||
private func lightwellPhase(session: VoiceSession, conversation: ConversationStateModel,
|
||||
textMode: Bool) -> LightwellView.Phase {
|
||||
if session.state.isFailed { return .failed }
|
||||
switch session.state {
|
||||
case .idle: return .idle
|
||||
case .connecting: return .connecting
|
||||
case .waitingAgent: return .waiting
|
||||
case .tearingDown: return .tearingDown
|
||||
case .failed: return .failed
|
||||
case .ready:
|
||||
// Only substitute idle input geometry when otherwise LISTENING.
|
||||
// Assistant output (speaking/thinking) must stay represented in
|
||||
// Type and muted modes per spec.
|
||||
switch conversation.phase {
|
||||
case .speaking: return .speaking(level: conversation.agentLevel)
|
||||
case .thinking: return .thinking
|
||||
default:
|
||||
if textMode || session.isMuted { return .readyNoInput }
|
||||
return .listening(level: conversation.micLevel)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private struct ConversationContent: View {
|
||||
@ObservedObject var settings: SettingsStore
|
||||
@ObservedObject var session: VoiceSession
|
||||
@ObservedObject private var transcripts: TranscriptHandler
|
||||
@ObservedObject private var conversation: ConversationStateModel
|
||||
@State private var composerText = ""
|
||||
@State private var sendError: String?
|
||||
@State private var isSending = false
|
||||
@FocusState private var composerFocused: Bool
|
||||
@Environment(\.dynamicTypeSize) private var typeSize
|
||||
@Environment(\.accessibilityReduceMotion) private var reduceMotion
|
||||
@Environment(\.colorSchemeContrast) private var contrast
|
||||
|
||||
init(settings: SettingsStore, session: VoiceSession) {
|
||||
self.settings = settings
|
||||
self.session = session
|
||||
transcripts = session.transcriptHandler
|
||||
conversation = session.stateModel
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 0) {
|
||||
if transcripts.events.isEmpty && !session.isReady {
|
||||
welcome
|
||||
if failureNotice != nil { failureBanner }
|
||||
} else {
|
||||
statusStrip
|
||||
if session.isReconnecting { reconnectBanner }
|
||||
if failureNotice != nil { failureBanner }
|
||||
TranscriptListView(events: transcripts.events, onClear: transcripts.clearAll)
|
||||
}
|
||||
}
|
||||
.background(Nearlight.ground)
|
||||
.navigationTitle("Hope")
|
||||
.navigationBarTitleDisplayMode(.inline)
|
||||
.safeAreaInset(edge: .bottom, spacing: 0) { dock }
|
||||
.onChange(of: session.state) { _, state in
|
||||
if state == .ready && settings.textMode { composerFocused = true }
|
||||
}
|
||||
.sensoryFeedback(.success, trigger: session.state == .ready) { old, new in
|
||||
// Success only on the transition INTO readiness, not out.
|
||||
!old && new
|
||||
}
|
||||
.sensoryFeedback(.error, trigger: session.state.isFailed) { old, new in
|
||||
// Error only on entering failure, not leaving it.
|
||||
!old && new
|
||||
}
|
||||
}
|
||||
|
||||
private var statusText: String {
|
||||
// MARK: Welcome (pre-connection, no history)
|
||||
|
||||
private var welcome: some View {
|
||||
ScrollView {
|
||||
VStack(alignment: .leading, spacing: Nearlight.s24) {
|
||||
LightwellView(phase: lightwellPhase(session: session, conversation: conversation, textMode: settings.textMode))
|
||||
.padding(.top, Nearlight.s32)
|
||||
VStack(alignment: .leading, spacing: Nearlight.s12) {
|
||||
Text("A moment with Hope.")
|
||||
.font(NearlightFont.welcome(typeSize))
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
Text("Speak or type to your own assistant.")
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
}
|
||||
statusLine
|
||||
}
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.padding(.bottom, Nearlight.s24)
|
||||
}
|
||||
}
|
||||
|
||||
/// Status text 24 below the supporting copy on welcome.
|
||||
private var statusLine: some View {
|
||||
Text(statusTitle)
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.padding(.top, Nearlight.s24)
|
||||
.accessibilityIdentifier("talk.status")
|
||||
}
|
||||
|
||||
// MARK: Status strip (active conversation)
|
||||
|
||||
private var statusStrip: some View {
|
||||
HStack(spacing: Nearlight.s12) {
|
||||
LightwellView(phase: lightwellPhase(session: session, conversation: conversation, textMode: settings.textMode), compact: true)
|
||||
VStack(alignment: .leading, spacing: 2) {
|
||||
Text(statusTitle)
|
||||
.font(NearlightFont.control(typeSize))
|
||||
.foregroundStyle(Nearlight.ink)
|
||||
.accessibilityIdentifier("talk.status")
|
||||
if showStatusDetail {
|
||||
Text(statusDetail)
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
}
|
||||
}
|
||||
Spacer(minLength: 0)
|
||||
}
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.padding(.vertical, Nearlight.s12)
|
||||
.frame(minHeight: 60, alignment: .leading)
|
||||
.background(Nearlight.ground)
|
||||
}
|
||||
|
||||
/// Subtitle omitted for ordinary listening/thinking/speaking per spec.
|
||||
private var showStatusDetail: Bool {
|
||||
guard session.isReady else { return true }
|
||||
if settings.textMode || session.isMuted { return true }
|
||||
return false
|
||||
}
|
||||
|
||||
private var reconnectBanner: some View {
|
||||
VStack(alignment: .leading, spacing: 2) {
|
||||
Label("Reconnecting…", systemImage: "wifi.exclamationmark")
|
||||
.font(NearlightFont.control(typeSize))
|
||||
Text("Your conversation is still here.")
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
}
|
||||
.padding(Nearlight.s12)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.background(Nearlight.surface)
|
||||
.clipShape(RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.accessibilityIdentifier("talk.connection.notice")
|
||||
}
|
||||
|
||||
/// Classified failure notice with actionable recovery.
|
||||
private var failureBanner: some View {
|
||||
VStack(alignment: .leading, spacing: Nearlight.s8) {
|
||||
Label(failureTitle, systemImage: "exclamationmark.circle")
|
||||
.font(NearlightFont.control(typeSize))
|
||||
.foregroundStyle(Nearlight.alert)
|
||||
Text(failureBody)
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.ink)
|
||||
if micDenied {
|
||||
Button("Open iPhone Settings") {
|
||||
if let url = URL(string: UIApplication.openSettingsURLString) {
|
||||
UIApplication.shared.open(url)
|
||||
}
|
||||
}
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.frame(minHeight: 44)
|
||||
.accessibilityIdentifier("talk.connection.openSystemSettings")
|
||||
}
|
||||
}
|
||||
.padding(Nearlight.s16)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.background(Nearlight.errorWash)
|
||||
.clipShape(RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.accessibilityIdentifier("talk.connection.notice")
|
||||
}
|
||||
|
||||
private var micDenied: Bool {
|
||||
guard session.state == .ready, let micError = session.micError else {
|
||||
return false
|
||||
}
|
||||
return AVAudioApplication.shared.recordPermission == .denied
|
||||
|| micError.localizedCaseInsensitiveContains("denied")
|
||||
}
|
||||
|
||||
private var failureTitle: String {
|
||||
if session.micError != nil { return "Microphone unavailable" }
|
||||
return session.everConnected ? "Connection interrupted" : "Couldn’t connect"
|
||||
}
|
||||
|
||||
private var failureBody: String {
|
||||
if let micError = session.micError {
|
||||
return micError
|
||||
}
|
||||
let raw = failureMessage ?? ""
|
||||
// Only the agent-timeout path is a known classified cause; every
|
||||
// other failure keeps its actual readable error per spec.
|
||||
if raw.contains("Agent never joined") {
|
||||
return "Check that Hope is running on your server."
|
||||
}
|
||||
return raw.isEmpty ? "The connection failed. Try again." : raw
|
||||
}
|
||||
|
||||
// MARK: Dock
|
||||
|
||||
private var dock: some View {
|
||||
VStack(spacing: Nearlight.s12) {
|
||||
if let sendError {
|
||||
Label(sendError, systemImage: "exclamationmark.circle")
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.alert)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.accessibilityIdentifier("talk.composer.error")
|
||||
}
|
||||
modeSelector
|
||||
if session.isReady {
|
||||
if settings.textMode { typeControls } else { voiceControls }
|
||||
} else if isBusy && !session.state.isFailed {
|
||||
// Connecting/waiting: keep the draft readable and expose the
|
||||
// existing cancellation path per spec.
|
||||
if settings.textMode { typeControls }
|
||||
else { HStack(spacing: Nearlight.s12) { muteButton; micMeter } }
|
||||
endButton
|
||||
} else {
|
||||
startControls
|
||||
}
|
||||
}
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.padding(.top, Nearlight.s12)
|
||||
.padding(.bottom, Nearlight.s12)
|
||||
.background(
|
||||
// Dock: Surface, 16-point top corners, decorative top separator.
|
||||
VStack(spacing: 0) {
|
||||
Rectangle().fill(Nearlight.separator).frame(height: 1 / UIScreen.main.scale)
|
||||
Nearlight.surface
|
||||
}
|
||||
.ignoresSafeArea(.container, edges: .bottom)
|
||||
.clipShape(UnevenRoundedRectangle(topLeadingRadius: Nearlight.radiusDock, topTrailingRadius: Nearlight.radiusDock))
|
||||
)
|
||||
}
|
||||
|
||||
@ViewBuilder private var modeSelector: some View {
|
||||
if typeSize.isAccessibilitySize {
|
||||
// Two stacked 52-point selection buttons at accessibility sizes.
|
||||
VStack(spacing: Nearlight.s8) {
|
||||
modeButton("Voice", systemImage: "mic", selected: !settings.textMode, id: "talk.mode.voice") {
|
||||
settings.textMode = false
|
||||
}
|
||||
modeButton("Type", systemImage: "keyboard", selected: settings.textMode, id: "talk.mode.type") {
|
||||
settings.textMode = true
|
||||
}
|
||||
}
|
||||
.disabled(isBusy)
|
||||
} else {
|
||||
Picker("Conversation mode", selection: $settings.textMode) {
|
||||
Label("Voice", systemImage: "mic").tag(false)
|
||||
Label("Type", systemImage: "keyboard").tag(true)
|
||||
}
|
||||
.pickerStyle(.segmented)
|
||||
.disabled(isBusy)
|
||||
}
|
||||
}
|
||||
|
||||
private func modeButton(_ title: String, systemImage: String, selected: Bool,
|
||||
id: String, action: @escaping () -> Void) -> some View {
|
||||
Button(action: action) {
|
||||
HStack(spacing: Nearlight.s8) {
|
||||
Image(systemName: systemImage)
|
||||
Text(title)
|
||||
if selected { Spacer(); Image(systemName: "checkmark") }
|
||||
}
|
||||
.font(NearlightFont.control(typeSize))
|
||||
.foregroundStyle(selected ? Nearlight.rosewood : Nearlight.ink)
|
||||
.frame(maxWidth: .infinity, minHeight: 52)
|
||||
.background(selected && contrast == .increased ? Nearlight.surface : selected ? Nearlight.selectedWash : Nearlight.surface,
|
||||
in: RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: Nearlight.radiusControl)
|
||||
.strokeBorder(selected && contrast == .increased ? Nearlight.rosewood : selected ? Nearlight.rosewood.opacity(0.4) : Nearlight.quiet,
|
||||
lineWidth: selected && contrast == .increased ? 2 : 1)
|
||||
)
|
||||
}
|
||||
.accessibilityIdentifier(id)
|
||||
.accessibilityAddTraits(selected ? .isSelected : [])
|
||||
}
|
||||
|
||||
private var startControls: some View {
|
||||
VStack(spacing: Nearlight.s8) {
|
||||
Text(settings.textMode ? "Your microphone stays off." : "Your microphone starts after you connect.")
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
Button {
|
||||
sendError = nil
|
||||
Task { await session.connect() }
|
||||
} label: {
|
||||
HStack(spacing: Nearlight.s8) {
|
||||
if isBusy {
|
||||
ProgressView().tint(Nearlight.onRosewood)
|
||||
} else {
|
||||
Image(systemName: settings.textMode ? "keyboard" : "mic")
|
||||
}
|
||||
Text(isBusy ? statusTitle : session.state.isFailed ? "Try again" : "Start chatting")
|
||||
.font(NearlightFont.control(typeSize))
|
||||
}
|
||||
.frame(maxWidth: .infinity, minHeight: 52)
|
||||
.background(isBusy ? Nearlight.rosewood.opacity(0.6) : Nearlight.rosewood,
|
||||
in: RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.foregroundStyle(Nearlight.onRosewood)
|
||||
}
|
||||
.disabled(isBusy)
|
||||
.accessibilityIdentifier("start.chatting")
|
||||
}
|
||||
}
|
||||
|
||||
private var voiceControls: some View {
|
||||
Group {
|
||||
if typeSize.isAccessibilitySize {
|
||||
VStack(spacing: Nearlight.s12) {
|
||||
HStack(spacing: Nearlight.s12) {
|
||||
muteButton
|
||||
micMeter
|
||||
}
|
||||
endButton
|
||||
}
|
||||
} else {
|
||||
HStack(spacing: Nearlight.s12) {
|
||||
muteButton
|
||||
micMeter
|
||||
endButton
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var muteButton: some View {
|
||||
Button {
|
||||
Task { await session.setMuted(!session.isMuted) }
|
||||
} label: {
|
||||
Image(systemName: session.isMuted ? "mic.slash" : "mic")
|
||||
.font(.system(size: 20, weight: .medium))
|
||||
.frame(width: 52, height: 52)
|
||||
.background(session.isMuted ? Nearlight.selectedWash : Nearlight.surface,
|
||||
in: RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: Nearlight.radiusControl)
|
||||
.strokeBorder(Nearlight.quiet, lineWidth: 1)
|
||||
)
|
||||
.foregroundStyle(session.isMuted ? Nearlight.rosewood : Nearlight.ink)
|
||||
}
|
||||
.accessibilityLabel(session.isMuted ? "Unmute microphone" : "Mute microphone")
|
||||
.accessibilityIdentifier("talk.microphone.toggle")
|
||||
.sensoryFeedback(.selection, trigger: session.isMuted)
|
||||
}
|
||||
|
||||
/// Five-bar capsule meter, 64×24, leading aligned.
|
||||
private var micMeter: some View {
|
||||
HStack(spacing: 6) {
|
||||
let level = session.isMuted || settings.textMode ? 0.0 : conversation.micLevel
|
||||
let filled = Int(ceil(level * 5))
|
||||
ForEach(0..<5, id: \.self) { index in
|
||||
Capsule()
|
||||
.fill(index < filled ? Nearlight.rosewood : Color.clear)
|
||||
.overlay(index < filled ? nil : Capsule().strokeBorder(Nearlight.quiet, lineWidth: 1))
|
||||
.frame(width: 6, height: [6.0, 12.0, 24.0, 12.0, 6.0][index])
|
||||
}
|
||||
}
|
||||
.frame(width: 64, height: 24, alignment: .leading)
|
||||
.accessibilityElement(children: .ignore)
|
||||
.accessibilityLabel("Microphone level")
|
||||
.accessibilityValue(session.isMuted || settings.textMode ? "Off" : "\(Int(round(conversation.micLevel * 100)))%")
|
||||
}
|
||||
|
||||
private var typeControls: some View {
|
||||
VStack(spacing: Nearlight.s12) {
|
||||
HStack(alignment: .bottom, spacing: Nearlight.s8) {
|
||||
TextField("Message Hope", text: $composerText, axis: .vertical)
|
||||
.lineLimit(1...5)
|
||||
.padding(Nearlight.s12)
|
||||
.background(Nearlight.surface, in: RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: Nearlight.radiusControl)
|
||||
.strokeBorder(Nearlight.quiet, lineWidth: 1)
|
||||
)
|
||||
.focused($composerFocused)
|
||||
.disabled(isSending)
|
||||
.accessibilityLabel("Message Hope")
|
||||
Button(action: sendComposer) {
|
||||
if isSending {
|
||||
ProgressView().frame(width: 52, height: 52)
|
||||
} else {
|
||||
Image(systemName: "arrow.up")
|
||||
.font(.system(size: 20, weight: .medium))
|
||||
.frame(width: 52, height: 52)
|
||||
.background(Nearlight.rosewood, in: RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.foregroundStyle(Nearlight.onRosewood)
|
||||
}
|
||||
}
|
||||
.disabled(composerText.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty || isSending || session.isReconnecting || !session.isReady)
|
||||
.accessibilityLabel("Send message")
|
||||
}
|
||||
endButton
|
||||
}
|
||||
}
|
||||
|
||||
private var endButton: some View {
|
||||
Button(role: .destructive) {
|
||||
composerFocused = false
|
||||
Task { await session.disconnect() }
|
||||
} label: {
|
||||
Label("End conversation", systemImage: "stop.fill")
|
||||
.font(NearlightFont.control(typeSize))
|
||||
.lineLimit(2...3)
|
||||
.frame(maxWidth: .infinity, minHeight: 52)
|
||||
.background(Nearlight.surface, in: RoundedRectangle(cornerRadius: Nearlight.radiusControl))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: Nearlight.radiusControl)
|
||||
.strokeBorder(Nearlight.alert, lineWidth: 1)
|
||||
)
|
||||
.foregroundStyle(Nearlight.alert)
|
||||
}
|
||||
.accessibilityIdentifier("end.conversation")
|
||||
}
|
||||
|
||||
// MARK: Status copy
|
||||
|
||||
private var isBusy: Bool {
|
||||
session.state == .connecting || session.state == .waitingAgent || session.state == .tearingDown
|
||||
}
|
||||
private var failureMessage: String? {
|
||||
if case .failed(let message) = session.state { return message }
|
||||
return nil
|
||||
}
|
||||
private var failureNotice: String? { session.micError ?? failureMessage }
|
||||
|
||||
private var statusTitle: String {
|
||||
switch session.state {
|
||||
case .idle: return "Ready to connect"
|
||||
case .connecting: return "Connecting…"
|
||||
case .waitingAgent: return "Waiting for agent…"
|
||||
case .tearingDown: return "Cleaning up…"
|
||||
case .idle: return "Ready when you are"
|
||||
case .connecting: return "Connecting to your server…"
|
||||
case .waitingAgent: return "Waiting for Hope…"
|
||||
case .tearingDown: return "Ending conversation…"
|
||||
case .failed: return "Connection interrupted"
|
||||
case .ready:
|
||||
if settings.textMode { return "Text mode" }
|
||||
switch session.stateModel.phase {
|
||||
case .listening: return "Listening"
|
||||
case .thinking: return "Thinking it through"
|
||||
// Assistant output outranks input-mode substitution.
|
||||
switch conversation.phase {
|
||||
case .speaking: return "Hope is speaking"
|
||||
default: return "Ready when you are"
|
||||
case .thinking: return "Thinking…"
|
||||
default:
|
||||
if settings.textMode { return "Ready for your message" }
|
||||
if session.isMuted { return "Microphone off" }
|
||||
return "Listening"
|
||||
}
|
||||
case .failed(let message): return "Failed: \(message)"
|
||||
}
|
||||
}
|
||||
|
||||
private func startSession() {
|
||||
connectError = nil
|
||||
// Starting a conversation preserves the restored last-50 history; only
|
||||
// the explicit Clear button erases it.
|
||||
Task { await session.connect() }
|
||||
}
|
||||
|
||||
private func endSession() {
|
||||
Task { await session.disconnect() }
|
||||
private var statusDetail: String {
|
||||
if session.state == .waitingAgent { return "The server is connected. Hope has not joined yet." }
|
||||
if session.state == .connecting { return "Reaching your server." }
|
||||
if !session.isReady { return "" }
|
||||
if settings.textMode { return "Microphone off · Type whenever you’re ready" }
|
||||
if session.isMuted { return "You can still hear Hope." }
|
||||
return ""
|
||||
}
|
||||
|
||||
private func sendComposer() {
|
||||
let text = composerText.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !text.isEmpty else { return }
|
||||
composerText = ""
|
||||
guard !text.isEmpty, !isSending else { return }
|
||||
isSending = true
|
||||
sendError = nil
|
||||
Task {
|
||||
do {
|
||||
try await session.sendText(text)
|
||||
composerText = ""
|
||||
} catch {
|
||||
connectError = error.localizedDescription
|
||||
sendError = "Your message wasn’t sent. Try again when connected."
|
||||
}
|
||||
isSending = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#Preview {
|
||||
ConversationView(settings: SettingsStore())
|
||||
extension VoiceSession {
|
||||
/// Whether any connection reached ready during this session lifetime.
|
||||
/// Used to title the failure notice per spec.
|
||||
var everConnected: Bool {
|
||||
// Sessions are recreated on replacement; reaching ready is recorded
|
||||
// indirectly through transcript presence. Conservative: treat any
|
||||
// prior transcript activity as evidence of a completed connection.
|
||||
!transcriptHandler.events.isEmpty
|
||||
}
|
||||
}
|
||||
|
||||
#Preview { ConversationView(settings: SettingsStore()) }
|
||||
@@ -0,0 +1,151 @@
|
||||
import SwiftUI
|
||||
import UIKit
|
||||
|
||||
/// The Nearlight signature element: an upright, softly lit aperture.
|
||||
///
|
||||
/// Two capsules in a ZStack — no assets, shaders, blur, or particles. The
|
||||
/// inner capsule carries a vertical Rosewood gradient; its geometry encodes
|
||||
/// conversation state and audio level. Noninteractive and accessibility
|
||||
/// hidden: the companion status text carries meaning.
|
||||
struct LightwellView: View {
|
||||
/// Lightwell presentation state, mapped from session/conversation models.
|
||||
enum Phase: Equatable {
|
||||
case idle
|
||||
case connecting
|
||||
case waiting
|
||||
case listening(level: Double)
|
||||
case thinking
|
||||
case speaking(level: Double)
|
||||
case tearingDown
|
||||
case failed
|
||||
/// Input never drives the light (muted or Type mode).
|
||||
case readyNoInput
|
||||
}
|
||||
|
||||
let phase: Phase
|
||||
/// Compact (24×36) in the conversation status strip; full (64×96) on welcome.
|
||||
var compact: Bool = false
|
||||
@Environment(\.accessibilityReduceMotion) private var reduceMotion
|
||||
/// Frame dimensions never animate; inner dimensions multiply by 0.375
|
||||
/// in the compact frame.
|
||||
private var frameSize: CGSize {
|
||||
compact ? CGSize(width: 24, height: 36) : CGSize(width: 64, height: 96)
|
||||
}
|
||||
private var baseInner: CGSize {
|
||||
let base = CGSize(width: 16, height: 40)
|
||||
return compact
|
||||
? CGSize(width: base.width * 0.375, height: base.height * 0.375)
|
||||
: base
|
||||
}
|
||||
|
||||
@Environment(\.colorSchemeContrast) private var contrast
|
||||
|
||||
var body: some View {
|
||||
Group {
|
||||
if pulseWanted {
|
||||
// Isolated TimelineView: ticks only while pending states are
|
||||
// showing; pauses when offscreen/inactive automatically.
|
||||
TimelineView(.periodic(from: .now, by: 0.05)) { context in
|
||||
lightwell(pulse: pulseValue(at: context.date))
|
||||
}
|
||||
} else {
|
||||
lightwell(pulse: 0)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func lightwell(pulse: Double) -> some View {
|
||||
let inner = innerSize
|
||||
return ZStack {
|
||||
Capsule()
|
||||
.fill(Nearlight.surface)
|
||||
Capsule()
|
||||
.fill(innerFill)
|
||||
.frame(width: inner.width, height: inner.height)
|
||||
.opacity(overallOpacity(pulse: pulse))
|
||||
}
|
||||
.frame(width: frameSize.width, height: frameSize.height)
|
||||
.overlay(
|
||||
Capsule().strokeBorder(
|
||||
contrast == .increased ? Nearlight.rosewood : Nearlight.quiet,
|
||||
lineWidth: 1
|
||||
)
|
||||
)
|
||||
.accessibilityHidden(true)
|
||||
.animation(reduceMotion ? nil : Nearlight.settle, value: innerSize)
|
||||
}
|
||||
|
||||
/// Increase Contrast promotes the gradient to solid Rosewood.
|
||||
private var innerFill: AnyShapeStyle {
|
||||
if contrast == .increased {
|
||||
return AnyShapeStyle(Nearlight.rosewood)
|
||||
}
|
||||
return AnyShapeStyle(
|
||||
LinearGradient(
|
||||
stops: [
|
||||
.init(color: Nearlight.rosewood.opacity(0.35), location: 0),
|
||||
.init(color: Nearlight.rosewood.opacity(1), location: 0.5),
|
||||
.init(color: Nearlight.rosewood.opacity(0.55), location: 1),
|
||||
],
|
||||
startPoint: .top, endPoint: .bottom
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
/// Whether a pulse TimelineView should run for this phase.
|
||||
private var pulseWanted: Bool {
|
||||
switch phase {
|
||||
case .connecting, .waiting, .thinking: return !reduceMotion
|
||||
default: return false
|
||||
}
|
||||
}
|
||||
|
||||
/// Triangle wave 0→1→0 over the 2.4s cycle from the timeline date.
|
||||
private func pulseValue(at date: Date) -> Double {
|
||||
let t = date.timeIntervalSinceReferenceDate.truncatingRemainder(dividingBy: 2.4) / 2.4
|
||||
return t < 0.5 ? t * 2 : (1 - t) * 2
|
||||
}
|
||||
|
||||
/// Inner capsule geometry per spec table. `a` is the smoothed level.
|
||||
/// Reduce Motion: static 16×40 (idle/listening) and 16×48 (pending/
|
||||
/// thinking/speaking) with no level-driven resizing.
|
||||
private var innerSize: CGSize {
|
||||
let base = baseInner
|
||||
if reduceMotion {
|
||||
switch phase {
|
||||
case .waiting, .thinking, .speaking:
|
||||
return CGSize(width: base.width, height: base.height + (compact ? 3 : 8))
|
||||
default:
|
||||
return base
|
||||
}
|
||||
}
|
||||
switch phase {
|
||||
case .listening(let a):
|
||||
let level = min(max(a, 0), 1)
|
||||
return CGSize(width: base.width + (compact ? 3 : 8) * level,
|
||||
height: base.height + (compact ? 6 : 16) * level)
|
||||
case .speaking(let a):
|
||||
let level = min(max(a, 0), 1)
|
||||
return CGSize(width: base.width + (compact ? 4.5 : 12) * level,
|
||||
height: base.height + (compact ? 9 : 24) * level)
|
||||
case .waiting, .thinking:
|
||||
// Pulse drives opacity; height steps up one band.
|
||||
return CGSize(width: base.width, height: base.height + (compact ? 3 : 8))
|
||||
case .connecting, .idle, .failed, .tearingDown, .readyNoInput:
|
||||
return base
|
||||
}
|
||||
}
|
||||
|
||||
private func overallOpacity(pulse: Double) -> Double {
|
||||
switch phase {
|
||||
case .connecting, .waiting:
|
||||
return 0.45 + 0.45 * pulse
|
||||
case .thinking:
|
||||
return 0.65 + 0.35 * pulse
|
||||
case .idle, .failed, .tearingDown:
|
||||
return 0.45
|
||||
case .listening, .speaking, .readyNoInput:
|
||||
return 1
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,150 +0,0 @@
|
||||
import SwiftUI
|
||||
|
||||
struct OrbView: View {
|
||||
enum OrbState {
|
||||
case idle
|
||||
case connecting
|
||||
case listening
|
||||
case speaking
|
||||
case thinking
|
||||
case failed
|
||||
}
|
||||
|
||||
let state: OrbState
|
||||
|
||||
@State private var animating = false
|
||||
|
||||
private var baseColor: Color {
|
||||
switch state {
|
||||
case .idle, .connecting: .blue
|
||||
case .listening: .green
|
||||
case .speaking, .thinking: .orange
|
||||
case .failed: .red
|
||||
}
|
||||
}
|
||||
|
||||
private var scale: CGFloat {
|
||||
switch state {
|
||||
case .connecting, .listening, .speaking, .thinking: 1.1
|
||||
default: 1.0
|
||||
}
|
||||
}
|
||||
|
||||
private var label: String {
|
||||
switch state {
|
||||
case .idle: "Idle"
|
||||
case .connecting: "Connecting"
|
||||
case .listening: "Listening"
|
||||
case .speaking: "Speaking"
|
||||
case .thinking: "Thinking"
|
||||
case .failed: "Connection failed"
|
||||
}
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
let shape = Circle()
|
||||
.fill(baseColor.gradient)
|
||||
.frame(width: 120, height: 120)
|
||||
let animated = shape
|
||||
.scaleEffect(animating ? scale : 0.95)
|
||||
.opacity(state == .idle || state == .failed ? 0.7 : 1.0)
|
||||
let anim: Animation? = state == .idle || state == .failed
|
||||
? nil
|
||||
: .easeInOut(duration: 0.8).repeatForever(autoreverses: true)
|
||||
animated
|
||||
.animation(anim, value: animating)
|
||||
.onAppear { animating = true }
|
||||
.onChange(of: state) { _, _ in animating = true }
|
||||
.accessibilityElement()
|
||||
.accessibilityLabel("Conversation status")
|
||||
.accessibilityValue(label)
|
||||
}
|
||||
}
|
||||
|
||||
struct TranscriptListView: View {
|
||||
/// The single canonical visible transcript (restored persisted finals plus
|
||||
/// current live events). The view renders exactly this collection — there is
|
||||
/// no second "restored" store, so the UI and persistence can never diverge.
|
||||
let events: [TranscriptEvent]
|
||||
var onClear: () -> Void
|
||||
|
||||
private static let deliveryMarkers = #"\[(?:appreciative|calm|concerned|curious|encouraging|excited|happy|joking|quiet|reassuring|reflective|sad|serious|surprised|sympathetic|urgent|whispering|laughter|sighing|breathing|neutral)\]\s*"#
|
||||
|
||||
private static let timestampFormatter: DateFormatter = {
|
||||
let formatter = DateFormatter()
|
||||
formatter.dateStyle = .none
|
||||
formatter.timeStyle = .short
|
||||
return formatter
|
||||
}()
|
||||
|
||||
private var visibleRows: [TranscriptEvent] { events }
|
||||
|
||||
var body: some View {
|
||||
ScrollViewReader { proxy in
|
||||
VStack(alignment: .leading, spacing: 0) {
|
||||
HStack {
|
||||
Text("Transcript")
|
||||
.font(.caption.bold())
|
||||
.foregroundStyle(.secondary)
|
||||
Spacer()
|
||||
Button(role: .destructive, action: onClear) {
|
||||
Label("Clear", systemImage: "trash")
|
||||
.font(.caption)
|
||||
}
|
||||
}
|
||||
.padding(.horizontal)
|
||||
.padding(.bottom, 4)
|
||||
|
||||
ScrollView {
|
||||
LazyVStack(alignment: .leading, spacing: 8) {
|
||||
ForEach(visibleRows) { event in
|
||||
transcriptRow(event)
|
||||
}
|
||||
}
|
||||
.padding(.horizontal)
|
||||
}
|
||||
.onChange(of: visibleRows.count) { _, _ in
|
||||
if let last = visibleRows.last {
|
||||
withAnimation { proxy.scrollTo(last.id, anchor: .bottom) }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func transcriptRow(_ event: TranscriptEvent) -> some View {
|
||||
let roleName = event.role == .user ? "You" : "Hope"
|
||||
let nameColor: Color = event.role == .user ? .secondary : .blue
|
||||
let rowBackground: Color = event.role == .user
|
||||
? Color(.secondarySystemBackground)
|
||||
: Color.blue.opacity(0.08)
|
||||
let textOpacity: Double = event.isFinal ? 1.0 : 0.55
|
||||
return VStack(alignment: .leading, spacing: 2) {
|
||||
HStack {
|
||||
Text(roleName)
|
||||
.font(.caption2.bold())
|
||||
.foregroundStyle(nameColor)
|
||||
Spacer()
|
||||
Text(Self.timestampFormatter.string(from: event.timestamp))
|
||||
.font(.caption2)
|
||||
.foregroundStyle(.tertiary)
|
||||
}
|
||||
Text(stripped(event.text))
|
||||
.font(.body)
|
||||
.opacity(textOpacity)
|
||||
}
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(8)
|
||||
.background(rowBackground)
|
||||
.cornerRadius(10)
|
||||
.id(event.id)
|
||||
}
|
||||
|
||||
private func stripped(_ text: String) -> String {
|
||||
text.replacingOccurrences(
|
||||
of: Self.deliveryMarkers,
|
||||
with: "",
|
||||
options: .regularExpression
|
||||
).trimmingCharacters(in: .whitespaces)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,236 @@
|
||||
import SwiftUI
|
||||
|
||||
/// Nearlight transcript: a legible shared record. Full-width rows, no
|
||||
/// bubbles; metadata row per event; "Live" placeholder on partials.
|
||||
struct TranscriptListView: View {
|
||||
let events: [TranscriptEvent]
|
||||
var onClear: () -> Void
|
||||
@State private var confirmClear = false
|
||||
@State private var followingLatest = true
|
||||
@State private var selectionActive = false
|
||||
@State private var viewportHeight: CGFloat = 0
|
||||
@State private var latestBottom: CGFloat = 0
|
||||
@AccessibilityFocusState private var focusedEventID: String?
|
||||
@Environment(\.accessibilityReduceMotion) private var reduceMotion
|
||||
@Environment(\.dynamicTypeSize) private var typeSize
|
||||
|
||||
init(events: [TranscriptEvent], onClear: @escaping () -> Void) {
|
||||
self.events = events
|
||||
self.onClear = onClear
|
||||
_followingLatest = State(initialValue: events.isEmpty)
|
||||
}
|
||||
|
||||
/// Auto-follow eligibility: never scroll while the user is selecting
|
||||
/// text or reading history.
|
||||
private var isFollowingEligible: Bool {
|
||||
followingLatest && !selectionActive && focusedEventID == nil
|
||||
}
|
||||
private static let deliveryMarkers = #"\[(?:appreciative|calm|concerned|curious|encouraging|excited|happy|joking|quiet|reassuring|reflective|sad|serious|surprised|sympathetic|urgent|whispering|laughter|sighing|breathing|neutral)\]\s*"#
|
||||
|
||||
var body: some View {
|
||||
ScrollViewReader { proxy in
|
||||
VStack(spacing: 0) {
|
||||
HStack {
|
||||
Text("Conversation")
|
||||
.font(NearlightFont.support(typeSize).weight(.semibold))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
Spacer()
|
||||
Menu {
|
||||
Button("Clear conversation", systemImage: "trash", role: .destructive) { confirmClear = true }
|
||||
} label: {
|
||||
Image(systemName: "ellipsis").frame(width: 44, height: 44)
|
||||
}
|
||||
.accessibilityLabel("Conversation options")
|
||||
.disabled(events.isEmpty)
|
||||
}
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
ScrollView {
|
||||
LazyVStack(alignment: .leading, spacing: Nearlight.s24) {
|
||||
if events.isEmpty {
|
||||
Text("Your conversation will appear here.")
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.padding(.top, Nearlight.s24)
|
||||
}
|
||||
ForEach(events) { event in transcriptRow(event) }
|
||||
GeometryReader { geometry in
|
||||
Color.clear.preference(
|
||||
key: TranscriptLatestBottomKey.self,
|
||||
value: geometry.frame(in: .named("transcriptScroll")).maxY
|
||||
)
|
||||
}
|
||||
.frame(height: 1)
|
||||
.id("latest")
|
||||
}
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.padding(.top, Nearlight.s16)
|
||||
.padding(.bottom, Nearlight.s20)
|
||||
.frame(minHeight: 200, alignment: .topLeading)
|
||||
}
|
||||
.coordinateSpace(name: "transcriptScroll")
|
||||
.background {
|
||||
GeometryReader { geometry in
|
||||
Color.clear.preference(
|
||||
key: TranscriptViewportHeightKey.self,
|
||||
value: geometry.size.height
|
||||
)
|
||||
}
|
||||
}
|
||||
.scrollDismissesKeyboard(.interactively)
|
||||
.defaultScrollAnchor(.top)
|
||||
.simultaneousGesture(DragGesture().onChanged { _ in
|
||||
// Any manual drag suspends automatic following.
|
||||
followingLatest = false
|
||||
})
|
||||
.onChange(of: events.last?.text) { _, _ in
|
||||
guard isFollowingEligible else { return }
|
||||
proxy.scrollTo("latest", anchor: .bottom)
|
||||
}
|
||||
.onChange(of: events.count) { _, count in
|
||||
if count == 0 {
|
||||
selectionActive = false
|
||||
followingLatest = true
|
||||
} else if isFollowingEligible {
|
||||
proxy.scrollTo("latest", anchor: .bottom)
|
||||
}
|
||||
}
|
||||
.onChange(of: selectionActive) { _, active in
|
||||
// Beginning text selection suspends auto-follow so the
|
||||
// user's reading position is never disturbed.
|
||||
if active { followingLatest = false }
|
||||
}
|
||||
.onChange(of: focusedEventID) { _, eventID in
|
||||
// VoiceOver focus on any transcript row means the user is
|
||||
// reading history. Only Latest resumes automatic following.
|
||||
if eventID != nil { followingLatest = false }
|
||||
}
|
||||
.onPreferenceChange(TranscriptViewportHeightKey.self) { height in
|
||||
viewportHeight = height
|
||||
updateFollowingEligibility(bottom: latestBottom, viewport: height)
|
||||
}
|
||||
.onPreferenceChange(TranscriptLatestBottomKey.self) { bottom in
|
||||
latestBottom = bottom
|
||||
updateFollowingEligibility(bottom: bottom, viewport: viewportHeight)
|
||||
}
|
||||
if !followingLatest && !events.isEmpty {
|
||||
HStack {
|
||||
Spacer()
|
||||
Button {
|
||||
// Explicit Latest restores full eligibility,
|
||||
// clearing any stale selection suspension.
|
||||
selectionActive = false
|
||||
focusedEventID = nil
|
||||
followingLatest = true
|
||||
withAnimation(reduceMotion ? nil : Nearlight.standard) {
|
||||
proxy.scrollTo("latest", anchor: .bottom)
|
||||
}
|
||||
} label: {
|
||||
Label("Latest", systemImage: "arrow.down")
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.padding(.horizontal, Nearlight.s12)
|
||||
.frame(minHeight: 44)
|
||||
.background(Nearlight.surface, in: Capsule())
|
||||
.overlay(Capsule().strokeBorder(Nearlight.quiet, lineWidth: 1))
|
||||
}
|
||||
.accessibilityIdentifier("talk.latest")
|
||||
}
|
||||
.padding(.horizontal, Nearlight.s12)
|
||||
.padding(.bottom, Nearlight.s12)
|
||||
}
|
||||
}
|
||||
}
|
||||
.alert("Clear this conversation?", isPresented: $confirmClear) {
|
||||
Button("Clear conversation", role: .destructive, action: onClear)
|
||||
Button("Cancel", role: .cancel) {}
|
||||
} message: {
|
||||
Text("This removes the transcript saved on this iPhone. It doesn’t reset Hope’s memory or end your current conversation.")
|
||||
}
|
||||
}
|
||||
|
||||
private func updateFollowingEligibility(bottom: CGFloat, viewport: CGFloat) {
|
||||
guard bottom > 0,
|
||||
viewport > 0,
|
||||
!selectionActive,
|
||||
focusedEventID == nil else { return }
|
||||
let shouldFollow = bottom <= viewport + 48
|
||||
if followingLatest != shouldFollow {
|
||||
followingLatest = shouldFollow
|
||||
}
|
||||
}
|
||||
|
||||
private func transcriptRow(_ event: TranscriptEvent) -> some View {
|
||||
let isUser = event.role == .user
|
||||
let text = event.text.replacingOccurrences(of: Self.deliveryMarkers, with: "", options: .regularExpression)
|
||||
.trimmingCharacters(in: .whitespaces)
|
||||
return VStack(alignment: .leading, spacing: Nearlight.s8) {
|
||||
// Metadata row: speaker, timestamp trailing. Stacks at
|
||||
// accessibility sizes per spec.
|
||||
Group {
|
||||
if typeSize.isAccessibilitySize {
|
||||
VStack(alignment: .leading, spacing: Nearlight.s4) {
|
||||
speakerLabel(isUser)
|
||||
timeLabel(event)
|
||||
}
|
||||
} else {
|
||||
HStack(spacing: Nearlight.s8) {
|
||||
speakerLabel(isUser)
|
||||
timeLabel(event)
|
||||
}
|
||||
}
|
||||
}
|
||||
Text(text)
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.ink)
|
||||
.textSelection(.enabled)
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
}
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.onLongPressGesture(minimumDuration: 0.05, pressing: { pressing in
|
||||
// Text-selection interaction begins with long-press on the row;
|
||||
// suspend auto-follow while pressing/held and restore when the
|
||||
// press ends so following can resume.
|
||||
selectionActive = pressing
|
||||
}, perform: {})
|
||||
.contextMenu {
|
||||
Button("Copy message", systemImage: "doc.on.doc") { UIPasteboard.general.string = text }
|
||||
}
|
||||
.accessibilityElement(children: .combine)
|
||||
.accessibilityFocused($focusedEventID, equals: event.id)
|
||||
.id(event.id)
|
||||
}
|
||||
|
||||
private func speakerLabel(_ isUser: Bool) -> some View {
|
||||
Text(isUser ? "You" : "Hope")
|
||||
.font(NearlightFont.metadata(typeSize).weight(.semibold))
|
||||
.foregroundStyle(Nearlight.ink)
|
||||
}
|
||||
|
||||
private func timeLabel(_ event: TranscriptEvent) -> some View {
|
||||
Group {
|
||||
if event.isFinal {
|
||||
Text(event.timestamp, style: .time)
|
||||
} else {
|
||||
Text("Live")
|
||||
}
|
||||
}
|
||||
.font(NearlightFont.metadata(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.monospacedDigit()
|
||||
}
|
||||
}
|
||||
|
||||
private struct TranscriptViewportHeightKey: PreferenceKey {
|
||||
static var defaultValue: CGFloat = 0
|
||||
|
||||
static func reduce(value: inout CGFloat, nextValue: () -> CGFloat) {
|
||||
value = nextValue()
|
||||
}
|
||||
}
|
||||
|
||||
private struct TranscriptLatestBottomKey: PreferenceKey {
|
||||
static var defaultValue: CGFloat = 0
|
||||
|
||||
static func reduce(value: inout CGFloat, nextValue: () -> CGFloat) {
|
||||
value = nextValue()
|
||||
}
|
||||
}
|
||||
@@ -1,516 +0,0 @@
|
||||
import SwiftUI
|
||||
|
||||
struct MediaPanelView: View {
|
||||
@ObservedObject var playerController: MediaPlayerController
|
||||
@ObservedObject var settings: SettingsStore
|
||||
|
||||
@State private var searchQuery = ""
|
||||
@State private var searchResults: [MediaSearchResult]?
|
||||
@State private var isSearching = false
|
||||
@State private var searchError: String?
|
||||
@State private var libraryTracks: [MediaTrack]?
|
||||
@State private var libraryError: String?
|
||||
@State private var streamName = ""
|
||||
@State private var streamURL = ""
|
||||
@State private var streamMessage: String?
|
||||
@State private var trackToDelete: MediaTrack?
|
||||
|
||||
var body: some View {
|
||||
ScrollView {
|
||||
VStack(spacing: 20) {
|
||||
if let stateError = playerController.stateError {
|
||||
Label(stateError, systemImage: "wifi.exclamationmark")
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.red)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(12)
|
||||
.background(Color.red.opacity(0.1), in: RoundedRectangle(cornerRadius: 14))
|
||||
}
|
||||
nowPlayingCard
|
||||
transportControls
|
||||
seekSection
|
||||
volumeSection
|
||||
pendingSection
|
||||
choicesSection
|
||||
searchSection
|
||||
librarySection
|
||||
addStreamSection
|
||||
}
|
||||
.padding()
|
||||
}
|
||||
.navigationTitle("Media")
|
||||
.onAppear {
|
||||
playerController.startDefaultPolling()
|
||||
refreshLibrary()
|
||||
}
|
||||
.alert("Remove from library?", isPresented: Binding(
|
||||
get: { trackToDelete != nil },
|
||||
set: { if !$0 { trackToDelete = nil } }
|
||||
)) {
|
||||
Button("Remove", role: .destructive) {
|
||||
guard let track = trackToDelete else { return }
|
||||
Task { await deleteTrack(track) }
|
||||
}
|
||||
Button("Cancel", role: .cancel) { trackToDelete = nil }
|
||||
} message: {
|
||||
Text(trackToDelete?.title ?? "")
|
||||
}
|
||||
}
|
||||
|
||||
private var nowPlayingCard: some View {
|
||||
HStack(spacing: 12) {
|
||||
if let artURL = playerController.track?.artURL, let url = URL(string: absolute(artURL)) {
|
||||
AsyncImage(url: url) { phase in
|
||||
switch phase {
|
||||
case .success(let image):
|
||||
image.resizable().scaledToFill()
|
||||
default:
|
||||
Image(systemName: "music.note")
|
||||
.font(.title2)
|
||||
.foregroundStyle(.secondary)
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
.background(Color(.systemGray6))
|
||||
}
|
||||
}
|
||||
.frame(width: 72, height: 72)
|
||||
.clipShape(RoundedRectangle(cornerRadius: 10))
|
||||
} else {
|
||||
RoundedRectangle(cornerRadius: 10)
|
||||
.fill(Color(.systemGray6))
|
||||
.frame(width: 72, height: 72)
|
||||
.overlay(
|
||||
Image(systemName: "music.note")
|
||||
.font(.title2)
|
||||
.foregroundStyle(.secondary)
|
||||
)
|
||||
}
|
||||
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
Text(title)
|
||||
.font(.headline)
|
||||
.lineLimit(2)
|
||||
Text(subtitle)
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.secondary)
|
||||
.lineLimit(1)
|
||||
if playerController.track != nil {
|
||||
Text(statusText)
|
||||
.font(.caption)
|
||||
.foregroundStyle(playerController.desired == "playing" ? Color.green : Color.secondary)
|
||||
}
|
||||
}
|
||||
Spacer()
|
||||
}
|
||||
.padding(12)
|
||||
.background(Color(.secondarySystemBackground), in: RoundedRectangle(cornerRadius: 14))
|
||||
}
|
||||
|
||||
private var transportControls: some View {
|
||||
HStack(spacing: 24) {
|
||||
Button(action: seekBackward) {
|
||||
Image(systemName: "gobackward.10")
|
||||
.font(.title3)
|
||||
}
|
||||
.accessibilityLabel("Seek back 10 seconds")
|
||||
.disabled(playerController.track == nil)
|
||||
|
||||
Button(action: togglePlayPause) {
|
||||
Image(systemName: playerController.desired == "playing" && playerController.track != nil ? "pause.circle.fill" : "play.circle.fill")
|
||||
.font(.system(size: 44))
|
||||
.foregroundStyle(playerController.track == nil ? Color.secondary : Color.accentColor)
|
||||
}
|
||||
.accessibilityLabel(playerController.desired == "playing" && playerController.track != nil ? "Pause" : "Play")
|
||||
.disabled(playerController.track == nil)
|
||||
|
||||
Button(action: seekForward) {
|
||||
Image(systemName: "goforward.10")
|
||||
.font(.title3)
|
||||
}
|
||||
.accessibilityLabel("Seek forward 10 seconds")
|
||||
.disabled(playerController.track == nil)
|
||||
|
||||
Button(role: .destructive, action: stopPlayback) {
|
||||
Image(systemName: "stop.circle")
|
||||
.font(.title2)
|
||||
}
|
||||
.accessibilityLabel("Stop playback")
|
||||
.disabled(playerController.track == nil)
|
||||
}
|
||||
}
|
||||
|
||||
private var seekSection: some View {
|
||||
VStack(spacing: 4) {
|
||||
if playerController.track != nil {
|
||||
Slider(
|
||||
value: Binding(
|
||||
get: { playerController.currentTime },
|
||||
set: { newValue in Task { playerController.seek(to: newValue) } }
|
||||
),
|
||||
in: 0...max(playerController.duration, 1)
|
||||
)
|
||||
HStack {
|
||||
Text(fmtTime(playerController.currentTime))
|
||||
.font(.caption.monospacedDigit())
|
||||
.foregroundStyle(.secondary)
|
||||
Spacer()
|
||||
Text(fmtTime(playerController.duration))
|
||||
.font(.caption.monospacedDigit())
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
} else {
|
||||
Text("Nothing loaded")
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.secondary)
|
||||
.frame(maxWidth: .infinity, alignment: .center)
|
||||
.padding(.vertical, 8)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var volumeSection: some View {
|
||||
HStack(spacing: 12) {
|
||||
Image(systemName: "speaker.fill")
|
||||
.foregroundStyle(.secondary)
|
||||
Slider(value: $settings.volume, in: 0...1)
|
||||
.onChange(of: settings.volume) { _, newValue in
|
||||
playerController.setVolume(newValue)
|
||||
}
|
||||
Image(systemName: "speaker.wave.3.fill")
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var pendingSection: some View {
|
||||
if !playerController.pendingJobs.isEmpty {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
ForEach(playerController.pendingJobs, id: \.jobID) { job in
|
||||
HStack {
|
||||
Text(job.stage == "queued" ? "Queued…" : "Downloading…")
|
||||
.font(.subheadline)
|
||||
Spacer()
|
||||
Text("\(Int((job.progress ?? 0)))%")
|
||||
.font(.caption.monospacedDigit())
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
}
|
||||
}
|
||||
.padding(12)
|
||||
.background(Color(.secondarySystemBackground), in: RoundedRectangle(cornerRadius: 14))
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var choicesSection: some View {
|
||||
if !playerController.choices.isEmpty {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
Text("Hope's suggestions")
|
||||
.font(.headline)
|
||||
ForEach(playerController.choices, id: \.videoID) { choice in
|
||||
mediaRow(
|
||||
title: choice.title ?? choice.videoID ?? "Unknown",
|
||||
detail: rowDetail(channel: choice.channel, duration: choice.duration),
|
||||
playEnabled: true
|
||||
) {
|
||||
Task {
|
||||
await playerController.playVideo(
|
||||
videoID: choice.videoID,
|
||||
title: choice.title,
|
||||
channel: choice.channel
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var searchSection: some View {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
Text("Search YouTube Music")
|
||||
.font(.headline)
|
||||
HStack(spacing: 8) {
|
||||
TextField("Song, artist, lyrics…", text: $searchQuery)
|
||||
.textFieldStyle(.roundedBorder)
|
||||
.submitLabel(.search)
|
||||
.onSubmit(performSearch)
|
||||
Button("Find", action: performSearch)
|
||||
.buttonStyle(.bordered)
|
||||
.disabled(searchQuery.trimmingCharacters(in: .whitespaces).isEmpty || isSearching)
|
||||
}
|
||||
if isSearching {
|
||||
HStack(spacing: 8) {
|
||||
ProgressView()
|
||||
Text("Searching YouTube Music…")
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
} else if let searchError {
|
||||
Text(searchError)
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.red)
|
||||
} else if let results = searchResults {
|
||||
if results.isEmpty {
|
||||
Text("No results.")
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.secondary)
|
||||
} else {
|
||||
ForEach(results, id: \.videoID) { result in
|
||||
mediaRow(
|
||||
title: result.title ?? result.videoID ?? "Unknown",
|
||||
detail: rowDetail(channel: result.channel, duration: result.duration, inLibrary: result.inLibrary),
|
||||
playEnabled: true
|
||||
) {
|
||||
Task {
|
||||
if let trackID = result.trackID {
|
||||
await playerController.playTrack(id: trackID)
|
||||
} else {
|
||||
await playerController.playVideo(
|
||||
videoID: result.videoID,
|
||||
title: result.title,
|
||||
channel: result.channel
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var librarySection: some View {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
HStack {
|
||||
Text("Library")
|
||||
.font(.headline)
|
||||
Spacer()
|
||||
Button(action: refreshLibrary) {
|
||||
Image(systemName: "arrow.clockwise")
|
||||
}
|
||||
.accessibilityLabel("Refresh library")
|
||||
}
|
||||
if let libraryError {
|
||||
Text(libraryError)
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.red)
|
||||
} else if let tracks = libraryTracks {
|
||||
if tracks.isEmpty {
|
||||
Text("Downloaded songs and streams live here.")
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.secondary)
|
||||
} else {
|
||||
ForEach(tracks, id: \.id) { track in
|
||||
HStack(spacing: 8) {
|
||||
mediaRow(
|
||||
title: track.title ?? track.id,
|
||||
detail: rowDetail(
|
||||
channel: track.channel ?? (track.kind == "stream" ? "Live stream" : ""),
|
||||
duration: track.duration,
|
||||
status: track.status
|
||||
),
|
||||
playEnabled: track.status == "ready"
|
||||
) {
|
||||
Task { await playerController.playTrack(id: track.id) }
|
||||
}
|
||||
Button(role: .destructive) {
|
||||
trackToDelete = track
|
||||
} label: {
|
||||
Image(systemName: "xmark")
|
||||
.font(.subheadline.weight(.semibold))
|
||||
.foregroundStyle(.red)
|
||||
}
|
||||
.accessibilityLabel("Remove \(track.title ?? track.id)")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var addStreamSection: some View {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
Text("Add live stream")
|
||||
.font(.headline)
|
||||
TextField("Station name", text: $streamName)
|
||||
.textFieldStyle(.roundedBorder)
|
||||
TextField("https://… audio stream URL", text: $streamURL)
|
||||
.textFieldStyle(.roundedBorder)
|
||||
.keyboardType(.URL)
|
||||
.autocorrectionDisabled()
|
||||
.textContentType(.URL)
|
||||
.textInputAutocapitalization(.never)
|
||||
if let streamMessage {
|
||||
Text(streamMessage)
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(streamMessage.hasPrefix("Added") ? Color.green : Color.red)
|
||||
}
|
||||
Button("Add", action: addStream)
|
||||
.buttonStyle(.bordered)
|
||||
.disabled(streamURL.trimmingCharacters(in: .whitespaces).isEmpty)
|
||||
}
|
||||
}
|
||||
|
||||
private func mediaRow(
|
||||
title: String,
|
||||
detail: String,
|
||||
playEnabled: Bool,
|
||||
onPlay: @escaping () -> Void
|
||||
) -> some View {
|
||||
HStack(spacing: 10) {
|
||||
VStack(alignment: .leading, spacing: 2) {
|
||||
Text(title)
|
||||
.font(.subheadline.weight(.semibold))
|
||||
.lineLimit(1)
|
||||
if !detail.isEmpty {
|
||||
Text(detail)
|
||||
.font(.caption)
|
||||
.foregroundStyle(.secondary)
|
||||
.lineLimit(1)
|
||||
}
|
||||
}
|
||||
Spacer()
|
||||
Button("Play", action: onPlay)
|
||||
.buttonStyle(.bordered)
|
||||
.controlSize(.small)
|
||||
.disabled(!playEnabled)
|
||||
}
|
||||
.padding(10)
|
||||
.background(Color(.secondarySystemBackground), in: RoundedRectangle(cornerRadius: 12))
|
||||
}
|
||||
|
||||
private func rowDetail(channel: String?, duration: Double?, status: String? = nil, inLibrary: Bool? = nil) -> String {
|
||||
var parts: [String] = []
|
||||
if let channel, !channel.isEmpty { parts.append(channel) }
|
||||
if let duration, duration > 0 { parts.append(fmtTime(duration)) }
|
||||
if let status, status != "ready" { parts.append(status) }
|
||||
if inLibrary == true { parts.append("in library") }
|
||||
return parts.joined(separator: " · ")
|
||||
}
|
||||
|
||||
private func performSearch() {
|
||||
let query = searchQuery.trimmingCharacters(in: .whitespaces)
|
||||
guard !query.isEmpty else { return }
|
||||
isSearching = true
|
||||
searchError = nil
|
||||
Task {
|
||||
do {
|
||||
let response = try await MediaAPI(config: serverConfig).search(room: "default", query: query, limit: 6)
|
||||
searchResults = response.results
|
||||
} catch {
|
||||
searchError = error.localizedDescription
|
||||
}
|
||||
isSearching = false
|
||||
}
|
||||
}
|
||||
|
||||
private func refreshLibrary() {
|
||||
Task {
|
||||
do {
|
||||
let library = try await MediaAPI(config: serverConfig).library()
|
||||
libraryTracks = library.tracks
|
||||
libraryError = nil
|
||||
} catch {
|
||||
libraryError = error.localizedDescription
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func addStream() {
|
||||
let url = streamURL.trimmingCharacters(in: .whitespaces)
|
||||
guard !url.isEmpty else { return }
|
||||
let name = streamName.trimmingCharacters(in: .whitespaces).isEmpty ? url : streamName.trimmingCharacters(in: .whitespaces)
|
||||
Task {
|
||||
do {
|
||||
try await MediaAPI(config: serverConfig).addStream(room: "default", name: name, url: url)
|
||||
streamMessage = "Added \(name)"
|
||||
streamName = ""
|
||||
streamURL = ""
|
||||
refreshLibrary()
|
||||
} catch {
|
||||
streamMessage = "Could not add stream: \(error.localizedDescription)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func deleteTrack(_ track: MediaTrack) async {
|
||||
do {
|
||||
try await MediaAPI(config: serverConfig).deleteTrack(id: track.id)
|
||||
if playerController.track?.id == track.id {
|
||||
Task { await playerController.stop() }
|
||||
}
|
||||
refreshLibrary()
|
||||
} catch {
|
||||
libraryError = error.localizedDescription
|
||||
}
|
||||
}
|
||||
|
||||
private var serverConfig: ServerConfig {
|
||||
// Use the configured URL directly. If it's invalid, fall back to the
|
||||
// documented default (media operations are less critical than the main
|
||||
// session, and the Settings view will show a validation error).
|
||||
if case .success(let config) = ServerConfig.parse(settings.trimmedServerURL) {
|
||||
return config
|
||||
}
|
||||
let fallback = ServerConfig.parse("https://192.168.86.2:9178")
|
||||
if case .success(let config) = fallback {
|
||||
return config
|
||||
}
|
||||
fatalError("Unreachable: default server URL is valid")
|
||||
}
|
||||
|
||||
private var title: String {
|
||||
playerController.track?.title ?? "Nothing loaded"
|
||||
}
|
||||
|
||||
private var subtitle: String {
|
||||
guard let track = playerController.track else { return "" }
|
||||
if track.status == "failed" {
|
||||
return "Download failed — pick another result or try again."
|
||||
}
|
||||
if track.status == "downloading" {
|
||||
return track.channel.map { "\($0) · downloading" } ?? "Downloading…"
|
||||
}
|
||||
return track.channel ?? (track.kind == "stream" ? "Live stream" : "")
|
||||
}
|
||||
|
||||
private var statusText: String {
|
||||
switch playerController.desired {
|
||||
case "playing": return "Playing"
|
||||
case "paused": return "Paused"
|
||||
default: return "Stopped"
|
||||
}
|
||||
}
|
||||
|
||||
private func absolute(_ path: String) -> String {
|
||||
if path.hasPrefix("http") { return path }
|
||||
let base = serverConfig.baseURL.absoluteString
|
||||
let suffix = path.hasPrefix("/") ? path : "/" + path
|
||||
return base + suffix
|
||||
}
|
||||
|
||||
private func togglePlayPause() {
|
||||
Task { await playerController.togglePlayPause() }
|
||||
}
|
||||
|
||||
private func seekBackward() {
|
||||
Task { await playerController.seek(by: -10) }
|
||||
}
|
||||
|
||||
private func seekForward() {
|
||||
Task { await playerController.seek(by: 10) }
|
||||
}
|
||||
|
||||
private func stopPlayback() {
|
||||
Task { await playerController.stop() }
|
||||
}
|
||||
|
||||
private func fmtTime(_ sec: Double) -> String {
|
||||
guard sec.isFinite, sec >= 0 else { return "0:00" }
|
||||
let total = Int(sec)
|
||||
let s = total % 60
|
||||
let m = (total / 60) % 60
|
||||
let h = total / 3600
|
||||
return h > 0 ? "\(h):\(String(format: "%02d", m)):\(String(format: "%02d", s))" : "\(m):\(String(format: "%02d", s))"
|
||||
}
|
||||
}
|
||||
@@ -26,7 +26,7 @@ struct ConnectionTestView: View {
|
||||
if isBusy {
|
||||
ProgressView()
|
||||
} else {
|
||||
Text("Get token")
|
||||
Text("Test connection")
|
||||
}
|
||||
Spacer()
|
||||
}
|
||||
@@ -44,7 +44,7 @@ struct ConnectionTestView: View {
|
||||
LabeledContent("User") {
|
||||
Text(response.user.displayName ?? response.user.id)
|
||||
}
|
||||
Label("Token endpoint and TLS are working.", systemImage: "checkmark.circle")
|
||||
Label("Connected to your server. Start a conversation to check Hope’s voice and replies.", systemImage: "checkmark.circle")
|
||||
.foregroundStyle(.green)
|
||||
}
|
||||
case .failure(let error):
|
||||
@@ -57,6 +57,7 @@ struct ConnectionTestView: View {
|
||||
}
|
||||
}
|
||||
.navigationTitle("Connection test")
|
||||
.navigationBarTitleDisplayMode(.inline)
|
||||
}
|
||||
|
||||
private func runTest() {
|
||||
|
||||
@@ -12,8 +12,9 @@ struct SettingsView: View {
|
||||
@State private var urlText: String = ""
|
||||
@State private var urlError: String?
|
||||
@State private var showVoicePicker = false
|
||||
@State private var savedURLConfirmation: String?
|
||||
@State private var showModelPicker = false
|
||||
|
||||
@FocusState private var urlFocused: Bool
|
||||
init(settings: SettingsStore, session: VoiceSession) {
|
||||
self.settings = settings
|
||||
self.session = session
|
||||
@@ -30,56 +31,92 @@ struct SettingsView: View {
|
||||
|
||||
var body: some View {
|
||||
Form {
|
||||
Section("Server") {
|
||||
Section {
|
||||
Button {
|
||||
showVoicePicker = true
|
||||
} label: {
|
||||
LabeledContent("Speaking voice", value: Voices.display(for: settings.voiceID) ?? settings.voiceID)
|
||||
}
|
||||
Picker("Conversation mode", selection: $settings.textMode) {
|
||||
Text("Voice").tag(false)
|
||||
Text("Type · Microphone off").tag(true)
|
||||
}
|
||||
} header: {
|
||||
Text("Make it yours")
|
||||
} footer: {
|
||||
Text("Voice and typing preferences are saved on this iPhone. You can switch modes from the Talk tab.")
|
||||
}
|
||||
Section {
|
||||
if session.isReady, !session.models.isEmpty {
|
||||
Button {
|
||||
showModelPicker = true
|
||||
} label: {
|
||||
LabeledContent("Model", value: session.selectedModel ?? settings.modelID ?? "Server default")
|
||||
}
|
||||
} else {
|
||||
LabeledContent("Model", value: settings.modelID ?? "Server default")
|
||||
Text("Connect from Talk to see the models available on your server.")
|
||||
.font(.footnote).foregroundStyle(Nearlight.quiet)
|
||||
}
|
||||
} header: { Text("Intelligence") }
|
||||
Section {
|
||||
Text("Server URL")
|
||||
.font(.footnote.weight(.semibold))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
TextField(SettingsStore.defaultServerURL, text: $urlText)
|
||||
.keyboardType(.URL)
|
||||
.autocorrectionDisabled()
|
||||
.textInputAutocapitalization(.never)
|
||||
.accessibilityLabel("Server URL")
|
||||
.focused($urlFocused)
|
||||
.onAppear { urlText = settings.serverURL }
|
||||
.onChange(of: urlText) { _, newValue in
|
||||
// Keep confirmation across the asynchronous onChange
|
||||
// caused by successful whitespace normalization.
|
||||
guard newValue != savedURLConfirmation else { return }
|
||||
savedURLConfirmation = nil
|
||||
urlError = nil
|
||||
}
|
||||
.overlay(alignment: .trailing) {
|
||||
if savedURLConfirmation != nil {
|
||||
Image(systemName: "checkmark.circle")
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.padding(.trailing, 8)
|
||||
.accessibilityHidden(true)
|
||||
}
|
||||
}
|
||||
if let urlError {
|
||||
Text(urlError)
|
||||
Label(urlError, systemImage: "exclamationmark.circle")
|
||||
.font(.footnote)
|
||||
.foregroundStyle(.red)
|
||||
.foregroundStyle(Nearlight.alert)
|
||||
.accessibilityIdentifier("settings.server.error")
|
||||
} else if savedURLConfirmation != nil {
|
||||
Text("Server address saved.")
|
||||
.font(.footnote)
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.accessibilityIdentifier("settings.server.saved")
|
||||
}
|
||||
Button("Apply server URL") { applyServerURL() }
|
||||
.disabled(!canApplyServerURL || urlText.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty)
|
||||
Button("Save server address") { applyServerURL() }
|
||||
.disabled(!canApplyServerURL || urlText.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty || urlText == settings.serverURL)
|
||||
if !canApplyServerURL {
|
||||
Text(session.state == .tearingDown
|
||||
? "Waiting for failure cleanup to finish…"
|
||||
: "End the conversation before changing servers.")
|
||||
.font(.footnote)
|
||||
.foregroundStyle(.secondary)
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
}
|
||||
}
|
||||
Section("Voice") {
|
||||
Button {
|
||||
showVoicePicker = true
|
||||
} label: {
|
||||
LabeledContent("Voice", value: Voices.display(for: settings.voiceID) ?? settings.voiceID)
|
||||
}
|
||||
}
|
||||
Section("Model") {
|
||||
if session.isReady, !session.models.isEmpty {
|
||||
Button {
|
||||
showModelPicker = true
|
||||
} label: {
|
||||
LabeledContent("Model", value: session.selectedModel ?? settings.modelID ?? "—")
|
||||
}
|
||||
} else {
|
||||
Text("Start a conversation to load the model list.")
|
||||
.font(.footnote)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
}
|
||||
Section("Interaction") {
|
||||
Toggle("Text mode", isOn: $settings.textMode)
|
||||
}
|
||||
Section {
|
||||
NavigationLink("Connection test") {
|
||||
ConnectionTestView(settings: settings)
|
||||
}
|
||||
} header: {
|
||||
Text("Connection")
|
||||
} footer: {
|
||||
Text("Hope runs on your own server. Your iPhone needs to be on the same network or a network that can reach it.")
|
||||
}
|
||||
}
|
||||
.navigationTitle("Settings")
|
||||
.navigationBarTitleDisplayMode(.inline)
|
||||
.tint(Nearlight.quiet)
|
||||
.sheet(isPresented: $showVoicePicker) {
|
||||
VoicePickerView(settings: settings, session: session)
|
||||
}
|
||||
@@ -93,6 +130,7 @@ struct SettingsView: View {
|
||||
/// left untouched so the active session keeps using its original URL.
|
||||
/// Guarded: only applies when the session is idle (no active call).
|
||||
private func applyServerURL() {
|
||||
savedURLConfirmation = nil
|
||||
guard canApplyServerURL else {
|
||||
urlError = "End the conversation before changing servers."
|
||||
return
|
||||
@@ -107,15 +145,18 @@ struct SettingsView: View {
|
||||
settings.serverURL = trimmed
|
||||
urlText = trimmed
|
||||
urlError = nil
|
||||
savedURLConfirmation = trimmed
|
||||
urlFocused = false
|
||||
case .failure(let error):
|
||||
urlError = error.localizedDescription
|
||||
urlFocused = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct VoicePickerView: View {
|
||||
@ObservedObject var settings: SettingsStore
|
||||
var session: VoiceSession
|
||||
@ObservedObject var session: VoiceSession
|
||||
@Environment(\.dismiss) private var dismiss
|
||||
|
||||
var body: some View {
|
||||
@@ -132,10 +173,11 @@ struct VoicePickerView: View {
|
||||
if settings.voiceID == voice.id {
|
||||
Spacer()
|
||||
Image(systemName: "checkmark")
|
||||
.foregroundStyle(.blue)
|
||||
.foregroundStyle(Nearlight.rosewood)
|
||||
}
|
||||
}
|
||||
}
|
||||
.accessibilityAddTraits(settings.voiceID == voice.id ? .isSelected : [])
|
||||
}
|
||||
}
|
||||
.navigationTitle("Voice")
|
||||
@@ -150,7 +192,7 @@ struct VoicePickerView: View {
|
||||
|
||||
struct ModelPickerView: View {
|
||||
@ObservedObject var settings: SettingsStore
|
||||
var session: VoiceSession
|
||||
@ObservedObject var session: VoiceSession
|
||||
@Environment(\.dismiss) private var dismiss
|
||||
|
||||
private var activeModelID: String? {
|
||||
@@ -176,10 +218,11 @@ struct ModelPickerView: View {
|
||||
if activeModelID == model.id {
|
||||
Spacer()
|
||||
Image(systemName: "checkmark")
|
||||
.foregroundStyle(.blue)
|
||||
.foregroundStyle(Nearlight.rosewood)
|
||||
}
|
||||
}
|
||||
}
|
||||
.accessibilityAddTraits(activeModelID == model.id ? .isSelected : [])
|
||||
}
|
||||
}
|
||||
.navigationTitle("Model")
|
||||
|
||||
@@ -1,137 +1,265 @@
|
||||
import SwiftUI
|
||||
|
||||
/// Nearlight Activity: a working log. Plain rows on Ground, no cards;
|
||||
/// live updates land in place without reordering while reading.
|
||||
struct TaskPanelView: View {
|
||||
@ObservedObject var taskStore: TaskStore
|
||||
@Environment(\.dynamicTypeSize) private var typeSize
|
||||
|
||||
var body: some View {
|
||||
Group {
|
||||
if taskStore.sortedTasks.isEmpty {
|
||||
emptyState
|
||||
} else {
|
||||
taskList
|
||||
}
|
||||
}
|
||||
.navigationTitle("Activity")
|
||||
.background(Nearlight.ground)
|
||||
}
|
||||
|
||||
private var taskList: some View {
|
||||
ScrollView {
|
||||
VStack(spacing: 12) {
|
||||
if taskStore.sortedTasks.isEmpty {
|
||||
emptyState
|
||||
} else {
|
||||
ForEach(taskStore.sortedTasks) { task in
|
||||
TaskCardView(task: task)
|
||||
LazyVStack(alignment: .leading, spacing: 0) {
|
||||
ForEach(taskStore.sortedTasks) { task in
|
||||
NavigationLink {
|
||||
TaskDetailView(taskStore: taskStore, taskID: task.id)
|
||||
} label: {
|
||||
TaskRowView(task: task)
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
Divider()
|
||||
.overlay(Nearlight.separator)
|
||||
}
|
||||
}
|
||||
.padding()
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
}
|
||||
.navigationTitle("Background tasks")
|
||||
#if DEBUG
|
||||
.toolbar {
|
||||
ToolbarItem(placement: .topBarTrailing) {
|
||||
Button("Sample", action: { taskStore.injectSampleEvents() })
|
||||
.font(.caption)
|
||||
}
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
private var emptyState: some View {
|
||||
VStack(spacing: 8) {
|
||||
Text("Nothing running")
|
||||
.font(.headline)
|
||||
Text("Ask Hope to research or compare something.")
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.secondary)
|
||||
.multilineTextAlignment(.center)
|
||||
VStack(alignment: .leading, spacing: Nearlight.s8) {
|
||||
Image(systemName: "checklist")
|
||||
.font(.system(size: 28))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
Text("No tasks yet")
|
||||
.font(NearlightFont.heading(typeSize))
|
||||
Text("Tasks from your conversations will appear here.")
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
}
|
||||
.frame(maxWidth: .infinity)
|
||||
.padding(.vertical, 48)
|
||||
.padding(.top, Nearlight.s32)
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.accessibilityIdentifier("activity.empty")
|
||||
}
|
||||
}
|
||||
|
||||
struct TaskCardView: View {
|
||||
/// One activity row: status symbol, title, current step/status, chevron.
|
||||
private struct TaskRowView: View {
|
||||
let task: TaskItem
|
||||
@Environment(\.dynamicTypeSize) private var typeSize
|
||||
|
||||
var body: some View {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
HStack(spacing: 8) {
|
||||
statusDot
|
||||
HStack(spacing: Nearlight.s12) {
|
||||
Image(systemName: statusSymbol)
|
||||
.font(.system(size: 20))
|
||||
.foregroundStyle(statusColor)
|
||||
.accessibilityHidden(true)
|
||||
VStack(alignment: .leading, spacing: Nearlight.s4) {
|
||||
Text(statusTitle)
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(statusColor)
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
Text(task.description)
|
||||
.font(.subheadline.weight(.semibold))
|
||||
.lineLimit(2)
|
||||
Spacer()
|
||||
.font(NearlightFont.control(typeSize))
|
||||
.foregroundStyle(Nearlight.ink)
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
if let latest = latestStep {
|
||||
Text(latest)
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
}
|
||||
Text(task.createdAt, style: .time)
|
||||
.font(NearlightFont.metadata(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.monospacedDigit()
|
||||
}
|
||||
if !task.steps.isEmpty {
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
ForEach(task.steps) { step in
|
||||
HStack(alignment: .top, spacing: 6) {
|
||||
Text(stepLabel(step.role))
|
||||
.font(.caption2.weight(.semibold))
|
||||
.foregroundStyle(stepColor(step.role))
|
||||
Text(step.content ?? "")
|
||||
.font(.caption)
|
||||
.foregroundStyle(stepColor(step.role))
|
||||
.lineLimit(3)
|
||||
Spacer(minLength: 0)
|
||||
Image(systemName: "chevron.right")
|
||||
.font(.system(size: 13, weight: .medium))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.accessibilityHidden(true)
|
||||
}
|
||||
.padding(.vertical, Nearlight.s16)
|
||||
.contentShape(Rectangle())
|
||||
}
|
||||
|
||||
private var statusTitle: String {
|
||||
switch task.status {
|
||||
case .pending: return "Queued"
|
||||
case .running: return "In progress"
|
||||
case .completed: return "Complete"
|
||||
case .failed: return "Needs attention"
|
||||
}
|
||||
}
|
||||
|
||||
private var latestStep: String? {
|
||||
if let error = task.error { return error }
|
||||
if let result = task.result { return result }
|
||||
return task.steps.last?.content
|
||||
}
|
||||
|
||||
private var statusSymbol: String {
|
||||
switch task.status {
|
||||
case .pending: return "circle.dotted"
|
||||
case .running: return "arrow.triangle.2.circlepath"
|
||||
case .completed: return "checkmark.circle"
|
||||
case .failed: return "exclamationmark.circle"
|
||||
}
|
||||
}
|
||||
|
||||
private var statusColor: Color {
|
||||
switch task.status {
|
||||
case .pending: return Nearlight.quiet
|
||||
case .running: return Nearlight.rosewood
|
||||
case .completed: return Nearlight.quiet
|
||||
case .failed: return Nearlight.alert
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Native push detail for one task: title, status, steps, result/error.
|
||||
/// Looks the task up live in the store so removal shows the missing state.
|
||||
struct TaskDetailView: View {
|
||||
@ObservedObject var taskStore: TaskStore
|
||||
let taskID: String
|
||||
@Environment(\.dynamicTypeSize) private var typeSize
|
||||
|
||||
private var task: TaskItem? {
|
||||
taskStore.tasks[taskID]
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
Group {
|
||||
if let task {
|
||||
TaskDetailContent(task: task)
|
||||
} else {
|
||||
Text("This task is no longer available.")
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.padding(Nearlight.s20)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
}
|
||||
.background(Nearlight.ground)
|
||||
.navigationTitle("Task")
|
||||
.navigationBarTitleDisplayMode(.inline)
|
||||
}
|
||||
}
|
||||
|
||||
private struct TaskDetailContent: View {
|
||||
let task: TaskItem
|
||||
@Environment(\.dynamicTypeSize) private var typeSize
|
||||
|
||||
var body: some View {
|
||||
ScrollView {
|
||||
VStack(alignment: .leading, spacing: Nearlight.s24) {
|
||||
VStack(alignment: .leading, spacing: Nearlight.s12) {
|
||||
Text(task.description)
|
||||
.font(NearlightFont.heading(typeSize))
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
Label(statusTitle, systemImage: statusSymbol)
|
||||
.font(NearlightFont.support(typeSize))
|
||||
.foregroundStyle(statusColor)
|
||||
Text(task.createdAt, style: .time)
|
||||
.font(NearlightFont.metadata(typeSize))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
.monospacedDigit()
|
||||
}
|
||||
VStack(alignment: .leading, spacing: 0) {
|
||||
Text("Steps")
|
||||
.font(NearlightFont.heading(typeSize))
|
||||
.padding(.bottom, Nearlight.s8)
|
||||
if task.steps.isEmpty {
|
||||
Text("No steps reported yet.")
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
} else {
|
||||
ForEach(task.steps) { step in
|
||||
VStack(alignment: .leading, spacing: Nearlight.s4) {
|
||||
if let role = step.role {
|
||||
Text(roleLabel(role))
|
||||
.font(NearlightFont.support(typeSize).weight(.semibold))
|
||||
.foregroundStyle(Nearlight.quiet)
|
||||
}
|
||||
Text(step.content ?? "")
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.ink)
|
||||
.textSelection(.enabled)
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
}
|
||||
.padding(.vertical, Nearlight.s16)
|
||||
}
|
||||
}
|
||||
}
|
||||
if let result = task.result {
|
||||
Text(result)
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.ink)
|
||||
.textSelection(.enabled)
|
||||
}
|
||||
if let error = task.error {
|
||||
Text(error)
|
||||
.font(.body)
|
||||
.foregroundStyle(Nearlight.alert)
|
||||
.textSelection(.enabled)
|
||||
}
|
||||
}
|
||||
if let result = task.result {
|
||||
block(result, color: .green, background: Color.green.opacity(0.12))
|
||||
}
|
||||
if let error = task.error {
|
||||
block(error, color: .red, background: Color.red.opacity(0.12))
|
||||
}
|
||||
.padding(.horizontal, Nearlight.s20)
|
||||
.padding(.top, Nearlight.s24)
|
||||
.padding(.bottom, Nearlight.s24)
|
||||
}
|
||||
.padding(12)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.background(Color(.secondarySystemBackground), in: RoundedRectangle(cornerRadius: 14))
|
||||
}
|
||||
|
||||
private var statusDot: some View {
|
||||
Circle()
|
||||
.fill(dotColor)
|
||||
.frame(width: 9, height: 9)
|
||||
}
|
||||
|
||||
private var dotColor: Color {
|
||||
private var statusTitle: String {
|
||||
switch task.status {
|
||||
case .pending: return .gray
|
||||
case .running: return .blue
|
||||
case .completed: return .green
|
||||
case .failed: return .red
|
||||
case .pending: return "Queued"
|
||||
case .running: return "In progress"
|
||||
case .completed: return "Complete"
|
||||
case .failed: return "Needs attention"
|
||||
}
|
||||
}
|
||||
|
||||
private func block(_ text: String, color: Color, background: Color) -> some View {
|
||||
Text(text)
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(color)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(10)
|
||||
.background(background, in: RoundedRectangle(cornerRadius: 10))
|
||||
private var statusSymbol: String {
|
||||
switch task.status {
|
||||
case .pending: return "circle.dotted"
|
||||
case .running: return "arrow.triangle.2.circlepath"
|
||||
case .completed: return "checkmark.circle"
|
||||
case .failed: return "exclamationmark.circle"
|
||||
}
|
||||
}
|
||||
|
||||
private func stepLabel(_ role: String?) -> String {
|
||||
private var statusColor: Color {
|
||||
switch task.status {
|
||||
case .pending: return Nearlight.quiet
|
||||
case .running: return Nearlight.rosewood
|
||||
case .completed: return Nearlight.quiet
|
||||
case .failed: return Nearlight.alert
|
||||
}
|
||||
}
|
||||
|
||||
private func roleLabel(_ role: String) -> String {
|
||||
switch role {
|
||||
case "thinking": return "Thinking"
|
||||
case "tool_call": return "Tool"
|
||||
case "tool_result": return "Result"
|
||||
case "text": return "Note"
|
||||
default: return role ?? "Step"
|
||||
}
|
||||
}
|
||||
|
||||
private func stepColor(_ role: String?) -> Color {
|
||||
switch role {
|
||||
case "thinking": return .blue
|
||||
case "tool_call": return .indigo
|
||||
case "tool_result": return .secondary
|
||||
default: return .primary
|
||||
default: return role
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#Preview {
|
||||
NavigationStack {
|
||||
TaskPanelView(taskStore: {
|
||||
let store = TaskStore()
|
||||
#if DEBUG
|
||||
store.injectSampleEvents()
|
||||
#endif
|
||||
return store
|
||||
}())
|
||||
TaskPanelView(taskStore: TaskStore())
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,7 @@ import Foundation
|
||||
import LiveKit
|
||||
|
||||
/// Outbound helpers for agent control topics and inbound parsing of
|
||||
/// voice-control / conversation-control / media-control data messages.
|
||||
/// voice-control / conversation-control data messages.
|
||||
@MainActor
|
||||
final class ControlChannel: NSObject, RoomDelegate {
|
||||
|
||||
@@ -109,11 +109,6 @@ final class ControlChannel: NSObject, RoomDelegate {
|
||||
try await publish(["type": "set_mode", "mode": mode], topic: "conversation-control", room: room)
|
||||
}
|
||||
|
||||
func sendMediaState(_ state: String, trackID: String?, room: Room) async throws {
|
||||
var object: [String: Any] = ["type": "media_state", "state": state]
|
||||
if let trackID { object["track_id"] = trackID }
|
||||
try await publish(object, topic: "media-control", room: room)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Minimal JSON value decoder for flexible agent payloads
|
||||
|
||||
@@ -20,7 +20,7 @@ final class TranscriptHistory: ObservableObject {
|
||||
static let maxEntries = 50
|
||||
|
||||
init() {
|
||||
if let raw = UserDefaults.standard.string(forKey: Self.key),
|
||||
if let raw = AppPreferences.store.string(forKey: Self.key),
|
||||
let data = raw.data(using: .utf8),
|
||||
let decoded = try? JSONDecoder().decode([HistoryEntry].self, from: data) {
|
||||
entries = Array(decoded.suffix(Self.maxEntries))
|
||||
@@ -45,6 +45,6 @@ final class TranscriptHistory: ObservableObject {
|
||||
|
||||
private func persist() {
|
||||
guard let data = try? JSONEncoder().encode(entries) else { return }
|
||||
UserDefaults.standard.set(String(decoding: data, as: UTF8.self), forKey: Self.key)
|
||||
AppPreferences.store.set(String(decoding: data, as: UTF8.self), forKey: Self.key)
|
||||
}
|
||||
}
|
||||
@@ -48,7 +48,6 @@ final class VoiceSession: ObservableObject {
|
||||
let transcriptHandler = TranscriptHandler()
|
||||
let stateModel = ConversationStateModel()
|
||||
let taskStore = TaskStore()
|
||||
let mediaController: MediaPlayerController
|
||||
private let config: ServerConfig
|
||||
private let tokenClient: TokenClient
|
||||
private let settings: SettingsStore
|
||||
@@ -90,9 +89,6 @@ final class VoiceSession: ObservableObject {
|
||||
self.config = config
|
||||
self.tokenClient = tokenClient
|
||||
self.settings = settings
|
||||
let mediaAPI = MediaAPI(config: config)
|
||||
self.mediaController = MediaPlayerController(api: mediaAPI, settings: settings)
|
||||
mediaController.attach(session: self)
|
||||
room.add(delegate: self)
|
||||
room.add(delegate: controlChannel)
|
||||
room.add(delegate: transcriptHandler)
|
||||
@@ -211,7 +207,6 @@ final class VoiceSession: ObservableObject {
|
||||
do {
|
||||
let token = try await tokenClient.fetchToken()
|
||||
logger.log("token ok room=\(token.room, privacy: .public)")
|
||||
await mediaController.enterMediaScope(roomName: token.room)
|
||||
try await room.connect(url: config.livekitURL.absoluteString, token: token.token, connectOptions: makeConnectOptions())
|
||||
result = .success(())
|
||||
} catch {
|
||||
@@ -263,8 +258,7 @@ final class VoiceSession: ObservableObject {
|
||||
private func handleAgentTimeout(generation: Int) async {
|
||||
guard generation == sessionGeneration, state == .waitingAgent else { return }
|
||||
guard !reconnectAttempted else {
|
||||
// Final failure: tear down the live room, restore the idle timer,
|
||||
// and end the media scope so no stale room or wake lock survives.
|
||||
// Final failure: tear down the live room and restore the idle timer.
|
||||
logger.log("agent timeout final, tearing down")
|
||||
await teardownForFailure(message: "Agent never joined. Is the agent worker running?")
|
||||
return
|
||||
@@ -281,8 +275,7 @@ final class VoiceSession: ObservableObject {
|
||||
/// current generation (so stale callbacks become no-ops), clear all
|
||||
/// per-room/session ephemeral published fields (agent, mute state, model
|
||||
/// list, error state, interruption ownership), disconnect and recreate the
|
||||
/// room, stop conversation state, restore the idle timer, and end the media
|
||||
/// scope. Only after all cleanup completes does the session publish
|
||||
/// room, stop conversation state, restore the idle timer, and publish
|
||||
/// `.failed(message)`, making retry safe: the old room is fully torn down
|
||||
/// before Start becomes available again.
|
||||
///
|
||||
@@ -308,7 +301,6 @@ final class VoiceSession: ObservableObject {
|
||||
interruptionSequence += 1
|
||||
stateModel.stop()
|
||||
UIApplication.shared.isIdleTimerDisabled = false
|
||||
await mediaController.endConversation()
|
||||
await teardownRoom()
|
||||
|
||||
// Final reset immediately before publishing .failed: any old-room
|
||||
@@ -540,15 +532,13 @@ final class VoiceSession: ObservableObject {
|
||||
interruptionSequence += 1
|
||||
stateModel.stop()
|
||||
UIApplication.shared.isIdleTimerDisabled = false
|
||||
await mediaController.endConversation()
|
||||
await teardownRoom()
|
||||
// Final reset before publishing .idle: any old-room callback delivered
|
||||
// during the teardownRoom() await may have repopulated ephemeral fields.
|
||||
// This ensures the idle state is clean for server replacement (Iteration 09).
|
||||
resetEphemeralFields()
|
||||
// Publish .idle only after media and room teardown have completed.
|
||||
// This makes idle a truthful readiness signal for server replacement:
|
||||
// the old session is fully torn down before the UI shows "ready to start".
|
||||
// Publish .idle only after room teardown has completed: the old session
|
||||
// is fully torn down before the UI shows "ready to start".
|
||||
state = .idle
|
||||
}
|
||||
|
||||
|
||||
@@ -1,249 +0,0 @@
|
||||
import Foundation
|
||||
|
||||
struct MediaTrack: Decodable, Equatable {
|
||||
let id: String
|
||||
let kind: String?
|
||||
let videoID: String?
|
||||
let title: String?
|
||||
let channel: String?
|
||||
let duration: Double?
|
||||
let status: String?
|
||||
let error: String?
|
||||
let audioURL: String?
|
||||
let artURL: String?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case id, kind, title, channel, duration, status, error
|
||||
case videoID = "video_id"
|
||||
case audioURL = "audio_url"
|
||||
case artURL = "art_url"
|
||||
}
|
||||
}
|
||||
|
||||
struct MediaPendingJob: Decodable, Equatable {
|
||||
let jobID: String?
|
||||
let trackID: String?
|
||||
let stage: String?
|
||||
let progress: Double?
|
||||
let error: String?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case stage, progress, error
|
||||
case jobID = "job_id"
|
||||
case trackID = "track_id"
|
||||
}
|
||||
}
|
||||
|
||||
struct MediaSeekHint: Decodable, Equatable {
|
||||
let ts: Double
|
||||
let position: Double
|
||||
}
|
||||
|
||||
struct MediaState: Decodable, Equatable {
|
||||
let room: String?
|
||||
let track: MediaTrack?
|
||||
let desired: String?
|
||||
let reported: MediaReported?
|
||||
let seekHint: MediaSeekHint?
|
||||
let choices: [MediaChoice]
|
||||
let pending: [MediaPendingJob]
|
||||
let version: Int?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case room, track, desired, reported, choices, pending, version
|
||||
case seekHint = "seek_hint"
|
||||
}
|
||||
}
|
||||
|
||||
struct MediaReported: Decodable, Equatable {
|
||||
let trackID: String?
|
||||
let state: String?
|
||||
let position: Double?
|
||||
let duration: Double?
|
||||
let ts: Double?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case state, position, duration, ts
|
||||
case trackID = "track_id"
|
||||
}
|
||||
}
|
||||
|
||||
struct MediaChoice: Decodable, Equatable {
|
||||
let videoID: String?
|
||||
let title: String?
|
||||
let channel: String?
|
||||
let duration: Double?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case title, channel, duration
|
||||
case videoID = "video_id"
|
||||
}
|
||||
}
|
||||
|
||||
struct MediaLibrary: Decodable, Equatable {
|
||||
let tracks: [MediaTrack]
|
||||
let version: Int?
|
||||
}
|
||||
|
||||
struct MediaSearchResult: Decodable, Equatable {
|
||||
let videoID: String?
|
||||
let trackID: String?
|
||||
let title: String?
|
||||
let channel: String?
|
||||
let duration: Double?
|
||||
let inLibrary: Bool?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case title, channel, duration
|
||||
case videoID = "video_id"
|
||||
case trackID = "track_id"
|
||||
case inLibrary = "in_library"
|
||||
}
|
||||
}
|
||||
|
||||
struct MediaSearchResponse: Decodable, Equatable {
|
||||
let results: [MediaSearchResult]
|
||||
}
|
||||
|
||||
enum MediaAPIError: LocalizedError {
|
||||
case invalidServerURL(String)
|
||||
case httpStatus(Int, String)
|
||||
case decodeFailed
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .invalidServerURL(let raw):
|
||||
return "Invalid server URL: \(raw)"
|
||||
case .httpStatus(let code, let detail):
|
||||
return "Media API request failed with HTTP \(code). \(detail)"
|
||||
case .decodeFailed:
|
||||
return "Unexpected response from the media endpoint."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
final class MediaAPI {
|
||||
let serverBase: URL
|
||||
private let baseURL: URL
|
||||
private let session: URLSession
|
||||
|
||||
init(config: ServerConfig) {
|
||||
self.serverBase = config.baseURL
|
||||
self.baseURL = config.baseURL.appendingPathComponent("media/api")
|
||||
self.session = TLSPinning.session()
|
||||
}
|
||||
|
||||
func state(room: String) async throws -> MediaState {
|
||||
var components = URLComponents(url: baseURL.appendingPathComponent("state"), resolvingAgainstBaseURL: false)!
|
||||
components.queryItems = [URLQueryItem(name: "room", value: room)]
|
||||
return try await get(MediaState.self, url: components.url!)
|
||||
}
|
||||
|
||||
func report(room: String, state: String, trackID: String?, position: Double, duration: Double?) async throws {
|
||||
var body: [String: Any] = ["room": room, "state": state, "position": position]
|
||||
if let trackID { body["track_id"] = trackID }
|
||||
if let duration { body["duration"] = duration }
|
||||
try await post("report", body: body)
|
||||
}
|
||||
|
||||
func play(room: String, videoID: String? = nil, title: String? = nil, channel: String? = nil, trackID: String? = nil, startPaused: Bool? = nil) async throws {
|
||||
var body: [String: Any] = ["room": room]
|
||||
if let videoID { body["video_id"] = videoID }
|
||||
if let title { body["title"] = title }
|
||||
if let channel { body["channel"] = channel }
|
||||
if let trackID { body["track_id"] = trackID }
|
||||
if let startPaused { body["start_paused"] = startPaused }
|
||||
try await post("play", body: body)
|
||||
}
|
||||
|
||||
func control(room: String, action: String) async throws {
|
||||
try await post("control", body: ["room": room, "action": action])
|
||||
}
|
||||
|
||||
func search(room: String, query: String, limit: Int = 6) async throws -> MediaSearchResponse {
|
||||
try await postDecoded("search", body: ["room": room, "query": query, "limit": limit], as: MediaSearchResponse.self)
|
||||
}
|
||||
|
||||
func library() async throws -> MediaLibrary {
|
||||
try await get(MediaLibrary.self, url: baseURL.appendingPathComponent("library"))
|
||||
}
|
||||
|
||||
func deleteTrack(id: String) async throws {
|
||||
var request = URLRequest(url: baseURL.appendingPathComponent("tracks").appending(path: id))
|
||||
request.httpMethod = "DELETE"
|
||||
let (data, response) = try await session.data(for: request)
|
||||
guard let http = response as? HTTPURLResponse else { throw MediaAPIError.decodeFailed }
|
||||
guard (200..<300).contains(http.statusCode) else {
|
||||
throw MediaAPIError.httpStatus(http.statusCode, String(data: data, encoding: .utf8) ?? "")
|
||||
}
|
||||
}
|
||||
|
||||
func addStream(room: String, name: String, url: String) async throws {
|
||||
try await post("streams", body: ["room": room, "name": name, "url": url])
|
||||
}
|
||||
|
||||
/// Returns the progressive-media endpoint URL for a track. The `/stream/{id}`
|
||||
/// path serves raw media bytes (not a text URL), so this constructs the
|
||||
/// endpoint without downloading or decoding the body.
|
||||
func streamURL(trackID: String) -> URL {
|
||||
baseURL.appendingPathComponent("stream").appending(path: trackID)
|
||||
}
|
||||
|
||||
func healthz() async throws -> Bool {
|
||||
let data = try await getRaw(url: baseURL.appendingPathComponent("healthz"))
|
||||
guard let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { return false }
|
||||
return json["ok"] as? Bool ?? false
|
||||
}
|
||||
|
||||
private func get<T: Decodable>(_ type: T.Type, url: URL) async throws -> T {
|
||||
let data = try await getRaw(url: url)
|
||||
do {
|
||||
return try JSONDecoder().decode(T.self, from: data)
|
||||
} catch {
|
||||
throw MediaAPIError.decodeFailed
|
||||
}
|
||||
}
|
||||
|
||||
private func getRaw(url: URL) async throws -> Data {
|
||||
let (data, response) = try await session.data(from: url)
|
||||
guard let http = response as? HTTPURLResponse else { throw MediaAPIError.decodeFailed }
|
||||
guard (200..<300).contains(http.statusCode) else {
|
||||
throw MediaAPIError.httpStatus(http.statusCode, String(data: data, encoding: .utf8) ?? "")
|
||||
}
|
||||
return data
|
||||
}
|
||||
|
||||
/// Fire-and-forget POST that only validates HTTP success. The media server
|
||||
/// returns `{"ok": true, ...snapshot}` for /play, /control, /report and
|
||||
/// /streams — callers don't need the body, so decoding it (as Data or any
|
||||
/// typed model) would be wasted work at best and a spurious failure mode.
|
||||
private func post(_ path: String, body: [String: Any]) async throws {
|
||||
_ = try await postRaw(path, body: body)
|
||||
}
|
||||
|
||||
private func postDecoded<T: Decodable>(_ path: String, body: [String: Any], as type: T.Type) async throws -> T {
|
||||
let data = try await postRaw(path, body: body)
|
||||
do {
|
||||
return try JSONDecoder().decode(T.self, from: data)
|
||||
} catch {
|
||||
throw MediaAPIError.decodeFailed
|
||||
}
|
||||
}
|
||||
|
||||
private func postRaw(_ path: String, body: [String: Any]) async throws -> Data {
|
||||
var request = URLRequest(url: baseURL.appendingPathComponent(path))
|
||||
request.httpMethod = "POST"
|
||||
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
|
||||
request.httpBody = try JSONSerialization.data(withJSONObject: body)
|
||||
let (data, response) = try await session.data(for: request)
|
||||
guard let http = response as? HTTPURLResponse else { throw MediaAPIError.decodeFailed }
|
||||
guard (200..<300).contains(http.statusCode) else {
|
||||
var detail = String(data: data, encoding: .utf8) ?? ""
|
||||
if let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any], let error = json["error"] as? String {
|
||||
detail = error
|
||||
}
|
||||
throw MediaAPIError.httpStatus(http.statusCode, detail)
|
||||
}
|
||||
return data
|
||||
}
|
||||
}
|
||||
@@ -1,595 +0,0 @@
|
||||
import AVFoundation
|
||||
import Combine
|
||||
import Foundation
|
||||
import os
|
||||
|
||||
private let logger = Logger(subsystem: "dev.shane.voice", category: "media")
|
||||
|
||||
/// AVPlayer-backed media player. The media server holds per-room desired state;
|
||||
/// this controller polls /state and reconciles the local player against it,
|
||||
/// porting the web UI's reconciliation rules (app.js reconcileMedia).
|
||||
@MainActor
|
||||
final class MediaPlayerController: ObservableObject {
|
||||
@Published private(set) var track: MediaTrack?
|
||||
@Published private(set) var desired: String = "stopped"
|
||||
@Published private(set) var currentTime: Double = 0
|
||||
@Published private(set) var duration: Double = 0
|
||||
@Published private(set) var pendingJobs: [MediaPendingJob] = []
|
||||
@Published private(set) var choices: [MediaChoice] = []
|
||||
@Published private(set) var stateError: String?
|
||||
|
||||
let player = AVPlayer()
|
||||
|
||||
private let api: MediaAPI
|
||||
private let settings: SettingsStore
|
||||
private let mediaLoader: PinnedMediaResourceLoader
|
||||
private weak var session: VoiceSession?
|
||||
|
||||
private var room: String?
|
||||
private var currentTrackID: String?
|
||||
private var loadedAudioPath: String?
|
||||
private var currentAsset: AVURLAsset?
|
||||
private var appliedSeekTs: Double = 0
|
||||
private var wasPlayingBeforeTalk = false
|
||||
private var lastReportDate = Date.distantPast
|
||||
private var pollTask: Task<Void, Never>?
|
||||
private var timeObserver: Any?
|
||||
private var statusCancellable: AnyCancellable?
|
||||
private var rateObservation: NSKeyValueObservation?
|
||||
|
||||
/// Bumped on every room-scope change so in-flight /state polls from the
|
||||
/// previous scope cannot reconcile into the new one (port of the web UI's
|
||||
/// `=== room` checks).
|
||||
private var pollGeneration = 0
|
||||
|
||||
init(api: MediaAPI, settings: SettingsStore) {
|
||||
self.api = api
|
||||
self.settings = settings
|
||||
self.mediaLoader = PinnedMediaResourceLoader(base: api.serverBase)
|
||||
player.volume = Float(settings.volume)
|
||||
NotificationCenter.default.addObserver(
|
||||
self, selector: #selector(handleInterruption),
|
||||
name: AVAudioSession.interruptionNotification, object: nil
|
||||
)
|
||||
}
|
||||
|
||||
deinit {
|
||||
if let timeObserver {
|
||||
player.removeTimeObserver(timeObserver)
|
||||
}
|
||||
rateObservation?.invalidate()
|
||||
NotificationCenter.default.removeObserver(self)
|
||||
}
|
||||
|
||||
/// Whether an audio session interruption is currently active. While true,
|
||||
/// reconciliation must not call player.play() (the next poll would restart
|
||||
/// audio during a phone call or Siri interaction).
|
||||
private var isInterruptionActive = false
|
||||
|
||||
/// Separate resume-suppression flag: set when an interruption ends without
|
||||
/// `.shouldResume` and the pause control to the server has not yet been
|
||||
/// confirmed. While true, reconciliation must not call player.play() even
|
||||
/// after `isInterruptionActive` is cleared. Cleared only by an explicit user
|
||||
/// play/resume action or a confirmed coherent paused state from the server.
|
||||
private var resumeSuppressed = false
|
||||
|
||||
/// Whether playback was actually active (rate > 0) when the interruption
|
||||
/// began. Used to decide whether a `.shouldResume` end should actually
|
||||
/// resume (a track that was already paused before the interruption must not
|
||||
/// be force-resumed).
|
||||
private var wasPlayingBeforeInterruption = false
|
||||
|
||||
/// Sequence number for media interruption operations. Incremented on every
|
||||
/// `.began`; a resume task captures the sequence and rechecks it immediately
|
||||
/// before `player.play()`. A newer `.began` makes an old resume task
|
||||
/// incapable of playing (Iteration 07 Item 5).
|
||||
private var mediaInterruptionSequence = 0
|
||||
|
||||
/// The retained operation chain for media interruption control work. Both
|
||||
/// pause and resume operations are assigned here so a newer begin cancels
|
||||
/// and supersedes prior control work.
|
||||
private var mediaInterruptionTask: Task<Void, Never>?
|
||||
|
||||
@objc private func handleInterruption(_ notification: Notification) {
|
||||
guard let info = notification.userInfo else { return }
|
||||
guard let typeValue = info[AVAudioSessionInterruptionTypeKey] as? UInt,
|
||||
let type = AVAudioSession.InterruptionType(rawValue: typeValue) else { return }
|
||||
switch type {
|
||||
case .began:
|
||||
// Increment the sequence so any in-flight resume task from a prior
|
||||
// interruption sees a stale sequence and cannot play.
|
||||
mediaInterruptionSequence += 1
|
||||
let seq = mediaInterruptionSequence
|
||||
// Cancel/supersede prior control work (resume or pause) so it cannot
|
||||
// complete out of order.
|
||||
mediaInterruptionTask?.cancel()
|
||||
isInterruptionActive = true
|
||||
wasPlayingBeforeInterruption = !player.rate.isZero
|
||||
let wasPlaying = wasPlayingBeforeInterruption
|
||||
player.pause()
|
||||
logger.log("audio interruption began, paused media (wasPlaying=\(wasPlaying), seq=\(seq))")
|
||||
case .ended:
|
||||
defer { isInterruptionActive = false }
|
||||
applyVolume()
|
||||
let optionsValue = info[AVAudioSessionInterruptionOptionKey] as? UInt ?? 0
|
||||
let shouldResume = AVAudioSession.InterruptionOptions(rawValue: optionsValue).contains(.shouldResume)
|
||||
if shouldResume, wasPlayingBeforeInterruption, currentTrackID != nil, desired == "playing", loadedAudioPath != nil {
|
||||
// Synchronize the server before resuming local playback so the
|
||||
// two never diverge. Capture generation/room/sequence before the
|
||||
// async hop. The task is retained in mediaInterruptionTask so a
|
||||
// newer .began can cancel it.
|
||||
let gen = pollGeneration
|
||||
let targetRoom = room
|
||||
let seq = mediaInterruptionSequence
|
||||
mediaInterruptionTask = Task { [weak self] in
|
||||
guard let self else { return }
|
||||
if let targetRoom {
|
||||
do {
|
||||
try await self.api.control(room: targetRoom, action: "resume")
|
||||
logger.log("interruption resume: server synchronized (seq=\(seq))")
|
||||
} catch {
|
||||
logger.log("interruption resume: server sync failed (poll will retry): \(error.localizedDescription)")
|
||||
}
|
||||
}
|
||||
// Recheck the sequence immediately before play: a newer .began
|
||||
// increments mediaInterruptionSequence, making this stale task
|
||||
// incapable of restarting audio during the newer interruption.
|
||||
guard !Task.isCancelled,
|
||||
self.mediaInterruptionSequence == seq,
|
||||
!self.isInterruptionActive,
|
||||
self.pollGeneration == gen,
|
||||
self.room == targetRoom else { return }
|
||||
self.player.play()
|
||||
logger.log("audio interruption ended, resumed media (seq=\(seq))")
|
||||
}
|
||||
} else {
|
||||
// No-resume path: keep suppression even if the pause control
|
||||
// fails. The flag is cleared only by an explicit user play/resume
|
||||
// action or a confirmed coherent paused state from the server.
|
||||
resumeSuppressed = true
|
||||
let seq = mediaInterruptionSequence
|
||||
if let room {
|
||||
mediaInterruptionTask = Task { [weak self] in
|
||||
guard let self else { return }
|
||||
do {
|
||||
try await self.api.control(room: room, action: "pause")
|
||||
logger.log("audio interruption ended without resume, sent pause to server (seq=\(seq))")
|
||||
} catch {
|
||||
// Keep suppression even on failure — the next poll
|
||||
// will see desired=="paused" and clear it.
|
||||
logger.log("audio interruption ended without resume, pause control failed (suppression retained): \(error.localizedDescription)")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@unknown default:
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: Lifecycle
|
||||
|
||||
func attach(session: VoiceSession) {
|
||||
self.session = session
|
||||
observePlayer()
|
||||
observePlaybackState()
|
||||
applyVolume()
|
||||
}
|
||||
|
||||
/// Called just before joining the LiveKit room. Carries whatever is loaded
|
||||
/// into the new room-scoped player state, paused (talking pauses music).
|
||||
func enterMediaScope(roomName: String) async {
|
||||
pollGeneration += 1
|
||||
let wasPlaying = currentTrackID != nil && !player.rate.isZero
|
||||
wasPlayingBeforeTalk = wasPlaying
|
||||
let carriedTrack = currentTrackID
|
||||
room = roomName
|
||||
if let carriedTrack {
|
||||
do {
|
||||
try await api.play(room: roomName, trackID: carriedTrack, startPaused: true)
|
||||
logger.log("media scope carry-over: \(carriedTrack, privacy: .public) start_paused")
|
||||
} catch {
|
||||
logger.log("media scope transfer failed: \(String(describing: error), privacy: .public)")
|
||||
}
|
||||
player.pause()
|
||||
}
|
||||
startPolling()
|
||||
}
|
||||
|
||||
/// Called when the conversation ends. If music was playing before the talk,
|
||||
/// hand the room back to the player. The previous scope (room name) is kept
|
||||
/// so the panel can resume polling it via `startDefaultPolling()`; a fresh
|
||||
/// conversation re-scopes via `enterMediaScope`. Polling restarts here so
|
||||
/// the resumed state stays visible without the panel being open.
|
||||
func endConversation() async {
|
||||
stopPolling()
|
||||
pollGeneration += 1
|
||||
if wasPlayingBeforeTalk {
|
||||
wasPlayingBeforeTalk = false
|
||||
guard let room else { return }
|
||||
do {
|
||||
try await api.control(room: room, action: "resume")
|
||||
logger.log("resumed playback after talk ended")
|
||||
} catch {
|
||||
logger.log("post-talk resume failed (poll will retry): \(String(describing: error), privacy: .public)")
|
||||
}
|
||||
}
|
||||
startPolling()
|
||||
}
|
||||
|
||||
/// Drop the room scope entirely (app-level teardown). After this, polling
|
||||
/// targets the default room again.
|
||||
func resetScope() {
|
||||
stopPolling()
|
||||
pollGeneration += 1
|
||||
room = nil
|
||||
wasPlayingBeforeTalk = false
|
||||
}
|
||||
|
||||
private func startPolling() {
|
||||
guard pollTask == nil else { return }
|
||||
pollTask = Task { [weak self] in
|
||||
while !Task.isCancelled {
|
||||
await self?.pollState()
|
||||
try? await Task.sleep(for: .milliseconds(1500))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func stopPolling() {
|
||||
pollTask?.cancel()
|
||||
pollTask = nil
|
||||
}
|
||||
|
||||
/// Resume polling whenever the panel becomes visible and nothing is being
|
||||
/// polled yet — whether that's the default room (no session) or a previous
|
||||
/// conversation's room scope (kept after `endConversation`).
|
||||
func startDefaultPolling() {
|
||||
guard pollTask == nil else { return }
|
||||
startPolling()
|
||||
}
|
||||
|
||||
// MARK: Poll + reconcile (ported from app.js pollMediaState/reconcileMedia)
|
||||
|
||||
private func pollState() async {
|
||||
let generation = pollGeneration
|
||||
let scope = room ?? "default"
|
||||
do {
|
||||
let state = try await api.state(room: scope)
|
||||
guard generation == pollGeneration else { return }
|
||||
stateError = nil
|
||||
reconcile(state)
|
||||
} catch {
|
||||
guard generation == pollGeneration else { return }
|
||||
logger.log("poll failed: \(String(describing: error), privacy: .public)")
|
||||
stateError = "Media server unreachable: \(error.localizedDescription)"
|
||||
}
|
||||
}
|
||||
|
||||
/// Reconciles the local player against a server state snapshot. Exposed
|
||||
/// as internal (not private) so tests can drive deterministic state
|
||||
/// transitions without a live server.
|
||||
func reconcile(_ s: MediaState) {
|
||||
choices = s.choices
|
||||
pendingJobs = s.pending
|
||||
// Keep the published desired state in sync with the server so the UI's
|
||||
// play/pause icon and status text reflect reality (the server is the
|
||||
// source of truth for per-room desired state).
|
||||
desired = s.desired ?? "stopped"
|
||||
|
||||
let track = s.track
|
||||
if let track, track.id != currentTrackID {
|
||||
loadTrack(track)
|
||||
} else if let track, let audioURL = track.audioURL, loadedAudioPath != audioURL {
|
||||
// Track was loaded before it had a stream (still downloading) and
|
||||
// now has one — wire it up so playback can start without a reload.
|
||||
wireSource(audioURL)
|
||||
}
|
||||
if track == nil, currentTrackID != nil, s.desired == "stopped" {
|
||||
clearMedia()
|
||||
}
|
||||
|
||||
applyDesiredState(s, track: track)
|
||||
|
||||
if let hint = s.seekHint, hint.ts != appliedSeekTs, duration.isFinite, duration > 0 {
|
||||
appliedSeekTs = hint.ts
|
||||
let target = min(hint.position, duration)
|
||||
if target.isFinite {
|
||||
seek(to: max(0, target))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func applyDesiredState(_ s: MediaState, track: MediaTrack?) {
|
||||
guard let desiredValue = s.desired else { return }
|
||||
switch desiredValue {
|
||||
case "playing":
|
||||
// Suppress play during an active audio interruption (phone call,
|
||||
// Siri). Without this guard the next 1.5s poll would restart audio
|
||||
// immediately after .began paused it. The player.rate.isZero check
|
||||
// makes this idempotent: once playing, subsequent polls are no-ops.
|
||||
// This also handles the late-audio case where desired=playing
|
||||
// arrives before audio_url is set; when the source is later wired
|
||||
// via wireSource(), the next poll retries play because rate is
|
||||
// still zero.
|
||||
// Also suppress while resumeSuppressed is true (no-resume
|
||||
// interruption ended, server not yet confirmed paused).
|
||||
guard !isInterruptionActive, !resumeSuppressed else { return }
|
||||
if let track, track.audioURL != nil, loadedAudioPath != nil, player.rate.isZero {
|
||||
player.play()
|
||||
}
|
||||
case "paused", "stopped":
|
||||
if !player.rate.isZero {
|
||||
player.pause()
|
||||
}
|
||||
// The server has confirmed a coherent paused/stopped state. If we
|
||||
// were suppressing resume due to a no-resume interruption, the
|
||||
// pause control succeeded (or the user paused independently), so
|
||||
// it is safe to clear the suppression flag.
|
||||
if resumeSuppressed {
|
||||
resumeSuppressed = false
|
||||
logger.log("resume suppression cleared: server confirmed \(desiredValue)")
|
||||
}
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
private func loadTrack(_ track: MediaTrack) {
|
||||
currentTrackID = track.id
|
||||
self.track = track
|
||||
if let audioURL = track.audioURL {
|
||||
wireSource(audioURL)
|
||||
} else {
|
||||
// Downloading or failed: no stream yet. Reconcile wires the stream
|
||||
// URL in via polling as soon as one is available.
|
||||
clearCurrentAsset()
|
||||
player.replaceCurrentItem(with: nil)
|
||||
}
|
||||
}
|
||||
|
||||
private func wireSource(_ path: String) {
|
||||
guard let url = URL(string: path) else { return }
|
||||
loadedAudioPath = path
|
||||
let pinned = mediaLoader.rewrite(url)
|
||||
// Remove the previous asset's loader mapping before wiring a new one so
|
||||
// the loaderURLs dictionary stays bounded (Finding 11 / Item 7).
|
||||
clearCurrentAsset()
|
||||
let asset = mediaLoader.assetWithPinnedURL(pinned)
|
||||
currentAsset = asset
|
||||
logger.log("wiring source \(pinned.absoluteString, privacy: .public)")
|
||||
let item = AVPlayerItem(asset: asset)
|
||||
observeItem(item)
|
||||
player.replaceCurrentItem(with: item)
|
||||
}
|
||||
|
||||
/// Removes the loader mapping for the current asset (if any) and clears the
|
||||
/// reference. Called before wiring a new source or clearing media so the
|
||||
/// resource loader's internal dictionaries stay bounded.
|
||||
private func clearCurrentAsset() {
|
||||
if let asset = currentAsset {
|
||||
mediaLoader.removeLoader(asset)
|
||||
currentAsset = nil
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: Player events
|
||||
|
||||
private func observePlayer() {
|
||||
if timeObserver == nil {
|
||||
timeObserver = player.addPeriodicTimeObserver(
|
||||
forInterval: CMTime(seconds: 0.5, preferredTimescale: 600), queue: .main
|
||||
) { [weak self] time in
|
||||
Task { @MainActor [weak self] in
|
||||
guard let self else { return }
|
||||
self.currentTime = time.seconds
|
||||
if let itemDuration = self.player.currentItem?.duration {
|
||||
let seconds = itemDuration.seconds
|
||||
if seconds.isFinite {
|
||||
self.duration = seconds
|
||||
}
|
||||
}
|
||||
if !self.player.rate.isZero, Date().timeIntervalSince(self.lastReportDate) >= 5 {
|
||||
self.lastReportDate = Date()
|
||||
self.report("playing")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Playback state observation
|
||||
|
||||
/// Observe the player's actual rate so every real play/pause transition is
|
||||
/// reported promptly, mirroring the web UI's `play`/`pause` listeners. The
|
||||
/// KVO block runs on an arbitrary queue; hop to MainActor before touching
|
||||
/// any controller state.
|
||||
private func observePlaybackState() {
|
||||
rateObservation = player.observe(\.rate, options: [.initial, .new]) { [weak self] player, _ in
|
||||
Task { @MainActor [weak self] in
|
||||
guard let self else { return }
|
||||
guard self.currentTrackID != nil else { return }
|
||||
if player.rate > 0 {
|
||||
// Actual playback began (user play, interruption resume, or
|
||||
// reconcile-driven play). Report + publish the transition.
|
||||
self.lastReportDate = Date()
|
||||
self.report("playing")
|
||||
self.publishMediaState("playing")
|
||||
} else if player.currentItem != nil {
|
||||
// Rate dropped to zero with an item still loaded: a pause
|
||||
// that didn't originate from a user action (interruption,
|
||||
// reconcile). User-initiated pauses already report before
|
||||
// this fires; the lastReportDate guard makes duplicates no-ops.
|
||||
if Date().timeIntervalSince(self.lastReportDate) >= 1 {
|
||||
self.lastReportDate = Date()
|
||||
self.report("paused")
|
||||
self.publishMediaState("paused")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func observeItem(_ item: AVPlayerItem?) {
|
||||
NotificationCenter.default.removeObserver(
|
||||
self, name: .AVPlayerItemDidPlayToEndTime, object: nil
|
||||
)
|
||||
guard let item else { return }
|
||||
NotificationCenter.default.addObserver(
|
||||
self, selector: #selector(playerItemDidEnd(_:)),
|
||||
name: .AVPlayerItemDidPlayToEndTime, object: item
|
||||
)
|
||||
statusCancellable = item.publisher(for: \.status).sink { status in
|
||||
switch status {
|
||||
case .readyToPlay:
|
||||
logger.log("item readyToPlay")
|
||||
case .failed:
|
||||
logger.log("item failed: \(String(describing: item.error), privacy: .public)")
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@objc private func playerItemDidEnd(_ notification: Notification) {
|
||||
lastReportDate = Date()
|
||||
report("stopped")
|
||||
publishMediaState("stopped")
|
||||
Task { [weak self] in
|
||||
guard let self, let room = self.room else { return }
|
||||
try? await self.api.control(room: room, action: "stop")
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: Reporting + publishing
|
||||
|
||||
private func report(_ state: String, trackID: String? = nil) {
|
||||
guard let room else { return }
|
||||
let position = currentTime.isFinite ? currentTime : 0
|
||||
var durationValue: Double?
|
||||
if duration.isFinite, duration > 0 {
|
||||
durationValue = duration
|
||||
}
|
||||
let resolvedTrackID = trackID ?? currentTrackID
|
||||
Task { [weak self] in
|
||||
guard let self else { return }
|
||||
do {
|
||||
try await self.api.report(room: room, state: state, trackID: resolvedTrackID, position: position, duration: durationValue)
|
||||
} catch {
|
||||
logger.log("report failed: \(String(describing: error), privacy: .public)")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func publishMediaState(_ state: String, trackID: String? = nil) {
|
||||
guard let session else { return }
|
||||
// Capture the exact room and generation before spawning so a delayed
|
||||
// publish from an old lifecycle cannot escape into a new conversation
|
||||
// room (Finding 10).
|
||||
let targetRoom = session.room
|
||||
let generation = session.sessionGeneration
|
||||
let pollGen = pollGeneration
|
||||
let resolvedTrackID = trackID ?? currentTrackID
|
||||
Task { [weak self] in
|
||||
guard let self, self.pollGeneration == pollGen else { return }
|
||||
guard session.sessionGeneration == generation, session.room === targetRoom else { return }
|
||||
do {
|
||||
try await session.controlChannel.sendMediaState(state, trackID: resolvedTrackID, room: targetRoom)
|
||||
} catch {
|
||||
logger.log("media_state publish failed: \(String(describing: error), privacy: .public)")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: Play requests (search results, choices, library)
|
||||
|
||||
func playVideo(videoID: String?, title: String?, channel: String?) async {
|
||||
let targetRoom = room ?? "default"
|
||||
do {
|
||||
try await api.play(room: targetRoom, videoID: videoID, title: title, channel: channel)
|
||||
logger.log("play requested video_id=\(videoID ?? "nil", privacy: .public) room=\(targetRoom, privacy: .public)")
|
||||
} catch {
|
||||
logger.log("play failed: \(String(describing: error), privacy: .public)")
|
||||
}
|
||||
}
|
||||
|
||||
func playTrack(id: String) async {
|
||||
let targetRoom = room ?? "default"
|
||||
do {
|
||||
try await api.play(room: targetRoom, trackID: id)
|
||||
logger.log("play requested track_id=\(id, privacy: .public) room=\(targetRoom, privacy: .public)")
|
||||
} catch {
|
||||
logger.log("play failed: \(String(describing: error), privacy: .public)")
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: Transport controls (parity with web UI play/pause/seek/stop)
|
||||
|
||||
func togglePlayPause() async {
|
||||
guard currentTrackID != nil, loadedAudioPath != nil else { return }
|
||||
if player.rate.isZero {
|
||||
// Explicit user play: clear any resume suppression (the user is
|
||||
// deliberately overriding the no-resume interruption state).
|
||||
resumeSuppressed = false
|
||||
// The rate observer reports + publishes the actual transition;
|
||||
// the control call updates server desired state.
|
||||
player.play()
|
||||
if let room { try? await api.control(room: room, action: "resume") }
|
||||
} else {
|
||||
lastReportDate = Date()
|
||||
report("paused")
|
||||
publishMediaState("paused")
|
||||
player.pause()
|
||||
if let room { try? await api.control(room: room, action: "pause") }
|
||||
}
|
||||
}
|
||||
|
||||
func seek(by delta: Double) async {
|
||||
guard loadedAudioPath != nil else { return }
|
||||
let target = max(0, min(duration > 0 ? duration - 0.5 : .infinity, currentTime + delta))
|
||||
if target.isFinite {
|
||||
seek(to: target)
|
||||
}
|
||||
}
|
||||
|
||||
func seek(to position: Double) {
|
||||
player.seek(to: CMTime(seconds: position, preferredTimescale: 600))
|
||||
currentTime = position
|
||||
report(player.rate.isZero ? "paused" : "playing")
|
||||
}
|
||||
|
||||
func stop() async {
|
||||
// Capture the track ID before clearing so the async report/publish
|
||||
// tasks can read a non-nil value even after clearMedia() runs.
|
||||
let trackID = currentTrackID
|
||||
report("stopped", trackID: trackID)
|
||||
clearMedia()
|
||||
if let room { try? await api.control(room: room, action: "stop") }
|
||||
publishMediaState("stopped", trackID: trackID)
|
||||
}
|
||||
|
||||
private func clearMedia() {
|
||||
currentTrackID = nil
|
||||
track = nil
|
||||
loadedAudioPath = nil
|
||||
// Remove the loader mapping for the asset being cleared so the
|
||||
// resource loader's internal dictionaries stay bounded (Item 7).
|
||||
clearCurrentAsset()
|
||||
observeItem(nil)
|
||||
player.replaceCurrentItem(with: nil)
|
||||
currentTime = 0
|
||||
duration = 0
|
||||
stateError = nil
|
||||
}
|
||||
|
||||
// MARK: Volume
|
||||
|
||||
func applyVolume() {
|
||||
player.volume = Float(settings.volume)
|
||||
}
|
||||
|
||||
func setVolume(_ volume: Double) {
|
||||
settings.volume = volume
|
||||
player.volume = Float(volume)
|
||||
}
|
||||
}
|
||||
@@ -1,220 +0,0 @@
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
import os
|
||||
|
||||
private let logger = Logger(subsystem: "dev.shane.voice", category: "media")
|
||||
|
||||
/// Proxies {base}/media/* bytes to AVPlayer over the pinned-TLS session, since
|
||||
/// AVPlayer cannot trust the self-signed LAN cert. AVPlayer loads https natively
|
||||
/// (ignoring registered URLProtocol subclasses), so the asset uses a custom
|
||||
/// `pinned-media` scheme and this delegate answers every data request by
|
||||
/// fetching the real https URL through TLSPinning.session(), forwarding Range.
|
||||
///
|
||||
/// **Lifecycle invariants (Finding 11 / Iteration 06 Item 7):**
|
||||
/// - Tasks are keyed by `ObjectIdentifier(loadingRequest)` directly, not a
|
||||
/// derived hash, so two distinct requests can never collide.
|
||||
/// - All mutable state (`loaderURLs`, `activeTasks`) is accessed only on the
|
||||
/// main queue: the AVAssetResourceLoader delegate is set with `queue: .main`
|
||||
/// and the pinned URLSession uses `delegateQueue: .main`, so both the
|
||||
/// resource-loading callbacks and the data-task completion handlers run on
|
||||
/// the same serialization domain. This is enforced in code, not by comment.
|
||||
/// - For HTTP 206, `contentLength` is set only when `Content-Range` provides a
|
||||
/// known total; it never falls back to the range's `expectedContentLength`.
|
||||
/// - For HTTP 200, `expectedContentLength` is already the full resource size.
|
||||
/// - Cleanup (task removal) occurs on success, error, cancellation, and item
|
||||
/// replacement via `removeLoader`.
|
||||
final class PinnedMediaResourceLoader: NSObject, AVAssetResourceLoaderDelegate {
|
||||
|
||||
static let scheme = "pinned-media"
|
||||
|
||||
private let base: URL
|
||||
/// Maps each asset's resource loader to the real https URL it proxies.
|
||||
/// Accessed only on the main queue (the delegate queue).
|
||||
private var loaderURLs: [ObjectIdentifier: URL] = [:]
|
||||
/// Maps each loading request's identity to its upstream data task so
|
||||
/// cancellation can cancel the in-flight download. Keyed directly by
|
||||
/// `ObjectIdentifier(loadingRequest)` — no hash derivation, no collisions.
|
||||
private var activeTasks: [ObjectIdentifier: URLSessionDataTask] = [:]
|
||||
/// Shared pinned session, created once and invalidated on deinit.
|
||||
/// Uses `delegateQueue: .main` so all completion handlers (and therefore
|
||||
/// all mutations of `activeTasks`) run on the main queue — the same
|
||||
/// serialization domain as the AVAssetResourceLoader delegate callbacks
|
||||
/// (which are also dispatched to `.main`). This makes map access truly
|
||||
/// serialized in code, not just by convention.
|
||||
private lazy var pinnedSession: URLSession = {
|
||||
let config = URLSessionConfiguration.default
|
||||
config.timeoutIntervalForRequest = 30
|
||||
return URLSession(configuration: config, delegate: PinnedTrustDelegate(), delegateQueue: .main)
|
||||
}()
|
||||
|
||||
init(base: URL) {
|
||||
self.base = base
|
||||
}
|
||||
|
||||
deinit {
|
||||
pinnedSession.invalidateAndCancel()
|
||||
}
|
||||
|
||||
/// Maps a media URL (absolute https or /media/... path) to the pinned-media
|
||||
/// scheme so AVPlayer routes it through this delegate instead of loading
|
||||
/// https natively. Built as a string because URLComponents.url returns nil
|
||||
/// for schemes not in Foundation's known-scheme list.
|
||||
func rewrite(_ url: URL) -> URL {
|
||||
let absolute: String
|
||||
if let scheme = url.scheme?.lowercased(), scheme.hasPrefix("http") {
|
||||
var c = URLComponents(url: url, resolvingAgainstBaseURL: false)!
|
||||
c.scheme = Self.scheme
|
||||
return c.url!
|
||||
} else {
|
||||
let path = url.absoluteString.hasPrefix("/") ? String(url.absoluteString.dropFirst()) : url.absoluteString
|
||||
guard let host = base.host, !host.isEmpty else { return url }
|
||||
let port = base.port.map { ":\($0)" } ?? ""
|
||||
absolute = "\(Self.scheme)://\(host)\(port)/\(path)"
|
||||
}
|
||||
return URL(string: absolute) ?? url
|
||||
}
|
||||
|
||||
/// Maps a pinned-media URL back to the real https URL it proxies.
|
||||
func resolve(_ url: URL) -> URL {
|
||||
guard let scheme = url.scheme?.lowercased(), scheme == Self.scheme else { return url }
|
||||
var c = URLComponents(url: url, resolvingAgainstBaseURL: false)!
|
||||
c.scheme = "https"
|
||||
return c.url ?? url
|
||||
}
|
||||
|
||||
/// Creates an AVURLAsset that routes through this loader's delegate. The
|
||||
/// mapping from the asset's resource loader to the real URL is recorded so
|
||||
/// `resourceLoader(_:shouldWaitForLoadingOfRequestedResource:)` can resolve
|
||||
/// it. All mutations happen on the main queue (the delegate queue).
|
||||
func assetWithPinnedURL(_ pinned: URL) -> AVURLAsset {
|
||||
let asset = AVURLAsset(url: pinned)
|
||||
let realURL = resolve(pinned)
|
||||
// The delegate queue is .main, so this runs on the main thread.
|
||||
loaderURLs[ObjectIdentifier(asset.resourceLoader)] = realURL
|
||||
asset.resourceLoader.setDelegate(self, queue: .main)
|
||||
return asset
|
||||
}
|
||||
|
||||
/// Removes the loader mapping and any in-flight tasks for an asset that is
|
||||
/// no longer needed. Called by MediaPlayerController when replacing or
|
||||
/// clearing media items to bound the dictionaries (Finding 11).
|
||||
func removeLoader(_ asset: AVURLAsset) {
|
||||
let key = ObjectIdentifier(asset.resourceLoader)
|
||||
loaderURLs[key] = nil
|
||||
// Cancel any in-flight tasks. URLSessionDataTask has no isCancelled
|
||||
// property; calling cancel() on an already-cancelled task is a no-op.
|
||||
for (_, task) in activeTasks {
|
||||
task.cancel()
|
||||
}
|
||||
activeTasks.removeAll()
|
||||
}
|
||||
|
||||
func resourceLoader(
|
||||
_ resourceLoader: AVAssetResourceLoader,
|
||||
shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest
|
||||
) -> Bool {
|
||||
// All state access is on the main queue (the delegate queue).
|
||||
guard let dataRequest = loadingRequest.dataRequest else { return false }
|
||||
guard let url = loaderURLs[ObjectIdentifier(resourceLoader)] else {
|
||||
logger.log("no resolved URL for resource loader")
|
||||
loadingRequest.finishLoading(with: URLError(.badURL))
|
||||
return false
|
||||
}
|
||||
|
||||
var upstream = URLRequest(url: url)
|
||||
let offset = dataRequest.requestedOffset
|
||||
let length = Int64(dataRequest.requestedLength)
|
||||
if length > 0 {
|
||||
upstream.setValue("bytes=\(offset)-\(offset + length - 1)", forHTTPHeaderField: "Range")
|
||||
} else {
|
||||
upstream.setValue("bytes=\(offset)-", forHTTPHeaderField: "Range")
|
||||
}
|
||||
logger.log("fetching \(url.absoluteString, privacy: .public) range=\(upstream.value(forHTTPHeaderField: "Range") ?? "-", privacy: .public)")
|
||||
|
||||
// Key the task by the loading request's identity directly — no hash
|
||||
// derivation, so two distinct requests can never collide.
|
||||
let requestKey = ObjectIdentifier(loadingRequest)
|
||||
let task = pinnedSession.dataTask(with: upstream) { [weak self] data, response, error in
|
||||
guard let self else { return }
|
||||
// Cleanup: remove the task from the active map regardless of outcome.
|
||||
// Runs on .main (the session's delegateQueue), same domain as all
|
||||
// other map mutations.
|
||||
self.activeTasks[requestKey] = nil
|
||||
|
||||
if let error {
|
||||
logger.log("fetch failed \(url.absoluteString, privacy: .public): \(error.localizedDescription, privacy: .public)")
|
||||
loadingRequest.finishLoading(with: error)
|
||||
return
|
||||
}
|
||||
guard let http = response as? HTTPURLResponse else {
|
||||
loadingRequest.finishLoading(with: URLError(.badServerResponse))
|
||||
return
|
||||
}
|
||||
// Reject non-200/206 responses (404, 500, etc.) instead of feeding
|
||||
// error JSON into AVPlayer as media bytes.
|
||||
guard http.statusCode == 200 || http.statusCode == 206 else {
|
||||
logger.log("unexpected status \(http.statusCode) for \(url.absoluteString, privacy: .public)")
|
||||
loadingRequest.finishLoading(with: URLError(.badServerResponse))
|
||||
return
|
||||
}
|
||||
if let info = loadingRequest.contentInformationRequest {
|
||||
info.contentType = http.mimeType ?? "application/octet-stream"
|
||||
// Content-length handling (Iteration 05 Item 7):
|
||||
// - For 206: set full content length ONLY when Content-Range
|
||||
// provides a known total. Never fall back to the range's
|
||||
// expectedContentLength (that would tell AVPlayer the total
|
||||
// resource is only as large as this one range).
|
||||
// - For 200: expectedContentLength is already the full size.
|
||||
if http.statusCode == 206 {
|
||||
if let total = Self.parseTotalLength(http), total >= 0 {
|
||||
info.contentLength = total
|
||||
}
|
||||
// If Content-Range has no known total (e.g. "bytes 0-1023/*"),
|
||||
// do NOT set contentLength — AVPlayer will handle it as
|
||||
// unknown-length, which is correct for a range response.
|
||||
} else {
|
||||
// 200: expectedContentLength is the full resource size.
|
||||
if http.expectedContentLength >= 0 {
|
||||
info.contentLength = Int64(http.expectedContentLength)
|
||||
}
|
||||
}
|
||||
info.isByteRangeAccessSupported = true
|
||||
}
|
||||
if let data {
|
||||
dataRequest.respond(with: data)
|
||||
}
|
||||
loadingRequest.finishLoading()
|
||||
}
|
||||
activeTasks[requestKey] = task
|
||||
task.resume()
|
||||
return true
|
||||
}
|
||||
|
||||
func resourceLoader(
|
||||
_ resourceLoader: AVAssetResourceLoader,
|
||||
didCancel loadingRequest: AVAssetResourceLoadingRequest
|
||||
) {
|
||||
// Cancel the upstream data task so a seek or item swap doesn't keep
|
||||
// downloading cancelled ranges. Keyed by ObjectIdentifier directly.
|
||||
let key = ObjectIdentifier(loadingRequest)
|
||||
activeTasks[key]?.cancel()
|
||||
activeTasks[key] = nil
|
||||
logger.log("resource loader cancelled request, upstream task cancelled")
|
||||
}
|
||||
|
||||
/// Parses the total resource length from an HTTP 206 response's
|
||||
/// `Content-Range` header (e.g. "bytes 0-1023/4096" → 4096). Returns nil
|
||||
/// for non-206 responses, missing headers, unknown totals ("*"), or
|
||||
/// malformed values.
|
||||
static func parseTotalLength(_ response: HTTPURLResponse) -> Int64? {
|
||||
if response.statusCode == 206,
|
||||
let contentRange = response.value(forHTTPHeaderField: "Content-Range") {
|
||||
// Format: "bytes <start>-<end>/<total>" or "bytes <start>-<end>/*"
|
||||
let afterSlash = contentRange.split(separator: "/").last?.trimmingCharacters(in: .whitespaces)
|
||||
if let afterSlash, afterSlash != "*", let total = Int64(afterSlash) {
|
||||
return total
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
@@ -1,125 +0,0 @@
|
||||
import Foundation
|
||||
import Testing
|
||||
@testable import Voice
|
||||
|
||||
/// Minimal HTTP/1.1 stub built on BSD sockets that answers every request with a
|
||||
/// successful JSON object body (mirroring the real media server's
|
||||
/// `{"ok": true, ...}` shape). Used to prove MediaAPI no-response POSTs succeed
|
||||
/// against an object body instead of throwing decodeFailed.
|
||||
@MainActor
|
||||
final class MediaAPIStubServer {
|
||||
private(set) var url: URL!
|
||||
nonisolated(unsafe) private var serverFD: Int32 = -1
|
||||
nonisolated(unsafe) private var acceptThread: Thread?
|
||||
nonisolated(unsafe) private var running = false
|
||||
nonisolated(unsafe) private var lastPath: String?
|
||||
|
||||
func start() throws {
|
||||
let fd = socket(AF_INET, SOCK_STREAM, 0)
|
||||
guard fd >= 0 else { throw MediaAPIError.decodeFailed }
|
||||
var yes: Int32 = 1
|
||||
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, socklen_t(MemoryLayout<Int32>.size))
|
||||
|
||||
var addr = sockaddr_in()
|
||||
addr.sin_family = sa_family_t(AF_INET)
|
||||
addr.sin_port = in_port_t(0).bigEndian
|
||||
addr.sin_addr = in_addr(s_addr: inet_addr("127.0.0.1"))
|
||||
let bindResult = withUnsafePointer(to: &addr) { ptr in
|
||||
ptr.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
|
||||
bind(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size))
|
||||
}
|
||||
}
|
||||
guard bindResult == 0 else { close(fd); throw MediaAPIError.decodeFailed }
|
||||
guard listen(fd, 8) == 0 else { close(fd); throw MediaAPIError.decodeFailed }
|
||||
|
||||
var bound = sockaddr_in()
|
||||
var len = socklen_t(MemoryLayout<sockaddr_in>.size)
|
||||
withUnsafeMutablePointer(to: &bound) { ptr in
|
||||
_ = ptr.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
|
||||
getsockname(fd, sa, &len)
|
||||
}
|
||||
}
|
||||
let port = Int(UInt16(bigEndian: bound.sin_port))
|
||||
serverFD = fd
|
||||
running = true
|
||||
url = URL(string: "http://127.0.0.1:\(port)")!
|
||||
|
||||
let thread = Thread { [weak self] in
|
||||
self?.acceptLoop()
|
||||
}
|
||||
acceptThread = thread
|
||||
thread.start()
|
||||
}
|
||||
|
||||
func stop() {
|
||||
running = false
|
||||
if serverFD >= 0 { close(serverFD) }
|
||||
serverFD = -1
|
||||
acceptThread?.cancel()
|
||||
}
|
||||
|
||||
var lastRequestPath: String? { lastPath }
|
||||
|
||||
nonisolated private func acceptLoop() {
|
||||
while running {
|
||||
var clientAddr = sockaddr_in()
|
||||
var len = socklen_t(MemoryLayout<sockaddr_in>.size)
|
||||
let clientFD = withUnsafeMutablePointer(to: &clientAddr) { ptr in
|
||||
ptr.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
|
||||
accept(serverFD, sa, &len)
|
||||
}
|
||||
}
|
||||
guard clientFD >= 0 else { continue }
|
||||
handleClient(clientFD)
|
||||
}
|
||||
}
|
||||
|
||||
nonisolated private func handleClient(_ clientFD: Int32) {
|
||||
defer { close(clientFD) }
|
||||
var buffer = [UInt8](repeating: 0, count: 4096)
|
||||
let bytesRead = read(clientFD, &buffer, buffer.count)
|
||||
guard bytesRead > 0 else { return }
|
||||
guard let request = String(bytes: buffer[0..<bytesRead], encoding: .utf8) else { return }
|
||||
|
||||
var path = ""
|
||||
if let firstLine = request.split(separator: "\r\n").first {
|
||||
let parts = firstLine.split(separator: " ")
|
||||
if parts.count >= 2 { path = String(parts[1]) }
|
||||
}
|
||||
lastPath = path
|
||||
|
||||
let body = #"{"ok": true, "room": "test", "desired": "stopped"}"#
|
||||
let response = """
|
||||
HTTP/1.1 200 OK\r
|
||||
Content-Type: application/json\r
|
||||
Content-Length: \(body.utf8.count)\r
|
||||
Connection: close\r
|
||||
|
||||
\(body)
|
||||
"""
|
||||
_ = response.withCString { cstr in
|
||||
write(clientFD, cstr, response.utf8.count)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@MainActor
|
||||
struct MediaAPITests {
|
||||
@Test("successful JSON object response does not throw decodeFailed")
|
||||
func successfulObjectResponse() async throws {
|
||||
let stub = MediaAPIStubServer()
|
||||
try stub.start()
|
||||
defer { stub.stop() }
|
||||
|
||||
// Bypass the https-only parse path: build the config directly.
|
||||
let base = stub.url!
|
||||
let config = ServerConfig(baseURL: base, livekitURL: base)
|
||||
let api = MediaAPI(config: config)
|
||||
|
||||
try await api.control(room: "test", action: "pause")
|
||||
try await api.report(room: "test", state: "paused", trackID: nil, position: 0, duration: nil)
|
||||
try await api.play(room: "test", videoID: nil, title: nil, channel: nil, trackID: nil, startPaused: nil)
|
||||
|
||||
#expect(stub.lastRequestPath == "/media/api/play")
|
||||
}
|
||||
}
|
||||
@@ -1,124 +0,0 @@
|
||||
import Foundation
|
||||
import Testing
|
||||
@testable import Voice
|
||||
|
||||
@MainActor
|
||||
struct MediaReconcileTests {
|
||||
private func makeController() -> MediaPlayerController {
|
||||
let config = ServerConfig(
|
||||
baseURL: URL(string: "http://127.0.0.1:9")!,
|
||||
livekitURL: URL(string: "ws://127.0.0.1:9")!
|
||||
)
|
||||
let api = MediaAPI(config: config)
|
||||
return MediaPlayerController(api: api, settings: SettingsStore())
|
||||
}
|
||||
|
||||
private func makeState(
|
||||
desired: String? = "stopped",
|
||||
trackID: String? = nil,
|
||||
audioURL: String? = nil
|
||||
) -> MediaState {
|
||||
let track: MediaTrack? = trackID != nil ? MediaTrack(
|
||||
id: trackID!,
|
||||
kind: nil,
|
||||
videoID: nil,
|
||||
title: nil,
|
||||
channel: nil,
|
||||
duration: nil,
|
||||
status: nil,
|
||||
error: nil,
|
||||
audioURL: audioURL,
|
||||
artURL: nil
|
||||
) : nil
|
||||
return MediaState(
|
||||
room: "default",
|
||||
track: track,
|
||||
desired: desired,
|
||||
reported: nil,
|
||||
seekHint: nil,
|
||||
choices: [],
|
||||
pending: [],
|
||||
version: 1
|
||||
)
|
||||
}
|
||||
|
||||
@Test("reconcile updates published desired to playing")
|
||||
func reconcileDesiredPlaying() {
|
||||
let controller = makeController()
|
||||
#expect(controller.desired == "stopped")
|
||||
|
||||
controller.reconcile(makeState(desired: "playing"))
|
||||
#expect(controller.desired == "playing")
|
||||
}
|
||||
|
||||
@Test("reconcile updates published desired to paused")
|
||||
func reconcileDesiredPaused() {
|
||||
let controller = makeController()
|
||||
controller.reconcile(makeState(desired: "playing"))
|
||||
#expect(controller.desired == "playing")
|
||||
|
||||
controller.reconcile(makeState(desired: "paused"))
|
||||
#expect(controller.desired == "paused")
|
||||
}
|
||||
|
||||
@Test("reconcile updates published desired to stopped")
|
||||
func reconcileDesiredStopped() {
|
||||
let controller = makeController()
|
||||
controller.reconcile(makeState(desired: "playing"))
|
||||
#expect(controller.desired == "playing")
|
||||
|
||||
controller.reconcile(makeState(desired: "stopped"))
|
||||
#expect(controller.desired == "stopped")
|
||||
}
|
||||
|
||||
@Test("reconcile with nil desired defaults to stopped")
|
||||
func reconcileNilDesiredDefaultsToStopped() {
|
||||
let controller = makeController()
|
||||
controller.reconcile(makeState(desired: "playing"))
|
||||
#expect(controller.desired == "playing")
|
||||
|
||||
controller.reconcile(makeState(desired: nil))
|
||||
#expect(controller.desired == "stopped")
|
||||
}
|
||||
|
||||
@Test("late audio: desired=playing before audio_url, then audio_url arrives")
|
||||
func lateAudioReconciliation() {
|
||||
let controller = makeController()
|
||||
|
||||
// Phase 1: server says playing but the track has no audio_url yet
|
||||
// (still downloading). The source is not wired, so play cannot start.
|
||||
let stateNoURL = makeState(desired: "playing", trackID: "track-1", audioURL: nil)
|
||||
controller.reconcile(stateNoURL)
|
||||
#expect(controller.desired == "playing")
|
||||
#expect(controller.track?.id == "track-1")
|
||||
// Player rate should still be zero because no source was wired.
|
||||
#expect(controller.player.rate.isZero)
|
||||
|
||||
// Phase 2: the same track now has an audio_url. The reconcile path
|
||||
// wires the source (wireSource) and then applyDesiredState retries
|
||||
// play because desired is still "playing" and rate is still zero.
|
||||
let stateWithURL = makeState(desired: "playing", trackID: "track-1", audioURL: "/stream/track-1.mp3")
|
||||
controller.reconcile(stateWithURL)
|
||||
#expect(controller.desired == "playing")
|
||||
// The source should now be wired (loadedAudioPath is set internally).
|
||||
// We can't directly observe loadedAudioPath, but we can verify the
|
||||
// player has a current item after wireSource ran.
|
||||
#expect(controller.player.currentItem != nil)
|
||||
}
|
||||
|
||||
@Test("startDefaultPolling restarts after endConversation keeps scope")
|
||||
func pollingRestartsAfterEndConversation() async {
|
||||
let controller = makeController()
|
||||
controller.startDefaultPolling()
|
||||
await controller.endConversation()
|
||||
controller.startDefaultPolling()
|
||||
try? await Task.sleep(for: .milliseconds(50))
|
||||
}
|
||||
|
||||
@Test("togglePlayPause without a loaded track is a no-op")
|
||||
func toggleWithoutTrack() async {
|
||||
let controller = makeController()
|
||||
await controller.togglePlayPause()
|
||||
#expect(controller.track == nil)
|
||||
}
|
||||
}
|
||||
@@ -1,126 +0,0 @@
|
||||
import Foundation
|
||||
import Testing
|
||||
@testable import Voice
|
||||
|
||||
/// Focused tests for PinnedMediaResourceLoader.parseTotalLength, covering
|
||||
/// 200, 206 with known total, 206 with unknown total (*), and malformed
|
||||
/// Content-Range headers (Finding 1 / 11).
|
||||
struct PinnedMediaResourceLoaderTests {
|
||||
|
||||
private func makeResponse(status: Int, contentRange: String? = nil, expectedLength: Int64 = -1) -> HTTPURLResponse {
|
||||
var headers: [String: String] = [:]
|
||||
if let contentRange {
|
||||
headers["Content-Range"] = contentRange
|
||||
}
|
||||
return HTTPURLResponse(
|
||||
url: URL(string: "https://example.com/media/test.mp3")!,
|
||||
statusCode: status,
|
||||
httpVersion: "HTTP/1.1",
|
||||
headerFields: headers
|
||||
)!
|
||||
}
|
||||
|
||||
@Test("206 with known total parses Content-Range total")
|
||||
func parse206KnownTotal() {
|
||||
let response = makeResponse(status: 206, contentRange: "bytes 0-1023/4096", expectedLength: 1024)
|
||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == 4096)
|
||||
}
|
||||
|
||||
@Test("206 with unknown total (*) returns nil")
|
||||
func parse206UnknownTotal() {
|
||||
let response = makeResponse(status: 206, contentRange: "bytes 0-1023/*", expectedLength: 1024)
|
||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
||||
}
|
||||
|
||||
@Test("200 response returns nil (expectedContentLength is already the full size)")
|
||||
func parse200ReturnsNil() {
|
||||
let response = makeResponse(status: 200, expectedLength: 8192)
|
||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
||||
}
|
||||
|
||||
@Test("206 without Content-Range header returns nil")
|
||||
func parse206NoContentRange() {
|
||||
let response = makeResponse(status: 206, expectedLength: 512)
|
||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
||||
}
|
||||
|
||||
@Test("malformed Content-Range returns nil")
|
||||
func parseMalformedContentRange() {
|
||||
let response = makeResponse(status: 206, contentRange: "garbage", expectedLength: 100)
|
||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
||||
}
|
||||
|
||||
@Test("206 with large total parses correctly")
|
||||
func parseLargeTotal() {
|
||||
let response = makeResponse(status: 206, contentRange: "bytes 1048576-2097151/1073741824", expectedLength: 1048576)
|
||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == 1_073_741_824)
|
||||
}
|
||||
|
||||
@Test("streamURL constructs endpoint without downloading")
|
||||
func streamURLConstruction() {
|
||||
let config = ServerConfig(
|
||||
baseURL: URL(string: "https://192.168.86.2:9178")!,
|
||||
livekitURL: URL(string: "ws://192.168.86.2:7880")!
|
||||
)
|
||||
let api = MediaAPI(config: config)
|
||||
let url = api.streamURL(trackID: "track-123")
|
||||
// streamURL builds under the /media/api base (MediaAPI.baseURL).
|
||||
#expect(url.absoluteString == "https://192.168.86.2:9178/media/api/stream/track-123")
|
||||
}
|
||||
|
||||
// MARK: Rewrite / Resolve round-trip (Item 7)
|
||||
|
||||
@Test("rewrite maps https URL to pinned-media scheme and resolve inverts it")
|
||||
func rewriteResolveRoundTrip() {
|
||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
||||
let original = URL(string: "https://192.168.86.2:9178/media/track-123.mp3")!
|
||||
let pinned = loader.rewrite(original)
|
||||
#expect(pinned.scheme == "pinned-media")
|
||||
#expect(pinned.absoluteString == "pinned-media://192.168.86.2:9178/media/track-123.mp3")
|
||||
|
||||
let resolved = loader.resolve(pinned)
|
||||
#expect(resolved.absoluteString == original.absoluteString)
|
||||
}
|
||||
|
||||
@Test("rewrite maps path-only URL using base host and port")
|
||||
func rewritePathOnly() {
|
||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
||||
let pathURL = URL(string: "/media/track-456.mp3")!
|
||||
let pinned = loader.rewrite(pathURL)
|
||||
#expect(pinned.scheme == "pinned-media")
|
||||
#expect(pinned.absoluteString == "pinned-media://192.168.86.2:9178/media/track-456.mp3")
|
||||
}
|
||||
|
||||
@Test("resolve leaves non-pinned URLs unchanged")
|
||||
func resolveNonPinned() {
|
||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
||||
let https = URL(string: "https://example.com/foo.mp3")!
|
||||
#expect(loader.resolve(https) == https)
|
||||
}
|
||||
|
||||
// MARK: Asset lifecycle (Item 7)
|
||||
|
||||
@Test("assetWithPinnedURL creates asset and removeLoader clears mapping")
|
||||
func assetLifecycle() {
|
||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
||||
let pinned = loader.rewrite(URL(string: "https://192.168.86.2:9178/media/test.mp3")!)
|
||||
let asset = loader.assetWithPinnedURL(pinned)
|
||||
|
||||
// The asset should have a valid URL matching the pinned scheme.
|
||||
#expect(asset.url.scheme == PinnedMediaResourceLoader.scheme)
|
||||
|
||||
// Remove the loader mapping — subsequent calls should not crash.
|
||||
loader.removeLoader(asset)
|
||||
}
|
||||
|
||||
@Test("removeLoader is idempotent (safe to call twice)")
|
||||
func removeLoaderIdempotent() {
|
||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
||||
let pinned = loader.rewrite(URL(string: "https://192.168.86.2:9178/media/test.mp3")!)
|
||||
let asset = loader.assetWithPinnedURL(pinned)
|
||||
|
||||
loader.removeLoader(asset)
|
||||
// Second call should be a no-op, not a crash.
|
||||
loader.removeLoader(asset)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
import XCTest
|
||||
|
||||
/// Opt-in tests against the actual NAS deployment. No injected replies.
|
||||
final class VoiceLiveTests: XCTestCase {
|
||||
private var app: XCUIApplication!
|
||||
|
||||
override func record(_ issue: XCTIssue) {
|
||||
if let app, app.state == .runningForeground {
|
||||
capture("Failure state")
|
||||
let hierarchy = XCTAttachment(string: app.debugDescription)
|
||||
hierarchy.name = "Failure UI hierarchy"
|
||||
hierarchy.lifetime = .keepAlways
|
||||
add(hierarchy)
|
||||
}
|
||||
super.record(issue)
|
||||
}
|
||||
|
||||
override func tearDownWithError() throws {
|
||||
app?.terminate()
|
||||
}
|
||||
|
||||
override func setUpWithError() throws {
|
||||
try XCTSkipUnless(ProcessInfo.processInfo.environment["HOPE_RUN_LIVE_TESTS"] == "1", "Requires the live Hope server")
|
||||
continueAfterFailure = false
|
||||
app = XCUIApplication()
|
||||
app.launchArguments = ["--ui-testing", "--reset-ui-testing", "--live-testing"]
|
||||
app.launch()
|
||||
}
|
||||
|
||||
func testLiveConversationAndBackgroundTask() {
|
||||
app.buttons["Start chatting"].tap()
|
||||
let composer = app.descendants(matching: .any).matching(identifier: "Message Hope").firstMatch
|
||||
XCTAssertTrue(composer.waitForExistence(timeout: 60), "Agent must join the real LiveKit room")
|
||||
composer.tap()
|
||||
composer.typeText("What is six times seven? Reply in one short sentence.")
|
||||
app.buttons["Send message"].tap()
|
||||
let reply = app.staticTexts.matching(NSPredicate(format: "label CONTAINS[c] 'forty-two' OR label CONTAINS[c] 'forty two' OR label CONTAINS '42'")).firstMatch
|
||||
XCTAssertTrue(reply.waitForExistence(timeout: 60), "Qwen must produce a real response")
|
||||
capture("Live Qwen reply")
|
||||
|
||||
composer.tap()
|
||||
composer.typeText("Compare espresso and filter coffee in a short background task. Use your existing knowledge; no web search needed.")
|
||||
app.buttons["Send message"].tap()
|
||||
app.tabBars.buttons["Activity"].tap()
|
||||
let complete = app.staticTexts["Complete"].firstMatch
|
||||
XCTAssertTrue(complete.waitForExistence(timeout: 90), "Background task must finish and deliver its result")
|
||||
capture("Live background result")
|
||||
app.tabBars.buttons["Talk"].tap()
|
||||
app.buttons["End conversation"].tap()
|
||||
XCTAssertTrue(app.buttons["Start chatting"].waitForExistence(timeout: 20))
|
||||
// A second room must connect cleanly after the first was torn down.
|
||||
app.buttons["Start chatting"].tap()
|
||||
XCTAssertTrue(composer.waitForExistence(timeout: 60))
|
||||
app.buttons["End conversation"].tap()
|
||||
XCTAssertTrue(app.buttons["Start chatting"].waitForExistence(timeout: 20))
|
||||
}
|
||||
|
||||
private func capture(_ name: String) {
|
||||
let attachment = XCTAttachment(screenshot: app.screenshot())
|
||||
attachment.name = name
|
||||
attachment.lifetime = .keepAlways
|
||||
add(attachment)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
import XCTest
|
||||
|
||||
/// Real UI interactions with isolated preferences. These cover the client,
|
||||
/// including offline recovery; they do not claim to verify the live backend.
|
||||
final class VoiceUITests: XCTestCase {
|
||||
private var app: XCUIApplication!
|
||||
|
||||
override func setUpWithError() throws {
|
||||
continueAfterFailure = false
|
||||
app = XCUIApplication()
|
||||
app.launchArguments = ["--ui-testing", "--reset-ui-testing"]
|
||||
}
|
||||
|
||||
func testWelcomeNavigationAndOfflineRecovery() {
|
||||
app.launch()
|
||||
XCTAssertTrue(app.buttons["Start chatting"].waitForExistence(timeout: 10))
|
||||
XCTAssertTrue(app.staticTexts["Your microphone starts after you connect."].exists)
|
||||
app.buttons["Type"].tap()
|
||||
XCTAssertTrue(app.buttons["Start chatting"].exists)
|
||||
XCTAssertTrue(app.staticTexts["Your microphone stays off."].exists)
|
||||
app.buttons["Start chatting"].tap()
|
||||
XCTAssertTrue(app.buttons["Try again"].waitForExistence(timeout: 20))
|
||||
XCTAssertTrue(app.staticTexts["Couldn’t connect"].exists)
|
||||
app.buttons["Settings"].tap()
|
||||
XCTAssertTrue(app.textFields["Server URL"].waitForExistence(timeout: 5))
|
||||
capture("Offline recovery")
|
||||
}
|
||||
|
||||
func testVoiceSelectionPersistsAndURLValidation() {
|
||||
app.launch()
|
||||
app.buttons["Settings"].tap()
|
||||
app.buttons.matching(NSPredicate(format: "label CONTAINS 'Speaking voice'")).firstMatch.tap()
|
||||
let voice = app.buttons.matching(NSPredicate(format: "label CONTAINS 'Jenny'")).firstMatch
|
||||
XCTAssertTrue(voice.waitForExistence(timeout: 5))
|
||||
voice.tap()
|
||||
XCTAssertTrue(app.buttons.matching(NSPredicate(format: "label CONTAINS 'Speaking voice' AND label CONTAINS 'Jenny'")).firstMatch.exists)
|
||||
let server = app.textFields["Server URL"]
|
||||
server.tap()
|
||||
let original = server.value as? String ?? ""
|
||||
server.typeText(String(repeating: XCUIKeyboardKey.delete.rawValue, count: original.count) + "http://example.com")
|
||||
app.buttons["Save server address"].tap()
|
||||
XCTAssertTrue(app.staticTexts.matching(NSPredicate(format: "label CONTAINS 'must use https'")).firstMatch.waitForExistence(timeout: 5))
|
||||
|
||||
let invalid = server.value as? String ?? ""
|
||||
server.tap()
|
||||
server.typeText(String(repeating: XCUIKeyboardKey.delete.rawValue, count: invalid.count) + " https://127.0.0.1:1 ")
|
||||
app.buttons["Save server address"].tap()
|
||||
XCTAssertEqual(server.value as? String, "https://127.0.0.1:1")
|
||||
XCTAssertTrue(app.staticTexts["Server address saved."].waitForExistence(timeout: 5))
|
||||
|
||||
let unchanged = server.value as? String ?? ""
|
||||
server.tap()
|
||||
server.typeText(String(repeating: XCUIKeyboardKey.delete.rawValue, count: unchanged.count) + " https://127.0.0.1:2 ")
|
||||
app.buttons["Save server address"].tap()
|
||||
Thread.sleep(forTimeInterval: 2)
|
||||
XCTAssertTrue(app.navigationBars["Settings"].exists)
|
||||
XCTAssertTrue(app.staticTexts["Server address saved."].exists)
|
||||
XCTAssertEqual(server.value as? String, "https://127.0.0.1:2")
|
||||
|
||||
app.terminate()
|
||||
app.launchArguments = ["--ui-testing"]
|
||||
app.launch()
|
||||
app.buttons["Settings"].tap()
|
||||
XCTAssertTrue(app.buttons.matching(NSPredicate(format: "label CONTAINS 'Speaking voice' AND label CONTAINS 'Jenny'")).firstMatch.waitForExistence(timeout: 5))
|
||||
XCTAssertEqual(app.textFields["Server URL"].value as? String, "https://127.0.0.1:2")
|
||||
}
|
||||
|
||||
|
||||
func testTranscriptClearConfirmationAndActivityDetails() {
|
||||
app.launchArguments.append("--ui-fixtures")
|
||||
app.launch()
|
||||
XCTAssertTrue(app.staticTexts["Help me plan a quiet weekend."].waitForExistence(timeout: 10))
|
||||
app.buttons["Conversation options"].tap()
|
||||
app.buttons["Clear conversation"].tap()
|
||||
app.buttons["Cancel"].tap()
|
||||
XCTAssertTrue(app.staticTexts["Help me plan a quiet weekend."].exists)
|
||||
app.buttons["Conversation options"].tap()
|
||||
app.buttons["Clear conversation"].tap()
|
||||
app.buttons["Clear conversation"].tap()
|
||||
XCTAssertTrue(app.buttons["Start chatting"].exists)
|
||||
XCTAssertFalse(app.staticTexts["Help me plan a quiet weekend."].exists)
|
||||
app.tabBars.buttons["Activity"].tap()
|
||||
XCTAssertTrue(app.staticTexts["Needs attention"].waitForExistence(timeout: 5))
|
||||
XCTAssertTrue(app.staticTexts["Search backend timed out after 30s"].exists)
|
||||
app.staticTexts["Search backend timed out after 30s"].tap()
|
||||
XCTAssertTrue(app.staticTexts["Fetching benchmark articles…"].waitForExistence(timeout: 5))
|
||||
capture("Activity details")
|
||||
}
|
||||
|
||||
func testRestoredTranscriptStaysAtTopUntilLatest() {
|
||||
app.launchArguments.append("--ui-long-transcript")
|
||||
app.launch()
|
||||
|
||||
let first = app.staticTexts.matching(NSPredicate(format: "label CONTAINS 'Earlier message 1:'")).firstMatch
|
||||
let last = app.staticTexts.matching(NSPredicate(format: "label CONTAINS 'Earlier message 30:'")).firstMatch
|
||||
XCTAssertTrue(first.waitForExistence(timeout: 10))
|
||||
XCTAssertTrue(app.buttons["Latest"].exists)
|
||||
XCTAssertFalse(last.isHittable)
|
||||
|
||||
app.buttons["Latest"].tap()
|
||||
XCTAssertTrue(last.waitForExistence(timeout: 5))
|
||||
XCTAssertTrue(last.isHittable)
|
||||
}
|
||||
|
||||
func testLargeTypeNavigation() {
|
||||
app.launchArguments += ["-UIPreferredContentSizeCategoryName", "UICTContentSizeCategoryAccessibilityXXXL"]
|
||||
app.launch()
|
||||
XCTAssertTrue(app.buttons["Start chatting"].waitForExistence(timeout: 10))
|
||||
XCTAssertTrue(app.buttons["Start chatting"].isHittable)
|
||||
app.tabBars.buttons["Activity"].tap()
|
||||
XCTAssertTrue(app.staticTexts["No tasks yet"].waitForExistence(timeout: 5))
|
||||
app.buttons["Settings"].tap()
|
||||
XCTAssertTrue(app.navigationBars["Settings"].exists)
|
||||
capture("Accessibility text size")
|
||||
}
|
||||
|
||||
private func capture(_ name: String) {
|
||||
let attachment = XCTAttachment(screenshot: app.screenshot())
|
||||
attachment.name = name
|
||||
attachment.lifetime = .keepAlways
|
||||
add(attachment)
|
||||
}
|
||||
}
|
||||
+120
-49
@@ -12,29 +12,26 @@
|
||||
28659D009399CA9C4FE57B74 /* ControlChannel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 61C71C48230C683D199D6F4B /* ControlChannel.swift */; };
|
||||
302BEC55CA9A10E49C307977 /* ServerConfig.swift in Sources */ = {isa = PBXBuildFile; fileRef = F5683993A1DE677121FFA1F3 /* ServerConfig.swift */; };
|
||||
36DF44D9A7896F18A064847D /* ConversationView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3F3FA9F420065D2127B0AEC0 /* ConversationView.swift */; };
|
||||
3E01A0AF44FD3324FA72C43C /* OrbView.swift in Sources */ = {isa = PBXBuildFile; fileRef = A62BAE22E3A165DCB5DFFEE1 /* OrbView.swift */; };
|
||||
57E6476135CE30495E02D723 /* VoiceUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8309AC6885ED7EB0047AE211 /* VoiceUITests.swift */; };
|
||||
64291BBC8D0DAD47E07E94E0 /* ConversationState.swift in Sources */ = {isa = PBXBuildFile; fileRef = A0C23305A6C55A83F8617D9D /* ConversationState.swift */; };
|
||||
70E4B860147494D9657E8C01 /* VoiceLiveTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5F6A697985B36B3A1A3F2AD5 /* VoiceLiveTests.swift */; };
|
||||
720C26EC163AE46713B02DD8 /* TLSPinning.swift in Sources */ = {isa = PBXBuildFile; fileRef = D9B40076934C28BA4B92D808 /* TLSPinning.swift */; };
|
||||
72F3B4AEC214521015200583 /* MediaReconcileTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 09FB71841B3D72CD1D69E47A /* MediaReconcileTests.swift */; };
|
||||
73DA73811F8AEB2AFE340D10 /* MediaPlayerController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3964F565A6C6A06399EBA4DF /* MediaPlayerController.swift */; };
|
||||
89B47754E5C39418BFD839E6 /* TranscriptHistory.swift in Sources */ = {isa = PBXBuildFile; fileRef = 68A45228EF59186CFC219F2E /* TranscriptHistory.swift */; };
|
||||
8DE1363CC0119B5AF9E4E117 /* ConnectionTestView.swift in Sources */ = {isa = PBXBuildFile; fileRef = AC4B9073EE8EAB51F8C55760 /* ConnectionTestView.swift */; };
|
||||
943BCE971A01F220FAE9E250 /* TaskPanelView.swift in Sources */ = {isa = PBXBuildFile; fileRef = E85BACA267ADD462306C9920 /* TaskPanelView.swift */; };
|
||||
9461C0886FA294FDD9742E51 /* TranscriptListView.swift in Sources */ = {isa = PBXBuildFile; fileRef = BD31F76E1880AC76ED31DD4D /* TranscriptListView.swift */; };
|
||||
98BA1A0F92DB7B98A8BA7814 /* SettingsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 48F76881B4AE51DEE74511CC /* SettingsView.swift */; };
|
||||
A5868238CF4958D6D0C5908E /* TaskStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = C0B60FBBB9C47BB9ADACA7CA /* TaskStore.swift */; };
|
||||
A86EFB492F40C26D4AA315E9 /* PinnedMediaResourceLoader.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6DC3AC57ED58FC517E212F19 /* PinnedMediaResourceLoader.swift */; };
|
||||
A66BC894BEEC870D879F991F /* LightwellView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 87DD77AA383E1ECCEEEFB399 /* LightwellView.swift */; };
|
||||
A94F6C509E3FE215115B2C58 /* SettingsStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6D0D153557BB1C756ED04C61 /* SettingsStore.swift */; };
|
||||
B94FAFA796A9AF6011B03DEC /* PinnedMediaResourceLoaderTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8A20F8AFF1AAC5A944173549 /* PinnedMediaResourceLoaderTests.swift */; };
|
||||
BCAB602C722D74F6C96C5E36 /* ServerReplacementCoordinatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */; };
|
||||
C00D1878C2AB3D8B6F9562B6 /* AgentMonitor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0FC167F6658CFEC9546A7E9F /* AgentMonitor.swift */; };
|
||||
C956D0FDFAC43925979E4077 /* MediaAPI.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9191AD60FF96E44F191864E3 /* MediaAPI.swift */; };
|
||||
CDE38FE9679472D770C601E6 /* VoiceSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = EB58EEF9490F81FB2E1CE31C /* VoiceSession.swift */; };
|
||||
D18AD347DFF72AB6361165BA /* TokenClient.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9681E7410B684D2D38752A70 /* TokenClient.swift */; };
|
||||
D2E343C8B8657316A993698E /* TranscriptHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = D82D665165A7BC1A4CBCCC6F /* TranscriptHandler.swift */; };
|
||||
D958BE8EE0FF9568F8FABB11 /* Nearlight.swift in Sources */ = {isa = PBXBuildFile; fileRef = CA6FA2A93116B237CEA712E6 /* Nearlight.swift */; };
|
||||
D9A0059E342955A06A021D6C /* LiveKit in Frameworks */ = {isa = PBXBuildFile; productRef = A6BAD55715F383C0DB5F306F /* LiveKit */; };
|
||||
DE1F24919E906EE13319C7A3 /* MediaPanelView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 68FACB1A5EA6FEFD1D2613E9 /* MediaPanelView.swift */; };
|
||||
EE0C5762EC97B123A1880FFB /* Voices.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2E5DF207EDB787F63CB3A64D /* Voices.swift */; };
|
||||
F26037AD028ED105E127CBD3 /* MediaAPITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 10C194656F19E89DFD656A91 /* MediaAPITests.swift */; };
|
||||
FC696C6E74057D4C93E6DA67 /* ConversationStateTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BDD0AA8B0E9B45A325D1E5E7 /* ConversationStateTests.swift */; };
|
||||
/* End PBXBuildFile section */
|
||||
|
||||
@@ -46,33 +43,38 @@
|
||||
remoteGlobalIDString = 57E3A292BDE525A22EB7ADE4;
|
||||
remoteInfo = Voice_iOS;
|
||||
};
|
||||
ED7A5BB58354A8C9A5994257 /* PBXContainerItemProxy */ = {
|
||||
isa = PBXContainerItemProxy;
|
||||
containerPortal = 201BFA602F0A78910611E5F5 /* Project object */;
|
||||
proxyType = 1;
|
||||
remoteGlobalIDString = 57E3A292BDE525A22EB7ADE4;
|
||||
remoteInfo = Voice_iOS;
|
||||
};
|
||||
/* End PBXContainerItemProxy section */
|
||||
|
||||
/* Begin PBXFileReference section */
|
||||
09FB71841B3D72CD1D69E47A /* MediaReconcileTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaReconcileTests.swift; sourceTree = "<group>"; };
|
||||
0FC167F6658CFEC9546A7E9F /* AgentMonitor.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AgentMonitor.swift; sourceTree = "<group>"; };
|
||||
10C194656F19E89DFD656A91 /* MediaAPITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaAPITests.swift; sourceTree = "<group>"; };
|
||||
20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ServerReplacementCoordinatorTests.swift; sourceTree = "<group>"; };
|
||||
2A4C3CE2CA6FF828452803D1 /* Voice.app */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.application; path = Voice.app; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||
2E5DF207EDB787F63CB3A64D /* Voices.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Voices.swift; sourceTree = "<group>"; };
|
||||
2FED8976B60FBC13BC980FCB /* VoiceApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceApp.swift; sourceTree = "<group>"; };
|
||||
3964F565A6C6A06399EBA4DF /* MediaPlayerController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaPlayerController.swift; sourceTree = "<group>"; };
|
||||
3F3FA9F420065D2127B0AEC0 /* ConversationView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConversationView.swift; sourceTree = "<group>"; };
|
||||
48F76881B4AE51DEE74511CC /* SettingsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsView.swift; sourceTree = "<group>"; };
|
||||
5759EB570DB87BBC121D946D /* VoiceUITests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = VoiceUITests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||
5F6A697985B36B3A1A3F2AD5 /* VoiceLiveTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceLiveTests.swift; sourceTree = "<group>"; };
|
||||
61C71C48230C683D199D6F4B /* ControlChannel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ControlChannel.swift; sourceTree = "<group>"; };
|
||||
68A45228EF59186CFC219F2E /* TranscriptHistory.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TranscriptHistory.swift; sourceTree = "<group>"; };
|
||||
68FACB1A5EA6FEFD1D2613E9 /* MediaPanelView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaPanelView.swift; sourceTree = "<group>"; };
|
||||
6D0D153557BB1C756ED04C61 /* SettingsStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsStore.swift; sourceTree = "<group>"; };
|
||||
6DC3AC57ED58FC517E212F19 /* PinnedMediaResourceLoader.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PinnedMediaResourceLoader.swift; sourceTree = "<group>"; };
|
||||
8A20F8AFF1AAC5A944173549 /* PinnedMediaResourceLoaderTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PinnedMediaResourceLoaderTests.swift; sourceTree = "<group>"; };
|
||||
9191AD60FF96E44F191864E3 /* MediaAPI.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaAPI.swift; sourceTree = "<group>"; };
|
||||
8309AC6885ED7EB0047AE211 /* VoiceUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceUITests.swift; sourceTree = "<group>"; };
|
||||
87DD77AA383E1ECCEEEFB399 /* LightwellView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LightwellView.swift; sourceTree = "<group>"; };
|
||||
9681E7410B684D2D38752A70 /* TokenClient.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TokenClient.swift; sourceTree = "<group>"; };
|
||||
A0C23305A6C55A83F8617D9D /* ConversationState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConversationState.swift; sourceTree = "<group>"; };
|
||||
A62BAE22E3A165DCB5DFFEE1 /* OrbView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OrbView.swift; sourceTree = "<group>"; };
|
||||
AC4B9073EE8EAB51F8C55760 /* ConnectionTestView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConnectionTestView.swift; sourceTree = "<group>"; };
|
||||
BD31F76E1880AC76ED31DD4D /* TranscriptListView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TranscriptListView.swift; sourceTree = "<group>"; };
|
||||
BDD0AA8B0E9B45A325D1E5E7 /* ConversationStateTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConversationStateTests.swift; sourceTree = "<group>"; };
|
||||
C0B60FBBB9C47BB9ADACA7CA /* TaskStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TaskStore.swift; sourceTree = "<group>"; };
|
||||
C22975C69240AACD3418091F /* VoiceTests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = VoiceTests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||
CA6FA2A93116B237CEA712E6 /* Nearlight.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Nearlight.swift; sourceTree = "<group>"; };
|
||||
D82D665165A7BC1A4CBCCC6F /* TranscriptHandler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TranscriptHandler.swift; sourceTree = "<group>"; };
|
||||
D9B40076934C28BA4B92D808 /* TLSPinning.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TLSPinning.swift; sourceTree = "<group>"; };
|
||||
E85BACA267ADD462306C9920 /* TaskPanelView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TaskPanelView.swift; sourceTree = "<group>"; };
|
||||
@@ -93,14 +95,6 @@
|
||||
/* End PBXFrameworksBuildPhase section */
|
||||
|
||||
/* Begin PBXGroup section */
|
||||
0D2E974CBC286D7C886BA814 /* Media */ = {
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
68FACB1A5EA6FEFD1D2613E9 /* MediaPanelView.swift */,
|
||||
);
|
||||
path = Media;
|
||||
sourceTree = "<group>";
|
||||
};
|
||||
0DF0591DF1D767F89099F62C /* Settings */ = {
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
@@ -124,6 +118,15 @@
|
||||
path = LiveKit;
|
||||
sourceTree = "<group>";
|
||||
};
|
||||
6735DFBC89D9A939C44D763A /* UITests */ = {
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
5F6A697985B36B3A1A3F2AD5 /* VoiceLiveTests.swift */,
|
||||
8309AC6885ED7EB0047AE211 /* VoiceUITests.swift */,
|
||||
);
|
||||
path = UITests;
|
||||
sourceTree = "<group>";
|
||||
};
|
||||
6B2AB34274D9B37E77178C6C /* Networking */ = {
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
@@ -139,6 +142,7 @@
|
||||
children = (
|
||||
2A4C3CE2CA6FF828452803D1 /* Voice.app */,
|
||||
C22975C69240AACD3418091F /* VoiceTests.xctest */,
|
||||
5759EB570DB87BBC121D946D /* VoiceUITests.xctest */,
|
||||
);
|
||||
name = Products;
|
||||
sourceTree = "<group>";
|
||||
@@ -147,7 +151,8 @@
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
3F3FA9F420065D2127B0AEC0 /* ConversationView.swift */,
|
||||
A62BAE22E3A165DCB5DFFEE1 /* OrbView.swift */,
|
||||
87DD77AA383E1ECCEEEFB399 /* LightwellView.swift */,
|
||||
BD31F76E1880AC76ED31DD4D /* TranscriptListView.swift */,
|
||||
);
|
||||
path = Conversation;
|
||||
sourceTree = "<group>";
|
||||
@@ -156,7 +161,6 @@
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
84ADAF12A43A57888CA178A6 /* Conversation */,
|
||||
0D2E974CBC286D7C886BA814 /* Media */,
|
||||
0DF0591DF1D767F89099F62C /* Settings */,
|
||||
F168B25F8B3C6F36B6E88228 /* Tasks */,
|
||||
);
|
||||
@@ -168,23 +172,15 @@
|
||||
children = (
|
||||
FCC61E9990B71498171F9D34 /* Sources */,
|
||||
DC8495F2B27E1770E6620A05 /* Tests */,
|
||||
6735DFBC89D9A939C44D763A /* UITests */,
|
||||
6F99423E59017FED59B4D71D /* Products */,
|
||||
);
|
||||
sourceTree = "<group>";
|
||||
};
|
||||
C12426487989358DE21A5426 /* Media */ = {
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
9191AD60FF96E44F191864E3 /* MediaAPI.swift */,
|
||||
3964F565A6C6A06399EBA4DF /* MediaPlayerController.swift */,
|
||||
6DC3AC57ED58FC517E212F19 /* PinnedMediaResourceLoader.swift */,
|
||||
);
|
||||
path = Media;
|
||||
sourceTree = "<group>";
|
||||
};
|
||||
CE49A38E40AAF659A9281B85 /* App */ = {
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
CA6FA2A93116B237CEA712E6 /* Nearlight.swift */,
|
||||
6D0D153557BB1C756ED04C61 /* SettingsStore.swift */,
|
||||
2FED8976B60FBC13BC980FCB /* VoiceApp.swift */,
|
||||
2E5DF207EDB787F63CB3A64D /* Voices.swift */,
|
||||
@@ -196,9 +192,6 @@
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
BDD0AA8B0E9B45A325D1E5E7 /* ConversationStateTests.swift */,
|
||||
10C194656F19E89DFD656A91 /* MediaAPITests.swift */,
|
||||
09FB71841B3D72CD1D69E47A /* MediaReconcileTests.swift */,
|
||||
8A20F8AFF1AAC5A944173549 /* PinnedMediaResourceLoaderTests.swift */,
|
||||
20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */,
|
||||
FAEF89B2DEEE68F7168A8014 /* TranscriptDedupeTests.swift */,
|
||||
);
|
||||
@@ -219,7 +212,6 @@
|
||||
CE49A38E40AAF659A9281B85 /* App */,
|
||||
8DA8BEF2EE3E0F4181A94EB9 /* Features */,
|
||||
2820A418F921E18A2FA7E4FB /* LiveKit */,
|
||||
C12426487989358DE21A5426 /* Media */,
|
||||
6B2AB34274D9B37E77178C6C /* Networking */,
|
||||
);
|
||||
path = Sources;
|
||||
@@ -246,6 +238,24 @@
|
||||
productReference = C22975C69240AACD3418091F /* VoiceTests.xctest */;
|
||||
productType = "com.apple.product-type.bundle.unit-test";
|
||||
};
|
||||
1C8CC2859E99A4AC83FC6374 /* VoiceUITests */ = {
|
||||
isa = PBXNativeTarget;
|
||||
buildConfigurationList = 286F46F3E4A225D8ABDCEA48 /* Build configuration list for PBXNativeTarget "VoiceUITests" */;
|
||||
buildPhases = (
|
||||
889523F0856C91F5A5E8FD41 /* Sources */,
|
||||
);
|
||||
buildRules = (
|
||||
);
|
||||
dependencies = (
|
||||
A506145EF5A1B3978391C33C /* PBXTargetDependency */,
|
||||
);
|
||||
name = VoiceUITests;
|
||||
packageProductDependencies = (
|
||||
);
|
||||
productName = VoiceUITests;
|
||||
productReference = 5759EB570DB87BBC121D946D /* VoiceUITests.xctest */;
|
||||
productType = "com.apple.product-type.bundle.ui-testing";
|
||||
};
|
||||
57E3A292BDE525A22EB7ADE4 /* Voice_iOS */ = {
|
||||
isa = PBXNativeTarget;
|
||||
buildConfigurationList = 67CE95EFE28F976D1CF1A59A /* Build configuration list for PBXNativeTarget "Voice_iOS" */;
|
||||
@@ -274,7 +284,11 @@
|
||||
BuildIndependentTargetsInParallel = YES;
|
||||
LastUpgradeCheck = 1430;
|
||||
TargetAttributes = {
|
||||
1C8CC2859E99A4AC83FC6374 = {
|
||||
TestTargetID = 57E3A292BDE525A22EB7ADE4;
|
||||
};
|
||||
57E3A292BDE525A22EB7ADE4 = {
|
||||
DevelopmentTeam = YJFYRB92NW;
|
||||
ProvisioningStyle = Automatic;
|
||||
};
|
||||
};
|
||||
@@ -296,8 +310,9 @@
|
||||
projectDirPath = "";
|
||||
projectRoot = "";
|
||||
targets = (
|
||||
1385EEF29189F331F2634382 /* VoiceTests_iOS */,
|
||||
57E3A292BDE525A22EB7ADE4 /* Voice_iOS */,
|
||||
1385EEF29189F331F2634382 /* VoiceTests_iOS */,
|
||||
1C8CC2859E99A4AC83FC6374 /* VoiceUITests */,
|
||||
);
|
||||
};
|
||||
/* End PBXProject section */
|
||||
@@ -312,11 +327,8 @@
|
||||
28659D009399CA9C4FE57B74 /* ControlChannel.swift in Sources */,
|
||||
64291BBC8D0DAD47E07E94E0 /* ConversationState.swift in Sources */,
|
||||
36DF44D9A7896F18A064847D /* ConversationView.swift in Sources */,
|
||||
C956D0FDFAC43925979E4077 /* MediaAPI.swift in Sources */,
|
||||
DE1F24919E906EE13319C7A3 /* MediaPanelView.swift in Sources */,
|
||||
73DA73811F8AEB2AFE340D10 /* MediaPlayerController.swift in Sources */,
|
||||
3E01A0AF44FD3324FA72C43C /* OrbView.swift in Sources */,
|
||||
A86EFB492F40C26D4AA315E9 /* PinnedMediaResourceLoader.swift in Sources */,
|
||||
A66BC894BEEC870D879F991F /* LightwellView.swift in Sources */,
|
||||
D958BE8EE0FF9568F8FABB11 /* Nearlight.swift in Sources */,
|
||||
302BEC55CA9A10E49C307977 /* ServerConfig.swift in Sources */,
|
||||
A94F6C509E3FE215115B2C58 /* SettingsStore.swift in Sources */,
|
||||
98BA1A0F92DB7B98A8BA7814 /* SettingsView.swift in Sources */,
|
||||
@@ -326,20 +338,27 @@
|
||||
D18AD347DFF72AB6361165BA /* TokenClient.swift in Sources */,
|
||||
D2E343C8B8657316A993698E /* TranscriptHandler.swift in Sources */,
|
||||
89B47754E5C39418BFD839E6 /* TranscriptHistory.swift in Sources */,
|
||||
9461C0886FA294FDD9742E51 /* TranscriptListView.swift in Sources */,
|
||||
1ECD63F10A8EAF33FD2A5B62 /* VoiceApp.swift in Sources */,
|
||||
CDE38FE9679472D770C601E6 /* VoiceSession.swift in Sources */,
|
||||
EE0C5762EC97B123A1880FFB /* Voices.swift in Sources */,
|
||||
);
|
||||
runOnlyForDeploymentPostprocessing = 0;
|
||||
};
|
||||
889523F0856C91F5A5E8FD41 /* Sources */ = {
|
||||
isa = PBXSourcesBuildPhase;
|
||||
buildActionMask = 2147483647;
|
||||
files = (
|
||||
70E4B860147494D9657E8C01 /* VoiceLiveTests.swift in Sources */,
|
||||
57E6476135CE30495E02D723 /* VoiceUITests.swift in Sources */,
|
||||
);
|
||||
runOnlyForDeploymentPostprocessing = 0;
|
||||
};
|
||||
C735E25A3F84D701AEBFC3F1 /* Sources */ = {
|
||||
isa = PBXSourcesBuildPhase;
|
||||
buildActionMask = 2147483647;
|
||||
files = (
|
||||
FC696C6E74057D4C93E6DA67 /* ConversationStateTests.swift in Sources */,
|
||||
F26037AD028ED105E127CBD3 /* MediaAPITests.swift in Sources */,
|
||||
72F3B4AEC214521015200583 /* MediaReconcileTests.swift in Sources */,
|
||||
B94FAFA796A9AF6011B03DEC /* PinnedMediaResourceLoaderTests.swift in Sources */,
|
||||
BCAB602C722D74F6C96C5E36 /* ServerReplacementCoordinatorTests.swift in Sources */,
|
||||
1813A1CB0921D04FCAEC92D0 /* TranscriptDedupeTests.swift in Sources */,
|
||||
);
|
||||
@@ -353,6 +372,11 @@
|
||||
target = 57E3A292BDE525A22EB7ADE4 /* Voice_iOS */;
|
||||
targetProxy = 1C0045D170F971FC571D0E54 /* PBXContainerItemProxy */;
|
||||
};
|
||||
A506145EF5A1B3978391C33C /* PBXTargetDependency */ = {
|
||||
isa = PBXTargetDependency;
|
||||
target = 57E3A292BDE525A22EB7ADE4 /* Voice_iOS */;
|
||||
targetProxy = ED7A5BB58354A8C9A5994257 /* PBXContainerItemProxy */;
|
||||
};
|
||||
/* End PBXTargetDependency section */
|
||||
|
||||
/* Begin XCBuildConfiguration section */
|
||||
@@ -438,6 +462,24 @@
|
||||
};
|
||||
name = Debug;
|
||||
};
|
||||
452ED10E7C45C3EDEDC466B5 /* Release */ = {
|
||||
isa = XCBuildConfiguration;
|
||||
buildSettings = {
|
||||
BUNDLE_LOADER = "$(TEST_HOST)";
|
||||
GENERATE_INFOPLIST_FILE = YES;
|
||||
LD_RUNPATH_SEARCH_PATHS = (
|
||||
"$(inherited)",
|
||||
"@executable_path/Frameworks",
|
||||
"@loader_path/Frameworks",
|
||||
);
|
||||
PRODUCT_BUNDLE_IDENTIFIER = dev.shane.voice.uitests;
|
||||
SDKROOT = iphoneos;
|
||||
SWIFT_VERSION = 5.9;
|
||||
TARGETED_DEVICE_FAMILY = "1,2";
|
||||
TEST_TARGET_NAME = Voice_iOS;
|
||||
};
|
||||
name = Release;
|
||||
};
|
||||
8CD9F2FF6EF1EB5438545350 /* Release */ = {
|
||||
isa = XCBuildConfiguration;
|
||||
buildSettings = {
|
||||
@@ -464,6 +506,7 @@
|
||||
CODE_SIGN_IDENTITY = "iPhone Developer";
|
||||
CODE_SIGN_STYLE = Automatic;
|
||||
CURRENT_PROJECT_VERSION = 1;
|
||||
DEVELOPMENT_TEAM = YJFYRB92NW;
|
||||
GENERATE_INFOPLIST_FILE = NO;
|
||||
INFOPLIST_FILE = Resources/Info.plist;
|
||||
LD_RUNPATH_SEARCH_PATHS = (
|
||||
@@ -486,6 +529,7 @@
|
||||
CODE_SIGN_IDENTITY = "iPhone Developer";
|
||||
CODE_SIGN_STYLE = Automatic;
|
||||
CURRENT_PROJECT_VERSION = 1;
|
||||
DEVELOPMENT_TEAM = YJFYRB92NW;
|
||||
GENERATE_INFOPLIST_FILE = NO;
|
||||
INFOPLIST_FILE = Resources/Info.plist;
|
||||
LD_RUNPATH_SEARCH_PATHS = (
|
||||
@@ -557,9 +601,36 @@
|
||||
};
|
||||
name = Release;
|
||||
};
|
||||
EEDD1CAF56A9063A94459A9B /* Debug */ = {
|
||||
isa = XCBuildConfiguration;
|
||||
buildSettings = {
|
||||
BUNDLE_LOADER = "$(TEST_HOST)";
|
||||
GENERATE_INFOPLIST_FILE = YES;
|
||||
LD_RUNPATH_SEARCH_PATHS = (
|
||||
"$(inherited)",
|
||||
"@executable_path/Frameworks",
|
||||
"@loader_path/Frameworks",
|
||||
);
|
||||
PRODUCT_BUNDLE_IDENTIFIER = dev.shane.voice.uitests;
|
||||
SDKROOT = iphoneos;
|
||||
SWIFT_VERSION = 5.9;
|
||||
TARGETED_DEVICE_FAMILY = "1,2";
|
||||
TEST_TARGET_NAME = Voice_iOS;
|
||||
};
|
||||
name = Debug;
|
||||
};
|
||||
/* End XCBuildConfiguration section */
|
||||
|
||||
/* Begin XCConfigurationList section */
|
||||
286F46F3E4A225D8ABDCEA48 /* Build configuration list for PBXNativeTarget "VoiceUITests" */ = {
|
||||
isa = XCConfigurationList;
|
||||
buildConfigurations = (
|
||||
EEDD1CAF56A9063A94459A9B /* Debug */,
|
||||
452ED10E7C45C3EDEDC466B5 /* Release */,
|
||||
);
|
||||
defaultConfigurationIsVisible = 0;
|
||||
defaultConfigurationName = Debug;
|
||||
};
|
||||
67CE95EFE28F976D1CF1A59A /* Build configuration list for PBXNativeTarget "Voice_iOS" */ = {
|
||||
isa = XCConfigurationList;
|
||||
buildConfigurations = (
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# NAS deployment — 2026-09-08
|
||||
|
||||
The `voice` container on `192.168.86.2` hosts LiveKit, the voice agent,
|
||||
token service, music API and nginx. The app connects to
|
||||
`https://192.168.86.2:9178`. Docker uses host networking and an
|
||||
`unless-stopped` restart policy.
|
||||
|
||||
- Image: `hope-voice-api-voice:68f4c46-ios-audio`.
|
||||
- LLM: **qwen2.5-vl-3b**, through OpenWebUI at `http://192.168.86.2:3005/api`.
|
||||
- Speech: existing Azure configuration, unchanged.
|
||||
- Persistent certs, skills and music: `/mnt/user/dev/hope-voice-api` mounts.
|
||||
- Build source: `/mnt/user/dev/hope-voice-deploy-68f4c46`, a separate clean
|
||||
archive plus the iOS audio compatibility patch. The pre-existing dirty
|
||||
backend checkout was not overwritten.
|
||||
- Local compatibility source: sibling `hope-voice-api/web/media_server.py`;
|
||||
regression test: `hope-voice-api/tests/test_ios_audio.py`.
|
||||
|
||||
The music service retains original WebM files. An explicit `?format=m4a`
|
||||
request generates and caches AAC audio under `media/files/ios`, serving
|
||||
byte ranges for iOS playback and seeking. Browser requests remain unchanged.
|
||||
|
||||
## Health and rollback
|
||||
|
||||
```sh
|
||||
ssh [email protected] 'docker inspect --format "{{.State.Status}} {{.State.Health.Status}}" voice'
|
||||
```
|
||||
|
||||
The stopped container `voice-before-ios-audio` retains the baseline deployment.
|
||||
If rollback is needed, first inspect both containers, then stop the current
|
||||
`voice`, rename it to a unique recovery name, rename `voice-before-ios-audio`
|
||||
to `voice`, and start it. This restores the baseline without deleting images,
|
||||
containers or the music library; it does not include iOS AAC conversion.
|
||||
|
||||
`scripts/deploy-backend.sh` records the launch configuration and refuses to
|
||||
replace an existing `voice` container. Run it on the NAS only after building
|
||||
the named image and explicitly handling the existing container. It reads
|
||||
credentials on the NAS and never prints them. No source changes were pushed;
|
||||
future CI deployments must include the compatibility patch to preserve it.
|
||||
+7
-18
@@ -1,7 +1,7 @@
|
||||
# Device testing checklist
|
||||
|
||||
Simulator verification covered connect/disconnect, transcripts (text mode), media
|
||||
reconciliation/UI, tasks panel, and the hardening pass. The items below need a **real
|
||||
Simulator verification covered connect/disconnect, transcripts (text mode), tasks
|
||||
panel, and the hardening pass. The items below need a **real
|
||||
iPhone on the LAN** (`https://192.168.86.2:9178`) because mic input, WebRTC capture,
|
||||
and audio session behavior don't work in this simulator environment (CoreAudio `-4010`,
|
||||
0 input channels).
|
||||
@@ -12,7 +12,7 @@ ensure the server cert pin is current (see README — re-export after any server
|
||||
## 1. Mic input with real speech
|
||||
|
||||
- [ ] Start → agent joins → mic enables **without** `-4010`.
|
||||
- [ ] Speak: orb/status transitions to **listening**; mic meter moves with voice level.
|
||||
- [ ] Speak: Lightwell/status transitions to **listening**; mic meter moves with voice level.
|
||||
- [ ] Stop speaking, wait >2 s while agent is silent → **thinking** chip appears.
|
||||
- [ ] Agent replies: state goes to **speaking** (remote audio drives it), TTS audible.
|
||||
- [ ] User transcript finals arrive in the list with correct role/timestamps.
|
||||
@@ -31,29 +31,18 @@ ensure the server cert pin is current (see README — re-export after any server
|
||||
side button to lock the device. It only disables the automatic idle timeout.
|
||||
- [ ] After hang-up, auto-lock returns (idle timer re-enabled).
|
||||
|
||||
## 4. Audible media playback
|
||||
|
||||
- [ ] Requires a track in an iOS-decodable format (m4a/aac/mp3) — current library tracks
|
||||
are webm and will fail with `-11828` until the server serves a supported format.
|
||||
- [ ] Play a library track: audio is audible, now-playing card shows art/title/status.
|
||||
- [ ] Pause / ±10 s / stop / seek all work; agent mics off while playing
|
||||
(`media_state` published), mics back on when stopped.
|
||||
- [ ] End conversation while music was playing before → playback resumes.
|
||||
|
||||
## 5. Audio interruption (phone call / Siri)
|
||||
## 4. Audio interruption (phone call / Siri)
|
||||
|
||||
- [ ] Mid-conversation, trigger an interruption (incoming call or "Hey Siri").
|
||||
- [ ] Mic is muted on interruption begin; conversation/media pause cleanly.
|
||||
- [ ] On end (with `shouldResume`), mic and media resume automatically.
|
||||
- [ ] Media panel: AVPlayer pauses on interruption and resumes after.
|
||||
- [ ] On end (with `shouldResume`), mic and conversation resume automatically.
|
||||
|
||||
## 6. Mic permission denied fallback
|
||||
## 5. Mic permission denied fallback
|
||||
|
||||
- [ ] Deny microphone permission in iOS Settings for the app, relaunch.
|
||||
- [ ] Start → session reaches **ready** despite mic failure (non-fatal); orange error
|
||||
banner shows the mic error; text mode remains fully usable.
|
||||
|
||||
## 7. General device sanity
|
||||
## 6. General device sanity
|
||||
|
||||
- [ ] Connect / hang-up / reconnect matrix (3×) — no crashes, state resets cleanly.
|
||||
- [ ] Change server URL in Settings while idle → session recreates without crash.
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
# Project status and handoff
|
||||
|
||||
Last updated: 2026-09-12.
|
||||
|
||||
## Current direction
|
||||
|
||||
Keep **Qwen 2.5**. The user explicitly canceled the earlier request to switch to
|
||||
a ChatGPT subscription/device-token login. Do not revive that migration because
|
||||
an old saved goal mentions it. Current work is the native iOS UI/UX redesign
|
||||
and end-to-end qualification.
|
||||
|
||||
**2026-09-12: the embedded music player was removed** by user decision — the
|
||||
feature is gone from the app (Media tab, `Sources/Media/`, `MediaPanelView`,
|
||||
`MediaAPI`, `MediaPlayerController`, `PinnedMediaResourceLoader`, media tests,
|
||||
and the `media_state` control publish). The backend media API still exists and
|
||||
is untouched. The backend AAC compatibility patch (below) is now irrelevant to
|
||||
the iOS app but remains deployed.
|
||||
|
||||
The **Nearlight** native redesign is implemented across Talk, Activity, Settings,
|
||||
pickers, task details, and connection diagnostics. Current simulator qualification
|
||||
is green, and GPT-6 Astra's final blocker re-verification returned **PASS**. The
|
||||
final source also produced a signed generic-iOS build, and once the phone
|
||||
reconnected the final build was installed and launched on the paired iPhone.
|
||||
Hardware audio and interruption qualification remains open.
|
||||
|
||||
## Completed work
|
||||
|
||||
### Native interface and usability
|
||||
|
||||
- The app now has native **Talk** and **Activity** tabs sharing one session;
|
||||
changing tabs never ends a conversation.
|
||||
- Implemented Astra's **Nearlight** direction: an adaptive plum/rosewood semantic
|
||||
palette, restrained rounded typography, shared spacing/radius/motion tokens, and
|
||||
the compact Lightwell aperture as the conversation state signature.
|
||||
- Rebuilt Talk around a clear pre-connection welcome, Voice/Type choice, explicit
|
||||
start/end controls, compact active status, microphone meter, persistent Type
|
||||
draft, and a dock that keeps primary actions reachable.
|
||||
- Reworked the transcript as a readable shared record with speaker/time metadata,
|
||||
live partials, copy/selection, explicit destructive confirmation, and measured
|
||||
follow-latest behavior. Restored history stays at the top; manual reading,
|
||||
selection, and VoiceOver row focus suspend following until **Latest**.
|
||||
- Made connection and ready-session microphone failures visible without discarding
|
||||
the real error. Permission denial exposes an iPhone Settings recovery action;
|
||||
Type mode remains available.
|
||||
- Reworked Settings around voice, mode, model, and connection. Server edits are
|
||||
validated and explicitly saved; normalization confirmation survives session
|
||||
replacement and clears on the next user edit.
|
||||
- Improved Activity hierarchy, active-count badge, empty state, state-specific
|
||||
rows, and task detail presentation.
|
||||
- Removed forced dark appearance; all redesigned surfaces support light/dark mode,
|
||||
Dynamic Type, Reduce Motion, increased contrast branches, and 44-point-or-larger
|
||||
interaction targets where applicable.
|
||||
|
||||
### Functional fixes
|
||||
|
||||
- Observed nested transcript, conversation-state, and task objects so asynchronous
|
||||
updates render.
|
||||
- Preserved latest-wins server replacement while keeping an open Settings
|
||||
navigation stack and its success feedback intact.
|
||||
- Preserved Type-mode microphone privacy, pending-message send gating, clean session
|
||||
teardown, transcript persistence/deduplication, and interruption ordering.
|
||||
- All media-related client behavior was removed with the music player on 2026-09-12.
|
||||
|
||||
### Tests and tooling
|
||||
|
||||
- Installed XcodeBuildMCP and its skills; used it for simulator build/run/test work.
|
||||
- The current combined simulator run passed **50 tests, 0 failures**, with the
|
||||
opt-in live suite skipped once. Five offline UI scenarios cover recovery,
|
||||
settings validation/persistence and save feedback, transcript/task flows,
|
||||
restored-history reading position, and accessibility-size navigation.
|
||||
- A post-focus-change targeted restored-history UI regression passed independently.
|
||||
- DEBUG UI fixtures use isolated preferences; ordinary application preferences and
|
||||
the NAS are not modified by offline UI runs.
|
||||
- GPT-6 Astra completed the design specification and four verification rounds. Its
|
||||
final scoped review found B1–B3 resolved, N1/N2 preserved, and no concrete
|
||||
remaining release blocker. Accepted visual/adaptive/accessibility polish remains
|
||||
documented as debt, not as exhaustive WCAG or VoiceOver qualification.
|
||||
|
||||
## Backend deployment and cross-repository dependency
|
||||
|
||||
SSH host: `[email protected]` (Unraid). App URL: `https://192.168.86.2:9178`.
|
||||
|
||||
- `voice` image: `hope-voice-api-voice:68f4c46-ios-audio`, host networking,
|
||||
restart policy `unless-stopped`. Includes LiveKit, voice agent, token service,
|
||||
media API, and nginx.
|
||||
- LLM: `qwen2.5-vl-3b` through existing OpenWebUI at
|
||||
`http://192.168.86.2:3005/api`. Existing Azure speech configuration retained.
|
||||
- `voice`, `open-webui`, and `local-vl-models` were present and healthy when the
|
||||
user suspected deletion. Do not recreate healthy services unnecessarily.
|
||||
- Original library, certificates, and skills were retained under
|
||||
`/mnt/user/dev/hope-voice-api`; the dirty remote checkout was not overwritten.
|
||||
- Separate deployment source: `/mnt/user/dev/hope-voice-deploy-68f4c46`.
|
||||
- Stopped rollback container: `voice-before-ios-audio`.
|
||||
- Backend AAC conversion preserves original WebM files and serves cached AAC with
|
||||
byte-range support. Browser/raw requests retain their original behavior.
|
||||
|
||||
**Important:** the backend patch and its test are in a different local repository:
|
||||
`/Users/shane/dev/hope-voice-api/web/media_server.py` and
|
||||
`/Users/shane/dev/hope-voice-api/tests/test_ios_audio.py`. They were still uncommitted
|
||||
at this handoff. Committing/pushing `voice-ios` does **not** publish those files.
|
||||
The deployed image contains the patch, but a future backend CI rebuild can lose it
|
||||
unless it is committed and deployed through that repository's workflow. Inspect
|
||||
that workflow before pushing: it may replace the live container.
|
||||
|
||||
Launch configuration and rollback details: [DEPLOYMENT.md](DEPLOYMENT.md).
|
||||
Do not copy credentials or `.env` contents into documentation or commits.
|
||||
|
||||
## Verification performed
|
||||
|
||||
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator
|
||||
`AFDA137E-1657-428E-BF01-E5CD7989772C`.
|
||||
|
||||
- Current combined run: **50 passed, 0 failed, 1 live skip** in 104.9 seconds.
|
||||
Result bundle:
|
||||
`test_sim_2026-09-12T14-03-09-058Z_pid95846_2ca9da7e.xcresult`.
|
||||
- Post-focus-change restored-history regression: **1 passed, 0 failed**.
|
||||
Result bundle:
|
||||
`test_sim_2026-09-12T14-05-21-070Z_pid98187_bd3e531a.xcresult`.
|
||||
- A fresh isolated simulator launch showed restored history top-anchored with
|
||||
**Latest** visible; the UI regression then tapped Latest and verified the final
|
||||
event became visible.
|
||||
- Astra's targeted isolated runtime check verified a padded changed-address save
|
||||
stays in Settings, normalizes the field, shows its checkmark and confirmation,
|
||||
and clears both only after the next user edit. Final verdict: **PASS**.
|
||||
- iPhone 14 Pro Max `BF93D35E-0853-5470-A125-6EB0DB3F17AB` was paired, signed,
|
||||
installed, trusted, and launched successfully. The final Nearlight build
|
||||
(`/tmp/voice-ios-device-derived/Build/Products/Debug-iphoneos/Voice.app`,
|
||||
`dev.shane.voice`) was reinstalled and launched via `devicectl` after the phone
|
||||
reconnected. Free provisioning expires after seven days; reinstall to re-sign.
|
||||
- Historical live NAS runs verified Qwen replies, background task completion, and
|
||||
reconnect. Backend integration remains 16 passed, 7 skipped, 2 deselected.
|
||||
|
||||
Exact historical qualifications: [VERIFICATION.md](VERIFICATION.md). Artifacts
|
||||
live on this Mac under the XcodeBuildMCP workspace, not in Git.
|
||||
|
||||
## What remains, in order
|
||||
|
||||
1. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
|
||||
barge-in, Siri/phone interruptions, lock/background behavior,
|
||||
microphone-permission-denied fallback, and network-loss recovery. These require
|
||||
the user's speech/listening or a real interruption; simulator state tests do not
|
||||
prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
|
||||
2. **Broader live regression:** exercise slow-network cancellation, network-loss
|
||||
recovery, repeated server replacement, and voice/model selection against the
|
||||
live agent.
|
||||
3. **Manual accessibility/visual qualification:** inspect populated/error screens
|
||||
in light and dark mode, accessibility text sizes, narrow/landscape layouts, and
|
||||
a real VoiceOver focus session. Astra accepted the remaining presentation debt
|
||||
for this personal-app release; that is not proof of exhaustive conformance.
|
||||
4. **Publish the backend compatibility source safely** in the sibling repository
|
||||
and ensure its normal deployment preserves mounts, certificates, and Qwen
|
||||
settings. This is lower priority because the iOS app no longer consumes the
|
||||
media API, but the deployed container still serves it for the web UI.
|
||||
|
||||
## Repository handoff
|
||||
|
||||
The iOS work and this report belong to branch `feat/ios-app`, remote `origin`
|
||||
(`https://git.srmr.xyz/snowspeeder/hope-voice-ios.git`). Keep the Nearlight
|
||||
implementation, tests, physical-device notes, and this handoff together when
|
||||
committing. Check Git history and remote status for the final publish result.
|
||||
@@ -0,0 +1,85 @@
|
||||
# Verification — 2026-09-12
|
||||
|
||||
## Scope
|
||||
|
||||
Qwen 2.5 remains configured; the canceled subscription-login migration was not
|
||||
revived. The embedded music player was removed by user decision. The current app
|
||||
is the Astra-designed **Nearlight** native interface with shared Talk and Activity
|
||||
tabs, adaptive semantic styling, the Lightwell conversation indicator, rebuilt
|
||||
conversation controls, readable transcripts, actionable failure states, explicit
|
||||
Settings saves, and revised Activity/detail surfaces.
|
||||
|
||||
## Current iOS evidence
|
||||
|
||||
Project: `Voice.xcodeproj`; scheme: `Voice_iOS`; simulator: iPhone 17 Pro
|
||||
`AFDA137E-1657-428E-BF01-E5CD7989772C`.
|
||||
|
||||
- Full combined simulator run: **50 passed, 0 failed, 1 live skip** in 104.9
|
||||
seconds. It includes unit/state tests and five offline UI scenarios.
|
||||
- The offline UI scenarios cover navigation/offline recovery; persisted voice and
|
||||
server validation; normalized same-address and changed-address save feedback;
|
||||
transcript clearing and Activity details; restored-history reading position and
|
||||
explicit Latest recovery; and accessibility-size navigation.
|
||||
- A second targeted run after the final VoiceOver eligibility strengthening passed
|
||||
`testRestoredTranscriptStaysAtTopUntilLatest`: **1 passed, 0 failed**.
|
||||
- Fresh runtime inspection showed the long restored transcript at its beginning
|
||||
with the **Latest** affordance visible. The regression tapped Latest and verified
|
||||
the final row became visible.
|
||||
- GPT-6 Astra re-read its design specification and prior findings, inspected the
|
||||
final source, and performed an isolated Settings runtime reproduction. Its
|
||||
round-four verdict was **PASS**: reading eligibility, ready-session microphone
|
||||
recovery visibility, and save-confirmation lifetime were resolved; the earlier
|
||||
selection and pending-Send fixes remained intact.
|
||||
- Astra's Settings reproduction saved ` https://127.0.0.1:2 ` using isolated
|
||||
UI-test preferences. Settings stayed open, the field normalized, and the
|
||||
checkmark plus “Server address saved.” remained visible. The next user edit
|
||||
cleared both indicators. It did not touch normal preferences or the NAS.
|
||||
|
||||
Result bundles under
|
||||
`~/Library/Developer/XcodeBuildMCP/workspaces/voice-ios-9eb036fdea81/result-bundles/`:
|
||||
|
||||
- Full run:
|
||||
`test_sim_2026-09-12T14-03-09-058Z_pid95846_2ca9da7e.xcresult`
|
||||
- Targeted restored-history run:
|
||||
`test_sim_2026-09-12T14-05-21-070Z_pid98187_bd3e531a.xcresult`
|
||||
|
||||
## Physical-device evidence
|
||||
|
||||
iPhone 14 Pro Max `BF93D35E-0853-5470-A125-6EB0DB3F17AB` was paired, built for,
|
||||
signed with personal team `YJFYRB92NW`, installed, trusted, and launched through
|
||||
`devicectl`. This proves device signing/install/launch only, not microphone or
|
||||
speaker behavior. Free provisioning expires after seven days.
|
||||
|
||||
The final source also produced a signed device build and, once the phone
|
||||
reconnected, `devicectl` installed and launched it successfully on 2026-09-12.
|
||||
This proves device signing/install/launch of the final binary, not microphone or
|
||||
speaker behavior.
|
||||
|
||||
## Historical live and backend evidence
|
||||
|
||||
Earlier live NAS runs verified Qwen replies, background task completion,
|
||||
disconnect/reconnect, and three consecutive Qwen/task/reconnect iterations. Older
|
||||
music playback evidence is historical only: the player and its tests no longer
|
||||
exist in the iOS app.
|
||||
|
||||
Backend integration previously passed 16 tests with 7 skipped and 2 deselected.
|
||||
The AAC compatibility regression passed and the deployed byte-range request
|
||||
returned HTTP 206. That patch remains in a separate backend repository and is not
|
||||
published by this iOS branch.
|
||||
|
||||
## Not yet qualified
|
||||
|
||||
- Audible speaker/headphone output, physical microphone capture, barge-in,
|
||||
phone/Siri interruption recovery, lock/background behavior, and live
|
||||
network-loss recovery.
|
||||
- A real microphone-permission-denied session. The visibility/recovery branches
|
||||
are source-reviewed, but no permission was changed during final verification.
|
||||
- A full VoiceOver audio/focus session, exhaustive contrast audit, every Dynamic
|
||||
Type/layout combination, or strict conformance to every Nearlight design detail.
|
||||
Astra explicitly accepted the remaining visual/adaptive/presentation debt for
|
||||
this personal release.
|
||||
- The final corrected binary on the physical iPhone while it remains unavailable.
|
||||
|
||||
Follow [DEVICE_TESTING.md](DEVICE_TESTING.md) for hardware qualification and
|
||||
[PROJECT_STATUS.md](PROJECT_STATUS.md) for current priorities and deployment
|
||||
dependencies.
|
||||
+14
@@ -41,6 +41,7 @@ targets:
|
||||
TARGETED_DEVICE_FAMILY: "1"
|
||||
CURRENT_PROJECT_VERSION: 1
|
||||
MARKETING_VERSION: "0.1.0"
|
||||
DEVELOPMENT_TEAM: YJFYRB92NW
|
||||
CODE_SIGN_STYLE: Automatic
|
||||
VoiceTests:
|
||||
type: bundle.unit-test
|
||||
@@ -54,3 +55,16 @@ targets:
|
||||
SWIFT_VERSION: "5.9"
|
||||
GENERATE_INFOPLIST_FILE: YES
|
||||
PRODUCT_BUNDLE_IDENTIFIER: dev.shane.voice.tests
|
||||
VoiceUITests:
|
||||
type: bundle.ui-testing
|
||||
platform: iOS
|
||||
sources:
|
||||
- UITests
|
||||
dependencies:
|
||||
- target: Voice_iOS
|
||||
settings:
|
||||
base:
|
||||
SWIFT_VERSION: "5.9"
|
||||
GENERATE_INFOPLIST_FILE: YES
|
||||
PRODUCT_BUNDLE_IDENTIFIER: dev.shane.voice.uitests
|
||||
TEST_TARGET_NAME: Voice_iOS
|
||||
@@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run on xNAS after building the clean backend source into the pinned image.
|
||||
# Credentials stay on the NAS. Existing checkout, library and certs are retained.
|
||||
set -euo pipefail
|
||||
deployment_source=/mnt/user/dev/hope-voice-deploy-68f4c46
|
||||
deployment_data=/mnt/user/dev/hope-voice-api
|
||||
deployment_image=${HOPE_DEPLOY_IMAGE:-hope-voice-api-voice:68f4c46-ios-audio}
|
||||
test -f "$deployment_data/.env"
|
||||
docker image inspect "$deployment_image" >/dev/null
|
||||
if docker container inspect voice >/dev/null 2>&1; then
|
||||
echo 'A voice container already exists; inspect it before replacing it.' >&2
|
||||
exit 1
|
||||
fi
|
||||
# Recover the OpenWebUI credential used by this exact backend deployment source.
|
||||
LLM_API_KEY=$(sed -n 's/.*-e LLM_API_KEY=\([^ ]*\).*/\1/p' "$deployment_source/.gitea/workflows/rebuild.yml")
|
||||
test -n "$LLM_API_KEY"
|
||||
export LLM_API_KEY
|
||||
export LLM_BASE_URL=http://192.168.86.2:3005/api
|
||||
export LLM_MODEL=qwen2.5-vl-3b
|
||||
docker run -d --name voice --network host --restart unless-stopped \
|
||||
--env-file "$deployment_data/.env" \
|
||||
--env LLM_BASE_URL --env LLM_MODEL --env LLM_API_KEY \
|
||||
--env LIVEKIT_URL=http://localhost:7880 \
|
||||
--env VOICE_USERS_FILE=/etc/hope/users.json \
|
||||
--env VOICE_DEFAULT_USER=shane --env MEDIA_DIR=/data/media \
|
||||
--env MEDIA_SERVER_PORT=8093 --env WEB_PORT=9178 \
|
||||
--env YTDLP_COOKIES=/data/media/cookies.txt \
|
||||
--volume "$deployment_data/certs:/etc/voice/certs" \
|
||||
--volume "$deployment_data/skills:/skills" \
|
||||
--volume "$deployment_data/media:/data/media" \
|
||||
--volume "$deployment_source/livekit.yaml:/etc/livekit.yaml:ro" \
|
||||
--health-cmd='curl -sfk https://localhost:9178/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null && curl -sf http://localhost:8093/media/api/healthz -o /dev/null' \
|
||||
--health-interval=15s --health-timeout=5s --health-retries=3 --health-start-period=30s \
|
||||
"$deployment_image"
|
||||
Reference in new issue
Block a user