Archived
Remove embedded music player
The music feature is retired: delete the Media tab, MediaAPI, MediaPlayerController, PinnedMediaResourceLoader, MediaPanelView, the media_state control publish, media scope handling in VoiceSession, the volume setting, and all media tests. App is now Talk + Activity tabs. Backend media API in hope-voice-api is untouched (still serves the web UI).
This commit is contained in:
1 parent
3620a2387b
commit
ed329213de
18 files changed
+55
-2393
No files matched your search
@@ -1,19 +1,19 @@
|
|||||||
# Voice — iOS client for Hope Voice
|
# Voice — iOS client for Hope Voice
|
||||||
|
|
||||||
Native SwiftUI iOS app (iOS 17+) that talks to the [Hope Voice](https://github.com/) agent:
|
Native SwiftUI iOS app (iOS 17+) that talks to the [Hope Voice](https://github.com/) agent:
|
||||||
voice/text conversation, transcript history, voice & model pickers, media player, and a
|
voice/text conversation, transcript history, voice & model pickers, and a background-tasks
|
||||||
background-tasks panel. Uses `hope-voice-api`, including cached AAC conversion for iOS music playback.
|
panel. Uses `hope-voice-api`.
|
||||||
|
|
||||||
> **Status:** Core functionality is implemented and verified in simulator to the extent the
|
> **Status:** Core functionality is implemented and verified in simulator to the extent the
|
||||||
> environment allows. Full device qualification (mic capture, audible media playback,
|
> environment allows. Full device qualification (mic capture, interruption behavior)
|
||||||
> interruption behavior) remains pending. The native Talk/Music/Activity redesign is implemented —
|
> remains pending. The native Talk/Activity redesign is implemented —
|
||||||
> see [PLAN.md](PLAN.md) and [docs/DEVICE_TESTING.md](docs/DEVICE_TESTING.md).
|
> see [PLAN.md](PLAN.md) and [docs/DEVICE_TESTING.md](docs/DEVICE_TESTING.md).
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- Xcode 16+ (Swift 6), iOS 17 simulator or a real iPhone on the LAN.
|
- Xcode 16+ (Swift 6), iOS 17 simulator or a real iPhone on the LAN.
|
||||||
- [xcodegen](https://github.com/yonaskolb/XcodeGen) (`brew install xcodegen`).
|
- [xcodegen](https://github.com/yonaskolb/XcodeGen) (`brew install xcodegen`).
|
||||||
- The `voice` container running on Unraid (LiveKit + token server + media API).
|
- The `voice` container running on Unraid (LiveKit + token server).
|
||||||
- Mic: the simulator mic is unreliable in this environment (CoreAudio `-4010`, 0 input
|
- Mic: the simulator mic is unreliable in this environment (CoreAudio `-4010`, 0 input
|
||||||
channels) — voice mode needs a real device. If the simulator was wiped, grant permission
|
channels) — voice mode needs a real device. If the simulator was wiped, grant permission
|
||||||
first: `xcrun simctl privacy booted grant microphone dev.shane.voice`.
|
first: `xcrun simctl privacy booted grant microphone dev.shane.voice`.
|
||||||
@@ -33,7 +33,7 @@ To run in the simulator, use Xcode or `xcrun simctl` with the built app.
|
|||||||
|
|
||||||
Default base URL is `https://192.168.86.2:9178` (see `Sources/App/SettingsStore.swift`).
|
Default base URL is `https://192.168.86.2:9178` (see `Sources/App/SettingsStore.swift`).
|
||||||
Change it at runtime in **Settings → Server URL** (the app disconnects and recreates the
|
Change it at runtime in **Settings → Server URL** (the app disconnects and recreates the
|
||||||
session when idle). The app derives LiveKit signaling and media endpoints from this base URL.
|
session when idle). The app derives the LiveKit signaling endpoint from this base URL.
|
||||||
|
|
||||||
### TLS cert pinning
|
### TLS cert pinning
|
||||||
|
|
||||||
@@ -46,7 +46,7 @@ openssl s_client -connect 192.168.86.2:9178 </dev/null 2>/dev/null \
|
|||||||
| openssl x509 -outform der | openssl dgst -sha256
|
| openssl x509 -outform der | openssl dgst -sha256
|
||||||
```
|
```
|
||||||
|
|
||||||
Paste the new hash into `TLSPinning.pinnedCertSHA256Hex`. Until you do, token/media calls
|
Paste the new hash into `TLSPinning.pinnedCertSHA256Hex`. Until you do, token calls
|
||||||
fail with a pin mismatch (not a network error).
|
fail with a pin mismatch (not a network error).
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
@@ -58,10 +58,6 @@ fail with a pin mismatch (not a network error).
|
|||||||
stripping, last-50 history persisted to UserDefaults, clear button.
|
stripping, last-50 history persisted to UserDefaults, clear button.
|
||||||
- **Voice & model pickers** — 13 Azure TTS voices; LLM models from the agent's `models`
|
- **Voice & model pickers** — 13 Azure TTS voices; LLM models from the agent's `models`
|
||||||
message. Both persist and sync on join (`set_voice`/`set_model`).
|
message. Both persist and sync on join (`set_voice`/`set_model`).
|
||||||
- **Media player** — now-playing card (art/title/channel/status), transport (play/pause,
|
|
||||||
±10 s, stop), seek + volume sliders, search with "in library" tags, agent-offered choices,
|
|
||||||
library list (play/remove), add-stream form. AVPlayer-backed with a pinned-TLS Range proxy;
|
|
||||||
1.5 s `/state` polling and full web-parity reconciliation/reporting.
|
|
||||||
- **Activity tab** — live `task_event` updates (steps, result, error) with an active
|
- **Activity tab** — live `task_event` updates (steps, result, error) with an active
|
||||||
count badge on the tab.
|
count badge on the tab.
|
||||||
|
|
||||||
@@ -69,11 +65,9 @@ fail with a pin mismatch (not a network error).
|
|||||||
|
|
||||||
- **Simulator mic**: CoreAudio returns `-4010` (no usable input source) in this environment;
|
- **Simulator mic**: CoreAudio returns `-4010` (no usable input source) in this environment;
|
||||||
voice capture must be verified on a real device. Text mode works in the simulator.
|
voice capture must be verified on a real device. Text mode works in the simulator.
|
||||||
- **Music compatibility**: library WebM files are requested as cached AAC on iOS.
|
|
||||||
This requires the deployed backend compatibility patch; see [deployment](docs/DEPLOYMENT.md).
|
|
||||||
- **Cert pin rotation**: see above — re-export the pin after each server cert regeneration.
|
- **Cert pin rotation**: see above — re-export the pin after each server cert regeneration.
|
||||||
- **LiveKit signaling** uses plain `ws://192.168.86.2:7880` (not wss) because the Swift SDK
|
- **LiveKit signaling** uses plain `ws://192.168.86.2:7880` (not wss) because the Swift SDK
|
||||||
cannot trust the self-signed cert on its WebSocket; token + media still go over pinned TLS.
|
cannot trust the self-signed cert on its WebSocket; token calls still go over pinned TLS.
|
||||||
|
|
||||||
## Docs
|
## Docs
|
||||||
|
|
||||||
|
|||||||
@@ -14,16 +14,12 @@ final class SettingsStore: ObservableObject {
|
|||||||
@Published var textMode: Bool {
|
@Published var textMode: Bool {
|
||||||
didSet { AppPreferences.store.set(textMode, forKey: Keys.textMode) }
|
didSet { AppPreferences.store.set(textMode, forKey: Keys.textMode) }
|
||||||
}
|
}
|
||||||
@Published var volume: Double {
|
|
||||||
didSet { AppPreferences.store.set(volume, forKey: Keys.volume) }
|
|
||||||
}
|
|
||||||
|
|
||||||
private enum Keys {
|
private enum Keys {
|
||||||
static let serverURL = "serverURL"
|
static let serverURL = "serverURL"
|
||||||
static let voiceID = "voiceID"
|
static let voiceID = "voiceID"
|
||||||
static let modelID = "modelID"
|
static let modelID = "modelID"
|
||||||
static let textMode = "textMode"
|
static let textMode = "textMode"
|
||||||
static let volume = "volume"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
static let defaultServerURL = "https://192.168.86.2:9178"
|
static let defaultServerURL = "https://192.168.86.2:9178"
|
||||||
@@ -47,7 +43,6 @@ final class SettingsStore: ObservableObject {
|
|||||||
voiceID = Voices.all.contains { $0.id == storedVoice } ? (storedVoice ?? "") : Voices.defaultID
|
voiceID = Voices.all.contains { $0.id == storedVoice } ? (storedVoice ?? "") : Voices.defaultID
|
||||||
modelID = d.string(forKey: Keys.modelID)
|
modelID = d.string(forKey: Keys.modelID)
|
||||||
textMode = d.bool(forKey: Keys.textMode)
|
textMode = d.bool(forKey: Keys.textMode)
|
||||||
volume = d.object(forKey: Keys.volume) as? Double ?? 0.8
|
|
||||||
}
|
}
|
||||||
|
|
||||||
var trimmedServerURL: String {
|
var trimmedServerURL: String {
|
||||||
|
|||||||
@@ -40,15 +40,8 @@ struct ConversationView: View {
|
|||||||
.tabItem { Label("Talk", systemImage: "waveform") }
|
.tabItem { Label("Talk", systemImage: "waveform") }
|
||||||
.tag(0)
|
.tag(0)
|
||||||
|
|
||||||
NavigationStack {
|
|
||||||
MediaPanelView(playerController: session.mediaController, settings: settings)
|
|
||||||
.toolbar { settingsLink }
|
|
||||||
}
|
|
||||||
.tabItem { Label("Music", systemImage: "music.note") }
|
|
||||||
.tag(1)
|
|
||||||
|
|
||||||
ActivityTab(taskStore: session.taskStore, settings: settings, session: session)
|
ActivityTab(taskStore: session.taskStore, settings: settings, session: session)
|
||||||
.tag(2)
|
.tag(1)
|
||||||
}
|
}
|
||||||
.id(sessionID)
|
.id(sessionID)
|
||||||
.tint(HopeStyle.tint)
|
.tint(HopeStyle.tint)
|
||||||
@@ -79,9 +72,7 @@ struct ConversationView: View {
|
|||||||
// Coalesce rapid Apply actions; construct from the captured URL.
|
// Coalesce rapid Apply actions; construct from the captured URL.
|
||||||
while let next = pendingServerURL {
|
while let next = pendingServerURL {
|
||||||
pendingServerURL = nil
|
pendingServerURL = nil
|
||||||
guard next != activeServerURL else { continue }
|
|
||||||
await session.disconnect()
|
await session.disconnect()
|
||||||
session.mediaController.resetScope()
|
|
||||||
activeServerURL = next
|
activeServerURL = next
|
||||||
session = Self.makeSession(next, settings: settings)
|
session = Self.makeSession(next, settings: settings)
|
||||||
sessionID = UUID()
|
sessionID = UUID()
|
||||||
@@ -157,13 +148,12 @@ private struct ConversationContent: View {
|
|||||||
Text("A little help.\nA good conversation.")
|
Text("A little help.\nA good conversation.")
|
||||||
.font(.largeTitle.weight(.semibold))
|
.font(.largeTitle.weight(.semibold))
|
||||||
.fixedSize(horizontal: false, vertical: true)
|
.fixedSize(horizontal: false, vertical: true)
|
||||||
Text("Talk things through, find something out, or put on a favorite song.")
|
Text("Talk things through, find something out, or get something done.")
|
||||||
.font(.body).foregroundStyle(.secondary)
|
.font(.body).foregroundStyle(.secondary)
|
||||||
}
|
}
|
||||||
VStack(alignment: .leading, spacing: 16) {
|
VStack(alignment: .leading, spacing: 16) {
|
||||||
Label("Think out loud", systemImage: "bubble.left.and.bubble.right")
|
Label("Think out loud", systemImage: "bubble.left.and.bubble.right")
|
||||||
Label("Find an answer", systemImage: "sparkle.magnifyingglass")
|
Label("Find an answer", systemImage: "sparkle.magnifyingglass")
|
||||||
Label("Make time for music", systemImage: "music.note")
|
|
||||||
}
|
}
|
||||||
.font(.subheadline).foregroundStyle(.secondary)
|
.font(.subheadline).foregroundStyle(.secondary)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,610 +0,0 @@
|
|||||||
import SwiftUI
|
|
||||||
|
|
||||||
struct MediaPanelView: View {
|
|
||||||
@ObservedObject var playerController: MediaPlayerController
|
|
||||||
@ObservedObject var settings: SettingsStore
|
|
||||||
|
|
||||||
@State private var searchQuery = ""
|
|
||||||
@State private var searchResults: [MediaSearchResult]?
|
|
||||||
@State private var isSearching = false
|
|
||||||
@State private var searchError: String?
|
|
||||||
@State private var libraryTracks: [MediaTrack]?
|
|
||||||
@State private var libraryError: String?
|
|
||||||
@State private var streamName = ""
|
|
||||||
@State private var streamURL = ""
|
|
||||||
@State private var streamMessage: String?
|
|
||||||
@State private var trackToDelete: MediaTrack?
|
|
||||||
@State private var browseMode = 0
|
|
||||||
@State private var showAddStream = false
|
|
||||||
@State private var isAddingStream = false
|
|
||||||
@State private var isLoadingLibrary = false
|
|
||||||
@State private var seekPosition: Double = 0
|
|
||||||
@State private var isSeeking = false
|
|
||||||
|
|
||||||
var body: some View {
|
|
||||||
ScrollView {
|
|
||||||
LazyVStack(alignment: .leading, spacing: 24) {
|
|
||||||
if playerController.stateError != nil {
|
|
||||||
VStack(alignment: .leading, spacing: 8) {
|
|
||||||
Label("Music server unavailable", systemImage: "wifi.exclamationmark")
|
|
||||||
.font(.headline)
|
|
||||||
Text("Check that your Hope server is running and you’re connected to its network. We’ll reconnect automatically.")
|
|
||||||
.font(.subheadline).foregroundStyle(.secondary)
|
|
||||||
}
|
|
||||||
.frame(maxWidth: .infinity, alignment: .leading)
|
|
||||||
}
|
|
||||||
if playerController.track != nil {
|
|
||||||
VStack(spacing: 20) {
|
|
||||||
nowPlayingCard
|
|
||||||
seekSection
|
|
||||||
transportControls
|
|
||||||
volumeSection
|
|
||||||
}
|
|
||||||
.padding(16)
|
|
||||||
.background(Color(.secondarySystemGroupedBackground), in: RoundedRectangle(cornerRadius: 16))
|
|
||||||
}
|
|
||||||
if let error = playerController.playbackError {
|
|
||||||
Label(error, systemImage: "exclamationmark.circle")
|
|
||||||
.font(.subheadline).foregroundStyle(.red)
|
|
||||||
}
|
|
||||||
pendingSection
|
|
||||||
choicesSection
|
|
||||||
Picker("Browse music", selection: $browseMode) {
|
|
||||||
Text("Library").tag(0)
|
|
||||||
Text("Search").tag(1)
|
|
||||||
}.pickerStyle(.segmented)
|
|
||||||
if browseMode == 0 { librarySection }
|
|
||||||
else { searchSection }
|
|
||||||
}
|
|
||||||
.padding()
|
|
||||||
}
|
|
||||||
.background(Color(.systemGroupedBackground))
|
|
||||||
.navigationTitle("Music")
|
|
||||||
.toolbar {
|
|
||||||
ToolbarItem(placement: .topBarTrailing) {
|
|
||||||
Button {
|
|
||||||
streamMessage = nil
|
|
||||||
showAddStream = true
|
|
||||||
} label: {
|
|
||||||
Image(systemName: "plus").frame(minWidth: 44, minHeight: 44)
|
|
||||||
}
|
|
||||||
.accessibilityLabel("Add radio station")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
.sheet(isPresented: $showAddStream) {
|
|
||||||
NavigationStack {
|
|
||||||
Form { addStreamSection }
|
|
||||||
.navigationTitle("Add a station")
|
|
||||||
.navigationBarTitleDisplayMode(.inline)
|
|
||||||
.toolbar {
|
|
||||||
ToolbarItem(placement: .cancellationAction) {
|
|
||||||
Button("Cancel") { showAddStream = false }.disabled(isAddingStream)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
.interactiveDismissDisabled(isAddingStream)
|
|
||||||
}
|
|
||||||
.onAppear {
|
|
||||||
playerController.startDefaultPolling()
|
|
||||||
refreshLibrary()
|
|
||||||
}
|
|
||||||
.alert("Remove from library?", isPresented: Binding(
|
|
||||||
get: { trackToDelete != nil },
|
|
||||||
set: { if !$0 { trackToDelete = nil } }
|
|
||||||
)) {
|
|
||||||
Button("Remove", role: .destructive) {
|
|
||||||
guard let track = trackToDelete else { return }
|
|
||||||
Task { await deleteTrack(track) }
|
|
||||||
}
|
|
||||||
Button("Cancel", role: .cancel) { trackToDelete = nil }
|
|
||||||
} message: {
|
|
||||||
Text(trackToDelete?.title ?? "")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var nowPlayingCard: some View {
|
|
||||||
HStack(spacing: 12) {
|
|
||||||
if let artURL = playerController.track?.artURL, let url = URL(string: absolute(artURL)) {
|
|
||||||
PinnedArtwork(url: url)
|
|
||||||
.frame(width: 72, height: 72)
|
|
||||||
.clipShape(RoundedRectangle(cornerRadius: 10))
|
|
||||||
} else {
|
|
||||||
RoundedRectangle(cornerRadius: 10)
|
|
||||||
.fill(Color(.systemGray6))
|
|
||||||
.frame(width: 72, height: 72)
|
|
||||||
.overlay(
|
|
||||||
Image(systemName: "music.note")
|
|
||||||
.font(.title2)
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
VStack(alignment: .leading, spacing: 4) {
|
|
||||||
Text(title)
|
|
||||||
.font(.headline)
|
|
||||||
.lineLimit(2)
|
|
||||||
Text(subtitle)
|
|
||||||
.font(.subheadline)
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
.lineLimit(2)
|
|
||||||
if playerController.track != nil {
|
|
||||||
Text(statusText)
|
|
||||||
.font(.caption)
|
|
||||||
.foregroundStyle(playerController.desired == "playing" ? Color.green : Color.secondary)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Spacer()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var transportControls: some View {
|
|
||||||
HStack(spacing: 24) {
|
|
||||||
Button(action: seekBackward) {
|
|
||||||
Image(systemName: "gobackward.10")
|
|
||||||
.font(.title3)
|
|
||||||
.frame(width: 44, height: 44)
|
|
||||||
}
|
|
||||||
.accessibilityLabel("Seek back 10 seconds")
|
|
||||||
.disabled(playerController.duration <= 0 || playerController.track?.kind == "stream")
|
|
||||||
|
|
||||||
Button(action: togglePlayPause) {
|
|
||||||
Image(systemName: playerController.desired == "playing" && playerController.track != nil ? "pause.circle.fill" : "play.circle.fill")
|
|
||||||
.font(.largeTitle)
|
|
||||||
.frame(width: 56, height: 56)
|
|
||||||
.foregroundStyle(playerController.track == nil ? Color.secondary : Color.accentColor)
|
|
||||||
}
|
|
||||||
.accessibilityLabel(playerController.desired == "playing" && playerController.track != nil ? "Pause" : "Play")
|
|
||||||
.disabled(playerController.track == nil)
|
|
||||||
|
|
||||||
Button(action: seekForward) {
|
|
||||||
Image(systemName: "goforward.10")
|
|
||||||
.font(.title3)
|
|
||||||
.frame(width: 44, height: 44)
|
|
||||||
}
|
|
||||||
.accessibilityLabel("Seek forward 10 seconds")
|
|
||||||
.disabled(playerController.duration <= 0 || playerController.track?.kind == "stream")
|
|
||||||
|
|
||||||
Button(role: .destructive, action: stopPlayback) {
|
|
||||||
Image(systemName: "stop.circle")
|
|
||||||
.font(.title2)
|
|
||||||
.frame(width: 44, height: 44)
|
|
||||||
}
|
|
||||||
.accessibilityLabel("Stop playback")
|
|
||||||
.disabled(playerController.track == nil)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var seekSection: some View {
|
|
||||||
VStack(spacing: 4) {
|
|
||||||
if playerController.track?.kind == "stream" {
|
|
||||||
Label("Live radio", systemImage: "dot.radiowaves.left.and.right")
|
|
||||||
.font(.subheadline).foregroundStyle(.secondary)
|
|
||||||
} else if playerController.track != nil {
|
|
||||||
Slider(
|
|
||||||
value: Binding(
|
|
||||||
get: { min(isSeeking ? seekPosition : playerController.currentTime, max(playerController.duration, 1)) },
|
|
||||||
set: { seekPosition = $0 }
|
|
||||||
),
|
|
||||||
in: 0...max(playerController.duration, 1),
|
|
||||||
onEditingChanged: { editing in
|
|
||||||
isSeeking = editing
|
|
||||||
if !editing { playerController.seek(to: seekPosition) }
|
|
||||||
}
|
|
||||||
)
|
|
||||||
.disabled(playerController.duration <= 0)
|
|
||||||
.accessibilityLabel("Playback position")
|
|
||||||
HStack {
|
|
||||||
Text(fmtTime(playerController.currentTime))
|
|
||||||
.font(.caption.monospacedDigit())
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
Spacer()
|
|
||||||
Text(fmtTime(playerController.duration))
|
|
||||||
.font(.caption.monospacedDigit())
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
Text("Nothing loaded")
|
|
||||||
.font(.subheadline)
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
.frame(maxWidth: .infinity, alignment: .center)
|
|
||||||
.padding(.vertical, 8)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var volumeSection: some View {
|
|
||||||
HStack(spacing: 12) {
|
|
||||||
Image(systemName: "speaker.fill")
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
Slider(value: $settings.volume, in: 0...1)
|
|
||||||
.accessibilityLabel("Music volume")
|
|
||||||
.onChange(of: settings.volume) { _, newValue in
|
|
||||||
playerController.setVolume(newValue)
|
|
||||||
}
|
|
||||||
Image(systemName: "speaker.wave.3.fill")
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@ViewBuilder
|
|
||||||
private var pendingSection: some View {
|
|
||||||
if !playerController.pendingJobs.isEmpty {
|
|
||||||
VStack(alignment: .leading, spacing: 8) {
|
|
||||||
ForEach(playerController.pendingJobs, id: \.jobID) { job in
|
|
||||||
HStack {
|
|
||||||
Text(job.stage == "queued" ? "Queued…" : "Downloading…")
|
|
||||||
.font(.subheadline)
|
|
||||||
Spacer()
|
|
||||||
Text("\(Int((job.progress ?? 0)))%")
|
|
||||||
.font(.caption.monospacedDigit())
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
.padding(12)
|
|
||||||
.background(Color(.secondarySystemBackground), in: RoundedRectangle(cornerRadius: 14))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@ViewBuilder
|
|
||||||
private var choicesSection: some View {
|
|
||||||
if !playerController.choices.isEmpty {
|
|
||||||
VStack(alignment: .leading, spacing: 8) {
|
|
||||||
Text("Hope's suggestions")
|
|
||||||
.font(.headline)
|
|
||||||
ForEach(playerController.choices, id: \.videoID) { choice in
|
|
||||||
mediaRow(
|
|
||||||
title: choice.title ?? choice.videoID ?? "Unknown",
|
|
||||||
detail: rowDetail(channel: choice.channel, duration: choice.duration),
|
|
||||||
playEnabled: true
|
|
||||||
) {
|
|
||||||
Task {
|
|
||||||
await playerController.playVideo(
|
|
||||||
videoID: choice.videoID,
|
|
||||||
title: choice.title,
|
|
||||||
channel: choice.channel
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var searchSection: some View {
|
|
||||||
VStack(alignment: .leading, spacing: 8) {
|
|
||||||
Text("Find your next favorite")
|
|
||||||
.font(.headline)
|
|
||||||
Text("Search songs and artists on YouTube Music.")
|
|
||||||
.font(.subheadline).foregroundStyle(.secondary)
|
|
||||||
HStack(spacing: 8) {
|
|
||||||
TextField("Song, artist, lyrics…", text: $searchQuery)
|
|
||||||
.textFieldStyle(.roundedBorder)
|
|
||||||
.submitLabel(.search)
|
|
||||||
.onSubmit(performSearch)
|
|
||||||
Button("Search", action: performSearch)
|
|
||||||
.accessibilityIdentifier("music.search")
|
|
||||||
.buttonStyle(.bordered)
|
|
||||||
.frame(minHeight: 44)
|
|
||||||
.disabled(searchQuery.trimmingCharacters(in: .whitespaces).isEmpty || isSearching)
|
|
||||||
}
|
|
||||||
if isSearching {
|
|
||||||
HStack(spacing: 8) {
|
|
||||||
ProgressView()
|
|
||||||
Text("Searching YouTube Music…")
|
|
||||||
.font(.subheadline)
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
}
|
|
||||||
} else if let searchError {
|
|
||||||
Text(searchError)
|
|
||||||
.font(.subheadline)
|
|
||||||
.foregroundStyle(.red)
|
|
||||||
} else if let results = searchResults {
|
|
||||||
if results.isEmpty {
|
|
||||||
Text("No matches. Try a different song or artist.")
|
|
||||||
.font(.subheadline)
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
} else {
|
|
||||||
ForEach(results, id: \.videoID) { result in
|
|
||||||
mediaRow(
|
|
||||||
title: result.title ?? result.videoID ?? "Unknown",
|
|
||||||
detail: rowDetail(channel: result.channel, duration: result.duration, inLibrary: result.inLibrary),
|
|
||||||
playEnabled: true
|
|
||||||
) {
|
|
||||||
Task {
|
|
||||||
if let trackID = result.trackID {
|
|
||||||
await playerController.playTrack(id: trackID)
|
|
||||||
} else {
|
|
||||||
await playerController.playVideo(
|
|
||||||
videoID: result.videoID,
|
|
||||||
title: result.title,
|
|
||||||
channel: result.channel
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var librarySection: some View {
|
|
||||||
VStack(alignment: .leading, spacing: 8) {
|
|
||||||
HStack {
|
|
||||||
Text("Library")
|
|
||||||
.font(.headline)
|
|
||||||
Spacer()
|
|
||||||
Button(action: refreshLibrary) {
|
|
||||||
Image(systemName: "arrow.clockwise").frame(width: 44, height: 44)
|
|
||||||
}
|
|
||||||
.disabled(isLoadingLibrary)
|
|
||||||
.accessibilityLabel("Refresh library")
|
|
||||||
}
|
|
||||||
if let libraryError, playerController.stateError == nil {
|
|
||||||
Text(libraryError)
|
|
||||||
.font(.subheadline)
|
|
||||||
.foregroundStyle(.red)
|
|
||||||
} else if let tracks = libraryTracks {
|
|
||||||
if tracks.isEmpty {
|
|
||||||
ContentUnavailableView {
|
|
||||||
Label("A home for your favorites", systemImage: "music.note.list")
|
|
||||||
} description: {
|
|
||||||
Text("Search for a song to save it here, or add a radio station using the + button.")
|
|
||||||
} actions: {
|
|
||||||
Button("Find music") { browseMode = 1 }.buttonStyle(.bordered)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
ForEach(tracks, id: \.id) { track in
|
|
||||||
HStack(spacing: 8) {
|
|
||||||
mediaRow(
|
|
||||||
title: track.title ?? track.id,
|
|
||||||
detail: rowDetail(
|
|
||||||
channel: track.channel ?? (track.kind == "stream" ? "Live stream" : ""),
|
|
||||||
duration: track.duration,
|
|
||||||
status: track.status
|
|
||||||
),
|
|
||||||
playEnabled: track.status == "ready"
|
|
||||||
) {
|
|
||||||
Task { await playerController.playTrack(id: track.id) }
|
|
||||||
}
|
|
||||||
Button(role: .destructive) {
|
|
||||||
trackToDelete = track
|
|
||||||
} label: {
|
|
||||||
Image(systemName: "xmark")
|
|
||||||
.font(.subheadline.weight(.semibold))
|
|
||||||
.foregroundStyle(.red)
|
|
||||||
.frame(width: 44, height: 44)
|
|
||||||
}
|
|
||||||
.accessibilityLabel("Remove \(track.title ?? track.id)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else if isLoadingLibrary {
|
|
||||||
ProgressView("Loading your library…").frame(maxWidth: .infinity).padding(.vertical, 24)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var addStreamSection: some View {
|
|
||||||
Section {
|
|
||||||
TextField("Station name", text: $streamName)
|
|
||||||
.accessibilityLabel("Station name")
|
|
||||||
TextField("https://… audio stream URL", text: $streamURL)
|
|
||||||
.accessibilityLabel("Audio stream URL")
|
|
||||||
.keyboardType(.URL)
|
|
||||||
.autocorrectionDisabled()
|
|
||||||
.textContentType(.URL)
|
|
||||||
.textInputAutocapitalization(.never)
|
|
||||||
if let streamMessage {
|
|
||||||
Text(streamMessage)
|
|
||||||
.font(.subheadline)
|
|
||||||
.foregroundStyle(streamMessage.hasPrefix("Added") ? Color.green : Color.red)
|
|
||||||
}
|
|
||||||
Button(isAddingStream ? "Adding station…" : "Add station", action: addStream)
|
|
||||||
.disabled(streamURL.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty || isAddingStream)
|
|
||||||
} footer: {
|
|
||||||
Text("Use a direct MP3 or AAC audio stream address, rather than the station’s website.")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func mediaRow(
|
|
||||||
title: String,
|
|
||||||
detail: String,
|
|
||||||
playEnabled: Bool,
|
|
||||||
onPlay: @escaping () -> Void
|
|
||||||
) -> some View {
|
|
||||||
HStack(spacing: 10) {
|
|
||||||
VStack(alignment: .leading, spacing: 2) {
|
|
||||||
Text(title)
|
|
||||||
.font(.subheadline.weight(.semibold))
|
|
||||||
.lineLimit(2)
|
|
||||||
if !detail.isEmpty {
|
|
||||||
Text(detail)
|
|
||||||
.font(.caption)
|
|
||||||
.foregroundStyle(.secondary)
|
|
||||||
.lineLimit(2)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Spacer()
|
|
||||||
Button(action: onPlay) {
|
|
||||||
Image(systemName: "play.fill").frame(width: 44, height: 44)
|
|
||||||
}
|
|
||||||
.accessibilityLabel("Play \(title)")
|
|
||||||
.disabled(!playEnabled)
|
|
||||||
}
|
|
||||||
.padding(.vertical, 10)
|
|
||||||
}
|
|
||||||
|
|
||||||
private func rowDetail(channel: String?, duration: Double?, status: String? = nil, inLibrary: Bool? = nil) -> String {
|
|
||||||
var parts: [String] = []
|
|
||||||
if let channel, !channel.isEmpty { parts.append(channel) }
|
|
||||||
if let duration, duration > 0 { parts.append(fmtTime(duration)) }
|
|
||||||
if let status, status != "ready" { parts.append(status) }
|
|
||||||
if inLibrary == true { parts.append("in library") }
|
|
||||||
return parts.joined(separator: " · ")
|
|
||||||
}
|
|
||||||
|
|
||||||
private func performSearch() {
|
|
||||||
let query = searchQuery.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
||||||
guard !query.isEmpty, !isSearching else { return }
|
|
||||||
isSearching = true
|
|
||||||
searchError = nil
|
|
||||||
Task {
|
|
||||||
do {
|
|
||||||
let response = try await MediaAPI(config: serverConfig).search(room: "default", query: query, limit: 6)
|
|
||||||
searchResults = response.results
|
|
||||||
} catch {
|
|
||||||
searchError = error.localizedDescription
|
|
||||||
}
|
|
||||||
isSearching = false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func refreshLibrary() {
|
|
||||||
guard !isLoadingLibrary else { return }
|
|
||||||
isLoadingLibrary = true
|
|
||||||
Task {
|
|
||||||
defer { isLoadingLibrary = false }
|
|
||||||
do {
|
|
||||||
let library = try await MediaAPI(config: serverConfig).library()
|
|
||||||
libraryTracks = library.tracks
|
|
||||||
libraryError = nil
|
|
||||||
} catch {
|
|
||||||
libraryError = error.localizedDescription
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func addStream() {
|
|
||||||
let url = streamURL.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
||||||
guard !isAddingStream else { return }
|
|
||||||
guard let parsed = URL(string: url), ["https", "http"].contains(parsed.scheme?.lowercased() ?? ""), parsed.host != nil else {
|
|
||||||
streamMessage = "Enter a complete http:// or https:// audio stream address."
|
|
||||||
return
|
|
||||||
}
|
|
||||||
isAddingStream = true
|
|
||||||
let name = streamName.trimmingCharacters(in: .whitespaces).isEmpty ? url : streamName.trimmingCharacters(in: .whitespaces)
|
|
||||||
Task {
|
|
||||||
defer { isAddingStream = false }
|
|
||||||
do {
|
|
||||||
try await MediaAPI(config: serverConfig).addStream(room: "default", name: name, url: url)
|
|
||||||
streamMessage = "Added \(name)"
|
|
||||||
streamName = ""
|
|
||||||
streamURL = ""
|
|
||||||
showAddStream = false
|
|
||||||
refreshLibrary()
|
|
||||||
} catch {
|
|
||||||
streamMessage = "Could not add stream: \(error.localizedDescription)"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func deleteTrack(_ track: MediaTrack) async {
|
|
||||||
do {
|
|
||||||
try await MediaAPI(config: serverConfig).deleteTrack(id: track.id)
|
|
||||||
if playerController.track?.id == track.id {
|
|
||||||
Task { await playerController.stop() }
|
|
||||||
}
|
|
||||||
refreshLibrary()
|
|
||||||
} catch {
|
|
||||||
libraryError = error.localizedDescription
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private var serverConfig: ServerConfig {
|
|
||||||
// Use the configured URL directly. If it's invalid, fall back to the
|
|
||||||
// documented default (media operations are less critical than the main
|
|
||||||
// session, and the Settings view will show a validation error).
|
|
||||||
if case .success(let config) = ServerConfig.parse(settings.trimmedServerURL) {
|
|
||||||
return config
|
|
||||||
}
|
|
||||||
let fallback = ServerConfig.parse("https://192.168.86.2:9178")
|
|
||||||
if case .success(let config) = fallback {
|
|
||||||
return config
|
|
||||||
}
|
|
||||||
fatalError("Unreachable: default server URL is valid")
|
|
||||||
}
|
|
||||||
|
|
||||||
private var title: String {
|
|
||||||
playerController.track?.title ?? "Nothing loaded"
|
|
||||||
}
|
|
||||||
|
|
||||||
private var subtitle: String {
|
|
||||||
guard let track = playerController.track else { return "" }
|
|
||||||
if track.status == "failed" {
|
|
||||||
return "Download failed — pick another result or try again."
|
|
||||||
}
|
|
||||||
if track.status == "downloading" {
|
|
||||||
return track.channel.map { "\($0) · downloading" } ?? "Downloading…"
|
|
||||||
}
|
|
||||||
return track.channel ?? (track.kind == "stream" ? "Live stream" : "")
|
|
||||||
}
|
|
||||||
|
|
||||||
private var statusText: String {
|
|
||||||
switch playerController.desired {
|
|
||||||
case "playing": return "Playing"
|
|
||||||
case "paused": return "Paused"
|
|
||||||
default: return "Stopped"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func absolute(_ path: String) -> String {
|
|
||||||
if path.hasPrefix("http") { return path }
|
|
||||||
let base = serverConfig.baseURL.absoluteString
|
|
||||||
let suffix = path.hasPrefix("/") ? path : "/" + path
|
|
||||||
return base + suffix
|
|
||||||
}
|
|
||||||
|
|
||||||
private func togglePlayPause() {
|
|
||||||
Task { await playerController.togglePlayPause() }
|
|
||||||
}
|
|
||||||
|
|
||||||
private func seekBackward() {
|
|
||||||
Task { await playerController.seek(by: -10) }
|
|
||||||
}
|
|
||||||
|
|
||||||
private func seekForward() {
|
|
||||||
Task { await playerController.seek(by: 10) }
|
|
||||||
}
|
|
||||||
|
|
||||||
private func stopPlayback() {
|
|
||||||
Task { await playerController.stop() }
|
|
||||||
}
|
|
||||||
|
|
||||||
private func fmtTime(_ sec: Double) -> String {
|
|
||||||
guard sec.isFinite, sec >= 0 else { return "0:00" }
|
|
||||||
let total = Int(sec)
|
|
||||||
let s = total % 60
|
|
||||||
let m = (total / 60) % 60
|
|
||||||
let h = total / 3600
|
|
||||||
return h > 0 ? "\(h):\(String(format: "%02d", m)):\(String(format: "%02d", s))" : "\(m):\(String(format: "%02d", s))"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Artwork uses the same trust policy as token and media requests.
|
|
||||||
private struct PinnedArtwork: View {
|
|
||||||
let url: URL
|
|
||||||
@State private var artwork: UIImage?
|
|
||||||
|
|
||||||
var body: some View {
|
|
||||||
Group {
|
|
||||||
if let artwork { Image(uiImage: artwork).resizable().scaledToFill() }
|
|
||||||
else {
|
|
||||||
Image(systemName: "music.note").font(.title2).foregroundStyle(.secondary)
|
|
||||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
|
||||||
.background(Color(.tertiarySystemFill))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
.accessibilityHidden(true)
|
|
||||||
.task(id: url) {
|
|
||||||
artwork = nil
|
|
||||||
let session = TLSPinning.session()
|
|
||||||
defer { session.invalidateAndCancel() }
|
|
||||||
guard let (data, response) = try? await session.data(from: url),
|
|
||||||
(response as? HTTPURLResponse)?.statusCode == 200,
|
|
||||||
!Task.isCancelled else { return }
|
|
||||||
artwork = UIImage(data: data)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -2,7 +2,7 @@ import Foundation
|
|||||||
import LiveKit
|
import LiveKit
|
||||||
|
|
||||||
/// Outbound helpers for agent control topics and inbound parsing of
|
/// Outbound helpers for agent control topics and inbound parsing of
|
||||||
/// voice-control / conversation-control / media-control data messages.
|
/// voice-control / conversation-control data messages.
|
||||||
@MainActor
|
@MainActor
|
||||||
final class ControlChannel: NSObject, RoomDelegate {
|
final class ControlChannel: NSObject, RoomDelegate {
|
||||||
|
|
||||||
@@ -109,11 +109,6 @@ final class ControlChannel: NSObject, RoomDelegate {
|
|||||||
try await publish(["type": "set_mode", "mode": mode], topic: "conversation-control", room: room)
|
try await publish(["type": "set_mode", "mode": mode], topic: "conversation-control", room: room)
|
||||||
}
|
}
|
||||||
|
|
||||||
func sendMediaState(_ state: String, trackID: String?, room: Room) async throws {
|
|
||||||
var object: [String: Any] = ["type": "media_state", "state": state]
|
|
||||||
if let trackID { object["track_id"] = trackID }
|
|
||||||
try await publish(object, topic: "media-control", room: room)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// MARK: - Minimal JSON value decoder for flexible agent payloads
|
// MARK: - Minimal JSON value decoder for flexible agent payloads
|
||||||
|
|||||||
@@ -48,7 +48,6 @@ final class VoiceSession: ObservableObject {
|
|||||||
let transcriptHandler = TranscriptHandler()
|
let transcriptHandler = TranscriptHandler()
|
||||||
let stateModel = ConversationStateModel()
|
let stateModel = ConversationStateModel()
|
||||||
let taskStore = TaskStore()
|
let taskStore = TaskStore()
|
||||||
let mediaController: MediaPlayerController
|
|
||||||
private let config: ServerConfig
|
private let config: ServerConfig
|
||||||
private let tokenClient: TokenClient
|
private let tokenClient: TokenClient
|
||||||
private let settings: SettingsStore
|
private let settings: SettingsStore
|
||||||
@@ -90,9 +89,6 @@ final class VoiceSession: ObservableObject {
|
|||||||
self.config = config
|
self.config = config
|
||||||
self.tokenClient = tokenClient
|
self.tokenClient = tokenClient
|
||||||
self.settings = settings
|
self.settings = settings
|
||||||
let mediaAPI = MediaAPI(config: config)
|
|
||||||
self.mediaController = MediaPlayerController(api: mediaAPI, settings: settings)
|
|
||||||
mediaController.attach(session: self)
|
|
||||||
room.add(delegate: self)
|
room.add(delegate: self)
|
||||||
room.add(delegate: controlChannel)
|
room.add(delegate: controlChannel)
|
||||||
room.add(delegate: transcriptHandler)
|
room.add(delegate: transcriptHandler)
|
||||||
@@ -211,7 +207,6 @@ final class VoiceSession: ObservableObject {
|
|||||||
do {
|
do {
|
||||||
let token = try await tokenClient.fetchToken()
|
let token = try await tokenClient.fetchToken()
|
||||||
logger.log("token ok room=\(token.room, privacy: .public)")
|
logger.log("token ok room=\(token.room, privacy: .public)")
|
||||||
await mediaController.enterMediaScope(roomName: token.room)
|
|
||||||
try await room.connect(url: config.livekitURL.absoluteString, token: token.token, connectOptions: makeConnectOptions())
|
try await room.connect(url: config.livekitURL.absoluteString, token: token.token, connectOptions: makeConnectOptions())
|
||||||
result = .success(())
|
result = .success(())
|
||||||
} catch {
|
} catch {
|
||||||
@@ -263,8 +258,7 @@ final class VoiceSession: ObservableObject {
|
|||||||
private func handleAgentTimeout(generation: Int) async {
|
private func handleAgentTimeout(generation: Int) async {
|
||||||
guard generation == sessionGeneration, state == .waitingAgent else { return }
|
guard generation == sessionGeneration, state == .waitingAgent else { return }
|
||||||
guard !reconnectAttempted else {
|
guard !reconnectAttempted else {
|
||||||
// Final failure: tear down the live room, restore the idle timer,
|
// Final failure: tear down the live room and restore the idle timer.
|
||||||
// and end the media scope so no stale room or wake lock survives.
|
|
||||||
logger.log("agent timeout final, tearing down")
|
logger.log("agent timeout final, tearing down")
|
||||||
await teardownForFailure(message: "Agent never joined. Is the agent worker running?")
|
await teardownForFailure(message: "Agent never joined. Is the agent worker running?")
|
||||||
return
|
return
|
||||||
@@ -281,8 +275,7 @@ final class VoiceSession: ObservableObject {
|
|||||||
/// current generation (so stale callbacks become no-ops), clear all
|
/// current generation (so stale callbacks become no-ops), clear all
|
||||||
/// per-room/session ephemeral published fields (agent, mute state, model
|
/// per-room/session ephemeral published fields (agent, mute state, model
|
||||||
/// list, error state, interruption ownership), disconnect and recreate the
|
/// list, error state, interruption ownership), disconnect and recreate the
|
||||||
/// room, stop conversation state, restore the idle timer, and end the media
|
/// room, stop conversation state, restore the idle timer, and publish
|
||||||
/// scope. Only after all cleanup completes does the session publish
|
|
||||||
/// `.failed(message)`, making retry safe: the old room is fully torn down
|
/// `.failed(message)`, making retry safe: the old room is fully torn down
|
||||||
/// before Start becomes available again.
|
/// before Start becomes available again.
|
||||||
///
|
///
|
||||||
@@ -308,7 +301,6 @@ final class VoiceSession: ObservableObject {
|
|||||||
interruptionSequence += 1
|
interruptionSequence += 1
|
||||||
stateModel.stop()
|
stateModel.stop()
|
||||||
UIApplication.shared.isIdleTimerDisabled = false
|
UIApplication.shared.isIdleTimerDisabled = false
|
||||||
await mediaController.endConversation()
|
|
||||||
await teardownRoom()
|
await teardownRoom()
|
||||||
|
|
||||||
// Final reset immediately before publishing .failed: any old-room
|
// Final reset immediately before publishing .failed: any old-room
|
||||||
@@ -540,15 +532,13 @@ final class VoiceSession: ObservableObject {
|
|||||||
interruptionSequence += 1
|
interruptionSequence += 1
|
||||||
stateModel.stop()
|
stateModel.stop()
|
||||||
UIApplication.shared.isIdleTimerDisabled = false
|
UIApplication.shared.isIdleTimerDisabled = false
|
||||||
await mediaController.endConversation()
|
|
||||||
await teardownRoom()
|
await teardownRoom()
|
||||||
// Final reset before publishing .idle: any old-room callback delivered
|
// Final reset before publishing .idle: any old-room callback delivered
|
||||||
// during the teardownRoom() await may have repopulated ephemeral fields.
|
// during the teardownRoom() await may have repopulated ephemeral fields.
|
||||||
// This ensures the idle state is clean for server replacement (Iteration 09).
|
// This ensures the idle state is clean for server replacement (Iteration 09).
|
||||||
resetEphemeralFields()
|
resetEphemeralFields()
|
||||||
// Publish .idle only after media and room teardown have completed.
|
// Publish .idle only after room teardown has completed: the old session
|
||||||
// This makes idle a truthful readiness signal for server replacement:
|
// is fully torn down before the UI shows "ready to start".
|
||||||
// the old session is fully torn down before the UI shows "ready to start".
|
|
||||||
state = .idle
|
state = .idle
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,249 +0,0 @@
|
|||||||
import Foundation
|
|
||||||
|
|
||||||
struct MediaTrack: Decodable, Equatable {
|
|
||||||
let id: String
|
|
||||||
let kind: String?
|
|
||||||
let videoID: String?
|
|
||||||
let title: String?
|
|
||||||
let channel: String?
|
|
||||||
let duration: Double?
|
|
||||||
let status: String?
|
|
||||||
let error: String?
|
|
||||||
let audioURL: String?
|
|
||||||
let artURL: String?
|
|
||||||
|
|
||||||
enum CodingKeys: String, CodingKey {
|
|
||||||
case id, kind, title, channel, duration, status, error
|
|
||||||
case videoID = "video_id"
|
|
||||||
case audioURL = "audio_url"
|
|
||||||
case artURL = "art_url"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaPendingJob: Decodable, Equatable {
|
|
||||||
let jobID: String?
|
|
||||||
let trackID: String?
|
|
||||||
let stage: String?
|
|
||||||
let progress: Double?
|
|
||||||
let error: String?
|
|
||||||
|
|
||||||
enum CodingKeys: String, CodingKey {
|
|
||||||
case stage, progress, error
|
|
||||||
case jobID = "job_id"
|
|
||||||
case trackID = "track_id"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaSeekHint: Decodable, Equatable {
|
|
||||||
let ts: Double
|
|
||||||
let position: Double
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaState: Decodable, Equatable {
|
|
||||||
let room: String?
|
|
||||||
let track: MediaTrack?
|
|
||||||
let desired: String?
|
|
||||||
let reported: MediaReported?
|
|
||||||
let seekHint: MediaSeekHint?
|
|
||||||
let choices: [MediaChoice]
|
|
||||||
let pending: [MediaPendingJob]
|
|
||||||
let version: Int?
|
|
||||||
|
|
||||||
enum CodingKeys: String, CodingKey {
|
|
||||||
case room, track, desired, reported, choices, pending, version
|
|
||||||
case seekHint = "seek_hint"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaReported: Decodable, Equatable {
|
|
||||||
let trackID: String?
|
|
||||||
let state: String?
|
|
||||||
let position: Double?
|
|
||||||
let duration: Double?
|
|
||||||
let ts: Double?
|
|
||||||
|
|
||||||
enum CodingKeys: String, CodingKey {
|
|
||||||
case state, position, duration, ts
|
|
||||||
case trackID = "track_id"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaChoice: Decodable, Equatable {
|
|
||||||
let videoID: String?
|
|
||||||
let title: String?
|
|
||||||
let channel: String?
|
|
||||||
let duration: Double?
|
|
||||||
|
|
||||||
enum CodingKeys: String, CodingKey {
|
|
||||||
case title, channel, duration
|
|
||||||
case videoID = "video_id"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaLibrary: Decodable, Equatable {
|
|
||||||
let tracks: [MediaTrack]
|
|
||||||
let version: Int?
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaSearchResult: Decodable, Equatable {
|
|
||||||
let videoID: String?
|
|
||||||
let trackID: String?
|
|
||||||
let title: String?
|
|
||||||
let channel: String?
|
|
||||||
let duration: Double?
|
|
||||||
let inLibrary: Bool?
|
|
||||||
|
|
||||||
enum CodingKeys: String, CodingKey {
|
|
||||||
case title, channel, duration
|
|
||||||
case videoID = "video_id"
|
|
||||||
case trackID = "track_id"
|
|
||||||
case inLibrary = "in_library"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
struct MediaSearchResponse: Decodable, Equatable {
|
|
||||||
let results: [MediaSearchResult]
|
|
||||||
}
|
|
||||||
|
|
||||||
enum MediaAPIError: LocalizedError {
|
|
||||||
case invalidServerURL(String)
|
|
||||||
case httpStatus(Int, String)
|
|
||||||
case decodeFailed
|
|
||||||
|
|
||||||
var errorDescription: String? {
|
|
||||||
switch self {
|
|
||||||
case .invalidServerURL(let raw):
|
|
||||||
return "Invalid server URL: \(raw)"
|
|
||||||
case .httpStatus(let code, let detail):
|
|
||||||
return "Media API request failed with HTTP \(code). \(detail)"
|
|
||||||
case .decodeFailed:
|
|
||||||
return "Unexpected response from the media endpoint."
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
final class MediaAPI {
|
|
||||||
let serverBase: URL
|
|
||||||
private let baseURL: URL
|
|
||||||
private let session: URLSession
|
|
||||||
|
|
||||||
init(config: ServerConfig, session: URLSession? = nil) {
|
|
||||||
self.serverBase = config.baseURL
|
|
||||||
self.baseURL = config.baseURL.appendingPathComponent("media/api")
|
|
||||||
self.session = session ?? TLSPinning.session()
|
|
||||||
}
|
|
||||||
|
|
||||||
func state(room: String) async throws -> MediaState {
|
|
||||||
var components = URLComponents(url: baseURL.appendingPathComponent("state"), resolvingAgainstBaseURL: false)!
|
|
||||||
components.queryItems = [URLQueryItem(name: "room", value: room)]
|
|
||||||
return try await get(MediaState.self, url: components.url!)
|
|
||||||
}
|
|
||||||
|
|
||||||
func report(room: String, state: String, trackID: String?, position: Double, duration: Double?) async throws {
|
|
||||||
var body: [String: Any] = ["room": room, "state": state, "position": position]
|
|
||||||
if let trackID { body["track_id"] = trackID }
|
|
||||||
if let duration { body["duration"] = duration }
|
|
||||||
try await post("report", body: body)
|
|
||||||
}
|
|
||||||
|
|
||||||
func play(room: String, videoID: String? = nil, title: String? = nil, channel: String? = nil, trackID: String? = nil, startPaused: Bool? = nil) async throws {
|
|
||||||
var body: [String: Any] = ["room": room]
|
|
||||||
if let videoID { body["video_id"] = videoID }
|
|
||||||
if let title { body["title"] = title }
|
|
||||||
if let channel { body["channel"] = channel }
|
|
||||||
if let trackID { body["track_id"] = trackID }
|
|
||||||
if let startPaused { body["start_paused"] = startPaused }
|
|
||||||
try await post("play", body: body)
|
|
||||||
}
|
|
||||||
|
|
||||||
func control(room: String, action: String) async throws {
|
|
||||||
try await post("control", body: ["room": room, "action": action])
|
|
||||||
}
|
|
||||||
|
|
||||||
func search(room: String, query: String, limit: Int = 6) async throws -> MediaSearchResponse {
|
|
||||||
try await postDecoded("search", body: ["room": room, "query": query, "limit": limit], as: MediaSearchResponse.self)
|
|
||||||
}
|
|
||||||
|
|
||||||
func library() async throws -> MediaLibrary {
|
|
||||||
try await get(MediaLibrary.self, url: baseURL.appendingPathComponent("library"))
|
|
||||||
}
|
|
||||||
|
|
||||||
func deleteTrack(id: String) async throws {
|
|
||||||
var request = URLRequest(url: baseURL.appendingPathComponent("tracks").appending(path: id))
|
|
||||||
request.httpMethod = "DELETE"
|
|
||||||
let (data, response) = try await session.data(for: request)
|
|
||||||
guard let http = response as? HTTPURLResponse else { throw MediaAPIError.decodeFailed }
|
|
||||||
guard (200..<300).contains(http.statusCode) else {
|
|
||||||
throw MediaAPIError.httpStatus(http.statusCode, String(data: data, encoding: .utf8) ?? "")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func addStream(room: String, name: String, url: String) async throws {
|
|
||||||
try await post("streams", body: ["room": room, "name": name, "url": url])
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Returns the progressive-media endpoint URL for a track. The `/stream/{id}`
|
|
||||||
/// path serves raw media bytes (not a text URL), so this constructs the
|
|
||||||
/// endpoint without downloading or decoding the body.
|
|
||||||
func streamURL(trackID: String) -> URL {
|
|
||||||
baseURL.appendingPathComponent("stream").appending(path: trackID)
|
|
||||||
}
|
|
||||||
|
|
||||||
func healthz() async throws -> Bool {
|
|
||||||
let data = try await getRaw(url: baseURL.appendingPathComponent("healthz"))
|
|
||||||
guard let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { return false }
|
|
||||||
return json["ok"] as? Bool ?? false
|
|
||||||
}
|
|
||||||
|
|
||||||
private func get<T: Decodable>(_ type: T.Type, url: URL) async throws -> T {
|
|
||||||
let data = try await getRaw(url: url)
|
|
||||||
do {
|
|
||||||
return try JSONDecoder().decode(T.self, from: data)
|
|
||||||
} catch {
|
|
||||||
throw MediaAPIError.decodeFailed
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func getRaw(url: URL) async throws -> Data {
|
|
||||||
let (data, response) = try await session.data(from: url)
|
|
||||||
guard let http = response as? HTTPURLResponse else { throw MediaAPIError.decodeFailed }
|
|
||||||
guard (200..<300).contains(http.statusCode) else {
|
|
||||||
throw MediaAPIError.httpStatus(http.statusCode, String(data: data, encoding: .utf8) ?? "")
|
|
||||||
}
|
|
||||||
return data
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Fire-and-forget POST that only validates HTTP success. The media server
|
|
||||||
/// returns `{"ok": true, ...snapshot}` for /play, /control, /report and
|
|
||||||
/// /streams — callers don't need the body, so decoding it (as Data or any
|
|
||||||
/// typed model) would be wasted work at best and a spurious failure mode.
|
|
||||||
private func post(_ path: String, body: [String: Any]) async throws {
|
|
||||||
_ = try await postRaw(path, body: body)
|
|
||||||
}
|
|
||||||
|
|
||||||
private func postDecoded<T: Decodable>(_ path: String, body: [String: Any], as type: T.Type) async throws -> T {
|
|
||||||
let data = try await postRaw(path, body: body)
|
|
||||||
do {
|
|
||||||
return try JSONDecoder().decode(T.self, from: data)
|
|
||||||
} catch {
|
|
||||||
throw MediaAPIError.decodeFailed
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func postRaw(_ path: String, body: [String: Any]) async throws -> Data {
|
|
||||||
var request = URLRequest(url: baseURL.appendingPathComponent(path))
|
|
||||||
request.httpMethod = "POST"
|
|
||||||
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
|
|
||||||
request.httpBody = try JSONSerialization.data(withJSONObject: body)
|
|
||||||
let (data, response) = try await session.data(for: request)
|
|
||||||
guard let http = response as? HTTPURLResponse else { throw MediaAPIError.decodeFailed }
|
|
||||||
guard (200..<300).contains(http.statusCode) else {
|
|
||||||
var detail = String(data: data, encoding: .utf8) ?? ""
|
|
||||||
if let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any], let error = json["error"] as? String {
|
|
||||||
detail = error
|
|
||||||
}
|
|
||||||
throw MediaAPIError.httpStatus(http.statusCode, detail)
|
|
||||||
}
|
|
||||||
return data
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,633 +0,0 @@
|
|||||||
import AVFoundation
|
|
||||||
import Combine
|
|
||||||
import Foundation
|
|
||||||
import os
|
|
||||||
|
|
||||||
private let logger = Logger(subsystem: "dev.shane.voice", category: "media")
|
|
||||||
|
|
||||||
/// AVPlayer-backed media player. The media server holds per-room desired state;
|
|
||||||
/// this controller polls /state and reconciles the local player against it,
|
|
||||||
/// porting the web UI's reconciliation rules (app.js reconcileMedia).
|
|
||||||
@MainActor
|
|
||||||
final class MediaPlayerController: ObservableObject {
|
|
||||||
@Published private(set) var track: MediaTrack?
|
|
||||||
@Published private(set) var desired: String = "stopped"
|
|
||||||
@Published private(set) var currentTime: Double = 0
|
|
||||||
@Published private(set) var duration: Double = 0
|
|
||||||
@Published private(set) var pendingJobs: [MediaPendingJob] = []
|
|
||||||
@Published private(set) var choices: [MediaChoice] = []
|
|
||||||
@Published private(set) var stateError: String?
|
|
||||||
@Published private(set) var playbackError: String?
|
|
||||||
|
|
||||||
let player = AVPlayer()
|
|
||||||
|
|
||||||
private let api: MediaAPI
|
|
||||||
private let settings: SettingsStore
|
|
||||||
private let mediaLoader: PinnedMediaResourceLoader
|
|
||||||
private weak var session: VoiceSession?
|
|
||||||
|
|
||||||
private var room: String?
|
|
||||||
private var currentTrackID: String?
|
|
||||||
private var loadedAudioPath: String?
|
|
||||||
private var currentAsset: AVURLAsset?
|
|
||||||
private var appliedSeekTs: Double = 0
|
|
||||||
private var wasPlayingBeforeTalk = false
|
|
||||||
private var lastReportDate = Date.distantPast
|
|
||||||
private var pollTask: Task<Void, Never>?
|
|
||||||
private var timeObserver: Any?
|
|
||||||
private var statusCancellable: AnyCancellable?
|
|
||||||
private var rateObservation: NSKeyValueObservation?
|
|
||||||
|
|
||||||
/// Bumped on every room-scope change so in-flight /state polls from the
|
|
||||||
/// previous scope cannot reconcile into the new one (port of the web UI's
|
|
||||||
/// `=== room` checks).
|
|
||||||
private var pollGeneration = 0
|
|
||||||
|
|
||||||
init(api: MediaAPI, settings: SettingsStore) {
|
|
||||||
self.api = api
|
|
||||||
self.settings = settings
|
|
||||||
self.mediaLoader = PinnedMediaResourceLoader(base: api.serverBase)
|
|
||||||
player.volume = Float(settings.volume)
|
|
||||||
NotificationCenter.default.addObserver(
|
|
||||||
self, selector: #selector(handleInterruption),
|
|
||||||
name: AVAudioSession.interruptionNotification, object: nil
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
deinit {
|
|
||||||
if let timeObserver {
|
|
||||||
player.removeTimeObserver(timeObserver)
|
|
||||||
}
|
|
||||||
rateObservation?.invalidate()
|
|
||||||
NotificationCenter.default.removeObserver(self)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Whether an audio session interruption is currently active. While true,
|
|
||||||
/// reconciliation must not call player.play() (the next poll would restart
|
|
||||||
/// audio during a phone call or Siri interaction).
|
|
||||||
private var isInterruptionActive = false
|
|
||||||
|
|
||||||
/// Separate resume-suppression flag: set when an interruption ends without
|
|
||||||
/// `.shouldResume` and the pause control to the server has not yet been
|
|
||||||
/// confirmed. While true, reconciliation must not call player.play() even
|
|
||||||
/// after `isInterruptionActive` is cleared. Cleared only by an explicit user
|
|
||||||
/// play/resume action or a confirmed coherent paused state from the server.
|
|
||||||
private var resumeSuppressed = false
|
|
||||||
|
|
||||||
/// Whether playback was actually active (rate > 0) when the interruption
|
|
||||||
/// began. Used to decide whether a `.shouldResume` end should actually
|
|
||||||
/// resume (a track that was already paused before the interruption must not
|
|
||||||
/// be force-resumed).
|
|
||||||
private var wasPlayingBeforeInterruption = false
|
|
||||||
|
|
||||||
/// Sequence number for media interruption operations. Incremented on every
|
|
||||||
/// `.began`; a resume task captures the sequence and rechecks it immediately
|
|
||||||
/// before `player.play()`. A newer `.began` makes an old resume task
|
|
||||||
/// incapable of playing (Iteration 07 Item 5).
|
|
||||||
private var mediaInterruptionSequence = 0
|
|
||||||
|
|
||||||
/// The retained operation chain for media interruption control work. Both
|
|
||||||
/// pause and resume operations are assigned here so a newer begin cancels
|
|
||||||
/// and supersedes prior control work.
|
|
||||||
private var mediaInterruptionTask: Task<Void, Never>?
|
|
||||||
|
|
||||||
@objc private func handleInterruption(_ notification: Notification) {
|
|
||||||
guard let info = notification.userInfo else { return }
|
|
||||||
guard let typeValue = info[AVAudioSessionInterruptionTypeKey] as? UInt,
|
|
||||||
let type = AVAudioSession.InterruptionType(rawValue: typeValue) else { return }
|
|
||||||
switch type {
|
|
||||||
case .began:
|
|
||||||
// Increment the sequence so any in-flight resume task from a prior
|
|
||||||
// interruption sees a stale sequence and cannot play.
|
|
||||||
mediaInterruptionSequence += 1
|
|
||||||
let seq = mediaInterruptionSequence
|
|
||||||
// Cancel/supersede prior control work (resume or pause) so it cannot
|
|
||||||
// complete out of order.
|
|
||||||
mediaInterruptionTask?.cancel()
|
|
||||||
isInterruptionActive = true
|
|
||||||
wasPlayingBeforeInterruption = !player.rate.isZero
|
|
||||||
let wasPlaying = wasPlayingBeforeInterruption
|
|
||||||
player.pause()
|
|
||||||
logger.log("audio interruption began, paused media (wasPlaying=\(wasPlaying), seq=\(seq))")
|
|
||||||
case .ended:
|
|
||||||
defer { isInterruptionActive = false }
|
|
||||||
applyVolume()
|
|
||||||
let optionsValue = info[AVAudioSessionInterruptionOptionKey] as? UInt ?? 0
|
|
||||||
let shouldResume = AVAudioSession.InterruptionOptions(rawValue: optionsValue).contains(.shouldResume)
|
|
||||||
if shouldResume, wasPlayingBeforeInterruption, currentTrackID != nil, desired == "playing", loadedAudioPath != nil {
|
|
||||||
// Synchronize the server before resuming local playback so the
|
|
||||||
// two never diverge. Capture generation/room/sequence before the
|
|
||||||
// async hop. The task is retained in mediaInterruptionTask so a
|
|
||||||
// newer .began can cancel it.
|
|
||||||
let gen = pollGeneration
|
|
||||||
let targetRoom = room
|
|
||||||
let seq = mediaInterruptionSequence
|
|
||||||
mediaInterruptionTask = Task { [weak self] in
|
|
||||||
guard let self else { return }
|
|
||||||
if let targetRoom {
|
|
||||||
do {
|
|
||||||
try await self.api.control(room: targetRoom, action: "resume")
|
|
||||||
logger.log("interruption resume: server synchronized (seq=\(seq))")
|
|
||||||
} catch {
|
|
||||||
logger.log("interruption resume: server sync failed (poll will retry): \(error.localizedDescription)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Recheck the sequence immediately before play: a newer .began
|
|
||||||
// increments mediaInterruptionSequence, making this stale task
|
|
||||||
// incapable of restarting audio during the newer interruption.
|
|
||||||
guard !Task.isCancelled,
|
|
||||||
self.mediaInterruptionSequence == seq,
|
|
||||||
!self.isInterruptionActive,
|
|
||||||
self.pollGeneration == gen,
|
|
||||||
self.room == targetRoom else { return }
|
|
||||||
self.player.play()
|
|
||||||
logger.log("audio interruption ended, resumed media (seq=\(seq))")
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// No-resume path: keep suppression even if the pause control
|
|
||||||
// fails. The flag is cleared only by an explicit user play/resume
|
|
||||||
// action or a confirmed coherent paused state from the server.
|
|
||||||
resumeSuppressed = true
|
|
||||||
let seq = mediaInterruptionSequence
|
|
||||||
if let room {
|
|
||||||
mediaInterruptionTask = Task { [weak self] in
|
|
||||||
guard let self else { return }
|
|
||||||
do {
|
|
||||||
try await self.api.control(room: room, action: "pause")
|
|
||||||
logger.log("audio interruption ended without resume, sent pause to server (seq=\(seq))")
|
|
||||||
} catch {
|
|
||||||
// Keep suppression even on failure — the next poll
|
|
||||||
// will see desired=="paused" and clear it.
|
|
||||||
logger.log("audio interruption ended without resume, pause control failed (suppression retained): \(error.localizedDescription)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@unknown default:
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Lifecycle
|
|
||||||
|
|
||||||
func attach(session: VoiceSession) {
|
|
||||||
self.session = session
|
|
||||||
observePlayer()
|
|
||||||
observePlaybackState()
|
|
||||||
applyVolume()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Called just before joining the LiveKit room. Carries whatever is loaded
|
|
||||||
/// into the new room-scoped player state, paused (talking pauses music).
|
|
||||||
func enterMediaScope(roomName: String) async {
|
|
||||||
pollGeneration += 1
|
|
||||||
let wasPlaying = currentTrackID != nil && !player.rate.isZero
|
|
||||||
wasPlayingBeforeTalk = wasPlaying
|
|
||||||
let carriedTrack = currentTrackID
|
|
||||||
room = roomName
|
|
||||||
if let carriedTrack {
|
|
||||||
do {
|
|
||||||
try await api.play(room: roomName, trackID: carriedTrack, startPaused: true)
|
|
||||||
logger.log("media scope carry-over: \(carriedTrack, privacy: .public) start_paused")
|
|
||||||
} catch {
|
|
||||||
logger.log("media scope transfer failed: \(String(describing: error), privacy: .public)")
|
|
||||||
}
|
|
||||||
player.pause()
|
|
||||||
}
|
|
||||||
startPolling()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Called when the conversation ends. If music was playing before the talk,
|
|
||||||
/// hand the room back to the player. The previous scope (room name) is kept
|
|
||||||
/// so the panel can resume polling it via `startDefaultPolling()`; a fresh
|
|
||||||
/// conversation re-scopes via `enterMediaScope`. Polling restarts here so
|
|
||||||
/// the resumed state stays visible without the panel being open.
|
|
||||||
func endConversation() async {
|
|
||||||
stopPolling()
|
|
||||||
pollGeneration += 1
|
|
||||||
if wasPlayingBeforeTalk {
|
|
||||||
wasPlayingBeforeTalk = false
|
|
||||||
guard let room else { return }
|
|
||||||
do {
|
|
||||||
try await api.control(room: room, action: "resume")
|
|
||||||
logger.log("resumed playback after talk ended")
|
|
||||||
} catch {
|
|
||||||
logger.log("post-talk resume failed (poll will retry): \(String(describing: error), privacy: .public)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
startPolling()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Drop the room scope entirely (app-level teardown). After this, polling
|
|
||||||
/// targets the default room again.
|
|
||||||
func resetScope() {
|
|
||||||
stopPolling()
|
|
||||||
pollGeneration += 1
|
|
||||||
room = nil
|
|
||||||
wasPlayingBeforeTalk = false
|
|
||||||
clearMedia()
|
|
||||||
}
|
|
||||||
|
|
||||||
private func startPolling() {
|
|
||||||
guard pollTask == nil else { return }
|
|
||||||
pollTask = Task { [weak self] in
|
|
||||||
while !Task.isCancelled {
|
|
||||||
await self?.pollState()
|
|
||||||
try? await Task.sleep(for: .milliseconds(1500))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func stopPolling() {
|
|
||||||
pollTask?.cancel()
|
|
||||||
pollTask = nil
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Resume polling whenever the panel becomes visible and nothing is being
|
|
||||||
/// polled yet — whether that's the default room (no session) or a previous
|
|
||||||
/// conversation's room scope (kept after `endConversation`).
|
|
||||||
func startDefaultPolling() {
|
|
||||||
guard pollTask == nil else { return }
|
|
||||||
startPolling()
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Poll + reconcile (ported from app.js pollMediaState/reconcileMedia)
|
|
||||||
|
|
||||||
private func pollState() async {
|
|
||||||
let generation = pollGeneration
|
|
||||||
let scope = room ?? "default"
|
|
||||||
do {
|
|
||||||
let state = try await api.state(room: scope)
|
|
||||||
guard generation == pollGeneration else { return }
|
|
||||||
stateError = nil
|
|
||||||
reconcile(state)
|
|
||||||
} catch {
|
|
||||||
guard generation == pollGeneration else { return }
|
|
||||||
logger.log("poll failed: \(String(describing: error), privacy: .public)")
|
|
||||||
stateError = "Media server unreachable: \(error.localizedDescription)"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Reconciles the local player against a server state snapshot. Exposed
|
|
||||||
/// as internal (not private) so tests can drive deterministic state
|
|
||||||
/// transitions without a live server.
|
|
||||||
func reconcile(_ s: MediaState) {
|
|
||||||
choices = s.choices
|
|
||||||
pendingJobs = s.pending
|
|
||||||
// Keep the published desired state in sync with the server so the UI's
|
|
||||||
// play/pause icon and status text reflect reality (the server is the
|
|
||||||
// source of truth for per-room desired state).
|
|
||||||
desired = s.desired ?? "stopped"
|
|
||||||
|
|
||||||
let track = s.track
|
|
||||||
self.track = track
|
|
||||||
if let track, track.id != currentTrackID {
|
|
||||||
loadTrack(track)
|
|
||||||
} else if let track, let audioURL = track.audioURL, loadedAudioPath != audioURL {
|
|
||||||
// Track was loaded before it had a stream (still downloading) and
|
|
||||||
// now has one — wire it up so playback can start without a reload.
|
|
||||||
wireSource(audioURL)
|
|
||||||
}
|
|
||||||
if track == nil, currentTrackID != nil, s.desired == "stopped" {
|
|
||||||
clearMedia()
|
|
||||||
}
|
|
||||||
|
|
||||||
applyDesiredState(s, track: track)
|
|
||||||
|
|
||||||
if let hint = s.seekHint, hint.ts != appliedSeekTs, duration.isFinite, duration > 0 {
|
|
||||||
appliedSeekTs = hint.ts
|
|
||||||
let target = min(hint.position, duration)
|
|
||||||
if target.isFinite {
|
|
||||||
seek(to: max(0, target))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func applyDesiredState(_ s: MediaState, track: MediaTrack?) {
|
|
||||||
guard let desiredValue = s.desired else { return }
|
|
||||||
switch desiredValue {
|
|
||||||
case "playing":
|
|
||||||
// Suppress play during an active audio interruption (phone call,
|
|
||||||
// Siri). Without this guard the next 1.5s poll would restart audio
|
|
||||||
// immediately after .began paused it. The player.rate.isZero check
|
|
||||||
// makes this idempotent: once playing, subsequent polls are no-ops.
|
|
||||||
// This also handles the late-audio case where desired=playing
|
|
||||||
// arrives before audio_url is set; when the source is later wired
|
|
||||||
// via wireSource(), the next poll retries play because rate is
|
|
||||||
// still zero.
|
|
||||||
// Also suppress while resumeSuppressed is true (no-resume
|
|
||||||
// interruption ended, server not yet confirmed paused).
|
|
||||||
guard !isInterruptionActive, !resumeSuppressed else { return }
|
|
||||||
if let track, track.audioURL != nil, loadedAudioPath != nil, player.rate.isZero {
|
|
||||||
player.play()
|
|
||||||
}
|
|
||||||
case "paused", "stopped":
|
|
||||||
if !player.rate.isZero {
|
|
||||||
player.pause()
|
|
||||||
}
|
|
||||||
// The server has confirmed a coherent paused/stopped state. If we
|
|
||||||
// were suppressing resume due to a no-resume interruption, the
|
|
||||||
// pause control succeeded (or the user paused independently), so
|
|
||||||
// it is safe to clear the suppression flag.
|
|
||||||
if resumeSuppressed {
|
|
||||||
resumeSuppressed = false
|
|
||||||
logger.log("resume suppression cleared: server confirmed \(desiredValue)")
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func loadTrack(_ track: MediaTrack) {
|
|
||||||
currentTrackID = track.id
|
|
||||||
self.track = track
|
|
||||||
if let audioURL = track.audioURL {
|
|
||||||
wireSource(audioURL)
|
|
||||||
} else {
|
|
||||||
// Downloading or failed: no stream yet. Reconcile wires the stream
|
|
||||||
// URL in via polling as soon as one is available.
|
|
||||||
clearCurrentAsset()
|
|
||||||
player.replaceCurrentItem(with: nil)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func wireSource(_ path: String) {
|
|
||||||
// A complete file is required for the server's cached AAC conversion.
|
|
||||||
// Keep showing download progress; the next ready-state poll wires it.
|
|
||||||
guard track?.kind != "youtube" || track?.status != "downloading" else { return }
|
|
||||||
guard var components = URLComponents(string: path) else { return }
|
|
||||||
if track?.kind == "youtube", components.path.hasPrefix("/media/api/stream/") {
|
|
||||||
var items = components.queryItems ?? []
|
|
||||||
items.removeAll { $0.name == "format" }
|
|
||||||
items.append(URLQueryItem(name: "format", value: "m4a"))
|
|
||||||
components.queryItems = items
|
|
||||||
}
|
|
||||||
guard let url = components.url else { return }
|
|
||||||
loadedAudioPath = path
|
|
||||||
let pinned = mediaLoader.rewrite(url)
|
|
||||||
// Remove the previous asset's loader mapping before wiring a new one so
|
|
||||||
// the loaderURLs dictionary stays bounded (Finding 11 / Item 7).
|
|
||||||
clearCurrentAsset()
|
|
||||||
let asset = mediaLoader.assetWithPinnedURL(pinned)
|
|
||||||
currentAsset = asset
|
|
||||||
logger.log("wiring source \(pinned.absoluteString, privacy: .public)")
|
|
||||||
let item = AVPlayerItem(asset: asset)
|
|
||||||
observeItem(item)
|
|
||||||
player.replaceCurrentItem(with: item)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Removes the loader mapping for the current asset (if any) and clears the
|
|
||||||
/// reference. Called before wiring a new source or clearing media so the
|
|
||||||
/// resource loader's internal dictionaries stay bounded.
|
|
||||||
private func clearCurrentAsset() {
|
|
||||||
if let asset = currentAsset {
|
|
||||||
mediaLoader.removeLoader(asset)
|
|
||||||
currentAsset = nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Player events
|
|
||||||
|
|
||||||
private func observePlayer() {
|
|
||||||
if timeObserver == nil {
|
|
||||||
timeObserver = player.addPeriodicTimeObserver(
|
|
||||||
forInterval: CMTime(seconds: 0.5, preferredTimescale: 600), queue: .main
|
|
||||||
) { [weak self] time in
|
|
||||||
Task { @MainActor [weak self] in
|
|
||||||
guard let self else { return }
|
|
||||||
self.currentTime = time.seconds
|
|
||||||
if let itemDuration = self.player.currentItem?.duration {
|
|
||||||
let seconds = itemDuration.seconds
|
|
||||||
if seconds.isFinite {
|
|
||||||
self.duration = seconds
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if !self.player.rate.isZero, Date().timeIntervalSince(self.lastReportDate) >= 5 {
|
|
||||||
self.lastReportDate = Date()
|
|
||||||
self.report("playing")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: - Playback state observation
|
|
||||||
|
|
||||||
/// Observe the player's actual rate so every real play/pause transition is
|
|
||||||
/// reported promptly, mirroring the web UI's `play`/`pause` listeners. The
|
|
||||||
/// KVO block runs on an arbitrary queue; hop to MainActor before touching
|
|
||||||
/// any controller state.
|
|
||||||
private func observePlaybackState() {
|
|
||||||
rateObservation = player.observe(\.rate, options: [.initial, .new]) { [weak self] player, _ in
|
|
||||||
Task { @MainActor [weak self] in
|
|
||||||
guard let self else { return }
|
|
||||||
guard self.currentTrackID != nil else { return }
|
|
||||||
if player.rate > 0 {
|
|
||||||
// Actual playback began (user play, interruption resume, or
|
|
||||||
// reconcile-driven play). Report + publish the transition.
|
|
||||||
self.lastReportDate = Date()
|
|
||||||
self.report("playing")
|
|
||||||
self.publishMediaState("playing")
|
|
||||||
} else if player.currentItem != nil {
|
|
||||||
// Rate dropped to zero with an item still loaded: a pause
|
|
||||||
// that didn't originate from a user action (interruption,
|
|
||||||
// reconcile). User-initiated pauses already report before
|
|
||||||
// this fires; the lastReportDate guard makes duplicates no-ops.
|
|
||||||
if Date().timeIntervalSince(self.lastReportDate) >= 1 {
|
|
||||||
self.lastReportDate = Date()
|
|
||||||
self.report("paused")
|
|
||||||
self.publishMediaState("paused")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func observeItem(_ item: AVPlayerItem?) {
|
|
||||||
NotificationCenter.default.removeObserver(
|
|
||||||
self, name: .AVPlayerItemDidPlayToEndTime, object: nil
|
|
||||||
)
|
|
||||||
guard let item else { return }
|
|
||||||
NotificationCenter.default.addObserver(
|
|
||||||
self, selector: #selector(playerItemDidEnd(_:)),
|
|
||||||
name: .AVPlayerItemDidPlayToEndTime, object: item
|
|
||||||
)
|
|
||||||
statusCancellable = item.publisher(for: \.status).sink { [weak self, weak item] status in
|
|
||||||
Task { @MainActor in
|
|
||||||
guard let self, let item, self.player.currentItem === item else { return }
|
|
||||||
switch status {
|
|
||||||
case .readyToPlay:
|
|
||||||
self.playbackError = nil
|
|
||||||
case .failed:
|
|
||||||
self.playbackError = "This audio couldn’t play on your iPhone. Try another track or an MP3/AAC station."
|
|
||||||
logger.log("item failed: \(String(describing: item.error), privacy: .public)")
|
|
||||||
default: break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@objc private func playerItemDidEnd(_ notification: Notification) {
|
|
||||||
lastReportDate = Date()
|
|
||||||
report("stopped")
|
|
||||||
publishMediaState("stopped")
|
|
||||||
Task { [weak self] in
|
|
||||||
guard let self else { return }
|
|
||||||
try? await self.api.control(room: self.room ?? "default", action: "stop")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Reporting + publishing
|
|
||||||
|
|
||||||
private func report(_ state: String, trackID: String? = nil) {
|
|
||||||
let room = room ?? "default"
|
|
||||||
let position = currentTime.isFinite ? currentTime : 0
|
|
||||||
var durationValue: Double?
|
|
||||||
if duration.isFinite, duration > 0 {
|
|
||||||
durationValue = duration
|
|
||||||
}
|
|
||||||
let resolvedTrackID = trackID ?? currentTrackID
|
|
||||||
Task { [weak self] in
|
|
||||||
guard let self else { return }
|
|
||||||
do {
|
|
||||||
try await self.api.report(room: room, state: state, trackID: resolvedTrackID, position: position, duration: durationValue)
|
|
||||||
} catch {
|
|
||||||
logger.log("report failed: \(String(describing: error), privacy: .public)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private func publishMediaState(_ state: String, trackID: String? = nil) {
|
|
||||||
guard let session else { return }
|
|
||||||
// Capture the exact room and generation before spawning so a delayed
|
|
||||||
// publish from an old lifecycle cannot escape into a new conversation
|
|
||||||
// room (Finding 10).
|
|
||||||
let targetRoom = session.room
|
|
||||||
let generation = session.sessionGeneration
|
|
||||||
let pollGen = pollGeneration
|
|
||||||
let resolvedTrackID = trackID ?? currentTrackID
|
|
||||||
Task { [weak self] in
|
|
||||||
guard let self, self.pollGeneration == pollGen else { return }
|
|
||||||
guard session.sessionGeneration == generation, session.room === targetRoom else { return }
|
|
||||||
do {
|
|
||||||
try await session.controlChannel.sendMediaState(state, trackID: resolvedTrackID, room: targetRoom)
|
|
||||||
} catch {
|
|
||||||
logger.log("media_state publish failed: \(String(describing: error), privacy: .public)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Play requests (search results, choices, library)
|
|
||||||
|
|
||||||
func playVideo(videoID: String?, title: String?, channel: String?) async {
|
|
||||||
let targetRoom = room ?? "default"
|
|
||||||
let generation = pollGeneration
|
|
||||||
playbackError = nil
|
|
||||||
do {
|
|
||||||
try await api.play(room: targetRoom, videoID: videoID, title: title, channel: channel)
|
|
||||||
logger.log("play requested video_id=\(videoID ?? "nil", privacy: .public) room=\(targetRoom, privacy: .public)")
|
|
||||||
} catch {
|
|
||||||
guard generation == pollGeneration else { return }
|
|
||||||
playbackError = "Couldn’t play this song. Check your connection and try again."
|
|
||||||
logger.log("play failed: \(String(describing: error), privacy: .public)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func playTrack(id: String) async {
|
|
||||||
let targetRoom = room ?? "default"
|
|
||||||
let generation = pollGeneration
|
|
||||||
playbackError = nil
|
|
||||||
do {
|
|
||||||
try await api.play(room: targetRoom, trackID: id)
|
|
||||||
logger.log("play requested track_id=\(id, privacy: .public) room=\(targetRoom, privacy: .public)")
|
|
||||||
} catch {
|
|
||||||
guard generation == pollGeneration else { return }
|
|
||||||
playbackError = "Couldn’t play this track. Check your connection and try again."
|
|
||||||
logger.log("play failed: \(String(describing: error), privacy: .public)")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Transport controls (parity with web UI play/pause/seek/stop)
|
|
||||||
|
|
||||||
func togglePlayPause() async {
|
|
||||||
guard currentTrackID != nil, loadedAudioPath != nil else { return }
|
|
||||||
if player.rate.isZero {
|
|
||||||
// Explicit user play: clear any resume suppression (the user is
|
|
||||||
// deliberately overriding the no-resume interruption state).
|
|
||||||
resumeSuppressed = false
|
|
||||||
// The rate observer reports + publishes the actual transition;
|
|
||||||
// the control call updates server desired state.
|
|
||||||
player.play()
|
|
||||||
await sendControl("resume")
|
|
||||||
} else {
|
|
||||||
lastReportDate = Date()
|
|
||||||
report("paused")
|
|
||||||
publishMediaState("paused")
|
|
||||||
player.pause()
|
|
||||||
await sendControl("pause")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func seek(by delta: Double) async {
|
|
||||||
guard loadedAudioPath != nil else { return }
|
|
||||||
let target = max(0, min(duration > 0 ? duration - 0.5 : .infinity, currentTime + delta))
|
|
||||||
if target.isFinite {
|
|
||||||
seek(to: target)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func seek(to position: Double) {
|
|
||||||
guard position.isFinite, position >= 0 else { return }
|
|
||||||
player.seek(to: CMTime(seconds: position, preferredTimescale: 600))
|
|
||||||
currentTime = position
|
|
||||||
report(player.rate.isZero ? "paused" : "playing")
|
|
||||||
}
|
|
||||||
|
|
||||||
func stop() async {
|
|
||||||
// Capture the track ID before clearing so the async report/publish
|
|
||||||
// tasks can read a non-nil value even after clearMedia() runs.
|
|
||||||
let trackID = currentTrackID
|
|
||||||
report("stopped", trackID: trackID)
|
|
||||||
clearMedia()
|
|
||||||
await sendControl("stop")
|
|
||||||
publishMediaState("stopped", trackID: trackID)
|
|
||||||
}
|
|
||||||
|
|
||||||
private func clearMedia() {
|
|
||||||
playbackError = nil
|
|
||||||
currentTrackID = nil
|
|
||||||
track = nil
|
|
||||||
loadedAudioPath = nil
|
|
||||||
// Remove the loader mapping for the asset being cleared so the
|
|
||||||
// resource loader's internal dictionaries stay bounded (Item 7).
|
|
||||||
clearCurrentAsset()
|
|
||||||
observeItem(nil)
|
|
||||||
player.replaceCurrentItem(with: nil)
|
|
||||||
currentTime = 0
|
|
||||||
duration = 0
|
|
||||||
stateError = nil
|
|
||||||
}
|
|
||||||
|
|
||||||
private func sendControl(_ action: String) async {
|
|
||||||
let generation = pollGeneration
|
|
||||||
do {
|
|
||||||
try await api.control(room: room ?? "default", action: action)
|
|
||||||
guard generation == pollGeneration else { return }
|
|
||||||
playbackError = nil
|
|
||||||
} catch {
|
|
||||||
guard generation == pollGeneration else { return }
|
|
||||||
playbackError = "Couldn’t update playback on the server. Check your connection and try again."
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Volume
|
|
||||||
|
|
||||||
func applyVolume() {
|
|
||||||
player.volume = Float(settings.volume)
|
|
||||||
}
|
|
||||||
|
|
||||||
func setVolume(_ volume: Double) {
|
|
||||||
settings.volume = volume
|
|
||||||
player.volume = Float(volume)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,238 +0,0 @@
|
|||||||
import AVFoundation
|
|
||||||
import Foundation
|
|
||||||
import UniformTypeIdentifiers
|
|
||||||
import os
|
|
||||||
|
|
||||||
private let logger = Logger(subsystem: "dev.shane.voice", category: "media")
|
|
||||||
|
|
||||||
/// Proxies {base}/media/* bytes to AVPlayer over the pinned-TLS session, since
|
|
||||||
/// AVPlayer cannot trust the self-signed LAN cert. AVPlayer loads https natively
|
|
||||||
/// (ignoring registered URLProtocol subclasses), so the asset uses a custom
|
|
||||||
/// `pinned-media` scheme and this delegate answers every data request by
|
|
||||||
/// fetching the real https URL through TLSPinning.session(), forwarding Range.
|
|
||||||
///
|
|
||||||
/// **Lifecycle invariants (Finding 11 / Iteration 06 Item 7):**
|
|
||||||
/// - Tasks are keyed by `ObjectIdentifier(loadingRequest)` directly, not a
|
|
||||||
/// derived hash, so two distinct requests can never collide.
|
|
||||||
/// - All mutable state (`loaderURLs`, `activeTasks`) is accessed only on the
|
|
||||||
/// main queue: the AVAssetResourceLoader delegate is set with `queue: .main`
|
|
||||||
/// and the pinned URLSession uses `delegateQueue: .main`, so both the
|
|
||||||
/// resource-loading callbacks and the data-task completion handlers run on
|
|
||||||
/// the same serialization domain. This is enforced in code, not by comment.
|
|
||||||
/// - For HTTP 206, `contentLength` is set only when `Content-Range` provides a
|
|
||||||
/// known total; it never falls back to the range's `expectedContentLength`.
|
|
||||||
/// - For HTTP 200, `expectedContentLength` is already the full resource size.
|
|
||||||
/// - Cleanup (task removal) occurs on success, error, cancellation, and item
|
|
||||||
/// replacement via `removeLoader`.
|
|
||||||
final class PinnedMediaResourceLoader: NSObject, AVAssetResourceLoaderDelegate {
|
|
||||||
|
|
||||||
static let scheme = "pinned-media"
|
|
||||||
|
|
||||||
private let base: URL
|
|
||||||
/// Maps each asset's resource loader to the real https URL it proxies.
|
|
||||||
/// Accessed only on the main queue (the delegate queue).
|
|
||||||
private var loaderURLs: [ObjectIdentifier: URL] = [:]
|
|
||||||
/// Maps each loading request's identity to its upstream data task so
|
|
||||||
/// cancellation can cancel the in-flight download. Keyed directly by
|
|
||||||
/// `ObjectIdentifier(loadingRequest)` — no hash derivation, no collisions.
|
|
||||||
private var activeTasks: [ObjectIdentifier: URLSessionDataTask] = [:]
|
|
||||||
/// Shared pinned session, created once and invalidated on deinit.
|
|
||||||
/// Uses `delegateQueue: .main` so all completion handlers (and therefore
|
|
||||||
/// all mutations of `activeTasks`) run on the main queue — the same
|
|
||||||
/// serialization domain as the AVAssetResourceLoader delegate callbacks
|
|
||||||
/// (which are also dispatched to `.main`). This makes map access truly
|
|
||||||
/// serialized in code, not just by convention.
|
|
||||||
private lazy var pinnedSession: URLSession = {
|
|
||||||
let config = URLSessionConfiguration.default
|
|
||||||
config.timeoutIntervalForRequest = 30
|
|
||||||
return URLSession(configuration: config, delegate: PinnedTrustDelegate(), delegateQueue: .main)
|
|
||||||
}()
|
|
||||||
|
|
||||||
init(base: URL) {
|
|
||||||
self.base = base
|
|
||||||
}
|
|
||||||
|
|
||||||
deinit {
|
|
||||||
pinnedSession.invalidateAndCancel()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Maps a media URL (absolute https or /media/... path) to the pinned-media
|
|
||||||
/// scheme so AVPlayer routes it through this delegate instead of loading
|
|
||||||
/// https natively. Built as a string because URLComponents.url returns nil
|
|
||||||
/// for schemes not in Foundation's known-scheme list.
|
|
||||||
func rewrite(_ url: URL) -> URL {
|
|
||||||
let absolute: String
|
|
||||||
if let scheme = url.scheme?.lowercased(), scheme.hasPrefix("http") {
|
|
||||||
var c = URLComponents(url: url, resolvingAgainstBaseURL: false)!
|
|
||||||
c.scheme = Self.scheme
|
|
||||||
return c.url!
|
|
||||||
} else {
|
|
||||||
let path = url.absoluteString.hasPrefix("/") ? String(url.absoluteString.dropFirst()) : url.absoluteString
|
|
||||||
guard let host = base.host, !host.isEmpty else { return url }
|
|
||||||
let port = base.port.map { ":\($0)" } ?? ""
|
|
||||||
absolute = "\(Self.scheme)://\(host)\(port)/\(path)"
|
|
||||||
}
|
|
||||||
return URL(string: absolute) ?? url
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Maps a pinned-media URL back to the real https URL it proxies.
|
|
||||||
func resolve(_ url: URL) -> URL {
|
|
||||||
guard let scheme = url.scheme?.lowercased(), scheme == Self.scheme else { return url }
|
|
||||||
var c = URLComponents(url: url, resolvingAgainstBaseURL: false)!
|
|
||||||
c.scheme = "https"
|
|
||||||
return c.url ?? url
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Creates an AVURLAsset that routes through this loader's delegate. The
|
|
||||||
/// mapping from the asset's resource loader to the real URL is recorded so
|
|
||||||
/// `resourceLoader(_:shouldWaitForLoadingOfRequestedResource:)` can resolve
|
|
||||||
/// it. All mutations happen on the main queue (the delegate queue).
|
|
||||||
func assetWithPinnedURL(_ pinned: URL) -> AVURLAsset {
|
|
||||||
let asset = AVURLAsset(url: pinned)
|
|
||||||
let realURL = resolve(pinned)
|
|
||||||
// The delegate queue is .main, so this runs on the main thread.
|
|
||||||
loaderURLs[ObjectIdentifier(asset.resourceLoader)] = realURL
|
|
||||||
asset.resourceLoader.setDelegate(self, queue: .main)
|
|
||||||
return asset
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Removes the loader mapping and any in-flight tasks for an asset that is
|
|
||||||
/// no longer needed. Called by MediaPlayerController when replacing or
|
|
||||||
/// clearing media items to bound the dictionaries (Finding 11).
|
|
||||||
func removeLoader(_ asset: AVURLAsset) {
|
|
||||||
let key = ObjectIdentifier(asset.resourceLoader)
|
|
||||||
loaderURLs[key] = nil
|
|
||||||
// Cancel any in-flight tasks. URLSessionDataTask has no isCancelled
|
|
||||||
// property; calling cancel() on an already-cancelled task is a no-op.
|
|
||||||
for (_, task) in activeTasks {
|
|
||||||
task.cancel()
|
|
||||||
}
|
|
||||||
activeTasks.removeAll()
|
|
||||||
}
|
|
||||||
|
|
||||||
func resourceLoader(
|
|
||||||
_ resourceLoader: AVAssetResourceLoader,
|
|
||||||
shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest
|
|
||||||
) -> Bool {
|
|
||||||
// All state access is on the main queue (the delegate queue).
|
|
||||||
guard loadingRequest.dataRequest != nil else { return false }
|
|
||||||
guard let url = loaderURLs[ObjectIdentifier(resourceLoader)] else {
|
|
||||||
logger.log("no resolved URL for resource loader")
|
|
||||||
loadingRequest.finishLoading(with: URLError(.badURL))
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
fetchChunk(loadingRequest, url: url)
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Return bounded chunks promptly instead of buffering an entire song before
|
|
||||||
/// AVPlayer receives its first usable bytes. The same loading request stays
|
|
||||||
/// open until its requested range (or EOF) has been delivered.
|
|
||||||
private func fetchChunk(_ loadingRequest: AVAssetResourceLoadingRequest, url: URL) {
|
|
||||||
guard !loadingRequest.isCancelled, let dataRequest = loadingRequest.dataRequest else { return }
|
|
||||||
var upstream = URLRequest(url: url)
|
|
||||||
let offset = max(dataRequest.requestedOffset, dataRequest.currentOffset)
|
|
||||||
let length = Int64(dataRequest.requestedLength)
|
|
||||||
let requestedEnd = dataRequest.requestedOffset + length
|
|
||||||
let end = dataRequest.requestsAllDataToEndOfResource || length == 0
|
|
||||||
? offset + 262_143 : min(requestedEnd - 1, offset + 262_143)
|
|
||||||
upstream.setValue("bytes=\(offset)-\(end)", forHTTPHeaderField: "Range")
|
|
||||||
logger.log("fetching \(url.absoluteString, privacy: .public) range=\(upstream.value(forHTTPHeaderField: "Range") ?? "-", privacy: .public)")
|
|
||||||
|
|
||||||
// Key the task by the loading request's identity directly — no hash
|
|
||||||
// derivation, so two distinct requests can never collide.
|
|
||||||
let requestKey = ObjectIdentifier(loadingRequest)
|
|
||||||
let task = pinnedSession.dataTask(with: upstream) { [weak self] data, response, error in
|
|
||||||
guard let self else { return }
|
|
||||||
// Cleanup: remove the task from the active map regardless of outcome.
|
|
||||||
// Runs on .main (the session's delegateQueue), same domain as all
|
|
||||||
// other map mutations.
|
|
||||||
self.activeTasks[requestKey] = nil
|
|
||||||
guard !loadingRequest.isCancelled else { return }
|
|
||||||
|
|
||||||
if let error {
|
|
||||||
logger.log("fetch failed \(url.absoluteString, privacy: .public): \(error.localizedDescription, privacy: .public)")
|
|
||||||
loadingRequest.finishLoading(with: error)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
guard let http = response as? HTTPURLResponse else {
|
|
||||||
loadingRequest.finishLoading(with: URLError(.badServerResponse))
|
|
||||||
return
|
|
||||||
}
|
|
||||||
// Reject non-200/206 responses (404, 500, etc.) instead of feeding
|
|
||||||
// error JSON into AVPlayer as media bytes.
|
|
||||||
guard http.statusCode == 200 || http.statusCode == 206 else {
|
|
||||||
logger.log("unexpected status \(http.statusCode) for \(url.absoluteString, privacy: .public)")
|
|
||||||
loadingRequest.finishLoading(with: URLError(.badServerResponse))
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if let info = loadingRequest.contentInformationRequest {
|
|
||||||
// AVFoundation requires a UTI, not the HTTP MIME string.
|
|
||||||
info.contentType = http.mimeType.flatMap { UTType(mimeType: $0)?.identifier }
|
|
||||||
// Content-length handling (Iteration 05 Item 7):
|
|
||||||
// - For 206: set full content length ONLY when Content-Range
|
|
||||||
// provides a known total. Never fall back to the range's
|
|
||||||
// expectedContentLength (that would tell AVPlayer the total
|
|
||||||
// resource is only as large as this one range).
|
|
||||||
// - For 200: expectedContentLength is already the full size.
|
|
||||||
if http.statusCode == 206 {
|
|
||||||
if let total = Self.parseTotalLength(http), total >= 0 {
|
|
||||||
info.contentLength = total
|
|
||||||
}
|
|
||||||
// If Content-Range has no known total (e.g. "bytes 0-1023/*"),
|
|
||||||
// do NOT set contentLength — AVPlayer will handle it as
|
|
||||||
// unknown-length, which is correct for a range response.
|
|
||||||
} else {
|
|
||||||
// 200: expectedContentLength is the full resource size.
|
|
||||||
if http.expectedContentLength >= 0 {
|
|
||||||
info.contentLength = Int64(http.expectedContentLength)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
info.isByteRangeAccessSupported = true
|
|
||||||
}
|
|
||||||
if let data, !data.isEmpty {
|
|
||||||
dataRequest.respond(with: data)
|
|
||||||
let nextOffset = offset + Int64(data.count)
|
|
||||||
let total = Self.parseTotalLength(http)
|
|
||||||
let reachedEOF = total.map { nextOffset >= $0 } ?? (nextOffset <= end)
|
|
||||||
let needsMore = dataRequest.requestsAllDataToEndOfResource || nextOffset < requestedEnd
|
|
||||||
if http.statusCode == 206, !reachedEOF, needsMore {
|
|
||||||
self.fetchChunk(loadingRequest, url: url)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
|
||||||
loadingRequest.finishLoading()
|
|
||||||
}
|
|
||||||
activeTasks[requestKey] = task
|
|
||||||
task.resume()
|
|
||||||
}
|
|
||||||
|
|
||||||
func resourceLoader(
|
|
||||||
_ resourceLoader: AVAssetResourceLoader,
|
|
||||||
didCancel loadingRequest: AVAssetResourceLoadingRequest
|
|
||||||
) {
|
|
||||||
// Cancel the upstream data task so a seek or item swap doesn't keep
|
|
||||||
// downloading cancelled ranges. Keyed by ObjectIdentifier directly.
|
|
||||||
let key = ObjectIdentifier(loadingRequest)
|
|
||||||
activeTasks[key]?.cancel()
|
|
||||||
activeTasks[key] = nil
|
|
||||||
logger.log("resource loader cancelled request, upstream task cancelled")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parses the total resource length from an HTTP 206 response's
|
|
||||||
/// `Content-Range` header (e.g. "bytes 0-1023/4096" → 4096). Returns nil
|
|
||||||
/// for non-206 responses, missing headers, unknown totals ("*"), or
|
|
||||||
/// malformed values.
|
|
||||||
static func parseTotalLength(_ response: HTTPURLResponse) -> Int64? {
|
|
||||||
if response.statusCode == 206,
|
|
||||||
let contentRange = response.value(forHTTPHeaderField: "Content-Range") {
|
|
||||||
// Format: "bytes <start>-<end>/<total>" or "bytes <start>-<end>/*"
|
|
||||||
let afterSlash = contentRange.split(separator: "/").last?.trimmingCharacters(in: .whitespaces)
|
|
||||||
if let afterSlash, afterSlash != "*", let total = Int64(afterSlash) {
|
|
||||||
return total
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,125 +0,0 @@
|
|||||||
import Foundation
|
|
||||||
import Testing
|
|
||||||
@testable import Voice
|
|
||||||
|
|
||||||
/// Minimal HTTP/1.1 stub built on BSD sockets that answers every request with a
|
|
||||||
/// successful JSON object body (mirroring the real media server's
|
|
||||||
/// `{"ok": true, ...}` shape). Used to prove MediaAPI no-response POSTs succeed
|
|
||||||
/// against an object body instead of throwing decodeFailed.
|
|
||||||
@MainActor
|
|
||||||
final class MediaAPIStubServer {
|
|
||||||
private(set) var url: URL!
|
|
||||||
nonisolated(unsafe) private var serverFD: Int32 = -1
|
|
||||||
nonisolated(unsafe) private var acceptThread: Thread?
|
|
||||||
nonisolated(unsafe) private var running = false
|
|
||||||
nonisolated(unsafe) private var lastPath: String?
|
|
||||||
|
|
||||||
func start() throws {
|
|
||||||
let fd = socket(AF_INET, SOCK_STREAM, 0)
|
|
||||||
guard fd >= 0 else { throw MediaAPIError.decodeFailed }
|
|
||||||
var yes: Int32 = 1
|
|
||||||
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, socklen_t(MemoryLayout<Int32>.size))
|
|
||||||
|
|
||||||
var addr = sockaddr_in()
|
|
||||||
addr.sin_family = sa_family_t(AF_INET)
|
|
||||||
addr.sin_port = in_port_t(0).bigEndian
|
|
||||||
addr.sin_addr = in_addr(s_addr: inet_addr("127.0.0.1"))
|
|
||||||
let bindResult = withUnsafePointer(to: &addr) { ptr in
|
|
||||||
ptr.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
|
|
||||||
bind(fd, sa, socklen_t(MemoryLayout<sockaddr_in>.size))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
guard bindResult == 0 else { close(fd); throw MediaAPIError.decodeFailed }
|
|
||||||
guard listen(fd, 8) == 0 else { close(fd); throw MediaAPIError.decodeFailed }
|
|
||||||
|
|
||||||
var bound = sockaddr_in()
|
|
||||||
var len = socklen_t(MemoryLayout<sockaddr_in>.size)
|
|
||||||
withUnsafeMutablePointer(to: &bound) { ptr in
|
|
||||||
_ = ptr.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
|
|
||||||
getsockname(fd, sa, &len)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
let port = Int(UInt16(bigEndian: bound.sin_port))
|
|
||||||
serverFD = fd
|
|
||||||
running = true
|
|
||||||
url = URL(string: "http://127.0.0.1:\(port)")!
|
|
||||||
|
|
||||||
let thread = Thread { [weak self] in
|
|
||||||
self?.acceptLoop()
|
|
||||||
}
|
|
||||||
acceptThread = thread
|
|
||||||
thread.start()
|
|
||||||
}
|
|
||||||
|
|
||||||
func stop() {
|
|
||||||
running = false
|
|
||||||
if serverFD >= 0 { close(serverFD) }
|
|
||||||
serverFD = -1
|
|
||||||
acceptThread?.cancel()
|
|
||||||
}
|
|
||||||
|
|
||||||
var lastRequestPath: String? { lastPath }
|
|
||||||
|
|
||||||
nonisolated private func acceptLoop() {
|
|
||||||
while running {
|
|
||||||
var clientAddr = sockaddr_in()
|
|
||||||
var len = socklen_t(MemoryLayout<sockaddr_in>.size)
|
|
||||||
let clientFD = withUnsafeMutablePointer(to: &clientAddr) { ptr in
|
|
||||||
ptr.withMemoryRebound(to: sockaddr.self, capacity: 1) { sa in
|
|
||||||
accept(serverFD, sa, &len)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
guard clientFD >= 0 else { continue }
|
|
||||||
handleClient(clientFD)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
nonisolated private func handleClient(_ clientFD: Int32) {
|
|
||||||
defer { close(clientFD) }
|
|
||||||
var buffer = [UInt8](repeating: 0, count: 4096)
|
|
||||||
let bytesRead = read(clientFD, &buffer, buffer.count)
|
|
||||||
guard bytesRead > 0 else { return }
|
|
||||||
guard let request = String(bytes: buffer[0..<bytesRead], encoding: .utf8) else { return }
|
|
||||||
|
|
||||||
var path = ""
|
|
||||||
if let firstLine = request.split(separator: "\r\n").first {
|
|
||||||
let parts = firstLine.split(separator: " ")
|
|
||||||
if parts.count >= 2 { path = String(parts[1]) }
|
|
||||||
}
|
|
||||||
lastPath = path
|
|
||||||
|
|
||||||
let body = #"{"ok": true, "room": "test", "desired": "stopped"}"#
|
|
||||||
let response = """
|
|
||||||
HTTP/1.1 200 OK\r
|
|
||||||
Content-Type: application/json\r
|
|
||||||
Content-Length: \(body.utf8.count)\r
|
|
||||||
Connection: close\r
|
|
||||||
|
|
||||||
\(body)
|
|
||||||
"""
|
|
||||||
_ = response.withCString { cstr in
|
|
||||||
write(clientFD, cstr, response.utf8.count)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@MainActor
|
|
||||||
struct MediaAPITests {
|
|
||||||
@Test("successful JSON object response does not throw decodeFailed")
|
|
||||||
func successfulObjectResponse() async throws {
|
|
||||||
let stub = MediaAPIStubServer()
|
|
||||||
try stub.start()
|
|
||||||
defer { stub.stop() }
|
|
||||||
|
|
||||||
// Bypass the https-only parse path: build the config directly.
|
|
||||||
let base = stub.url!
|
|
||||||
let config = ServerConfig(baseURL: base, livekitURL: base)
|
|
||||||
let api = MediaAPI(config: config)
|
|
||||||
|
|
||||||
try await api.control(room: "test", action: "pause")
|
|
||||||
try await api.report(room: "test", state: "paused", trackID: nil, position: 0, duration: nil)
|
|
||||||
try await api.play(room: "test", videoID: nil, title: nil, channel: nil, trackID: nil, startPaused: nil)
|
|
||||||
|
|
||||||
#expect(stub.lastRequestPath == "/media/api/play")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,89 +0,0 @@
|
|||||||
import Foundation
|
|
||||||
import Testing
|
|
||||||
@testable import Voice
|
|
||||||
|
|
||||||
private final class PlaybackProtocol: URLProtocol, @unchecked Sendable {
|
|
||||||
static let lock = NSLock()
|
|
||||||
nonisolated(unsafe) static var status = 200
|
|
||||||
nonisolated(unsafe) static var requests: [URLRequest] = []
|
|
||||||
override class func canInit(with request: URLRequest) -> Bool { true }
|
|
||||||
override class func canonicalRequest(for request: URLRequest) -> URLRequest { request }
|
|
||||||
override func startLoading() {
|
|
||||||
Self.lock.lock()
|
|
||||||
Self.requests.append(request)
|
|
||||||
let code = Self.status
|
|
||||||
Self.lock.unlock()
|
|
||||||
let response = HTTPURLResponse(url: request.url!, statusCode: code, httpVersion: nil, headerFields: ["Content-Type": "application/json"])!
|
|
||||||
client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
|
|
||||||
client?.urlProtocol(self, didLoad: Data("{\"ok\":true}".utf8))
|
|
||||||
client?.urlProtocolDidFinishLoading(self)
|
|
||||||
}
|
|
||||||
override func stopLoading() {}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Suite(.serialized)
|
|
||||||
@MainActor struct MediaPlaybackTests {
|
|
||||||
private func controller(status: Int = 200) -> MediaPlayerController {
|
|
||||||
PlaybackProtocol.lock.withLock {
|
|
||||||
PlaybackProtocol.status = status
|
|
||||||
PlaybackProtocol.requests = []
|
|
||||||
}
|
|
||||||
let configuration = URLSessionConfiguration.ephemeral
|
|
||||||
configuration.protocolClasses = [PlaybackProtocol.self]
|
|
||||||
let url = URL(string: "https://media.test")!
|
|
||||||
let api = MediaAPI(config: ServerConfig(baseURL: url, livekitURL: url), session: URLSession(configuration: configuration))
|
|
||||||
return MediaPlayerController(api: api, settings: SettingsStore())
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("standalone playback sends stop to the default server room")
|
|
||||||
func standaloneStop() async throws {
|
|
||||||
let player = controller()
|
|
||||||
await player.stop()
|
|
||||||
let calls = PlaybackProtocol.lock.withLock { PlaybackProtocol.requests }
|
|
||||||
let stop = try #require(calls.first { $0.url?.path == "/media/api/control" })
|
|
||||||
// URLSession may expose the encoded body as an InputStream to URLProtocol.
|
|
||||||
let body: Data
|
|
||||||
if let data = stop.httpBody { body = data }
|
|
||||||
else {
|
|
||||||
let stream = try #require(stop.httpBodyStream)
|
|
||||||
stream.open()
|
|
||||||
defer { stream.close() }
|
|
||||||
var data = Data()
|
|
||||||
var buffer = [UInt8](repeating: 0, count: 1024)
|
|
||||||
while stream.hasBytesAvailable {
|
|
||||||
let count = stream.read(&buffer, maxLength: buffer.count)
|
|
||||||
if count <= 0 { break }
|
|
||||||
data.append(contentsOf: buffer.prefix(count))
|
|
||||||
}
|
|
||||||
body = data
|
|
||||||
}
|
|
||||||
let json = try #require(JSONSerialization.jsonObject(with: body) as? [String: Any])
|
|
||||||
#expect(json["room"] as? String == "default")
|
|
||||||
#expect(json["action"] as? String == "stop")
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("failed play requests are visible and a successful retry clears the error")
|
|
||||||
func failureAndRetry() async {
|
|
||||||
let player = controller(status: 503)
|
|
||||||
await player.playTrack(id: "song")
|
|
||||||
#expect(player.playbackError != nil)
|
|
||||||
PlaybackProtocol.lock.withLock { PlaybackProtocol.status = 200 }
|
|
||||||
await player.playTrack(id: "song")
|
|
||||||
#expect(player.playbackError == nil)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("metadata refreshes when a download becomes ready")
|
|
||||||
func readyMetadata() {
|
|
||||||
let player = controller()
|
|
||||||
func state(_ status: String) -> MediaState {
|
|
||||||
MediaState(room: "default", track: MediaTrack(id: "song", kind: "youtube", videoID: nil, title: status, channel: nil, duration: 180, status: status, error: nil, audioURL: nil, artURL: nil), desired: "paused", reported: nil, seekHint: nil, choices: [], pending: [], version: 1)
|
|
||||||
}
|
|
||||||
player.reconcile(state("downloading"))
|
|
||||||
player.reconcile(state("ready"))
|
|
||||||
#expect(player.track?.status == "ready")
|
|
||||||
#expect(player.track?.title == "ready")
|
|
||||||
player.resetScope()
|
|
||||||
#expect(player.track == nil)
|
|
||||||
#expect(player.player.currentItem == nil)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,124 +0,0 @@
|
|||||||
import Foundation
|
|
||||||
import Testing
|
|
||||||
@testable import Voice
|
|
||||||
|
|
||||||
@MainActor
|
|
||||||
struct MediaReconcileTests {
|
|
||||||
private func makeController() -> MediaPlayerController {
|
|
||||||
let config = ServerConfig(
|
|
||||||
baseURL: URL(string: "http://127.0.0.1:9")!,
|
|
||||||
livekitURL: URL(string: "ws://127.0.0.1:9")!
|
|
||||||
)
|
|
||||||
let api = MediaAPI(config: config)
|
|
||||||
return MediaPlayerController(api: api, settings: SettingsStore())
|
|
||||||
}
|
|
||||||
|
|
||||||
private func makeState(
|
|
||||||
desired: String? = "stopped",
|
|
||||||
trackID: String? = nil,
|
|
||||||
audioURL: String? = nil
|
|
||||||
) -> MediaState {
|
|
||||||
let track: MediaTrack? = trackID != nil ? MediaTrack(
|
|
||||||
id: trackID!,
|
|
||||||
kind: nil,
|
|
||||||
videoID: nil,
|
|
||||||
title: nil,
|
|
||||||
channel: nil,
|
|
||||||
duration: nil,
|
|
||||||
status: nil,
|
|
||||||
error: nil,
|
|
||||||
audioURL: audioURL,
|
|
||||||
artURL: nil
|
|
||||||
) : nil
|
|
||||||
return MediaState(
|
|
||||||
room: "default",
|
|
||||||
track: track,
|
|
||||||
desired: desired,
|
|
||||||
reported: nil,
|
|
||||||
seekHint: nil,
|
|
||||||
choices: [],
|
|
||||||
pending: [],
|
|
||||||
version: 1
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("reconcile updates published desired to playing")
|
|
||||||
func reconcileDesiredPlaying() {
|
|
||||||
let controller = makeController()
|
|
||||||
#expect(controller.desired == "stopped")
|
|
||||||
|
|
||||||
controller.reconcile(makeState(desired: "playing"))
|
|
||||||
#expect(controller.desired == "playing")
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("reconcile updates published desired to paused")
|
|
||||||
func reconcileDesiredPaused() {
|
|
||||||
let controller = makeController()
|
|
||||||
controller.reconcile(makeState(desired: "playing"))
|
|
||||||
#expect(controller.desired == "playing")
|
|
||||||
|
|
||||||
controller.reconcile(makeState(desired: "paused"))
|
|
||||||
#expect(controller.desired == "paused")
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("reconcile updates published desired to stopped")
|
|
||||||
func reconcileDesiredStopped() {
|
|
||||||
let controller = makeController()
|
|
||||||
controller.reconcile(makeState(desired: "playing"))
|
|
||||||
#expect(controller.desired == "playing")
|
|
||||||
|
|
||||||
controller.reconcile(makeState(desired: "stopped"))
|
|
||||||
#expect(controller.desired == "stopped")
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("reconcile with nil desired defaults to stopped")
|
|
||||||
func reconcileNilDesiredDefaultsToStopped() {
|
|
||||||
let controller = makeController()
|
|
||||||
controller.reconcile(makeState(desired: "playing"))
|
|
||||||
#expect(controller.desired == "playing")
|
|
||||||
|
|
||||||
controller.reconcile(makeState(desired: nil))
|
|
||||||
#expect(controller.desired == "stopped")
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("late audio: desired=playing before audio_url, then audio_url arrives")
|
|
||||||
func lateAudioReconciliation() {
|
|
||||||
let controller = makeController()
|
|
||||||
|
|
||||||
// Phase 1: server says playing but the track has no audio_url yet
|
|
||||||
// (still downloading). The source is not wired, so play cannot start.
|
|
||||||
let stateNoURL = makeState(desired: "playing", trackID: "track-1", audioURL: nil)
|
|
||||||
controller.reconcile(stateNoURL)
|
|
||||||
#expect(controller.desired == "playing")
|
|
||||||
#expect(controller.track?.id == "track-1")
|
|
||||||
// Player rate should still be zero because no source was wired.
|
|
||||||
#expect(controller.player.rate.isZero)
|
|
||||||
|
|
||||||
// Phase 2: the same track now has an audio_url. The reconcile path
|
|
||||||
// wires the source (wireSource) and then applyDesiredState retries
|
|
||||||
// play because desired is still "playing" and rate is still zero.
|
|
||||||
let stateWithURL = makeState(desired: "playing", trackID: "track-1", audioURL: "/stream/track-1.mp3")
|
|
||||||
controller.reconcile(stateWithURL)
|
|
||||||
#expect(controller.desired == "playing")
|
|
||||||
// The source should now be wired (loadedAudioPath is set internally).
|
|
||||||
// We can't directly observe loadedAudioPath, but we can verify the
|
|
||||||
// player has a current item after wireSource ran.
|
|
||||||
#expect(controller.player.currentItem != nil)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("startDefaultPolling restarts after endConversation keeps scope")
|
|
||||||
func pollingRestartsAfterEndConversation() async {
|
|
||||||
let controller = makeController()
|
|
||||||
controller.startDefaultPolling()
|
|
||||||
await controller.endConversation()
|
|
||||||
controller.startDefaultPolling()
|
|
||||||
try? await Task.sleep(for: .milliseconds(50))
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("togglePlayPause without a loaded track is a no-op")
|
|
||||||
func toggleWithoutTrack() async {
|
|
||||||
let controller = makeController()
|
|
||||||
await controller.togglePlayPause()
|
|
||||||
#expect(controller.track == nil)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,126 +0,0 @@
|
|||||||
import Foundation
|
|
||||||
import Testing
|
|
||||||
@testable import Voice
|
|
||||||
|
|
||||||
/// Focused tests for PinnedMediaResourceLoader.parseTotalLength, covering
|
|
||||||
/// 200, 206 with known total, 206 with unknown total (*), and malformed
|
|
||||||
/// Content-Range headers (Finding 1 / 11).
|
|
||||||
struct PinnedMediaResourceLoaderTests {
|
|
||||||
|
|
||||||
private func makeResponse(status: Int, contentRange: String? = nil, expectedLength: Int64 = -1) -> HTTPURLResponse {
|
|
||||||
var headers: [String: String] = [:]
|
|
||||||
if let contentRange {
|
|
||||||
headers["Content-Range"] = contentRange
|
|
||||||
}
|
|
||||||
return HTTPURLResponse(
|
|
||||||
url: URL(string: "https://example.com/media/test.mp3")!,
|
|
||||||
statusCode: status,
|
|
||||||
httpVersion: "HTTP/1.1",
|
|
||||||
headerFields: headers
|
|
||||||
)!
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("206 with known total parses Content-Range total")
|
|
||||||
func parse206KnownTotal() {
|
|
||||||
let response = makeResponse(status: 206, contentRange: "bytes 0-1023/4096", expectedLength: 1024)
|
|
||||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == 4096)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("206 with unknown total (*) returns nil")
|
|
||||||
func parse206UnknownTotal() {
|
|
||||||
let response = makeResponse(status: 206, contentRange: "bytes 0-1023/*", expectedLength: 1024)
|
|
||||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("200 response returns nil (expectedContentLength is already the full size)")
|
|
||||||
func parse200ReturnsNil() {
|
|
||||||
let response = makeResponse(status: 200, expectedLength: 8192)
|
|
||||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("206 without Content-Range header returns nil")
|
|
||||||
func parse206NoContentRange() {
|
|
||||||
let response = makeResponse(status: 206, expectedLength: 512)
|
|
||||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("malformed Content-Range returns nil")
|
|
||||||
func parseMalformedContentRange() {
|
|
||||||
let response = makeResponse(status: 206, contentRange: "garbage", expectedLength: 100)
|
|
||||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == nil)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("206 with large total parses correctly")
|
|
||||||
func parseLargeTotal() {
|
|
||||||
let response = makeResponse(status: 206, contentRange: "bytes 1048576-2097151/1073741824", expectedLength: 1048576)
|
|
||||||
#expect(PinnedMediaResourceLoader.parseTotalLength(response) == 1_073_741_824)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("streamURL constructs endpoint without downloading")
|
|
||||||
func streamURLConstruction() {
|
|
||||||
let config = ServerConfig(
|
|
||||||
baseURL: URL(string: "https://192.168.86.2:9178")!,
|
|
||||||
livekitURL: URL(string: "ws://192.168.86.2:7880")!
|
|
||||||
)
|
|
||||||
let api = MediaAPI(config: config)
|
|
||||||
let url = api.streamURL(trackID: "track-123")
|
|
||||||
// streamURL builds under the /media/api base (MediaAPI.baseURL).
|
|
||||||
#expect(url.absoluteString == "https://192.168.86.2:9178/media/api/stream/track-123")
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Rewrite / Resolve round-trip (Item 7)
|
|
||||||
|
|
||||||
@Test("rewrite maps https URL to pinned-media scheme and resolve inverts it")
|
|
||||||
func rewriteResolveRoundTrip() {
|
|
||||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
|
||||||
let original = URL(string: "https://192.168.86.2:9178/media/track-123.mp3")!
|
|
||||||
let pinned = loader.rewrite(original)
|
|
||||||
#expect(pinned.scheme == "pinned-media")
|
|
||||||
#expect(pinned.absoluteString == "pinned-media://192.168.86.2:9178/media/track-123.mp3")
|
|
||||||
|
|
||||||
let resolved = loader.resolve(pinned)
|
|
||||||
#expect(resolved.absoluteString == original.absoluteString)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("rewrite maps path-only URL using base host and port")
|
|
||||||
func rewritePathOnly() {
|
|
||||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
|
||||||
let pathURL = URL(string: "/media/track-456.mp3")!
|
|
||||||
let pinned = loader.rewrite(pathURL)
|
|
||||||
#expect(pinned.scheme == "pinned-media")
|
|
||||||
#expect(pinned.absoluteString == "pinned-media://192.168.86.2:9178/media/track-456.mp3")
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("resolve leaves non-pinned URLs unchanged")
|
|
||||||
func resolveNonPinned() {
|
|
||||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
|
||||||
let https = URL(string: "https://example.com/foo.mp3")!
|
|
||||||
#expect(loader.resolve(https) == https)
|
|
||||||
}
|
|
||||||
|
|
||||||
// MARK: Asset lifecycle (Item 7)
|
|
||||||
|
|
||||||
@Test("assetWithPinnedURL creates asset and removeLoader clears mapping")
|
|
||||||
func assetLifecycle() {
|
|
||||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
|
||||||
let pinned = loader.rewrite(URL(string: "https://192.168.86.2:9178/media/test.mp3")!)
|
|
||||||
let asset = loader.assetWithPinnedURL(pinned)
|
|
||||||
|
|
||||||
// The asset should have a valid URL matching the pinned scheme.
|
|
||||||
#expect(asset.url.scheme == PinnedMediaResourceLoader.scheme)
|
|
||||||
|
|
||||||
// Remove the loader mapping — subsequent calls should not crash.
|
|
||||||
loader.removeLoader(asset)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test("removeLoader is idempotent (safe to call twice)")
|
|
||||||
func removeLoaderIdempotent() {
|
|
||||||
let loader = PinnedMediaResourceLoader(base: URL(string: "https://192.168.86.2:9178")!)
|
|
||||||
let pinned = loader.rewrite(URL(string: "https://192.168.86.2:9178/media/test.mp3")!)
|
|
||||||
let asset = loader.assetWithPinnedURL(pinned)
|
|
||||||
|
|
||||||
loader.removeLoader(asset)
|
|
||||||
// Second call should be a no-op, not a crash.
|
|
||||||
loader.removeLoader(asset)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,9 +1,8 @@
|
|||||||
import XCTest
|
import XCTest
|
||||||
|
|
||||||
/// Opt-in tests against the actual NAS deployment. No injected replies or media.
|
/// Opt-in tests against the actual NAS deployment. No injected replies.
|
||||||
final class VoiceLiveTests: XCTestCase {
|
final class VoiceLiveTests: XCTestCase {
|
||||||
private var app: XCUIApplication!
|
private var app: XCUIApplication!
|
||||||
private var startedMusic = false
|
|
||||||
|
|
||||||
override func record(_ issue: XCTIssue) {
|
override func record(_ issue: XCTIssue) {
|
||||||
if let app, app.state == .runningForeground {
|
if let app, app.state == .runningForeground {
|
||||||
@@ -17,20 +16,13 @@ final class VoiceLiveTests: XCTestCase {
|
|||||||
}
|
}
|
||||||
|
|
||||||
override func tearDownWithError() throws {
|
override func tearDownWithError() throws {
|
||||||
defer { app?.terminate() }
|
app?.terminate()
|
||||||
guard let app, app.state == .runningForeground else { return }
|
|
||||||
// Stop only playback initiated by this test, including assertion failures.
|
|
||||||
// Do not leave the shared default room playing between test runs.
|
|
||||||
if startedMusic, app.buttons["Stop playback"].exists {
|
|
||||||
app.buttons["Stop playback"].tap()
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
override func setUpWithError() throws {
|
override func setUpWithError() throws {
|
||||||
try XCTSkipUnless(ProcessInfo.processInfo.environment["HOPE_RUN_LIVE_TESTS"] == "1", "Requires the live Hope server")
|
try XCTSkipUnless(ProcessInfo.processInfo.environment["HOPE_RUN_LIVE_TESTS"] == "1", "Requires the live Hope server")
|
||||||
continueAfterFailure = false
|
continueAfterFailure = false
|
||||||
app = XCUIApplication()
|
app = XCUIApplication()
|
||||||
startedMusic = false
|
|
||||||
app.launchArguments = ["--ui-testing", "--reset-ui-testing", "--live-testing"]
|
app.launchArguments = ["--ui-testing", "--reset-ui-testing", "--live-testing"]
|
||||||
app.launch()
|
app.launch()
|
||||||
}
|
}
|
||||||
@@ -63,31 +55,6 @@ final class VoiceLiveTests: XCTestCase {
|
|||||||
XCTAssertTrue(app.buttons["Start chatting"].waitForExistence(timeout: 20))
|
XCTAssertTrue(app.buttons["Start chatting"].waitForExistence(timeout: 20))
|
||||||
}
|
}
|
||||||
|
|
||||||
func testLiveLibraryPlayback() {
|
|
||||||
app.tabBars.buttons["Music"].tap()
|
|
||||||
let play = app.buttons.matching(NSPredicate(format: "label BEGINSWITH 'Play Sheryl Crow'")).firstMatch
|
|
||||||
XCTAssertTrue(play.waitForExistence(timeout: 20), "Existing library must load from NAS")
|
|
||||||
startedMusic = true
|
|
||||||
play.tap()
|
|
||||||
XCTAssertTrue(app.buttons["Pause"].waitForExistence(timeout: 30))
|
|
||||||
// Elapsed media time proves AVPlayer decoded and advanced, beyond a server desired-state label.
|
|
||||||
let position = app.sliders["Playback position"]
|
|
||||||
XCTAssertTrue(position.waitForExistence(timeout: 10))
|
|
||||||
let advanced = NSPredicate { _, _ in
|
|
||||||
let value = position.value as? String ?? "0%"
|
|
||||||
return value != "0%" && value != "0" && !value.isEmpty
|
|
||||||
}
|
|
||||||
expectation(for: advanced, evaluatedWith: position)
|
|
||||||
waitForExpectations(timeout: 35)
|
|
||||||
app.buttons["Pause"].tap()
|
|
||||||
XCTAssertTrue(app.buttons["Play"].waitForExistence(timeout: 10))
|
|
||||||
app.buttons["Seek forward 10 seconds"].tap()
|
|
||||||
capture("Live music playback")
|
|
||||||
app.buttons["Stop playback"].tap()
|
|
||||||
XCTAssertFalse(app.buttons["Pause"].exists)
|
|
||||||
startedMusic = false
|
|
||||||
}
|
|
||||||
|
|
||||||
private func capture(_ name: String) {
|
private func capture(_ name: String) {
|
||||||
let attachment = XCTAttachment(screenshot: app.screenshot())
|
let attachment = XCTAttachment(screenshot: app.screenshot())
|
||||||
attachment.name = name
|
attachment.name = name
|
||||||
|
|||||||
@@ -48,29 +48,6 @@ final class VoiceUITests: XCTestCase {
|
|||||||
XCTAssertEqual(app.textFields["Server URL"].value as? String, "https://127.0.0.1:1")
|
XCTAssertEqual(app.textFields["Server URL"].value as? String, "https://127.0.0.1:1")
|
||||||
}
|
}
|
||||||
|
|
||||||
func testMusicSearchAndStationValidation() {
|
|
||||||
app.launch()
|
|
||||||
app.tabBars.buttons["Music"].tap()
|
|
||||||
XCTAssertTrue(app.buttons["Add radio station"].waitForExistence(timeout: 5))
|
|
||||||
XCTAssertTrue(app.staticTexts["Music server unavailable"].waitForExistence(timeout: 10))
|
|
||||||
app.segmentedControls.buttons["Search"].tap()
|
|
||||||
let query = app.textFields["Song, artist, lyrics…"]
|
|
||||||
query.tap()
|
|
||||||
query.typeText("Piano")
|
|
||||||
app.buttons["music.search"].tap()
|
|
||||||
XCTAssertTrue(app.staticTexts.matching(NSPredicate(format: "label CONTAINS 'connect to the server'")).firstMatch.waitForExistence(timeout: 15))
|
|
||||||
app.buttons["Add radio station"].tap()
|
|
||||||
let url = app.textFields["Audio stream URL"]
|
|
||||||
XCTAssertTrue(url.waitForExistence(timeout: 5))
|
|
||||||
url.tap()
|
|
||||||
url.typeText("not-a-url")
|
|
||||||
app.buttons["Add station"].tap()
|
|
||||||
XCTAssertTrue(app.staticTexts["Enter a complete http:// or https:// audio stream address."].exists)
|
|
||||||
app.buttons["Cancel"].tap()
|
|
||||||
XCTAssertTrue(app.buttons["Add radio station"].exists)
|
|
||||||
capture("Music search")
|
|
||||||
}
|
|
||||||
|
|
||||||
func testTranscriptClearConfirmationAndActivityDetails() {
|
func testTranscriptClearConfirmationAndActivityDetails() {
|
||||||
app.launchArguments.append("--ui-fixtures")
|
app.launchArguments.append("--ui-fixtures")
|
||||||
app.launch()
|
app.launch()
|
||||||
|
|||||||
@@ -17,27 +17,19 @@
|
|||||||
64291BBC8D0DAD47E07E94E0 /* ConversationState.swift in Sources */ = {isa = PBXBuildFile; fileRef = A0C23305A6C55A83F8617D9D /* ConversationState.swift */; };
|
64291BBC8D0DAD47E07E94E0 /* ConversationState.swift in Sources */ = {isa = PBXBuildFile; fileRef = A0C23305A6C55A83F8617D9D /* ConversationState.swift */; };
|
||||||
70E4B860147494D9657E8C01 /* VoiceLiveTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5F6A697985B36B3A1A3F2AD5 /* VoiceLiveTests.swift */; };
|
70E4B860147494D9657E8C01 /* VoiceLiveTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5F6A697985B36B3A1A3F2AD5 /* VoiceLiveTests.swift */; };
|
||||||
720C26EC163AE46713B02DD8 /* TLSPinning.swift in Sources */ = {isa = PBXBuildFile; fileRef = D9B40076934C28BA4B92D808 /* TLSPinning.swift */; };
|
720C26EC163AE46713B02DD8 /* TLSPinning.swift in Sources */ = {isa = PBXBuildFile; fileRef = D9B40076934C28BA4B92D808 /* TLSPinning.swift */; };
|
||||||
72F3B4AEC214521015200583 /* MediaReconcileTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 09FB71841B3D72CD1D69E47A /* MediaReconcileTests.swift */; };
|
|
||||||
73DA73811F8AEB2AFE340D10 /* MediaPlayerController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3964F565A6C6A06399EBA4DF /* MediaPlayerController.swift */; };
|
|
||||||
89B47754E5C39418BFD839E6 /* TranscriptHistory.swift in Sources */ = {isa = PBXBuildFile; fileRef = 68A45228EF59186CFC219F2E /* TranscriptHistory.swift */; };
|
89B47754E5C39418BFD839E6 /* TranscriptHistory.swift in Sources */ = {isa = PBXBuildFile; fileRef = 68A45228EF59186CFC219F2E /* TranscriptHistory.swift */; };
|
||||||
8DE1363CC0119B5AF9E4E117 /* ConnectionTestView.swift in Sources */ = {isa = PBXBuildFile; fileRef = AC4B9073EE8EAB51F8C55760 /* ConnectionTestView.swift */; };
|
8DE1363CC0119B5AF9E4E117 /* ConnectionTestView.swift in Sources */ = {isa = PBXBuildFile; fileRef = AC4B9073EE8EAB51F8C55760 /* ConnectionTestView.swift */; };
|
||||||
943BCE971A01F220FAE9E250 /* TaskPanelView.swift in Sources */ = {isa = PBXBuildFile; fileRef = E85BACA267ADD462306C9920 /* TaskPanelView.swift */; };
|
943BCE971A01F220FAE9E250 /* TaskPanelView.swift in Sources */ = {isa = PBXBuildFile; fileRef = E85BACA267ADD462306C9920 /* TaskPanelView.swift */; };
|
||||||
98BA1A0F92DB7B98A8BA7814 /* SettingsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 48F76881B4AE51DEE74511CC /* SettingsView.swift */; };
|
98BA1A0F92DB7B98A8BA7814 /* SettingsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 48F76881B4AE51DEE74511CC /* SettingsView.swift */; };
|
||||||
A5868238CF4958D6D0C5908E /* TaskStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = C0B60FBBB9C47BB9ADACA7CA /* TaskStore.swift */; };
|
A5868238CF4958D6D0C5908E /* TaskStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = C0B60FBBB9C47BB9ADACA7CA /* TaskStore.swift */; };
|
||||||
A86EFB492F40C26D4AA315E9 /* PinnedMediaResourceLoader.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6DC3AC57ED58FC517E212F19 /* PinnedMediaResourceLoader.swift */; };
|
|
||||||
A94F6C509E3FE215115B2C58 /* SettingsStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6D0D153557BB1C756ED04C61 /* SettingsStore.swift */; };
|
A94F6C509E3FE215115B2C58 /* SettingsStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6D0D153557BB1C756ED04C61 /* SettingsStore.swift */; };
|
||||||
AF719B1668ADA929AA0237BF /* MediaPlaybackTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E492DE95BFFF99D7AC1E8970 /* MediaPlaybackTests.swift */; };
|
|
||||||
B94FAFA796A9AF6011B03DEC /* PinnedMediaResourceLoaderTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8A20F8AFF1AAC5A944173549 /* PinnedMediaResourceLoaderTests.swift */; };
|
|
||||||
BCAB602C722D74F6C96C5E36 /* ServerReplacementCoordinatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */; };
|
BCAB602C722D74F6C96C5E36 /* ServerReplacementCoordinatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */; };
|
||||||
C00D1878C2AB3D8B6F9562B6 /* AgentMonitor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0FC167F6658CFEC9546A7E9F /* AgentMonitor.swift */; };
|
C00D1878C2AB3D8B6F9562B6 /* AgentMonitor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0FC167F6658CFEC9546A7E9F /* AgentMonitor.swift */; };
|
||||||
C956D0FDFAC43925979E4077 /* MediaAPI.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9191AD60FF96E44F191864E3 /* MediaAPI.swift */; };
|
|
||||||
CDE38FE9679472D770C601E6 /* VoiceSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = EB58EEF9490F81FB2E1CE31C /* VoiceSession.swift */; };
|
CDE38FE9679472D770C601E6 /* VoiceSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = EB58EEF9490F81FB2E1CE31C /* VoiceSession.swift */; };
|
||||||
D18AD347DFF72AB6361165BA /* TokenClient.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9681E7410B684D2D38752A70 /* TokenClient.swift */; };
|
D18AD347DFF72AB6361165BA /* TokenClient.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9681E7410B684D2D38752A70 /* TokenClient.swift */; };
|
||||||
D2E343C8B8657316A993698E /* TranscriptHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = D82D665165A7BC1A4CBCCC6F /* TranscriptHandler.swift */; };
|
D2E343C8B8657316A993698E /* TranscriptHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = D82D665165A7BC1A4CBCCC6F /* TranscriptHandler.swift */; };
|
||||||
D9A0059E342955A06A021D6C /* LiveKit in Frameworks */ = {isa = PBXBuildFile; productRef = A6BAD55715F383C0DB5F306F /* LiveKit */; };
|
D9A0059E342955A06A021D6C /* LiveKit in Frameworks */ = {isa = PBXBuildFile; productRef = A6BAD55715F383C0DB5F306F /* LiveKit */; };
|
||||||
DE1F24919E906EE13319C7A3 /* MediaPanelView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 68FACB1A5EA6FEFD1D2613E9 /* MediaPanelView.swift */; };
|
|
||||||
EE0C5762EC97B123A1880FFB /* Voices.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2E5DF207EDB787F63CB3A64D /* Voices.swift */; };
|
EE0C5762EC97B123A1880FFB /* Voices.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2E5DF207EDB787F63CB3A64D /* Voices.swift */; };
|
||||||
F26037AD028ED105E127CBD3 /* MediaAPITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 10C194656F19E89DFD656A91 /* MediaAPITests.swift */; };
|
|
||||||
FC696C6E74057D4C93E6DA67 /* ConversationStateTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BDD0AA8B0E9B45A325D1E5E7 /* ConversationStateTests.swift */; };
|
FC696C6E74057D4C93E6DA67 /* ConversationStateTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BDD0AA8B0E9B45A325D1E5E7 /* ConversationStateTests.swift */; };
|
||||||
/* End PBXBuildFile section */
|
/* End PBXBuildFile section */
|
||||||
|
|
||||||
@@ -59,26 +51,19 @@
|
|||||||
/* End PBXContainerItemProxy section */
|
/* End PBXContainerItemProxy section */
|
||||||
|
|
||||||
/* Begin PBXFileReference section */
|
/* Begin PBXFileReference section */
|
||||||
09FB71841B3D72CD1D69E47A /* MediaReconcileTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaReconcileTests.swift; sourceTree = "<group>"; };
|
|
||||||
0FC167F6658CFEC9546A7E9F /* AgentMonitor.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AgentMonitor.swift; sourceTree = "<group>"; };
|
0FC167F6658CFEC9546A7E9F /* AgentMonitor.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AgentMonitor.swift; sourceTree = "<group>"; };
|
||||||
10C194656F19E89DFD656A91 /* MediaAPITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaAPITests.swift; sourceTree = "<group>"; };
|
|
||||||
20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ServerReplacementCoordinatorTests.swift; sourceTree = "<group>"; };
|
20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ServerReplacementCoordinatorTests.swift; sourceTree = "<group>"; };
|
||||||
2A4C3CE2CA6FF828452803D1 /* Voice.app */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.application; path = Voice.app; sourceTree = BUILT_PRODUCTS_DIR; };
|
2A4C3CE2CA6FF828452803D1 /* Voice.app */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.application; path = Voice.app; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||||
2E5DF207EDB787F63CB3A64D /* Voices.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Voices.swift; sourceTree = "<group>"; };
|
2E5DF207EDB787F63CB3A64D /* Voices.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Voices.swift; sourceTree = "<group>"; };
|
||||||
2FED8976B60FBC13BC980FCB /* VoiceApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceApp.swift; sourceTree = "<group>"; };
|
2FED8976B60FBC13BC980FCB /* VoiceApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceApp.swift; sourceTree = "<group>"; };
|
||||||
3964F565A6C6A06399EBA4DF /* MediaPlayerController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaPlayerController.swift; sourceTree = "<group>"; };
|
|
||||||
3F3FA9F420065D2127B0AEC0 /* ConversationView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConversationView.swift; sourceTree = "<group>"; };
|
3F3FA9F420065D2127B0AEC0 /* ConversationView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConversationView.swift; sourceTree = "<group>"; };
|
||||||
48F76881B4AE51DEE74511CC /* SettingsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsView.swift; sourceTree = "<group>"; };
|
48F76881B4AE51DEE74511CC /* SettingsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsView.swift; sourceTree = "<group>"; };
|
||||||
5759EB570DB87BBC121D946D /* VoiceUITests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = VoiceUITests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
|
5759EB570DB87BBC121D946D /* VoiceUITests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = VoiceUITests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||||
5F6A697985B36B3A1A3F2AD5 /* VoiceLiveTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceLiveTests.swift; sourceTree = "<group>"; };
|
5F6A697985B36B3A1A3F2AD5 /* VoiceLiveTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceLiveTests.swift; sourceTree = "<group>"; };
|
||||||
61C71C48230C683D199D6F4B /* ControlChannel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ControlChannel.swift; sourceTree = "<group>"; };
|
61C71C48230C683D199D6F4B /* ControlChannel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ControlChannel.swift; sourceTree = "<group>"; };
|
||||||
68A45228EF59186CFC219F2E /* TranscriptHistory.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TranscriptHistory.swift; sourceTree = "<group>"; };
|
68A45228EF59186CFC219F2E /* TranscriptHistory.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TranscriptHistory.swift; sourceTree = "<group>"; };
|
||||||
68FACB1A5EA6FEFD1D2613E9 /* MediaPanelView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaPanelView.swift; sourceTree = "<group>"; };
|
|
||||||
6D0D153557BB1C756ED04C61 /* SettingsStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsStore.swift; sourceTree = "<group>"; };
|
6D0D153557BB1C756ED04C61 /* SettingsStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsStore.swift; sourceTree = "<group>"; };
|
||||||
6DC3AC57ED58FC517E212F19 /* PinnedMediaResourceLoader.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PinnedMediaResourceLoader.swift; sourceTree = "<group>"; };
|
|
||||||
8309AC6885ED7EB0047AE211 /* VoiceUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceUITests.swift; sourceTree = "<group>"; };
|
8309AC6885ED7EB0047AE211 /* VoiceUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceUITests.swift; sourceTree = "<group>"; };
|
||||||
8A20F8AFF1AAC5A944173549 /* PinnedMediaResourceLoaderTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PinnedMediaResourceLoaderTests.swift; sourceTree = "<group>"; };
|
|
||||||
9191AD60FF96E44F191864E3 /* MediaAPI.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaAPI.swift; sourceTree = "<group>"; };
|
|
||||||
9681E7410B684D2D38752A70 /* TokenClient.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TokenClient.swift; sourceTree = "<group>"; };
|
9681E7410B684D2D38752A70 /* TokenClient.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TokenClient.swift; sourceTree = "<group>"; };
|
||||||
A0C23305A6C55A83F8617D9D /* ConversationState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConversationState.swift; sourceTree = "<group>"; };
|
A0C23305A6C55A83F8617D9D /* ConversationState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConversationState.swift; sourceTree = "<group>"; };
|
||||||
A62BAE22E3A165DCB5DFFEE1 /* OrbView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OrbView.swift; sourceTree = "<group>"; };
|
A62BAE22E3A165DCB5DFFEE1 /* OrbView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OrbView.swift; sourceTree = "<group>"; };
|
||||||
@@ -88,7 +73,6 @@
|
|||||||
C22975C69240AACD3418091F /* VoiceTests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = VoiceTests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
|
C22975C69240AACD3418091F /* VoiceTests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = VoiceTests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||||
D82D665165A7BC1A4CBCCC6F /* TranscriptHandler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TranscriptHandler.swift; sourceTree = "<group>"; };
|
D82D665165A7BC1A4CBCCC6F /* TranscriptHandler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TranscriptHandler.swift; sourceTree = "<group>"; };
|
||||||
D9B40076934C28BA4B92D808 /* TLSPinning.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TLSPinning.swift; sourceTree = "<group>"; };
|
D9B40076934C28BA4B92D808 /* TLSPinning.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TLSPinning.swift; sourceTree = "<group>"; };
|
||||||
E492DE95BFFF99D7AC1E8970 /* MediaPlaybackTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MediaPlaybackTests.swift; sourceTree = "<group>"; };
|
|
||||||
E85BACA267ADD462306C9920 /* TaskPanelView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TaskPanelView.swift; sourceTree = "<group>"; };
|
E85BACA267ADD462306C9920 /* TaskPanelView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TaskPanelView.swift; sourceTree = "<group>"; };
|
||||||
EB58EEF9490F81FB2E1CE31C /* VoiceSession.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceSession.swift; sourceTree = "<group>"; };
|
EB58EEF9490F81FB2E1CE31C /* VoiceSession.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VoiceSession.swift; sourceTree = "<group>"; };
|
||||||
F5683993A1DE677121FFA1F3 /* ServerConfig.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ServerConfig.swift; sourceTree = "<group>"; };
|
F5683993A1DE677121FFA1F3 /* ServerConfig.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ServerConfig.swift; sourceTree = "<group>"; };
|
||||||
@@ -107,14 +91,6 @@
|
|||||||
/* End PBXFrameworksBuildPhase section */
|
/* End PBXFrameworksBuildPhase section */
|
||||||
|
|
||||||
/* Begin PBXGroup section */
|
/* Begin PBXGroup section */
|
||||||
0D2E974CBC286D7C886BA814 /* Media */ = {
|
|
||||||
isa = PBXGroup;
|
|
||||||
children = (
|
|
||||||
68FACB1A5EA6FEFD1D2613E9 /* MediaPanelView.swift */,
|
|
||||||
);
|
|
||||||
path = Media;
|
|
||||||
sourceTree = "<group>";
|
|
||||||
};
|
|
||||||
0DF0591DF1D767F89099F62C /* Settings */ = {
|
0DF0591DF1D767F89099F62C /* Settings */ = {
|
||||||
isa = PBXGroup;
|
isa = PBXGroup;
|
||||||
children = (
|
children = (
|
||||||
@@ -180,7 +156,6 @@
|
|||||||
isa = PBXGroup;
|
isa = PBXGroup;
|
||||||
children = (
|
children = (
|
||||||
84ADAF12A43A57888CA178A6 /* Conversation */,
|
84ADAF12A43A57888CA178A6 /* Conversation */,
|
||||||
0D2E974CBC286D7C886BA814 /* Media */,
|
|
||||||
0DF0591DF1D767F89099F62C /* Settings */,
|
0DF0591DF1D767F89099F62C /* Settings */,
|
||||||
F168B25F8B3C6F36B6E88228 /* Tasks */,
|
F168B25F8B3C6F36B6E88228 /* Tasks */,
|
||||||
);
|
);
|
||||||
@@ -197,16 +172,6 @@
|
|||||||
);
|
);
|
||||||
sourceTree = "<group>";
|
sourceTree = "<group>";
|
||||||
};
|
};
|
||||||
C12426487989358DE21A5426 /* Media */ = {
|
|
||||||
isa = PBXGroup;
|
|
||||||
children = (
|
|
||||||
9191AD60FF96E44F191864E3 /* MediaAPI.swift */,
|
|
||||||
3964F565A6C6A06399EBA4DF /* MediaPlayerController.swift */,
|
|
||||||
6DC3AC57ED58FC517E212F19 /* PinnedMediaResourceLoader.swift */,
|
|
||||||
);
|
|
||||||
path = Media;
|
|
||||||
sourceTree = "<group>";
|
|
||||||
};
|
|
||||||
CE49A38E40AAF659A9281B85 /* App */ = {
|
CE49A38E40AAF659A9281B85 /* App */ = {
|
||||||
isa = PBXGroup;
|
isa = PBXGroup;
|
||||||
children = (
|
children = (
|
||||||
@@ -221,10 +186,6 @@
|
|||||||
isa = PBXGroup;
|
isa = PBXGroup;
|
||||||
children = (
|
children = (
|
||||||
BDD0AA8B0E9B45A325D1E5E7 /* ConversationStateTests.swift */,
|
BDD0AA8B0E9B45A325D1E5E7 /* ConversationStateTests.swift */,
|
||||||
10C194656F19E89DFD656A91 /* MediaAPITests.swift */,
|
|
||||||
E492DE95BFFF99D7AC1E8970 /* MediaPlaybackTests.swift */,
|
|
||||||
09FB71841B3D72CD1D69E47A /* MediaReconcileTests.swift */,
|
|
||||||
8A20F8AFF1AAC5A944173549 /* PinnedMediaResourceLoaderTests.swift */,
|
|
||||||
20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */,
|
20A82F8996DEA166C94A7956 /* ServerReplacementCoordinatorTests.swift */,
|
||||||
FAEF89B2DEEE68F7168A8014 /* TranscriptDedupeTests.swift */,
|
FAEF89B2DEEE68F7168A8014 /* TranscriptDedupeTests.swift */,
|
||||||
);
|
);
|
||||||
@@ -245,7 +206,6 @@
|
|||||||
CE49A38E40AAF659A9281B85 /* App */,
|
CE49A38E40AAF659A9281B85 /* App */,
|
||||||
8DA8BEF2EE3E0F4181A94EB9 /* Features */,
|
8DA8BEF2EE3E0F4181A94EB9 /* Features */,
|
||||||
2820A418F921E18A2FA7E4FB /* LiveKit */,
|
2820A418F921E18A2FA7E4FB /* LiveKit */,
|
||||||
C12426487989358DE21A5426 /* Media */,
|
|
||||||
6B2AB34274D9B37E77178C6C /* Networking */,
|
6B2AB34274D9B37E77178C6C /* Networking */,
|
||||||
);
|
);
|
||||||
path = Sources;
|
path = Sources;
|
||||||
@@ -360,11 +320,7 @@
|
|||||||
28659D009399CA9C4FE57B74 /* ControlChannel.swift in Sources */,
|
28659D009399CA9C4FE57B74 /* ControlChannel.swift in Sources */,
|
||||||
64291BBC8D0DAD47E07E94E0 /* ConversationState.swift in Sources */,
|
64291BBC8D0DAD47E07E94E0 /* ConversationState.swift in Sources */,
|
||||||
36DF44D9A7896F18A064847D /* ConversationView.swift in Sources */,
|
36DF44D9A7896F18A064847D /* ConversationView.swift in Sources */,
|
||||||
C956D0FDFAC43925979E4077 /* MediaAPI.swift in Sources */,
|
|
||||||
DE1F24919E906EE13319C7A3 /* MediaPanelView.swift in Sources */,
|
|
||||||
73DA73811F8AEB2AFE340D10 /* MediaPlayerController.swift in Sources */,
|
|
||||||
3E01A0AF44FD3324FA72C43C /* OrbView.swift in Sources */,
|
3E01A0AF44FD3324FA72C43C /* OrbView.swift in Sources */,
|
||||||
A86EFB492F40C26D4AA315E9 /* PinnedMediaResourceLoader.swift in Sources */,
|
|
||||||
302BEC55CA9A10E49C307977 /* ServerConfig.swift in Sources */,
|
302BEC55CA9A10E49C307977 /* ServerConfig.swift in Sources */,
|
||||||
A94F6C509E3FE215115B2C58 /* SettingsStore.swift in Sources */,
|
A94F6C509E3FE215115B2C58 /* SettingsStore.swift in Sources */,
|
||||||
98BA1A0F92DB7B98A8BA7814 /* SettingsView.swift in Sources */,
|
98BA1A0F92DB7B98A8BA7814 /* SettingsView.swift in Sources */,
|
||||||
@@ -394,10 +350,6 @@
|
|||||||
buildActionMask = 2147483647;
|
buildActionMask = 2147483647;
|
||||||
files = (
|
files = (
|
||||||
FC696C6E74057D4C93E6DA67 /* ConversationStateTests.swift in Sources */,
|
FC696C6E74057D4C93E6DA67 /* ConversationStateTests.swift in Sources */,
|
||||||
F26037AD028ED105E127CBD3 /* MediaAPITests.swift in Sources */,
|
|
||||||
AF719B1668ADA929AA0237BF /* MediaPlaybackTests.swift in Sources */,
|
|
||||||
72F3B4AEC214521015200583 /* MediaReconcileTests.swift in Sources */,
|
|
||||||
B94FAFA796A9AF6011B03DEC /* PinnedMediaResourceLoaderTests.swift in Sources */,
|
|
||||||
BCAB602C722D74F6C96C5E36 /* ServerReplacementCoordinatorTests.swift in Sources */,
|
BCAB602C722D74F6C96C5E36 /* ServerReplacementCoordinatorTests.swift in Sources */,
|
||||||
1813A1CB0921D04FCAEC92D0 /* TranscriptDedupeTests.swift in Sources */,
|
1813A1CB0921D04FCAEC92D0 /* TranscriptDedupeTests.swift in Sources */,
|
||||||
);
|
);
|
||||||
|
|||||||
+6
-17
@@ -1,7 +1,7 @@
|
|||||||
# Device testing checklist
|
# Device testing checklist
|
||||||
|
|
||||||
Simulator verification covered connect/disconnect, transcripts (text mode), media
|
Simulator verification covered connect/disconnect, transcripts (text mode), tasks
|
||||||
reconciliation/UI, tasks panel, and the hardening pass. The items below need a **real
|
panel, and the hardening pass. The items below need a **real
|
||||||
iPhone on the LAN** (`https://192.168.86.2:9178`) because mic input, WebRTC capture,
|
iPhone on the LAN** (`https://192.168.86.2:9178`) because mic input, WebRTC capture,
|
||||||
and audio session behavior don't work in this simulator environment (CoreAudio `-4010`,
|
and audio session behavior don't work in this simulator environment (CoreAudio `-4010`,
|
||||||
0 input channels).
|
0 input channels).
|
||||||
@@ -31,29 +31,18 @@ ensure the server cert pin is current (see README — re-export after any server
|
|||||||
side button to lock the device. It only disables the automatic idle timeout.
|
side button to lock the device. It only disables the automatic idle timeout.
|
||||||
- [ ] After hang-up, auto-lock returns (idle timer re-enabled).
|
- [ ] After hang-up, auto-lock returns (idle timer re-enabled).
|
||||||
|
|
||||||
## 4. Audible media playback
|
## 4. Audio interruption (phone call / Siri)
|
||||||
|
|
||||||
- [ ] The deployed server now serves cached AAC for library tracks requested with
|
|
||||||
`?format=m4a`; verify actual speaker/headphone output on hardware.
|
|
||||||
- [ ] Play a library track: audio is audible, now-playing card shows art/title/status.
|
|
||||||
- [ ] Pause / ±10 s / stop / seek all work; agent mics off while playing
|
|
||||||
(`media_state` published), mics back on when stopped.
|
|
||||||
- [ ] End conversation while music was playing before → playback resumes.
|
|
||||||
|
|
||||||
## 5. Audio interruption (phone call / Siri)
|
|
||||||
|
|
||||||
- [ ] Mid-conversation, trigger an interruption (incoming call or "Hey Siri").
|
- [ ] Mid-conversation, trigger an interruption (incoming call or "Hey Siri").
|
||||||
- [ ] Mic is muted on interruption begin; conversation/media pause cleanly.
|
- [ ] On end (with `shouldResume`), mic and conversation resume automatically.
|
||||||
- [ ] On end (with `shouldResume`), mic and media resume automatically.
|
|
||||||
- [ ] Media panel: AVPlayer pauses on interruption and resumes after.
|
|
||||||
|
|
||||||
## 6. Mic permission denied fallback
|
## 5. Mic permission denied fallback
|
||||||
|
|
||||||
- [ ] Deny microphone permission in iOS Settings for the app, relaunch.
|
- [ ] Deny microphone permission in iOS Settings for the app, relaunch.
|
||||||
- [ ] Start → session reaches **ready** despite mic failure (non-fatal); orange error
|
- [ ] Start → session reaches **ready** despite mic failure (non-fatal); orange error
|
||||||
banner shows the mic error; text mode remains fully usable.
|
banner shows the mic error; text mode remains fully usable.
|
||||||
|
|
||||||
## 7. General device sanity
|
## 6. General device sanity
|
||||||
|
|
||||||
- [ ] Connect / hang-up / reconnect matrix (3×) — no crashes, state resets cleanly.
|
- [ ] Connect / hang-up / reconnect matrix (3×) — no crashes, state resets cleanly.
|
||||||
- [ ] Change server URL in Settings while idle → session recreates without crash.
|
- [ ] Change server URL in Settings while idle → session recreates without crash.
|
||||||
|
|||||||
+32
-25
@@ -1,13 +1,20 @@
|
|||||||
# Project status and handoff
|
# Project status and handoff
|
||||||
|
|
||||||
Last updated: 2026-09-09.
|
Last updated: 2026-09-12.
|
||||||
|
|
||||||
## Current direction
|
## Current direction
|
||||||
|
|
||||||
Keep **Qwen 2.5**. The user explicitly canceled the earlier request to switch to
|
Keep **Qwen 2.5**. The user explicitly canceled the earlier request to switch to
|
||||||
a ChatGPT subscription/device-token login. Do not revive that migration because
|
a ChatGPT subscription/device-token login. Do not revive that migration because
|
||||||
an old saved goal mentions it. Current work is the native iOS UI/UX redesign,
|
an old saved goal mentions it. Current work is the native iOS UI/UX redesign
|
||||||
restoring the music/AI server, and end-to-end qualification.
|
and end-to-end qualification.
|
||||||
|
|
||||||
|
**2026-09-12: the embedded music player was removed** by user decision — the
|
||||||
|
feature is gone from the app (Media tab, `Sources/Media/`, `MediaPanelView`,
|
||||||
|
`MediaAPI`, `MediaPlayerController`, `PinnedMediaResourceLoader`, media tests,
|
||||||
|
and the `media_state` control publish). The backend media API still exists and
|
||||||
|
is untouched. The backend AAC compatibility patch (below) is now irrelevant to
|
||||||
|
the iOS app but remains deployed.
|
||||||
|
|
||||||
The UI and playback changes are implemented, and the NAS deployment was verified
|
The UI and playback changes are implemented, and the NAS deployment was verified
|
||||||
on September 8. Full app qualification is **not complete**: physical-device tests
|
on September 8. Full app qualification is **not complete**: physical-device tests
|
||||||
@@ -19,6 +26,8 @@ server health and device availability must be rechecked when resuming.
|
|||||||
### Native interface and usability
|
### Native interface and usability
|
||||||
|
|
||||||
- Reorganized the app into native **Talk, Music, Activity** tabs, sharing one session.
|
- Reorganized the app into native **Talk, Music, Activity** tabs, sharing one session.
|
||||||
|
(The Music tab was removed again on 2026-09-12 by user decision; the app now has
|
||||||
|
**Talk** and **Activity** tabs.)
|
||||||
- Replaced the large animated orb with a compact waveform/status presentation;
|
- Replaced the large animated orb with a compact waveform/status presentation;
|
||||||
added a clearer welcome screen, Voice/Type selection before connecting, explicit
|
added a clearer welcome screen, Voice/Type selection before connecting, explicit
|
||||||
start/end controls, microphone meter, and recoverable connection errors.
|
start/end controls, microphone meter, and recoverable connection errors.
|
||||||
@@ -27,7 +36,7 @@ server health and device availability must be rechecked when resuming.
|
|||||||
- Reworked Settings around voice, input mode, model, and connection; server changes
|
- Reworked Settings around voice, input mode, model, and connection; server changes
|
||||||
require an explicit validated save. Voice selection persists.
|
require an explicit validated save. Voice selection persists.
|
||||||
- Improved Music library/search navigation, now-playing controls, seek/volume,
|
- Improved Music library/search navigation, now-playing controls, seek/volume,
|
||||||
empty/loading/error states, and validated station-entry sheet.
|
empty/loading/error states, and validated station-entry sheet. (Removed 2026-09-12.)
|
||||||
- Improved Activity hierarchy and task detail presentation with an active-task badge.
|
- Improved Activity hierarchy and task detail presentation with an active-task badge.
|
||||||
- Removed forced dark appearance; adopted adaptive system styling, Dynamic Type,
|
- Removed forced dark appearance; adopted adaptive system styling, Dynamic Type,
|
||||||
and larger interaction targets, informed by the Impeccable native-iOS guidance.
|
and larger interaction targets, informed by the Impeccable native-iOS guidance.
|
||||||
@@ -35,14 +44,11 @@ server health and device availability must be rechecked when resuming.
|
|||||||
### Functional fixes
|
### Functional fixes
|
||||||
|
|
||||||
- Observed nested transcript/state/task objects so asynchronous updates render.
|
- Observed nested transcript/state/task objects so asynchronous updates render.
|
||||||
- Preserved latest-wins server replacement; clear old media when switching servers.
|
- Preserved latest-wins server replacement.
|
||||||
- Sent standalone media controls and reports to the default server room.
|
(All media-related behaviors — media clearing on server switch, standalone media
|
||||||
- Refreshed track metadata as downloads become ready; exposed request/decoder errors.
|
controls/reports, track metadata refresh, pinned-TLS artwork, cached-AAC requests,
|
||||||
- Loaded artwork through pinned TLS for the NAS self-signed certificate.
|
UTI content-type fix, bounded-range media loader — were removed with the music
|
||||||
- Requested cached AAC (`?format=m4a`) for YouTube library tracks on iOS.
|
player on 2026-09-12.)
|
||||||
- Corrected AVFoundation's content type to a UTI rather than an HTTP MIME string.
|
|
||||||
- Changed the pinned media loader to deliver 256 KB byte ranges instead of buffering
|
|
||||||
an entire song before delivering usable audio, addressing a reproduced timeout.
|
|
||||||
|
|
||||||
### Tests and tooling
|
### Tests and tooling
|
||||||
|
|
||||||
@@ -50,9 +56,10 @@ server health and device availability must be rechecked when resuming.
|
|||||||
- Added the `VoiceUITests` target in `project.yml` and regenerated the Xcode project.
|
- Added the `VoiceUITests` target in `project.yml` and regenerated the Xcode project.
|
||||||
- Added isolated DEBUG UI-test preferences and optional fixtures; normal application
|
- Added isolated DEBUG UI-test preferences and optional fixtures; normal application
|
||||||
preferences are not reset by those UI-test launch arguments.
|
preferences are not reset by those UI-test launch arguments.
|
||||||
- Added media-control/error/metadata tests and opt-in real-NAS UI tests.
|
- Added media-control/error/metadata tests and opt-in real-NAS UI tests. (Media tests
|
||||||
|
were removed with the music player on 2026-09-12.)
|
||||||
- Live tests retain failure screenshots/UI hierarchy and attempt to stop music
|
- Live tests retain failure screenshots/UI hierarchy and attempt to stop music
|
||||||
they initiated even when an assertion fails.
|
they initiated even when an assertion fails. (Music playback test removed 2026-09-12.)
|
||||||
|
|
||||||
## Backend deployment and cross-repository dependency
|
## Backend deployment and cross-repository dependency
|
||||||
|
|
||||||
@@ -87,15 +94,14 @@ Do not copy credentials or `.env` contents into documentation or commits.
|
|||||||
|
|
||||||
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator.
|
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator.
|
||||||
|
|
||||||
- 68 unit tests passed; five offline UI tests passed (navigation/error recovery,
|
- 49 unit tests passed (after the 2026-09-12 music-player removal; previously 68 with
|
||||||
settings persistence/validation, music validation, transcript/task UI, large text).
|
media tests); four offline UI tests passed (navigation/error recovery,
|
||||||
|
settings persistence/validation, transcript/task UI, large text).
|
||||||
- Real NAS tests verified Qwen replies, background task completion, reconnect,
|
- Real NAS tests verified Qwen replies, background task completion, reconnect,
|
||||||
library loading, advancing playback time, pause, forward-seek control and stop.
|
library loading, advancing playback time, pause, forward-seek control and stop.
|
||||||
|
(Playback evidence predates the music-player removal; the feature no longer exists
|
||||||
|
in the app.)
|
||||||
- Qwen/task/reconnect passed three consecutive iterations.
|
- Qwen/task/reconnect passed three consecutive iterations.
|
||||||
- Music initially failed intermittently. Logs captured an AVPlayer timeout while
|
|
||||||
buffering a whole 6 MB response. After bounded-range delivery, three consecutive
|
|
||||||
live playback iterations passed. The earlier isolated agent-join failure remains
|
|
||||||
unexplained; do not claim all intermittent connection issues are resolved.
|
|
||||||
- Backend integration: 16 passed, 7 skipped, 2 deselected.
|
- Backend integration: 16 passed, 7 skipped, 2 deselected.
|
||||||
- Backend AAC regression: one passed (conversion, cache reuse, original preservation,
|
- Backend AAC regression: one passed (conversion, cache reuse, original preservation,
|
||||||
and byte ranges); deployed range request returned HTTP 206.
|
and byte ranges); deployed range request returned HTTP 206.
|
||||||
@@ -115,19 +121,20 @@ Artifacts live on this Mac under the XcodeBuildMCP workspace, not in Git.
|
|||||||
2. **Build, sign, install and run on that iPhone.** No successful device install is
|
2. **Build, sign, install and run on that iPhone.** No successful device install is
|
||||||
recorded. Check Developer Mode, trust, signing team/profile, and LAN access.
|
recorded. Check Developer Mode, trust, signing team/profile, and LAN access.
|
||||||
3. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
|
3. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
|
||||||
barge-in, media/voice handoff, Siri/phone interruptions, lock/background behavior,
|
barge-in, Siri/phone interruptions, lock/background behavior,
|
||||||
microphone-permission-denied fallback, and network-loss recovery. Some checks
|
microphone-permission-denied fallback, and network-loss recovery. Some checks
|
||||||
require the user's speech/listening or a real interruption; simulator state tests
|
require the user's speech/listening or a real interruption; simulator state tests
|
||||||
do not prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
|
do not prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
|
||||||
4. **Broader regression:** rerun the full suite after the range-loader change; test
|
4. **Broader regression:** rerun the full suite after the music-player removal;
|
||||||
longer playback, seek near EOF, slow network/cancellation and server replacement.
|
test slow network/cancellation and server replacement. Exercise voice/model
|
||||||
Exercise new downloads, test-owned station add/play/remove, and voice/model
|
selection against the live agent.
|
||||||
selection against the live agent. Do not delete the user's existing library song.
|
|
||||||
5. **Final visual/accessibility pass:** inspect all populated/error screens in light
|
5. **Final visual/accessibility pass:** inspect all populated/error screens in light
|
||||||
and dark mode, large text and small-device layouts; test VoiceOver and focus.
|
and dark mode, large text and small-device layouts; test VoiceOver and focus.
|
||||||
The existing large-text navigation test is not a full accessibility audit.
|
The existing large-text navigation test is not a full accessibility audit.
|
||||||
6. **Publish the backend compatibility source safely** in the sibling repository
|
6. **Publish the backend compatibility source safely** in the sibling repository
|
||||||
and ensure its normal deployment preserves mounts, certificates and Qwen settings.
|
and ensure its normal deployment preserves mounts, certificates and Qwen settings.
|
||||||
|
(Lower priority now: the iOS app no longer consumes the media API, but the deployed
|
||||||
|
container still serves it for the web UI.)
|
||||||
7. Update this report with results and remaining limitations. Do not mark the whole
|
7. Update this report with results and remaining limitations. Do not mark the whole
|
||||||
app qualified until the outstanding end-to-end gates have evidence.
|
app qualified until the outstanding end-to-end gates have evidence.
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user