github/openclaw
1
1
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-02-19 18:39:20 -05:00
Code Issues Packages Projects Releases Wiki Activity

Revert "iOS: wire node services and tests"

Browse Source 
This reverts commit 7b0a0f3dac.
This commit is contained in:
Mariano Belinky
2026-02-02 17:27:56 +00:00
parent c83bdb73a4
commit 4ab814fd50
59 changed files with 182 additions and 6234 deletions
Download Patch File Download Diff File Expand all files Collapse all files

110
IOS-PRIORITIES.md
View File

@@ -1,110 +0,0 @@
# iOS App Priorities (OpenClaw / Moltbot)
This report is based on repo code + docs in `/Users/mariano/Coding/openclaw`, with focus on:
- iOS Swift sources under `apps/ios/Sources`
- Shared Swift packages under `apps/shared/OpenClawKit`
- Gateway protocol + node docs in `docs/`
- macOS node implementation under `apps/macos/Sources/OpenClaw/NodeMode`
## Current iOS state (what works today)
**Gateway connectivity + pairing**
- Uses the unified Gateway WebSocket protocol with device identity + challenge signing (via `GatewayChannel` in OpenClawKit).
- Discovery via Bonjour (`NWBrowser`) for `_openclaw-gw._tcp` plus manual host/port fallback and TLS pinning support (`apps/ios/Sources/Gateway/*`).
- Stores gateway token/password in Keychain (`GatewaySettingsStore.swift`).
**Node command handling** (implemented in `NodeAppModel.handleInvoke`)
- Canvas: `canvas.present`, `canvas.hide`, `canvas.navigate`, `canvas.eval`, `canvas.snapshot`.
- A2UI: `canvas.a2ui.reset`, `canvas.a2ui.push`, `canvas.a2ui.pushJsonl`.
- Camera: `camera.list`, `camera.snap`, `camera.clip`.
- Screen: `screen.record` (ReplayKit-based screen recording).
- Location: `location.get` (CoreLocation-based).
- Foreground gating: returns `NODE_BACKGROUND_UNAVAILABLE` for canvas/camera/screen when backgrounded.
**Voice features**
- Voice Wake: continuous speech recognition with wake-word gating and gateway sync (`VoiceWakeManager.swift`).
- Talk Mode: speech-to-text + chat.send + ElevenLabs streaming TTS + system voice fallback (`TalkModeManager.swift`).
**Chat UI**
- Uses shared SwiftUI chat client (`OpenClawChatUI`) and Gateway chat APIs (`IOSGatewayChatTransport.swift`).
**UI surface**
- Full-screen canvas with overlay controls for chat, settings, and Talk orb (`RootCanvas.swift`).
- Settings for gateway selection, voice, camera, location, screen prevent-sleep, and debug flags (`SettingsTab.swift`).
## Protocol requirements the iOS app must honor
From `docs/gateway/protocol.md` + `docs/nodes/index.md` + OpenClawKit:
- WebSocket `connect` handshake with `role: "node"`, `caps`, `commands`, and `permissions` claims.
- Device identity + challenge signing on connect; device token persistence.
- Respond to `node.invoke.request` with `node.invoke.result`.
- Emit node events (`node.event`) for voice transcripts and agent requests.
- Use gateway RPCs needed by the iOS UI: `config.get`, `voicewake.get/set`, `chat.*`, `sessions.list`.
## Gaps / incomplete or mismatched behavior
**1) Declared commands exceed iOS implementation**
`GatewayConnectionController.currentCommands()` includes:
- `system.run`, `system.which`, `system.notify`, `system.execApprovals.get`, `system.execApprovals.set`
…but `NodeAppModel.handleInvoke` does not implement any `system.*` commands and will return `INVALID_REQUEST: unknown command` for them. This is a protocol-level mismatch: the gateway will believe iOS supports system execution + notifications, but the node cannot fulfill those requests.
**2) Permissions map is always empty**
iOS sends `permissions: [:]` in its connect options, while macOS node reports real permission states via `PermissionManager`. This means the gateway cannot reason about iOS permission availability even though camera/mic/location/screen limitations materially affect command success.
**3) Canvas parity gaps**
- `canvas.hide` is currently a no-op on iOS (returns ok but doesn’t change UI).
- `canvas.present` ignores placement params (macOS supports window placement).
These may be acceptable platform limitations, but they should be explicitly handled/documented so the node surface is consistent and predictable.
## iOS vs. macOS node feature parity
macOS node mode (`apps/macos/Sources/OpenClaw/NodeMode/*`) supports:
- `system.run`, `system.which`, `system.notify`, `system.execApprovals.get/set`.
- Permission reporting in `connect.permissions`.
- Canvas window placement + hide.
iOS currently implements the shared node surface (canvas/camera/screen/location + voice) but does **not** match macOS on the system/exec side and permission reporting.
## Prioritized work items (ordered by importance)
1. **Fix the command/implementation mismatch for `system.*`**
- Either remove `system.*` from iOS `currentCommands()` **or** implement iOS equivalents (at minimum `system.notify` via local notifications) with clear error semantics for unsupported actions.
- This is the highest risk mismatch because it misleads the gateway and any operator about what the iOS node can actually do.
2. **Report real iOS permission state in `connect.permissions`**
- Mirror macOS behavior by sending camera/microphone/location/screen-recording permission flags.
- This enables the gateway to make better decisions and reduces “it failed because permissions” surprises.
3. **Clarify/normalize iOS canvas behaviors**
- Decide how `canvas.hide` should behave on iOS (e.g., return to the local scaffold) and implement it.
- Document that `canvas.present` ignores placement on iOS, or add a platform-specific best effort.
4. **Explicitly document platform deltas vs. macOS node**
- The docs currently describe `system.*` under “Nodes” and cite macOS/headless node support. iOS should be clearly marked as not supporting system exec to avoid incorrect user expectations.
5. **Release readiness (if the goal is to move beyond internal preview)**
- Docs state the iOS app is “internal preview” (`docs/platforms/ios.md`).
- If public distribution is desired, build out TestFlight/App Store release steps (fastlane exists in `apps/ios/fastlane/`).
## Files referenced (key evidence)
- iOS node behavior: `apps/ios/Sources/Model/NodeAppModel.swift`
- iOS command declarations: `apps/ios/Sources/Gateway/GatewayConnectionController.swift`
- iOS discovery + TLS: `apps/ios/Sources/Gateway/*`
- iOS voice: `apps/ios/Sources/Voice/*`
- iOS screen/camera/location: `apps/ios/Sources/Screen/*`, `apps/ios/Sources/Camera/*`, `apps/ios/Sources/Location/*`
- Shared protocol + commands: `apps/shared/OpenClawKit/Sources/OpenClawKit/*`
- macOS node runtime: `apps/macos/Sources/OpenClaw/NodeMode/*`
- Node + protocol docs: `docs/nodes/index.md`, `docs/gateway/protocol.md`, `docs/platforms/ios.md`

173
apps/ios/Sources/Calendar/CalendarService.swift
View File

@@ -1,173 +0,0 @@
import EventKit
import Foundation
import OpenClawKit
final class CalendarService: CalendarServicing {
func events(params: OpenClawCalendarEventsParams) async throws -> OpenClawCalendarEventsPayload {
let store = EKEventStore()
let status = EKEventStore.authorizationStatus(for: .event)
let authorized = await Self.ensureAuthorization(store: store, status: status)
guard authorized else {
throw NSError(domain: "Calendar", code: 1, userInfo: [
NSLocalizedDescriptionKey: "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission",
])
}
let (start, end) = Self.resolveRange(
startISO: params.startISO,
endISO: params.endISO)
let predicate = store.predicateForEvents(withStart: start, end: end, calendars: nil)
let events = store.events(matching: predicate)
let limit = max(1, min(params.limit ?? 50, 500))
let selected = Array(events.prefix(limit))
let formatter = ISO8601DateFormatter()
let payload = selected.map { event in
OpenClawCalendarEventPayload(
identifier: event.eventIdentifier ?? UUID().uuidString,
title: event.title ?? "(untitled)",
startISO: formatter.string(from: event.startDate),
endISO: formatter.string(from: event.endDate),
isAllDay: event.isAllDay,
location: event.location,
calendarTitle: event.calendar.title)
}
return OpenClawCalendarEventsPayload(events: payload)
}
func add(params: OpenClawCalendarAddParams) async throws -> OpenClawCalendarAddPayload {
let store = EKEventStore()
let status = EKEventStore.authorizationStatus(for: .event)
let authorized = await Self.ensureWriteAuthorization(store: store, status: status)
guard authorized else {
throw NSError(domain: "Calendar", code: 2, userInfo: [
NSLocalizedDescriptionKey: "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission",
])
}
let title = params.title.trimmingCharacters(in: .whitespacesAndNewlines)
guard !title.isEmpty else {
throw NSError(domain: "Calendar", code: 3, userInfo: [
NSLocalizedDescriptionKey: "CALENDAR_INVALID: title required",
])
}
let formatter = ISO8601DateFormatter()
guard let start = formatter.date(from: params.startISO) else {
throw NSError(domain: "Calendar", code: 4, userInfo: [
NSLocalizedDescriptionKey: "CALENDAR_INVALID: startISO required",
])
}
guard let end = formatter.date(from: params.endISO) else {
throw NSError(domain: "Calendar", code: 5, userInfo: [
NSLocalizedDescriptionKey: "CALENDAR_INVALID: endISO required",
])
}
let event = EKEvent(eventStore: store)
event.title = title
event.startDate = start
event.endDate = end
event.isAllDay = params.isAllDay ?? false
if let location = params.location?.trimmingCharacters(in: .whitespacesAndNewlines), !location.isEmpty {
event.location = location
}
if let notes = params.notes?.trimmingCharacters(in: .whitespacesAndNewlines), !notes.isEmpty {
event.notes = notes
}
event.calendar = try Self.resolveCalendar(
store: store,
calendarId: params.calendarId,
calendarTitle: params.calendarTitle)
try store.save(event, span: .thisEvent)
let payload = OpenClawCalendarEventPayload(
identifier: event.eventIdentifier ?? UUID().uuidString,
title: event.title ?? title,
startISO: formatter.string(from: event.startDate),
endISO: formatter.string(from: event.endDate),
isAllDay: event.isAllDay,
location: event.location,
calendarTitle: event.calendar.title)
return OpenClawCalendarAddPayload(event: payload)
}
private static func ensureAuthorization(store: EKEventStore, status: EKAuthorizationStatus) async -> Bool {
switch status {
case .authorized:
return true
case .notDetermined:
return await withCheckedContinuation { cont in
store.requestAccess(to: .event) { granted, _ in
cont.resume(returning: granted)
}
}
case .restricted, .denied:
return false
case .fullAccess:
return true
case .writeOnly:
return false
@unknown default:
return false
}
}
private static func ensureWriteAuthorization(store: EKEventStore, status: EKAuthorizationStatus) async -> Bool {
switch status {
case .authorized, .fullAccess, .writeOnly:
return true
case .notDetermined:
return await withCheckedContinuation { cont in
store.requestAccess(to: .event) { granted, _ in
cont.resume(returning: granted)
}
}
case .restricted, .denied:
return false
@unknown default:
return false
}
}
private static func resolveCalendar(
store: EKEventStore,
calendarId: String?,
calendarTitle: String?) throws -> EKCalendar
{
if let id = calendarId?.trimmingCharacters(in: .whitespacesAndNewlines), !id.isEmpty,
let calendar = store.calendar(withIdentifier: id)
{
return calendar
}
if let title = calendarTitle?.trimmingCharacters(in: .whitespacesAndNewlines), !title.isEmpty {
if let calendar = store.calendars(for: .event).first(where: {
$0.title.compare(title, options: [.caseInsensitive, .diacriticInsensitive]) == .orderedSame
}) {
return calendar
}
throw NSError(domain: "Calendar", code: 6, userInfo: [
NSLocalizedDescriptionKey: "CALENDAR_NOT_FOUND: no calendar named \(title)",
])
}
if let fallback = store.defaultCalendarForNewEvents {
return fallback
}
throw NSError(domain: "Calendar", code: 7, userInfo: [
NSLocalizedDescriptionKey: "CALENDAR_NOT_FOUND: no default calendar",
])
}
private static func resolveRange(startISO: String?, endISO: String?) -> (Date, Date) {
let formatter = ISO8601DateFormatter()
let start = startISO.flatMap { formatter.date(from: $0) } ?? Date()
let end = endISO.flatMap { formatter.date(from: $0) } ?? start.addingTimeInterval(7 * 24 * 3600)
return (start, end)
}
}

25
apps/ios/Sources/Capabilities/NodeCapabilityRouter.swift
View File

@@ -1,25 +0,0 @@
import Foundation
import OpenClawKit
@MainActor
final class NodeCapabilityRouter {
enum RouterError: Error {
case unknownCommand
case handlerUnavailable
}
typealias Handler = (BridgeInvokeRequest) async throws -> BridgeInvokeResponse
private let handlers: [String: Handler]
init(handlers: [String: Handler]) {
self.handlers = handlers
}
func handle(_ request: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
guard let handler = handlers[request.command] else {
throw RouterError.unknownCommand
}
return try await handler(request)
}
}

214
apps/ios/Sources/Contacts/ContactsService.swift
View File

@@ -1,214 +0,0 @@
import Contacts
import Foundation
import OpenClawKit
final class ContactsService: ContactsServicing {
private static var payloadKeys: [CNKeyDescriptor] {
[
CNContactIdentifierKey as CNKeyDescriptor,
CNContactGivenNameKey as CNKeyDescriptor,
CNContactFamilyNameKey as CNKeyDescriptor,
CNContactOrganizationNameKey as CNKeyDescriptor,
CNContactPhoneNumbersKey as CNKeyDescriptor,
CNContactEmailAddressesKey as CNKeyDescriptor,
]
}
func search(params: OpenClawContactsSearchParams) async throws -> OpenClawContactsSearchPayload {
let store = CNContactStore()
let status = CNContactStore.authorizationStatus(for: .contacts)
let authorized = await Self.ensureAuthorization(store: store, status: status)
guard authorized else {
throw NSError(domain: "Contacts", code: 1, userInfo: [
NSLocalizedDescriptionKey: "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission",
])
}
let limit = max(1, min(params.limit ?? 25, 200))
var contacts: [CNContact] = []
if let query = params.query?.trimmingCharacters(in: .whitespacesAndNewlines), !query.isEmpty {
let predicate = CNContact.predicateForContacts(matchingName: query)
contacts = try store.unifiedContacts(matching: predicate, keysToFetch: Self.payloadKeys)
} else {
let request = CNContactFetchRequest(keysToFetch: Self.payloadKeys)
try store.enumerateContacts(with: request) { contact, stop in
contacts.append(contact)
if contacts.count >= limit {
stop.pointee = true
}
}
}
let sliced = Array(contacts.prefix(limit))
let payload = sliced.map { Self.payload(from: $0) }
return OpenClawContactsSearchPayload(contacts: payload)
}
func add(params: OpenClawContactsAddParams) async throws -> OpenClawContactsAddPayload {
let store = CNContactStore()
let status = CNContactStore.authorizationStatus(for: .contacts)
let authorized = await Self.ensureAuthorization(store: store, status: status)
guard authorized else {
throw NSError(domain: "Contacts", code: 1, userInfo: [
NSLocalizedDescriptionKey: "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission",
])
}
let givenName = params.givenName?.trimmingCharacters(in: .whitespacesAndNewlines)
let familyName = params.familyName?.trimmingCharacters(in: .whitespacesAndNewlines)
let organizationName = params.organizationName?.trimmingCharacters(in: .whitespacesAndNewlines)
let displayName = params.displayName?.trimmingCharacters(in: .whitespacesAndNewlines)
let phoneNumbers = Self.normalizeStrings(params.phoneNumbers)
let emails = Self.normalizeStrings(params.emails, lowercased: true)
let hasName = !(givenName ?? "").isEmpty || !(familyName ?? "").isEmpty || !(displayName ?? "").isEmpty
let hasOrg = !(organizationName ?? "").isEmpty
let hasDetails = !phoneNumbers.isEmpty || !emails.isEmpty
guard hasName || hasOrg || hasDetails else {
throw NSError(domain: "Contacts", code: 2, userInfo: [
NSLocalizedDescriptionKey: "CONTACTS_INVALID: include a name, organization, phone, or email",
])
}
if !phoneNumbers.isEmpty || !emails.isEmpty {
if let existing = try Self.findExistingContact(
store: store,
phoneNumbers: phoneNumbers,
emails: emails)
{
return OpenClawContactsAddPayload(contact: Self.payload(from: existing))
}
}
let contact = CNMutableContact()
contact.givenName = givenName ?? ""
contact.familyName = familyName ?? ""
contact.organizationName = organizationName ?? ""
if contact.givenName.isEmpty && contact.familyName.isEmpty, let displayName {
contact.givenName = displayName
}
contact.phoneNumbers = phoneNumbers.map {
CNLabeledValue(label: CNLabelPhoneNumberMobile, value: CNPhoneNumber(stringValue: $0))
}
contact.emailAddresses = emails.map {
CNLabeledValue(label: CNLabelHome, value: $0 as NSString)
}
let save = CNSaveRequest()
save.add(contact, toContainerWithIdentifier: nil)
try store.execute(save)
let persisted: CNContact
if !contact.identifier.isEmpty {
persisted = try store.unifiedContact(
withIdentifier: contact.identifier,
keysToFetch: Self.payloadKeys)
} else {
persisted = contact
}
return OpenClawContactsAddPayload(contact: Self.payload(from: persisted))
}
private static func ensureAuthorization(store: CNContactStore, status: CNAuthorizationStatus) async -> Bool {
switch status {
case .authorized, .limited:
return true
case .notDetermined:
return await withCheckedContinuation { cont in
store.requestAccess(for: .contacts) { granted, _ in
cont.resume(returning: granted)
}
}
case .restricted, .denied:
return false
@unknown default:
return false
}
}
private static func normalizeStrings(_ values: [String]?, lowercased: Bool = false) -> [String] {
(values ?? [])
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
.map { lowercased ? $0.lowercased() : $0 }
}
private static func findExistingContact(
store: CNContactStore,
phoneNumbers: [String],
emails: [String]) throws -> CNContact?
{
if phoneNumbers.isEmpty && emails.isEmpty {
return nil
}
var matches: [CNContact] = []
for phone in phoneNumbers {
let predicate = CNContact.predicateForContacts(matching: CNPhoneNumber(stringValue: phone))
let contacts = try store.unifiedContacts(matching: predicate, keysToFetch: Self.payloadKeys)
matches.append(contentsOf: contacts)
}
for email in emails {
let predicate = CNContact.predicateForContacts(matchingEmailAddress: email)
let contacts = try store.unifiedContacts(matching: predicate, keysToFetch: Self.payloadKeys)
matches.append(contentsOf: contacts)
}
return Self.matchContacts(contacts: matches, phoneNumbers: phoneNumbers, emails: emails)
}
private static func matchContacts(
contacts: [CNContact],
phoneNumbers: [String],
emails: [String]) -> CNContact?
{
let normalizedPhones = Set(phoneNumbers.map { normalizePhone($0) }.filter { !$0.isEmpty })
let normalizedEmails = Set(emails.map { $0.lowercased() }.filter { !$0.isEmpty })
var seen = Set<String>()
for contact in contacts {
guard seen.insert(contact.identifier).inserted else { continue }
let contactPhones = Set(contact.phoneNumbers.map { normalizePhone($0.value.stringValue) })
let contactEmails = Set(contact.emailAddresses.map { String($0.value).lowercased() })
if !normalizedPhones.isEmpty, !contactPhones.isDisjoint(with: normalizedPhones) {
return contact
}
if !normalizedEmails.isEmpty, !contactEmails.isDisjoint(with: normalizedEmails) {
return contact
}
}
return nil
}
private static func normalizePhone(_ phone: String) -> String {
let trimmed = phone.trimmingCharacters(in: .whitespacesAndNewlines)
let digits = trimmed.unicodeScalars.filter { CharacterSet.decimalDigits.contains($0) }
let normalized = String(String.UnicodeScalarView(digits))
return normalized.isEmpty ? trimmed : normalized
}
private static func payload(from contact: CNContact) -> OpenClawContactPayload {
OpenClawContactPayload(
identifier: contact.identifier,
displayName: CNContactFormatter.string(from: contact, style: .fullName)
?? "\(contact.givenName) \(contact.familyName)".trimmingCharacters(in: .whitespacesAndNewlines),
givenName: contact.givenName,
familyName: contact.familyName,
organizationName: contact.organizationName,
phoneNumbers: contact.phoneNumbers.map { $0.value.stringValue },
emails: contact.emailAddresses.map { String($0.value) })
}
#if DEBUG
static func _test_matches(contact: CNContact, phoneNumbers: [String], emails: [String]) -> Bool {
matchContacts(contacts: [contact], phoneNumbers: phoneNumbers, emails: emails) != nil
}
#endif
}

87
apps/ios/Sources/Device/DeviceStatusService.swift
View File

@@ -1,87 +0,0 @@
import Foundation
import OpenClawKit
import UIKit
final class DeviceStatusService: DeviceStatusServicing {
private let networkStatus: NetworkStatusService
init(networkStatus: NetworkStatusService = NetworkStatusService()) {
self.networkStatus = networkStatus
}
func status() async throws -> OpenClawDeviceStatusPayload {
let battery = self.batteryStatus()
let thermal = self.thermalStatus()
let storage = self.storageStatus()
let network = await self.networkStatus.currentStatus()
let uptime = ProcessInfo.processInfo.systemUptime
return OpenClawDeviceStatusPayload(
battery: battery,
thermal: thermal,
storage: storage,
network: network,
uptimeSeconds: uptime)
}
func info() -> OpenClawDeviceInfoPayload {
let device = UIDevice.current
let appVersion = Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "dev"
let appBuild = Bundle.main.infoDictionary?["CFBundleVersion"] as? String ?? "0"
let locale = Locale.preferredLanguages.first ?? Locale.current.identifier
return OpenClawDeviceInfoPayload(
deviceName: device.name,
modelIdentifier: Self.modelIdentifier(),
systemName: device.systemName,
systemVersion: device.systemVersion,
appVersion: appVersion,
appBuild: appBuild,
locale: locale)
}
private func batteryStatus() -> OpenClawBatteryStatusPayload {
let device = UIDevice.current
device.isBatteryMonitoringEnabled = true
let level = device.batteryLevel >= 0 ? Double(device.batteryLevel) : nil
let state: OpenClawBatteryState = switch device.batteryState {
case .charging: .charging
case .full: .full
case .unplugged: .unplugged
case .unknown: .unknown
@unknown default: .unknown
}
return OpenClawBatteryStatusPayload(
level: level,
state: state,
lowPowerModeEnabled: ProcessInfo.processInfo.isLowPowerModeEnabled)
}
private func thermalStatus() -> OpenClawThermalStatusPayload {
let state: OpenClawThermalState = switch ProcessInfo.processInfo.thermalState {
case .nominal: .nominal
case .fair: .fair
case .serious: .serious
case .critical: .critical
@unknown default: .nominal
}
return OpenClawThermalStatusPayload(state: state)
}
private func storageStatus() -> OpenClawStorageStatusPayload {
let attrs = (try? FileManager.default.attributesOfFileSystem(forPath: NSHomeDirectory())) ?? [:]
let total = (attrs[.systemSize] as? NSNumber)?.int64Value ?? 0
let free = (attrs[.systemFreeSize] as? NSNumber)?.int64Value ?? 0
let used = max(0, total - free)
return OpenClawStorageStatusPayload(totalBytes: total, freeBytes: free, usedBytes: used)
}
private static func modelIdentifier() -> String {
var systemInfo = utsname()
uname(&systemInfo)
let machine = withUnsafeBytes(of: &systemInfo.machine) { ptr in
String(bytes: ptr.prefix { $0 != 0 }, encoding: .utf8)
}
let trimmed = machine?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
return trimmed.isEmpty ? "unknown" : trimmed
}
}

69
apps/ios/Sources/Device/NetworkStatusService.swift
View File

@@ -1,69 +0,0 @@
import Foundation
import Network
import OpenClawKit
final class NetworkStatusService: @unchecked Sendable {
func currentStatus(timeoutMs: Int = 1500) async -> OpenClawNetworkStatusPayload {
await withCheckedContinuation { cont in
let monitor = NWPathMonitor()
let queue = DispatchQueue(label: "bot.molt.ios.network-status")
let state = NetworkStatusState()
monitor.pathUpdateHandler = { path in
guard state.markCompleted() else { return }
monitor.cancel()
cont.resume(returning: Self.payload(from: path))
}
monitor.start(queue: queue)
queue.asyncAfter(deadline: .now() + .milliseconds(timeoutMs)) {
guard state.markCompleted() else { return }
monitor.cancel()
cont.resume(returning: Self.fallbackPayload())
}
}
}
private static func payload(from path: NWPath) -> OpenClawNetworkStatusPayload {
let status: OpenClawNetworkPathStatus = switch path.status {
case .satisfied: .satisfied
case .requiresConnection: .requiresConnection
case .unsatisfied: .unsatisfied
@unknown default: .unsatisfied
}
var interfaces: [OpenClawNetworkInterfaceType] = []
if path.usesInterfaceType(.wifi) { interfaces.append(.wifi) }
if path.usesInterfaceType(.cellular) { interfaces.append(.cellular) }
if path.usesInterfaceType(.wiredEthernet) { interfaces.append(.wired) }
if interfaces.isEmpty { interfaces.append(.other) }
return OpenClawNetworkStatusPayload(
status: status,
isExpensive: path.isExpensive,
isConstrained: path.isConstrained,
interfaces: interfaces)
}
private static func fallbackPayload() -> OpenClawNetworkStatusPayload {
OpenClawNetworkStatusPayload(
status: .unsatisfied,
isExpensive: false,
isConstrained: false,
interfaces: [.other])
}
}
private final class NetworkStatusState: @unchecked Sendable {
private let lock = NSLock()
private var completed = false
func markCompleted() -> Bool {
self.lock.lock()
defer { self.lock.unlock() }
if self.completed { return false }
self.completed = true
return true
}
}

48
apps/ios/Sources/Device/NodeDisplayName.swift
View File

@@ -1,48 +0,0 @@
import Foundation
import UIKit
enum NodeDisplayName {
private static let genericNames: Set<String> = ["iOS Node", "iPhone Node", "iPad Node"]
static func isGeneric(_ name: String) -> Bool {
Self.genericNames.contains(name)
}
static func defaultValue(for interfaceIdiom: UIUserInterfaceIdiom) -> String {
switch interfaceIdiom {
case .phone:
return "iPhone Node"
case .pad:
return "iPad Node"
default:
return "iOS Node"
}
}
static func resolve(
existing: String?,
deviceName: String,
interfaceIdiom: UIUserInterfaceIdiom
) -> String {
let trimmedExisting = existing?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
if !trimmedExisting.isEmpty, !Self.isGeneric(trimmedExisting) {
return trimmedExisting
}
let trimmedDevice = deviceName.trimmingCharacters(in: .whitespacesAndNewlines)
if let normalized = Self.normalizedDeviceName(trimmedDevice) {
return normalized
}
return Self.defaultValue(for: interfaceIdiom)
}
private static func normalizedDeviceName(_ deviceName: String) -> String? {
guard !deviceName.isEmpty else { return nil }
let lower = deviceName.lowercased()
if lower.contains("iphone") || lower.contains("ipad") || lower.contains("ios") {
return deviceName
}
return nil
}
}

312
apps/ios/Sources/Gateway/GatewayConnectionController.swift
View File

@@ -1,15 +1,8 @@
import AVFoundation
import Contacts
import CoreLocation
import CoreMotion
import EventKit
import Foundation
import OpenClawKit
import Darwin
import Foundation
import Network
import Observation
import Photos
import ReplayKit
import Speech
import SwiftUI
import UIKit
@@ -67,11 +60,6 @@ final class GatewayConnectionController {
port: port,
useTLS: tlsParams?.required == true)
else { return }
GatewaySettingsStore.saveLastGatewayConnection(
host: host,
port: port,
useTLS: tlsParams?.required == true,
stableID: gateway.stableID)
self.didAutoConnect = true
self.startAutoConnect(
url: url,
@@ -86,24 +74,13 @@ final class GatewayConnectionController {
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let token = GatewaySettingsStore.loadGatewayToken(instanceId: instanceId)
let password = GatewaySettingsStore.loadGatewayPassword(instanceId: instanceId)
let resolvedUseTLS = useTLS || self.shouldForceTLS(host: host)
guard let resolvedPort = self.resolveManualPort(host: host, port: port, useTLS: resolvedUseTLS)
else { return }
let stableID = self.manualStableID(host: host, port: resolvedPort)
let tlsParams = self.resolveManualTLSParams(
stableID: stableID,
tlsEnabled: resolvedUseTLS,
allowTOFUReset: self.shouldForceTLS(host: host))
let stableID = self.manualStableID(host: host, port: port)
let tlsParams = self.resolveManualTLSParams(stableID: stableID, tlsEnabled: useTLS)
guard let url = self.buildGatewayURL(
host: host,
port: resolvedPort,
port: port,
useTLS: tlsParams?.required == true)
else { return }
GatewaySettingsStore.saveLastGatewayConnection(
host: host,
port: resolvedPort,
useTLS: tlsParams?.required == true,
stableID: stableID)
self.didAutoConnect = true
self.startAutoConnect(
url: url,
@@ -113,38 +90,6 @@ final class GatewayConnectionController {
password: password)
}
func connectLastKnown() async {
guard let last = GatewaySettingsStore.loadLastGatewayConnection() else { return }
let instanceId = UserDefaults.standard.string(forKey: "node.instanceId")?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let token = GatewaySettingsStore.loadGatewayToken(instanceId: instanceId)
let password = GatewaySettingsStore.loadGatewayPassword(instanceId: instanceId)
let resolvedUseTLS = last.useTLS || self.shouldForceTLS(host: last.host)
let tlsParams = self.resolveManualTLSParams(
stableID: last.stableID,
tlsEnabled: resolvedUseTLS,
allowTOFUReset: self.shouldForceTLS(host: last.host))
guard let url = self.buildGatewayURL(
host: last.host,
port: last.port,
useTLS: tlsParams?.required == true)
else { return }
if resolvedUseTLS != last.useTLS {
GatewaySettingsStore.saveLastGatewayConnection(
host: last.host,
port: last.port,
useTLS: resolvedUseTLS,
stableID: last.stableID)
}
self.didAutoConnect = true
self.startAutoConnect(
url: url,
gatewayStableID: last.stableID,
tls: tlsParams,
token: token,
password: password)
}
private func updateFromDiscovery() {
let newGateways = self.discovery.gateways
self.gateways = newGateways
@@ -189,19 +134,11 @@ final class GatewayConnectionController {
guard !manualHost.isEmpty else { return }
let manualPort = defaults.integer(forKey: "gateway.manual.port")
let resolvedPort = manualPort > 0 ? manualPort : 18789
let manualTLS = defaults.bool(forKey: "gateway.manual.tls")
let resolvedUseTLS = manualTLS || self.shouldForceTLS(host: manualHost)
guard let resolvedPort = self.resolveManualPort(
host: manualHost,
port: manualPort,
useTLS: resolvedUseTLS)
else { return }
let stableID = self.manualStableID(host: manualHost, port: resolvedPort)
let tlsParams = self.resolveManualTLSParams(
stableID: stableID,
tlsEnabled: resolvedUseTLS,
allowTOFUReset: self.shouldForceTLS(host: manualHost))
let tlsParams = self.resolveManualTLSParams(stableID: stableID, tlsEnabled: manualTLS)
guard let url = self.buildGatewayURL(
host: manualHost,
@@ -219,70 +156,30 @@ final class GatewayConnectionController {
return
}
if let lastKnown = GatewaySettingsStore.loadLastGatewayConnection() {
let resolvedUseTLS = lastKnown.useTLS || self.shouldForceTLS(host: lastKnown.host)
let tlsParams = self.resolveManualTLSParams(
stableID: lastKnown.stableID,
tlsEnabled: resolvedUseTLS,
allowTOFUReset: self.shouldForceTLS(host: lastKnown.host))
guard let url = self.buildGatewayURL(
host: lastKnown.host,
port: lastKnown.port,
useTLS: tlsParams?.required == true)
else { return }
self.didAutoConnect = true
self.startAutoConnect(
url: url,
gatewayStableID: lastKnown.stableID,
tls: tlsParams,
token: token,
password: password)
return
}
let preferredStableID = defaults.string(forKey: "gateway.preferredStableID")?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let lastDiscoveredStableID = defaults.string(forKey: "gateway.lastDiscoveredStableID")?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let candidates = [preferredStableID, lastDiscoveredStableID].filter { !$0.isEmpty }
if let targetStableID = candidates.first(where: { id in
guard let targetStableID = candidates.first(where: { id in
self.gateways.contains(where: { $0.stableID == id })
}) {
guard let target = self.gateways.first(where: { $0.stableID == targetStableID }) else { return }
guard let host = self.resolveGatewayHost(target) else { return }
let port = target.gatewayPort ?? 18789
let tlsParams = self.resolveDiscoveredTLSParams(gateway: target)
guard let url = self.buildGatewayURL(host: host, port: port, useTLS: tlsParams?.required == true)
else { return }
}) else { return }
self.didAutoConnect = true
self.startAutoConnect(
url: url,
gatewayStableID: target.stableID,
tls: tlsParams,
token: token,
password: password)
return
}
guard let target = self.gateways.first(where: { $0.stableID == targetStableID }) else { return }
guard let host = self.resolveGatewayHost(target) else { return }
let port = target.gatewayPort ?? 18789
let tlsParams = self.resolveDiscoveredTLSParams(gateway: target)
guard let url = self.buildGatewayURL(host: host, port: port, useTLS: tlsParams?.required == true)
else { return }
if self.gateways.count == 1, let gateway = self.gateways.first {
guard let host = self.resolveGatewayHost(gateway) else { return }
let port = gateway.gatewayPort ?? 18789
let tlsParams = self.resolveDiscoveredTLSParams(gateway: gateway)
guard let url = self.buildGatewayURL(host: host, port: port, useTLS: tlsParams?.required == true)
else { return }
self.didAutoConnect = true
self.startAutoConnect(
url: url,
gatewayStableID: gateway.stableID,
tls: tlsParams,
token: token,
password: password)
return
}
self.didAutoConnect = true
self.startAutoConnect(
url: url,
gatewayStableID: target.stableID,
tls: tlsParams,
token: token,
password: password)
}
private func updateLastDiscoveredGateway(from gateways: [GatewayDiscoveryModel.DiscoveredGateway]) {
@@ -308,10 +205,10 @@ final class GatewayConnectionController {
password: String?)
{
guard let appModel else { return }
let connectOptions = self.makeConnectOptions(stableID: gatewayStableID)
let connectOptions = self.makeConnectOptions()
Task { [weak appModel] in
guard let appModel else { return }
Task { [weak self] in
guard let self else { return }
await MainActor.run {
appModel.gatewayStatusText = "Connecting…"
}
@@ -340,17 +237,13 @@ final class GatewayConnectionController {
return nil
}
private func resolveManualTLSParams(
stableID: String,
tlsEnabled: Bool,
allowTOFUReset: Bool = false) -> GatewayTLSParams?
{
private func resolveManualTLSParams(stableID: String, tlsEnabled: Bool) -> GatewayTLSParams? {
let stored = GatewayTLSStore.loadFingerprint(stableID: stableID)
if tlsEnabled || stored != nil {
return GatewayTLSParams(
required: true,
expectedFingerprint: stored,
allowTOFU: stored == nil || allowTOFUReset,
allowTOFU: stored == nil,
storeKey: stableID)
}
@@ -358,12 +251,12 @@ final class GatewayConnectionController {
}
private func resolveGatewayHost(_ gateway: GatewayDiscoveryModel.DiscoveredGateway) -> String? {
if let tailnet = gateway.tailnetDns?.trimmingCharacters(in: .whitespacesAndNewlines), !tailnet.isEmpty {
return tailnet
}
if let lanHost = gateway.lanHost?.trimmingCharacters(in: .whitespacesAndNewlines), !lanHost.isEmpty {
return lanHost
}
if let tailnet = gateway.tailnetDns?.trimmingCharacters(in: .whitespacesAndNewlines), !tailnet.isEmpty {
return tailnet
}
return nil
}
@@ -376,69 +269,38 @@ final class GatewayConnectionController {
return components.url
}
private func shouldForceTLS(host: String) -> Bool {
let trimmed = host.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
if trimmed.isEmpty { return false }
return trimmed.hasSuffix(".ts.net") || trimmed.hasSuffix(".ts.net.")
}
private func manualStableID(host: String, port: Int) -> String {
"manual|\(host.lowercased())|\(port)"
}
private func makeConnectOptions(stableID: String?) -> GatewayConnectOptions {
private func makeConnectOptions() -> GatewayConnectOptions {
let defaults = UserDefaults.standard
let displayName = self.resolvedDisplayName(defaults: defaults)
let resolvedClientId = self.resolvedClientId(defaults: defaults, stableID: stableID)
return GatewayConnectOptions(
role: "node",
scopes: [],
caps: self.currentCaps(),
commands: self.currentCommands(),
permissions: self.currentPermissions(),
clientId: resolvedClientId,
permissions: [:],
clientId: "openclaw-ios",
clientMode: "node",
clientDisplayName: displayName)
}
private func resolvedClientId(defaults: UserDefaults, stableID: String?) -> String {
if let stableID,
let override = GatewaySettingsStore.loadGatewayClientIdOverride(stableID: stableID) {
return override
}
let manualClientId = defaults.string(forKey: "gateway.manual.clientId")?
.trimmingCharacters(in: .whitespacesAndNewlines)
if manualClientId?.isEmpty == false {
return manualClientId!
}
return "openclaw-ios"
}
private func resolveManualPort(host: String, port: Int, useTLS: Bool) -> Int? {
if port > 0 {
return port <= 65535 ? port : nil
}
let trimmedHost = host.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmedHost.isEmpty else { return nil }
if useTLS && self.shouldForceTLS(host: trimmedHost) {
return 443
}
return 18789
}
private func resolvedDisplayName(defaults: UserDefaults) -> String {
let key = "node.displayName"
let existingRaw = defaults.string(forKey: key)
let resolved = NodeDisplayName.resolve(
existing: existingRaw,
deviceName: UIDevice.current.name,
interfaceIdiom: UIDevice.current.userInterfaceIdiom)
let existing = existingRaw?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
if existing.isEmpty || NodeDisplayName.isGeneric(existing) {
defaults.set(resolved, forKey: key)
let existing = defaults.string(forKey: key)?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
if !existing.isEmpty, existing != "iOS Node" { return existing }
let deviceName = UIDevice.current.name.trimmingCharacters(in: .whitespacesAndNewlines)
let candidate = deviceName.isEmpty ? "iOS Node" : deviceName
if existing.isEmpty || existing == "iOS Node" {
defaults.set(candidate, forKey: key)
}
return resolved
return candidate
}
private func currentCaps() -> [String] {
@@ -458,15 +320,6 @@ final class GatewayConnectionController {
let locationMode = OpenClawLocationMode(rawValue: locationModeRaw) ?? .off
if locationMode != .off { caps.append(OpenClawCapability.location.rawValue) }
caps.append(OpenClawCapability.device.rawValue)
caps.append(OpenClawCapability.photos.rawValue)
caps.append(OpenClawCapability.contacts.rawValue)
caps.append(OpenClawCapability.calendar.rawValue)
caps.append(OpenClawCapability.reminders.rawValue)
if Self.motionAvailable() {
caps.append(OpenClawCapability.motion.rawValue)
}
return caps
}
@@ -482,11 +335,10 @@ final class GatewayConnectionController {
OpenClawCanvasA2UICommand.reset.rawValue,
OpenClawScreenCommand.record.rawValue,
OpenClawSystemCommand.notify.rawValue,
OpenClawChatCommand.push.rawValue,
OpenClawTalkCommand.pttStart.rawValue,
OpenClawTalkCommand.pttStop.rawValue,
OpenClawTalkCommand.pttCancel.rawValue,
OpenClawTalkCommand.pttOnce.rawValue,
OpenClawSystemCommand.which.rawValue,
OpenClawSystemCommand.run.rawValue,
OpenClawSystemCommand.execApprovalsGet.rawValue,
OpenClawSystemCommand.execApprovalsSet.rawValue,
]
let caps = Set(self.currentCaps())
@@ -498,76 +350,10 @@ final class GatewayConnectionController {
if caps.contains(OpenClawCapability.location.rawValue) {
commands.append(OpenClawLocationCommand.get.rawValue)
}
if caps.contains(OpenClawCapability.device.rawValue) {
commands.append(OpenClawDeviceCommand.status.rawValue)
commands.append(OpenClawDeviceCommand.info.rawValue)
}
if caps.contains(OpenClawCapability.photos.rawValue) {
commands.append(OpenClawPhotosCommand.latest.rawValue)
}
if caps.contains(OpenClawCapability.contacts.rawValue) {
commands.append(OpenClawContactsCommand.search.rawValue)
commands.append(OpenClawContactsCommand.add.rawValue)
}
if caps.contains(OpenClawCapability.calendar.rawValue) {
commands.append(OpenClawCalendarCommand.events.rawValue)
commands.append(OpenClawCalendarCommand.add.rawValue)
}
if caps.contains(OpenClawCapability.reminders.rawValue) {
commands.append(OpenClawRemindersCommand.list.rawValue)
commands.append(OpenClawRemindersCommand.add.rawValue)
}
if caps.contains(OpenClawCapability.motion.rawValue) {
commands.append(OpenClawMotionCommand.activity.rawValue)
commands.append(OpenClawMotionCommand.pedometer.rawValue)
}
return commands
}
private func currentPermissions() -> [String: Bool] {
var permissions: [String: Bool] = [:]
permissions["camera"] = AVCaptureDevice.authorizationStatus(for: .video) == .authorized
permissions["microphone"] = AVCaptureDevice.authorizationStatus(for: .audio) == .authorized
permissions["speechRecognition"] = SFSpeechRecognizer.authorizationStatus() == .authorized
permissions["location"] = Self.isLocationAuthorized(
status: CLLocationManager().authorizationStatus)
&& CLLocationManager.locationServicesEnabled()
permissions["screenRecording"] = RPScreenRecorder.shared().isAvailable
let photoStatus = PHPhotoLibrary.authorizationStatus(for: .readWrite)
permissions["photos"] = photoStatus == .authorized || photoStatus == .limited
let contactsStatus = CNContactStore.authorizationStatus(for: .contacts)
permissions["contacts"] = contactsStatus == .authorized || contactsStatus == .limited
let calendarStatus = EKEventStore.authorizationStatus(for: .event)
permissions["calendar"] =
calendarStatus == .authorized || calendarStatus == .fullAccess || calendarStatus == .writeOnly
let remindersStatus = EKEventStore.authorizationStatus(for: .reminder)
permissions["reminders"] =
remindersStatus == .authorized || remindersStatus == .fullAccess || remindersStatus == .writeOnly
let motionStatus = CMMotionActivityManager.authorizationStatus()
let pedometerStatus = CMPedometer.authorizationStatus()
permissions["motion"] =
motionStatus == .authorized || pedometerStatus == .authorized
return permissions
}
private static func isLocationAuthorized(status: CLAuthorizationStatus) -> Bool {
switch status {
case .authorizedAlways, .authorizedWhenInUse, .authorized:
return true
default:
return false
}
}
private static func motionAvailable() -> Bool {
CMMotionActivityManager.isActivityAvailable() || CMPedometer.isStepCountingAvailable()
}
private func platformString() -> String {
let v = ProcessInfo.processInfo.operatingSystemVersion
let name = switch UIDevice.current.userInterfaceIdiom {
@@ -621,10 +407,6 @@ extension GatewayConnectionController {
self.currentCommands()
}
func _test_currentPermissions() -> [String: Bool] {
self.currentPermissions()
}
func _test_platformString() -> String {
self.platformString()
}

85
apps/ios/Sources/Gateway/GatewayHealthMonitor.swift
View File

@@ -1,85 +0,0 @@
import Foundation
import OpenClawKit
@MainActor
final class GatewayHealthMonitor {
struct Config: Sendable {
var intervalSeconds: Double
var timeoutSeconds: Double
var maxFailures: Int
}
private let config: Config
private let sleep: @Sendable (UInt64) async -> Void
private var task: Task<Void, Never>?
init(
config: Config = Config(intervalSeconds: 15, timeoutSeconds: 5, maxFailures: 3),
sleep: @escaping @Sendable (UInt64) async -> Void = { nanoseconds in
try? await Task.sleep(nanoseconds: nanoseconds)
}
) {
self.config = config
self.sleep = sleep
}
func start(
check: @escaping @Sendable () async throws -> Bool,
onFailure: @escaping @Sendable (_ failureCount: Int) async -> Void)
{
self.stop()
let config = self.config
let sleep = self.sleep
self.task = Task { @MainActor in
var failures = 0
while !Task.isCancelled {
let ok = await Self.runCheck(check: check, timeoutSeconds: config.timeoutSeconds)
if ok {
failures = 0
} else {
failures += 1
if failures >= max(1, config.maxFailures) {
await onFailure(failures)
failures = 0
}
}
if Task.isCancelled { break }
let interval = max(0.0, config.intervalSeconds)
let nanos = UInt64(interval * 1_000_000_000)
if nanos > 0 {
await sleep(nanos)
} else {
await Task.yield()
}
}
}
}
func stop() {
self.task?.cancel()
self.task = nil
}
private static func runCheck(
check: @escaping @Sendable () async throws -> Bool,
timeoutSeconds: Double) async -> Bool
{
let timeout = max(0.0, timeoutSeconds)
if timeout == 0 {
return (try? await check()) ?? false
}
do {
let timeoutError = NSError(
domain: "GatewayHealthMonitor",
code: 1,
userInfo: [NSLocalizedDescriptionKey: "health check timed out"])
return try await AsyncTimeout.withTimeout(
seconds: timeout,
onTimeout: { timeoutError },
operation: check)
} catch {
return false
}
}
}

69
apps/ios/Sources/Gateway/GatewaySettingsStore.swift
View File

@@ -11,13 +11,7 @@ enum GatewaySettingsStore {
private static let manualHostDefaultsKey = "gateway.manual.host"
private static let manualPortDefaultsKey = "gateway.manual.port"
private static let manualTlsDefaultsKey = "gateway.manual.tls"
private static let manualPasswordDefaultsKey = "gateway.manual.password"
private static let discoveryDebugLogsDefaultsKey = "gateway.discovery.debugLogs"
private static let lastGatewayHostDefaultsKey = "gateway.last.host"
private static let lastGatewayPortDefaultsKey = "gateway.last.port"
private static let lastGatewayTlsDefaultsKey = "gateway.last.tls"
private static let lastGatewayStableIDDefaultsKey = "gateway.last.stableID"
private static let clientIdOverrideDefaultsPrefix = "gateway.clientIdOverride."
private static let instanceIdAccount = "instanceId"
private static let preferredGatewayStableIDAccount = "preferredStableID"
@@ -27,7 +21,6 @@ enum GatewaySettingsStore {
self.ensureStableInstanceID()
self.ensurePreferredGatewayStableID()
self.ensureLastDiscoveredGatewayStableID()
self.ensureManualGatewayPassword()
}
static func loadStableInstanceID() -> String? {
@@ -114,49 +107,6 @@ enum GatewaySettingsStore {
account: self.gatewayPasswordAccount(instanceId: instanceId))
}
static func saveLastGatewayConnection(host: String, port: Int, useTLS: Bool, stableID: String) {
let defaults = UserDefaults.standard
defaults.set(host, forKey: self.lastGatewayHostDefaultsKey)
defaults.set(port, forKey: self.lastGatewayPortDefaultsKey)
defaults.set(useTLS, forKey: self.lastGatewayTlsDefaultsKey)
defaults.set(stableID, forKey: self.lastGatewayStableIDDefaultsKey)
}
static func loadLastGatewayConnection() -> (host: String, port: Int, useTLS: Bool, stableID: String)? {
let defaults = UserDefaults.standard
let host = defaults.string(forKey: self.lastGatewayHostDefaultsKey)?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let port = defaults.integer(forKey: self.lastGatewayPortDefaultsKey)
let useTLS = defaults.bool(forKey: self.lastGatewayTlsDefaultsKey)
let stableID = defaults.string(forKey: self.lastGatewayStableIDDefaultsKey)?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !host.isEmpty, port > 0, port <= 65535, !stableID.isEmpty else { return nil }
return (host: host, port: port, useTLS: useTLS, stableID: stableID)
}
static func loadGatewayClientIdOverride(stableID: String) -> String? {
let trimmedID = stableID.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmedID.isEmpty else { return nil }
let key = self.clientIdOverrideDefaultsPrefix + trimmedID
let value = UserDefaults.standard.string(forKey: key)?
.trimmingCharacters(in: .whitespacesAndNewlines)
if value?.isEmpty == false { return value }
return nil
}
static func saveGatewayClientIdOverride(stableID: String, clientId: String?) {
let trimmedID = stableID.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmedID.isEmpty else { return }
let key = self.clientIdOverrideDefaultsPrefix + trimmedID
let trimmedClientId = clientId?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
if trimmedClientId.isEmpty {
UserDefaults.standard.removeObject(forKey: key)
} else {
UserDefaults.standard.set(trimmedClientId, forKey: key)
}
}
private static func gatewayTokenAccount(instanceId: String) -> String {
"gateway-token.\(instanceId)"
}
@@ -224,23 +174,4 @@ enum GatewaySettingsStore {
}
}
private static func ensureManualGatewayPassword() {
let defaults = UserDefaults.standard
let instanceId = defaults.string(forKey: self.instanceIdDefaultsKey)?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !instanceId.isEmpty else { return }
let manualPassword = defaults.string(forKey: self.manualPasswordDefaultsKey)?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !manualPassword.isEmpty else { return }
if self.loadGatewayPassword(instanceId: instanceId) == nil {
self.saveGatewayPassword(manualPassword, instanceId: instanceId)
}
if self.loadGatewayPassword(instanceId: instanceId) == manualPassword {
defaults.removeObject(forKey: self.manualPasswordDefaultsKey)
}
}
}

103
apps/ios/Sources/Media/PhotoLibraryService.swift
View File

@@ -1,103 +0,0 @@
import Foundation
import Photos
import OpenClawKit
import UIKit
final class PhotoLibraryService: PhotosServicing {
func latest(params: OpenClawPhotosLatestParams) async throws -> OpenClawPhotosLatestPayload {
let status = await Self.ensureAuthorization()
guard status == .authorized || status == .limited else {
throw NSError(domain: "Photos", code: 1, userInfo: [
NSLocalizedDescriptionKey: "PHOTOS_PERMISSION_REQUIRED: grant Photos permission",
])
}
let limit = max(1, min(params.limit ?? 1, 20))
let fetchOptions = PHFetchOptions()
fetchOptions.fetchLimit = limit
fetchOptions.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: false)]
let assets = PHAsset.fetchAssets(with: .image, options: fetchOptions)
var results: [OpenClawPhotoPayload] = []
let maxWidth = params.maxWidth.flatMap { $0 > 0 ? $0 : nil } ?? 1600
let quality = params.quality.map { max(0.1, min(1.0, $0)) } ?? 0.85
let formatter = ISO8601DateFormatter()
assets.enumerateObjects { asset, _, stop in
if results.count >= limit { stop.pointee = true; return }
if let payload = try? Self.renderAsset(
asset,
maxWidth: maxWidth,
quality: quality,
formatter: formatter)
{
results.append(payload)
}
}
return OpenClawPhotosLatestPayload(photos: results)
}
private static func ensureAuthorization() async -> PHAuthorizationStatus {
let current = PHPhotoLibrary.authorizationStatus(for: .readWrite)
if current == .notDetermined {
return await withCheckedContinuation { cont in
PHPhotoLibrary.requestAuthorization(for: .readWrite) { status in
cont.resume(returning: status)
}
}
}
return current
}
private static func renderAsset(
_ asset: PHAsset,
maxWidth: Int,
quality: Double,
formatter: ISO8601DateFormatter) throws -> OpenClawPhotoPayload
{
let manager = PHImageManager.default()
let options = PHImageRequestOptions()
options.isSynchronous = true
options.isNetworkAccessAllowed = true
options.deliveryMode = .highQualityFormat
let targetSize: CGSize = {
guard maxWidth > 0 else { return PHImageManagerMaximumSize }
let aspect = CGFloat(asset.pixelHeight) / CGFloat(max(1, asset.pixelWidth))
let width = CGFloat(maxWidth)
return CGSize(width: width, height: width * aspect)
}()
var image: UIImage?
manager.requestImage(
for: asset,
targetSize: targetSize,
contentMode: .aspectFit,
options: options)
{ result, _ in
image = result
}
guard let image else {
throw NSError(domain: "Photos", code: 2, userInfo: [
NSLocalizedDescriptionKey: "photo load failed",
])
}
let jpeg = image.jpegData(compressionQuality: quality)
guard let data = jpeg else {
throw NSError(domain: "Photos", code: 3, userInfo: [
NSLocalizedDescriptionKey: "photo encode failed",
])
}
let created = asset.creationDate.map { formatter.string(from: $0) }
return OpenClawPhotoPayload(
format: "jpeg",
base64: data.base64EncodedString(),
width: Int(image.size.width),
height: Int(image.size.height),
createdAt: created)
}
}

609
apps/ios/Sources/Model/NodeAppModel.swift
View File

@@ -1,41 +1,8 @@
import OpenClawChatUI
import OpenClawKit
import Network
import Observation
import SwiftUI
import UIKit
import UserNotifications
// Wrap errors without pulling non-Sendable types into async notification paths.
private struct NotificationCallError: Error, Sendable {
let message: String
}
// Ensures notification requests return promptly even if the system prompt blocks.
private final class NotificationInvokeLatch<T: Sendable>: @unchecked Sendable {
private let lock = NSLock()
private var continuation: CheckedContinuation<Result<T, NotificationCallError>, Never>?
private var resumed = false
func setContinuation(_ continuation: CheckedContinuation<Result<T, NotificationCallError>, Never>) {
self.lock.lock()
defer { self.lock.unlock() }
self.continuation = continuation
}
func resume(_ response: Result<T, NotificationCallError>) {
let cont: CheckedContinuation<Result<T, NotificationCallError>, Never>?
self.lock.lock()
if self.resumed {
self.lock.unlock()
return
}
self.resumed = true
cont = self.continuation
self.continuation = nil
self.lock.unlock()
cont?.resume(returning: response)
}
}
@MainActor
@Observable
@@ -48,9 +15,9 @@ final class NodeAppModel {
}
var isBackgrounded: Bool = false
let screen: ScreenController
private let camera: any CameraServicing
private let screenRecorder: any ScreenRecordingServicing
let screen = ScreenController()
let camera = CameraController()
private let screenRecorder = ScreenRecordService()
var gatewayStatusText: String = "Offline"
var gatewayServerName: String?
var gatewayRemoteAddress: String?
@@ -62,20 +29,10 @@ final class NodeAppModel {
private var gatewayTask: Task<Void, Never>?
private var voiceWakeSyncTask: Task<Void, Never>?
@ObservationIgnored private var cameraHUDDismissTask: Task<Void, Never>?
@ObservationIgnored private lazy var capabilityRouter: NodeCapabilityRouter = self.buildCapabilityRouter()
private let gatewayHealthMonitor = GatewayHealthMonitor()
private let notificationCenter: NotificationCentering
let voiceWake = VoiceWakeManager()
let talkMode: TalkModeManager
private let locationService: any LocationServicing
private let deviceStatusService: any DeviceStatusServicing
private let photosService: any PhotosServicing
private let contactsService: any ContactsServicing
private let calendarService: any CalendarServicing
private let remindersService: any RemindersServicing
private let motionService: any MotionServicing
let talkMode = TalkModeManager()
private let locationService = LocationService()
private var lastAutoA2uiURL: String?
private var pttVoiceWakeSuspended = false
private var gatewayConnected = false
var gatewaySession: GatewayNodeSession { self.gateway }
@@ -85,33 +42,7 @@ final class NodeAppModel {
var cameraFlashNonce: Int = 0
var screenRecordActive: Bool = false
init(
screen: ScreenController = ScreenController(),
camera: any CameraServicing = CameraController(),
screenRecorder: any ScreenRecordingServicing = ScreenRecordService(),
locationService: any LocationServicing = LocationService(),
notificationCenter: NotificationCentering = LiveNotificationCenter(),
deviceStatusService: any DeviceStatusServicing = DeviceStatusService(),
photosService: any PhotosServicing = PhotoLibraryService(),
contactsService: any ContactsServicing = ContactsService(),
calendarService: any CalendarServicing = CalendarService(),
remindersService: any RemindersServicing = RemindersService(),
motionService: any MotionServicing = MotionService(),
talkMode: TalkModeManager = TalkModeManager())
{
self.screen = screen
self.camera = camera
self.screenRecorder = screenRecorder
self.locationService = locationService
self.notificationCenter = notificationCenter
self.deviceStatusService = deviceStatusService
self.photosService = photosService
self.contactsService = contactsService
self.calendarService = calendarService
self.remindersService = remindersService
self.motionService = motionService
self.talkMode = talkMode
init() {
self.voiceWake.configure { [weak self] cmd in
guard let self else { return }
let sessionKey = await MainActor.run { self.mainSessionKey }
@@ -176,10 +107,7 @@ final class NodeAppModel {
return raw.isEmpty ? "-" : raw
}()
let host = NodeDisplayName.resolve(
existing: UserDefaults.standard.string(forKey: "node.displayName"),
deviceName: UIDevice.current.name,
interfaceIdiom: UIDevice.current.userInterfaceIdiom)
let host = UserDefaults.standard.string(forKey: "node.displayName") ?? UIDevice.current.name
let instanceId = (UserDefaults.standard.string(forKey: "node.instanceId") ?? "ios-node").lowercased()
let contextJSON = OpenClawCanvasA2UIAction.compactJSON(userAction["context"])
let sessionKey = self.mainSessionKey
@@ -247,12 +175,8 @@ final class NodeAppModel {
switch phase {
case .background:
self.isBackgrounded = true
self.stopGatewayHealthMonitor()
case .active, .inactive:
self.isBackgrounded = false
if self.gatewayConnected {
self.startGatewayHealthMonitor()
}
@unknown default:
self.isBackgrounded = false
}
@@ -288,7 +212,6 @@ final class NodeAppModel {
connectOptions: GatewayConnectOptions)
{
self.gatewayTask?.cancel()
self.gatewayHealthMonitor.stop()
self.gatewayServerName = nil
self.gatewayRemoteAddress = nil
let id = gatewayStableID.trimmingCharacters(in: .whitespacesAndNewlines)
@@ -300,9 +223,6 @@ final class NodeAppModel {
self.gatewayTask = Task {
var attempt = 0
var currentOptions = connectOptions
var didFallbackClientId = false
let trimmedStableID = gatewayStableID.trimmingCharacters(in: .whitespacesAndNewlines)
while !Task.isCancelled {
await MainActor.run {
if attempt == 0 {
@@ -319,7 +239,7 @@ final class NodeAppModel {
url: url,
token: token,
password: password,
connectOptions: currentOptions,
connectOptions: connectOptions,
sessionBox: sessionBox,
onConnected: { [weak self] in
guard let self else { return }
@@ -327,7 +247,6 @@ final class NodeAppModel {
self.gatewayStatusText = "Connected"
self.gatewayServerName = url.host ?? "gateway"
self.gatewayConnected = true
self.talkMode.updateGatewayConnected(true)
}
if let addr = await self.gateway.currentRemoteAddress() {
await MainActor.run {
@@ -336,7 +255,6 @@ final class NodeAppModel {
}
await self.refreshBrandingFromGateway()
await self.startVoiceWakeSync()
await MainActor.run { self.startGatewayHealthMonitor() }
await self.showA2UIOnConnectIfNeeded()
},
onDisconnected: { [weak self] reason in
@@ -345,11 +263,9 @@ final class NodeAppModel {
self.gatewayStatusText = "Disconnected"
self.gatewayRemoteAddress = nil
self.gatewayConnected = false
self.talkMode.updateGatewayConnected(false)
self.showLocalCanvasOnDisconnect()
self.gatewayStatusText = "Disconnected: \(reason)"
}
await MainActor.run { self.stopGatewayHealthMonitor() }
},
onInvoke: { [weak self] req in
guard let self else {
@@ -368,30 +284,12 @@ final class NodeAppModel {
try? await Task.sleep(nanoseconds: 1_000_000_000)
} catch {
if Task.isCancelled { break }
if !didFallbackClientId,
let fallbackClientId = self.legacyClientIdFallback(
currentClientId: currentOptions.clientId,
error: error)
{
didFallbackClientId = true
currentOptions.clientId = fallbackClientId
if !trimmedStableID.isEmpty {
GatewaySettingsStore.saveGatewayClientIdOverride(
stableID: trimmedStableID,
clientId: fallbackClientId)
}
await MainActor.run {
self.gatewayStatusText = "Gateway rejected client id. Retrying…"
}
continue
}
attempt += 1
await MainActor.run {
self.gatewayStatusText = "Gateway error: \(error.localizedDescription)"
self.gatewayServerName = nil
self.gatewayRemoteAddress = nil
self.gatewayConnected = false
self.talkMode.updateGatewayConnected(false)
self.showLocalCanvasOnDisconnect()
}
let sleepSeconds = min(8.0, 0.5 * pow(1.7, Double(attempt)))
@@ -405,7 +303,6 @@ final class NodeAppModel {
self.gatewayRemoteAddress = nil
self.connectedGatewayID = nil
self.gatewayConnected = false
self.talkMode.updateGatewayConnected(false)
self.seamColorHex = nil
if !SessionKey.isCanonicalMainSessionKey(self.mainSessionKey) {
self.mainSessionKey = "main"
@@ -416,29 +313,17 @@ final class NodeAppModel {
}
}
private func legacyClientIdFallback(currentClientId: String, error: Error) -> String? {
let normalizedClientId = currentClientId.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
guard normalizedClientId == "openclaw-ios" else { return nil }
let message = error.localizedDescription.lowercased()
guard message.contains("invalid connect params"), message.contains("/client/id") else {
return nil
}
return "moltbot-ios"
}
func disconnectGateway() {
self.gatewayTask?.cancel()
self.gatewayTask = nil
self.voiceWakeSyncTask?.cancel()
self.voiceWakeSyncTask = nil
self.gatewayHealthMonitor.stop()
Task { await self.gateway.disconnect() }
self.gatewayStatusText = "Offline"
self.gatewayServerName = nil
self.gatewayRemoteAddress = nil
self.connectedGatewayID = nil
self.gatewayConnected = false
self.talkMode.updateGatewayConnected(false)
self.seamColorHex = nil
if !SessionKey.isCanonicalMainSessionKey(self.mainSessionKey) {
self.mainSessionKey = "main"
@@ -533,30 +418,6 @@ final class NodeAppModel {
}
}
private func startGatewayHealthMonitor() {
self.gatewayHealthMonitor.start(
check: { [weak self] in
guard let self else { return false }
do {
let data = try await self.gateway.request(method: "health", paramsJSON: nil, timeoutSeconds: 6)
guard let decoded = try? JSONDecoder().decode(OpenClawGatewayHealthOK.self, from: data) else {
return false
}
return decoded.ok ?? false
} catch {
return false
}
},
onFailure: { [weak self] _ in
guard let self else { return }
await self.gateway.disconnect()
})
}
private func stopGatewayHealthMonitor() {
self.gatewayHealthMonitor.stop()
}
private func refreshWakeWordsFromGateway() async {
do {
let data = try await self.gateway.request(method: "voicewake.get", paramsJSON: "{}", timeoutSeconds: 8)
@@ -662,19 +523,30 @@ final class NodeAppModel {
}
do {
return try await self.capabilityRouter.handle(req)
} catch let error as NodeCapabilityRouter.RouterError {
switch error {
case .unknownCommand:
switch command {
case OpenClawLocationCommand.get.rawValue:
return try await self.handleLocationInvoke(req)
case OpenClawCanvasCommand.present.rawValue,
OpenClawCanvasCommand.hide.rawValue,
OpenClawCanvasCommand.navigate.rawValue,
OpenClawCanvasCommand.evalJS.rawValue,
OpenClawCanvasCommand.snapshot.rawValue:
return try await self.handleCanvasInvoke(req)
case OpenClawCanvasA2UICommand.reset.rawValue,
OpenClawCanvasA2UICommand.push.rawValue,
OpenClawCanvasA2UICommand.pushJSONL.rawValue:
return try await self.handleCanvasA2UIInvoke(req)
case OpenClawCameraCommand.list.rawValue,
OpenClawCameraCommand.snap.rawValue,
OpenClawCameraCommand.clip.rawValue:
return try await self.handleCameraInvoke(req)
case OpenClawScreenCommand.record.rawValue:
return try await self.handleScreenRecordInvoke(req)
default:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: unknown command"))
case .handlerUnavailable:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .unavailable, message: "node handler unavailable"))
}
} catch {
if command.hasPrefix("camera.") {
@@ -689,8 +561,7 @@ final class NodeAppModel {
}
private func isBackgroundRestricted(_ command: String) -> Bool {
command.hasPrefix("canvas.") || command.hasPrefix("camera.") || command.hasPrefix("screen.") ||
command.hasPrefix("talk.")
command.hasPrefix("canvas.") || command.hasPrefix("camera.") || command.hasPrefix("screen.")
}
private func handleLocationInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
@@ -755,7 +626,6 @@ final class NodeAppModel {
private func handleCanvasInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
switch req.command {
case OpenClawCanvasCommand.present.rawValue:
// iOS ignores placement hints; canvas always fills the screen.
let params = (try? Self.decodeParams(OpenClawCanvasPresentParams.self, from: req.paramsJSON)) ??
OpenClawCanvasPresentParams()
let url = params.url?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
@@ -766,7 +636,6 @@ final class NodeAppModel {
}
return BridgeInvokeResponse(id: req.id, ok: true)
case OpenClawCanvasCommand.hide.rawValue:
self.screen.showDefaultCanvas()
return BridgeInvokeResponse(id: req.id, ok: true)
case OpenClawCanvasCommand.navigate.rawValue:
let params = try Self.decodeParams(OpenClawCanvasNavigateParams.self, from: req.paramsJSON)
@@ -990,427 +859,9 @@ final class NodeAppModel {
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: payload)
}
private func handleSystemNotify(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
let params = try Self.decodeParams(OpenClawSystemNotifyParams.self, from: req.paramsJSON)
let title = params.title.trimmingCharacters(in: .whitespacesAndNewlines)
let body = params.body.trimmingCharacters(in: .whitespacesAndNewlines)
if title.isEmpty, body.isEmpty {
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: empty notification"))
}
let finalStatus = await self.requestNotificationAuthorizationIfNeeded()
guard finalStatus == .authorized || finalStatus == .provisional || finalStatus == .ephemeral else {
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .unavailable, message: "NOT_AUTHORIZED: notifications"))
}
let addResult = await self.runNotificationCall(timeoutSeconds: 2.0) { [notificationCenter] in
let content = UNMutableNotificationContent()
content.title = title
content.body = body
if #available(iOS 15.0, *) {
switch params.priority ?? .active {
case .passive:
content.interruptionLevel = .passive
case .timeSensitive:
content.interruptionLevel = .timeSensitive
case .active:
content.interruptionLevel = .active
}
}
let soundValue = params.sound?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
if let soundValue, ["none", "silent", "off", "false", "0"].contains(soundValue) {
content.sound = nil
} else {
content.sound = .default
}
let request = UNNotificationRequest(
identifier: UUID().uuidString,
content: content,
trigger: nil)
try await notificationCenter.add(request)
}
if case let .failure(error) = addResult {
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .unavailable, message: "NOTIFICATION_FAILED: \(error.message)"))
}
return BridgeInvokeResponse(id: req.id, ok: true)
}
private func handleChatPushInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
let params = try Self.decodeParams(OpenClawChatPushParams.self, from: req.paramsJSON)
let text = params.text.trimmingCharacters(in: .whitespacesAndNewlines)
guard !text.isEmpty else {
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: empty chat.push text"))
}
let finalStatus = await self.requestNotificationAuthorizationIfNeeded()
let messageId = UUID().uuidString
if finalStatus == .authorized || finalStatus == .provisional || finalStatus == .ephemeral {
let addResult = await self.runNotificationCall(timeoutSeconds: 2.0) { [notificationCenter] in
let content = UNMutableNotificationContent()
content.title = "OpenClaw"
content.body = text
content.sound = .default
content.userInfo = ["messageId": messageId]
let request = UNNotificationRequest(
identifier: messageId,
content: content,
trigger: nil)
try await notificationCenter.add(request)
}
if case let .failure(error) = addResult {
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .unavailable, message: "NOTIFICATION_FAILED: \(error.message)"))
}
}
if params.speak ?? true {
let toSpeak = text
Task { @MainActor in
try? await TalkSystemSpeechSynthesizer.shared.speak(text: toSpeak)
}
}
let payload = OpenClawChatPushPayload(messageId: messageId)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
}
private func requestNotificationAuthorizationIfNeeded() async -> NotificationAuthorizationStatus {
let status = await self.notificationAuthorizationStatus()
guard status == .notDetermined else { return status }
// Avoid hanging invoke requests if the permission prompt is never answered.
_ = await self.runNotificationCall(timeoutSeconds: 2.0) { [notificationCenter] in
_ = try await notificationCenter.requestAuthorization(options: [.alert, .sound, .badge])
}
return await self.notificationAuthorizationStatus()
}
private func notificationAuthorizationStatus() async -> NotificationAuthorizationStatus {
let result = await self.runNotificationCall(timeoutSeconds: 1.5) { [notificationCenter] in
await notificationCenter.authorizationStatus()
}
switch result {
case let .success(status):
return status
case .failure:
return .denied
}
}
private func runNotificationCall<T: Sendable>(
timeoutSeconds: Double,
operation: @escaping @Sendable () async throws -> T
) async -> Result<T, NotificationCallError> {
let latch = NotificationInvokeLatch<T>()
var opTask: Task<Void, Never>?
var timeoutTask: Task<Void, Never>?
defer {
opTask?.cancel()
timeoutTask?.cancel()
}
let clamped = max(0.0, timeoutSeconds)
return await withCheckedContinuation { (cont: CheckedContinuation<Result<T, NotificationCallError>, Never>) in
latch.setContinuation(cont)
opTask = Task { @MainActor in
do {
let value = try await operation()
latch.resume(.success(value))
} catch {
latch.resume(.failure(NotificationCallError(message: error.localizedDescription)))
}
}
timeoutTask = Task.detached {
if clamped > 0 {
try? await Task.sleep(nanoseconds: UInt64(clamped * 1_000_000_000))
}
latch.resume(.failure(NotificationCallError(message: "notification request timed out")))
}
}
}
private func handleDeviceInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
switch req.command {
case OpenClawDeviceCommand.status.rawValue:
let payload = try await self.deviceStatusService.status()
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawDeviceCommand.info.rawValue:
let payload = self.deviceStatusService.info()
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
default:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: unknown command"))
}
}
private func handlePhotosInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
let params = (try? Self.decodeParams(OpenClawPhotosLatestParams.self, from: req.paramsJSON)) ??
OpenClawPhotosLatestParams()
let payload = try await self.photosService.latest(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
}
private func handleContactsInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
switch req.command {
case OpenClawContactsCommand.search.rawValue:
let params = (try? Self.decodeParams(OpenClawContactsSearchParams.self, from: req.paramsJSON)) ??
OpenClawContactsSearchParams()
let payload = try await self.contactsService.search(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawContactsCommand.add.rawValue:
let params = try Self.decodeParams(OpenClawContactsAddParams.self, from: req.paramsJSON)
let payload = try await self.contactsService.add(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
default:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: unknown command"))
}
}
private func handleCalendarInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
switch req.command {
case OpenClawCalendarCommand.events.rawValue:
let params = (try? Self.decodeParams(OpenClawCalendarEventsParams.self, from: req.paramsJSON)) ??
OpenClawCalendarEventsParams()
let payload = try await self.calendarService.events(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawCalendarCommand.add.rawValue:
let params = try Self.decodeParams(OpenClawCalendarAddParams.self, from: req.paramsJSON)
let payload = try await self.calendarService.add(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
default:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: unknown command"))
}
}
private func handleRemindersInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
switch req.command {
case OpenClawRemindersCommand.list.rawValue:
let params = (try? Self.decodeParams(OpenClawRemindersListParams.self, from: req.paramsJSON)) ??
OpenClawRemindersListParams()
let payload = try await self.remindersService.list(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawRemindersCommand.add.rawValue:
let params = try Self.decodeParams(OpenClawRemindersAddParams.self, from: req.paramsJSON)
let payload = try await self.remindersService.add(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
default:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: unknown command"))
}
}
private func handleMotionInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
switch req.command {
case OpenClawMotionCommand.activity.rawValue:
let params = (try? Self.decodeParams(OpenClawMotionActivityParams.self, from: req.paramsJSON)) ??
OpenClawMotionActivityParams()
let payload = try await self.motionService.activities(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawMotionCommand.pedometer.rawValue:
let params = (try? Self.decodeParams(OpenClawPedometerParams.self, from: req.paramsJSON)) ??
OpenClawPedometerParams()
let payload = try await self.motionService.pedometer(params: params)
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
default:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: unknown command"))
}
}
private func handleTalkInvoke(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse {
switch req.command {
case OpenClawTalkCommand.pttStart.rawValue:
self.pttVoiceWakeSuspended = self.voiceWake.suspendForExternalAudioCapture()
let payload = try await self.talkMode.beginPushToTalk()
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawTalkCommand.pttStop.rawValue:
let payload = await self.talkMode.endPushToTalk()
self.voiceWake.resumeAfterExternalAudioCapture(wasSuspended: self.pttVoiceWakeSuspended)
self.pttVoiceWakeSuspended = false
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawTalkCommand.pttCancel.rawValue:
let payload = await self.talkMode.cancelPushToTalk()
self.voiceWake.resumeAfterExternalAudioCapture(wasSuspended: self.pttVoiceWakeSuspended)
self.pttVoiceWakeSuspended = false
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
case OpenClawTalkCommand.pttOnce.rawValue:
self.pttVoiceWakeSuspended = self.voiceWake.suspendForExternalAudioCapture()
defer {
self.voiceWake.resumeAfterExternalAudioCapture(wasSuspended: self.pttVoiceWakeSuspended)
self.pttVoiceWakeSuspended = false
}
let payload = try await self.talkMode.runPushToTalkOnce()
let json = try Self.encodePayload(payload)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json)
default:
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: OpenClawNodeError(code: .invalidRequest, message: "INVALID_REQUEST: unknown command"))
}
}
}
private extension NodeAppModel {
// Central registry for node invoke routing to keep commands in one place.
func buildCapabilityRouter() -> NodeCapabilityRouter {
var handlers: [String: NodeCapabilityRouter.Handler] = [:]
func register(_ commands: [String], handler: @escaping NodeCapabilityRouter.Handler) {
for command in commands {
handlers[command] = handler
}
}
register([OpenClawLocationCommand.get.rawValue]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleLocationInvoke(req)
}
register([
OpenClawCanvasCommand.present.rawValue,
OpenClawCanvasCommand.hide.rawValue,
OpenClawCanvasCommand.navigate.rawValue,
OpenClawCanvasCommand.evalJS.rawValue,
OpenClawCanvasCommand.snapshot.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleCanvasInvoke(req)
}
register([
OpenClawCanvasA2UICommand.reset.rawValue,
OpenClawCanvasA2UICommand.push.rawValue,
OpenClawCanvasA2UICommand.pushJSONL.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleCanvasA2UIInvoke(req)
}
register([
OpenClawCameraCommand.list.rawValue,
OpenClawCameraCommand.snap.rawValue,
OpenClawCameraCommand.clip.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleCameraInvoke(req)
}
register([OpenClawScreenCommand.record.rawValue]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleScreenRecordInvoke(req)
}
register([OpenClawSystemCommand.notify.rawValue]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleSystemNotify(req)
}
register([OpenClawChatCommand.push.rawValue]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleChatPushInvoke(req)
}
register([
OpenClawDeviceCommand.status.rawValue,
OpenClawDeviceCommand.info.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleDeviceInvoke(req)
}
register([OpenClawPhotosCommand.latest.rawValue]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handlePhotosInvoke(req)
}
register([
OpenClawContactsCommand.search.rawValue,
OpenClawContactsCommand.add.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleContactsInvoke(req)
}
register([
OpenClawCalendarCommand.events.rawValue,
OpenClawCalendarCommand.add.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleCalendarInvoke(req)
}
register([
OpenClawRemindersCommand.list.rawValue,
OpenClawRemindersCommand.add.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleRemindersInvoke(req)
}
register([
OpenClawMotionCommand.activity.rawValue,
OpenClawMotionCommand.pedometer.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleMotionInvoke(req)
}
register([
OpenClawTalkCommand.pttStart.rawValue,
OpenClawTalkCommand.pttStop.rawValue,
OpenClawTalkCommand.pttCancel.rawValue,
OpenClawTalkCommand.pttOnce.rawValue,
]) { [weak self] req in
guard let self else { throw NodeCapabilityRouter.RouterError.handlerUnavailable }
return try await self.handleTalkInvoke(req)
}
return NodeCapabilityRouter(handlers: handlers)
}
func locationMode() -> OpenClawLocationMode {
let raw = UserDefaults.standard.string(forKey: "location.enabledMode") ?? "off"
return OpenClawLocationMode(rawValue: raw) ?? .off

88
apps/ios/Sources/Motion/MotionService.swift
View File

@@ -1,88 +0,0 @@
import CoreMotion
import Foundation
import OpenClawKit
final class MotionService: MotionServicing {
func activities(params: OpenClawMotionActivityParams) async throws -> OpenClawMotionActivityPayload {
guard CMMotionActivityManager.isActivityAvailable() else {
throw NSError(domain: "Motion", code: 1, userInfo: [
NSLocalizedDescriptionKey: "MOTION_UNAVAILABLE: activity not supported on this device",
])
}
let (start, end) = Self.resolveRange(startISO: params.startISO, endISO: params.endISO)
let limit = max(1, min(params.limit ?? 200, 1000))
let manager = CMMotionActivityManager()
let mapped = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[OpenClawMotionActivityEntry], Error>) in
manager.queryActivityStarting(from: start, to: end, to: OperationQueue()) { activity, error in
if let error {
cont.resume(throwing: error)
} else {
let formatter = ISO8601DateFormatter()
let sliced = Array((activity ?? []).suffix(limit))
let entries = sliced.map { entry in
OpenClawMotionActivityEntry(
startISO: formatter.string(from: entry.startDate),
endISO: formatter.string(from: end),
confidence: Self.confidenceString(entry.confidence),
isWalking: entry.walking,
isRunning: entry.running,
isCycling: entry.cycling,
isAutomotive: entry.automotive,
isStationary: entry.stationary,
isUnknown: entry.unknown)
}
cont.resume(returning: entries)
}
}
}
return OpenClawMotionActivityPayload(activities: mapped)
}
func pedometer(params: OpenClawPedometerParams) async throws -> OpenClawPedometerPayload {
guard CMPedometer.isStepCountingAvailable() else {
throw NSError(domain: "Motion", code: 2, userInfo: [
NSLocalizedDescriptionKey: "PEDOMETER_UNAVAILABLE: step counting not supported",
])
}
let (start, end) = Self.resolveRange(startISO: params.startISO, endISO: params.endISO)
let pedometer = CMPedometer()
let payload = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<OpenClawPedometerPayload, Error>) in
pedometer.queryPedometerData(from: start, to: end) { data, error in
if let error {
cont.resume(throwing: error)
} else {
let formatter = ISO8601DateFormatter()
let payload = OpenClawPedometerPayload(
startISO: formatter.string(from: start),
endISO: formatter.string(from: end),
steps: data?.numberOfSteps.intValue,
distanceMeters: data?.distance?.doubleValue,
floorsAscended: data?.floorsAscended?.intValue,
floorsDescended: data?.floorsDescended?.intValue)
cont.resume(returning: payload)
}
}
}
return payload
}
private static func resolveRange(startISO: String?, endISO: String?) -> (Date, Date) {
let formatter = ISO8601DateFormatter()
let start = startISO.flatMap { formatter.date(from: $0) } ?? Calendar.current.startOfDay(for: Date())
let end = endISO.flatMap { formatter.date(from: $0) } ?? Date()
return (start, end)
}
private static func confidenceString(_ confidence: CMMotionActivityConfidence) -> String {
switch confidence {
case .low: "low"
case .medium: "medium"
case .high: "high"
@unknown default: "unknown"
}
}
}

311
apps/ios/Sources/Onboarding/GatewayOnboardingView.swift
View File

@@ -1,311 +0,0 @@
import SwiftUI
import UIKit
struct GatewayOnboardingView: View {
@Environment(NodeAppModel.self) private var appModel: NodeAppModel
@Environment(GatewayConnectionController.self) private var gatewayController: GatewayConnectionController
@AppStorage("gateway.preferredStableID") private var preferredGatewayStableID: String = ""
@AppStorage("gateway.lastDiscoveredStableID") private var lastDiscoveredGatewayStableID: String = ""
@AppStorage("gateway.manual.enabled") private var manualGatewayEnabled: Bool = false
@AppStorage("gateway.manual.host") private var manualGatewayHost: String = ""
@AppStorage("gateway.manual.port") private var manualGatewayPort: Int = 18789
@AppStorage("gateway.manual.tls") private var manualGatewayTLS: Bool = true
@State private var connectStatusText: String?
@State private var connectingGatewayID: String?
@State private var showManualEntry: Bool = false
@State private var manualGatewayPortText: String = ""
var body: some View {
NavigationStack {
Form {
Section {
Text("Connect to your gateway to get started.")
LabeledContent("Discovery", value: self.gatewayController.discoveryStatusText)
LabeledContent("Status", value: self.appModel.gatewayStatusText)
}
Section("Gateways") {
self.gatewayList()
}
Section {
DisclosureGroup(isExpanded: self.$showManualEntry) {
TextField("Host", text: self.$manualGatewayHost)
.textInputAutocapitalization(.never)
.autocorrectionDisabled()
TextField("Port (optional)", text: self.manualPortBinding)
.keyboardType(.numberPad)
Toggle("Use TLS", isOn: self.$manualGatewayTLS)
Button {
Task { await self.connectManual() }
} label: {
if self.connectingGatewayID == "manual" {
HStack(spacing: 8) {
ProgressView()
.progressViewStyle(.circular)
Text("Connecting...")
}
} else {
Text("Connect manual gateway")
}
}
.disabled(self.connectingGatewayID != nil || self.manualGatewayHost
.trimmingCharacters(in: .whitespacesAndNewlines)
.isEmpty || !self.manualPortIsValid)
Button("Paste gateway URL") {
self.pasteGatewayURL()
}
Text(
"Use this when discovery is blocked. "
+ "Leave port empty for 443 on tailnet DNS (TLS) or 18789 otherwise.")
.font(.footnote)
.foregroundStyle(.secondary)
} label: {
Text("Manual gateway")
}
}
if let text = self.connectStatusText {
Section {
Text(text)
.font(.footnote)
.foregroundStyle(.secondary)
}
}
}
.navigationTitle("Connect Gateway")
.onAppear {
self.syncManualPortText()
}
.onChange(of: self.manualGatewayPort) { _, _ in
self.syncManualPortText()
}
.onChange(of: self.appModel.gatewayServerName) { _, _ in
self.connectStatusText = nil
}
}
}
@ViewBuilder
private func gatewayList() -> some View {
if self.gatewayController.gateways.isEmpty {
VStack(alignment: .leading, spacing: 10) {
Text("No gateways found yet.")
.foregroundStyle(.secondary)
Text("Make sure you are on the same Wi-Fi as your gateway, or your tailnet DNS is set.")
.font(.footnote)
.foregroundStyle(.secondary)
if let lastKnown = GatewaySettingsStore.loadLastGatewayConnection() {
Button {
Task { await self.connectLastKnown() }
} label: {
self.lastKnownButtonLabel(host: lastKnown.host, port: lastKnown.port)
}
.disabled(self.connectingGatewayID != nil)
.buttonStyle(.borderedProminent)
.tint(self.appModel.seamColor)
}
}
} else {
ForEach(self.gatewayController.gateways) { gateway in
HStack {
VStack(alignment: .leading, spacing: 2) {
Text(gateway.name)
let detailLines = self.gatewayDetailLines(gateway)
ForEach(detailLines, id: \.self) { line in
Text(line)
.font(.footnote)
.foregroundStyle(.secondary)
}
}
Spacer()
Button {
Task { await self.connect(gateway) }
} label: {
if self.connectingGatewayID == gateway.id {
ProgressView()
.progressViewStyle(.circular)
} else {
Text("Connect")
}
}
.disabled(self.connectingGatewayID != nil)
}
}
}
}
private func connect(_ gateway: GatewayDiscoveryModel.DiscoveredGateway) async {
self.connectingGatewayID = gateway.id
self.manualGatewayEnabled = false
self.preferredGatewayStableID = gateway.stableID
GatewaySettingsStore.savePreferredGatewayStableID(gateway.stableID)
self.lastDiscoveredGatewayStableID = gateway.stableID
GatewaySettingsStore.saveLastDiscoveredGatewayStableID(gateway.stableID)
defer { self.connectingGatewayID = nil }
await self.gatewayController.connect(gateway)
}
private func connectLastKnown() async {
self.connectingGatewayID = "last-known"
defer { self.connectingGatewayID = nil }
await self.gatewayController.connectLastKnown()
}
private var manualPortBinding: Binding<String> {
Binding(
get: { self.manualGatewayPortText },
set: { newValue in
let filtered = newValue.filter(\.isNumber)
if self.manualGatewayPortText != filtered {
self.manualGatewayPortText = filtered
}
if filtered.isEmpty {
if self.manualGatewayPort != 0 {
self.manualGatewayPort = 0
}
} else if let port = Int(filtered), self.manualGatewayPort != port {
self.manualGatewayPort = port
}
})
}
private var manualPortIsValid: Bool {
if self.manualGatewayPortText.isEmpty { return true }
return self.manualGatewayPort >= 1 && self.manualGatewayPort <= 65535
}
private func syncManualPortText() {
if self.manualGatewayPort > 0 {
let next = String(self.manualGatewayPort)
if self.manualGatewayPortText != next {
self.manualGatewayPortText = next
}
} else if !self.manualGatewayPortText.isEmpty {
self.manualGatewayPortText = ""
}
}
@ViewBuilder
private func lastKnownButtonLabel(host: String, port: Int) -> some View {
if self.connectingGatewayID == "last-known" {
HStack(spacing: 8) {
ProgressView()
.progressViewStyle(.circular)
Text("Connecting...")
}
.frame(maxWidth: .infinity)
} else {
HStack(spacing: 8) {
Image(systemName: "bolt.horizontal.circle.fill")
VStack(alignment: .leading, spacing: 2) {
Text("Connect last known")
Text("\(host):\(port)")
.font(.footnote)
.foregroundStyle(.secondary)
}
Spacer()
}
.frame(maxWidth: .infinity)
}
}
private func connectManual() async {
let host = self.manualGatewayHost.trimmingCharacters(in: .whitespacesAndNewlines)
guard !host.isEmpty else {
self.connectStatusText = "Failed: host required"
return
}
guard self.manualPortIsValid else {
self.connectStatusText = "Failed: invalid port"
return
}
self.connectingGatewayID = "manual"
self.manualGatewayEnabled = true
defer { self.connectingGatewayID = nil }
await self.gatewayController.connectManual(
host: host,
port: self.manualGatewayPort,
useTLS: self.manualGatewayTLS)
}
private func pasteGatewayURL() {
guard let text = UIPasteboard.general.string else {
self.connectStatusText = "Clipboard is empty."
return
}
if self.applyGatewayInput(text) {
self.connectStatusText = nil
self.showManualEntry = true
} else {
self.connectStatusText = "Could not parse gateway URL."
}
}
private func applyGatewayInput(_ text: String) -> Bool {
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return false }
if let components = URLComponents(string: trimmed),
let host = components.host?.trimmingCharacters(in: .whitespacesAndNewlines),
!host.isEmpty
{
let scheme = components.scheme?.lowercased()
let defaultPort: Int = {
let hostLower = host.lowercased()
if (scheme == "wss" || scheme == "https"), hostLower.hasSuffix(".ts.net") {
return 443
}
return 18789
}()
let port = components.port ?? defaultPort
if scheme == "wss" || scheme == "https" {
self.manualGatewayTLS = true
} else if scheme == "ws" || scheme == "http" {
self.manualGatewayTLS = false
}
self.manualGatewayHost = host
self.manualGatewayPort = port
self.manualGatewayPortText = String(port)
return true
}
if let hostPort = SettingsNetworkingHelpers.parseHostPort(from: trimmed) {
self.manualGatewayHost = hostPort.host
self.manualGatewayPort = hostPort.port
self.manualGatewayPortText = String(hostPort.port)
return true
}
return false
}
private func gatewayDetailLines(_ gateway: GatewayDiscoveryModel.DiscoveredGateway) -> [String] {
var lines: [String] = []
if let lanHost = gateway.lanHost { lines.append("LAN: \(lanHost)") }
if let tailnet = gateway.tailnetDns { lines.append("Tailnet: \(tailnet)") }
let gatewayPort = gateway.gatewayPort
let canvasPort = gateway.canvasPort
if gatewayPort != nil || canvasPort != nil {
let gw = gatewayPort.map(String.init) ?? "-"
let canvas = canvasPort.map(String.init) ?? "-"
lines.append("Ports: gateway \(gw) / canvas \(canvas)")
}
if lines.isEmpty {
lines.append(gateway.debugID)
}
return lines
}
}

2
apps/ios/Sources/OpenClawApp.swift
View File

@@ -15,7 +15,7 @@ struct OpenClawApp: App {
var body: some Scene {
WindowGroup {
RootView()
RootCanvas()
.environment(self.appModel)
.environment(self.appModel.voiceWake)
.environment(self.gatewayController)

171
apps/ios/Sources/Reminders/RemindersService.swift
View File

@@ -1,171 +0,0 @@
import EventKit
import Foundation
import OpenClawKit
final class RemindersService: RemindersServicing {
func list(params: OpenClawRemindersListParams) async throws -> OpenClawRemindersListPayload {
let store = EKEventStore()
let status = EKEventStore.authorizationStatus(for: .reminder)
let authorized = await Self.ensureAuthorization(store: store, status: status)
guard authorized else {
throw NSError(domain: "Reminders", code: 1, userInfo: [
NSLocalizedDescriptionKey: "REMINDERS_PERMISSION_REQUIRED: grant Reminders permission",
])
}
let limit = max(1, min(params.limit ?? 50, 500))
let statusFilter = params.status ?? .incomplete
let predicate = store.predicateForReminders(in: nil)
let payload = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[OpenClawReminderPayload], Error>) in
store.fetchReminders(matching: predicate) { items in
let formatter = ISO8601DateFormatter()
let filtered = (items ?? []).filter { reminder in
switch statusFilter {
case .all:
return true
case .completed:
return reminder.isCompleted
case .incomplete:
return !reminder.isCompleted
}
}
let selected = Array(filtered.prefix(limit))
let payload = selected.map { reminder in
let due = reminder.dueDateComponents.flatMap { Calendar.current.date(from: $0) }
return OpenClawReminderPayload(
identifier: reminder.calendarItemIdentifier,
title: reminder.title,
dueISO: due.map { formatter.string(from: $0) },
completed: reminder.isCompleted,
listName: reminder.calendar.title)
}
cont.resume(returning: payload)
}
}
return OpenClawRemindersListPayload(reminders: payload)
}
func add(params: OpenClawRemindersAddParams) async throws -> OpenClawRemindersAddPayload {
let store = EKEventStore()
let status = EKEventStore.authorizationStatus(for: .reminder)
let authorized = await Self.ensureWriteAuthorization(store: store, status: status)
guard authorized else {
throw NSError(domain: "Reminders", code: 2, userInfo: [
NSLocalizedDescriptionKey: "REMINDERS_PERMISSION_REQUIRED: grant Reminders permission",
])
}
let title = params.title.trimmingCharacters(in: .whitespacesAndNewlines)
guard !title.isEmpty else {
throw NSError(domain: "Reminders", code: 3, userInfo: [
NSLocalizedDescriptionKey: "REMINDERS_INVALID: title required",
])
}
let reminder = EKReminder(eventStore: store)
reminder.title = title
if let notes = params.notes?.trimmingCharacters(in: .whitespacesAndNewlines), !notes.isEmpty {
reminder.notes = notes
}
reminder.calendar = try Self.resolveList(
store: store,
listId: params.listId,
listName: params.listName)
if let dueISO = params.dueISO?.trimmingCharacters(in: .whitespacesAndNewlines), !dueISO.isEmpty {
let formatter = ISO8601DateFormatter()
guard let dueDate = formatter.date(from: dueISO) else {
throw NSError(domain: "Reminders", code: 4, userInfo: [
NSLocalizedDescriptionKey: "REMINDERS_INVALID: dueISO must be ISO-8601",
])
}
reminder.dueDateComponents = Calendar.current.dateComponents(
[.year, .month, .day, .hour, .minute, .second],
from: dueDate)
}
try store.save(reminder, commit: true)
let formatter = ISO8601DateFormatter()
let due = reminder.dueDateComponents.flatMap { Calendar.current.date(from: $0) }
let payload = OpenClawReminderPayload(
identifier: reminder.calendarItemIdentifier,
title: reminder.title,
dueISO: due.map { formatter.string(from: $0) },
completed: reminder.isCompleted,
listName: reminder.calendar.title)
return OpenClawRemindersAddPayload(reminder: payload)
}
private static func ensureAuthorization(store: EKEventStore, status: EKAuthorizationStatus) async -> Bool {
switch status {
case .authorized:
return true
case .notDetermined:
return await withCheckedContinuation { cont in
store.requestAccess(to: .reminder) { granted, _ in
cont.resume(returning: granted)
}
}
case .restricted, .denied:
return false
case .fullAccess:
return true
case .writeOnly:
return false
@unknown default:
return false
}
}
private static func ensureWriteAuthorization(store: EKEventStore, status: EKAuthorizationStatus) async -> Bool {
switch status {
case .authorized, .fullAccess, .writeOnly:
return true
case .notDetermined:
return await withCheckedContinuation { cont in
store.requestAccess(to: .reminder) { granted, _ in
cont.resume(returning: granted)
}
}
case .restricted, .denied:
return false
@unknown default:
return false
}
}
private static func resolveList(
store: EKEventStore,
listId: String?,
listName: String?) throws -> EKCalendar
{
if let id = listId?.trimmingCharacters(in: .whitespacesAndNewlines), !id.isEmpty,
let calendar = store.calendar(withIdentifier: id)
{
return calendar
}
if let title = listName?.trimmingCharacters(in: .whitespacesAndNewlines), !title.isEmpty {
if let calendar = store.calendars(for: .reminder).first(where: {
$0.title.compare(title, options: [.caseInsensitive, .diacriticInsensitive]) == .orderedSame
}) {
return calendar
}
throw NSError(domain: "Reminders", code: 5, userInfo: [
NSLocalizedDescriptionKey: "REMINDERS_LIST_NOT_FOUND: no list named \(title)",
])
}
if let fallback = store.defaultCalendarForNewReminders() {
return fallback
}
throw NSError(domain: "Reminders", code: 6, userInfo: [
NSLocalizedDescriptionKey: "REMINDERS_LIST_NOT_FOUND: no default list",
])
}
}

46
apps/ios/Sources/RootView.swift
View File

@@ -1,46 +0,0 @@
import SwiftUI
struct RootView: View {
@Environment(NodeAppModel.self) private var appModel: NodeAppModel
@AppStorage("gateway.onboardingComplete") private var onboardingComplete: Bool = false
@AppStorage("gateway.preferredStableID") private var preferredGatewayStableID: String = ""
@AppStorage("gateway.manual.enabled") private var manualGatewayEnabled: Bool = false
@AppStorage("gateway.manual.host") private var manualGatewayHost: String = ""
var body: some View {
Group {
if self.shouldShowOnboarding {
GatewayOnboardingView()
} else {
RootCanvas()
}
}
.onAppear { self.bootstrapOnboardingIfNeeded() }
.onChange(of: self.appModel.gatewayServerName) { _, newValue in
if newValue != nil {
self.onboardingComplete = true
}
}
}
private var shouldShowOnboarding: Bool {
if self.appModel.gatewayServerName != nil { return false }
if self.onboardingComplete { return false }
if self.hasExistingGatewayConfig { return false }
return true
}
private var hasExistingGatewayConfig: Bool {
if GatewaySettingsStore.loadLastGatewayConnection() != nil { return true }
let preferred = self.preferredGatewayStableID.trimmingCharacters(in: .whitespacesAndNewlines)
if !preferred.isEmpty { return true }
let manualHost = self.manualGatewayHost.trimmingCharacters(in: .whitespacesAndNewlines)
return self.manualGatewayEnabled && !manualHost.isEmpty
}
private func bootstrapOnboardingIfNeeded() {
if !self.onboardingComplete, self.hasExistingGatewayConfig {
self.onboardingComplete = true
}
}
}

64
apps/ios/Sources/Services/NodeServiceProtocols.swift
View File

@@ -1,64 +0,0 @@
import CoreLocation
import Foundation
import OpenClawKit
import UIKit
protocol CameraServicing: Sendable {
func listDevices() async -> [CameraController.CameraDeviceInfo]
func snap(params: OpenClawCameraSnapParams) async throws -> (format: String, base64: String, width: Int, height: Int)
func clip(params: OpenClawCameraClipParams) async throws -> (format: String, base64: String, durationMs: Int, hasAudio: Bool)
}
protocol ScreenRecordingServicing: Sendable {
func record(
screenIndex: Int?,
durationMs: Int?,
fps: Double?,
includeAudio: Bool?,
outPath: String?) async throws -> String
}
@MainActor
protocol LocationServicing: Sendable {
func authorizationStatus() -> CLAuthorizationStatus
func accuracyAuthorization() -> CLAccuracyAuthorization
func ensureAuthorization(mode: OpenClawLocationMode) async -> CLAuthorizationStatus
func currentLocation(
params: OpenClawLocationGetParams,
desiredAccuracy: OpenClawLocationAccuracy,
maxAgeMs: Int?,
timeoutMs: Int?) async throws -> CLLocation
}
protocol DeviceStatusServicing: Sendable {
func status() async throws -> OpenClawDeviceStatusPayload
func info() -> OpenClawDeviceInfoPayload
}
protocol PhotosServicing: Sendable {
func latest(params: OpenClawPhotosLatestParams) async throws -> OpenClawPhotosLatestPayload
}
protocol ContactsServicing: Sendable {
func search(params: OpenClawContactsSearchParams) async throws -> OpenClawContactsSearchPayload
func add(params: OpenClawContactsAddParams) async throws -> OpenClawContactsAddPayload
}
protocol CalendarServicing: Sendable {
func events(params: OpenClawCalendarEventsParams) async throws -> OpenClawCalendarEventsPayload
func add(params: OpenClawCalendarAddParams) async throws -> OpenClawCalendarAddPayload
}
protocol RemindersServicing: Sendable {
func list(params: OpenClawRemindersListParams) async throws -> OpenClawRemindersListPayload
func add(params: OpenClawRemindersAddParams) async throws -> OpenClawRemindersAddPayload
}
protocol MotionServicing: Sendable {
func activities(params: OpenClawMotionActivityParams) async throws -> OpenClawMotionActivityPayload
func pedometer(params: OpenClawPedometerParams) async throws -> OpenClawPedometerPayload
}
extension CameraController: CameraServicing {}
extension ScreenRecordService: ScreenRecordingServicing {}
extension LocationService: LocationServicing {}

58
apps/ios/Sources/Services/NotificationService.swift
View File

@@ -1,58 +0,0 @@
import Foundation
import UserNotifications
enum NotificationAuthorizationStatus: Sendable {
case notDetermined
case denied
case authorized
case provisional
case ephemeral
}
protocol NotificationCentering: Sendable {
func authorizationStatus() async -> NotificationAuthorizationStatus
func requestAuthorization(options: UNAuthorizationOptions) async throws -> Bool
func add(_ request: UNNotificationRequest) async throws
}
struct LiveNotificationCenter: NotificationCentering, @unchecked Sendable {
private let center: UNUserNotificationCenter
init(center: UNUserNotificationCenter = .current()) {
self.center = center
}
func authorizationStatus() async -> NotificationAuthorizationStatus {
let settings = await self.center.notificationSettings()
return switch settings.authorizationStatus {
case .authorized:
.authorized
case .provisional:
.provisional
case .ephemeral:
.ephemeral
case .denied:
.denied
case .notDetermined:
.notDetermined
@unknown default:
.denied
}
}
func requestAuthorization(options: UNAuthorizationOptions) async throws -> Bool {
try await self.center.requestAuthorization(options: options)
}
func add(_ request: UNNotificationRequest) async throws {
try await withCheckedThrowingContinuation { (cont: CheckedContinuation<Void, Error>) in
self.center.add(request) { error in
if let error {
cont.resume(throwing: error)
} else {
cont.resume(returning: ())
}
}
}
}
}

100
apps/ios/Sources/Settings/SettingsTab.swift
View File

@@ -17,8 +17,7 @@ struct SettingsTab: View {
@Environment(VoiceWakeManager.self) private var voiceWake: VoiceWakeManager
@Environment(GatewayConnectionController.self) private var gatewayController: GatewayConnectionController
@Environment(\.dismiss) private var dismiss
@AppStorage("node.displayName") private var displayName: String = NodeDisplayName.defaultValue(
for: UIDevice.current.userInterfaceIdiom)
@AppStorage("node.displayName") private var displayName: String = "iOS Node"
@AppStorage("node.instanceId") private var instanceId: String = UUID().uuidString
@AppStorage("voiceWake.enabled") private var voiceWakeEnabled: Bool = false
@AppStorage("talk.enabled") private var talkEnabled: Bool = false
@@ -41,7 +40,6 @@ struct SettingsTab: View {
@State private var lastLocationModeRaw: String = OpenClawLocationMode.off.rawValue
@State private var gatewayToken: String = ""
@State private var gatewayPassword: String = ""
@State private var manualGatewayPortText: String = ""
var body: some View {
NavigationStack {
@@ -122,7 +120,7 @@ struct SettingsTab: View {
.textInputAutocapitalization(.never)
.autocorrectionDisabled()
TextField("Port (optional)", text: self.manualPortBinding)
TextField("Port", value: self.$manualGatewayPort, format: .number)
.keyboardType(.numberPad)
Toggle("Use TLS", isOn: self.$manualGatewayTLS)
@@ -142,11 +140,11 @@ struct SettingsTab: View {
}
.disabled(self.connectingGatewayID != nil || self.manualGatewayHost
.trimmingCharacters(in: .whitespacesAndNewlines)
.isEmpty || !self.manualPortIsValid)
.isEmpty || self.manualGatewayPort <= 0 || self.manualGatewayPort > 65535)
Text(
"Use this when mDNS/Bonjour discovery is blocked. "
+ "Leave port empty for 443 on tailnet DNS (TLS) or 18789 otherwise.")
+ "The gateway WebSocket listens on port 18789 by default.")
.font(.footnote)
.foregroundStyle(.secondary)
@@ -234,7 +232,6 @@ struct SettingsTab: View {
.onAppear {
self.localIPAddress = Self.primaryIPv4Address()
self.lastLocationModeRaw = self.locationEnabledModeRaw
self.syncManualPortText()
let trimmedInstanceId = self.instanceId.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmedInstanceId.isEmpty {
self.gatewayToken = GatewaySettingsStore.loadGatewayToken(instanceId: trimmedInstanceId) ?? ""
@@ -258,9 +255,6 @@ struct SettingsTab: View {
guard !instanceId.isEmpty else { return }
GatewaySettingsStore.saveGatewayPassword(trimmed, instanceId: instanceId)
}
.onChange(of: self.manualGatewayPort) { _, _ in
self.syncManualPortText()
}
.onChange(of: self.appModel.gatewayServerName) { _, _ in
self.connectStatus.text = nil
}
@@ -284,24 +278,8 @@ struct SettingsTab: View {
@ViewBuilder
private func gatewayList(showing: GatewayListMode) -> some View {
if self.gatewayController.gateways.isEmpty {
VStack(alignment: .leading, spacing: 12) {
Text("No gateways found yet.")
.foregroundStyle(.secondary)
Text("If your gateway is on another network, connect it and ensure DNS is working.")
.font(.footnote)
.foregroundStyle(.secondary)
if let lastKnown = GatewaySettingsStore.loadLastGatewayConnection() {
Button {
Task { await self.connectLastKnown() }
} label: {
self.lastKnownButtonLabel(host: lastKnown.host, port: lastKnown.port)
}
.disabled(self.connectingGatewayID != nil)
.buttonStyle(.borderedProminent)
.tint(self.appModel.seamColor)
}
}
Text("No gateways found yet.")
.foregroundStyle(.secondary)
} else {
let connectedID = self.appModel.connectedGatewayID
let rows = self.gatewayController.gateways.filter { gateway in
@@ -399,77 +377,13 @@ struct SettingsTab: View {
await self.gatewayController.connect(gateway)
}
private func connectLastKnown() async {
self.connectingGatewayID = "last-known"
defer { self.connectingGatewayID = nil }
await self.gatewayController.connectLastKnown()
}
@ViewBuilder
private func lastKnownButtonLabel(host: String, port: Int) -> some View {
if self.connectingGatewayID == "last-known" {
HStack(spacing: 8) {
ProgressView()
.progressViewStyle(.circular)
Text("Connecting…")
}
.frame(maxWidth: .infinity)
} else {
HStack(spacing: 8) {
Image(systemName: "bolt.horizontal.circle.fill")
VStack(alignment: .leading, spacing: 2) {
Text("Connect last known")
Text("\(host):\(port)")
.font(.footnote)
.foregroundStyle(.secondary)
}
Spacer()
}
.frame(maxWidth: .infinity)
}
}
private var manualPortBinding: Binding<String> {
Binding(
get: { self.manualGatewayPortText },
set: { newValue in
let filtered = newValue.filter(\.isNumber)
if self.manualGatewayPortText != filtered {
self.manualGatewayPortText = filtered
}
if filtered.isEmpty {
if self.manualGatewayPort != 0 {
self.manualGatewayPort = 0
}
} else if let port = Int(filtered), self.manualGatewayPort != port {
self.manualGatewayPort = port
}
})
}
private var manualPortIsValid: Bool {
if self.manualGatewayPortText.isEmpty { return true }
return self.manualGatewayPort >= 1 && self.manualGatewayPort <= 65535
}
private func syncManualPortText() {
if self.manualGatewayPort > 0 {
let next = String(self.manualGatewayPort)
if self.manualGatewayPortText != next {
self.manualGatewayPortText = next
}
} else if !self.manualGatewayPortText.isEmpty {
self.manualGatewayPortText = ""
}
}
private func connectManual() async {
let host = self.manualGatewayHost.trimmingCharacters(in: .whitespacesAndNewlines)
guard !host.isEmpty else {
self.connectStatus.text = "Failed: host required"
return
}
guard self.manualPortIsValid else {
guard self.manualGatewayPort > 0, self.manualGatewayPort <= 65535 else {
self.connectStatus.text = "Failed: invalid port"
return
}

8
apps/ios/Sources/Status/StatusPill.swift
View File

@@ -72,6 +72,12 @@ struct StatusPill: View {
.lineLimit(1)
}
.transition(.opacity.combined(with: .move(edge: .top)))
} else {
Image(systemName: self.voiceWakeEnabled ? "mic.fill" : "mic.slash")
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(self.voiceWakeEnabled ? .primary : .secondary)
.accessibilityLabel(self.voiceWakeEnabled ? "Voice Wake enabled" : "Voice Wake disabled")
.transition(.opacity.combined(with: .move(edge: .top)))
}
}
.padding(.vertical, 8)
@@ -104,7 +110,7 @@ struct StatusPill: View {
if let activity {
return "\(self.gateway.title), \(activity.title)"
}
return self.gateway.title
return "\(self.gateway.title), Voice Wake \(self.voiceWakeEnabled ? "enabled" : "disabled")"
}
private func updatePulse(for gateway: GatewayState, scenePhase: ScenePhase) {

948
apps/ios/Sources/Voice/TalkModeManager.swift
View File

File diff suppressed because it is too large Load Diff

102
apps/ios/Sources/Voice/VoiceWakeManager.swift
View File

@@ -1,7 +1,6 @@
import AVFAudio
import Foundation
import Observation
import OpenClawKit
import Speech
import SwabbleKit
@@ -160,18 +159,14 @@ final class VoiceWakeManager: NSObject {
let micOk = await Self.requestMicrophonePermission()
guard micOk else {
self.statusText = Self.permissionMessage(
kind: "Microphone",
status: AVAudioSession.sharedInstance().recordPermission)
self.statusText = "Microphone permission denied"
self.isListening = false
return
}
let speechOk = await Self.requestSpeechPermission()
guard speechOk else {
self.statusText = Self.permissionMessage(
kind: "Speech recognition",
status: SFSpeechRecognizer.authorizationStatus())
self.statusText = "Speech recognition permission denied"
self.isListening = false
return
}
@@ -369,101 +364,20 @@ final class VoiceWakeManager: NSObject {
}
private nonisolated static func requestMicrophonePermission() async -> Bool {
let session = AVAudioSession.sharedInstance()
switch session.recordPermission {
case .granted:
return true
case .denied:
return false
case .undetermined:
break
@unknown default:
return false
}
return await self.requestPermissionWithTimeout { completion in
AVAudioSession.sharedInstance().requestRecordPermission { ok in
completion(ok)
await withCheckedContinuation(isolation: nil) { cont in
AVAudioApplication.requestRecordPermission { ok in
cont.resume(returning: ok)
}
}
}
private nonisolated static func requestSpeechPermission() async -> Bool {
let status = SFSpeechRecognizer.authorizationStatus()
switch status {
case .authorized:
return true
case .denied, .restricted:
return false
case .notDetermined:
break
@unknown default:
return false
}
return await self.requestPermissionWithTimeout { completion in
SFSpeechRecognizer.requestAuthorization { authStatus in
completion(authStatus == .authorized)
await withCheckedContinuation(isolation: nil) { cont in
SFSpeechRecognizer.requestAuthorization { status in
cont.resume(returning: status == .authorized)
}
}
}
private nonisolated static func requestPermissionWithTimeout(
_ operation: @escaping @Sendable (@escaping (Bool) -> Void) -> Void) async -> Bool
{
do {
return try await AsyncTimeout.withTimeout(
seconds: 8,
onTimeout: { NSError(domain: "VoiceWake", code: 6, userInfo: [
NSLocalizedDescriptionKey: "permission request timed out",
]) },
operation: {
await withCheckedContinuation(isolation: nil) { cont in
Task { @MainActor in
operation { ok in
cont.resume(returning: ok)
}
}
}
})
} catch {
return false
}
}
private static func permissionMessage(
kind: String,
status: AVAudioSession.RecordPermission) -> String
{
switch status {
case .denied:
return "\(kind) permission denied"
case .undetermined:
return "\(kind) permission not granted"
case .granted:
return "\(kind) permission denied"
@unknown default:
return "\(kind) permission denied"
}
}
private static func permissionMessage(
kind: String,
status: SFSpeechRecognizerAuthorizationStatus) -> String
{
switch status {
case .denied:
return "\(kind) permission denied"
case .restricted:
return "\(kind) permission restricted"
case .notDetermined:
return "\(kind) permission not granted"
case .authorized:
return "\(kind) permission denied"
@unknown default:
return "\(kind) permission denied"
}
}
}
#if DEBUG

20
apps/ios/SwiftSources.input.xcfilelist
View File

@@ -2,32 +2,20 @@ Sources/Gateway/GatewayConnectionController.swift
Sources/Gateway/GatewayDiscoveryDebugLogView.swift
Sources/Gateway/GatewayDiscoveryModel.swift
Sources/Gateway/GatewaySettingsStore.swift
Sources/Gateway/GatewayHealthMonitor.swift
Sources/Gateway/KeychainStore.swift
Sources/Capabilities/NodeCapabilityRouter.swift
Sources/Camera/CameraController.swift
Sources/Chat/ChatSheet.swift
Sources/Chat/IOSGatewayChatTransport.swift
Sources/Contacts/ContactsService.swift
Sources/Device/DeviceStatusService.swift
Sources/Device/NodeDisplayName.swift
Sources/Device/NetworkStatusService.swift
Sources/OpenClawApp.swift
Sources/Location/LocationService.swift
Sources/Media/PhotoLibraryService.swift
Sources/Motion/MotionService.swift
Sources/Model/NodeAppModel.swift
Sources/RootCanvas.swift
Sources/RootTabs.swift
Sources/Services/NodeServiceProtocols.swift
Sources/Services/NotificationService.swift
Sources/Screen/ScreenController.swift
Sources/Screen/ScreenRecordService.swift
Sources/Screen/ScreenTab.swift
Sources/Screen/ScreenWebView.swift
Sources/SessionKey.swift
Sources/Calendar/CalendarService.swift
Sources/Reminders/RemindersService.swift
Sources/Settings/SettingsNetworkingHelpers.swift
Sources/Settings/SettingsTab.swift
Sources/Settings/VoiceWakeWordsSettingsView.swift
@@ -52,7 +40,6 @@ Sources/Voice/VoiceWakePreferences.swift
../shared/OpenClawKit/Sources/OpenClawKit/BonjourEscapes.swift
../shared/OpenClawKit/Sources/OpenClawKit/BonjourTypes.swift
../shared/OpenClawKit/Sources/OpenClawKit/BridgeFrames.swift
../shared/OpenClawKit/Sources/OpenClawKit/CalendarCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/CameraCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/CanvasA2UIAction.swift
../shared/OpenClawKit/Sources/OpenClawKit/CanvasA2UICommands.swift
@@ -60,20 +47,13 @@ Sources/Voice/VoiceWakePreferences.swift
../shared/OpenClawKit/Sources/OpenClawKit/CanvasCommandParams.swift
../shared/OpenClawKit/Sources/OpenClawKit/CanvasCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/Capabilities.swift
../shared/OpenClawKit/Sources/OpenClawKit/ChatCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/ContactsCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/DeviceCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/OpenClawKitResources.swift
../shared/OpenClawKit/Sources/OpenClawKit/DeepLinks.swift
../shared/OpenClawKit/Sources/OpenClawKit/JPEGTranscoder.swift
../shared/OpenClawKit/Sources/OpenClawKit/MotionCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/NodeError.swift
../shared/OpenClawKit/Sources/OpenClawKit/PhotosCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/RemindersCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/ScreenCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/StoragePaths.swift
../shared/OpenClawKit/Sources/OpenClawKit/SystemCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/TalkCommands.swift
../shared/OpenClawKit/Sources/OpenClawKit/TalkDirective.swift
../../Swabble/Sources/SwabbleKit/WakeWordGate.swift
Sources/Voice/TalkModeManager.swift

20
apps/ios/Tests/ContactsServiceTests.swift
View File

@@ -1,20 +0,0 @@
import Contacts
import Testing
@testable import OpenClaw
@Suite(.serialized) struct ContactsServiceTests {
@Test func matchesPhoneOrEmailForDedupe() {
let contact = CNMutableContact()
contact.givenName = "Test"
contact.phoneNumbers = [
CNLabeledValue(label: CNLabelPhoneNumberMobile, value: CNPhoneNumber(stringValue: "+1 (555) 000-0000")),
]
contact.emailAddresses = [
CNLabeledValue(label: CNLabelHome, value: "test@example.com" as NSString),
]
#expect(ContactsService._test_matches(contact: contact, phoneNumbers: ["15550000000"], emails: []))
#expect(ContactsService._test_matches(contact: contact, phoneNumbers: [], emails: ["TEST@example.com"]))
#expect(!ContactsService._test_matches(contact: contact, phoneNumbers: ["999"], emails: ["nope@example.com"]))
}
}

50
apps/ios/Tests/GatewayConnectionControllerTests.swift
View File

@@ -40,7 +40,6 @@ private func withUserDefaults<T>(_ updates: [String: Any?], _ body: () throws ->
let resolved = controller._test_resolvedDisplayName(defaults: defaults)
#expect(!resolved.isEmpty)
#expect(resolved != "iOS Node")
#expect(defaults.string(forKey: displayKey) == resolved)
}
}
@@ -62,11 +61,6 @@ private func withUserDefaults<T>(_ updates: [String: Any?], _ body: () throws ->
#expect(caps.contains(OpenClawCapability.camera.rawValue))
#expect(caps.contains(OpenClawCapability.location.rawValue))
#expect(caps.contains(OpenClawCapability.voiceWake.rawValue))
#expect(caps.contains(OpenClawCapability.device.rawValue))
#expect(caps.contains(OpenClawCapability.photos.rawValue))
#expect(caps.contains(OpenClawCapability.contacts.rawValue))
#expect(caps.contains(OpenClawCapability.calendar.rawValue))
#expect(caps.contains(OpenClawCapability.reminders.rawValue))
}
}
@@ -82,48 +76,4 @@ private func withUserDefaults<T>(_ updates: [String: Any?], _ body: () throws ->
#expect(commands.contains(OpenClawLocationCommand.get.rawValue))
}
}
@Test @MainActor func currentCommandsExcludeShellAndIncludeNotifyAndDevice() {
withUserDefaults([
"node.instanceId": "ios-test",
]) {
let appModel = NodeAppModel()
let controller = GatewayConnectionController(appModel: appModel, startDiscovery: false)
let commands = Set(controller._test_currentCommands())
#expect(commands.contains(OpenClawSystemCommand.notify.rawValue))
#expect(commands.contains(OpenClawChatCommand.push.rawValue))
#expect(!commands.contains(OpenClawSystemCommand.run.rawValue))
#expect(!commands.contains(OpenClawSystemCommand.which.rawValue))
#expect(!commands.contains(OpenClawSystemCommand.execApprovalsGet.rawValue))
#expect(!commands.contains(OpenClawSystemCommand.execApprovalsSet.rawValue))
#expect(commands.contains(OpenClawDeviceCommand.status.rawValue))
#expect(commands.contains(OpenClawDeviceCommand.info.rawValue))
#expect(commands.contains(OpenClawContactsCommand.add.rawValue))
#expect(commands.contains(OpenClawCalendarCommand.add.rawValue))
#expect(commands.contains(OpenClawRemindersCommand.add.rawValue))
#expect(commands.contains(OpenClawTalkCommand.pttStart.rawValue))
#expect(commands.contains(OpenClawTalkCommand.pttStop.rawValue))
#expect(commands.contains(OpenClawTalkCommand.pttCancel.rawValue))
#expect(commands.contains(OpenClawTalkCommand.pttOnce.rawValue))
}
}
@Test @MainActor func currentPermissionsIncludeExpectedKeys() {
let appModel = NodeAppModel()
let controller = GatewayConnectionController(appModel: appModel, startDiscovery: false)
let permissions = controller._test_currentPermissions()
let keys = Set(permissions.keys)
#expect(keys.contains("camera"))
#expect(keys.contains("microphone"))
#expect(keys.contains("location"))
#expect(keys.contains("screenRecording"))
#expect(keys.contains("photos"))
#expect(keys.contains("contacts"))
#expect(keys.contains("calendar"))
#expect(keys.contains("reminders"))
#expect(keys.contains("motion"))
}
}

60
apps/ios/Tests/GatewayHealthMonitorTests.swift
View File

@@ -1,60 +0,0 @@
import Foundation
import Testing
@testable import OpenClaw
private actor Counter {
private var value = 0
func increment() {
value += 1
}
func get() -> Int {
value
}
func set(_ newValue: Int) {
value = newValue
}
}
@Suite struct GatewayHealthMonitorTests {
@Test @MainActor func triggersFailureAfterThreshold() async {
let failureCount = Counter()
let monitor = GatewayHealthMonitor(
config: .init(intervalSeconds: 0.001, timeoutSeconds: 0.0, maxFailures: 2))
monitor.start(
check: { false },
onFailure: { _ in
await failureCount.increment()
await monitor.stop()
})
try? await Task.sleep(nanoseconds: 60_000_000)
#expect(await failureCount.get() == 1)
}
@Test @MainActor func resetsFailuresAfterSuccess() async {
let failureCount = Counter()
let calls = Counter()
let monitor = GatewayHealthMonitor(
config: .init(intervalSeconds: 0.001, timeoutSeconds: 0.0, maxFailures: 2))
monitor.start(
check: {
await calls.increment()
let callCount = await calls.get()
if callCount >= 6 {
await monitor.stop()
}
return callCount % 2 == 0
},
onFailure: { _ in
await failureCount.increment()
})
try? await Task.sleep(nanoseconds: 60_000_000)
#expect(await failureCount.get() == 0)
}
}

36
apps/ios/Tests/GatewaySettingsStoreTests.swift
View File

@@ -7,14 +7,11 @@ private struct KeychainEntry: Hashable {
let account: String
}
private let gatewayService = "ai.openclaw.gateway"
private let nodeService = "ai.openclaw.node"
private let gatewayService = "bot.molt.gateway"
private let nodeService = "bot.molt.node"
private let instanceIdEntry = KeychainEntry(service: nodeService, account: "instanceId")
private let preferredGatewayEntry = KeychainEntry(service: gatewayService, account: "preferredStableID")
private let lastGatewayEntry = KeychainEntry(service: gatewayService, account: "lastDiscoveredStableID")
private func gatewayPasswordEntry(instanceId: String) -> KeychainEntry {
KeychainEntry(service: gatewayService, account: "gateway-password.\(instanceId)")
}
private func snapshotDefaults(_ keys: [String]) -> [String: Any?] {
let defaults = UserDefaults.standard
@@ -127,33 +124,4 @@ private func restoreKeychain(_ snapshot: [KeychainEntry: String?]) {
#expect(defaults.string(forKey: "gateway.preferredStableID") == "preferred-from-keychain")
#expect(defaults.string(forKey: "gateway.lastDiscoveredStableID") == "last-from-keychain")
}
@Test func bootstrapCopiesManualPasswordToKeychainWhenMissing() {
let instanceId = "node-test"
let defaultsKeys = [
"node.instanceId",
"gateway.manual.password",
]
let passwordEntry = gatewayPasswordEntry(instanceId: instanceId)
let defaultsSnapshot = snapshotDefaults(defaultsKeys)
let keychainSnapshot = snapshotKeychain([passwordEntry, instanceIdEntry])
defer {
restoreDefaults(defaultsSnapshot)
restoreKeychain(keychainSnapshot)
}
applyDefaults([
"node.instanceId": instanceId,
"gateway.manual.password": "manual-secret",
])
applyKeychain([
passwordEntry: nil,
instanceIdEntry: nil,
])
GatewaySettingsStore.bootstrapPersistence()
#expect(KeychainStore.loadString(service: gatewayService, account: passwordEntry.account) == "manual-secret")
#expect(UserDefaults.standard.string(forKey: "gateway.manual.password") != "manual-secret")
}
}

677
apps/ios/Tests/NodeAppModelInvokeTests.swift
View File

@@ -1,9 +1,7 @@
import CoreLocation
import Foundation
import OpenClawKit
import Foundation
import Testing
import UIKit
import UserNotifications
@testable import OpenClaw
private func withUserDefaults<T>(_ updates: [String: Any?], _ body: () throws -> T) rethrows -> T {
@@ -31,210 +29,6 @@ private func withUserDefaults<T>(_ updates: [String: Any?], _ body: () throws ->
return try body()
}
private final class TestNotificationCenter: NotificationCentering, @unchecked Sendable {
private(set) var requestAuthorizationCalls = 0
private(set) var addedRequests: [UNNotificationRequest] = []
private var status: NotificationAuthorizationStatus
init(status: NotificationAuthorizationStatus) {
self.status = status
}
func authorizationStatus() async -> NotificationAuthorizationStatus {
status
}
func requestAuthorization(options: UNAuthorizationOptions) async throws -> Bool {
requestAuthorizationCalls += 1
status = .authorized
return true
}
func add(_ request: UNNotificationRequest) async throws {
addedRequests.append(request)
}
}
private struct TestCameraService: CameraServicing {
func listDevices() async -> [CameraController.CameraDeviceInfo] { [] }
func snap(params: OpenClawCameraSnapParams) async throws -> (format: String, base64: String, width: Int, height: Int) {
("jpeg", "dGVzdA==", 1, 1)
}
func clip(params: OpenClawCameraClipParams) async throws -> (format: String, base64: String, durationMs: Int, hasAudio: Bool) {
("mp4", "dGVzdA==", 1000, true)
}
}
private struct TestScreenRecorder: ScreenRecordingServicing {
func record(
screenIndex: Int?,
durationMs: Int?,
fps: Double?,
includeAudio: Bool?,
outPath: String?) async throws -> String
{
let url = FileManager.default.temporaryDirectory.appendingPathComponent("openclaw-screen-test.mp4")
FileManager.default.createFile(atPath: url.path, contents: Data())
return url.path
}
}
@MainActor
private struct TestLocationService: LocationServicing {
func authorizationStatus() -> CLAuthorizationStatus { .authorizedWhenInUse }
func accuracyAuthorization() -> CLAccuracyAuthorization { .fullAccuracy }
func ensureAuthorization(mode: OpenClawLocationMode) async -> CLAuthorizationStatus { .authorizedWhenInUse }
func currentLocation(
params: OpenClawLocationGetParams,
desiredAccuracy: OpenClawLocationAccuracy,
maxAgeMs: Int?,
timeoutMs: Int?) async throws -> CLLocation
{
CLLocation(latitude: 37.3349, longitude: -122.0090)
}
}
private struct TestDeviceStatusService: DeviceStatusServicing {
let statusPayload: OpenClawDeviceStatusPayload
let infoPayload: OpenClawDeviceInfoPayload
func status() async throws -> OpenClawDeviceStatusPayload { statusPayload }
func info() -> OpenClawDeviceInfoPayload { infoPayload }
}
private struct TestPhotosService: PhotosServicing {
let payload: OpenClawPhotosLatestPayload
func latest(params: OpenClawPhotosLatestParams) async throws -> OpenClawPhotosLatestPayload { payload }
}
private struct TestContactsService: ContactsServicing {
let searchPayload: OpenClawContactsSearchPayload
let addPayload: OpenClawContactsAddPayload
func search(params: OpenClawContactsSearchParams) async throws -> OpenClawContactsSearchPayload { searchPayload }
func add(params: OpenClawContactsAddParams) async throws -> OpenClawContactsAddPayload { addPayload }
}
private struct TestCalendarService: CalendarServicing {
let eventsPayload: OpenClawCalendarEventsPayload
let addPayload: OpenClawCalendarAddPayload
func events(params: OpenClawCalendarEventsParams) async throws -> OpenClawCalendarEventsPayload { eventsPayload }
func add(params: OpenClawCalendarAddParams) async throws -> OpenClawCalendarAddPayload { addPayload }
}
private struct TestRemindersService: RemindersServicing {
let listPayload: OpenClawRemindersListPayload
let addPayload: OpenClawRemindersAddPayload
func list(params: OpenClawRemindersListParams) async throws -> OpenClawRemindersListPayload { listPayload }
func add(params: OpenClawRemindersAddParams) async throws -> OpenClawRemindersAddPayload { addPayload }
}
private struct TestMotionService: MotionServicing {
let activityPayload: OpenClawMotionActivityPayload
let pedometerPayload: OpenClawPedometerPayload
func activities(params: OpenClawMotionActivityParams) async throws -> OpenClawMotionActivityPayload {
activityPayload
}
func pedometer(params: OpenClawPedometerParams) async throws -> OpenClawPedometerPayload {
pedometerPayload
}
}
@MainActor
private func makeTestAppModel(
notificationCenter: NotificationCentering = TestNotificationCenter(status: .authorized),
deviceStatusService: DeviceStatusServicing,
photosService: PhotosServicing,
contactsService: ContactsServicing,
calendarService: CalendarServicing,
remindersService: RemindersServicing,
motionService: MotionServicing,
talkMode: TalkModeManager = TalkModeManager(allowSimulatorCapture: true)) -> NodeAppModel
{
NodeAppModel(
screen: ScreenController(),
camera: TestCameraService(),
screenRecorder: TestScreenRecorder(),
locationService: TestLocationService(),
notificationCenter: notificationCenter,
deviceStatusService: deviceStatusService,
photosService: photosService,
contactsService: contactsService,
calendarService: calendarService,
remindersService: remindersService,
motionService: motionService,
talkMode: talkMode)
}
@MainActor
private func makeTalkTestAppModel(talkMode: TalkModeManager) -> NodeAppModel {
makeTestAppModel(
deviceStatusService: TestDeviceStatusService(
statusPayload: OpenClawDeviceStatusPayload(
battery: OpenClawBatteryStatusPayload(level: 0.5, state: .unplugged, lowPowerModeEnabled: false),
thermal: OpenClawThermalStatusPayload(state: .nominal),
storage: OpenClawStorageStatusPayload(totalBytes: 10, freeBytes: 5, usedBytes: 5),
network: OpenClawNetworkStatusPayload(
status: .satisfied,
isExpensive: false,
isConstrained: false,
interfaces: [.wifi]),
uptimeSeconds: 1),
infoPayload: OpenClawDeviceInfoPayload(
deviceName: "Test",
modelIdentifier: "Test1,1",
systemName: "iOS",
systemVersion: "1.0",
appVersion: "dev",
appBuild: "0",
locale: "en-US")),
photosService: TestPhotosService(payload: OpenClawPhotosLatestPayload(photos: [])),
contactsService: TestContactsService(
searchPayload: OpenClawContactsSearchPayload(contacts: []),
addPayload: OpenClawContactsAddPayload(contact: OpenClawContactPayload(
identifier: "c0",
displayName: "",
givenName: "",
familyName: "",
organizationName: "",
phoneNumbers: [],
emails: []))),
calendarService: TestCalendarService(
eventsPayload: OpenClawCalendarEventsPayload(events: []),
addPayload: OpenClawCalendarAddPayload(event: OpenClawCalendarEventPayload(
identifier: "e0",
title: "Test",
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T00:10:00Z",
isAllDay: false,
location: nil,
calendarTitle: nil))),
remindersService: TestRemindersService(
listPayload: OpenClawRemindersListPayload(reminders: []),
addPayload: OpenClawRemindersAddPayload(reminder: OpenClawReminderPayload(
identifier: "r0",
title: "Test",
dueISO: nil,
completed: false,
listName: nil))),
motionService: TestMotionService(
activityPayload: OpenClawMotionActivityPayload(activities: []),
pedometerPayload: OpenClawPedometerPayload(
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T01:00:00Z",
steps: nil,
distanceMeters: nil,
floorsAscended: nil,
floorsDescended: nil)),
talkMode: talkMode)
}
private func decodePayload<T: Decodable>(_ json: String?, as type: T.Type) throws -> T {
let data = try #require(json?.data(using: .utf8))
return try JSONDecoder().decode(type, from: data)
}
@Suite(.serialized) struct NodeAppModelInvokeTests {
@Test @MainActor func decodeParamsFailsWithoutJSON() {
#expect(throws: Error.self) {
@@ -330,11 +124,6 @@ private func decodePayload<T: Decodable>(_ json: String?, as type: T.Type) throw
let payloadData = try #require(evalRes.payloadJSON?.data(using: .utf8))
let payload = try JSONSerialization.jsonObject(with: payloadData) as? [String: Any]
#expect(payload?["result"] as? String == "2")
let hide = BridgeInvokeRequest(id: "hide", command: OpenClawCanvasCommand.hide.rawValue)
let hideRes = await appModel._test_handleInvoke(hide)
#expect(hideRes.ok == true)
#expect(appModel.screen.urlString.isEmpty)
}
@Test @MainActor func handleInvokeA2UICommandsFailWhenHostMissing() async throws {
@@ -366,470 +155,6 @@ private func decodePayload<T: Decodable>(_ json: String?, as type: T.Type) throw
#expect(res.error?.code == .invalidRequest)
}
@Test @MainActor func handleInvokeSystemNotifyCreatesNotificationRequest() async throws {
let notifier = TestNotificationCenter(status: .notDetermined)
let deviceStatus = TestDeviceStatusService(
statusPayload: OpenClawDeviceStatusPayload(
battery: OpenClawBatteryStatusPayload(level: 0.5, state: .charging, lowPowerModeEnabled: false),
thermal: OpenClawThermalStatusPayload(state: .nominal),
storage: OpenClawStorageStatusPayload(totalBytes: 100, freeBytes: 50, usedBytes: 50),
network: OpenClawNetworkStatusPayload(
status: .satisfied,
isExpensive: false,
isConstrained: false,
interfaces: [.wifi]),
uptimeSeconds: 10),
infoPayload: OpenClawDeviceInfoPayload(
deviceName: "Test",
modelIdentifier: "Test1,1",
systemName: "iOS",
systemVersion: "1.0",
appVersion: "dev",
appBuild: "0",
locale: "en-US"))
let emptyContact = OpenClawContactPayload(
identifier: "c0",
displayName: "",
givenName: "",
familyName: "",
organizationName: "",
phoneNumbers: [],
emails: [])
let emptyEvent = OpenClawCalendarEventPayload(
identifier: "e0",
title: "Test",
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T00:30:00Z",
isAllDay: false,
location: nil,
calendarTitle: nil)
let emptyReminder = OpenClawReminderPayload(
identifier: "r0",
title: "Test",
dueISO: nil,
completed: false,
listName: nil)
let appModel = makeTestAppModel(
notificationCenter: notifier,
deviceStatusService: deviceStatus,
photosService: TestPhotosService(payload: OpenClawPhotosLatestPayload(photos: [])),
contactsService: TestContactsService(
searchPayload: OpenClawContactsSearchPayload(contacts: []),
addPayload: OpenClawContactsAddPayload(contact: emptyContact)),
calendarService: TestCalendarService(
eventsPayload: OpenClawCalendarEventsPayload(events: []),
addPayload: OpenClawCalendarAddPayload(event: emptyEvent)),
remindersService: TestRemindersService(
listPayload: OpenClawRemindersListPayload(reminders: []),
addPayload: OpenClawRemindersAddPayload(reminder: emptyReminder)),
motionService: TestMotionService(
activityPayload: OpenClawMotionActivityPayload(activities: []),
pedometerPayload: OpenClawPedometerPayload(
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T01:00:00Z",
steps: nil,
distanceMeters: nil,
floorsAscended: nil,
floorsDescended: nil)))
let params = OpenClawSystemNotifyParams(title: "Hello", body: "World")
let data = try JSONEncoder().encode(params)
let json = String(decoding: data, as: UTF8.self)
let req = BridgeInvokeRequest(
id: "notify",
command: OpenClawSystemCommand.notify.rawValue,
paramsJSON: json)
let res = await appModel._test_handleInvoke(req)
#expect(res.ok == true)
#expect(notifier.requestAuthorizationCalls == 1)
#expect(notifier.addedRequests.count == 1)
let request = try #require(notifier.addedRequests.first)
#expect(request.content.title == "Hello")
#expect(request.content.body == "World")
}
@Test @MainActor func handleInvokeChatPushCreatesNotification() async throws {
let notifier = TestNotificationCenter(status: .authorized)
let deviceStatus = TestDeviceStatusService(
statusPayload: OpenClawDeviceStatusPayload(
battery: OpenClawBatteryStatusPayload(level: 0.5, state: .charging, lowPowerModeEnabled: false),
thermal: OpenClawThermalStatusPayload(state: .nominal),
storage: OpenClawStorageStatusPayload(totalBytes: 100, freeBytes: 50, usedBytes: 50),
network: OpenClawNetworkStatusPayload(
status: .satisfied,
isExpensive: false,
isConstrained: false,
interfaces: [.wifi]),
uptimeSeconds: 10),
infoPayload: OpenClawDeviceInfoPayload(
deviceName: "Test",
modelIdentifier: "Test1,1",
systemName: "iOS",
systemVersion: "1.0",
appVersion: "dev",
appBuild: "0",
locale: "en-US"))
let emptyContact = OpenClawContactPayload(
identifier: "c0",
displayName: "",
givenName: "",
familyName: "",
organizationName: "",
phoneNumbers: [],
emails: [])
let emptyEvent = OpenClawCalendarEventPayload(
identifier: "e0",
title: "Test",
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T00:30:00Z",
isAllDay: false,
location: nil,
calendarTitle: nil)
let emptyReminder = OpenClawReminderPayload(
identifier: "r0",
title: "Test",
dueISO: nil,
completed: false,
listName: nil)
let appModel = makeTestAppModel(
notificationCenter: notifier,
deviceStatusService: deviceStatus,
photosService: TestPhotosService(payload: OpenClawPhotosLatestPayload(photos: [])),
contactsService: TestContactsService(
searchPayload: OpenClawContactsSearchPayload(contacts: []),
addPayload: OpenClawContactsAddPayload(contact: emptyContact)),
calendarService: TestCalendarService(
eventsPayload: OpenClawCalendarEventsPayload(events: []),
addPayload: OpenClawCalendarAddPayload(event: emptyEvent)),
remindersService: TestRemindersService(
listPayload: OpenClawRemindersListPayload(reminders: []),
addPayload: OpenClawRemindersAddPayload(reminder: emptyReminder)),
motionService: TestMotionService(
activityPayload: OpenClawMotionActivityPayload(activities: []),
pedometerPayload: OpenClawPedometerPayload(
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T01:00:00Z",
steps: nil,
distanceMeters: nil,
floorsAscended: nil,
floorsDescended: nil)))
let params = OpenClawChatPushParams(text: "Ping", speak: false)
let data = try JSONEncoder().encode(params)
let json = String(decoding: data, as: UTF8.self)
let req = BridgeInvokeRequest(
id: "chat-push",
command: OpenClawChatCommand.push.rawValue,
paramsJSON: json)
let res = await appModel._test_handleInvoke(req)
#expect(res.ok == true)
#expect(notifier.addedRequests.count == 1)
let request = try #require(notifier.addedRequests.first)
#expect(request.content.title == "OpenClaw")
#expect(request.content.body == "Ping")
let payloadJSON = try #require(res.payloadJSON)
let decoded = try JSONDecoder().decode(OpenClawChatPushPayload.self, from: Data(payloadJSON.utf8))
#expect((decoded.messageId ?? "").isEmpty == false)
#expect(request.identifier == decoded.messageId)
}
@Test @MainActor func handleInvokeDeviceAndDataCommandsReturnPayloads() async throws {
let deviceStatusPayload = OpenClawDeviceStatusPayload(
battery: OpenClawBatteryStatusPayload(level: 0.25, state: .unplugged, lowPowerModeEnabled: false),
thermal: OpenClawThermalStatusPayload(state: .fair),
storage: OpenClawStorageStatusPayload(totalBytes: 200, freeBytes: 80, usedBytes: 120),
network: OpenClawNetworkStatusPayload(
status: .satisfied,
isExpensive: true,
isConstrained: false,
interfaces: [.cellular]),
uptimeSeconds: 42)
let deviceInfoPayload = OpenClawDeviceInfoPayload(
deviceName: "TestPhone",
modelIdentifier: "Test2,1",
systemName: "iOS",
systemVersion: "2.0",
appVersion: "dev",
appBuild: "1",
locale: "en-US")
let photosPayload = OpenClawPhotosLatestPayload(
photos: [
OpenClawPhotoPayload(format: "jpeg", base64: "dGVzdA==", width: 1, height: 1, createdAt: nil),
])
let contactsPayload = OpenClawContactsSearchPayload(
contacts: [
OpenClawContactPayload(
identifier: "c1",
displayName: "Jane Doe",
givenName: "Jane",
familyName: "Doe",
organizationName: "",
phoneNumbers: ["+1"],
emails: ["jane@example.com"]),
])
let contactsAddPayload = OpenClawContactsAddPayload(
contact: OpenClawContactPayload(
identifier: "c2",
displayName: "Added",
givenName: "Added",
familyName: "",
organizationName: "",
phoneNumbers: ["+2"],
emails: ["add@example.com"]))
let calendarPayload = OpenClawCalendarEventsPayload(
events: [
OpenClawCalendarEventPayload(
identifier: "e1",
title: "Standup",
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T00:30:00Z",
isAllDay: false,
location: nil,
calendarTitle: "Work"),
])
let calendarAddPayload = OpenClawCalendarAddPayload(
event: OpenClawCalendarEventPayload(
identifier: "e2",
title: "Added Event",
startISO: "2024-01-02T00:00:00Z",
endISO: "2024-01-02T01:00:00Z",
isAllDay: false,
location: "HQ",
calendarTitle: "Work"))
let remindersPayload = OpenClawRemindersListPayload(
reminders: [
OpenClawReminderPayload(
identifier: "r1",
title: "Ship build",
dueISO: "2024-01-02T00:00:00Z",
completed: false,
listName: "Inbox"),
])
let remindersAddPayload = OpenClawRemindersAddPayload(
reminder: OpenClawReminderPayload(
identifier: "r2",
title: "Added Reminder",
dueISO: "2024-01-03T00:00:00Z",
completed: false,
listName: "Inbox"))
let motionPayload = OpenClawMotionActivityPayload(
activities: [
OpenClawMotionActivityEntry(
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T00:10:00Z",
confidence: "high",
isWalking: true,
isRunning: false,
isCycling: false,
isAutomotive: false,
isStationary: false,
isUnknown: false),
])
let pedometerPayload = OpenClawPedometerPayload(
startISO: "2024-01-01T00:00:00Z",
endISO: "2024-01-01T01:00:00Z",
steps: 123,
distanceMeters: 456,
floorsAscended: 1,
floorsDescended: 2)
let appModel = makeTestAppModel(
deviceStatusService: TestDeviceStatusService(
statusPayload: deviceStatusPayload,
infoPayload: deviceInfoPayload),
photosService: TestPhotosService(payload: photosPayload),
contactsService: TestContactsService(
searchPayload: contactsPayload,
addPayload: contactsAddPayload),
calendarService: TestCalendarService(
eventsPayload: calendarPayload,
addPayload: calendarAddPayload),
remindersService: TestRemindersService(
listPayload: remindersPayload,
addPayload: remindersAddPayload),
motionService: TestMotionService(
activityPayload: motionPayload,
pedometerPayload: pedometerPayload))
let deviceStatusReq = BridgeInvokeRequest(id: "device", command: OpenClawDeviceCommand.status.rawValue)
let deviceStatusRes = await appModel._test_handleInvoke(deviceStatusReq)
#expect(deviceStatusRes.ok == true)
let decodedDeviceStatus = try decodePayload(deviceStatusRes.payloadJSON, as: OpenClawDeviceStatusPayload.self)
#expect(decodedDeviceStatus == deviceStatusPayload)
let deviceInfoReq = BridgeInvokeRequest(id: "device-info", command: OpenClawDeviceCommand.info.rawValue)
let deviceInfoRes = await appModel._test_handleInvoke(deviceInfoReq)
#expect(deviceInfoRes.ok == true)
let decodedDeviceInfo = try decodePayload(deviceInfoRes.payloadJSON, as: OpenClawDeviceInfoPayload.self)
#expect(decodedDeviceInfo == deviceInfoPayload)
let photosReq = BridgeInvokeRequest(id: "photos", command: OpenClawPhotosCommand.latest.rawValue)
let photosRes = await appModel._test_handleInvoke(photosReq)
#expect(photosRes.ok == true)
let decodedPhotos = try decodePayload(photosRes.payloadJSON, as: OpenClawPhotosLatestPayload.self)
#expect(decodedPhotos == photosPayload)
let contactsReq = BridgeInvokeRequest(id: "contacts", command: OpenClawContactsCommand.search.rawValue)
let contactsRes = await appModel._test_handleInvoke(contactsReq)
#expect(contactsRes.ok == true)
let decodedContacts = try decodePayload(contactsRes.payloadJSON, as: OpenClawContactsSearchPayload.self)
#expect(decodedContacts == contactsPayload)
let contactsAddParams = OpenClawContactsAddParams(
givenName: "Added",
phoneNumbers: ["+2"],
emails: ["add@example.com"])
let contactsAddData = try JSONEncoder().encode(contactsAddParams)
let contactsAddReq = BridgeInvokeRequest(
id: "contacts-add",
command: OpenClawContactsCommand.add.rawValue,
paramsJSON: String(decoding: contactsAddData, as: UTF8.self))
let contactsAddRes = await appModel._test_handleInvoke(contactsAddReq)
#expect(contactsAddRes.ok == true)
let decodedContactsAdd = try decodePayload(contactsAddRes.payloadJSON, as: OpenClawContactsAddPayload.self)
#expect(decodedContactsAdd == contactsAddPayload)
let calendarReq = BridgeInvokeRequest(id: "calendar", command: OpenClawCalendarCommand.events.rawValue)
let calendarRes = await appModel._test_handleInvoke(calendarReq)
#expect(calendarRes.ok == true)
let decodedCalendar = try decodePayload(calendarRes.payloadJSON, as: OpenClawCalendarEventsPayload.self)
#expect(decodedCalendar == calendarPayload)
let calendarAddParams = OpenClawCalendarAddParams(
title: "Added Event",
startISO: "2024-01-02T00:00:00Z",
endISO: "2024-01-02T01:00:00Z",
location: "HQ",
calendarTitle: "Work")
let calendarAddData = try JSONEncoder().encode(calendarAddParams)
let calendarAddReq = BridgeInvokeRequest(
id: "calendar-add",
command: OpenClawCalendarCommand.add.rawValue,
paramsJSON: String(decoding: calendarAddData, as: UTF8.self))
let calendarAddRes = await appModel._test_handleInvoke(calendarAddReq)
#expect(calendarAddRes.ok == true)
let decodedCalendarAdd = try decodePayload(calendarAddRes.payloadJSON, as: OpenClawCalendarAddPayload.self)
#expect(decodedCalendarAdd == calendarAddPayload)
let remindersReq = BridgeInvokeRequest(id: "reminders", command: OpenClawRemindersCommand.list.rawValue)
let remindersRes = await appModel._test_handleInvoke(remindersReq)
#expect(remindersRes.ok == true)
let decodedReminders = try decodePayload(remindersRes.payloadJSON, as: OpenClawRemindersListPayload.self)
#expect(decodedReminders == remindersPayload)
let remindersAddParams = OpenClawRemindersAddParams(
title: "Added Reminder",
dueISO: "2024-01-03T00:00:00Z",
listName: "Inbox")
let remindersAddData = try JSONEncoder().encode(remindersAddParams)
let remindersAddReq = BridgeInvokeRequest(
id: "reminders-add",
command: OpenClawRemindersCommand.add.rawValue,
paramsJSON: String(decoding: remindersAddData, as: UTF8.self))
let remindersAddRes = await appModel._test_handleInvoke(remindersAddReq)
#expect(remindersAddRes.ok == true)
let decodedRemindersAdd = try decodePayload(remindersAddRes.payloadJSON, as: OpenClawRemindersAddPayload.self)
#expect(decodedRemindersAdd == remindersAddPayload)
let motionReq = BridgeInvokeRequest(id: "motion", command: OpenClawMotionCommand.activity.rawValue)
let motionRes = await appModel._test_handleInvoke(motionReq)
#expect(motionRes.ok == true)
let decodedMotion = try decodePayload(motionRes.payloadJSON, as: OpenClawMotionActivityPayload.self)
#expect(decodedMotion == motionPayload)
let pedometerReq = BridgeInvokeRequest(id: "pedometer", command: OpenClawMotionCommand.pedometer.rawValue)
let pedometerRes = await appModel._test_handleInvoke(pedometerReq)
#expect(pedometerRes.ok == true)
let decodedPedometer = try decodePayload(pedometerRes.payloadJSON, as: OpenClawPedometerPayload.self)
#expect(decodedPedometer == pedometerPayload)
}
@Test @MainActor func handleInvokePushToTalkReturnsTranscriptStatus() async throws {
let talkMode = TalkModeManager(allowSimulatorCapture: true)
talkMode.updateGatewayConnected(false)
let appModel = makeTalkTestAppModel(talkMode: talkMode)
let startReq = BridgeInvokeRequest(id: "ptt-start", command: OpenClawTalkCommand.pttStart.rawValue)
let startRes = await appModel._test_handleInvoke(startReq)
#expect(startRes.ok == true)
let startPayload = try decodePayload(startRes.payloadJSON, as: OpenClawTalkPTTStartPayload.self)
#expect(!startPayload.captureId.isEmpty)
talkMode._test_seedTranscript("Hello from PTT")
let stopReq = BridgeInvokeRequest(id: "ptt-stop", command: OpenClawTalkCommand.pttStop.rawValue)
let stopRes = await appModel._test_handleInvoke(stopReq)
#expect(stopRes.ok == true)
let stopPayload = try decodePayload(stopRes.payloadJSON, as: OpenClawTalkPTTStopPayload.self)
#expect(stopPayload.captureId == startPayload.captureId)
#expect(stopPayload.transcript == "Hello from PTT")
#expect(stopPayload.status == "offline")
}
@Test @MainActor func handleInvokePushToTalkCancelStopsSession() async throws {
let talkMode = TalkModeManager(allowSimulatorCapture: true)
talkMode.updateGatewayConnected(false)
let appModel = makeTalkTestAppModel(talkMode: talkMode)
let startReq = BridgeInvokeRequest(id: "ptt-start", command: OpenClawTalkCommand.pttStart.rawValue)
let startRes = await appModel._test_handleInvoke(startReq)
#expect(startRes.ok == true)
let startPayload = try decodePayload(startRes.payloadJSON, as: OpenClawTalkPTTStartPayload.self)
let cancelReq = BridgeInvokeRequest(id: "ptt-cancel", command: OpenClawTalkCommand.pttCancel.rawValue)
let cancelRes = await appModel._test_handleInvoke(cancelReq)
#expect(cancelRes.ok == true)
let cancelPayload = try decodePayload(cancelRes.payloadJSON, as: OpenClawTalkPTTStopPayload.self)
#expect(cancelPayload.captureId == startPayload.captureId)
#expect(cancelPayload.status == "cancelled")
}
@Test @MainActor func handleInvokePushToTalkOnceAutoStopsAfterSilence() async throws {
let talkMode = TalkModeManager(allowSimulatorCapture: true)
talkMode.updateGatewayConnected(false)
let appModel = makeTalkTestAppModel(talkMode: talkMode)
let onceReq = BridgeInvokeRequest(id: "ptt-once", command: OpenClawTalkCommand.pttOnce.rawValue)
let onceTask = Task { await appModel._test_handleInvoke(onceReq) }
for _ in 0..<5 where !talkMode.isPushToTalkActive {
await Task.yield()
}
#expect(talkMode.isPushToTalkActive == true)
talkMode._test_seedTranscript("Hello from PTT once")
talkMode._test_backdateLastHeard(seconds: 1.0)
await talkMode._test_runSilenceCheck()
let onceRes = await onceTask.value
#expect(onceRes.ok == true)
let oncePayload = try decodePayload(onceRes.payloadJSON, as: OpenClawTalkPTTStopPayload.self)
#expect(oncePayload.transcript == "Hello from PTT once")
#expect(oncePayload.status == "offline")
}
@Test @MainActor func handleInvokePushToTalkOnceStopsOnFinalTranscript() async throws {
let talkMode = TalkModeManager(allowSimulatorCapture: true)
talkMode.updateGatewayConnected(false)
let appModel = makeTalkTestAppModel(talkMode: talkMode)
let onceReq = BridgeInvokeRequest(id: "ptt-once-final", command: OpenClawTalkCommand.pttOnce.rawValue)
let onceTask = Task { await appModel._test_handleInvoke(onceReq) }
for _ in 0..<5 where !talkMode.isPushToTalkActive {
await Task.yield()
}
#expect(talkMode.isPushToTalkActive == true)
await talkMode._test_handleTranscript("Hello final", isFinal: true)
let onceRes = await onceTask.value
#expect(onceRes.ok == true)
let oncePayload = try decodePayload(onceRes.payloadJSON, as: OpenClawTalkPTTStopPayload.self)
#expect(oncePayload.transcript == "Hello final")
#expect(oncePayload.status == "offline")
}
@Test @MainActor func handleDeepLinkSetsErrorWhenNotConnected() async {
let appModel = NodeAppModel()
let url = URL(string: "openclaw://agent?message=hello")!

34
apps/ios/Tests/NodeDisplayNameTests.swift
View File

@@ -1,34 +0,0 @@
import Testing
@testable import OpenClaw
struct NodeDisplayNameTests {
@Test func keepsCustomName() {
let resolved = NodeDisplayName.resolve(
existing: "Razor Phone",
deviceName: "iPhone",
interfaceIdiom: .phone)
#expect(resolved == "Razor Phone")
}
@Test func usesDeviceNameWhenMatchesIphone() {
let resolved = NodeDisplayName.resolve(
existing: "iOS Node",
deviceName: "iPhone 17 Pro",
interfaceIdiom: .phone)
#expect(resolved == "iPhone 17 Pro")
}
@Test func usesDefaultWhenDeviceNameIsGeneric() {
let resolved = NodeDisplayName.resolve(
existing: nil,
deviceName: "Work Phone",
interfaceIdiom: .phone)
#expect(NodeDisplayName.isGeneric(resolved))
}
@Test func identifiesGenericValues() {
#expect(NodeDisplayName.isGeneric("iOS Node"))
#expect(NodeDisplayName.isGeneric("iPhone Node"))
#expect(NodeDisplayName.isGeneric("iPad Node"))
}
}

33
apps/ios/Tests/TalkModeIncrementalTests.swift
View File

@@ -1,33 +0,0 @@
import Foundation
import Testing
@testable import OpenClaw
@Suite struct TalkModeIncrementalTests {
@Test @MainActor func incrementalSpeechSplitsOnBoundary() {
let talkMode = TalkModeManager(allowSimulatorCapture: true)
talkMode._test_incrementalReset()
let segments = talkMode._test_incrementalIngest("Hello world. Next", isFinal: false)
#expect(segments.count == 1)
#expect(segments.first == "Hello world.")
}
@Test @MainActor func incrementalSpeechSkipsDirectiveLine() {
let talkMode = TalkModeManager(allowSimulatorCapture: true)
talkMode._test_incrementalReset()
let segments = talkMode._test_incrementalIngest("{\"voice\":\"abc\"}\nHello.", isFinal: false)
#expect(segments.count == 1)
#expect(segments.first == "Hello.")
}
@Test @MainActor func incrementalSpeechIgnoresCodeBlocks() {
let talkMode = TalkModeManager(allowSimulatorCapture: true)
talkMode._test_incrementalReset()
let text = "Here is code:\n```js\nx=1\n```\nDone."
let segments = talkMode._test_incrementalIngest(text, isFinal: true)
#expect(segments.count == 1)
let value = segments.first ?? ""
#expect(value.contains("x=1") == false)
#expect(value.contains("Here is code") == true)
#expect(value.contains("Done.") == true)
}
}

18
apps/macos/Sources/OpenClaw/GatewayConnection.swift
View File

@@ -360,12 +360,11 @@ actor GatewayConnection {
await client.shutdown()
}
self.lastSnapshot = nil
let resolvedSessionBox = self.sessionBox ?? Self.buildSessionBox(url: url)
self.client = GatewayChannelActor(
url: url,
token: token,
password: password,
session: resolvedSessionBox,
session: self.sessionBox,
pushHandler: { [weak self] push in
await self?.handle(push: push)
})
@@ -381,21 +380,6 @@ actor GatewayConnection {
private static func defaultConfigProvider() async throws -> Config {
try await GatewayEndpointStore.shared.requireConfig()
}
private static func buildSessionBox(url: URL) -> WebSocketSessionBox? {
guard url.scheme?.lowercased() == "wss" else { return nil }
let host = url.host ?? "gateway"
let port = url.port ?? 443
let stableID = "\(host):\(port)"
let stored = GatewayTLSStore.loadFingerprint(stableID: stableID)
let params = GatewayTLSParams(
required: true,
expectedFingerprint: stored,
allowTOFU: true,
storeKey: stableID)
let session = GatewayTLSPinningSession(params: params)
return WebSocketSessionBox(session: session)
}
}
// MARK: - Typed gateway API

93
apps/shared/OpenClawKit/Sources/OpenClawKit/CalendarCommands.swift
View File

@@ -1,93 +0,0 @@
import Foundation
public enum OpenClawCalendarCommand: String, Codable, Sendable {
case events = "calendar.events"
case add = "calendar.add"
}
public struct OpenClawCalendarEventsParams: Codable, Sendable, Equatable {
public var startISO: String?
public var endISO: String?
public var limit: Int?
public init(startISO: String? = nil, endISO: String? = nil, limit: Int? = nil) {
self.startISO = startISO
self.endISO = endISO
self.limit = limit
}
}
public struct OpenClawCalendarAddParams: Codable, Sendable, Equatable {
public var title: String
public var startISO: String
public var endISO: String
public var isAllDay: Bool?
public var location: String?
public var notes: String?
public var calendarId: String?
public var calendarTitle: String?
public init(
title: String,
startISO: String,
endISO: String,
isAllDay: Bool? = nil,
location: String? = nil,
notes: String? = nil,
calendarId: String? = nil,
calendarTitle: String? = nil)
{
self.title = title
self.startISO = startISO
self.endISO = endISO
self.isAllDay = isAllDay
self.location = location
self.notes = notes
self.calendarId = calendarId
self.calendarTitle = calendarTitle
}
}
public struct OpenClawCalendarEventPayload: Codable, Sendable, Equatable {
public var identifier: String
public var title: String
public var startISO: String
public var endISO: String
public var isAllDay: Bool
public var location: String?
public var calendarTitle: String?
public init(
identifier: String,
title: String,
startISO: String,
endISO: String,
isAllDay: Bool,
location: String? = nil,
calendarTitle: String? = nil)
{
self.identifier = identifier
self.title = title
self.startISO = startISO
self.endISO = endISO
self.isAllDay = isAllDay
self.location = location
self.calendarTitle = calendarTitle
}
}
public struct OpenClawCalendarEventsPayload: Codable, Sendable, Equatable {
public var events: [OpenClawCalendarEventPayload]
public init(events: [OpenClawCalendarEventPayload]) {
self.events = events
}
}
public struct OpenClawCalendarAddPayload: Codable, Sendable, Equatable {
public var event: OpenClawCalendarEventPayload
public init(event: OpenClawCalendarEventPayload) {
self.event = event
}
}

6
apps/shared/OpenClawKit/Sources/OpenClawKit/Capabilities.swift
View File

@@ -6,10 +6,4 @@ public enum OpenClawCapability: String, Codable, Sendable {
case screen
case voiceWake
case location
case device
case photos
case contacts
case calendar
case reminders
case motion
}

23
apps/shared/OpenClawKit/Sources/OpenClawKit/ChatCommands.swift
View File

@@ -1,23 +0,0 @@
import Foundation
public enum OpenClawChatCommand: String, Codable, Sendable {
case push = "chat.push"
}
public struct OpenClawChatPushParams: Codable, Sendable, Equatable {
public var text: String
public var speak: Bool?
public init(text: String, speak: Bool? = nil) {
self.text = text
self.speak = speak
}
}
public struct OpenClawChatPushPayload: Codable, Sendable, Equatable {
public var messageId: String?
public init(messageId: String? = nil) {
self.messageId = messageId
}
}

85
apps/shared/OpenClawKit/Sources/OpenClawKit/ContactsCommands.swift
View File

@@ -1,85 +0,0 @@
import Foundation
public enum OpenClawContactsCommand: String, Codable, Sendable {
case search = "contacts.search"
case add = "contacts.add"
}
public struct OpenClawContactsSearchParams: Codable, Sendable, Equatable {
public var query: String?
public var limit: Int?
public init(query: String? = nil, limit: Int? = nil) {
self.query = query
self.limit = limit
}
}
public struct OpenClawContactsAddParams: Codable, Sendable, Equatable {
public var givenName: String?
public var familyName: String?
public var organizationName: String?
public var displayName: String?
public var phoneNumbers: [String]?
public var emails: [String]?
public init(
givenName: String? = nil,
familyName: String? = nil,
organizationName: String? = nil,
displayName: String? = nil,
phoneNumbers: [String]? = nil,
emails: [String]? = nil)
{
self.givenName = givenName
self.familyName = familyName
self.organizationName = organizationName
self.displayName = displayName
self.phoneNumbers = phoneNumbers
self.emails = emails
}
}
public struct OpenClawContactPayload: Codable, Sendable, Equatable {
public var identifier: String
public var displayName: String
public var givenName: String
public var familyName: String
public var organizationName: String
public var phoneNumbers: [String]
public var emails: [String]
public init(
identifier: String,
displayName: String,
givenName: String,
familyName: String,
organizationName: String,
phoneNumbers: [String],
emails: [String])
{
self.identifier = identifier
self.displayName = displayName
self.givenName = givenName
self.familyName = familyName
self.organizationName = organizationName
self.phoneNumbers = phoneNumbers
self.emails = emails
}
}
public struct OpenClawContactsSearchPayload: Codable, Sendable, Equatable {
public var contacts: [OpenClawContactPayload]
public init(contacts: [OpenClawContactPayload]) {
self.contacts = contacts
}
}
public struct OpenClawContactsAddPayload: Codable, Sendable, Equatable {
public var contact: OpenClawContactPayload
public init(contact: OpenClawContactPayload) {
self.contact = contact
}
}

134
apps/shared/OpenClawKit/Sources/OpenClawKit/DeviceCommands.swift
View File

@@ -1,134 +0,0 @@
import Foundation
public enum OpenClawDeviceCommand: String, Codable, Sendable {
case status = "device.status"
case info = "device.info"
}
public enum OpenClawBatteryState: String, Codable, Sendable {
case unknown
case unplugged
case charging
case full
}
public enum OpenClawThermalState: String, Codable, Sendable {
case nominal
case fair
case serious
case critical
}
public enum OpenClawNetworkPathStatus: String, Codable, Sendable {
case satisfied
case unsatisfied
case requiresConnection
}
public enum OpenClawNetworkInterfaceType: String, Codable, Sendable {
case wifi
case cellular
case wired
case other
}
public struct OpenClawBatteryStatusPayload: Codable, Sendable, Equatable {
public var level: Double?
public var state: OpenClawBatteryState
public var lowPowerModeEnabled: Bool
public init(level: Double?, state: OpenClawBatteryState, lowPowerModeEnabled: Bool) {
self.level = level
self.state = state
self.lowPowerModeEnabled = lowPowerModeEnabled
}
}
public struct OpenClawThermalStatusPayload: Codable, Sendable, Equatable {
public var state: OpenClawThermalState
public init(state: OpenClawThermalState) {
self.state = state
}
}
public struct OpenClawStorageStatusPayload: Codable, Sendable, Equatable {
public var totalBytes: Int64
public var freeBytes: Int64
public var usedBytes: Int64
public init(totalBytes: Int64, freeBytes: Int64, usedBytes: Int64) {
self.totalBytes = totalBytes
self.freeBytes = freeBytes
self.usedBytes = usedBytes
}
}
public struct OpenClawNetworkStatusPayload: Codable, Sendable, Equatable {
public var status: OpenClawNetworkPathStatus
public var isExpensive: Bool
public var isConstrained: Bool
public var interfaces: [OpenClawNetworkInterfaceType]
public init(
status: OpenClawNetworkPathStatus,
isExpensive: Bool,
isConstrained: Bool,
interfaces: [OpenClawNetworkInterfaceType])
{
self.status = status
self.isExpensive = isExpensive
self.isConstrained = isConstrained
self.interfaces = interfaces
}
}
public struct OpenClawDeviceStatusPayload: Codable, Sendable, Equatable {
public var battery: OpenClawBatteryStatusPayload
public var thermal: OpenClawThermalStatusPayload
public var storage: OpenClawStorageStatusPayload
public var network: OpenClawNetworkStatusPayload
public var uptimeSeconds: Double
public init(
battery: OpenClawBatteryStatusPayload,
thermal: OpenClawThermalStatusPayload,
storage: OpenClawStorageStatusPayload,
network: OpenClawNetworkStatusPayload,
uptimeSeconds: Double)
{
self.battery = battery
self.thermal = thermal
self.storage = storage
self.network = network
self.uptimeSeconds = uptimeSeconds
}
}
public struct OpenClawDeviceInfoPayload: Codable, Sendable, Equatable {
public var deviceName: String
public var modelIdentifier: String
public var systemName: String
public var systemVersion: String
public var appVersion: String
public var appBuild: String
public var locale: String
public init(
deviceName: String,
modelIdentifier: String,
systemName: String,
systemVersion: String,
appVersion: String,
appBuild: String,
locale: String)
{
self.deviceName = deviceName
self.modelIdentifier = modelIdentifier
self.systemName = systemName
self.systemVersion = systemVersion
self.appVersion = appVersion
self.appBuild = appBuild
self.locale = locale
}
}

60
apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayChannel.swift
View File

@@ -110,13 +110,7 @@ private enum ConnectChallengeError: Error {
public actor GatewayChannelActor {
private let logger = Logger(subsystem: "ai.openclaw", category: "gateway")
#if DEBUG
private var debugEventLogCount = 0
private var debugMessageLogCount = 0
private var debugListenLogCount = 0
#endif
private var task: WebSocketTaskBox?
private var listenTask: Task<Void, Never>?
private var pending: [String: CheckedContinuation<GatewayFrame, Error>] = [:]
private var connected = false
private var isConnecting = false
@@ -175,9 +169,6 @@ public actor GatewayChannelActor {
self.tickTask?.cancel()
self.tickTask = nil
self.listenTask?.cancel()
self.listenTask = nil
self.task?.cancel(with: .goingAway, reason: nil)
self.task = nil
@@ -230,8 +221,6 @@ public actor GatewayChannelActor {
self.isConnecting = true
defer { self.isConnecting = false }
self.listenTask?.cancel()
self.listenTask = nil
self.task?.cancel(with: .goingAway, reason: nil)
self.task = self.session.makeWebSocketTask(url: self.url)
self.task?.resume()
@@ -259,7 +248,6 @@ public actor GatewayChannelActor {
throw wrapped
}
self.listen()
self.logger.info("gateway ws listen registered")
self.connected = true
self.backoffMs = 500
self.lastSeq = nil
@@ -432,44 +420,24 @@ public actor GatewayChannelActor {
}
private func listen() {
#if DEBUG
if self.debugListenLogCount < 3 {
self.debugListenLogCount += 1
self.logger.info("gateway ws listen start")
}
#endif
self.listenTask?.cancel()
self.listenTask = Task { [weak self] in
self.task?.receive { [weak self] result in
guard let self else { return }
defer { Task { await self.clearListenTask() } }
while !Task.isCancelled {
guard let task = await self.currentTask() else { return }
do {
let msg = try await task.receive()
switch result {
case let .failure(err):
Task { await self.handleReceiveFailure(err) }
case let .success(msg):
Task {
await self.handle(msg)
} catch {
if Task.isCancelled { return }
await self.handleReceiveFailure(error)
return
await self.listen()
}
}
}
}
private func clearListenTask() {
self.listenTask = nil
}
private func currentTask() -> WebSocketTaskBox? {
self.task
}
private func handleReceiveFailure(_ err: Error) async {
let wrapped = self.wrap(err, context: "gateway receive")
self.logger.error("gateway ws receive failed \(wrapped.localizedDescription, privacy: .public)")
self.connected = false
self.listenTask?.cancel()
self.listenTask = nil
await self.disconnectHandler?("receive failed: \(wrapped.localizedDescription)")
await self.failPending(wrapped)
await self.scheduleReconnect()
@@ -481,13 +449,6 @@ public actor GatewayChannelActor {
case let .string(s): s.data(using: .utf8)
@unknown default: nil
}
#if DEBUG
if self.debugMessageLogCount < 8 {
self.debugMessageLogCount += 1
let size = data?.count ?? 0
self.logger.info("gateway ws message received size=\(size, privacy: .public)")
}
#endif
guard let data else { return }
guard let frame = try? self.decoder.decode(GatewayFrame.self, from: data) else {
self.logger.error("gateway decode failed")
@@ -501,13 +462,6 @@ public actor GatewayChannelActor {
}
case let .event(evt):
if evt.event == "connect.challenge" { return }
#if DEBUG
if self.debugEventLogCount < 12 {
self.debugEventLogCount += 1
self.logger.info(
"gateway event received event=\(evt.event, privacy: .public) payload=\(evt.payload != nil, privacy: .public)")
}
#endif
if let seq = evt.seq {
if let last = lastSeq, seq > last + 1 {
await self.pushHandler?(.seqGap(expected: last + 1, received: seq))

151
apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayNodeSession.swift
View File

@@ -11,7 +11,6 @@ private struct NodeInvokeRequestPayload: Codable, Sendable {
var idempotencyKey: String?
}
public actor GatewayNodeSession {
private let logger = Logger(subsystem: "ai.openclaw", category: "node.gateway")
private let decoder = JSONDecoder()
@@ -24,78 +23,34 @@ public actor GatewayNodeSession {
private var onConnected: (@Sendable () async -> Void)?
private var onDisconnected: (@Sendable (String) async -> Void)?
private var onInvoke: (@Sendable (BridgeInvokeRequest) async -> BridgeInvokeResponse)?
private var hasNotifiedConnected = false
private var snapshotReceived = false
private var snapshotWaiters: [CheckedContinuation<Bool, Never>] = []
static func invokeWithTimeout(
request: BridgeInvokeRequest,
timeoutMs: Int?,
onInvoke: @escaping @Sendable (BridgeInvokeRequest) async -> BridgeInvokeResponse
) async -> BridgeInvokeResponse {
let timeoutLogger = Logger(subsystem: "ai.openclaw", category: "node.gateway")
let timeout: Int = {
guard let timeoutMs else { return 0 }
return max(0, timeoutMs)
}()
let timeout = max(0, timeoutMs ?? 0)
guard timeout > 0 else {
return await onInvoke(request)
}
// Use an explicit latch so timeouts win even if onInvoke blocks (e.g., permission prompts).
final class InvokeLatch: @unchecked Sendable {
private let lock = NSLock()
private var continuation: CheckedContinuation<BridgeInvokeResponse, Never>?
private var resumed = false
func setContinuation(_ continuation: CheckedContinuation<BridgeInvokeResponse, Never>) {
self.lock.lock()
defer { self.lock.unlock() }
self.continuation = continuation
}
func resume(_ response: BridgeInvokeResponse) {
let cont: CheckedContinuation<BridgeInvokeResponse, Never>?
self.lock.lock()
if self.resumed {
self.lock.unlock()
return
}
self.resumed = true
cont = self.continuation
self.continuation = nil
self.lock.unlock()
cont?.resume(returning: response)
}
}
let latch = InvokeLatch()
var onInvokeTask: Task<Void, Never>?
var timeoutTask: Task<Void, Never>?
defer {
onInvokeTask?.cancel()
timeoutTask?.cancel()
}
let response = await withCheckedContinuation { (cont: CheckedContinuation<BridgeInvokeResponse, Never>) in
latch.setContinuation(cont)
onInvokeTask = Task.detached {
let result = await onInvoke(request)
latch.resume(result)
}
timeoutTask = Task.detached {
return await withTaskGroup(of: BridgeInvokeResponse.self) { group in
group.addTask { await onInvoke(request) }
group.addTask {
try? await Task.sleep(nanoseconds: UInt64(timeout) * 1_000_000)
timeoutLogger.info("node invoke timeout fired id=\(request.id, privacy: .public)")
latch.resume(BridgeInvokeResponse(
return BridgeInvokeResponse(
id: request.id,
ok: false,
error: OpenClawNodeError(
code: .unavailable,
message: "node invoke timed out")
))
)
}
let first = await group.next()!
group.cancelAll()
return first
}
timeoutLogger.info("node invoke race resolved id=\(request.id, privacy: .public) ok=\(response.ok, privacy: .public)")
return response
}
private var serverEventSubscribers: [UUID: AsyncStream<EventFrame>.Continuation] = [:]
private var canvasHostUrl: String?
@@ -123,7 +78,6 @@ public actor GatewayNodeSession {
self.onInvoke = onInvoke
if shouldReconnect {
self.resetConnectionState()
if let existing = self.channel {
await existing.shutdown()
}
@@ -153,10 +107,7 @@ public actor GatewayNodeSession {
do {
try await channel.connect()
let snapshotReady = await self.waitForSnapshot(timeoutMs: 500)
if snapshotReady {
await self.notifyConnectedIfNeeded()
}
await onConnected()
} catch {
await onDisconnected(error.localizedDescription)
throw error
@@ -169,7 +120,6 @@ public actor GatewayNodeSession {
self.activeURL = nil
self.activeToken = nil
self.activePassword = nil
self.resetConnectionState()
}
public func currentCanvasHostUrl() -> String? {
@@ -229,8 +179,7 @@ public actor GatewayNodeSession {
case let .snapshot(ok):
let raw = ok.canvashosturl?.trimmingCharacters(in: .whitespacesAndNewlines)
self.canvasHostUrl = (raw?.isEmpty == false) ? raw : nil
self.markSnapshotReceived()
await self.notifyConnectedIfNeeded()
await self.onConnected?()
case let .event(evt):
await self.handleEvent(evt)
default:
@@ -238,98 +187,28 @@ public actor GatewayNodeSession {
}
}
private func resetConnectionState() {
self.hasNotifiedConnected = false
self.snapshotReceived = false
if !self.snapshotWaiters.isEmpty {
let waiters = self.snapshotWaiters
self.snapshotWaiters.removeAll()
for waiter in waiters {
waiter.resume(returning: false)
}
}
}
private func markSnapshotReceived() {
self.snapshotReceived = true
if !self.snapshotWaiters.isEmpty {
let waiters = self.snapshotWaiters
self.snapshotWaiters.removeAll()
for waiter in waiters {
waiter.resume(returning: true)
}
}
}
private func waitForSnapshot(timeoutMs: Int) async -> Bool {
if self.snapshotReceived { return true }
let clamped = max(0, timeoutMs)
return await withCheckedContinuation { cont in
self.snapshotWaiters.append(cont)
Task { [weak self] in
guard let self else { return }
try? await Task.sleep(nanoseconds: UInt64(clamped) * 1_000_000)
await self.timeoutSnapshotWaiters()
}
}
}
private func timeoutSnapshotWaiters() {
guard !self.snapshotReceived else { return }
if !self.snapshotWaiters.isEmpty {
let waiters = self.snapshotWaiters
self.snapshotWaiters.removeAll()
for waiter in waiters {
waiter.resume(returning: false)
}
}
}
private func notifyConnectedIfNeeded() async {
guard !self.hasNotifiedConnected else { return }
self.hasNotifiedConnected = true
await self.onConnected?()
}
private func handleEvent(_ evt: EventFrame) async {
self.broadcastServerEvent(evt)
guard evt.event == "node.invoke.request" else { return }
self.logger.info("node invoke request received")
guard let payload = evt.payload else { return }
do {
let request = try self.decodeInvokeRequest(from: payload)
let timeoutLabel = request.timeoutMs.map(String.init) ?? "none"
self.logger.info("node invoke request decoded id=\(request.id, privacy: .public) command=\(request.command, privacy: .public) timeoutMs=\(timeoutLabel, privacy: .public)")
let data = try self.encoder.encode(payload)
let request = try self.decoder.decode(NodeInvokeRequestPayload.self, from: data)
guard let onInvoke else { return }
let req = BridgeInvokeRequest(id: request.id, command: request.command, paramsJSON: request.paramsJSON)
self.logger.info("node invoke executing id=\(request.id, privacy: .public)")
let response = await Self.invokeWithTimeout(
request: req,
timeoutMs: request.timeoutMs,
onInvoke: onInvoke
)
self.logger.info("node invoke completed id=\(request.id, privacy: .public) ok=\(response.ok, privacy: .public)")
await self.sendInvokeResult(request: request, response: response)
} catch {
self.logger.error("node invoke decode failed: \(error.localizedDescription, privacy: .public)")
}
}
private func decodeInvokeRequest(from payload: OpenClawProtocol.AnyCodable) throws -> NodeInvokeRequestPayload {
do {
let data = try self.encoder.encode(payload)
return try self.decoder.decode(NodeInvokeRequestPayload.self, from: data)
} catch {
if let raw = payload.value as? String, let data = raw.data(using: .utf8) {
return try self.decoder.decode(NodeInvokeRequestPayload.self, from: data)
}
throw error
}
}
private func sendInvokeResult(request: NodeInvokeRequestPayload, response: BridgeInvokeResponse) async {
guard let channel = self.channel else { return }
self.logger.info("node invoke result sending id=\(request.id, privacy: .public) ok=\(response.ok, privacy: .public)")
var params: [String: AnyCodable] = [
"id": AnyCodable(request.id),
"nodeId": AnyCodable(request.nodeId),
@@ -347,7 +226,7 @@ public actor GatewayNodeSession {
do {
try await channel.send(method: "node.invoke.result", params: params)
} catch {
self.logger.error("node invoke result failed id=\(request.id, privacy: .public) error=\(error.localizedDescription, privacy: .public)")
self.logger.error("node invoke result failed: \(error.localizedDescription, privacy: .public)")
}
}

5
apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayTLSPinning.swift
View File

@@ -73,11 +73,6 @@ public final class GatewayTLSPinningSession: NSObject, WebSocketSessioning, URLS
if let expected {
if fingerprint == expected {
completionHandler(.useCredential, URLCredential(trust: trust))
} else if params.allowTOFU {
if let storeKey = params.storeKey {
GatewayTLSStore.saveFingerprint(fingerprint, stableID: storeKey)
}
completionHandler(.useCredential, URLCredential(trust: trust))
} else {
completionHandler(.cancelAuthenticationChallenge, nil)
}

95
apps/shared/OpenClawKit/Sources/OpenClawKit/MotionCommands.swift
View File

@@ -1,95 +0,0 @@
import Foundation
public enum OpenClawMotionCommand: String, Codable, Sendable {
case activity = "motion.activity"
case pedometer = "motion.pedometer"
}
public struct OpenClawMotionActivityParams: Codable, Sendable, Equatable {
public var startISO: String?
public var endISO: String?
public var limit: Int?
public init(startISO: String? = nil, endISO: String? = nil, limit: Int? = nil) {
self.startISO = startISO
self.endISO = endISO
self.limit = limit
}
}
public struct OpenClawMotionActivityEntry: Codable, Sendable, Equatable {
public var startISO: String
public var endISO: String
public var confidence: String
public var isWalking: Bool
public var isRunning: Bool
public var isCycling: Bool
public var isAutomotive: Bool
public var isStationary: Bool
public var isUnknown: Bool
public init(
startISO: String,
endISO: String,
confidence: String,
isWalking: Bool,
isRunning: Bool,
isCycling: Bool,
isAutomotive: Bool,
isStationary: Bool,
isUnknown: Bool)
{
self.startISO = startISO
self.endISO = endISO
self.confidence = confidence
self.isWalking = isWalking
self.isRunning = isRunning
self.isCycling = isCycling
self.isAutomotive = isAutomotive
self.isStationary = isStationary
self.isUnknown = isUnknown
}
}
public struct OpenClawMotionActivityPayload: Codable, Sendable, Equatable {
public var activities: [OpenClawMotionActivityEntry]
public init(activities: [OpenClawMotionActivityEntry]) {
self.activities = activities
}
}
public struct OpenClawPedometerParams: Codable, Sendable, Equatable {
public var startISO: String?
public var endISO: String?
public init(startISO: String? = nil, endISO: String? = nil) {
self.startISO = startISO
self.endISO = endISO
}
}
public struct OpenClawPedometerPayload: Codable, Sendable, Equatable {
public var startISO: String
public var endISO: String
public var steps: Int?
public var distanceMeters: Double?
public var floorsAscended: Int?
public var floorsDescended: Int?
public init(
startISO: String,
endISO: String,
steps: Int?,
distanceMeters: Double?,
floorsAscended: Int?,
floorsDescended: Int?)
{
self.startISO = startISO
self.endISO = endISO
self.steps = steps
self.distanceMeters = distanceMeters
self.floorsAscended = floorsAscended
self.floorsDescended = floorsDescended
}
}

41
apps/shared/OpenClawKit/Sources/OpenClawKit/PhotosCommands.swift
View File

@@ -1,41 +0,0 @@
import Foundation
public enum OpenClawPhotosCommand: String, Codable, Sendable {
case latest = "photos.latest"
}
public struct OpenClawPhotosLatestParams: Codable, Sendable, Equatable {
public var limit: Int?
public var maxWidth: Int?
public var quality: Double?
public init(limit: Int? = nil, maxWidth: Int? = nil, quality: Double? = nil) {
self.limit = limit
self.maxWidth = maxWidth
self.quality = quality
}
}
public struct OpenClawPhotoPayload: Codable, Sendable, Equatable {
public var format: String
public var base64: String
public var width: Int
public var height: Int
public var createdAt: String?
public init(format: String, base64: String, width: Int, height: Int, createdAt: String? = nil) {
self.format = format
self.base64 = base64
self.width = width
self.height = height
self.createdAt = createdAt
}
}
public struct OpenClawPhotosLatestPayload: Codable, Sendable, Equatable {
public var photos: [OpenClawPhotoPayload]
public init(photos: [OpenClawPhotoPayload]) {
self.photos = photos
}
}

82
apps/shared/OpenClawKit/Sources/OpenClawKit/RemindersCommands.swift
View File

@@ -1,82 +0,0 @@
import Foundation
public enum OpenClawRemindersCommand: String, Codable, Sendable {
case list = "reminders.list"
case add = "reminders.add"
}
public enum OpenClawReminderStatusFilter: String, Codable, Sendable {
case incomplete
case completed
case all
}
public struct OpenClawRemindersListParams: Codable, Sendable, Equatable {
public var status: OpenClawReminderStatusFilter?
public var limit: Int?
public init(status: OpenClawReminderStatusFilter? = nil, limit: Int? = nil) {
self.status = status
self.limit = limit
}
}
public struct OpenClawRemindersAddParams: Codable, Sendable, Equatable {
public var title: String
public var dueISO: String?
public var notes: String?
public var listId: String?
public var listName: String?
public init(
title: String,
dueISO: String? = nil,
notes: String? = nil,
listId: String? = nil,
listName: String? = nil)
{
self.title = title
self.dueISO = dueISO
self.notes = notes
self.listId = listId
self.listName = listName
}
}
public struct OpenClawReminderPayload: Codable, Sendable, Equatable {
public var identifier: String
public var title: String
public var dueISO: String?
public var completed: Bool
public var listName: String?
public init(
identifier: String,
title: String,
dueISO: String? = nil,
completed: Bool,
listName: String? = nil)
{
self.identifier = identifier
self.title = title
self.dueISO = dueISO
self.completed = completed
self.listName = listName
}
}
public struct OpenClawRemindersListPayload: Codable, Sendable, Equatable {
public var reminders: [OpenClawReminderPayload]
public init(reminders: [OpenClawReminderPayload]) {
self.reminders = reminders
}
}
public struct OpenClawRemindersAddPayload: Codable, Sendable, Equatable {
public var reminder: OpenClawReminderPayload
public init(reminder: OpenClawReminderPayload) {
self.reminder = reminder
}
}

4
apps/shared/OpenClawKit/Sources/OpenClawKit/Resources/tool-display.json
View File

@@ -123,10 +123,6 @@
"screen_record": {
"label": "screen record",
"detailKeys": ["node", "nodeId", "duration", "durationMs", "fps", "screenIndex"]
},
"invoke": {
"label": "invoke",
"detailKeys": ["node", "nodeId", "invokeCommand"]
}
}
},

28
apps/shared/OpenClawKit/Sources/OpenClawKit/TalkCommands.swift
View File

@@ -1,28 +0,0 @@
import Foundation
public enum OpenClawTalkCommand: String, Codable, Sendable {
case pttStart = "talk.ptt.start"
case pttStop = "talk.ptt.stop"
case pttCancel = "talk.ptt.cancel"
case pttOnce = "talk.ptt.once"
}
public struct OpenClawTalkPTTStartPayload: Codable, Sendable, Equatable {
public var captureId: String
public init(captureId: String) {
self.captureId = captureId
}
}
public struct OpenClawTalkPTTStopPayload: Codable, Sendable, Equatable {
public var captureId: String
public var transcript: String?
public var status: String
public init(captureId: String, transcript: String?, status: String) {
self.captureId = captureId
self.transcript = transcript
self.status = status
}
}

310
apps/shared/OpenClawKit/Tests/OpenClawKitTests/GatewayNodeInvokeTests.swift
View File

@@ -1,310 +0,0 @@
import Foundation
import Testing
@testable import OpenClawKit
import OpenClawProtocol
@Suite struct GatewayNodeInvokeTests {
@Test
func nodeInvokeRequestSendsInvokeResult() async throws {
let task = TestWebSocketTask()
let session = TestWebSocketSession(task: task)
task.enqueue(Self.makeEventMessage(
event: "connect.challenge",
payload: ["nonce": "test-nonce"]))
let tracker = InvokeTracker()
let gateway = GatewayNodeSession()
try await gateway.connect(
url: URL(string: "ws://127.0.0.1:18789")!,
token: nil,
password: "test-password",
connectOptions: GatewayConnectOptions(
role: "node",
scopes: [],
caps: [],
commands: ["device.info"],
permissions: [:],
clientId: "openclaw-ios",
clientMode: "node",
clientDisplayName: "Test iOS Node"),
sessionBox: WebSocketSessionBox(session: session),
onConnected: {},
onDisconnected: { _ in },
onInvoke: { req in
await tracker.set(req)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: "{\"ok\":true}")
})
task.enqueue(Self.makeEventMessage(
event: "node.invoke.request",
payload: [
"id": "invoke-1",
"nodeId": "node-1",
"command": "device.info",
"timeoutMs": 15000,
"idempotencyKey": "abc123",
]))
let resultFrame = try await waitForSentMethod(
task,
method: "node.invoke.result",
timeoutSeconds: 1.0)
let sentParams = resultFrame.params?.value as? [String: OpenClawProtocol.AnyCodable]
#expect(sentParams?["id"]?.value as? String == "invoke-1")
#expect(sentParams?["nodeId"]?.value as? String == "node-1")
#expect(sentParams?["ok"]?.value as? Bool == true)
let captured = await tracker.get()
#expect(captured?.command == "device.info")
#expect(captured?.id == "invoke-1")
}
@Test
func nodeInvokeRequestHandlesStringPayload() async throws {
let task = TestWebSocketTask()
let session = TestWebSocketSession(task: task)
task.enqueue(Self.makeEventMessage(
event: "connect.challenge",
payload: ["nonce": "test-nonce"]))
let tracker = InvokeTracker()
let gateway = GatewayNodeSession()
try await gateway.connect(
url: URL(string: "ws://127.0.0.1:18789")!,
token: nil,
password: "test-password",
connectOptions: GatewayConnectOptions(
role: "node",
scopes: [],
caps: [],
commands: ["device.info"],
permissions: [:],
clientId: "openclaw-ios",
clientMode: "node",
clientDisplayName: "Test iOS Node"),
sessionBox: WebSocketSessionBox(session: session),
onConnected: {},
onDisconnected: { _ in },
onInvoke: { req in
await tracker.set(req)
return BridgeInvokeResponse(id: req.id, ok: true)
})
let payload = """
{"id":"invoke-2","nodeId":"node-1","command":"device.info"}
"""
task.enqueue(Self.makeEventMessage(
event: "node.invoke.request",
payload: payload))
let resultFrame = try await waitForSentMethod(
task,
method: "node.invoke.result",
timeoutSeconds: 1.0)
let sentParams = resultFrame.params?.value as? [String: OpenClawProtocol.AnyCodable]
#expect(sentParams?["id"]?.value as? String == "invoke-2")
#expect(sentParams?["nodeId"]?.value as? String == "node-1")
#expect(sentParams?["ok"]?.value as? Bool == true)
let captured = await tracker.get()
#expect(captured?.command == "device.info")
#expect(captured?.id == "invoke-2")
}
}
private enum TestError: Error {
case timeout
}
private func waitForSentMethod(
_ task: TestWebSocketTask,
method: String,
timeoutSeconds: Double
) async throws -> RequestFrame {
try await AsyncTimeout.withTimeout(
seconds: timeoutSeconds,
onTimeout: { TestError.timeout },
operation: {
while true {
let frames = task.sentRequests()
if let match = frames.first(where: { $0.method == method }) {
return match
}
try? await Task.sleep(nanoseconds: 50_000_000)
}
})
}
private actor InvokeTracker {
private var request: BridgeInvokeRequest?
func set(_ req: BridgeInvokeRequest) {
self.request = req
}
func get() -> BridgeInvokeRequest? {
self.request
}
}
private final class TestWebSocketSession: WebSocketSessioning {
private let task: TestWebSocketTask
init(task: TestWebSocketTask) {
self.task = task
}
func makeWebSocketTask(url: URL) -> WebSocketTaskBox {
WebSocketTaskBox(task: self.task)
}
}
private final class TestWebSocketTask: WebSocketTasking, @unchecked Sendable {
private let lock = NSLock()
private var _state: URLSessionTask.State = .suspended
private var receiveQueue: [URLSessionWebSocketTask.Message] = []
private var receiveContinuations: [CheckedContinuation<URLSessionWebSocketTask.Message, Error>] = []
private var receiveHandlers: [@Sendable (Result<URLSessionWebSocketTask.Message, Error>) -> Void] = []
private var sent: [URLSessionWebSocketTask.Message] = []
var state: URLSessionTask.State {
self.lock.withLock { self._state }
}
func resume() {
self.lock.withLock { self._state = .running }
}
func cancel(with closeCode: URLSessionWebSocketTask.CloseCode, reason: Data?) {
self.lock.withLock { self._state = .canceling }
}
func send(_ message: URLSessionWebSocketTask.Message) async throws {
self.lock.withLock { self.sent.append(message) }
guard let frame = Self.decodeRequestFrame(message) else { return }
guard frame.method == "connect" else { return }
let id = frame.id
let response = Self.connectResponse(for: id)
self.enqueue(.data(response))
}
func receive() async throws -> URLSessionWebSocketTask.Message {
try await withCheckedThrowingContinuation { cont in
var next: URLSessionWebSocketTask.Message?
self.lock.withLock {
if !self.receiveQueue.isEmpty {
next = self.receiveQueue.removeFirst()
} else {
self.receiveContinuations.append(cont)
}
}
if let next { cont.resume(returning: next) }
}
}
func receive(completionHandler: @escaping @Sendable (Result<URLSessionWebSocketTask.Message, Error>) -> Void) {
var next: URLSessionWebSocketTask.Message?
self.lock.withLock {
if !self.receiveQueue.isEmpty {
next = self.receiveQueue.removeFirst()
} else {
self.receiveHandlers.append(completionHandler)
}
}
if let next {
completionHandler(.success(next))
}
}
func enqueue(_ message: URLSessionWebSocketTask.Message) {
var handler: (@Sendable (Result<URLSessionWebSocketTask.Message, Error>) -> Void)?
var continuation: CheckedContinuation<URLSessionWebSocketTask.Message, Error>?
self.lock.withLock {
if !self.receiveHandlers.isEmpty {
handler = self.receiveHandlers.removeFirst()
} else if !self.receiveContinuations.isEmpty {
continuation = self.receiveContinuations.removeFirst()
} else {
self.receiveQueue.append(message)
}
}
if let handler {
handler(.success(message))
} else if let continuation {
continuation.resume(returning: message)
}
}
func sentRequests() -> [RequestFrame] {
let messages = self.lock.withLock { self.sent }
return messages.compactMap(Self.decodeRequestFrame)
}
private static func decodeRequestFrame(_ message: URLSessionWebSocketTask.Message) -> RequestFrame? {
let data: Data?
switch message {
case let .data(raw): data = raw
case let .string(text): data = text.data(using: .utf8)
@unknown default: data = nil
}
guard let data else { return nil }
return try? JSONDecoder().decode(RequestFrame.self, from: data)
}
private static func connectResponse(for id: String) -> Data {
let payload: [String: Any] = [
"type": "hello-ok",
"protocol": 3,
"server": [
"version": "dev",
"connId": "test-conn",
],
"features": [
"methods": [],
"events": [],
],
"snapshot": [
"presence": [],
"health": ["ok": true],
"stateVersion": ["presence": 0, "health": 0],
"uptimeMs": 0,
],
"policy": [
"maxPayload": 1,
"maxBufferedBytes": 1,
"tickIntervalMs": 1000,
],
]
let frame: [String: Any] = [
"type": "res",
"id": id,
"ok": true,
"payload": payload,
]
return (try? JSONSerialization.data(withJSONObject: frame)) ?? Data()
}
}
private extension GatewayNodeInvokeTests {
static func makeEventMessage(event: String, payload: Any) -> URLSessionWebSocketTask.Message {
let frame: [String: Any] = [
"type": "event",
"event": event,
"payload": payload,
]
let data = try? JSONSerialization.data(withJSONObject: frame)
return .data(data ?? Data())
}
}
private extension NSLock {
func withLock<T>(_ body: () -> T) -> T {
self.lock()
defer { self.unlock() }
return body()
}
}

1
docs/tools/slash-commands.md
View File

@@ -75,7 +75,6 @@ Text + native (when enabled):
- `/usage off|tokens|full|cost` (per-response usage footer or local cost summary)
- `/tts off|always|inbound|tagged|status|provider|limit|summary|audio` (control TTS; see [/tts](/tts))
- Discord: native command is `/voice` (Discord reserves `/tts`); text `/tts` still works.
- `/ptt start|stop|once|cancel [node=<id>]` (push-to-talk controls for a paired node)
- `/stop`
- `/restart`
- `/dock-telegram` (alias: `/dock_telegram`) (switch replies to Telegram)

49
src/agents/openclaw-tools.camera.test.ts
View File

@@ -133,52 +133,3 @@ describe("nodes run", () => {
});
});
});
describe("nodes invoke", () => {
beforeEach(() => {
callGateway.mockReset();
});
it("invokes arbitrary commands with params JSON", async () => {
callGateway.mockImplementation(async ({ method, params }) => {
if (method === "node.list") {
return { nodes: [{ nodeId: "ios-1" }] };
}
if (method === "node.invoke") {
expect(params).toMatchObject({
nodeId: "ios-1",
command: "device.info",
params: { includeBattery: true },
timeoutMs: 12_000,
});
return {
ok: true,
nodeId: "ios-1",
command: "device.info",
payload: { deviceName: "iPhone" },
};
}
throw new Error(`unexpected method: ${String(method)}`);
});
const tool = createOpenClawTools().find((candidate) => candidate.name === "nodes");
if (!tool) {
throw new Error("missing nodes tool");
}
const result = await tool.execute("call1", {
action: "invoke",
node: "ios-1",
invokeCommand: "device.info",
invokeParamsJson: JSON.stringify({ includeBattery: true }),
invokeTimeoutMs: 12_000,
});
expect(result.details).toMatchObject({
ok: true,
nodeId: "ios-1",
command: "device.info",
payload: { deviceName: "iPhone" },
});
});
});

4
src/agents/system-prompt.ts
View File

@@ -229,7 +229,7 @@ export function buildAgentSystemPrompt(params: {
// Channel docking: add login tools here when a channel needs interactive linking.
browser: "Control web browser",
canvas: "Present/eval/snapshot the Canvas",
nodes: "List/describe/notify/camera/screen/invoke on paired nodes",
nodes: "List/describe/notify/camera/screen on paired nodes",
cron: "Manage cron jobs and wake events (use for reminders; when scheduling a reminder, write the systemEvent text as something that will read like a reminder when it fires, and mention that it is a reminder depending on the time gap between setting and firing; include recent context in reminder text if appropriate)",
message: "Send messages and channel actions",
gateway: "Restart, apply config, or run updates on the running OpenClaw process",
@@ -382,7 +382,7 @@ export function buildAgentSystemPrompt(params: {
`- ${processToolName}: manage background exec sessions`,
"- browser: control openclaw's dedicated browser",
"- canvas: present/eval/snapshot the Canvas",
"- nodes: list/describe/notify/camera/screen/invoke on paired nodes",
"- nodes: list/describe/notify/camera/screen on paired nodes",
"- cron: manage cron jobs and wake events (use for reminders; when scheduling a reminder, write the systemEvent text as something that will read like a reminder when it fires, and mention that it is a reminder depending on the time gap between setting and firing; include recent context in reminder text if appropriate)",
"- sessions_list: list sessions",
"- sessions_history: fetch session history",

4
src/agents/tool-display.json
View File

@@ -140,10 +140,6 @@
"screen_record": {
"label": "screen record",
"detailKeys": ["node", "nodeId", "duration", "durationMs", "fps", "screenIndex"]
},
"invoke": {
"label": "invoke",
"detailKeys": ["node", "nodeId", "invokeCommand"]
}
}
},

31
src/agents/tools/nodes-tool.ts
View File

@@ -37,7 +37,6 @@ const NODES_TOOL_ACTIONS = [
"screen_record",
"location_get",
"run",
"invoke",
] as const;
const NOTIFY_PRIORITIES = ["passive", "active", "timeSensitive"] as const;
@@ -85,9 +84,6 @@ const NodesToolSchema = Type.Object({
commandTimeoutMs: Type.Optional(Type.Number()),
invokeTimeoutMs: Type.Optional(Type.Number()),
needsScreenRecording: Type.Optional(Type.Boolean()),
// invoke
invokeCommand: Type.Optional(Type.String()),
invokeParamsJson: Type.Optional(Type.String()),
});
export function createNodesTool(options?: {
@@ -103,7 +99,7 @@ export function createNodesTool(options?: {
label: "Nodes",
name: "nodes",
description:
"Discover and control paired nodes (status/describe/pairing/notify/camera/screen/location/run/invoke).",
"Discover and control paired nodes (status/describe/pairing/notify/camera/screen/location/run).",
parameters: NodesToolSchema,
execute: async (_toolCallId, args) => {
const params = args as Record<string, unknown>;
@@ -442,31 +438,6 @@ export function createNodesTool(options?: {
});
return jsonResult(raw?.payload ?? {});
}
case "invoke": {
const node = readStringParam(params, "node", { required: true });
const nodeId = await resolveNodeId(gatewayOpts, node);
const invokeCommand = readStringParam(params, "invokeCommand", { required: true });
const invokeParamsJson =
typeof params.invokeParamsJson === "string" ? params.invokeParamsJson.trim() : "";
let invokeParams: unknown = {};
if (invokeParamsJson) {
try {
invokeParams = JSON.parse(invokeParamsJson);
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
throw new Error(`invokeParamsJson must be valid JSON: ${message}`, { cause: err });
}
}
const invokeTimeoutMs = parseTimeoutMs(params.invokeTimeoutMs);
const raw = await callGatewayTool("node.invoke", gatewayOpts, {
nodeId,
command: invokeCommand,
params: invokeParams,
timeoutMs: invokeTimeoutMs,
idempotencyKey: crypto.randomUUID(),
});
return jsonResult(raw ?? {});
}
default:
throw new Error(`Unknown action: ${action}`);
}

22
src/auto-reply/commands-registry.data.ts
View File

@@ -273,28 +273,6 @@ function buildChatCommands(): ChatCommandDefinition[] {
],
argsMenu: "auto",
}),
defineChatCommand({
key: "ptt",
nativeName: "ptt",
description: "Push-to-talk controls for a paired node.",
textAlias: "/ptt",
acceptsArgs: true,
argsParsing: "none",
category: "tools",
args: [
{
name: "action",
description: "start, stop, once, or cancel",
type: "string",
choices: ["start", "stop", "once", "cancel"],
},
{
name: "node",
description: "node=<id> (optional)",
type: "string",
},
],
}),
defineChatCommand({
key: "config",
nativeName: "config",

2
src/auto-reply/reply/commands-core.ts
View File

@@ -21,7 +21,6 @@ import {
} from "./commands-info.js";
import { handleModelsCommand } from "./commands-models.js";
import { handlePluginCommand } from "./commands-plugin.js";
import { handlePTTCommand } from "./commands-ptt.js";
import {
handleAbortTrigger,
handleActivationCommand,
@@ -47,7 +46,6 @@ export async function handleCommands(params: HandleCommandsParams): Promise<Comm
handleUsageCommand,
handleRestartCommand,
handleTtsCommands,
handlePTTCommand,
handleHelpCommand,
handleCommandsListCommand,
handleStatusCommand,

94
src/auto-reply/reply/commands-ptt.test.ts
View File

@@ -1,94 +0,0 @@
import { describe, expect, it, vi } from "vitest";
import type { OpenClawConfig } from "../../config/config.js";
import type { MsgContext } from "../templating.js";
import { buildCommandContext, handleCommands } from "./commands.js";
import { parseInlineDirectives } from "./directive-handling.js";
const callGateway = vi.fn(async (_opts: { method?: string }) => ({ ok: true }));
vi.mock("../../gateway/call.js", () => ({
callGateway: (opts: unknown) => callGateway(opts as { method?: string }),
randomIdempotencyKey: () => "idem-test",
}));
function buildParams(commandBody: string, cfg: OpenClawConfig, ctxOverrides?: Partial<MsgContext>) {
const ctx = {
Body: commandBody,
CommandBody: commandBody,
CommandSource: "text",
CommandAuthorized: true,
Provider: "telegram",
Surface: "telegram",
...ctxOverrides,
} as MsgContext;
const command = buildCommandContext({
ctx,
cfg,
isGroup: false,
triggerBodyNormalized: commandBody.trim().toLowerCase(),
commandAuthorized: true,
});
return {
ctx,
cfg,
command,
directives: parseInlineDirectives(commandBody),
elevated: { enabled: true, allowed: true, failures: [] },
sessionKey: "agent:main:main",
workspaceDir: "/tmp",
defaultGroupActivation: () => "mention",
resolvedVerboseLevel: "off" as const,
resolvedReasoningLevel: "off" as const,
resolveDefaultThinkingLevel: async () => undefined,
provider: "telegram",
model: "test-model",
contextTokens: 0,
isGroup: false,
};
}
describe("handleCommands /ptt", () => {
it("invokes talk.ptt.once on the default iOS node", async () => {
callGateway.mockImplementation(async (opts: { method?: string; params?: unknown }) => {
if (opts.method === "node.list") {
return {
nodes: [
{
nodeId: "ios-1",
displayName: "iPhone",
platform: "ios",
connected: true,
},
],
};
}
if (opts.method === "node.invoke") {
return {
ok: true,
nodeId: "ios-1",
command: "talk.ptt.once",
payload: { status: "offline" },
};
}
return { ok: true };
});
const cfg = {
commands: { text: true },
channels: { telegram: { allowFrom: ["*"] } },
} as OpenClawConfig;
const params = buildParams("/ptt once", cfg);
const result = await handleCommands(params);
expect(result.shouldContinue).toBe(false);
expect(result.reply?.text).toContain("PTT once");
expect(result.reply?.text).toContain("status: offline");
const invokeCall = callGateway.mock.calls.find((call) => call[0]?.method === "node.invoke");
expect(invokeCall).toBeTruthy();
expect(invokeCall?.[0]?.params?.command).toBe("talk.ptt.once");
expect(invokeCall?.[0]?.params?.idempotencyKey).toBe("idem-test");
});
});

208
src/auto-reply/reply/commands-ptt.ts
View File

@@ -1,208 +0,0 @@
import type { OpenClawConfig } from "../../config/config.js";
import type { CommandHandler } from "./commands-types.js";
import { callGateway, randomIdempotencyKey } from "../../gateway/call.js";
import { logVerbose } from "../../globals.js";
type NodeSummary = {
nodeId: string;
displayName?: string;
platform?: string;
deviceFamily?: string;
remoteIp?: string;
connected?: boolean;
};
const PTT_COMMANDS: Record<string, string> = {
start: "talk.ptt.start",
stop: "talk.ptt.stop",
once: "talk.ptt.once",
cancel: "talk.ptt.cancel",
};
function normalizeNodeKey(value: string) {
return value
.toLowerCase()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+/, "")
.replace(/-+$/, "");
}
function isIOSNode(node: NodeSummary): boolean {
const platform = node.platform?.toLowerCase() ?? "";
const family = node.deviceFamily?.toLowerCase() ?? "";
return (
platform.startsWith("ios") ||
family.includes("iphone") ||
family.includes("ipad") ||
family.includes("ios")
);
}
async function loadNodes(cfg: OpenClawConfig): Promise<NodeSummary[]> {
try {
const res = await callGateway<{ nodes?: NodeSummary[] }>({
method: "node.list",
params: {},
config: cfg,
});
return Array.isArray(res.nodes) ? res.nodes : [];
} catch {
const res = await callGateway<{ pending?: unknown[]; paired?: NodeSummary[] }>({
method: "node.pair.list",
params: {},
config: cfg,
});
return Array.isArray(res.paired) ? res.paired : [];
}
}
function describeNodes(nodes: NodeSummary[]) {
return nodes
.map((node) => node.displayName || node.remoteIp || node.nodeId)
.filter(Boolean)
.join(", ");
}
function resolveNodeId(nodes: NodeSummary[], query?: string): string {
const trimmed = String(query ?? "").trim();
if (trimmed) {
const qNorm = normalizeNodeKey(trimmed);
const matches = nodes.filter((node) => {
if (node.nodeId === trimmed) {
return true;
}
if (typeof node.remoteIp === "string" && node.remoteIp === trimmed) {
return true;
}
const name = typeof node.displayName === "string" ? node.displayName : "";
if (name && normalizeNodeKey(name) === qNorm) {
return true;
}
if (trimmed.length >= 6 && node.nodeId.startsWith(trimmed)) {
return true;
}
return false;
});
if (matches.length === 1) {
return matches[0].nodeId;
}
const known = describeNodes(nodes);
if (matches.length === 0) {
throw new Error(`unknown node: ${trimmed}${known ? ` (known: ${known})` : ""}`);
}
throw new Error(
`ambiguous node: ${trimmed} (matches: ${matches
.map((node) => node.displayName || node.remoteIp || node.nodeId)
.join(", ")})`,
);
}
const iosNodes = nodes.filter(isIOSNode);
const iosConnected = iosNodes.filter((node) => node.connected);
const iosCandidates = iosConnected.length > 0 ? iosConnected : iosNodes;
if (iosCandidates.length === 1) {
return iosCandidates[0].nodeId;
}
if (iosCandidates.length > 1) {
throw new Error(
`multiple iOS nodes found (${describeNodes(iosCandidates)}); specify node=<id>`,
);
}
const connected = nodes.filter((node) => node.connected);
const fallback = connected.length > 0 ? connected : nodes;
if (fallback.length === 1) {
return fallback[0].nodeId;
}
const known = describeNodes(nodes);
throw new Error(`node required${known ? ` (known: ${known})` : ""}`);
}
function parsePTTArgs(commandBody: string) {
const tokens = commandBody.trim().split(/\s+/).slice(1);
let action: string | undefined;
let node: string | undefined;
for (const token of tokens) {
if (!token) {
continue;
}
if (token.toLowerCase().startsWith("node=")) {
node = token.slice("node=".length);
continue;
}
if (!action) {
action = token;
}
}
return { action, node };
}
function buildPTTHelpText() {
return [
"Usage: /ptt <start|stop|once|cancel> [node=<id>]",
"Example: /ptt once node=iphone",
].join("\n");
}
export const handlePTTCommand: CommandHandler = async (params, allowTextCommands) => {
if (!allowTextCommands) {
return null;
}
const { command, cfg } = params;
const normalized = command.commandBodyNormalized.trim();
if (!normalized.startsWith("/ptt")) {
return null;
}
if (!command.isAuthorizedSender) {
logVerbose(`Ignoring /ptt from unauthorized sender: ${command.senderId || "<unknown>"}`);
return { shouldContinue: false, reply: { text: "PTT requires an authorized sender." } };
}
const parsed = parsePTTArgs(normalized);
const actionKey = parsed.action?.trim().toLowerCase() ?? "";
const commandId = PTT_COMMANDS[actionKey];
if (!commandId) {
return { shouldContinue: false, reply: { text: buildPTTHelpText() } };
}
try {
const nodes = await loadNodes(cfg);
const nodeId = resolveNodeId(nodes, parsed.node);
const invokeParams: Record<string, unknown> = {
nodeId,
command: commandId,
params: {},
idempotencyKey: randomIdempotencyKey(),
timeoutMs: 15_000,
};
const res = await callGateway<{
ok?: boolean;
payload?: Record<string, unknown>;
command?: string;
nodeId?: string;
}>({
method: "node.invoke",
params: invokeParams,
config: cfg,
});
const payload = res.payload && typeof res.payload === "object" ? res.payload : {};
const lines = [`PTT ${actionKey} → ${nodeId}`];
if (typeof payload.status === "string") {
lines.push(`status: ${payload.status}`);
}
if (typeof payload.captureId === "string") {
lines.push(`captureId: ${payload.captureId}`);
}
if (typeof payload.transcript === "string" && payload.transcript.trim()) {
lines.push(`transcript: ${payload.transcript}`);
}
return { shouldContinue: false, reply: { text: lines.join("\n") } };
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
return { shouldContinue: false, reply: { text: `PTT failed: ${message}` } };
}
};

19
src/cli/nodes-cli.coverage.test.ts
View File

@@ -251,23 +251,4 @@ describe("nodes-cli coverage", () => {
});
expect(invoke?.params?.timeoutMs).toBe(6000);
});
it("invokes talk.ptt.once via nodes talk ptt once", async () => {
runtimeLogs.length = 0;
runtimeErrors.length = 0;
callGateway.mockClear();
randomIdempotencyKey.mockClear();
const { registerNodesCli } = await import("./nodes-cli.js");
const program = new Command();
program.exitOverride();
registerNodesCli(program);
await program.parseAsync(["nodes", "talk", "ptt", "once", "--node", "mac-1"], { from: "user" });
const invoke = callGateway.mock.calls.find((call) => call[0]?.method === "node.invoke")?.[0];
expect(invoke).toBeTruthy();
expect(invoke?.params?.command).toBe("talk.ptt.once");
expect(invoke?.params?.idempotencyKey).toBe("rk_test");
});
});

79
src/cli/nodes-cli/register.talk.ts
View File

@@ -1,79 +0,0 @@
import type { Command } from "commander";
import type { NodesRpcOpts } from "./types.js";
import { randomIdempotencyKey } from "../../gateway/call.js";
import { defaultRuntime } from "../../runtime.js";
import { runNodesCommand } from "./cli-utils.js";
import { callGatewayCli, nodesCallOpts, resolveNodeId } from "./rpc.js";
type PTTAction = {
name: string;
command: string;
description: string;
};
const PTT_ACTIONS: PTTAction[] = [
{ name: "start", command: "talk.ptt.start", description: "Start push-to-talk capture" },
{ name: "stop", command: "talk.ptt.stop", description: "Stop push-to-talk capture" },
{ name: "once", command: "talk.ptt.once", description: "Run push-to-talk once" },
{ name: "cancel", command: "talk.ptt.cancel", description: "Cancel push-to-talk capture" },
];
export function registerNodesTalkCommands(nodes: Command) {
const talk = nodes.command("talk").description("Talk/voice controls on a paired node");
const ptt = talk.command("ptt").description("Push-to-talk controls");
for (const action of PTT_ACTIONS) {
nodesCallOpts(
ptt
.command(action.name)
.description(action.description)
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms (default 15000)", "15000")
.action(async (opts: NodesRpcOpts) => {
await runNodesCommand(`talk ptt ${action.name}`, async () => {
const nodeId = await resolveNodeId(opts, String(opts.node ?? ""));
const invokeTimeoutMs = opts.invokeTimeout
? Number.parseInt(String(opts.invokeTimeout), 10)
: undefined;
const invokeParams: Record<string, unknown> = {
nodeId,
command: action.command,
params: {},
idempotencyKey: randomIdempotencyKey(),
};
if (typeof invokeTimeoutMs === "number" && Number.isFinite(invokeTimeoutMs)) {
invokeParams.timeoutMs = invokeTimeoutMs;
}
const raw = await callGatewayCli("node.invoke", opts, invokeParams);
const res =
typeof raw === "object" && raw !== null ? (raw as { payload?: unknown }) : {};
const payload =
res.payload && typeof res.payload === "object"
? (res.payload as Record<string, unknown>)
: {};
if (opts.json) {
defaultRuntime.log(JSON.stringify(payload, null, 2));
return;
}
const lines = [`PTT ${action.name} → ${nodeId}`];
if (typeof payload.status === "string") {
lines.push(`status: ${payload.status}`);
}
if (typeof payload.captureId === "string") {
lines.push(`captureId: ${payload.captureId}`);
}
if (typeof payload.transcript === "string" && payload.transcript.trim()) {
lines.push(`transcript: ${payload.transcript}`);
}
defaultRuntime.log(lines.join("\n"));
});
}),
{ timeoutMs: 30_000 },
);
}
}

2
src/cli/nodes-cli/register.ts
View File

@@ -9,7 +9,6 @@ import { registerNodesNotifyCommand } from "./register.notify.js";
import { registerNodesPairingCommands } from "./register.pairing.js";
import { registerNodesScreenCommands } from "./register.screen.js";
import { registerNodesStatusCommands } from "./register.status.js";
import { registerNodesTalkCommands } from "./register.talk.js";
export function registerNodesCli(program: Command) {
const nodes = program
@@ -29,5 +28,4 @@ export function registerNodesCli(program: Command) {
registerNodesCameraCommands(nodes);
registerNodesScreenCommands(nodes);
registerNodesLocationCommands(nodes);
registerNodesTalkCommands(nodes);
}

34
src/gateway/node-command-policy.ts
View File

@@ -20,24 +20,6 @@ const LOCATION_COMMANDS = ["location.get"];
const SMS_COMMANDS = ["sms.send"];
const DEVICE_COMMANDS = ["device.status", "device.info"];
const PHOTOS_COMMANDS = ["photos.latest"];
const CONTACTS_COMMANDS = ["contacts.search", "contacts.add"];
const CALENDAR_COMMANDS = ["calendar.events", "calendar.add"];
const REMINDERS_COMMANDS = ["reminders.list", "reminders.add"];
const MOTION_COMMANDS = ["motion.activity", "motion.pedometer"];
const SYSTEM_NOTIFY_COMMANDS = ["system.notify"];
const CHAT_COMMANDS = ["chat.push"];
const TALK_COMMANDS = ["talk.ptt.start", "talk.ptt.stop", "talk.ptt.cancel", "talk.ptt.once"];
const SYSTEM_COMMANDS = [
"system.run",
"system.which",
@@ -48,21 +30,7 @@ const SYSTEM_COMMANDS = [
];
const PLATFORM_DEFAULTS: Record<string, string[]> = {
ios: [
...CANVAS_COMMANDS,
...CAMERA_COMMANDS,
...SCREEN_COMMANDS,
...LOCATION_COMMANDS,
...SYSTEM_NOTIFY_COMMANDS,
...CHAT_COMMANDS,
...DEVICE_COMMANDS,
...PHOTOS_COMMANDS,
...CONTACTS_COMMANDS,
...CALENDAR_COMMANDS,
...REMINDERS_COMMANDS,
...MOTION_COMMANDS,
...TALK_COMMANDS,
],
ios: [...CANVAS_COMMANDS, ...CAMERA_COMMANDS, ...SCREEN_COMMANDS, ...LOCATION_COMMANDS],
android: [
...CANVAS_COMMANDS,
...CAMERA_COMMANDS,