iroh default mobile transport: design + green Swift FFI spike (#5735)

* spike: iroh Swift C-FFI harness (macOS + iOS-sim) for mobile transport

Minimal Rust staticlib (iroh 1.0.0-rc.1) exposing a blocking C API:
bind endpoint, dial by EndpointId via n0 relays/discovery, one bi-stream
send/recv. Swift CLI harness with listen (echo) and dial (round-trip
proof) modes. build.sh builds aarch64-apple-darwin and
aarch64-apple-ios-sim variants; artifacts are gitignored.

Co-Authored-By: Claude Fable 5 <[email protected]>

* spike: green cross-platform iroh proof (iOS-sim dials Mac by EndpointId)

- link Network.framework for the ios-sim target (netdev nw_path_monitor)
- map clean peer close (application code 0) to end-of-stream in recv
- line-buffer harness stdout so orchestration can read the endpoint id
- README records bindings decision (official iroh-ffi is archived, n0
  recommends a custom wrapper), versions, build steps, proof transcript,
  and the ~7.7MB per-slice binary delta

Co-Authored-By: Claude Fable 5 <[email protected]>

* plans: iroh default mobile transport design

Substrate swap for the existing length-prefixed mobile-host protocol:
iroh QUIC dial-by-EndpointId as the default iOS-to-Mac transport,
Tailscale/LAN demoted to an opt-in fallback toggle. Covers the Mac
listener seam (MobileHostByteConnection), the phone CmxByteTransport
lane, registry route publication, E2E story (QUIC raw-public-key TLS
replaces the Noise IK plan on this lane), Keychain key custody, relay
strategy (n0 now, self-host later), iOS background/battery policy,
hive-design reconciliation, and a 5-PR stacked delivery plan.

Co-Authored-By: Claude Fable 5 <[email protected]>

* spike: iroh route priority 5 so it beats Tailscale in preferredRoute

CmxAttachTicket.preferredRoute sorts ascending and lower wins. The Mac
publishes debugLoopback at 0 and Tailscale at 10+; the spike's route
JSON claimed 20, which would have lost to Tailscale, contradicting the
design's iroh-by-default ordering. 5 sits below Tailscale (default) and
above debugLoopback (DEBUG/simulator keeps the loopback mock host).
Re-ran the cross-platform proof after the change: iOS-sim dialed the
Mac by EndpointId, 46 bytes echoed, 1.04s connect, rc=0 both sides.

Co-Authored-By: Claude Fable 5 <[email protected]>

* plans: EndpointId pinning ships in the first iroh lane, not later

Sharpen the security section: the real threat on a substituted route is
Stack-token exfiltration (the phone sends its bearer token on every
RPC), and iroh is the lane that can close it because the channel is
cryptographically bound to the dialed EndpointId. So pinning moves from
'defense in depth later' into PR 3/4: pin at first trust in
MobilePairedMacStore (QR = proximity, registry auto-pair = TOFU),
refuse to send Stack tokens to a non-matching EndpointId, and surface
EndpointId changes for explicit re-trust.

Co-Authored-By: Claude Fable 5 <[email protected]>

* spike: drain finished stream before connection close

Review finding (autoreview P2): finish() only queues the FIN plus
buffered stream data, while Connection::close is immediate and abandons
buffered data, so a final frame that send() already accepted could be
dropped by close(). Wait on SendStream::stopped() (peer acked all
finished data) with a 5s bound before closing, so a vanished peer
cannot wedge close. Re-ran the cross-platform proof: iOS-sim dial rc=0,
mac listener rc=0, 0.50s connect, no drain stall (2s wall total).

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
This commit is contained in:
Lawrence Chen
2026-06-09 21:04:40 -07:00
committed by GitHub
co-authored by Claude Fable 5
parent 4884c562a2
commit 6dd725a0f4
9 changed files with 5067 additions and 0 deletions
@@ -0,0 +1,2 @@
out/
rust/target/
@@ -0,0 +1,89 @@
# iroh Swift FFI spike
Gating-risk spike for making [iroh](https://www.iroh.computer/) the default
cmux iOS-to-Mac transport: prove a Swift process can bind an iroh endpoint,
dial another endpoint by EndpointId through the default n0 relays, and
exchange bytes over one bidirectional QUIC stream (the byte-stream substrate
the existing length-prefixed `CmxByteTransport` mobile protocol rides on).
Result: **green**. See `plans/feat-ios-iroh/DESIGN.md` for the production design.
## Bindings decision: Rust staticlib + minimal C FFI (not iroh-ffi)
The official uniffi bindings repo
[n0-computer/iroh-ffi](https://github.com/n0-computer/iroh-ffi) is **archived**
("provided as a reference example only"); its last release is v0.35.0
(2025-06-23), which tracks pre-1.0 iroh. n0's own guidance for other
languages is to write a small Rust wrapper that exposes only what the app
needs. That is what this spike does: `rust/src/lib.rs` is a ~400-line
staticlib exposing a blocking C API (bind, id, route JSON, online, accept,
connect, recv, send, close) over iroh 1.0.0-rc.1, consumed from Swift via a
plain bridging header (`include/cmux_iroh_ffi.h`).
## Versions
- iroh `1.0.0-rc.1` (n0-error `1.0.0-rc.0`, tokio `1.48`), Rust edition 2024
- rustc/cargo `1.94.0`, targets `aarch64-apple-darwin`, `aarch64-apple-ios-sim`
- Swift harness: `xcrun swiftc -O`, targets `arm64-apple-macos14.0` and
`arm64-apple-ios17.0-simulator`
- Proof ran on macOS 26.5 host + iPhone 17 simulator (iOS 26.4)
## Build steps
```bash
# once: rustup target add aarch64-apple-darwin aarch64-apple-ios-sim
./build.sh macos # staticlib + out/swift-harness-macos
./build.sh ios-sim # staticlib + out/swift-harness-ios-sim
```
Framework links the staticlib needs (found empirically; netdev inside iroh
uses `nw_path_monitor` on iOS and CoreWLAN on macOS):
- macOS: `SystemConfiguration CoreWLAN Security`
- iOS simulator: `SystemConfiguration Security Network`
## Proof
Terminal A (macOS):
```bash
./out/swift-harness-macos listen
# prints endpoint-id + CmxAttachRoute-shaped route JSON, then echoes
```
Terminal B (iOS simulator, booted):
```bash
xcrun simctl spawn booted "$PWD/out/swift-harness-ios-sim" dial <endpoint-id>
```
Observed 2026-06-09 (no relay/addr hints passed to the dialer, pure
dial-by-EndpointId via n0 discovery):
```
mac listener endpoint-id: 8b5505d8915e8389a3bcf1bd2ff1c7ec5f2184da8cf48fd72c2334757ec63c0e
PROOF: dialed by EndpointId, 45 byte(s) echoed in 1.03s connect
echoed 45 byte(s); peer closed stream
ios-sim dial rc=0, mac listener rc=0
```
macOS-to-macOS run of the same proof connected in 0.52s. Home relay assigned
was `usw1-1.relay.n0.iroh-canary.iroh.link` (1.0-rc endpoints currently land
on n0's canary relay fleet; pin/verify before shipping).
## Binary size
- staticlib `libcmux_iroh_ffi.a`: 15 MB per target (`opt-level = "s"`, fat
LTO, `codegen-units = 1`, debuginfo stripped)
- linked, dead-stripped Swift harness: 7.8 MB (macOS), 7.7 MB (iOS sim);
a trivial Swift CLI baseline with the same flags is 51 KB, so the
**real post-link app delta is about +7.7 MB per architecture slice**
- nothing binary is committed; `out/` and `rust/target/` are gitignored
## Known spike-level gaps (deliberate, covered in the design doc)
- fresh `SecretKey::generate()` per bind; production needs Keychain custody
- one global blocking tokio runtime; production wraps calls off-main from
Swift (the existing `CmxByteTransport` actor pattern already does this)
- no E2E payload encryption beyond iroh QUIC TLS; the Noise-IK / ticket
authorization story from the existing mobile protocol still applies
+53
View File
@@ -0,0 +1,53 @@
#!/bin/bash
# Builds the iroh FFI staticlib and the Swift harness.
#
# ./build.sh macos # aarch64-apple-darwin staticlib + macOS Swift harness
# ./build.sh ios-sim # aarch64-apple-ios-sim staticlib + iOS-simulator Swift harness
#
# Artifacts land in ./out/ (gitignored). No binaries are committed.
set -euo pipefail
cd "$(dirname "$0")"
mkdir -p out
target_kind="${1:-macos}"
case "$target_kind" in
macos)
rust_target="aarch64-apple-darwin"
sdk="macosx"
swift_target="arm64-apple-macos14.0"
frameworks=(-framework SystemConfiguration -framework CoreWLAN -framework Security)
;;
ios-sim)
rust_target="aarch64-apple-ios-sim"
sdk="iphonesimulator"
swift_target="arm64-apple-ios17.0-simulator"
# netdev (iroh's interface enumeration) uses Network.framework nw_path_monitor on iOS.
frameworks=(-framework SystemConfiguration -framework Security -framework Network)
;;
*)
echo "usage: $0 [macos|ios-sim]" >&2
exit 2
;;
esac
echo "==> cargo build --release --target $rust_target"
(cd rust && cargo build --release --target "$rust_target")
lib="rust/target/$rust_target/release/libcmux_iroh_ffi.a"
ls -lh "$lib"
out="out/swift-harness-$target_kind"
echo "==> swiftc -> $out"
xcrun -sdk "$sdk" swiftc \
-target "$swift_target" \
-O \
-import-objc-header include/cmux_iroh_ffi.h \
swift/main.swift \
"$lib" \
"${frameworks[@]}" \
-o "$out"
ls -lh "$out"
echo "==> done: $out"
@@ -0,0 +1,74 @@
// Minimal C FFI over iroh for the cmux mobile transport spike.
// See rust/src/lib.rs for semantics. All blocking; call off the main thread.
#ifndef CMUX_IROH_FFI_H
#define CMUX_IROH_FFI_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct CmuxIrohEndpoint CmuxIrohEndpoint;
typedef struct CmuxIrohConnection CmuxIrohConnection;
CmuxIrohEndpoint *cmux_iroh_endpoint_bind(
bool enable_relay,
bool accept_connections,
char *err_buf,
size_t err_cap);
char *cmux_iroh_endpoint_id(const CmuxIrohEndpoint *endpoint);
char *cmux_iroh_endpoint_route_json(const CmuxIrohEndpoint *endpoint);
int cmux_iroh_endpoint_online(
CmuxIrohEndpoint *endpoint,
uint64_t timeout_ms,
char *err_buf,
size_t err_cap);
CmuxIrohConnection *cmux_iroh_endpoint_accept(
CmuxIrohEndpoint *endpoint,
uint64_t timeout_ms,
char *err_buf,
size_t err_cap);
CmuxIrohConnection *cmux_iroh_endpoint_connect(
CmuxIrohEndpoint *endpoint,
const char *endpoint_id,
const char *relay_url,
const char *const *direct_addrs,
size_t direct_addr_count,
uint64_t timeout_ms,
char *err_buf,
size_t err_cap);
intptr_t cmux_iroh_connection_recv(
CmuxIrohConnection *connection,
uint8_t *buf,
size_t cap,
char *err_buf,
size_t err_cap);
int cmux_iroh_connection_send(
CmuxIrohConnection *connection,
const uint8_t *bytes,
size_t len,
char *err_buf,
size_t err_cap);
void cmux_iroh_connection_close(CmuxIrohConnection *connection);
void cmux_iroh_endpoint_close(CmuxIrohEndpoint *endpoint);
void cmux_iroh_string_free(char *string);
#ifdef __cplusplus
}
#endif
#endif // CMUX_IROH_FFI_H
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,21 @@
[package]
name = "cmux-iroh-ffi"
version = "0.1.0"
edition = "2024"
rust-version = "1.91"
publish = false
[lib]
crate-type = ["staticlib"]
[dependencies]
iroh = "1.0.0-rc.1"
n0-error = "1.0.0-rc.0"
serde_json = "1.0"
tokio = { version = "1.48", features = ["rt-multi-thread", "time"] }
[profile.release]
opt-level = "s"
lto = true
codegen-units = 1
strip = "debuginfo"
@@ -0,0 +1,438 @@
//! Minimal C FFI over iroh for the cmux mobile transport spike.
//!
//! This is spike code: it proves a Swift process (macOS app or iOS app) can
//! bind an iroh endpoint, dial another endpoint by EndpointId through the
//! default n0 relays, and exchange bytes over one bidirectional QUIC stream.
//! The single bi-stream is deliberate: it is the byte-stream substrate the
//! existing `CmxByteTransport` protocol (length-prefixed JSON frames) rides on.
//!
//! Shape notes for the production version (see plans/feat-ios-iroh/DESIGN.md):
//! - one blocking C call per `CmxByteTransport` operation (connect/recv/send/close),
//! called from Swift off the main thread; a shared tokio runtime lives in here.
//! - the dialer opens the stream and speaks first, which matches the existing
//! mobile protocol where the phone sends the first RPC frame (QUIC `accept_bi`
//! only resolves once the opener has sent bytes).
use std::{
ffi::{CStr, CString, c_char},
net::SocketAddr,
os::raw::c_int,
ptr,
str::FromStr,
sync::OnceLock,
time::Duration,
};
use iroh::{
Endpoint, EndpointAddr, EndpointId, RelayMode, RelayUrl, SecretKey, TransportAddr,
endpoint::{Connection, ConnectionError, ReadError, RecvStream, SendStream, presets},
};
use tokio::{runtime::Runtime, sync::Mutex};
const ALPN: &[u8] = b"dev.cmux.mobile.terminal/0";
fn runtime() -> &'static Runtime {
static RUNTIME: OnceLock<Runtime> = OnceLock::new();
RUNTIME.get_or_init(|| {
tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build()
.expect("tokio runtime should build")
})
}
/// Writes `message` into the caller-provided error buffer, truncating to fit.
fn set_error(err_buf: *mut c_char, err_cap: usize, message: &str) {
if err_buf.is_null() || err_cap == 0 {
return;
}
let bytes = message.as_bytes();
let len = bytes.len().min(err_cap - 1);
unsafe {
ptr::copy_nonoverlapping(bytes.as_ptr(), err_buf.cast::<u8>(), len);
*err_buf.add(len) = 0;
}
}
pub struct CmuxIrohEndpoint {
endpoint: Endpoint,
}
pub struct CmuxIrohConnection {
connection: Connection,
send: Mutex<SendStream>,
recv: Mutex<RecvStream>,
}
/// Binds an iroh endpoint using the default n0 preset (relays + discovery).
///
/// Returns null on failure with the cause in `err_buf`. The spike always
/// generates a fresh secret key; key custody is a production design topic.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_endpoint_bind(
enable_relay: bool,
accept_connections: bool,
err_buf: *mut c_char,
err_cap: usize,
) -> *mut CmuxIrohEndpoint {
let result = runtime().block_on(async move {
let mut builder = Endpoint::builder(presets::N0)
.secret_key(SecretKey::generate())
.relay_mode(if enable_relay {
RelayMode::Default
} else {
RelayMode::Disabled
});
if accept_connections {
builder = builder.alpns(vec![ALPN.to_vec()]);
}
builder.bind().await
});
match result {
Ok(endpoint) => Box::into_raw(Box::new(CmuxIrohEndpoint { endpoint })),
Err(error) => {
set_error(err_buf, err_cap, &format!("bind failed: {error:#}"));
ptr::null_mut()
}
}
}
/// Returns the endpoint's EndpointId (z-base-32) as a heap string.
/// Free with `cmux_iroh_string_free`.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_endpoint_id(endpoint: *const CmuxIrohEndpoint) -> *mut c_char {
let Some(endpoint) = (unsafe { endpoint.as_ref() }) else {
return ptr::null_mut();
};
string_to_c(endpoint.endpoint.id().to_string())
}
/// Returns a `CmxAttachRoute`-shaped JSON object for this endpoint
/// (id, direct addrs, relay URL). Free with `cmux_iroh_string_free`.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_endpoint_route_json(endpoint: *const CmuxIrohEndpoint) -> *mut c_char {
let Some(endpoint) = (unsafe { endpoint.as_ref() }) else {
return ptr::null_mut();
};
let addr = endpoint.endpoint.addr();
let direct_addrs = addr
.ip_addrs()
.map(|addr| addr.to_string())
.collect::<Vec<_>>();
let relay_url = addr.relay_urls().next().map(|url| url.to_string());
// `CmxAttachTicket.preferredRoute` sorts ascending and lower wins, so iroh
// must sit below the Mac's primary Tailscale route (priority 10) to be the
// default; 5 also stays above debugLoopback (0) so DEBUG/simulator runs
// keep preferring the loopback mock host.
let route = serde_json::json!({
"id": "iroh",
"kind": "iroh",
"endpoint": {
"type": "peer",
"id": endpoint.endpoint.id().to_string(),
"direct_addrs": direct_addrs,
"relay_url": relay_url,
},
"priority": 5,
});
string_to_c(route.to_string())
}
/// Waits until the endpoint has a home relay connection (so dial-by-id from
/// elsewhere can reach it). 0 on success, -1 on timeout.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_endpoint_online(
endpoint: *mut CmuxIrohEndpoint,
timeout_ms: u64,
err_buf: *mut c_char,
err_cap: usize,
) -> c_int {
let Some(endpoint) = (unsafe { endpoint.as_ref() }) else {
set_error(err_buf, err_cap, "null endpoint");
return -1;
};
let online = runtime().block_on(async {
tokio::time::timeout(
Duration::from_millis(timeout_ms.max(1)),
endpoint.endpoint.online(),
)
.await
});
match online {
Ok(()) => 0,
Err(_) => {
set_error(err_buf, err_cap, "timed out waiting for relay connection");
-1
}
}
}
/// Accepts one incoming connection and its first bidirectional stream.
/// Blocks up to `timeout_ms`. Returns null on failure/timeout.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_endpoint_accept(
endpoint: *mut CmuxIrohEndpoint,
timeout_ms: u64,
err_buf: *mut c_char,
err_cap: usize,
) -> *mut CmuxIrohConnection {
let Some(endpoint) = (unsafe { endpoint.as_ref() }) else {
set_error(err_buf, err_cap, "null endpoint");
return ptr::null_mut();
};
let result = runtime().block_on(async {
tokio::time::timeout(Duration::from_millis(timeout_ms.max(1)), async {
let incoming = endpoint
.endpoint
.accept()
.await
.ok_or_else(|| "endpoint closed".to_string())?;
let connection = incoming
.await
.map_err(|error| format!("incoming connection failed: {error:#}"))?;
let (send, recv) = connection
.accept_bi()
.await
.map_err(|error| format!("accept_bi failed: {error:#}"))?;
Ok::<_, String>((connection, send, recv))
})
.await
.map_err(|_| "accept timed out".to_string())?
});
finish_connection(result, err_buf, err_cap)
}
/// Dials `endpoint_id` (optionally with relay URL / direct addr hints) and
/// opens one bidirectional stream. With no hints, n0 discovery resolves the id.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_endpoint_connect(
endpoint: *mut CmuxIrohEndpoint,
endpoint_id: *const c_char,
relay_url: *const c_char,
direct_addrs: *const *const c_char,
direct_addr_count: usize,
timeout_ms: u64,
err_buf: *mut c_char,
err_cap: usize,
) -> *mut CmuxIrohConnection {
let Some(endpoint) = (unsafe { endpoint.as_ref() }) else {
set_error(err_buf, err_cap, "null endpoint");
return ptr::null_mut();
};
let Some(id_str) = c_to_str(endpoint_id) else {
set_error(err_buf, err_cap, "null or invalid endpoint id");
return ptr::null_mut();
};
let id = match EndpointId::from_str(id_str) {
Ok(id) => id,
Err(error) => {
set_error(err_buf, err_cap, &format!("invalid endpoint id: {error:#}"));
return ptr::null_mut();
}
};
let mut addrs: Vec<TransportAddr> = Vec::new();
if !direct_addrs.is_null() {
for index in 0..direct_addr_count {
let raw = unsafe { *direct_addrs.add(index) };
let Some(addr_str) = c_to_str(raw) else {
continue;
};
match SocketAddr::from_str(addr_str) {
Ok(addr) => addrs.push(TransportAddr::Ip(addr)),
Err(error) => {
set_error(
err_buf,
err_cap,
&format!("invalid direct addr {addr_str}: {error:#}"),
);
return ptr::null_mut();
}
}
}
}
if let Some(relay_str) = c_to_str(relay_url) {
match RelayUrl::from_str(relay_str) {
Ok(url) => addrs.push(TransportAddr::Relay(url)),
Err(error) => {
set_error(err_buf, err_cap, &format!("invalid relay url: {error:#}"));
return ptr::null_mut();
}
}
}
let addr = if addrs.is_empty() {
EndpointAddr::from(id)
} else {
EndpointAddr::from_parts(id, addrs)
};
let result = runtime().block_on(async {
tokio::time::timeout(Duration::from_millis(timeout_ms.max(1)), async {
let connection = endpoint
.endpoint
.connect(addr, ALPN)
.await
.map_err(|error| format!("connect failed: {error:#}"))?;
let (send, recv) = connection
.open_bi()
.await
.map_err(|error| format!("open_bi failed: {error:#}"))?;
Ok::<_, String>((connection, send, recv))
})
.await
.map_err(|_| "connect timed out".to_string())?
});
finish_connection(result, err_buf, err_cap)
}
/// Receives up to `cap` bytes. Returns bytes read (>0), 0 on clean end of
/// stream, or -1 on error.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_connection_recv(
connection: *mut CmuxIrohConnection,
buf: *mut u8,
cap: usize,
err_buf: *mut c_char,
err_cap: usize,
) -> isize {
let Some(connection) = (unsafe { connection.as_ref() }) else {
set_error(err_buf, err_cap, "null connection");
return -1;
};
if buf.is_null() || cap == 0 {
set_error(err_buf, err_cap, "null or empty receive buffer");
return -1;
}
let slice = unsafe { std::slice::from_raw_parts_mut(buf, cap) };
let result = runtime().block_on(async {
let mut recv = connection.recv.lock().await;
recv.read(slice).await
});
match result {
Ok(Some(read)) => read as isize,
Ok(None) => 0,
// A clean peer close (application error code 0) is end-of-stream,
// not an error: QUIC CONNECTION_CLOSE can race the stream FIN.
Err(ReadError::ConnectionLost(ConnectionError::ApplicationClosed(close)))
if u64::from(close.error_code) == 0 =>
{
0
}
Err(error) => {
set_error(err_buf, err_cap, &format!("recv failed: {error:#}"));
-1
}
}
}
/// Sends `len` bytes. Returns 0 on success, -1 on error.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_connection_send(
connection: *mut CmuxIrohConnection,
bytes: *const u8,
len: usize,
err_buf: *mut c_char,
err_cap: usize,
) -> c_int {
let Some(connection) = (unsafe { connection.as_ref() }) else {
set_error(err_buf, err_cap, "null connection");
return -1;
};
if len == 0 {
return 0;
}
if bytes.is_null() {
set_error(err_buf, err_cap, "null send buffer");
return -1;
}
let slice = unsafe { std::slice::from_raw_parts(bytes, len) };
let result = runtime().block_on(async {
let mut send = connection.send.lock().await;
send.write_all(slice).await
});
match result {
Ok(()) => 0,
Err(error) => {
set_error(err_buf, err_cap, &format!("send failed: {error:#}"));
-1
}
}
}
/// Closes the connection and frees its handle.
///
/// Graceful close: `finish()` only queues the FIN plus any buffered stream
/// data, while `Connection::close` is immediate and abandons buffered data.
/// Closing right after finishing could therefore drop a final frame that
/// `send()` already reported as accepted. `stopped()` resolves once the peer
/// acknowledges receipt of all finished stream data, so wait for it (bounded,
/// so a vanished peer cannot wedge close) before closing the connection.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_connection_close(connection: *mut CmuxIrohConnection) {
if connection.is_null() {
return;
}
let connection = unsafe { Box::from_raw(connection) };
runtime().block_on(async {
let mut send = connection.send.lock().await;
if send.finish().is_ok() {
let _ = tokio::time::timeout(Duration::from_secs(5), send.stopped()).await;
}
drop(send);
connection.connection.close(0u32.into(), b"close");
});
}
/// Closes the endpoint and frees its handle.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_endpoint_close(endpoint: *mut CmuxIrohEndpoint) {
if endpoint.is_null() {
return;
}
let endpoint = unsafe { Box::from_raw(endpoint) };
runtime().block_on(async {
endpoint.endpoint.close().await;
});
}
/// Frees a string returned by this library.
#[unsafe(no_mangle)]
pub extern "C" fn cmux_iroh_string_free(string: *mut c_char) {
if string.is_null() {
return;
}
drop(unsafe { CString::from_raw(string) });
}
fn finish_connection(
result: Result<(Connection, SendStream, RecvStream), String>,
err_buf: *mut c_char,
err_cap: usize,
) -> *mut CmuxIrohConnection {
match result {
Ok((connection, send, recv)) => Box::into_raw(Box::new(CmuxIrohConnection {
connection,
send: Mutex::new(send),
recv: Mutex::new(recv),
})),
Err(message) => {
set_error(err_buf, err_cap, &message);
ptr::null_mut()
}
}
}
fn string_to_c(string: String) -> *mut c_char {
match CString::new(string) {
Ok(cstring) => cstring.into_raw(),
Err(_) => ptr::null_mut(),
}
}
fn c_to_str<'a>(raw: *const c_char) -> Option<&'a str> {
if raw.is_null() {
return None;
}
unsafe { CStr::from_ptr(raw) }.to_str().ok()
}
@@ -0,0 +1,124 @@
// Swift harness for the cmux iroh FFI spike.
//
// listen: bind an endpoint, print its EndpointId + route JSON, echo one
// connection's bytes back until the peer closes.
// dial: bind an endpoint, dial a peer by EndpointId (n0 discovery + relays),
// send a payload, verify the echo round-trips.
//
// Build/run via ../build.sh. This is spike code, not app code; blocking calls
// on the CLI main thread are intentional.
import Foundation
// Line-buffer stdout even when piped, so orchestration scripts can react to
// the endpoint-id line before the process exits.
setvbuf(stdout, nil, _IOLBF, 0)
let errCap = 512
func lastError(_ buf: [CChar]) -> String {
String(cString: buf, encoding: .utf8) ?? "unknown error"
}
func takeString(_ raw: UnsafeMutablePointer<CChar>?) -> String {
guard let raw else { return "" }
defer { cmux_iroh_string_free(raw) }
return String(cString: raw)
}
func fail(_ message: String) -> Never {
FileHandle.standardError.write(Data("error: \(message)\n".utf8))
exit(1)
}
func bindEndpoint(acceptConnections: Bool) -> OpaquePointer {
var err = [CChar](repeating: 0, count: errCap)
guard let endpoint = cmux_iroh_endpoint_bind(true, acceptConnections, &err, errCap) else {
fail("bind: \(lastError(err))")
}
return endpoint
}
func waitOnline(_ endpoint: OpaquePointer) {
var err = [CChar](repeating: 0, count: errCap)
guard cmux_iroh_endpoint_online(endpoint, 30_000, &err, errCap) == 0 else {
fail("online: \(lastError(err))")
}
}
func runListen() {
print("binding endpoint...")
let endpoint = bindEndpoint(acceptConnections: true)
print("bound; waiting for relay connection...")
waitOnline(endpoint)
print("endpoint-id: \(takeString(cmux_iroh_endpoint_id(endpoint)))")
print("route: \(takeString(cmux_iroh_endpoint_route_json(endpoint)))")
print("listening; dial me with: swift-harness dial <endpoint-id>")
var err = [CChar](repeating: 0, count: errCap)
guard let connection = cmux_iroh_endpoint_accept(endpoint, 180_000, &err, errCap) else {
fail("accept: \(lastError(err))")
}
print("accepted connection; echoing")
var buf = [UInt8](repeating: 0, count: 64 * 1024)
var total = 0
while true {
let read = cmux_iroh_connection_recv(connection, &buf, buf.count, &err, errCap)
if read < 0 { fail("recv: \(lastError(err))") }
if read == 0 { break }
total += read
guard cmux_iroh_connection_send(connection, buf, read, &err, errCap) == 0 else {
fail("send: \(lastError(err))")
}
}
print("echoed \(total) byte(s); peer closed stream")
cmux_iroh_connection_close(connection)
cmux_iroh_endpoint_close(endpoint)
}
func runDial(endpointID: String, payload: String) {
let endpoint = bindEndpoint(acceptConnections: false)
var err = [CChar](repeating: 0, count: errCap)
let started = Date()
guard let connection = endpointID.withCString({ id in
cmux_iroh_endpoint_connect(endpoint, id, nil, nil, 0, 60_000, &err, errCap)
}) else {
fail("connect: \(lastError(err))")
}
let connectSeconds = Date().timeIntervalSince(started)
let message = Array(payload.utf8)
guard cmux_iroh_connection_send(connection, message, message.count, &err, errCap) == 0 else {
fail("send: \(lastError(err))")
}
var echoed = [UInt8]()
var buf = [UInt8](repeating: 0, count: 64 * 1024)
while echoed.count < message.count {
let read = cmux_iroh_connection_recv(connection, &buf, buf.count, &err, errCap)
if read < 0 { fail("recv: \(lastError(err))") }
if read == 0 { break }
echoed.append(contentsOf: buf[0..<read])
}
cmux_iroh_connection_close(connection)
cmux_iroh_endpoint_close(endpoint)
guard echoed == message else {
fail("echo mismatch: sent \(message.count) byte(s), received \(echoed.count)")
}
print(String(format: "PROOF: dialed by EndpointId, %d byte(s) echoed in %.2fs connect", message.count, connectSeconds))
}
let arguments = CommandLine.arguments
switch arguments.count > 1 ? arguments[1] : "" {
case "listen":
runListen()
case "dial" where arguments.count >= 3:
let payload = arguments.count >= 4
? arguments[3]
: "hello over iroh from swift @ \(Date().timeIntervalSince1970)"
runDial(endpointID: arguments[2], payload: payload)
default:
fail("usage: swift-harness listen | swift-harness dial <endpoint-id> [payload]")
}
+127
View File
@@ -0,0 +1,127 @@
# iroh as the default cmux iOS-to-Mac transport
Status: spike green, design committed, implementation planned as stacked PRs (see "Delivery plan"). Decision (Lawrence, 2026-06-09): iroh is the DEFAULT transport; Tailscale becomes opt-in. Onboarding "just works" with sign-in plus dial-by-EndpointId; no VPN install, no network setup.
## What this is and is not
This is a substrate swap, not a protocol rewrite. The existing mobile-host protocol (length-prefixed JSON frames, `MobileSyncFrameCodec`, the `mobileHostHandleRPC` allowlist, render-grid data plane, per-RPC Stack same-account auth) is unchanged byte for byte. Today those frames ride one TCP connection over Tailscale; after this they ride one iroh QUIC bidirectional stream, dialed by EndpointId. Everything above the byte stream stays put.
The codebase already reserved the seams for this:
- `CmxAttachTransportKind.iroh` exists and validates against `.peer` endpoints (`Packages/CMUXMobileCore/Sources/CMUXMobileCore/CmxTransport.swift`).
- `CmxAttachEndpoint.peer(id:relayHint:directAddrs:relayURL:)` already carries exactly what an iroh dial needs.
- `MobileShellRouteAuthPolicy` already classifies `(.iroh, .peer)` as an encrypted route that may carry Stack tokens (`Packages/CmuxMobileShellModel/Sources/CmuxMobileShellModel/MobileShellRouteAuthPolicy.swift`).
- The phone builds transports through `CmxRouteTransportFactory` keyed by route kind; adding a kind is one registration in `ios/cmux/cmuxApp.swift`.
- The device registry (merged, https://github.com/manaflow-ai/cmux/pull/5626) stores `CmxAttachRoute` lists as opaque jsonb, so an `iroh` route needs zero schema change, and the phone's `DeviceRegistryService` already skips unknown route kinds on old builds (forward compatible).
- Settings diagnostics already have a localized "Iroh" route label (`Sources/HostSettingsActions.swift`).
## Spike results (gating risk: retired)
Spike code: `experiments/iroh-swift-ffi-spike/` (full notes in its README).
- Bindings: the official uniffi bindings (https://github.com/n0-computer/iroh-ffi) are archived, last release v0.35.0 tracks pre-1.0 iroh. n0's guidance is a custom wrapper. We use a ~400-line Rust staticlib over iroh `1.0.0-rc.1` exposing a minimal blocking C API (bind, id, route JSON, online, accept, connect, recv, send, close), consumed from Swift via a bridging header.
- Proof: an arm64 iOS-simulator process (iPhone 17, iOS 26.4) dialed a macOS process by EndpointId alone (no address hints, n0 discovery plus relays) and round-tripped bytes; connect took 0.50 to 1.03s. Both sides exited clean.
- Size: staticlib 15 MB; real post-link, dead-stripped app delta is about +7.7 MB per architecture slice (measured harness vs trivial Swift CLI baseline).
- Frameworks needed: macOS `SystemConfiguration CoreWLAN Security`; iOS `SystemConfiguration Security Network`.
## Architecture
### Mac (host) side
`MobileHostService` gains an iroh listener lane next to the existing `NWListener`:
1. On host start, bind one long-lived iroh endpoint (ALPN `dev.cmux.mobile.terminal/0`) with a persisted secret key (Keychain, below). Bind is lazy with the same enable gates as the TCP listener.
2. Accept loop: each accepted connection plus first bi-stream becomes a `MobileHostConnection`. Today that actor talks to `NWConnection` directly in exactly three places (`receiveNext`, send, state handler). Introduce a small `MobileHostByteConnection` protocol (receive/send/close, the server-side mirror of `CmxByteTransport`) with an `NWConnection` adapter and an iroh adapter. All connection-lifecycle logic (frame codec, first-frame and idle timeouts, subscriptions, RPC dispatch, connection registry, max-connection cap) is already transport-agnostic and is reused unchanged.
3. Route publication: `MobileRouteResolver` adds one `iroh` route built from the endpoint (EndpointId, current relay URL, direct addrs) at a priority that beats Tailscale (lower number wins in `preferredRoute`). The route flows everywhere routes already flow: attach tickets, QR payloads, and `DeviceRegistryClient` POSTs to `/api/devices`.
4. The loopback-reject rule stays on the TCP lane. The iroh lane has no loopback concept; its equivalent floor is that QUIC handshake requires the dialer to know the EndpointId, and the Stack same-account check still gates every RPC.
### iPhone (client) side
One new transport, one registration:
1. `CmxIrohByteTransport: CmxByteTransport` (actor) wrapping the C FFI: `connect()` binds the phone endpoint (if needed) and dials the route's peer endpoint with relay/addr hints; `receive()`/`send()` map to stream reads/writes off the main thread; `close()` finishes the stream and closes. The blocking FFI calls run inside the actor like `CmxNetworkByteTransport` runs its continuation plumbing today; the shared tokio runtime in the staticlib does the async work.
2. Register `.iroh` in `cmuxApp.swift`'s `supportedKinds`. `CmxAttachTicket.preferredRoute(supportedKinds:)` then automatically prefers the iroh route on new builds while old builds keep picking Tailscale.
3. Connection failure classification: map iroh connect errors into the existing `CmxConnectFailureKind` so the UI keeps giving actionable messages ("Mac offline" vs "relay unreachable").
### Onboarding flow (the point of all this)
1. User signs into the iOS app (Stack).
2. `DeviceRegistryService` lists the account's Macs; each instance's routes jsonb now includes the iroh route with the Mac's EndpointId.
3. Phone dials by EndpointId. n0 discovery finds the Mac through its home relay; QUIC holepunches to a direct path when possible, relay carries traffic otherwise. No VPN, no LAN requirement, no QR.
4. Every RPC still carries the Stack access token; the Mac verifies same-account server-side. The registry is rendezvous, never authority (unchanged from https://github.com/manaflow-ai/cmux/pull/5626).
QR pairing remains exactly what it is today: first-trust UX and the fallback when the registry is unreachable. The QR payload's routes list simply includes the iroh route, so a QR pair also yields an EndpointId the phone can keep dialing from anywhere.
## Security and E2E story
iroh QUIC connections are end-to-end encrypted with TLS 1.3 using raw public keys: the EndpointId IS the peer's public key, and the handshake fails unless the dialed peer holds the matching secret key. Relays carry ciphertext only and cannot MITM a dial-by-EndpointId. This means the planned Noise IK layer (from the earlier pluggable-transports design, which assumed untrusted relays under plain TCP) is not needed for the iroh lane: the channel is already authenticated to the key we dialed.
What that reduces the trust problem to: distributing the authentic Mac EndpointId. The threat that matters is credential exfiltration, not host impersonation alone: the phone sends its Stack access token inside the protocol on every RPC, so a substituted route (compromised registry, authz bug, malicious route write) would receive a live bearer token from a phone that dials it. That exposure exists today with substituted host:port routes; iroh is the lane that can actually close it, because the channel is cryptographically bound to the EndpointId being dialed.
So EndpointId pinning ships in the first iroh lane (PR 3/4), not later:
- The phone pins the Mac's EndpointId in `MobilePairedMacStore` at first trust. QR pairing pins from physical proximity. Registry-only auto-pair pins on first attach (TOFU), which is no weaker than today's registry-trusted host:port baseline and strictly stronger afterwards.
- After pinning, the phone sends Stack tokens over iroh only to a connection whose dialed EndpointId matches the pinned one for that device row. A registry that later substitutes a different EndpointId cannot get a token: the dial either fails the QUIC handshake (wrong key) or fails the pin check before any frame is sent.
- A changed EndpointId for a known device (legitimate after a Mac Keychain reset) is surfaced to the user for explicit re-trust, never silently accepted.
Key custody: the iroh secret key is a 32-byte Ed25519 key. Mac and phone each generate once and store in Keychain as `kSecClassGenericPassword` with `kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly`, no iCloud sync (a synced key would make two devices claim one EndpointId). The FFI gains `bind` taking an optional caller-provided key plus a key-generation call, so key material lives in Swift/Keychain and is passed in, not minted and held inside Rust.
A stable phone-side key also gives every phone a stable EndpointId, which is the natural per-device identity the device-revoke design (cmuxterm-hq `plans/feat-ios-device-revoke/DESIGN.md`) was missing; the Mac can pin known client EndpointIds later. Not in scope for P1, but the keys are persisted from day one so the option exists.
## Relay strategy
Now: n0's default relay fleet via `presets::N0` (free public infrastructure run by n0; relays are dumb encrypted-byte forwarders). Note from the spike: iroh 1.0.0-rc endpoints currently land on n0's canary relays (`relay.n0.iroh-canary.iroh.link`); before shipping, pin the stable relay map and re-verify at iroh 1.0 GA.
Later: self-host `iroh-relay` (open source, stateless, cheap) under cmux.dev and set it as the relay map, keeping n0 as fallback. The `relay_url` field in routes means each Mac advertises whichever relay it actually homes on, so mixed fleets work during any migration. Self-hosting removes the third-party availability dependency and is the right place to add usage metrics.
## iOS background and battery behavior
An idle iroh endpoint maintains a relay connection and periodic keepalives. Policy: the phone's endpoint lifecycle is tied to the attach session and scene phase, exactly like today's TCP connections. Bind on first connect, close the endpoint when the app backgrounds (the existing scenePhase handling in `cmuxApp.swift` is the hook) and rebind on foreground attach. No background networking entitlement, no persistent background socket, so no new battery cost class: radio use happens only while the user is actually attached. Reconnect-on-foreground is fast (sub-second connect in the spike, plus the session/RPC layer already handles transport drops and re-dials).
The Mac side keeps its endpoint bound whenever the mobile host is enabled, same as the TCP listener today. Mac battery impact is a relay keepalive, negligible against a running cmux.
## Tailscale and LAN: opt-in fallback
New Settings toggle on the Mac (Mobile section): "Also publish Tailscale/LAN routes" (exact copy TBD, localized en+ja like every string in this feature). Semantics:
- ON: `MobileRouteResolver` publishes tailscale routes (current behavior) in addition to iroh, at lower preference.
- OFF (eventual default): only iroh (plus debug loopback in DEBUG).
Rollout compatibility: during the transition the toggle defaults ON so existing paired phones on old builds (which only support `.tailscale`) keep connecting; `preferredRoute` picks iroh on new phones automatically because of priority ordering. Flipping the default to OFF is a later, separate change once the fleet has the iroh-capable build. Direct LAN connectivity does not actually need Tailscale at all under iroh: direct addrs in the iroh route make same-LAN dials holepunch-free, so the toggle is genuinely only for people who want the tailnet path.
## Packaging (no vendored binaries)
The spike's Rust crate graduates to `native/cmux-iroh/` in-repo (source only, Cargo.lock committed). A `scripts/ensure-cmux-iroh.sh` builds `CmuxIrohFFI.xcframework` (macOS arm64+x86_64, iOS device arm64, iOS sim arm64) into a gitignored path, mirroring the GhosttyKit pattern (`scripts/ensure-ghosttykit.sh`, gitignored `GhosttyKit.xcframework`). CI and fleet builders gain a pinned Rust toolchain step next to the existing pinned Zig step (`scripts/install-zig-ci.sh` precedent). The xcframework links into the Mac app and the iOS app; binary cost is about +7.7 MB per slice (measured in the spike), which is acceptable against the existing GhosttyKit payload.
This toolchain addition (Rust on every CI runner and fleet builder) is the main reason P1 is its own PR rather than bundled with the spike: it touches build infrastructure shared by every job and deserves isolated review and a canary CI run.
## Reconciliation with the hive design
cmuxterm-hq `plans/feat-hive/DESIGN.md` assumed "Tailscale is the substrate the user sets up; cmux assumes it and verifies it" with iroh as a P3 seam. This decision inverts that for phone-to-host: iroh is the default substrate and Tailscale is the opt-in. What carries over unchanged from hive, because the hive node contract was deliberately transport-pluggable:
- The hive node contract (frame codec, Stack auth, capabilities, render-grid, registry) does not name a transport; an iroh route is just another `CmxAttachRoute` kind in the registry's opaque routes jsonb.
- Linux hive nodes get iroh almost free: the host body is the Go `cmuxd-remote`, which can cgo the same C FFI staticlib (it is the same Rust crate, built for linux targets), or n0's Go path if that is cleaner at implementation time. The "headless box needs its own Stack credential" problem from hive is unchanged and orthogonal.
- Hive's "verify, don't manage" Tailscale UX (the phone-side `TailscaleStatus` detector, https://github.com/manaflow-ai/cmux/pull/5722) becomes the opt-in lane's diagnostics instead of the default lane's.
- Hive's P1 "tailnet-only, WireGuard is the encryption" security story is superseded on the default lane by iroh's QUIC raw-public-key TLS, which is stronger in one respect: it is end to end per-peer rather than per-network.
The hive doc should be updated to say: default connectivity = registry routes dialed iroh-first; Tailscale/LAN = opt-in fallback routes. Nothing else in it changes.
## Delivery plan (stacked PRs)
P1 is not bundled here because it carries a build-toolchain change for all CI and fleet builders; the seam work itself is mechanical. Stack, each independently revertable:
1. **This PR**: spike (`experiments/iroh-swift-ffi-spike/`) plus this design doc. No app/runtime changes, no reload needed.
2. **PR 2, packaging**: `native/cmux-iroh/` crate (FFI grown to caller-provided keys and error-kind codes), `scripts/ensure-cmux-iroh.sh`, Rust toolchain in CI, xcframework linked into both apps but referenced by nothing. Proves the build matrix everywhere without behavior change.
3. **PR 3, phone dial lane**: `CmxIrohByteTransport` in `Packages/CmuxMobileTransport` plus registration, behind a feature flag (DEBUG default ON, release default OFF). Includes the EndpointId pin gate from the security section: pin at first trust in `MobilePairedMacStore`, refuse to send Stack tokens to a non-matching EndpointId, explicit re-trust UX on change. Unit tests with the existing transport test doubles; one loopback-style integration test dialing a local iroh listener.
4. **PR 4, Mac host lane**: `MobileHostByteConnection` seam extraction in `MobileHostService` (pure refactor commit first, NWConnection adapter only, zero behavior change), then the iroh accept loop, Keychain key custody, route publication in `MobileRouteResolver`, registry propagation. Same feature flag.
5. **PR 5, default-on plus Settings**: flag becomes a real Settings toggle pair (iroh on by default; "Also publish Tailscale/LAN routes" on by default for compat), strings en+ja, docs. Flip of the Tailscale-publication default is a later standalone change.
Dogfood gate between 4 and 5: phone on cellular (Tailscale off) attaches to the Mac by sign-in alone, terminal latency subjectively fine on both relay and holepunched paths, reconnect after backgrounding works.
## Open questions and risks
- **iroh 1.0 is rc**: API churn risk between rc.1 and GA is real but the surface we use (Endpoint, connect/accept, streams) is the stable core. Pin the version; re-verify the relay map at GA (canary-relay note above).
- **Discovery latency tail**: spike showed 0.5 to 1.0s connects on a warm network; cold relay discovery on bad networks needs measurement during PR 3/4 dogfood. The route's direct_addrs and relay_url hints cut discovery out of the hot path.
- **Staticlib size in the iOS app**: +7.7 MB/slice is fine for the dev/TestFlight app; recheck App Store thinning behavior at submission time.
- **Tokio runtime inside two apps**: one multi-thread runtime (2 workers) per process, sized in the FFI. No conflict with Swift concurrency observed in the spike; keep FFI calls off the main actor by construction (actor-isolated transport).
- **Same-account ceiling**: unchanged from today; team-shared Macs and per-device revoke remain future work (per-device identity now has a natural carrier in stable phone EndpointIds).