Enobarbus
Enobarbus — Claude Context
Computer vision and tracking package. Depends only on Horatio. Provides camera management, all trackers (Vision + ARKit), processing utilities, and diagnostic overlay views. All tracking code is #if os(iOS).
Architecture
Session Layer
Two mutually exclusive session types — ARSession and AVCaptureSession cannot run simultaneously on iOS.
| File |
Role |
TrackingSession.swift |
TrackingSession protocol + TrackingBackend enum (.vision / .arkit) |
VisionTrackingSession.swift |
Wraps CameraManager → conforms to TrackingSession |
ARTrackingSession.swift |
Owns ARSession, two configs: .face (front/TrueDepth) or .body (rear/A12+). Distributes ARFrame.capturedImage to pixel buffer handlers + routes typed anchor callbacks |
Camera
| File |
Role |
CameraManager.swift |
Owns AVCaptureSession, distributes frames via FrameHandler / PositionAwareFrameHandler / DepthHandler. Front/rear switching, orientation tracking, depth output config |
Vision Trackers
All follow the same pattern: processFrame(_:orientation:) → @Published [SourceType] on main thread. Also have processPixelBuffer(_:orientation:cameraPosition:) for ARKit interop.
| File |
Sources |
Count |
HandTracker.swift |
[HandInputSource] |
Up to 4 hands, chirality, finger count, identity resolution |
BodyTracker.swift |
[BodyJointInputSource] |
19 joints via VNDetectHumanBodyPoseRequest |
FaceTracker.swift |
[FaceLandmarkInputSource] |
11 sources: 3 positions + 5 expressions + 3 rotations |
DepthTracker.swift |
[DepthEnrichedInputSource] |
TrueDepth / LiDAR depth enrichment |
ARKit Trackers
Process ARKit anchors instead of Vision requests. Produce the same base source IDs as their Vision counterparts for compatibility.
| File |
Sources |
Notes |
ARKitFaceTracker.swift |
[FaceLandmarkInputSource] |
11 base IDs (same as Vision) + 52 face.arkit.{blendShape} extended sources. Uses FaceTracker.normalizeAngle for head rotation |
ARKitBodyTracker.swift |
[BodyJointInputSource] |
19 base IDs (same as Vision) + extended body.arkit.{joint} sources. 3D→2D via ARCamera.projectPoint |
ARKitCapabilities.swift |
Extension on TrackerCapabilities |
arkitCapabilities (runtime detection), allAvailable (Vision + ARKit) |
Processing
| File |
Role |
PositionStabilizer.swift |
EMA smoothing (α=0.6), velocity clamping (0.15), stale-frame recovery (5 rejections) |
RelationshipProcessor.swift |
Derives compound sources from joint pairs (distance, angle, deltas). 7 default definitions |
BodyCalibration.swift |
T-pose calibration for body tracking normalization |
TrackingMode.swift |
Enum: .hands, .body, .face, .relationships, .all |
Views (Views/)
Diagnostic overlay views for PhotoLab. All #if os(iOS).
| File |
Role |
CameraPreviewView.swift |
UIViewRepresentable wrapping AVCaptureVideoPreviewLayer |
TrackingSessionPreviewView.swift |
UIViewRepresentable wrapping any TrackingSession.makePreviewView() |
HandOverlayView.swift |
Dot overlay for hand positions |
BodyOverlayView.swift |
Joint dot + skeleton overlay |
FaceOverlayView.swift |
Face landmark overlay |
RelationshipOverlayView.swift |
Lines between related joints |
ConfidenceIndicator.swift |
Visual confidence badge |
SourceDataView.swift |
Scrollable data panel showing all active sources |
CalibrationView.swift |
T-pose calibration UI |
Key Implementation Details
- Coordinate transforms:
transformPosition(_:for:cameraPosition:) handles Vision→screen mapping with front-camera X mirror. All orientations supported.
- Hand identity:
resolveHandIdentities detects Vision chirality swaps between frames using position continuity (2-hand case, 2x threshold).
- Finger counting: Compares fingertip vs PIP joint (not MCP) for deliberate extension detection. Holds last count when tips leave frame.
- ARKit constraint: Face tracking = front camera only. Body tracking = rear camera only. Cannot run both ARKit configs simultaneously. In "all" mode: one ARKit config + Vision fallback for the other.
- Frame skipping: ARTrackingSession skips every other frame for hand detection when ARKit is active (
handDetectionFrameSkip).
Tests
cd Packages/Enobarbus && swift test
ARKitFaceTrackerTests, ARKitBodyTrackerTests, ARKitCapabilitiesTests) are #if os(iOS) gated — they run on iOS Simulator builds but not macOS swift test.
Rules
- All tracker
@Published properties publish on main thread via DispatchQueue.main.async.
- Source IDs are stable contracts — do not change
body.{joint} or face.{landmark} names without updating all downstream consumers.
processPixelBuffer must stay in sync with processFrame — they share the same core logic.- Adding new trackers: follow the existing pattern (ObservableObject,
@Published inputSources, processFrame/processPixelBuffer, allSourceIDs).