Skeleton Demo

The standalone skeleton demo displays a live camera view with estimated hand landmarks and connections for both hands and all five fingers. The full-body wireframe is not drawn. Each hand has 21 landmarks. The existing gestures are not graded. The optional comparison panel reports pose similarity; video and positions are not persisted.

A left palette offers Cube and Pyramid as miniature labeled 3D wireframes, using the same exact geometry, projection, solid/faded dashed edges, midpoint equality ticks, and AB annotation as the active figure on the right. Initially neither is selected. Place the midpoint of your thumb and index fingertips anywhere inside a figure button. The compact buttons use half-spacing padding and translucent theme-colored backgrounds so the camera remains visible behind fully opaque icons and labels. Whenever those fingertips are pinched, that button selects immediately. No prior opening or closing motion is required. Keep the pinch held while moving between buttons to switch figures. Middle-finger position does not block selection; a three-finger pinch over a button also selects it. Tracking reacquisition accepts an already pinched hand immediately. Selection preserves each figure's size and rotation. Hand tracking lines appear above the icons, with the camera image behind them. Click, touch and keyboard activation are also supported. Palette space consumes gestures without rotating or resizing. Rotation/resizing retain the controlling hand while it remains identifiable. Either valid hand can select a figure from the palette; an accepted selection transfers control and establishes a fresh reference. The existing status area explains targeting, pinch distance, and missing or uncertain hand tracking.

Away from the palette, turn your palm. The figure follows the palm plane in all three rotation axes with smoothing; no finger pose is required. Triple pinch all three tips, then move up to increase edge length/display size or down to decrease them. A quarter of the preview height doubles/halves length, within 0.25-4 units. Rotation stays unchanged during resizing. Each figure remembers its own rotation and size.

Both are exact labeled wireframes with solid visible edges, 30%-opacity dashed hidden edges, matching midpoint equality ticks, and no XYZ axes. Cube letters are A-H; pyramid A-D surround its square base with E at the apex. All edges start at 1 unit, and the AB label updates on resizing. Pyramid height is 1/sqrt(2), with its volume centroid one-quarter of that height above the base. A fixed orthographic view fits maximum size; text and ticks retain screen-sized dimensions. Letters and the AB annotation have fixed 3D anchors that rotate and scale with the solid. Text stays upright; labels are not rearranged to avoid overlaps or clamped independently at viewport edges. Anchors that project to the same screen position can overlap.

Slow movements apply, fast movements are ignored, and the next slower increment resumes immediately without holding still or replaying motion. Each mode/hand change establishes a fresh reference. Button selection takes priority inside the palette; triple pinch activates resizing outside it.

Lowering the hand quickly disengages through screen movement. Apparent hand-size changes provide only a coarse cue for movement toward/away from the camera; hand-centered 3D landmarks are not room-position measurements.

Rotation uses the wrist and index/little-finger base landmarks (0, 5, 17) in estimated 3D world coordinates. These define a palm plane and an in-plane axis, independent of fingertip positions, finger extension, or fingertip overlap. A palm triangle area ratio below 0.08 disables rotation; a normalized side-proportion change above 0.12 rejects that movement. The full relative orientation includes tilt and twist. Forward/backward wrist tilt is reversed about the camera-horizontal axis; side-to-side turning and clockwise/counterclockwise rotation keep their existing directions. This mapping is applied after smoothing in both mirrored and unmirrored views. A One Euro-style adaptive spherical orientation filter uses a 225 ms resting time constant and reduces it toward 100 ms during deliberate turns. Signed 3D angular velocity is low-pass filtered at 1 Hz before its magnitude sets the orientation cutoff (base 0.707 Hz plus 1 Hz per rad/s, capped at 1.592 Hz). Filtering the velocity vector lets alternating jitter cancel instead of automatically accelerating the filter. Both filters use elapsed-time exponential integration; orientation follows the shortest 3D rotation without Euler-angle averaging or a dead zone. This is an SO(3) adaptation of the One Euro filter, with a bounded cutoff. It does not predict motion or interpolate display frames between observations. Resizing requires the three fingertips together and takes priority over palm rotation. Outside the palette, triple pinch takes priority over rotation. Ambiguous directions, tracking gaps over 250 ms, mode/owner changes, and rejected motion establish a fresh smoothing reference and clear the angular-velocity estimate. Angular speed above 720 degrees/s, translation above 6 palm widths/s, or apparent size change above 4.5/s is ignored without replaying that movement. Selection accepts a thumb-index gap up to 80% of estimated 3D palm width, independently of prior frames or middle-finger spacing. Triple pinch activates resizing immediately at 80% and remains active up to 95% to avoid mode jitter. Its first observation establishes the position from which up/down movement changes size; there is no arming step, opening requirement, or dwell timer. Palm width is measured between index and little-finger bases in 3D, so ratios do not directly use foreshortened 2D camera distances. For a 60 mm palm, both selection and resizing accept gaps up to 48 mm (all three pairs for resizing). Fingertips do not need to touch or press together. These are model estimates requiring physical testing. Distances are model estimates, not calibrated physical measurements. The speed cutoffs are now three times the initial demo values: 720 degrees/s for turning, 6 palm widths/s for screen movement, and 4.5/s for apparent-size change. There is no hold-still timer, separate reconnection speed threshold, or reversal-specific acceleration cutoff. Tracking-quality checks remain in place.

Add "skeletondemosession": {} to a unit with a unitname. Do not combine it with another session selector, adaptive transitions, or enabled gazeTracking. The test package is C:\dev\mofacts_config\Skeleton_Demo.zip; editable source is in C:\dev\mofacts_config\Skeleton Demo. It contains exactly one demo unit and an empty stimulus file, with no questions or placeholder answers.

Use HTTPS or localhost. Choose Start camera, allow camera access, and move your hands and fingers to inspect alignment. Stop camera releases the camera; Continue finishes the unit through normal lesson progression. That existing completion state is the only saved information. Leaving the unit or switching to another tab stops the camera and tracking. Return and choose Start camera to resume the display. Stopping, switching, or hiding the camera clears gesture acquisition but retains both figures' displayed orientations and lengths within the mounted unit. Leaving discards them. Only current landmarks, a previous gesture reference, and per-figure transforms/lengths exist in browser memory; nothing is recorded or uploaded.

Camera and Wireframe visibility buttons are independent. Camera hides the video view; Wireframe hides the hand tracking lines and dots. The geometric figure remains visible in all four combinations. Hiding either or both keeps capture and gesture tracking running; Stop camera releases it. Both visibility settings initially start On and persist across camera restarts and switching until this demo unit is left. Pressed states are announced and the buttons work with the keyboard.

Start camera opens the default/preferred camera, including desktop webcams that do not identify themselves as front or rear cameras. Switch camera appears only after starting capture when the browser reports more than one camera. It cycles through those devices; a single-camera computer has no switch button. Switching releases the current camera before opening the selected one. The front/webcam view is mirrored; a camera reporting rear-facing is not. If switching fails, choose Start camera to reopen the default camera.

A compact toolbar, brief status, and short privacy note surround the preview. The lesson heading appears once in the application header. Small side margins, buttons, colors, and typography follow the main application's theme. The preview uses the remaining screen space and shrinks proportionally to fit both width and height, preserving the complete video and overlay without a scrollbar. The controls, demonstration canvas, and figure palette use the shared card style, including theme-controlled surface, border and rounding. The application theme also owns toggle/selection buttons and the figure's text/accent colors.

Both models run in the browser using locally hosted assets. No camera images or body positions are transmitted. Hidden joints may have no overlay; inferred depth is not a calibrated room measurement. The demo's labels are English. Three.js math projects both figures into a theme-owned SVG display loaded only for this demo. No WebGL context is required. Display loading/projection failure stops the camera and shows an explicit error. The distributed MIT notice is at /licenses/three-LICENSE.txt.

For acceptance, try both hands with repeated slow/fast/slow strokes, lowering the hand, finger bending, and movement toward the camera. Confirm the figure stays still during fast movement and resumes immediately on slower movement without jumping. Automated synthetic and layout checks do not establish physical gesture quality. Also test both palette targets, palm tilt and twist, palm rotation, triple pinch resizing in both directions, size-limit reversal and per-figure restoration. Automated browser checks cover all four layer combinations while retaining the camera stream and gesture control, plus Default/Dark Industrial card styling.

Implementation, setup, reuse boundaries, and verification commands are documented in the application repository's docs/skeleton-demo.md.

Partial hand visibility

Validation is specific to each gesture: button selection requires thumb and index tips; resizing requires thumb, index, and middle tips. Palm rotation needs no fingertips, only usable wrist and index/little-knuckle estimates. Missing, invalid, or off-screen fingertips do not block palm rotation. Missing middle fingertips do not block button selection. Pinches require their own fingertips in view. Palm estimates and confident hand identity remain necessary for scale and motion continuity; the tracker must still detect a hand. Missing points are not fabricated. Losing a required point stops that gesture; reacquiring it establishes a fresh motion reference, so rotation or resizing does not jump. Button selection remains immediate. Synthetic tests cover missing/invalid points and all four frame edges; physical camera reliability still requires testing with the actual device.

The Wireframe toggle shows or hides only hand landmark dots and connections. Body landmarks are not rendered. Camera visibility, figure visibility, and gesture controls keep their existing behavior. This display change does not change the tracking service's model loading or inference contract.

Experimental 3D pose comparison

At localhost, open Skeleton Demo and expand 3D pose matcher comparison in the preview's lower corner. Start the camera yourself. Keep both shoulders, elbows and wrists visible, capture a template, then choose Calibrate person and move both arms across 45 well-tracked frames. Return to the reference pose to compare simple distances with a skeleton corrected using fixed target-to-person segment-length ratios. Calibration uses segment medians independently of scores. Lower normalized squared error means closer geometry; no recognition threshold is claimed. One optional global scale fit applies to both methods; calibrated error without that fit is also shown. Labels remain left/right, and reflected geometry is treated as equivalent. Fingers are outside this first comparison.

The template, calibration samples and scores stay only in browser memory until Clear comparison or leaving the unit. Stop camera clears current scores and pauses collection. Existing selection, rotation and resizing remain active. The panel overlays the preview and scrolls when expanded on small screens.

The detailed method and deterministic/synthetic test protocol are in the app repository's pose matcher plan. Physical camera quality and recognition accuracy require a person to test them.

Camera suite lifecycle

Gaze and this demo share the application's cameraTrackingSession.ts ownership boundary. Starting this camera releases a paused gaze stream and invalidates its calibration. On return, a gaze-enabled unit prompts for calibration again; ordinary disabled units still retain it. Models and observations remain separate, and a unit still cannot enable both modes. Route exit, logout/account change, lesson change, page hide, unit unmount, Stop camera, and switching cancel pending work and release owned resources. A pending browser permission request must settle before a new owner opens its camera. A late stream is stopped before handoff proceeds.

poseTracking.ts schedules at most one inference frame at a time and at most 15 Hz, skipping duplicate decoded video frames. It transfers one temporary ImageBitmap to poseTrackingWorker.ts, which loads the same pinned local CPU pose/hand models. The worker closes each bitmap after inference; stopping terminates the worker, including pending WASM/model loading. No frames are recorded, retained as history, or uploaded. Observation timestamps are monotonic inference-input times, not physical camera capture times. Pose and hand landmarks/confidence reach the app; FigureOverlay.svelte applies the existing 0.7 handedness-confidence policy at the gesture-consumer boundary. Gesture interpretation stays in learning-components/.

The worker provides runtime isolation because WebGazer's older FaceMesh WASM and Tasks Vision both use Emscripten globals. Loading them in the same window reproduced a Module.noExitRuntime initialization failure. Separate globals resolve that collision without changing dependencies, upstream bundles, served assets, or CSP. A browser must support workers, transferable ImageBitmap, and OffscreenCanvas; unsupported/failed inference reports the existing explicit unavailable state. There is no substitute model or main-thread inference path.

Run node --test scripts/cameraTrackingSuite.test.cjs scripts/gazeTracking.test.cjs scripts/skeletonDemo.test.cjs for controlled lifecycle checks, and node --test scripts/skeletonDemoBrowser.test.cjs with the supported localhost watcher for the real-library transition and existing gesture/display checks. The latter uses synthetic video and does not establish physical tracking accuracy.