vrlizate library

vrlizate — VR 3D engine for Flutter.

Complete VR framework: scene graph, PBR materials, mesh rendering, lighting with shadows, stereoscopic projection, head/hand tracking, raycasting, spatial UI, locomotion, physics, animation with skeletal deformation, glTF loading, effects, and WebXR session management.

import 'package:vrlizate/vrlizate.dart';

final engine = VREngine();
engine.scene.add(LitMeshNode(
  geometry: CubeGeometry(),
  material: PBRMaterial(color: Color(0xFF2E90FA), metallic: 0.8),
));
engine.scene.add(Light.directional());
engine.enableHeadTracking();
engine.start();

CustomPaint(painter: engine.stereoPainter)

Classes

Aabb: Axis-aligned bounding box for frustum culling.
AimAction: The user is aiming with a hand ray; ray is the pointing ray in tracking space.
AnimationClip: A clip containing multiple tracks that animate a node.
AnimationMixer: Plays and blends multiple animation clips on a node.
AnimationPlayback: Handle to control a playing animation.
AnimationTrack: A track that animates a single property of a node.
BackAction: Navigation "back" / cancel (fist gesture or ultrasonic pull).
Billboard: A node that always faces the camera (billboarding).
BloomEffect: Simple bloom effect applied as a post-process overlay. Draws bright areas with gaussian blur to simulate light bleed.
Bone: A bone in a skeleton hierarchy.
CameraHandTrackingDriver: Drives HandState instances from real camera landmark frames.
CameraRig: Camera rig for VR: head position + rotation, with stereo eye offsets. Produces view and projection matrices for left/right eyes.
ConfirmAction: A confirm gesture (thumbs up / ultrasonic push while aiming).
ControllerState: Represents the state of a VR controller (left or right hand).
CsiFrame: A simulated raw Channel State Information frame. Represents OFDM subcarrier amplitudes affected by human movement.
CubeGeometry
CylinderGeometry: Closed cylinder along Y, centered on the origin, wound counter-clockwise from outside like CubeGeometry so back-face culling keeps the near side.
DebugOverlay: Debug overlay that displays performance metrics in VR.
DepthDisplacedGeometry: A 3D geometry generated by displacing a grid mesh using a depth map list.
DesktopInputDriver: Desktop input driver: mouse-look, WASD locomotion, and mouse-picking.
DesktopInputRegion: Widget wrapper that forwards mouse and keyboard input to a DesktopInputDriver. Place it around the VR viewport on desktop targets.
DeviceParams: VR headset device parameters. Different headsets have different lens properties.
DistortionMesh: Polynomial radial distortion for VR lens correction. Based on Google Cardboard SDK distortion algorithm.
FaceTrackerDriver: Driver that feeds live camera/MLKit 3D face position coordinates into CameraRig. Delivers the Looking-Glass "Holographic 3D Window" effect by altering off-axis projection.
FlyLocomotion: Free-fly locomotion: move in any direction including up/down.
FogEffect: Distance-based fog effect.
GazePointer: Gaze-based pointer for VR interaction (look to select). Fires a ray from the camera center forward direction.
Geometry: Raw vertex data for a 3D geometry.
GeometryPlacement: One source geometry and the local transform baked into a merged geometry.
GltfParser: Lightweight glTF 2.0 JSON parser for vrlizate. Handles meshes, materials, and node hierarchy. Does NOT handle textures (use PBRMaterial.colorMap for that).
GltfResult: Result of parsing a glTF file.
GPURenderer: GPU rendering abstraction layer.
GPURendererStub: Stub implementation that reports GPU is not available. Replace with real implementation when flutter_gpu is stable.
Grabbable: Makes a node grabbable via controller grip or pinch.
GridFloor: A grid rendered on the XZ plane for spatial reference.
HandBone: Represents a bone connection between a parent and child joint in the hand skeleton.
HandGestureEvent: A recognized gesture event emitted by HandGestureRecognizer.
HandGestureRecognizer: Converts raw HandState poses into debounced semantic gesture events.
HandLandmarkFrame: A single frame of 21 hand landmarks produced by an on-device ML source (MediaPipe Hand Landmarker, ML Kit, or any compatible detector).
HandState: Represents the tracked state of a hand with 26 joints following OpenXR standard.
HeadTracker: Head tracking input via device gyroscope with calibration and background Isolate.
HologramMeshNode: Advanced Volumetric Hologram Simulation Node implementing multi-pass shell rendering, vertex glitched offsets, time-based flickering, and scanline overlay.
HybridRayHit: Result of a low-cost world-space ray query.
HybridRayTracer: CPU ray queries designed for hybrid lighting on mobile devices.
InertialTapConfig: Configuration for the inertial head/visor tap detector.
InertialTapDetector: Zero-accessory mechanical trigger that detects physical taps on the side of the VR visor or the user's temple using the phone's accelerometer.
InputFusion: Fuses the joystick-free input channels of the engine into a single action stream with priority and conflict resolution:
Keyframe<T>: A single keyframe in an animation track.
LensDistortion: Lens distortion parameters for VR optics. Barrel distortion corrects for the magnification effect of VR lenses.
Light: Light source in the scene.
LitMeshNode: Renders a mesh with per-face lighting from scene lights.
MediaPipeHandDriver: Converts MediaPipe / MLKit 3D landmark coordinates to an OpenXR HandState.
MeshNode: A renderable 3D mesh node with geometry and material.
MoveAction: Continuous locomotion request in units per second along the head's ground-projected forward direction (0 = stop).
Node: Base node in the scene graph with parent-child hierarchy. All spatial objects inherit from Node.
Offset3D
OneEuroFilter: One Euro filter for low-latency smoothing of jittery landmark positions.
PBRMaterial: Physically-Based Rendering material with texture maps. Inspired by Thermion/Filament PBR and three_js MeshStandardMaterial.
PhysicsWorld: Simple physics world that manages rigid bodies.
PlaneGeometry
Pointable: Makes a node respond to pointer hover and click events.
ProjectedPoint: Projected 2D point with depth info for z-sorting.
QuaternionTrack: Animates a Quaternion property (rotation).
Raycaster: Raycaster for testing ray-mesh intersections in the scene.
RaycastHit: Result of a raycast hit.
RenderPass: Render pass that traverses the scene and renders to a canvas. Handles: frustum culling, depth sorting, lighting injection, background.
RigidBody: Simple 3D rigid body physics for VR interactions. Handles gravity, velocity, collision response, and damping.
RotationTarget: Interface for anything that can receive rotation input.
RttMeasurement: A single round-trip-time distance measurement to an anchor.
RttPosition: Room-scale position estimate produced by WifiRttPositioning.
Scene: Root of the scene graph. Contains the environment and all nodes.
SelectAction: A confirmed selection ("click") on a target, from any channel. targetId is null when the action came from a channel without a ray target (e.g. ultrasonic push with no gaze hit).
ShaderProgram: Abstraction for a shader program (vertex + fragment).
ShaderRegistry: Registry of available shader programs.
ShaderUniform: A shader uniform variable.
ShadowConfig: Shadow configuration for a light source.
ShadowRenderer: Simple shadow renderer that projects shadows onto the ground plane. Uses a simplified shadow map approach for Canvas rendering.
SharedPrimitives: Process-wide, immutable unit primitives for retained GPU rendering.
Skeleton: A skeleton is a hierarchy of bones used for mesh deformation.
SkinDeformer: Deforms mesh vertices using skeletal bone weights. CPU-based linear blend skinning (LBS).
SpatialButton: A clickable 3D button in VR space.
SpatialNavigationArrow: A gaze/tap-enabled 3D arrow that points toward the upper-left.
SpatialPanel: A floating UI panel in VR space. Renders a rectangle with background, border, and content callback.
SpatialText: 3D text that floats in VR space, always facing the camera.
SphereGeometry: UV sphere centered on the origin with analytic (exactly radial) normals, so shading is smooth at any tessellation instead of showing the faceting that averaged face normals leave on a coarse mesh.
SSAOEffect: Screen-Space Ambient Occlusion approximation for Canvas rendering. Uses a simplified approach: darkens corners and edges of the viewport to simulate ambient light being blocked by nearby surfaces.
StereoscopicProjection: Stereoscopic projection engine. Projects 3D points to left/right eye viewports.
TeleportLocomotion: Teleportation locomotion: point at floor, press trigger, move there.
Transform3D: Full 3D transform with position, rotation (quaternion), and scale. Uses dirty-flagged matrix caching for performance.
UltrasonicGestureChannel: Active-ultrasonic hand gesture channel (FingerIO / LLAP / AudioGest family): the device speaker emits an inaudible tone and hand motion near the microphone is detected from the Doppler-shifted echo.
UltrasonicGestureConfig: Configuration for the ultrasonic sensing channel.
UltrasonicGestureEvent: An event emitted by UltrasonicGestureChannel.
Vector3Track: Animates a Vector3 property (position, scale).
VignetteEffect: Vignette effect for VR lens simulation. Darkens the edges of each eye viewport to simulate lens optics.
VRAnaglyphPainter: Paints a VR scene in Red/Cyan Anaglyph 3D stereoscopic mode. Enables glasses-free or $1 3D glasses stereoscopic rendering.
VRCamera: Legacy VR camera with spherical orientation (theta/phi). Used by the original neom_vr particle system. For new code, prefer CameraRig which uses Matrix4 + Quaternion.
VRConnection: A line connecting two VR elements.
VrControlDescriptor: One visual and semantic control exposed by a remote device.
VrControllerArmModel: Head-relative pseudo-6DoF position model for a 3DoF phone controller.
VrControllerAvatarNode: Lightweight visual twin of a paired smartphone controller.
VrControllerMessage
VrControllerMessageCodec: Newline-friendly JSON codec used by the reference socket transport.
VrControllerProfile: Description sent once by the child so the visor can build a controller UI.
VrControllerTransport: Extended transport used by paired smartphone controllers.
VrDisconnectMessage
VRElement: Base class for any object rendered in the VR scene.
VREngine: Main VR engine. Manages the game loop, scene, camera, and rendering.
VREnginePainter: CustomPainter that renders the VR engine output.
VrErrorMessage
VrExternalRenderTargetDescriptor: Immutable descriptor defining the properties of an external GPU render target.
VrFrustum: View frustum defined by 6 planes. Used for culling objects outside the camera view.
VrGamepadDriver: Reference adapter from normalized gamepad state to canonical VR input.
VrHelloMessage
VrInputAction: High-level, device-agnostic VR actions produced by fusing every available input channel (gaze, camera hand tracking, inertial walk-in-place, and ultrasonic gestures).
VrInputArbiter: Resolves input conflicts without allocating in the per-frame event path.
VrInputDriver: Extension point for gamepads, phones, vision systems, and custom sensors.
VrInputEvent: Unified spatial input event.
VrInputEventBus: Central Event Bus for unified multimodal spatial input in VRlizate.
VrInputEventPool: Fixed-capacity pool for VrInputEvent instances.
VrInputEventSnapshot: Immutable copy of a potentially pooled VrInputEvent.
VrInputMessage
VrInputPrioritizedDriver: Optional per-driver priority override.
VrInputSink: Sink passed to a VrInputDriver when it is attached to an arbiter.
VrLaserPointerDriver: Reference 3DoF laser pointer for a remote or second smartphone.
VrlizateScene: High-quality scene graph used by VRlizate experiences.
VrLocalSocketTransport: Unsupported-platform placeholder. Use WebRTC/WebSocket on web builds.
VRMaterial: Material defines the visual appearance of a mesh surface.
VRMonoPainter: Paints a VR scene in monoscopic (single eye) mode.
VrPairingPayload: Immutable connection invitation shared by the parent visor and child controller.
VRParticle: A point/particle in VR space.
VrPingMessage
VrPongMessage
VrPoseMessage
VrProfileMessage
VrRay: A ray defined by origin and direction. Named VrRay to avoid collision with vector_math's Ray.
VrReadyMessage
VrRemoteBinaryCodec: Experimental binary wire codec for compact VRlizate pose streaming.
VrRemoteControllerSession: Parent-side bridge from a paired controller transport into input and scene.
VrRemoteImuController: Child-side IMU producer for a low-cost smartphone controller.
VrRemotePoseFrame: Immutable high-frequency sample sent by the child controller.
VrRemotePosePredictor: Allocation-free angular pose prediction for a networked 3DoF controller.
VrRemotePoseSmoothing: Opt-in One Euro smoothing for a remote IMU orientation, before prediction.
VrRemoteTouchpadDriver: Converts normalized coordinates from a remote phone into navigation input.
VRRenderer: Abstract renderer that VR scenes implement.
VRRing: A ring/circle in VR space (rendered as an arc at its projected position).
VRScene: A scene containing VR elements that can be rendered stereoscopically.
VrSensorCapabilities: Platform registration of the device-motion backend, not a guarantee that a particular device contains a sensor or that browser permission is granted.
VrSpatialInputState: Renderer-independent, borrowed-event-safe navigation and pointer state.
VRStereoPainter: Paints a VR scene in stereoscopic split-screen mode.
VRTexture: Texture for mapping images onto mesh surfaces.
VrTransport: Transport contract for direct sockets, Bluetooth LE, or Wi-Fi Direct.
WalkInPlaceDetector: Detects walking-in-place steps from the device accelerometer using head-bob analysis (VR-Step style).
WalkInPlaceLocomotion: Walk-in-place locomotion: converts detected steps into forward movement of the camera rig, in the head's ground-projected forward direction.
WalkLocomotion: Thumbstick-based walk locomotion.
WifiRadarNode: A 3D node representing the WiFi electromagnetic sensing grid and radar fields.
WifiRttAnchor: A Wi-Fi RTT (IEEE 802.11mc) anchor: an access point or RTT-capable device with a known physical position in the room.
WifiRttPositioning: Room-scale positioning via Wi-Fi RTT (IEEE 802.11mc / FTM).
WifiSensingSystem: SOTA WiFi Sensing Engine that processes raw CSI frames in a background Isolate to track movement, presence, and vitals (respiration) of human subjects.
WifiTrackedSubject: Represents a physical entity tracked in the room through WiFi CSI (Channel State Information).
XRSession: Abstract XR session for future WebXR/AndroidXR integration.
XRSessionStub: Stub implementation for platforms without XR support.

Enums

ControllerHand
CullResult
EasingFunction: Easing functions for interpolation.
HandGesture: Semantic gestures recognized from tracked hands.
HandJoint: 26 joints per hand following the OpenXR hand tracking standard.
LightType
TextureFilter
TextureWrap
UltrasonicGesture: Gestures detectable through active ultrasonic sensing (speaker emits an inaudible tone, the microphone listens to the hand's echo).
UniformType: Shader uniform value types.
VrControlKind
VrControllerHandedness
VrDeviceRole: Role of a device in a VRlizate controller pairing session.
VrGamepadButton: Standard face buttons and analog triggers exposed by a gamepad adapter.
VrGamepadStick: Standard analog sticks.
VrGpuBackend: GPU backend types supported for external render target presentation.
VrGpuTextureFormat: Common texture pixel formats used by VR swapchains and render targets.
VrImuScreenOrientation
VrInputPriority: Input priority used to resolve simultaneous multimodal interactions.
VrInputSource: Hardware or software source that originated the input event.
VrInputType: Canonical input interaction types across VRlizate XR experiences.
VrlizateRayTracingMode: Lighting ray-query strategy.
VrlizateSceneQuality: Hardware-aware scene quality for the modern VRlizate scene graph.
VrRangeSource
VrTransportConnectionState
VrTransportType: Transport selected for a VRlizate pairing session.
WifiRttCapability: Whether the device reports Wi-Fi RTT support. Resolved natively by the host app (WifiRttManager.isAvailable on Android 9+) and passed to WifiRttPositioning.capability.
XRReferenceSpaceType
XRSessionMode

Constants

vrControllerProtocolVersion → const int: Current wire protocol understood by VRlizate remote controllers.

Functions

applyEasing(double t, EasingFunction easing) → double: Applies easing to a normalized t (0-1).
lerpDouble(double a, double b, double t) → double: Interpolates between two doubles.
lerpVector3(Vector3 a, Vector3 b, double t) → Vector3: Interpolates between two Vector3.
slerpQuaternion(Quaternion a, Quaternion b, double t) → Quaternion: Spherical linear interpolation between two quaternions.

Typedefs

OnInertialTapCallback = void Function(): Callback when an inertial tap (temple/visor tap) is detected.
VrInputClock = int Function(): Monotonic time source in microseconds.
VrInputListener = void Function(VrInputEvent event): Synchronous listener used by the allocation-free arbitration path.
VrPoseClock = int Function(): Monotonic clock used by VrRemotePosePredictor.

vrlizate library

Classes

Enums

Constants

Functions

Typedefs

vrlizate package

vrlizate library