Flutter Virtual Background

A powerful Flutter Web plugin for real-time AI-powered background removal and replacement with dual-engine support and video recording capabilities.

Features

🚀 Dual AI Engine Support

• MediaPipe: Optimized for 60FPS performance
• BodyPix: Optimized for precise edge detection

🎨 Advanced Background Management

• Real-time background removal
• Custom background image selection
• Dynamic background switching

🎥 Professional Recording

• Synchronized audio/video recording
• Real-time timer callbacks
• WebM format export with VP8/Opus codecs

⚡ Solid-Core Pipeline

• Morphological dilation for edge enhancement
• WebGL-accelerated compositing
• Eliminates ghosting and tearing artifacts

Core Technical Architecture

The Hybrid Engine

Switch between two AI segmentation models in real-time:

• MediaPipe Selfie Segmentation: Lightning-fast processing for smooth 60FPS
• BodyPix: Superior edge precision with advanced dilation

The Solid-Core Pipeline

Raw Segmentation → Mask Enhancement → Polish & Blur → Composite Output

Morphological Dilation

Aggressive edge-expansion technique that prevents background bleed-through during fast motion.

Universal Compositor

Custom WebGL-accelerated canvas system for real-time video processing.

Installation

Add this to your package's pubspec.yaml file:

dependencies:
  flutter_virtual_background: ^1.0.0

Then run:

flutter pub get

Usage

Basic Setup

import 'package:flutter_virtual_background/flutter_virtual_background.dart';

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  VirtualBackgroundController? _controller;
  bool _isRecording = false;
  String _recordingTime = "00:00";
  VirtualEngine _currentEngine = VirtualEngine.mediapipe;

  @override
  void initState() {
    super.initState();
    _initializeCamera();
  }

  Future<void> _initializeCamera() async {
    _controller = VirtualBackgroundController(
      containerId: 'my-camera-view',
      config: VirtualBackgroundConfig(
        enableAudio: true,
        initialEngine: VirtualEngine.mediapipe,
        onReady: () {
          print('Camera is ready!');
        },
        onRecordingStart: () {
          setState(() => _isRecording = true);
        },
        onRecordingStop: () {
          setState(() {
            _isRecording = false;
            _recordingTime = "00:00";
          });
        },
        onRecordingError: (error) {
          print('Recording error: $error');
        },
        onTimerUpdate: (seconds) {
          final duration = Duration(seconds: seconds);
          setState(() {
            _recordingTime = "${duration.inMinutes.toString().padLeft(2, '0')}:"
                "${(duration.inSeconds % 60).toString().padLeft(2, '0')}";
          });
        },
      ),
    );

    await _controller!.initialize();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Stack(
        children: [
          // Camera view
          VirtualBackgroundView(
            controller: _controller,
            viewId: 'my-camera-view',
          ),

          // Your custom UI
          Positioned(
            bottom: 20,
            left: 0,
            right: 0,
            child: Row(
              mainAxisAlignment: MainAxisAlignment.center,
              children: [
                // Switch Engine
                IconButton(
                  icon: Icon(Icons.psychology),
                  onPressed: () async {
                    final newEngine = _currentEngine == VirtualEngine.mediapipe
                        ? VirtualEngine.bodypix
                        : VirtualEngine.mediapipe;
                    await _controller?.switchEngine(newEngine);
                    setState(() => _currentEngine = newEngine);
                  },
                ),
                
                // Select Background
                IconButton(
                  icon: Icon(Icons.image),
                  onPressed: () => _controller?.selectBackground(),
                ),
                
                // Start/Stop Recording
                IconButton(
                  icon: Icon(_isRecording ? Icons.stop : Icons.play_arrow),
                  onPressed: () {
                    if (_isRecording) {
                      _controller?.stopRecording();
                    } else {
                      _controller?.startRecording();
                    }
                  },
                ),
                
                // Clear Background
                IconButton(
                  icon: Icon(Icons.close),
                  onPressed: () => _controller?.clearBackground(),
                ),
              ],
            ),
          ),

          // Recording Timer
          if (_isRecording)
            Positioned(
              top: 50,
              child: Container(
                padding: EdgeInsets.all(8),
                color: Colors.red,
                child: Text(_recordingTime),
              ),
            ),
        ],
      ),
    );
  }

  @override
  void dispose() {
    _controller?.dispose();
    super.dispose();
  }
}

Advanced Configuration

VirtualBackgroundConfig(
  enableAudio: true,  // Enable/disable audio recording
  initialEngine: VirtualEngine.mediapipe,
  videoWidth: 1280,
  videoHeight: 720,
  onReady: () {
    // Camera initialized and ready
  },
  onRecordingStart: () {
    // Recording started
  },
  onRecordingStop: () {
    // Recording stopped and file downloaded
  },
  onRecordingError: (String error) {
    // Handle recording errors
  },
  onTimerUpdate: (int seconds) {
    // Update UI with recording duration
  },
)

API Reference

VirtualBackgroundController

Methods:

• Future<void> initialize() - Initialize camera and AI engines
• Future<void> switchEngine(VirtualEngine engine) - Switch between MediaPipe and BodyPix
• Future<void> selectBackground() - Open file picker for background image
• Future<void> setBackground(String imageUrl) - Set background from URL
• Future<void> clearBackground() - Remove background (show original)
• Future<void> startRecording() - Start recording with audio
• Future<void> stopRecording() - Stop recording and download file
• void dispose() - Clean up resources

VirtualEngine

enum VirtualEngine {
  mediapipe,  // Fast, 60FPS optimized
  bodypix,    // Precise edge detection
}

VirtualBackgroundConfig

VirtualBackgroundConfig({
  bool enableAudio = true,
  VirtualEngine initialEngine = VirtualEngine.mediapipe,
  int videoWidth = 1280,
  int videoHeight = 720,
  VoidCallback? onReady,
  VoidCallback? onRecordingStart,
  VoidCallback? onRecordingStop,
  ValueChanged<String>? onRecordingError,
  ValueChanged<int>? onTimerUpdate,
})

Platform Support

Platform	Support
Web	✅
Android	❌
iOS	❌
macOS	❌
Windows	❌
Linux	❌

This is a Web-only plugin using browser-specific APIs (MediaPipe, BodyPix, MediaRecorder).

Browser Compatibility

• ✅ Chrome 90+
• ✅ Edge 90+
• ✅ Firefox 88+
• ✅ Safari 14+ (limited support)
• ✅ Opera 76+

Technical Details

AI Models

• MediaPipe Selfie Segmentation: Google's optimized model for real-time person segmentation
• BodyPix: TensorFlow.js model with MobileNetV1 architecture

Video Processing

• Real-time canvas compositing at 30-60 FPS
• Morphological dilation for edge enhancement
• Blur and contrast filters for smooth masking

Recording Format

• Container: WebM
• Video Codec: VP8
• Audio Codec: Opus
• Frame Rate: 30 FPS

Example

Check out the example directory for a complete implementation.

Screenshots

Permission Request

MediaPipe Engine

BodyPix Engine

Background Selection

Final Output

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Credits

Developed by LogicPixel

Original Project: AI-LiveMask-Flutter-Web

Support

For issues, questions, or contributions, please visit the GitHub repository.