Flutter Virtual Background #

A powerful Flutter Web plugin for real-time AI-powered background removal and replacement with dual-engine support and video recording capabilities.

Features #

🚀 Dual AI Engine Support

MediaPipe: Optimized for 60FPS performance
BodyPix: Optimized for precise edge detection

🎨 Advanced Background Management

Real-time background removal
Custom background image selection
Dynamic background switching

🎥 Professional Recording

Synchronized audio/video recording
Real-time timer callbacks
WebM format export with VP8/Opus codecs

⚡ Solid-Core Pipeline

Morphological dilation for edge enhancement
WebGL-accelerated compositing
Eliminates ghosting and tearing artifacts

Core Technical Architecture #

The Hybrid Engine #

Switch between two AI segmentation models in real-time:

MediaPipe Selfie Segmentation: Lightning-fast processing for smooth 60FPS
BodyPix: Superior edge precision with advanced dilation

The Solid-Core Pipeline #

Raw Segmentation → Mask Enhancement → Polish & Blur → Composite Output

Morphological Dilation #

Aggressive edge-expansion technique that prevents background bleed-through during fast motion.

Universal Compositor #

Custom WebGL-accelerated canvas system for real-time video processing.

Installation #

Add this to your package's pubspec.yaml file:

dependencies:
  flutter_virtual_background: ^1.0.0

Then run:

flutter pub get

Usage #

Basic Setup #

import 'package:flutter_virtual_background/flutter_virtual_background.dart';

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  VirtualBackgroundController? _controller;
  bool _isRecording = false;
  String _recordingTime = "00:00";
  VirtualEngine _currentEngine = VirtualEngine.mediapipe;

  @override
  void initState() {
    super.initState();
    _initializeCamera();
  }

  Future<void> _initializeCamera() async {
    _controller = VirtualBackgroundController(
      containerId: 'my-camera-view',
      config: VirtualBackgroundConfig(
        enableAudio: true,
        initialEngine: VirtualEngine.mediapipe,
        onReady: () {
          print('Camera is ready!');
        },
        onRecordingStart: () {
          setState(() => _isRecording = true);
        },
        onRecordingStop: () {
          setState(() {
            _isRecording = false;
            _recordingTime = "00:00";
          });
        },
        onRecordingError: (error) {
          print('Recording error: $error');
        },
        onTimerUpdate: (seconds) {
          final duration = Duration(seconds: seconds);
          setState(() {
            _recordingTime = "${duration.inMinutes.toString().padLeft(2, '0')}:"
                "${(duration.inSeconds % 60).toString().padLeft(2, '0')}";
          });
        },
      ),
    );

    await _controller!.initialize();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Stack(
        children: [
          // Camera view
          VirtualBackgroundView(
            controller: _controller,
            viewId: 'my-camera-view',
          ),

          // Your custom UI
          Positioned(
            bottom: 20,
            left: 0,
            right: 0,
            child: Row(
              mainAxisAlignment: MainAxisAlignment.center,
              children: [
                // Switch Engine
                IconButton(
                  icon: Icon(Icons.psychology),
                  onPressed: () async {
                    final newEngine = _currentEngine == VirtualEngine.mediapipe
                        ? VirtualEngine.bodypix
                        : VirtualEngine.mediapipe;
                    await _controller?.switchEngine(newEngine);
                    setState(() => _currentEngine = newEngine);
                  },
                ),
                
                // Select Background
                IconButton(
                  icon: Icon(Icons.image),
                  onPressed: () => _controller?.selectBackground(),
                ),
                
                // Start/Stop Recording
                IconButton(
                  icon: Icon(_isRecording ? Icons.stop : Icons.play_arrow),
                  onPressed: () {
                    if (_isRecording) {
                      _controller?.stopRecording();
                    } else {
                      _controller?.startRecording();
                    }
                  },
                ),
                
                // Clear Background
                IconButton(
                  icon: Icon(Icons.close),
                  onPressed: () => _controller?.clearBackground(),
                ),
              ],
            ),
          ),

          // Recording Timer
          if (_isRecording)
            Positioned(
              top: 50,
              child: Container(
                padding: EdgeInsets.all(8),
                color: Colors.red,
                child: Text(_recordingTime),
              ),
            ),
        ],
      ),
    );
  }

  @override
  void dispose() {
    _controller?.dispose();
    super.dispose();
  }
}

Advanced Configuration #

VirtualBackgroundConfig(
  enableAudio: true,  // Enable/disable audio recording
  initialEngine: VirtualEngine.mediapipe,
  videoWidth: 1280,
  videoHeight: 720,
  onReady: () {
    // Camera initialized and ready
  },
  onRecordingStart: () {
    // Recording started
  },
  onRecordingStop: () {
    // Recording stopped and file downloaded
  },
  onRecordingError: (String error) {
    // Handle recording errors
  },
  onTimerUpdate: (int seconds) {
    // Update UI with recording duration
  },
)

API Reference #

VirtualBackgroundController

Methods:

Future<void> initialize() - Initialize camera and AI engines
Future<void> switchEngine(VirtualEngine engine) - Switch between MediaPipe and BodyPix
Future<void> selectBackground() - Open file picker for background image
Future<void> setBackground(String imageUrl) - Set background from URL
Future<void> clearBackground() - Remove background (show original)
Future<void> startRecording() - Start recording with audio
Future<void> stopRecording() - Stop recording and download file
void dispose() - Clean up resources

VirtualEngine

enum VirtualEngine {
  mediapipe,  // Fast, 60FPS optimized
  bodypix,    // Precise edge detection
}

VirtualBackgroundConfig

VirtualBackgroundConfig({
  bool enableAudio = true,
  VirtualEngine initialEngine = VirtualEngine.mediapipe,
  int videoWidth = 1280,
  int videoHeight = 720,
  VoidCallback? onReady,
  VoidCallback? onRecordingStart,
  VoidCallback? onRecordingStop,
  ValueChanged<String>? onRecordingError,
  ValueChanged<int>? onTimerUpdate,
})

Platform Support #

Platform	Support
Web	✅
Android	❌
iOS	❌
macOS	❌
Windows	❌
Linux	❌

This is a Web-only plugin using browser-specific APIs (MediaPipe, BodyPix, MediaRecorder).

Browser Compatibility #

✅ Chrome 90+
✅ Edge 90+
✅ Firefox 88+
✅ Safari 14+ (limited support)
✅ Opera 76+

Technical Details #

AI Models #

MediaPipe Selfie Segmentation: Google's optimized model for real-time person segmentation
BodyPix: TensorFlow.js model with MobileNetV1 architecture

Video Processing #

Real-time canvas compositing at 30-60 FPS
Morphological dilation for edge enhancement
Blur and contrast filters for smooth masking

Recording Format #

Container: WebM
Video Codec: VP8
Audio Codec: Opus
Frame Rate: 30 FPS

Example #

Check out the example directory for a complete implementation.

Screenshots #

Permission Request #

MediaPipe Engine #

BodyPix Engine #

Background Selection #

Final Output #

Contributing #

Contributions are welcome! Please feel free to submit a Pull Request.

License #

This project is licensed under the MIT License - see the LICENSE file for details.

Credits #

Developed by LogicPixel

Original Project: AI-LiveMask-Flutter-Web

Support #

For issues, questions, or contributions, please visit the GitHub repository.

flutter_virtual_background 1.0.1 flutter_virtual_background: ^1.0.1 copied to clipboard

Metadata