vocallabs_agent_sdk

A Flutter SDK that captures screenshots of your app's UI and sends them to a server via WebSocket communication. This is useful for remote assistance, session recording, and agent-based user support.

Features

• Establishes a WebSocket connection to a specified server.
• Captures the screen of a specific widget tree.
• Uploads the captured image to an HTTP endpoint.
• Handles WebSocket lifecycle events (connect, message, error, disconnect).
• Shows toast notifications on mobile platforms upon successful uploads.

How it Works

1. The service connects to a WebSocket URL with a given user ID.
2. It listens for incoming messages from the server.
3. Upon receiving a message (like a "ping" or a specific command), it triggers a screen capture.
4. The captured image is sent as a multipart form data POST request to a specified upload URL.
5. A toast message is displayed on mobile devices to confirm the upload.

Installation

1. Add this to your package's pubspec.yaml file:

   dependencies:
     vocallabs_agent_sdk: ^0.0.1 # Use the latest version
   

2. Install it by running the following command in your terminal:

   flutter pub get
   

Usage

To use this SDK, you need to wrap the widget you want to capture with a Screenshot widget.

1. Wrap your widget

First, create a ScreenshotController and wrap the desired part of your widget tree (e.g., your main Scaffold) with a Screenshot widget.

import 'package:flutter/material.dart';
import 'package:screenshot/screenshot.dart';
import 'package:vocallabs_agent_sdk/vocallabs_agent_sdk.dart';

class MyApp extends StatefulWidget {
  const MyApp({super.key});

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  // 1. Create a ScreenshotController
  final ScreenshotController _screenshotController = ScreenshotController();
  late final ScreenSyncerService _screenSyncerService;

  @override
  void initState() {
    super.initState();
    // 2. Initialize the service with the controller
    _screenSyncerService = ScreenSyncerService(screenshotController: _screenshotController);
    
    // 3. Connect to the server
    // Replace with your server URL and a unique user ID
    _screenSyncerService.connect('your-server-url.com', 'user-123');
  }

  @override
  void dispose() {
    // 4. Disconnect when the widget is disposed
    _screenSyncerService.disconnect();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    // 5. Wrap the widget you want to capture with the Screenshot widget
    return Screenshot(
      controller: _screenshotController,
      child: MaterialApp(
        home: Scaffold(
          appBar: AppBar(
            title: const Text('VocalLabs Agent'),
          ),
          body: const Center(
            child: Text('This screen will be captured.'),
          ),
        ),
      ),
    );
  }
}

2. API Reference

ScreenSyncerService({required this.screenshotController})

The constructor requires a ScreenshotController that is attached to a Screenshot widget in your UI.

void connect(String url, String userId)

Connects to the WebSocket server.

• url: The base URL of your server (e.g., your-server-url.com). The service will construct the WebSocket URL as wss://<url>/ws/<userId>.
• userId: A unique identifier for the user/client.

void sendMessage(String message)

Sends a text message to the connected WebSocket server.

_screenSyncerService.sendMessage('Hello from the client!');

void disconnect()

Closes the WebSocket connection. It's important to call this in your widget's dispose method to prevent memory leaks.

Contributing

Contributions are welcome! If you find a bug or have a feature request, please open an issue on the GitHub repository.