cloudproof 3.0.1

Cloudproof Encryption provides libraries and tools to encrypt large repositories of data - Big Data - with high performance and advanced secure techniques in zero trust environments. It also provides [...]

Cloudproof Flutter Library

The Cloudproof Flutter library provides a Flutter-friendly API to the Cosmian Cloudproof Encryption product.

In summary, Cloudproof Encryption product secures data repositories in the cloud with attributes-based access control encryption and encrypted search.

• Getting started
  • CoverCrypt
  • Findex
• Installation
• Example
• Tests
  • WARNINGS
  • Implementation details
• FFI libs notes
  • Generating .h
    • iOS WARNING
  • Building .so, .a…
    • Linux
    • Android
    • iOS
  • Supported versions
• Cloudproof versions Correspondence

Getting started

CoverCrypt

CoverCrypt allows to decrypt data previously encrypted with one of our libraries (Java, Python, Rust…).

Two classes are available: CoverCryptDecryption and CoverCryptDecryptionWithCache which is a little bit faster (omit the initialization phase during decryption). See test/covercrypt_test.dart.

Findex

Findex allows to do encrypted search queries on an encrypted index. To use Findex you need a driver which is able to store and update indexes (it could be SQLite, Redis, or any other storage method). You can find in test/findex_redis_test.dart and test/findex_sqlite_test.dart two example of implementation.

To search, you need:

1. copy/paste the following lines
2. replace TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction by the name of your class
3. implement fetchEntries and fetchChains

  static Future<List<IndexRow>> fetchEntries(Uids uids) async {
    // Implement me!
  }

  static Future<List<IndexRow>> fetchChains(Uids uids) async {
    // Implement me!
  }

  // --------------------------------------------------
  // Copy-paste code :AutoGeneratedImplementation
  // --------------------------------------------------

  static Future<List<IndexedValue>> search(
    Uint8List keyK,
    Uint8List label,
    List<Keyword> words,
  ) async {
    return await Findex.search(
      keyK,
      label,
      words,
      Pointer.fromFunction(
        fetchEntriesCallback,
        errorCodeInCaseOfCallbackException,
      ),
      Pointer.fromFunction(
        fetchChainsCallback,
        errorCodeInCaseOfCallbackException,
      ),
    );
  }

  static int fetchEntriesCallback(
    Pointer<Char> outputEntryTableLinesPointer,
    Pointer<UnsignedInt> outputEntryTableLinesLength,
    Pointer<UnsignedChar> uidsPointer,
    int uidsNumber,
  ) {
    try {
      final uids =
          Uids.deserialize(uidsPointer.cast<Uint8>().asTypedList(uidsNumber));
      final entryTableLines = SqliteFindex.fetchEntries(uids);
      IndexRow.serialize(outputEntryTableLinesPointer.cast<UnsignedChar>(),
          outputEntryTableLinesLength, entryTableLines);
      return 0;
    } catch (e, stacktrace) {
      log("Exception during fetchEntriesCallback $e $stacktrace");
      rethrow;
    }
  }

  static int fetchChainsCallback(
    Pointer<Char> outputChainTableLinesPointer,
    Pointer<UnsignedInt> outputChainTableLinesLength,
    Pointer<UnsignedChar> uidsPointer,
    int uidsNumber,
  ) {
    try {
      final uids =
          Uids.deserialize(uidsPointer.cast<Uint8>().asTypedList(uidsNumber));
      final entryTableLines = SqliteFindex.fetchChains(uids);
      IndexRow.serialize(outputChainTableLinesPointer.cast<UnsignedChar>(),
          outputChainTableLinesLength, entryTableLines);
      return 0;
    } catch (e, stacktrace) {
      log("Exception during fetchChainsCallback $e $stacktrace");
      rethrow;
    }
  }

To upsert, you need:

1. copy/paste the following lines
2. replace TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction by the name of your class
3. implement fetchEntries, upsertEntries and upsertChains

  static Future<List<IndexRow>> upsertEntries(List<UpsertData> entries) async {
    // Implement me!
  }

  static Future<List<IndexRow>> upsertChains(List<UpsertData> entries) async {
    // Implement me!
  }

  // --------------------------------------------------
  // Copy-paste code :AutoGeneratedImplementation
  // --------------------------------------------------

  static Future<void> upsert(
    MasterKey masterKey,
    Uint8List label,
    Map<IndexedValue, List<Word>> indexedValuesAndWords,
  ) async {
    await Findex.upsert(
      masterKey,
      label,
      indexedValuesAndWords,
      Pointer.fromFunction(
        fetchEntriesCallback,
        errorCodeInCaseOfCallbackException,
      ),
      Pointer.fromFunction(
        upsertEntriesCallback,
        errorCodeInCaseOfCallbackException,
      ),
      Pointer.fromFunction(
        upsertChainsCallback,
        errorCodeInCaseOfCallbackException,
      ),
    );
  }

  static void upsertEntriesCallback(
    Pointer<UnsignedChar> entriesListPointer,
    int entriesListLength,
    Pointer<UnsignedChar> outputRejectedEntriesListPointer,
    Pointer<UnsignedInt> outputRejectedEntriesListLength,
  ) {
    try {
      // Deserialize uids and values
      final uidsAndValues = UpsertData.deserialize(
          entriesListPointer.cast<Uint8>().asTypedList(entriesListLength));

      final rejectedEntries = SqliteFindex.upsertEntries(uidsAndValues);
      IndexRow.serialize(outputRejectedEntriesListPointer,
          outputRejectedEntriesListLength, rejectedEntries);
    } catch (e, stacktrace) {
      log("Exception during upsertEntriesCallback $e $stacktrace");
      rethrow;
    }
  }

  static void upsertChainsCallback(
    Pointer<UnsignedChar> chainsListPointer,
    int chainsListLength,
  ) {
    try {
      final uidsAndValues = IndexRow.deserialize(
          chainsListPointer.cast<Uint8>().asTypedList(chainsListLength));
      log("upsertWrapperWithoutIsolate: uidsAndValues: $uidsAndValues");

      SqliteFindex.upsertChains(uidsAndValues);
    } catch (e, stacktrace) {
      log("Exception during upsertChainsCallback $e $stacktrace");
      rethrow;
    }
  }

Note that if you search and upsert, the two implementation can share the same callback for fetchEntries.

Note that the copy/paste code could be removed in a future version when Dart implements https://github.com/dart-lang/language/issues/1482.

Installation

flutter pub get cloudproof

Example

To run the example, you need a Redis server configured. Then, update redisHost and redisPort at the top of the example/lib/findex_redis_implementation.dart file.

Tests

To run all tests:

flutter test

Some tests require a Redis database on localhost (default port).

If you ran the Java test which populate the Redis database, you can run the hidden test that read from this database.

RUN_JAVA_E2E_TESTS=1 flutter test --plain-name 'Search and decrypt with preallocate Redis by Java'

If you share the same Redis database between Java and Dart tests, flutter test will cleanup the Redis database (it could take some time and timeout on the first execution). So you may want to re-run mvn test to populate the Redis database again.

You can run the benchmarks with:

dart benchmark/cloudproof_benchmark.dart

WARNINGS

• fetchEntries, fetchChains, upsertEntries and upsertChains can not be static methods in a class or raw functions but should be static! You cannot put classic methods of an instance here.
• fetchEntries, fetchChains, upsertEntries and upsertChains cannot access the state of the program, they will run in a separate Isolate with no data from the main thread (for example static/global variables populated during an initialization phase of your program will not exist). If you need to access some data from the main thread, the only way we think we'll work is to save this information inside a file or a database and read it from the callback. This pattern will slow down the search process. If you don't need async in the callbacks (for example the sqlite library has sync functions, you can call *WrapperWithoutIsolate and keep all the process in the same thread, so you can use your global variables).

Implementation details

• The search and upsert methods will call the Rust FFI via native bindings synchronously. If you want to not stop your main thread, please call compute to run the search in a different Isolate.
• The Findex.fetchWrapper and Findex.upsertWrapper will wrap your callback inside an isolate to allow you to use asynchronous callbacks.
• The Findex.fetchWrapperWithoutIsolate and Findex.upsertWrapperWithoutIsolate will wrap just call your callback so you will not have access to async but your callbacks are executed inside the main isolate (so you have access to your global data)

FFI libs notes

This project has been first created via:

flutter create --org com.example --template=plugin --platforms=android,ios -a kotlin cloudproof

Generating .h

The lib/src/*/generated_bindings.dart are generated with ffigen with the config file ./ffigen_*.yml:

flutter pub run ffigen --config ffigen_cover_crypt.yaml
flutter pub run ffigen --config ffigen_findex.yaml

iOS WARNING

Use cbindgen, do not forget to remove str type in libcosmian_cover_crypt.h (last two lines) for iOS to compile (type str unknown in C headers).

The two .h need to be inside the ios/Classes folder. Android doesn't need .h files.

Building .so, .a…

Linux

Just copy .so file from the Rust projects to the resources folder. These .so are only useful to run the tests on Linux.

Android

Download artifacts from the Gitlab CI. You should get a jniLibs folder to copy to android/src/main.

Then:

cd example
flutter pub get
flutter run

iOS

If building with cargo lipo on Linux we only get aarch64-apple-ios and x86_64-apple-ios.

On codemagic.io:

• aarch64-apple-ios is failing with "ld: in /Users/builder/clone/ios/libcosmian_cover_crypt.a(cover_crypt.cover_crypt.aea4b2d2-cgu.0.rcgu.o), building for iOS Simulator, but linking in object file built for iOS, file '/Users/builder/clone/ios/libcosmian_cover_crypt.a' for architecture arm64"
• x86_64-apple-ios is failing with "ld: warning: ignoring file /Users/builder/clone/ios/libcosmian_cover_crypt.a, building for iOS Simulator-arm64 but attempting to link with file built for iOS Simulator-x86_64"

To make the flutter build succeed, 3 prerequisites are needed:

• declaring headers (CoverCrypt and Findex) in CloudproofPlugin.h (concat both headers)
• call artificially 1 function of each native library in SwiftCloudproofPlugin.swift
• use universal ios build: copy both .a in cloudproof_flutter/ios

Supported versions

Linux	Flutter	Dart	Android SDK	NDK	Glibc	LLVM	Smartphone Virtual Device
Ubuntu 22.04	3.3.4	2.18.2	Chipmunk 2021.2.1	r25	2.35	14.0.0-1	Pixel 5 API 30
Centos 7	3.3.4	2.18.2	Chipmunk 2021.2.1	r25	2.17	-	-

Mac	Flutter	Dart	OS	LLVM	Xcode	Smartphone Virtual Device
Catalina	3.3.4	2.18.2	Catalina	12.0.0		iPhone 12 PRO MAX

Cloudproof versions Correspondence

When using local encryption and decryption with CoverCrypt native libraries are required.

Check the main pages of the respective projects to build the native libraries appropriate for your systems. The test directory provides pre-built libraries for Linux GLIBC 2.17. These libraries should run fine on a system with a more recent GLIBC version.

This table shows the minimum versions correspondences between the various components

Flutter Lib	CoverCrypt lib	Findex
0.1.0	6.0.5	0.7.2
1.0.0	6.0.5	0.7.2
2.0.0	7.1.0	0.10.0
3.0.0	8.0.0	0.12.0