cloudproof 8.1.0

Cloudproof Encryption provides libraries and tools to encrypt large repositories of data - Big Data - with high performance and advanced secure techniques in zero trust environments. It also provides [...]

Cloudproof Flutter Library

The Cloudproof Flutter library provides a Flutter-friendly API to the Cosmian Cloudproof Encryption product.

In summary, Cloudproof Encryption product secures data repositories in the cloud with attributes-based access control encryption and encrypted search.

• Licensing
• Cryptographic primitives
  • Cosmian libraries functionalities overview
• Getting started
  • CoverCrypt
  • Findex
• Installation
  • Download required native libraries
• Example
• Tests
  • WARNINGS
  • Implementation details
• FFI libs notes
  • Generating Dart bindings from cloudproof.h
    • iOS WARNING
  • Building .so, .a…
    • Linux
    • Android
    • iOS
  • Supported versions
    • Downgrade Flutter version with snap installation
• Cloudproof versions Correspondence

Licensing

The library is available under a dual licensing scheme Affero GPL/v3 and commercial. See LICENSE.md for details.

Cryptographic primitives

The library is based on:

• CoverCrypt algorithm which allows creating ciphertexts for a set of attributes and issuing user keys with access policies over these attributes. CoverCrypt offers Post-Quantum resistance.

• Findex which is a cryptographic protocol designed to securely make search queries on an untrusted cloud server. Thanks to its encrypted indexes, large databases can securely be outsourced without compromising usability.

• FPE provides Format Preserving Encryption techniques for use in a zero-trust environment. These techniques are based on FPE-FF1 which is described in NIST:800-38G.

• Anonymization: Data anonymization is the process of transforming data in such a way that it can no longer be used to identify individuals without the use of additional information. This is often done to protect the privacy of individuals whose data is being collected or processed.

• KMS: The repository offers an implementation of the Key Management Services (server + client) handling the life cycle of all the symmetric and asymmetric keys.

Cosmian libraries functionalities overview

The following table presents the current support of the Cosmian cryptographic primitives:

	CoverCrypt	Findex	KMS client	FPE	Anonymization
cloudproof_rust	✅	✅	❌	✅	✅
cloudproof_java	✅	✅	✅	❌	❌
cloudproof_python	✅	✅	✅	✅	✅
cloudproof_js	✅	✅	✅	✅	✅
cloudproof_flutter	✅	✅	❌	❌	❌

Getting started

CoverCrypt

CoverCrypt allows to decrypt data previously encrypted with one of our libraries (Java, Python, Rust…).

To get an example of how to use CoverCrypt, take a look at test/cover_crypt/cover_crypt_test.dart.

Findex

Findex allows to do encrypted search queries on an encrypted index. To use Findex you need a driver which is able to store and update indexes (it could be SQLite, Redis, or any other storage method). You can find in test/findex_redis_test.dart and test/findex_sqlite_test.dart two example of implementation.

To search, you need:

1. copy/paste the following lines
2. replace TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction by the name of your class
3. implement fetchEntries and fetchChains

  static List<UidAndValue> fetchEntries(Uids uids) async {
    // Implement me!
  }

  static List<UidAndValue> fetchChains(Uids uids) async {
    // Implement me!
  }

  // --------------------------------------------------
  // Copy-paste code :AutoGeneratedImplementation
  // --------------------------------------------------
    static Future<Map<Keyword, Set<Location>>> search(Set<Keyword> keywords,
      {int findexHandle = -1}) async {
    return await Findex.search(keywords, findexHandle: findexHandle);
  }

  static int fetchEntriesCallback(
    Pointer<Uint8> outputEntryTableLinesPointer,
    Pointer<Uint32> outputEntryTableLinesLength,
    Pointer<Uint8> uidsPointer,
    int uidsNumber,
  ) {
    return Findex.wrapSyncFetchCallback(
      TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction.fetchEntries,
      outputEntryTableLinesPointer,
      outputEntryTableLinesLength,
      uidsPointer,
      uidsNumber,
    );
  }

  static int fetchChainsCallback(
    Pointer<Uint8> outputChainTableLinesPointer,
    Pointer<Uint32> outputChainTableLinesLength,
    Pointer<Uint8> uidsPointer,
    int uidsNumber,
  ) {
    return Findex.wrapSyncFetchCallback(
      TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction.fetchChains,
      outputChainTableLinesPointer,
      outputChainTableLinesLength,
      uidsPointer,
      uidsNumber,
    );
  }

To upsert, you need:

1. copy/paste the following lines
2. replace TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction by the name of your class
3. implement fetchEntries, upsertEntries and insertChains

  static List<UidAndValue> fetchEntries(Uids uids) async {
    // Implement me!
  }

  static List<UidAndValue> upsertEntries(UpsertData entries) {
    // Implement me!
  }

  static void insertChains(List<UidAndValue> chains) {
    // Implement me!
  }

  // --------------------------------------------------
  // Copy-paste code :AutoGeneratedImplementation
  // --------------------------------------------------

  static Future<Set<Keyword>> add(Map<IndexedValue, Set<Keyword>> additions,
      {int findexHandle = -1}) async {
    return Findex.add(additions, findexHandle: findexHandle);
  }

  static int upsertEntriesCallback(
    Pointer<Uint8> outputRejectedEntriesListPointer,
    Pointer<Uint32> outputRejectedEntriesListLength,
    Pointer<Uint8> oldValuesPointer,
    int oldValuesLength,
    Pointer<Uint8> newValuesPointer,
    int newValuesLength,
  ) {
    return Findex.wrapSyncUpsertEntriesCallback(
      TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction.upsertEntries,
      outputRejectedEntriesListPointer,
      outputRejectedEntriesListLength,
      oldValuesPointer,
      oldValuesLength,
      newValuesPointer,
      newValuesLength,
    );
  }

  static int insertChainsCallback(
    Pointer<Uint8> chainsListPointer,
    int chainsListLength,
  ) {
    return Findex.wrapSyncInsertChainsCallback(
      TODO_ReplaceThisByTheNameOfYourClassOrTheRawFunction.insertChains,
      chainsListPointer,
      chainsListLength,
    );
  }

Note that if you search and add, the two implementation can share the same callback for fetchEntries.

Note that if your callbacks are async, you can use Findex.wrapAsyncFetchCallback, wrapAsyncUpsertEntriesCallback and wrapAsyncInsertChainsCallback.

Note that the copy/paste code could be removed in a future version when Dart implements https://github.com/dart-lang/language/issues/1482.

Installation

flutter pub get cloudproof

Download required native libraries

The Cloudproof Flutter lib uses FFI to access functions of the following native cryptographic libraries:

• CoverCrypt
• Findex

Those libraries must be found in specific subfolders for Android, iOS and PC architectures.

• Android: in android/src/main/jniLibs/ subfolders:
  • arm64-v8a
  • armeabi-v7a
  • x86
  • x86_64
• iOS: in ios/ folder
• PC: in resources/ folder

To download them, please run the following script that will fetch the releases in the public URL package.cosmian.com: :

flutter pub get
python3 scripts/get_native_libraries.py

Otherwise, to build those libraries manually, please check the CoverCrypt and Findex projects on Github: their build directory contains instructions on how to build the native libraries for your system.

Example

To run the example, you need a Redis server configured. Then, update redisHost and redisPort at the top of the example/lib/findex_redis_implementation.dart file.

Tests

To run all tests:

flutter test

Some tests require a Redis database on localhost (default port).

You can run the benchmarks with:

dart benchmark/cloudproof_benchmark.dart

WARNINGS

• fetchEntries, fetchChains, upsertEntries and insertChains can not be static methods in a class or raw functions but should be static! You cannot put classic methods of an instance here.
• fetchEntries, fetchChains, upsertEntries and insertChains (if async) cannot access the state of the program, they will run in a separate Isolate with no data from the main thread (for example static/global variables populated during an initialization phase of your program will not exist). If you need to access some data from the main thread, the only way we think we'll work is to save this information inside a file or a database and read it from the callback. This pattern will slow down the search process. If you don't need async in the callbacks (for example the sqlite library has sync functions, you can call *WrapperWithoutIsolate and keep all the process in the same thread, so you can use your global variables).

Implementation details

• The search and upsert methods will call the Rust FFI via native bindings synchronously. If you want to not stop your main thread, please call compute to run the search in a different Isolate.

FFI libs notes

This project has been first created via:

flutter create --org com.example --template=plugin --platforms=android,ios -a kotlin cloudproof

Generating Dart bindings from cloudproof.h

The lib/src/generated_bindings.dart are generated with ffigen with the config file ./ffigen_*.yml:

flutter pub run ffigen --config ffigen_cloudproof.yaml

iOS WARNING

Use cbindgen, do not forget to remove str type in libcloudproof_cover_crypt.h (last two lines) for iOS to compile (type str unknown in C headers).

The two .h need to be inside the ios/Classes folder. Android doesn't need .h files.

Building .so, .a…

Linux

Just copy .so file from the Rust projects to the resources folder. These .so are only useful to run the tests on Linux.

Android

Download artifacts from the Gitlab CI. You should get a jniLibs folder to copy to android/src/main.

Then:

cd example
flutter pub get
flutter run

iOS

If building with cargo lipo on Linux we only get aarch64-apple-ios and x86_64-apple-ios.

On codemagic.io:

• aarch64-apple-ios is failing with "ld: in /Users/builder/clone/ios/libcloudproof_cover_crypt.a(cover_crypt.cover_crypt.aea4b2d2-cgu.0.rcgu.o), building for iOS Simulator, but linking in object file built for iOS, file '/Users/builder/clone/ios/libcloudproof_cover_crypt.a' for architecture arm64"
• x86_64-apple-ios is failing with "ld: warning: ignoring file /Users/builder/clone/ios/libcloudproof_cover_crypt.a, building for iOS Simulator-arm64 but attempting to link with file built for iOS Simulator-x86_64"

To make the flutter build succeed, 3 prerequisites are needed:

• declaring headers (CoverCrypt and Findex) in CloudproofPlugin.h (concat both headers)
• call artificially 1 function of each native library in SwiftCloudproofPlugin.swift
• use universal ios build: copy both .a in cloudproof_flutter/ios

Supported versions

Linux	Flutter	Dart	Android SDK	NDK	Glibc	LLVM	Smartphone Virtual Device
Ubuntu 22.04	3.3.4	2.18.2	Chipmunk 2021.2.1	r25	2.35	14.0.0-1	Pixel 5 API 30
Centos 7	3.3.4	2.18.2	Chipmunk 2021.2.1	r25	2.17	-	-

Mac	Flutter	Dart	OS	LLVM	Xcode	Smartphone Virtual Device
Catalina	3.3.4	2.18.2	Catalina	12.0.0		iPhone 12 PRO MAX

Downgrade Flutter version with snap installation

cd ~/snap/flutter/common/flutter/
git checkout 3.3.4

Cloudproof versions Correspondence

When using local encryption and decryption with CoverCrypt native libraries are required.

Check the main pages of the respective projects to build the native libraries appropriate for your systems. The resources directory provides pre-built libraries for Linux GLIBC 2.17. These libraries should run fine on a system with a more recent GLIBC version.

This table shows the minimum versions correspondences between the various components

Flutter Lib	CoverCrypt lib	Findex
0.1.0	6.0.5	0.7.2
1.0.0	6.0.5	0.7.2
2.0.0	7.1.0	0.10.0
3.0.0	8.0.0	0.12.0
4.0.0	8.0.0	1.0.1
4.0.1,4.0.2,4.0.3	8.0.0,8.0.1,8.0.2	2.0.0
5.0.0	10.0.0	2.0.1
5.1.0	10.0.0	2.1.0

From the version 6.0.0, cloudproof_flutter depends on cloudproof_rust which wraps the interfaces of CoverCrypt and Findex.

Flutter Lib	Cloudproof Rust lib
6.0.0	1.0.0
6.0.2	1.0.1
7.0.0	2.0.1
8.0.0	2.4.0
8.1.0	3.0.0