search 0.2.0

A simple search engine that enables natural öanguage searches in Flutter / web applications.

Overview

This is an information retrieval engine written in Dart.

Licensed under the Apache License 2.0.

Warning: expect many breaking changes before the project freezes the APIs.

How it works

SearchableDatastore wraps any other Datastore and intercepts search requests that contain one or more KeywordFilter instances.

The current implementation then simply visits every document in the collection and calculates score for each document. This is very inefficient strategy for large collections / many concurrent requests. However, for typical mobile and web applications, this is fine!

In the preprocessing step, we simplify both keyword and:

• Replace whitespace characters with a single space.
  • "hello,\n world" --> " hello world "
• Lowercase characters and replace some extended Latin characters with ASCII characters.
  • "Élysée" --> " elysee "
• Remove some suffices
  • "Joe's coffee" --> " joe coffee "

The document scoring algorithm is a quick hack at the moment. It attempts to raise score for:

• Higher count of substring search matches.
• Substring search matches near each other.
• Presence of exact (non-processed) substring matches.

Contributing

• github.com/terrier989/datastore

Getting started

In pubspec.yaml:

dependencies:
  datastore: any
  search: any

In lib/main.dart:

import 'package:datastore/datastore.dart';
import 'package:search/search.dart';

void main() {
  final datastore = SearchableDatastore(
    datastore: MemoryDatastore(),
  );
  final collection = datastore.collection('employee');
  final result = await collection.search(
    query: Query.parse(
      '(Hello OR Hi) world!',
      skip: 0,
      take: 10,
    ),
  );
}