add method

dynamic add(Map<String, dynamic> doc, [ Map<String, dynamic>? attributes ])

Adds a document to the index.

Before adding fields to the index the index should have been fully setup, with the document ref and all fields to index already having been specified.

The document must have a field name as specified by the ref (by default this is 'id') and it should have all fields defined for indexing, though null or undefined values will not cause errors.

Entire documents can be boosted at build time. Applying a boost to a document indicates that this document should rank higher in search results than other documents.

doc - The document to add to the index. attributes - Optional attributes associated with this document. attributes.boost=1 - Boost applied to all terms within this document.

Implementation

add(Map<String, dynamic> doc, [Map<String, dynamic>? attributes]) {
  String docRef = doc[ref]!;
  Iterable<String> fields = _fields.keys;

  _documents[docRef] = attributes ?? {};
  documentCount += 1;

  for (String fieldName in fields) {
    var extractor = _fields[fieldName]!['extractor'];
    String field =
        extractor is Function ? extractor(doc) : '${doc[fieldName]}';
    List<Token> tokens = tokenizer(field, {
      'fields': [fieldName]
    });
    List<Token?> terms = pipeline.run(tokens);

    FieldRef fieldRef = FieldRef(docRef, fieldName);
    TermFrequencies fieldTerms = {};
    fieldTermFrequencies[fieldRef] = fieldTerms;

    fieldLengths[fieldRef] = 0;

    // store the length of this field for this document
    fieldLengths[fieldRef] = fieldLengths[fieldRef]! + terms.length;

    // calculate term frequencies for this field
    for (Token? term in terms) {
      if (term == null) continue;
      if (fieldTerms[term] == null) {
        fieldTerms[term] = 0;
      }

      fieldTerms[term] = fieldTerms[term]! + 1;

      // add to inverted index
      // create an initial posting if one doesn't exist
      if (invertedIndex[term] == null) {
        Posting posting = Posting();
        posting.index = termIndex;
        termIndex += 1;

        for (String field in fields) {
          posting[field] = {};
        }

        invertedIndex[term] = posting;
      }

      // add an entry for this term/fieldName/docRef to the invertedIndex
      if (invertedIndex[term]![fieldName][docRef] == null) {
        invertedIndex[term]![fieldName][docRef] = {};
      }

      // store all whitelisted metadata about this token in the
      // inverted index
      for (String metadataKey in metadataWhitelist) {
        dynamic metadata = term.metadata![metadataKey];

        if (invertedIndex[term]![fieldName]?[docRef]?[metadataKey] == null) {
          invertedIndex[term]![fieldName][docRef][metadataKey] = [];
        }

        invertedIndex[term]![fieldName][docRef][metadataKey].add(metadata);
      }
    }
  }
}