preparePrompt method

String preparePrompt(
  1. String task, {
  2. String? question,
})

Prepare task-specific prompt tokens with special tokens added.

Adds the required special tokens for a given task to the tokenizer and returns the encoded prompt.

task: one of 'cord-v2', 'rvlcdip', 'docvqa', or a custom task name question: optional question text (for VQA tasks)

Implementation

String preparePrompt(String task, {String? question}) {
  final startToken = '<s_$task>';

  // Ensure special token exists
  tokenizer?.addSpecialTokens([startToken, '</s_$task>']);

  if (question != null) {
    tokenizer?.addSpecialTokens([
      '<s_question>',
      '</s_question>',
      '<s_answer>',
      '</s_answer>',
    ]);
    return '$startToken<s_question>$question</s_question><s_answer>';
  }

  return startToken;
}