core/nn/sentence/cross_encoder library
Cross-encoder reranker.
Mirrors the CrossEncoder class from
github.com/huggingface/sentence-transformers: it takes a
single joint tokenization of a (query, passage) pair and produces a
scalar relevance score. Slower than a bi-encoder SentenceEncoder
because the transformer sees both sides at once, but much more
accurate for reranking a shortlist retrieved by a bi-encoder.
Input tokens: the caller is responsible for concatenating the
query and passage token ids into one 1D [seqLen] sequence,
usually as [CLS] q0 q1 ... [SEP] d0 d1 ... [SEP].
Output: a [1, numLabels] logits tensor (defaults to numLabels = 1 for a single relevance score). For binary classification use
score() which passes the logit through a sigmoid.