llama_sample_classifier_free_guidance method

void llama_sample_classifier_free_guidance(
  1. Pointer<llama_context> ctx,
  2. Pointer<llama_token_data_array> candidates,
  3. Pointer<llama_context> guidance_ctx,
  4. double scale,
)

@details Apply classifier-free guidance to the logits as described in academic paper "Stay on topic with Classifier-Free Guidance" https://arxiv.org/abs/2306.17806 @param candidates A vector of llama_token_data containing the candidate tokens, the logits must be directly extracted from the original generation context without being sorted. @params guidance_ctx A separate context from the same model. Other than a negative prompt at the beginning, it should have all generated and user input tokens copied from the main context. @params scale Guidance strength. 1.0f means no guidance. Higher values mean stronger guidance.

Implementation

void llama_sample_classifier_free_guidance(
  ffi.Pointer<llama_context> ctx,
  ffi.Pointer<llama_token_data_array> candidates,
  ffi.Pointer<llama_context> guidance_ctx,
  double scale,
) {
  return _llama_sample_classifier_free_guidance(
    ctx,
    candidates,
    guidance_ctx,
    scale,
  );
}