Shortcuts

Hugging Face generation patch

lightrft.models.monkey_patch.hf_generate_patch replaces Transformers’ stopping-criteria builder with a variant that omits the EOS criterion. It is an explicit monkey patch: importing the module alone does not modify GenerationMixin; callers must invoke apply_monkey_patch_to_generation_mixin().

The implementation is tied to the Transformers generation API noted in the source file and should be revalidated when that dependency is upgraded.

def apply_monkey_patch_to_generation_mixin():
    """
    Apply the monkey patch to replace the original _get_stopping_criteria method.

    This function replaces the default _get_stopping_criteria implementation in GenerationMixin
    with our modified version that excludes EOS token stopping.

    :return: None
    """
    GenerationMixin._get_stopping_criteria = _get_stopping_criteria_hacked