Hugging Face generation patch¶
lightrft.models.monkey_patch.hf_generate_patch replaces Transformers’
stopping-criteria builder with a variant that omits the EOS criterion. It is an
explicit monkey patch: importing the module alone does not modify
GenerationMixin; callers must invoke
apply_monkey_patch_to_generation_mixin().
The implementation is tied to the Transformers generation API noted in the source file and should be revalidated when that dependency is upgraded.
def apply_monkey_patch_to_generation_mixin():
"""
Apply the monkey patch to replace the original _get_stopping_criteria method.
This function replaces the default _get_stopping_criteria implementation in GenerationMixin
with our modified version that excludes EOS token stopping.
:return: None
"""
GenerationMixin._get_stopping_criteria = _get_stopping_criteria_hacked