MODELS OF RESOURCE-EFFICIENT DISTILLATION OF TRANSFORMERS FOR PRESERVING CRITICAL KNOWLEDGE IN NER TASKS

Chudnov Ivan Ilyich, Romashkova Oxana Nikolaevna

The article discusses the development and experimental verification of resource- and energy-efficient distillation methods for transformer models in the task of named entity recognition (NER), with an emphasis on preserving verifiable and application-relevant knowledge. The proposed approach combines engineering distillation techniques, targeted evaluation by critical entity classes, and systematic evaluation of model resource characteristics. The work is focused on practical reproducibility: all experimental protocols are formalised and implemented as a reproducible software pipeline.

Keywords:knowledge distillation, transformers, Named Entity Recognition, resource efficiency, critical facts, verifiable knowledge.