ACTIVE LEARNING FOR INTENT CLASSIFICATION IN IN-CONTEXT LEARNING OF LARGE LANGUAGE MODELS

Dranga Daniil I.

In modern natural language processing, large language models have demonstrated high effectiveness across various tasks, including intent classification in dialogue systems. However, selecting the optimal set of examples for contextual learning remains challenging, especially with a limited annotation budget. This paper proposes an adapted active learning algorithm for the efficient selection of examples in contextual few-shot learning of large language models for intent classification tasks. The proposed method takes model uncertainty into account and ensures diversity in the selected examples, thereby enhancing classification accuracy. Experimental studies on three different datasets have shown that the developed approach outperforms alternative methods, providing higher classification quality for both closed models (e.g., GPT-4) and open models that perform well in Russian (e.g., Gemma 2 27b).

Keywords:active learning, contextual learning, large language models, intent classification, few-shot learning, uncertainty estimation