Task-oriented optimization of pre-trained language models with task projection network

Official URL: https://dx.doi.org/10.1109/SIU66497.2025.11112512

Large-scale artificial learning models are trained with large data sets and made suitable for general use. However, adapting these models to a specific topic or specific task often involves updating all the weights. This process results in high memory consumption and increased computational costs. In this study, instead of fine-tuning an entire model, we consider the method of customizing it with a task projection network added in front of it. This approach preserves the general structure of the model, while allowing it to be used directly for a specific task without retraining. Thus, computational and storage needs reduced and effective adaptations can be made with small data sets. The proposed method facilitates the rapid adaptation of large models to different areas, enabling the spread of artificial learning to wider areas of use. In our work, we demonstrate how the task projection network can be used effectively without the need to change a primary model and the practical benefits of this method.