Instruction-based fine-tuning of open-source llms forpredictıng customer purchase behaviors

In this study, the performance of various predictive models, including probabilistic baseline,CNN, LSTM, and fine-tuned LLMs, in forecasting merchant categories from financialtransaction data have been evaluated. Utilizing datasets from Bank A for training andBank B for testing, the superior predictive capabilities of the fine-tuned Mistral Instructmodel, which was trained using customer data converted into natural language formathave been demonstrated. The methodology of this study involves instruction fine-tuningMistral via LoRA (Low-Rank Adaptation of Large Language Models) to adapt its vastpre-trained knowledge to the specific domain of financial transactions. The Mistral modelsignificantly outperforms traditional sequential models, achieving higher F1 scores in thethree key merchant categories of bank transaction data—grocery, clothing, and gas stations—that is crucial for targeted marketing campaigns. This performance is attributedto the model’s enhanced semantic understanding and adaptability which enables it tobetter manage minority classes and predict transaction categories with greater accuracy.These findings highlight the potential of LLMs in predicting human behavior and revolutionizingfinancial decision-making processes.