Fine-tuning is additional training of an already prepared language model on narrow data, for example on code examples or medical questions. Without this step, AI copes worse with tasks in a specific domain. Usually data is divided among several graphics processing units (GPUs) randomly, and different accelerators may process similar examples repeatedly. A GPU is a device that quickly performs calculations for training neural networks.
A team led by Professor Zhe-Gil Lee from the KAIST School of Computing has developed the CluSTER method. First, the algorithm assesses what contribution each data fragment makes to training and combines examples with a similar effect. The grouping is not by topics or words, but by what skills the model acquires. Then data is selected from each group and distributed evenly among the GPUs so that each compute unit receives diverse information.
The experiments involved models with approximately 7 and 13 billion parameters. Parameters are internal numerical values that are adjusted during training and determine the model's knowledge. Three datasets from programming and medicine were used, and effectiveness was checked on eight performance tests.
The models retained almost the same accuracy even when they were trained on only 31% of the full dataset, and training time was reduced to 69.6%. If training previously took 10 hours, it can now be completed in about three. The main experiments were conducted on four GPUs, but on eight accelerators as well the method showed better results than traditional approaches.
The authors emphasize that the speedup is achieved not by increasing the number of video cards, but through a smarter selection and distribution of data. This could help companies and research organizations create specialized AI assistants faster and more cheaply, for example for customer support or helping doctors find information.
The results were presented in October 2026 at the Conference on Empirical Methods in Natural Language Processing (EMNLP 2026) and published on the arXiv preprint server. The first author was doctoral student Hyunjin Kim, and the co-authors include researchers from KAIST and the University of Toronto.
A limitation of the work is that the conclusions are based on specific datasets and experimental conditions. The study does not prove that any project will automatically receive the same gain. Nevertheless, it shows that the efficiency of AI training can be improved without scaling up computing power - it is enough to take a competent approach to selecting training material.