Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflows
By: Orlando Marquez Ayala , Patrice Bechard , Emily Chen and more
Potential Business Impact:
Small AI models make specific tasks better than big ones.
Large Language Models (LLMs) such as GPT-4o can handle a wide range of complex tasks with the right prompt. As per token costs are reduced, the advantages of fine-tuning Small Language Models (SLMs) for real-world applications -- faster inference, lower costs -- may no longer be clear. In this work, we present evidence that, for domain-specific tasks that require structured outputs, SLMs still have a quality advantage. We compare fine-tuning an SLM against prompting LLMs on the task of generating low-code workflows in JSON form. We observe that while a good prompt can yield reasonable results, fine-tuning improves quality by 10% on average. We also perform systematic error analysis to reveal model limitations.
Similar Papers
Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs
Computation and Language
Makes complex writing easier to understand.
Teaching LLMs How to Learn with Contextual Fine-Tuning
Machine Learning (CS)
Teaches computers to learn new things faster.
Fine-tuning for Better Few Shot Prompting: An Empirical Comparison for Short Answer Grading
Machine Learning (CS)
Teaches computers to grade homework faster.