Meta’s Self-Taught Evaluator enables LLMs to create their own training data
Human evaluation has been the gold standard for assessing the quality and accuracy of large language models (LLMs), especially for open-ended tasks such as creative writing and coding. However, human evaluation is slow, expensive, and often requires specialized expertise.
Researchers at Meta FAIR have introduced a novel approach called the Self-Taught Evaluator, which leverages synthetic data to train LLM evaluators without the need for human annotations. The method comes with a few caveats, but it could significantly improve the efficiency and scalability of LLM evaluation for enterprises that want to build custom models.
The challenges of LLM evaluation
LLMs are often used as evaluators themselves, playing a crucial role in aligning other models with human preferences or improving their own performance during training. This is especially important for tasks where multiple valid answers are possible, as is often the case with creative or complex instructions.
However, training accurate LLM evaluators typically relies on extensive human-annotated data, which is costly and time-consuming to acquire. This bottleneck becomes self-defeating, hindering the rapid development and deployment of new LLM-based applications.
The Self-Taught Evaluator addresses this challenge by using a training approach that eliminates the need for human-labeled data. It is built on top of the LLM-as-a-Judge concept, where the model is provided with an input, two possible answers, and an evaluation prompt. The LLM-as-a-Judge model aims to determine which response is better by generating a reasoning chain that reaches the correct result.
Self-Taught Evaluator starts with a seed LLM and a large collection of unlabeled human-written instructions, such as those commonly found in production systems.
Article