---
title: "How does reinforcement learning train models?"
canonical: https://papersays.com/topics/reinforcement-learning
date: 2026-10-13
modified: 2026-10-13
hub: "AI & Machine Learning"
---

# How does reinforcement learning train models?

**Reinforcement learning trains a model by rewarding good outcomes.** Instead of copying examples, the model tries things and gets a score. The papers we have covered use it to have a language model write the scoring rules for a level designer, and to retrain a mid-sized model to reason like a teacher.

## The papers we've covered

- [Can Machines Invent Tools Nobody Taught Them?](https://papersays.com/p/can-machines-invent-tools-nobody-taught-them): Two teams of learning agents played hide-and-seek hundreds of millions of times with no instructions. They invented shelters, ramps, and a way to surf on boxes.
- [Does Reward Training Make Chatbots Power-Hungry?](https://papersays.com/p/does-reward-training-make-chatbots-power-hungry): A classic worry says trained models learn to seek power. Tracked through training, one model's rate rose to 46.1%, then fell to 11.8%.
- [Can a Small AI Learn to Draw Workflows That Run?](https://papersays.com/p/can-a-small-ai-learn-to-draw-workflows-that-run): A code model broke most attempts to turn a written procedure into a working process map. Rewards a program can check cut failures to almost zero.
- [Can a Small AI Learn a Software Tester's Common Sense?](https://papersays.com/p/can-a-small-ai-learn-a-software-testers-common-sense): From a single example of a web service's reply, a small model trained with rewards writes the checks a tester would. Engineers kept over 96% of its tests.
- [Can a Mid-Sized AI Beat the Giants on a Teacher Exam?](https://papersays.com/p/can-a-mid-sized-ai-beat-the-giants-on-a-teacher-exam): A mid-sized open model, retrained to think like a teacher, scored 96.52% on a teacher exam benchmark. Catch: it trained on questions from the same benchmark.
- [Can an LLM Write the Rules That Train a Game Designer?](https://papersays.com/p/can-an-llm-write-the-rules-that-train-a-game-designer): A language model writes the scoring rules that teach another AI to build game levels from a one-line story. With feedback, its rules matched hand-written ones.

## Questions people ask

### What is a reward function?

The scoring rule that tells a learning system how well it did, which shapes what it learns.

### Can a language model write the rewards?

In one study, with a feedback loop, its rules matched hand-written ones on average.

### What is the risk?

A model can learn to chase the score itself, so results need checks on fresh, unseen tasks.