---
title: "What can large language models really do?"
canonical: https://papersays.com/topics/llms
date: 2026-10-13
modified: 2026-10-13
hub: "AI & Machine Learning"
---

# What can large language models really do?

**Large language models are text predictors trained on huge amounts of writing.** The papers we have covered test them on concrete jobs: spotting network attacks, knowing when to stay quiet, writing game rules, passing teacher exams and writing software tests. The pattern so far: they are often useful, rarely magic, and a plainer method sometimes wins.

## The papers we've covered

- [Does Reward Training Make Chatbots Power-Hungry?](https://papersays.com/p/does-reward-training-make-chatbots-power-hungry): A classic worry says trained models learn to seek power. Tracked through training, one model's rate rose to 46.1%, then fell to 11.8%.
- [How AI Learned to Read a Whole Sentence at Once](https://papersays.com/p/how-ai-learned-to-read-a-whole-sentence-at-once): In 2017, a Google team dropped word-by-word reading from machine translation. Their attention-only design beat the best systems and trained in days.
- [Can a Small AI Learn to Draw Workflows That Run?](https://papersays.com/p/can-a-small-ai-learn-to-draw-workflows-that-run): A code model broke most attempts to turn a written procedure into a working process map. Rewards a program can check cut failures to almost zero.
- [Can AI Give Doctors a Useful Second Opinion?](https://papersays.com/p/can-ai-give-doctors-a-useful-second-opinion): Eight AI models played a panel of specialists on 190 hard published cases. They reasoned well, but the final call was the weak spot.
- [Can a Small AI Learn a Software Tester's Common Sense?](https://papersays.com/p/can-a-small-ai-learn-a-software-testers-common-sense): From a single example of a web service's reply, a small model trained with rewards writes the checks a tester would. Engineers kept over 96% of its tests.
- [Can AI Write Tests That Catch Bugs in Quantum Code?](https://papersays.com/p/can-ai-write-tests-that-catch-bugs-in-quantum-code): A test that waves broken code through is worse than no test. This study checks whether AI-written tests for quantum programs can tell right from wrong.
- [Can a Mid-Sized AI Beat the Giants on a Teacher Exam?](https://papersays.com/p/can-a-mid-sized-ai-beat-the-giants-on-a-teacher-exam): A mid-sized open model, retrained to think like a teacher, scored 96.52% on a teacher exam benchmark. Catch: it trained on questions from the same benchmark.
- [Can We Stop AI From Adding Made-Up Words to Searches?](https://papersays.com/p/can-we-stop-ai-from-adding-made-up-words-to-searches): Search systems ask language models to add related words to short queries, and some are invented. This preprint cuts made-up terms by 85% without hurting search.
- [Can Harmless Requests Make an AI Agent Cause Harm?](https://papersays.com/p/can-harmless-requests-make-an-ai-agent-cause-harm): A harmless-looking request can make an AI assistant email an internal report outside the company. This study builds a system that hunts for such chains.
- [Can an AI Learn When to Say 'I Don't Know'?](https://papersays.com/p/can-an-ai-learn-when-to-say-i-dont-know): Chatbots guess confidently even when they don't know. This study compares two ways to teach a model when to hold back: retrain it, or add a separate checker.
- [Can an AI Model Hide a Skill That No Test Can Find?](https://papersays.com/p/can-an-ai-model-hide-a-skill-that-no-test-can-find): Safety teams test AI models for dangerous skills. This preprint hides an encrypted skill inside a model's weights, and the usual tests could not draw it out.
- [Can an LLM Write the Rules That Train a Game Designer?](https://papersays.com/p/can-an-llm-write-the-rules-that-train-a-game-designer): A language model writes the scoring rules that teach another AI to build game levels from a one-line story. With feedback, its rules matched hand-written ones.
- [Do LLMs Beat Classic Models at Catching Network Attacks?](https://papersays.com/p/do-llms-beat-classic-models-at-catching-network-attacks): Small language models learned to flag network attacks and scored above 95 percent. But a plain classic method, XGBoost, still won, faster and far cheaper.

## Questions people ask

### Do language models beat classic methods?

Not always. In one study they flagged network attacks well, but an older method still won, faster and cheaper.

### Can a language model know when it doesn't know?

Partly. One study compared retraining the model with adding a separate checker that decides which answers to show.

### Are benchmark scores trustworthy?

Read the fine print: one model scored highly on an exam benchmark after training on questions from it.