Do language models beat classic methods?
Not always. In one study they flagged network attacks well, but an older method still won, faster and cheaper.
Ask your own in the app → TopicLLMs · AI & Machine Learning
What can large language models really do?
Large language models are text predictors trained on huge amounts of writing. The papers we have covered test them on concrete jobs: spotting network attacks, knowing when to stay quiet, writing game rules, passing teacher exams and writing software tests. The pattern so far: they are often useful, rarely magic, and a plainer method sometimes wins.
Paper by paper
The papers we've covered
- AI & ML LLMs
arXiv8 min listen
Does Reward Training Make Chatbots Power-Hungry?
A classic worry says trained models learn to seek power. Tracked through training, one model's rate rose to 46.1%, then fell to 11.8%.
Preprint Checkpoint experiments1 model family Preliminary evidence
- AI & ML LLMs
NeurIPS 20178 min listen
How AI Learned to Read a Whole Sentence at Once
In 2017, a Google team dropped word-by-word reading from machine translation. Their attention-only design beat the best systems and trained in days.
Peer-reviewed Benchmark experiment Moderate evidence
- AI & ML LLMs
Process Science7 min listen
Can a Small AI Learn to Draw Workflows That Run?
A code model broke most attempts to turn a written procedure into a working process map. Rewards a program can check cut failures to almost zero.
Peer-reviewed Training experiment with benchmark Moderate evidence
- AI & ML LLMs
arXiv8 min listen
Can AI Give Doctors a Useful Second Opinion?
Eight AI models played a panel of specialists on 190 hard published cases. They reasoned well, but the final call was the weak spot.
Preprint Benchmark on published cases Preliminary evidence
- AI & ML LLMs
Proceedings of the ACM on Software Engineering8 min listen
Can a Small AI Learn a Software Tester's Common Sense?
From a single example of a web service's reply, a small model trained with rewards writes the checks a tester would. Engineers kept over 96% of its tests.
Peer-reviewed Industrial evaluationOne company Limited evidence
- AI & ML LLMs
arXiv8 min listen
Can AI Write Tests That Catch Bugs in Quantum Code?
A test that waves broken code through is worse than no test. This study checks whether AI-written tests for quantum programs can tell right from wrong.
Preprint Benchmark study Preliminary evidence
- AI & ML LLMs
Frontiers in Artificial Intelligence8 min listen
Can a Mid-Sized AI Beat the Giants on a Teacher Exam?
A mid-sized open model, retrained to think like a teacher, scored 96.52% on a teacher exam benchmark. Catch: it trained on questions from the same benchmark.
Peer-reviewed Benchmark training experimentBrief report Limited evidence
- AI & ML LLMs
arXiv8 min listen
Can We Stop AI From Adding Made-Up Words to Searches?
Search systems ask language models to add related words to short queries, and some are invented. This preprint cuts made-up terms by 85% without hurting search.
Preprint Benchmark experiments Preliminary evidence
- AI & ML LLMs
arXiv8 min listen
Can Harmless Requests Make an AI Agent Cause Harm?
A harmless-looking request can make an AI assistant email an internal report outside the company. This study builds a system that hunts for such chains.
Preprint Simulated red-teaming experiments Preliminary evidence
- AI & ML LLMs
arXiv8 min listen
Can an AI Learn When to Say 'I Don't Know'?
Chatbots guess confidently even when they don't know. This study compares two ways to teach a model when to hold back: retrain it, or add a separate checker.
Preprint Controlled experiments1 model Preliminary evidence
- AI & ML LLMs
arXiv9 min listen
Can an AI Model Hide a Skill That No Test Can Find?
Safety teams test AI models for dangerous skills. This preprint hides an encrypted skill inside a model's weights, and the usual tests could not draw it out.
Preprint Cryptographic construction + small experiments Preliminary evidence
- AI & ML LLMs
IEEE Transactions on Games8 min listen
Can an LLM Write the Rules That Train a Game Designer?
A language model writes the scoring rules that teach another AI to build game levels from a one-line story. With feedback, its rules matched hand-written ones.
Peer-reviewed Simulation experiments4 scenarios5 seeds Limited evidence
- AI & ML LLMs
Artificial Intelligence Review8 min listen
Do LLMs Beat Classic Models at Catching Network Attacks?
Small language models learned to flag network attacks and scored above 95 percent. But a plain classic method, XGBoost, still won, faster and far cheaper.
Peer-reviewed Benchmark1 dataset4 traffic classes Limited evidence
Questions people ask
Short answers, from the papers.
Can a language model know when it doesn't know?
Partly. One study compared retraining the model with adding a separate checker that decides which answers to show.
Ask your own in the app →Are benchmark scores trustworthy?
Read the fine print: one model scored highly on an exam benchmark after training on questions from it.
Ask your own in the app →Keep up with AI & Machine Learning research, ten minutes a paper.
The papers that matter in this hub, explained, and you can ask each one questions.
