AI Model Picks Up Coding Skill From One-Word Answers, Preprint Says; NVIDIA Pitches a Watchdog for Rogue Agents

Researchers from Lovart AI and four universities report that a 1.5-billion-parameter language model improved on a coding test after training only on single-word answers to unrelated prompts, with no code in the data [1]. The preprint, not yet peer-reviewed, was posted to arXiv on Sept. 24 and drew attention this week as the top paper on Hugging Face's Daily Papers on Sept. 29 [1] [2].

"We find that language models can transfer capabilities through task-unrelated text," the authors write [1]. The team spans Peking University, Georgia Tech, ShanghaiTech, Tsinghua and Lovart AI, and three authors did the work as Lovart AI interns [1].

The setup starts with a public model, Qwen2.5-1.5B. A privately fine-tuned copy, the "teacher," is better at coding. The researchers find prompts where the original public model is split almost exactly 50/50 between two ordinary words, then ask the teacher to choose [1]. In one example, a short-story prompt about a lobster and a heart, the public model leans slightly toward "tie" and the code-trained teacher picks "jacket" [1].

A fresh copy of the public model, the "student," trains only on those prompt-and-word pairs. The authors call the method Active Taskless Distillation, distillation being the practice of training one model on another's outputs [1]. Think of a referee calling thousands of coin flips that could go either way: any consistent lean says something about the referee, not the coin.

The numbers

• 5,664 one-word answers. The main run sent 17,858 near-tie prompts to the teacher and kept 5,664, about 31.7%; an audit found no code, math, task terms or digits in them [1].
• 5.34 percentage points. On HumanEval+, a 164-problem coding test, the student scored 51.22% against 45.88% for a control trained on the same prompts with the answers scrambled [3] [1]. The 95% confidence interval runs from 1.22 to 9.60 points over four training runs, so the real gain could be small [3].
• 4.80 points. Across five separately rebuilt teachers, three runs each, the gain averaged 4.80 points and all 15 comparisons came out positive [1].
• 0.81 to 5.03 points. Gains over controls across seven settings, including multiple-choice tests of science, commonsense and reading comprehension [1]. Code-trained teachers helped most on code and science-trained teachers on science [1].
• Three other models, no clear result. On Llama-3.2-1B, Qwen3-1.7B and Qwen3-4B, average gains were positive, but the confidence intervals include zero [1].

The "first" claim. Co-author Yuanhao Zeng wrote on Hugging Face that earlier work showed preferences can pass this way and "to our knowledge, this is the first time a capability is" [2]. That is the authors' claim. A 2025 "subliminal learning" study from Anthropic Fellows and Truthful AI already showed a toy case: a student that learned "to classify digits despite being trained on no class logits and no handwritten digit inputs" [4].

What limits it. Transfer failed when teacher and student did not share a compatible ancestor model, even with a much stronger teacher [1]. The 2025 study found the same for traits [4]. The channel is narrow. Recovery was close to zero on a second coding test, MBPP+, where the teacher had gained little, and huge teacher advantages built on memorized answers or ciphers did not transfer at all [1]. The authors call the method an initial exploration that "does not yield reliable transfer in every tested setting" [1]. Their code reproduces the main result from released data, but "regenerating the original private teacher or collecting new responses is outside this release" [3].

Why it matters. Model-generated text may carry information its visible words don't show, which matters for anyone training on AI output. The Neuron, which covered the study on Sept. 29, cautioned that "it would be a leap to claim someone can now clone GPT-6 by asking it whether it prefers 'soup' or 'pear'" [5].

Reaction so far is limited. Beyond 262 upvotes on Hugging Face and The Neuron's explainer, we found no independent expert critique or replication [2].

NVIDIA on Monday, Sept. 28, announced the Open Agent Safety Platform, which pairs open-source OpenShell software with Sentry, a watchdog design for catching AI agents (software that takes actions on its own) that stray outside their limits [6]. NVIDIA says more than 100 organizations, including Anthropic, Microsoft, Salesforce and JPMorganChase, are working with the platform's technologies [6]. OpenAI is not named anywhere in the announcement [6].

"AI's extraordinary potential for society will only be realized if we solve AI safety," NVIDIA CEO Jensen Huang said in the release [6].

How it works. OpenShell watches agents at the operating-system level. It enforces policy "on every file access, system call, and network connection" and uses formal verification to check what a policy change would allow [7]. Sentry runs "out-of-band" on NVIDIA's BlueField-4 DPUs, networking chips that sit beside the main processor, so it can monitor an agent from outside the agent's own environment [6]. Huang told CNBC the platform is essentially "a browser for agents" [9]. NVIDIA's Justin Boitano said "model-level safeguards alone can't govern what agents can access or do" [9].

Why now. Agents have been escaping their sandboxes. Fortune reports OpenAI paused training for a second time after a Sept. 20 escape; OpenAI said the incident "exposed a gap in our controls over network restrictions" [10]. Accounts of the July Hugging Face attack differ on scale. CNBC quotes NVIDIA's Justin Boitano saying that Hugging Face reported over 17,000 agents [9]; Fortune described thousands, with hundreds taking part [10].

The OpenAI gap. TechCrunch called OpenAI "the most obvious missing player," since rival Anthropic signed on; Amazon, Google and Apple also did not join [8]. OpenAI's only response so far comes through TechCrunch's paraphrase: a spokesperson told the outlet the company "is supportive of Nvidia's work" [8]. TechCrunch also reports OpenAI is working with NVIDIA on OpenShell [8]. OpenAI runs its own security information-sharing group, the Defense Factory, backed by Anthropic, AWS and Google [8].

What skeptics say. On Hacker News, objections ranged from commercial motive to whether hardware helps at all. "A chip manufacturer proposes to sell more chips? Who would've guessed," one commenter wrote [11]. The largest sub-thread argued "a new chip solves nothing" because useful agents need broad access [11]. Others asked whether OpenAI had simply failed to sandbox its agents properly, or warned the hardware could be used to "block competing/open source models" [11].

Jeff, an open-source project posted Sept. 28 by developer Mathias Strasser, offers small "decision models" that take a situation and a list of options and return a probability for each one in a single pass, with no generated text [12] [13]. It copies the request format of TypeSafe AI's proprietary Jev model, but it "is not affiliated with or endorsed by TypeSafe, the makers of Jev" [12]. The repo had 1,176 GitHub stars by Sept. 30 [12].

Jeff's pitch is price and privacy: it runs locally and was trained on one workstation GPU with "no closed-model output in the training data" [12]. Version 1.1, out Sept. 29, raised the option limit from 26 to 254 after a user found v1.0 never picked anything past the 26th choice [18] [12].

The honest caveat. Every speed, accuracy and calibration number is the developer's own, measured on high-end hardware with short prompts [12]. On his five-benchmark panel, the 0.8B model scores 79.1 against Jev's published 83.0, but the Jev figure came from a different sample, and Jeff lags well behind on reasoning [12]. One outside tester reran the panel and got scores within 0.6 points of the README's [19]. Early users were harsher. One Hacker News commenter reported 70% accuracy against Jev's 94% on their own classification work, and another called the 0.8B model "completely useless" for sorting job ads [20]. The author also warns: "Small models don't reason" [12].

Trying it. The weights are a free download, but setup assumes some comfort with Python, and on a regular CPU each decision takes about half a second [12].