19 August 2026

Liquid AI uses AI agents to build tokenizer software

• Liquid AI, a machine learning startup, deployed autonomous coding agents to construct toktoktok, a production tokenizer trainer (software that converts text into chunks for AI models to process).
• The agents completed the task by following concrete specifications, handling multiple different types of work, and using external verification to check their own progress.
• The successful project suggests that autonomous agents work best on real-world tasks when given clear instructions, varied challenges, and ways to verify their output.

How it was covered

TLDR AITLDR editorial team

Liquid AI used autonomous coding agents to build toktoktok, a production BPE tokenizer trainer. The experiment demonstrated that concrete specifications, multi-domain tasks, and external verification are key ingredients for reliable long-running agent workflows.