Can Claude Code Learn To Draw In MS PAINT?
Why It Matters
This experiment shows that language models can autonomously learn visual tasks through iterative feedback, suggesting new, low‑code approaches to AI‑assisted design and automation across industries.
Key Takeaways
- •Claude Code learns to draw via iterative browser automation
- •Goal‑oriented prompts guide LLM to replicate JS Paint images
- •Similarity metric drives refinement until ~95% match
- •New skills like brush‑stroke and pencil techniques emerge
- •Experiment shows LLMs can automate creative visual tasks
Summary
The video demonstrates how the Claude Code large language model can be trained to reproduce drawings in a web‑based MS Paint clone using only browser‑automation tools. By feeding the model a target PNG, a prompt, and access to Chrome’s debugging protocol, the author sets a clear objective: recreate the image in JS Paint and iteratively improve until a similarity threshold is reached.
The system reads the reference image, navigates to the paint canvas, and generates CDP scripts that move the mouse, select colors, and apply brush strokes. After each attempt, a screenshot comparison yields a similarity score, prompting Claude Code to adjust its script. Over several cycles the model refines details such as the fishing rod, facial features, and text orientation, eventually achieving around 95% similarity, albeit with some shortcuts.
Notable moments include the model’s ability to invent new “skills” like drawing abstract oil paintings, pencil portraits, and a dog in the snow, each built from repeated trial‑and‑error loops. The creator highlights a specific quote: “The hands are the most complex part…nature solves it by vibecoding,” framing the experiment as a proof‑of‑concept that LLMs can emulate evolutionary search to solve intricate tasks.
The implication is that LLMs, when coupled with automated testing and feedback, can extend beyond text generation to perform visual and creative workflows, opening pathways for AI‑driven design, rapid prototyping, and even autonomous art creation without hand‑coded instructions.
Comments
Want to join the conversation?
Loading comments...