Here's a Fun Exercize for ChatGPT

Why It Matters

The anecdote underscores risks of over-relying on large language models: they can provide useful guidance yet also self-contradict or mislead, so businesses and individuals must verify outputs and retain critical thinking. This has implications for training, decision-making, and trust in AI-assisted workflows.

Summary

A teacher recounts training a student to ask better questions rather than supplying answers, building investigative skills. The student then used ChatGPT to solve a problem; when prompted to critique its own prior response, the model produced a detailed rebuttal claiming the earlier answer was wrong. The teacher warns this reflects how the AI predicts word sequences rather than possessing knowledge, and that it may contradict itself to satisfy user prompts. He urges applying first principles and skepticism when evaluating AI-generated answers.

Original Description

Fun exercise: ask ChatGPT a question, get an answer (probably correct), and then ask it to explain, from the point of view of a senior engineer, why that answer is wrong.