Wil Reynolds

AI Product Comparisons Drown in Unverified Self‑published Listicles

Spent hours and hours vibe coding something that pulled answers to prompts like "I'm comparing company A and company B for xyz product or service tell me which is better for my use case" I've been manually doing evals and tweaking code. The one thing I've learned, when you use AI to compare companies or products, low quality listicles and self published stuff not verified by third parties rules the roost on both web enabled and training only models . Even if you want to avoid listicles, be wise about what you publish about yourself, I bet your company has all kinds of internal quality metrics you don't share publicly. You are going to want to do that. Sad part is there's very little scrutiny given to "self published" stats at the LLM level. Buyer beware, if you run a prompt like that, you really gotta check sources so much self published crap is in there. Found a company who is auto generating pages comparing them to every agency on the planet, at scale and the negatives they made up about other companies were CONSTANTLY cited as sources. If you don't self publish your stuff you'll find out the hard way that someone else can post about your average fees, clients, etc all not true and really ding your results.