miscellany

Is the chatbot's answer right? 3 checks you can do yourself

AI tools like ChatGPT, Google's AI answers, Gemini and Copilot can sound sure and still be wrong. These three checks take a few minutes and don't need a chatbot.

1. Does it exist?

If an AI answer names a book, article, quote, law or expert, look it up in the library catalog or a search engine before you rely on it.

Real example: In May 2025 the Chicago Sun-Times printed a summer reading list of 15 books. 10 of them don't exist. The list was written with AI and nobody checked it.

Source: CBS News Chicago, May 2025. www.cbsnews.com/chicago/news/summer-reading-ai-generated-titles-chicago-sun-times/

2. Ask the place itself.

For rules, prices, refunds, deals, hours or services, check the organization's own website, or call or ask them. Don't rely on what a chatbot says about them.

Real example: Air Canada's own website chatbot told a traveler they could claim a bereavement discount after the trip. The airline's real policy said to ask before travel. A Canadian tribunal ordered the airline to pay.

Source: CBS News, February 2024. www.cbsnews.com/news/aircanada-chatbot-discount-customer/

Real example: In August 2025 a Missouri restaurant asked customers, "Please do not use Google AI to find out our specials." Google's AI answers were showing deals the restaurant never offered.

Source: KMOV First Alert 4, August 2025. www.firstalert4.com/2025/08/20/please-do-not-use-google-ai-find-out-our-specials-wentzville-restaurant-asks-patrons/

3. The more it matters, the more you check.

For health, money, legal or immigration questions, treat an AI answer as a list of questions to bring to a qualified person (a doctor, pharmacist, lawyer, legal aid office or librarian), not as the answer.

Want help checking a specific answer? miscellany.io/check/ gives you a checking message to paste into a fresh chat. It is free and needs no account. (We haven't tested it in any chatbot yet.)