I tried the image of this real actual road collapse: https://www.tv2.no/nyheter/innenriks/60-mennesker-isolert-etter-veiras/12875776

I told ChatGPT it was fake and asked it to explain why. It assured me I was a special boy asking valid questions and helpfully made up some claims.

I dont agree with the argument that chat gpt should “push back”.

Me neither, but if they are being presented as “artificial people to chat with” they must.

I’d rather LLMs stay tools, not pretend people.

Are we expecting the llm to act like a psychologist, evaluating if the users state of mind is healthy before answering questions?

Some of the LLMs referred to are advertised as AI psychological help, so they must either act like psychologists (which they can’t) or stop being allowed as digital therapists.