I was a little disappointed that there weren’t more variables accounted for regarding the participants

I think it’s even worse than that. quoting from the study’s abstract:

In Study 1, participants (1,682 adults recruited from Prolific)

…

In Studies 2 and 3, participants (905 adults recruited from Prolific)

hadn’t heard of Profilic before, they describe themselves as:

“Prolific connects researchers, AI developers and product teams with real people who take part in studies online. It’s built for collecting high-quality, human data quickly and reliably, from training AI models to running behavioral research. Every participant is vetted, active and fairly paid, so you get trustworthy results without the usual noise.”

so the common theme is that 100% of the study participants have opted-in to this “help evaluate AI as a paid side-hustle” thing. that is going to introduce a huge selection bias that is definitely not representative of the population as a whole.

using an AI chatbot is “dabbling in computer science” in the same way that playing Operation in your dorm common room is “dabbling in med school”