Huge Study

*Looks inside

this latest study examined the chat logs of 19 real users of chatbots — primarily OpenAI’s ChatGPT — who reported experiencing psychological harm as a result of their chatbot use.

Pretty small sample size despite being a large dataset that they pulled from, its still the dataset of just 19 people.

AI sucks in a lot of ways sure, but this feels like fud.

I remember reading my old states book that said a minimum of 30 points needed for normal distribution. Also typically these small sets about proof of concept, so yeah you still got a point.

It’s about 300 samples for an estimate of the distribution with a 95% confidence iirc. That’s assuming the samples are representative (unbiased) and 95% confidence doesn’t mean it’s within 95% of reality, but that 5% of tests run in such a way would be expected to be inaccurate (and there’s no way of knowing for sure which one this particular sample is because even a meta study will have such an error rate, though you can increase the confidence with more samples or studies, just never to 100% unless you study every possible sample, including future ones).

That doesn’t make sense. What if your population is only 100?

Then any statistics you measure on that population might be fully accurate for those 100 but might be less able to predict what the next 100 will look like.

You can still measure stats with smaller groups, it just means the confidence interval is smaller. With 300, there’s a 95% chance your test results are close to reality. With 100 it might be more like 66%.

Population is a statistical term which means “everything”. There is no “next 100”.

The 300 number is specifically about very big populations where you’re trying to measure something like an average of an unknown variable. It doesn’t apply to just anything statistics.