The way people talk about AI training data sets me off every time

I keep seeing folks online say that more data always makes a model better. But I spent 3 years at a startup in Austin working on a medical imaging AI, and we found that adding noisy x-rays actually hurt our accuracy by 12 percent. It's not about the volume, it's about how clean your labels are. Has anyone else dealt with a project where bad data actually made things worse?

2 comments

2 Comments

campbell.tara6d ago

Wait, are you saying you had a 12% drop just from adding noisy images? That's pretty wild, but I believe it. We had a similar thing at my old company with a customer service chatbot. We kept feeding it more and more chat logs, but half of them were just people typing "asdfghjkl" or spam. The bot started answering real questions with those random strings. It got dumber the more data we gave it. So yeah, bad data definitely makes things worse, not better.

williamhill6d ago

You're spot on @campbell.tara, that 'asdfghjkl' thing is hilarious but sad. I've seen the same with image recognition models where people upload memes and random crap, and the AI starts classifying actual photos as 'unrelated nonsense.'