Loading…
27
I fed my training data the wrong labels for 3 months and a confusion matrix finally tipped me off
Honestly I was hand labeling my image set in batches and never once checked my own work, just trusted the count matched. Last Tuesday I ran a confusion matrix and saw 22% of my cat photos were tagged dog, which killed my model's accuracy. How do you all catch label drift before it burns a whole training run?
1 comments
Log in to join the discussion
Log In1 Comment
margaret_jackson7317d ago
This reminds me of how I organize my garage. I keep shoving stuff in bins thinking future me will sort it out, then six months later I can't find a screwdriver and realize I've been putting tools in the holiday box the whole time. Labeling data by hand has that same trap, you trust the pile matches the plan but nobody ever audits the pile. What saved me on a smaller project was spot checking like 10 random samples every couple hundred labels, just enough to catch when my brain started going on autopilot. The real killer is batch labeling when you're tired, your cat/dog call gets sloppy and you never notice because you're just counting to the end. Maybe run that confusion matrix weekly instead of at the finish line, at least on a small holdout set.
1