Shoutout to the person who posted about fine-tuning a local model for a niche task

I needed to generate consistent product descriptions for a vintage camera shop in Portland, but the base Llama 3 model kept adding weird, modern tech jargon. I figured I could just prompt-engineer my way out of it in an afternoon. It took me three full days of messing with the training data and parameters before it finally stopped calling a 1970s film camera 'AI-powered'. Anyone else get stuck on a simple-sounding tuning job for way longer than planned?

3 comments

3 Comments

henry60423h ago

What kind of data did you use for the training? I tried something similar for old tool descriptions and found the base model's modern word associations are way stronger than you'd guess. It took mixing in a bunch of scanned old catalog pages with my own notes to finally get the tone right. The hard part is knowing when to stop adding more examples versus tweaking the learning rate again.

jennyh411d ago

Ugh, same thing happened with my poetry bot.

uma40923h ago

Same thing happened" is the worst feeling. Henry604's point about modern word associations is huge, it's like the model has a default setting you can't just turn off. What kind of poetry was your bot messing up, the rhythm or the word choice?