Question about picking a local AI model for my small business

I had to choose between paying for a big cloud AI service or running a smaller, free model on my own computer. I went with the free one, Llama 3.1 8B, because the monthly fees were adding up. It's slower, but now I'm not locked into a subscription just to sort my customer emails. Anyone else try to run their own AI stuff to save cash?

3 comments

3 Comments

the_nina1mo ago

Hell yeah, that's the smart move. I did the same thing last year, got so sick of the monthly bill for a simple chat thing on my website. Switched to running a model on an old computer in the back office. It's not as fast to answer, but who cares? The money stays in my pocket now, and that's what matters for a small shop. Total game changer to cut out that subscription fee forever.

stella_baker1mo ago

There is one downside nobody talks about with this. Running a model on an old computer in your office means you are eating the electricity cost. A GPU running 24/7 can add a surprising amount to your bill, especially if you are in a place with expensive power. I did the math on my setup and it was like running a space heater year round. Not a dealbreaker for everyone but it definitely ate into my savings more than I expected. You have to factor that in when you think you are saving the full subscription amount. It is still cheaper overall for me, just not as much as I first thought.

jordang321mo ago

Nice, @the_nina. That's the real win for a small business, getting off the subscription treadmill. The slight delay is a fine trade for keeping that cash every month. It feels way better to own your own tools.