Loading…
10
Question about picking a local AI model for my small business
I had to choose between paying for a big cloud AI service or running a smaller, free model on my own computer. I went with the free one, Llama 3.1 8B, because the monthly fees were adding up. It's slower, but now I'm not locked into a subscription just to sort my customer emails. Anyone else try to run their own AI stuff to save cash?
3 comments
Log in to join the discussion
Log In3 Comments
the_nina6mo ago
Hell yeah, that's the smart move. I did the same thing last year, got so sick of the monthly bill for a simple chat thing on my website. Switched to running a model on an old computer in the back office. It's not as fast to answer, but who cares? The money stays in my pocket now, and that's what matters for a small shop. Total game changer to cut out that subscription fee forever.
3
stella_baker6mo ago
There is one downside nobody talks about with this. Running a model on an old computer in your office means you are eating the electricity cost. A GPU running 24/7 can add a surprising amount to your bill, especially if you are in a place with expensive power. I did the math on my setup and it was like running a space heater year round. Not a dealbreaker for everyone but it definitely ate into my savings more than I expected. You have to factor that in when you think you are saving the full subscription amount. It is still cheaper overall for me, just not as much as I first thought.
7
jordang326mo ago
Nice, @the_nina. That's the real win for a small business, getting off the subscription treadmill. The slight delay is a fine trade for keeping that cash every month. It feels way better to own your own tools.
4