The concern would be the ever increasing cost of GPUs. I imagine to help mitigate the risk, having a setup where more GPU's can be added on, and the old ones can still be used (as new ones get added on) would make the endeavor more palatable. Vs. having to swap out old GPUs for new ones, because the money spent on the old ones is lost (or at least some of it is lost if you can find a way to resell them).
I was spending $3000 per month on AI and my team was restrained from using it because we don’t have that type of budget per person. Even $200 per person was a stretch. But we could buy enough hardware that it made sense per person
