Ask HN: What computer to buy if you want to do small fine tuning and inference?
I am not looking at anything expensive, but want to buy something for fine tuning small models or run small models for experimenting and code assistance.
I am not looking at anything expensive, but want to buy something for fine tuning small models or run small models for experimenting and code assistance.
Check these out:
- https://www.hyperstack.cloud/technical-resources/tutorials/h...
- https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
- https://preview.redd.it/the-llm-gpu-buying-guide-august-2023...
Do you have a computer? What does it not do that you need? I am not an expert of any sort but I have seen lots of people (and myself) buy equipment (computers, tools, bicycles, etc) and not use but a fraction of the capabilities or have to go out and but something else because it was the wrong thing.
You can buy a 3060ti with modern CUDA support and 8gb of VRAM for ~$220 right now. If your only goal is inference/finetuning that should be the best option at it's price point.