Selfhosted

The budget friendly AI GPUs are in the shelf right next to the unicorn pen.

Afaik the budget friendliest local AI solutions currently are Mac Minis! Due to the CPU/GPU/RAM unified structure they are powerhouses for AI and astonishingly well priced for what they can put out.

IMO, a 5060 Ti 16gb is among the best values at $430.

Take a look at https://www.localscore.ai/. It helped me understand just what the difference in experience will be like.

It's all VRAM, that's the bottleneck for even the best GPUs. AMD support is spotty so you should stay in Nvidia's claws unless you know what you're doing. Figure out what kind of money you're willing to part with, and then get whatever Nvidia GPU gets you the most VRAM.

3090 24gb ($800 USD)

3060 12gb x 2 if you have 2 pcie slots (<$400 USD)

Radeon mi50 32gb with Vulkan (<$300 ) if you have more time, space, and will to tinker

Not sure if it counts as "budget friendly" but the best and cheapest method right now to run decently sized models is a Strix Halo machine like the Bosgame M5 or the Framework Desktop.

Not only does it have 128GB of VRAM/RAM, it sips power at 10W idle and 120W full load.

It can run models like gpt-oss-120b or glm-4.5-air (Q4/Q6) at full context length and even larger models like glm-4.6, qwen3-235b, or minimax-m2 at Q3 quantization.

Running these models is otherwise not currently possible without putting 128GB of RAM in a server mainboard or paying the Nvidia tax to get a RTX 6000 Pro.

Everyone is mentioning nvidia, but amds rocm has improved tremendously in the last few years, making a 6900xt 16gb an attractive option for me. I currently have a 6700xt 12gb that works no problem with ollama and comfyui, and an instinct mi25 16gb that works with some fiddling as well. From what I understand, an mi50 32gb requires less fiddling. However the instinct line is passively cooled, so finding a way to cool it might be a reason to stay away from them.

Edit: I should add, my experience is on a few Linux distributions, I can not attest to the experience on windows.

I heard about people using multiple used 3090s in a single motherboard for this. Apparently it delivers a lot of bang for the buck, as compared to a single card with loads of VRAM.

Intel has some GPUs that are more cost effective than NVIDIA's when it comes to VRAM.

Arc A770 is selling for $370 in the US, and the new B50 for $399, both with 16GB.

B60 has 24GB, but I'm not sure where to find it.

What does budget friendly mean to you?

Recent models run surprisignly well on CPUs if you have sufficient regular RAM. You can also use a low VRAM GPU and offload parts to the CPU. If you are just starting out and want to play around I would try that first. 64gb system RAM is a good amount for that.

I bought used rtx 2060 12 Gigz vram edition for about 150 bucks and it runs pretty well. 4B models run well, I even ran 12B models and while its not the fastest experience, its still decent enough. Sad truth is that nvidia gpus are miles better than any other cards for ai even running linux.

i will simply want to scan projects that i personally use to be aware of its current state and future changes, before i blindly update apps i host.

If you're just doing this for yourself then you still need to know the programming languages involved, what kind of vulnerabilities exist, how to validate them and quite a bit of how the projects operate.

The AI will output a lot of false positives and you will need to actually know if any of the "vulnerabilities" are valid or just hallucinations. Do you really want that extra workload?

Buying new: Basically all of the integrated memory units like macs and amd's new AI chips, after that any modern (last 5 years) gpu while focusing only on vram (currently nvidia is more properly supported in SOME tools)

Buying second hand: not likely to find any of the integrated memory stuff, so any GPU from the last decade that is still officially supported and focusing on vram.

8gb is enough to run basic small models, 20+ for pretty capable 20-30b models, 50+ for the 70b ones and 100-200+ for full sized models.

These are rough estimates, do your own research as well.

For the most part with LLMs for a single user you really only care about VRAM and storage speed(ssd) Any GPU will perform faster than you can read for anything that fully fits on it's VRAM, so the GPU only matters if you intend on running large models at extreme speeds (for automation tasks, etc) And the storage is a bottleneck at model load, so depending on your needs it might not be that big of an issue for you, but for example with a 30gb model you can expect to wait 2-10 minutes for it to load into the vram from an HDD, about 1 minute with a sata SSD, and about 4-30 seconds with an NVMe.

Don't forget that the "budget friendly" card cost does not include the "non budget friendly" power bill that goes with it.

RTX 3060 with 12gb vram. Cheap, when bought used, and larger VRAM when compared to other options in this price range, but it will be slow. Or if you can find one, go for Arc B50 16GB VRAM, very power efficient.