Selfhosted

52803 readers

650 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago

MODERATORS

HybridSarcasm@lemmy.world

HybridSarcasm@lemmy.hybridsarcasm.xyz

What budget friendly GPU for local AI workloads should I aim for? (lemmy.dbzer0.com)

submitted 1 day ago by pjusk@lemmy.dbzer0.com to c/selfhosted@lemmy.world

30 comments fedilink hide all child comments

Hiya,

Recently upgraded my server to an i5-12400 CPU, and have neen wanting to push my server a bit. Been looking to host my own LLM tasks and workloads, such as building pipelines to scan open-source projects for vulnerabilities and insecure code, to mention one of the things I want to start doing. Inspiration for this started after reading the recent scannings of the Curl project.

Sidenote: I have no intention of swamping devs with AI bugreports, i will simply want to scan projects that i personally use to be aware of its current state and future changes, before i blindly update apps i host.

What budget friendly GPU should i be looking for? Afaik VRAM is quite important, higher the better. What other features do i need to be on the look out for?

you are viewing a single comment's thread
view the rest of the comments

[–] Diplomjodler3@lemmy.world 61 points 1 day ago (2 children)

The budget friendly AI GPUs are in the shelf right next to the unicorn pen.

[–] breadsmasher@lemmy.world 11 points 1 day ago

Ooh do they have any magic beans? Im looking to trade a cow for some

[–] irmadlad@lemmy.world 3 points 1 day ago (1 children)

I've self hosted a few of the bite sized LLM. The thing that's keeping me from having a full blown, self hosted AI platform is my little GeForce 1650 just doesn't have the ass to really do it up right. If I'm going to consult with AI, I want the answers within at least 3 or 4 minutes, not hours. LOL

[–] Diplomjodler3@lemmy.world 7 points 1 day ago (1 children)

Quite so. The cheapest card that I'd put any kind of real AI workload on is the 16GB Radeon 9060XT. That's not what I would call budget friendly, which is why I consider a budget friendly AI GPU to be a mythical beast.

[–] irmadlad@lemmy.world 1 points 1 day ago

It would be super cool tho, to have a server dedicated to a totally on premise AI with no connectivity to external AI. I'm just not sure whether I can justify several thousand dollars of equipment. Because if I did, true to my nature, I'd want to go all in.