Selfhosted

46671 readers

458 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago

MODERATORS

HybridSarcasm@lemmy.world

HybridSarcasm@lemmy.hybridsarcasm.xyz

What can I use for an offline, selfhosted LLM client, pref with images,charts, python code execution (lemmy.world)

submitted 2 days ago* (last edited 2 days ago) by catty@lemmy.world to c/selfhosted@lemmy.world

24 comments fedilink hide all child comments

I was looking back at some old lemmee posts and came across GPT4All. Didn't get much sleep last night as it's awesome, even on my old (10yo) laptop with a Compute 5.0 NVidia card.

Still, I'm after more, I'd like to be able to get image creation and view it in the conversation, if it generates python code, to be able to run it (I'm using Debian, and have a default python env set up). Local file analysis also useful. CUDA Compute 5.0 / vulkan compatibility needed too with the option to use some of the smaller models (1-3B for example). Also a local API would be nice for my own python experiments.

Is there anything that can tick the boxes? Even if I have to scoot across models for some of the features? I'd prefer more of a desktop client application than a docker container running in the background.

you are viewing a single comment's thread
view the rest of the comments

[–] catty@lemmy.world 9 points 2 days ago* (last edited 1 day ago) (4 children)

I've discovered jan.ai which is far faster than GPT4All, and visually a little nicer.

EDIT: After using it for an hour or so, it seems to crash all the time, I keep on having to reset it, and currently am facing it freezing for no reason.

[–] voidspace@lemmy.world 1 points 1 day ago (1 children)

Took ages to produce answer, and only worked once on one model, then crashed since then.

[–] catty@lemmy.world 1 points 1 day ago* (last edited 1 day ago)

Try the beta on the github repo, and use a smaller model!

load more comments (2 replies)