[tux557]

Are there some people running some LLM models locally? What are your experiences with different models?

I've played with some recent Mistral 7B models, and they seem higher quality than earliel llama2 models of the same size. Do you have the same impression?
Also, maybe do you have some good places to download uncensored models? Because for now I've only tried huggingface, and it has some inforced rules.

[4NG3L]

I second that, it would be cool to know if anyone has an open-source and unrestricted LLM that can be run locally

[DredgenSun]

Just install LM Studio and go from there

[tengusec]

Just install LM Studio and go from there

I second this. Try out some bigger and smaller models, then use the OpenAI compatible API that's built-in to it to make tools and workflows. Best model to start is probably Mistral-Dolphin 7B.

[DredgenSun]

I ean, those basic toolsets are going to give you a smattering of what to do. Thanks to Civit.ai, I'm now able to do all ranges of image diffusing offline