Oct 24, 2024, 01:28 AM
This thread will discuss populer front end that use to running ai locally on pc and android
Disclaimer : all of this tool mention are open source , this file are free to view the code, if you still worry i advise you use virustotal.com to scan file, also this guide target windows user, but all of this tool still and work for linux user and mac os.
Koboldcpp
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable from Concedo, that builds off llama.cpp, and adds a versatile KoboldAI API endpoint, additional format support, Stable Diffusion image generation, speech-to-text, backward compatibility, as well as a fancy UI with persistent stories, editing tools, save formats, memory, world info, author's note, characters, scenarios.
Llamacpp
llama.cpp is an open source software library mostly written in C++ that performs inference on various Large Language Models such as Llama. A CLI and web server is included with the library. llama.cpp is co-developed alongside the GGML project, a general-purpose tensor library. this tool basicly grandfather of all Ai tool that we know today.
GPT4all
GPT4All is developed by Nomic-ai, like other tool is base on llamacpp but design effectively for cpu interference.
Phone
Please note you need modern smartphone with at least 8 gbram and good cpu(i frankly honestly don't know what good cpu new model or 2018 model maybe)
Chatter UI
ChatterUI is a native mobile frontend for managing chat files and character cards inspired by SillyTavern. It aims to provide a mobile friendly experience to inferface with Large Language models. ChatterUI supports multiple backends(openai,claude,reverse proxy,cohere etc) GGUF models locally on your device
Download Link
Disclaimer : all of this tool mention are open source , this file are free to view the code, if you still worry i advise you use virustotal.com to scan file, also this guide target windows user, but all of this tool still and work for linux user and mac os.
Koboldcpp
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable from Concedo, that builds off llama.cpp, and adds a versatile KoboldAI API endpoint, additional format support, Stable Diffusion image generation, speech-to-text, backward compatibility, as well as a fancy UI with persistent stories, editing tools, save formats, memory, world info, author's note, characters, scenarios.
Llamacpp
llama.cpp is an open source software library mostly written in C++ that performs inference on various Large Language Models such as Llama. A CLI and web server is included with the library. llama.cpp is co-developed alongside the GGML project, a general-purpose tensor library. this tool basicly grandfather of all Ai tool that we know today.
GPT4all
GPT4All is developed by Nomic-ai, like other tool is base on llamacpp but design effectively for cpu interference.
Phone
Please note you need modern smartphone with at least 8 gbram and good cpu(i frankly honestly don't know what good cpu new model or 2018 model maybe)
Chatter UI
ChatterUI is a native mobile frontend for managing chat files and character cards inspired by SillyTavern. It aims to provide a mobile friendly experience to inferface with Large Language models. ChatterUI supports multiple backends(openai,claude,reverse proxy,cohere etc) GGUF models locally on your device
Download Link
