Small Local LLMs
Generated by PortableLM Prompt: Generate an image of an an abstract AI I have been interested in local LLMs for some time. Unfortunately, my hardware setup is old and my Nvidia Quadro P400 has a pitiful amount of VRAM. I was intrigued to read about PortableLM , a private AI environment designed to run on a local hard drive or even a portable USB stick. I downloaded the repo and installed it on an external drive. It was dead easy to install. Once you clone the repo, you just run the setup routine and you're ready to go. The setup routine automatically installs a group of models and provides scripts to run them on Android, Linux, macOS, and Windows. It also provides portable Python and engine binaries so that you don't really need anything else. It doesn't require new system permissions, registry edits, or a package manager. The repo provides LLM chat models NemoMix-Unleashed, Phi-3.5-mini-instruct, Qwen3.5-9b-Uncensored, dolphin-2.9, gemma-2-2b-it-abliter...