# How to Run Local LLMs From a USB Drive on Windows - Original post: https://www.twitter.com/i/status/2117151459938844948 - Source account: [@CyberRacheal](https://x.com/CyberRacheal) - Source language: English - Duration: 4:57 - Published: 2026-11-02 - Canonical URL: https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows ## Transcript [0:00](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows) First, you'll need a USB 3 flash drive. I recommend sixty-four gigabytes or larger, so you can store more LLMs on it. Format the drive using the exFAT file Right click the drive, choose Format, and select exFAT as the file system. [0:26](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=26) For faster loading, plug the flash drive into a USB 3.0 port. [0:31](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=31) Next, go to the Llama File GitHub page. Link is in the description. Click the Releases link on the right. [0:38](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=38) The newest release will show at the top. [0:41](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=41) Scroll down to the Assets section and find the file that starts with Llama File followed by the version number like Llama File zero point nine three. [0:51](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=51) Click it to download. [0:55](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=55) After the file downloads, rename it to Llama File dot EXE and move it to your USB drive. [1:07](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=67) After this is done, go to the Hugging Face website. [1:10](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=70) Click on Models, then select GGUF in the left navigation bar under Libraries. Here you can search for models and sort them by categories. [1:21](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=81) Pick a model you like, but keep it fourteen billion parameters or smaller if it's a dense model or thirty billion or smaller for MoE models. I'll search for the Qwen three eight B model. [1:33](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=93) Once you find the model, click on it. Go to the Files and Versions tab at the top and download your version. [1:41](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=101) I will download Qwen three eight B quantized Q4 version. [1:47](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=107) After the download is done, move the GGUF file to your USB flash drive. [2:00](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=120) And the final step before running a local LLM from a USB flash drive is to create a simple .bat file to start it. [2:08](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=128) Open Notepad or any text editor, type the following command or copy and paste it from the description. [2:15](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=135) Then enter the full model file name exactly as it appears on your USB flash drive including the .gguf extension. [2:26](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=146) Save this file to the USB flash drive with a .bat extension and name it something that helps you identify the model like Run Qwen three eight b.bat. Then run the file. [2:44](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=164) The model will load into system memory. This can take a minute or more depending on the model size and your computer. [2:53](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=173) On my desktop with one hundred and twenty-eight gigabytes of DDR4 memory and Ryzen seven five thousand eight hundred X processor, it took thirty-four seconds to launch. [3:05](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=185) Once the load is done, Llama File will open in your browser. [3:10](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=190) For this tutorial, I'll adjust only the system prompt, username, and bot name. [3:18](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=198) I'll leave all other settings at their default values and start chatting by entering a question in the box at the bottom. [3:32](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=212) The Qwen three eight B Q4 model responded at seven point fifteen tokens per second. [3:42](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=222) Let's try another model. This time I'll use Qwen three thirty B A3B Q4. [3:50](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=230) It's a MoE model with thirty billion parameters. [4:18](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=258) This model responded at fifteen point sixty-seven tokens per second. [4:24](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=264) That's it. A fully portable setup for running local LLMs from a USB flash drive. [4:30](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=270) No installation, no internet, and no admin access required. [4:35](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=275) Just plug it into any Windows PC and start using it. [4:38](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=278) This setup is private, beginner friendly, and works entirely offline. You can switch between different models and different computers and run your own AI assistant directly from the USB drive. [4:51](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=291) Feel free to share your experience or tips in the comments. [4:54](https://www.translate.mom/transcript/CYdGNJJWhbRa/en/how-to-run-local-llms-from-a-usb-drive-on-windows?t=294) Thank you for watching.