Run AI-Generated Images with Local LLM on Raspberry Pi Without Linux

A new project called NightRun allows users to run local large language models (LLMs) directly on a Raspberry Pi’s bare metal, eliminating the need for an operating system like Linux. This development is significant because it enables users to run AI-generated images and other tasks without relying on cloud services or powerful hardware.

The creators of NightRun acknowledge that LLMs deserve serious ethical consideration, but running them locally on one’s own hardware can alleviate many concerns. To achieve this, centoslinux built NightRun using Rust programming language with the assistance of Claude AI. The project runs as a UEFI application at boot, pulling the LLM from removable storage and copying it into RAM to execute.

NightRun supports various lightweight LLMs on Raspberry Pi 5 models, including Llama 3.2-1B Instruct, which can run on the 4GB model. The 8GB and 16GB versions of the device support additional LLMs such as Granite 4.1-3B and Qwen3-4B Instruct 2507. These models are capable of running more complex tasks due to their increased memory capacity.

Although NightRun is not truly running on bare metal, it operates within UEFI Boot Services. This allows the application to accept keyboard input, read from storage, and output a terminal to a display without incurring significant overhead. The creators emphasize that even lightweight operating systems supporting peripherals are more demanding than UEFI Boot Services.

Users can try NightRun by downloading the files and installer script from GitHub. The installation process involves creating a removable storage image using the provided instructions. This project demonstrates the potential for running AI-generated images and other tasks locally on single-board computers without relying on cloud services or powerful hardware.