A minimalist proxy for your local LLM cluster (~1100 lines)

A lightweight proxy server for managing a local LLM cluster, implemented in approximately 1100 lines of code. It sits between clients and locally hosted language models, routing requests to the appropriate backend. Designed for simplicity and minimal overhead in self-hosted LLM setups.