Routing email with a self-hosted language model
Running a large language model on in-house GPU hardware to classify incoming email and route it to the right queue, without sending it to an outside service.
- My role
- Deployed and ran the model and its infrastructure.
- Context
- Incoming email needed to be sorted into different queues, and the content was better kept inside the company.
- Decision
- Deploy a self-hosted model with Ollama in Docker on in-house NVIDIA GPU hardware.
- Results
- Email classified and routed automatically
- Data kept on company infrastructure
The problem
Sorting incoming email by hand is slow, and sending business email to a public AI service raises questions about where that data goes.
The approach
I deployed a self-hosted large language model using Ollama in Docker, running on in-house hardware with NVIDIA GPUs. The model reads each incoming message, classifies it, and routes it to the appropriate queue.
The result
Email is classified and routed automatically, and the data never leaves company infrastructure. It’s a small, practical example of putting generative AI to work where privacy and cost both matter.