In-house AI platform with open-source LLMs, fine-tuning and RAG
MY ROLEProject lead, AI/infrastructure architect and developer
As project lead, AI/infrastructure architect and developer, I built and operated an in-house AI platform for local language models, speech recognition, fine-tuning and document-based knowledge retrieval.
Local models have been in use since the first Ollama release, now mainly through llama.cpp and LM Studio, with a self-hosted Open WebUI interface. A dedicated GPU server in the Proxmox cluster provides LLM inference, Whisper speech recognition and embedding, reranking and Qdrant services for RAG.
For the AI companion in BodySeasons, the open-source Ministral 3 14B model was trained with Unsloth using supervised fine-tuning on an approved bilingual dataset. The model runs through llama.cpp, with domain knowledge supplied at runtime through RAG.
Custom MCP servers connect products such as horizOn to AI coding tools. n8n workflows create multilingual blog posts for BodySeasons, horizOn and ProjectMakers. An internal service exposes the Claude Code, Codex and Gemini command-line tools through an HTTP interface for automation. Sensitive data can therefore be processed using our own models without leaving our network.
Result: Built and operated an in-house AI platform for inference, fine-tuning, speech recognition, RAG, MCP integration and content automation.