Medium-sized companies face different challenges than large corporations when it comes to AI initiatives. Budgets are smaller, teams are leaner and expectations often fluctuate between "this solves everything" and "this is just hype" within a single meeting. What is needed is an architectural framework that combines pilot speed with production stability.
The typical mistake: relying on hyperscalers too early
Many initial AI projects fail not because of the quality of the model, but because of the running costs. A chat application on your own data that generates 5000 euros per month in inference fees rarely survives the first budget review. Added to this is the outflow of data to the USA - a no-go for many B2B customers from regulated industries.
Instead, the pragmatic way to get started is via self-hosted, scalable infrastructure: a Kubernetes cluster (on-premise or with a European provider), an object store such as MinIO for documents, a Postgres with pgvector for embeddings and a gateway layer that can mediate between the local model and cloud fallback.
Four components that need to work together
The stack that has proven itself in our projects consists of four components: An indexer that reads documents from MinIO, chunks them and writes them to pgvector as embeddings. A gateway that bundles model requests, enforces rate limits and implements fallback logic. An Orchestrator that selects the right agent depending on the request - chat, search, catalog, download. And a front-end widget that can be embedded in existing websites via iframe.
The separation is not academic. Each of these components evolves at a different pace, and each has its own security and compliance profile. A monolithic approach forces them all into the same cost model and release cycle.
From pilot to production
The transition from pilot to production often fails due to the question: "Who will run this on a day-to-day basis?" Without a clear answer, the project ends up with the person who built it - and becomes a technical liability with the first change of personnel.
The clean answer is a GitOps pipeline with ArgoCD: every change to the system is a commit, every release is a tag, every rollback is a 'git revert'. This dramatically reduces the demands on the operations team and makes the system independent of individual people.
What comes next
We also help smaller companies to get a practical start on private AI with their own data, without overburdening them. The first steps start with a conversation about your expectations of AI - and our support in building a realistic roadmap and clear opportunities and limitations.
Take the first step and talk to us.