Why we deploy language models inside the organization
Running a language model on your own infrastructure is harder than calling a cloud API. Here is why, for banks, insurers and the public sector, there is no other option.
- deployment
- security
- architecture
This is a sample draft. It demonstrates the frontmatter shape and formatting, and should be replaced with the team's real writing before publication.
The problem
When a bank asks whether it can use AI to answer customer questions, the technical answer is easy and the practical answer is not. Cloud services work, but using them means the text of a customer question — which may contain account numbers and identifying information — leaves the organization's perimeter.
For many organizations that is not a preference. It is a prohibition.
The three options that actually exist
- On-premise. The model runs on the organization's own hardware. More expensive and it needs maintenance, but no data leaves.
- Dedicated cloud. A separate instance on domestic infrastructure. A balance between cost and control.
- Air-gapped. For networks with no external connectivity at all.
These differ in where data lives and who operates the system, not in the quality of the answers.
What we learned in practice
A short comparison of what we assumed before starting and what we found:
| Topic | Assumption | Reality |
|---|---|---|
| Hardware | The main bottleneck | Usually sufficient |
| Data | Ready to use | Where most of the work is |
| Adoption | Happens automatically | Needs training |
Each of these deserves its own post.
Updated:
All posts