Posted by 
It is tempting to adopt AI for business, especially on the operational side, where the productivity improvement is real rather than just a story for the share market. While I was studying enterprise network architecture, I kept asking myself how AI could take routine work off people’s hands. I have now worked through one possible answer, and I want to share it as a three-part case study, showing the roadmap from ageing servers to on-prem AI at a mid-sized company.
Part 1: Why Virtualisation Comes First
Part 2: Phase Zero Has No AI in It
Part 3: The One Box We Left Alone
Part 1 is mainly about virtualisation, which is the foundation on which AI capability gets built and iterated. Part 2 covers the preparation phase that has to happen before AI touches anything. Part 3 covers scaling that capability by adding a dedicated inference node.
One aside before we start. AI may have devalued writing as a commodity, but I still write for my own knowledge management, and of course I use AI to broaden the knowledge, sharpening the result rather than to replace it.
Case Study: User and Business Requirements
Fitnu is a mid-sized business of roughly 200 staff that develops, manufactures and distributes solar-powered scooters. The head office is in Barangaroo, manufacturing is in Wollongong, and fulfilment is in Newcastle, with three small distribution offices in Melbourne, Brisbane and Perth. Staff at every site need reliable AD logon, local DNS, local DHCP, and file and print services, even when the WAN link drops. Fitnu also wants a long-term work-from-home pathway and a youth-oriented website to grow its customer base.
Across the three main sites, At Barangaroo, the back-office consists of three Server 2016 member servers plus one Server 2012 member server that hosts both the SQL ordering database and the public website. The Server 2012 box is end-of-life and no longer receives security updates, yet it carries two business-critical services. The email server is outdated and slows internal communication. There is no work-from-home pathway and no DMZ, the safe subnetwork that sits between a private local area network and the untrusted public internet.
The heavy rack-mounted servers also draw a large amount of power and cooling, which raises operational cost and conflicts with Fitnu’s green-business policy calling for fewer physical servers, lower power draw and less cooling load. Together, these drive the move from one-service-per-rack-server to virtualisation.
As AI adoption accelerates, executive management is also keen to use AI to improve productivity: automatically generating wiki documentation, operational procedures and Infrastructure as Code templates, so that engineers spend less time on boilerplate and more on review, design and judgement.
Based on those requirements, here is a simplified logical topology. I will not expand every detail, but the core design reasoning is worth explaining.
Barangaroo is the head office and the hub for centralised services. Newcastle and Wollongong are the spoke sites. The three sites form a triangle of point-to-point WAN links, so any single link can fail without isolating a site. Each site keeps its own pair of physical domain controllers running AD, DNS and DHCP locally.
At Barangaroo the ASA firewall (note that it is a single point of failure) sits at the perimeter. The Layer-3 Catalyst 9200L core switch carries three VLANs (Corporate, Server, DMZ), routes between the two internal VLANs, and also handles routing to the remote sites (note that it is a single point of failure as well, and NIC teaming cannot fix that, since both teamed links terminate on the same switch). The DMZ VLAN is carried at Layer 2 only, which means no SVI (Switched Virtual Interface) is configured for it on the switch, so its default gateway is an ASA interface. Internal Corporate-to-Server traffic is therefore switched at ASIC line rate, while all traffic crossing the DMZ trust boundary is inspected and logged by the firewall. Three Hyper-V hosts (Internal, DMZ and AI) share the back-office workload by security zone.
Also, the design includes a separate virtualisation server: the Internal Hyper-V host, mainly because of Exchange. Microsoft strongly advises against installing Exchange on a domain controller, as doing so locks the server’s DC role and any later promotion or demotion will break Exchange. DC1 and DC2 must stay clean, running only Active Directory, DNS and DHCP, so Exchange cannot live on them and needs a home of its own. Rather than buying a standalone box just for Exchange, the design runs it as a virtual machine (Exch01) on one dedicated Internal Hyper-V host, which also has the capacity to host SQL01, DFS and RADIUS.
Moreover, here you can see we split the back-office into three Hyper-V hosts by security zone (Internal, AI and DMZ). This makes the DMZ boundary easy to audit and lets each host be sized for its actual workload. Why this split is industry standard: internet-facing services live in the DMZ on their own physical host, so if a DMZ VM is compromised, the attacker is still on the wrong side of the firewall and cannot pivot directly to Exchange, SQL, files or AD. The Internal host can be patched, hardened and audited without touching public-facing traffic. The current design does not include a Hyper-V Failover Cluster because that would need a second Internal host, which is necessary in the next phase however.
Why Virtualisation Helps the deployment of the AI Platform
The networking design above might be one of the solutions for Fitnu, but wait a second, one thing still needs to be clarified. Are virtualisation and AI deployment compatible with each other? The answer is not that straightforward. But on the whole, AI deployment does require some special characteristics, which happen to be the strengths of virtualisation.
1. Checkpoints and shadow mode together make controlled experiments possible
How can we justify the AI is working or not. The answer is from the test result of experiments. The whole point of AI preparation phase is to accumulate evidence. But accumulating evidence requires controlled comparison: you change exactly one variable and measure the result against a fixed reference.
- Swap the prompt template → does the acceptance rate of generated alert summaries rise or fall?
- Change the retrieval parameters → how far does first contact resolution move?
- Change the model version → is the new one measurably better, or does it only feel better?
On a virtual machine the loop is short. Take a checkpoint, make one change, run the golden test set (A fixed set of questions where you already know what a good answer looks like. You run the same set every time you change something, so you can tell whether the change helped or hurt), compare the score, and revert within seconds if the score dropped.
On bare metal there is no revert. In practice that leads to one of two outcomes: nobody changes anything because the risk is too high, or somebody changes something and cannot restore the previous state.
This is what determines whether the testing produces defensible numbers or only impressions.
2. A staging environment is only economical as a virtual machine
When we launch a IT product, like new AI agents go live, typically we would put the new version in the staging environment for testing. The value of pre-production comes entirely from how closely it matches production. Cloning AI server reproduces the configuration exactly and costs nothing beyond disk space. Achieving the same with physical hardware means buying a second server, then aligning its configuration by hand before every test.environment.
The environment is also idle most of the time. A workload that is needed occasionally but must be identical to production is the clearest case for virtualisation there is (For security, here we don not suggest keep it on all the time). On the contrast, with physical hardware, you have to buy a second box and then keep it in sync by hand.
3. Being able to rebuild AI quickly is itself a security control
AI without the internet somehow is comprimised its capability, so your AI have to expose to outside, but only some parts are allowed, which can be sits in the DMZ. The design rests on a specific claim: it stores no data, so compromising it yields little.
Virtualisation is what turns that claim into an operational procedure. If compromise is suspected, deploy a fresh instance from a clean template and be back in service within minutes. On the contrast, Reinstalling physical hardware takes hours, and because it takes hours, the decision driven by human nature tends to become “monitor it for now”, which is the wrong response.
Checkpoints add a second benefit. The compromised instance can be preserved in full, memory and disk, for forensic analysis while the replacement runs. That is impractical on physical hardware.
4. A model rollback has to move several components as one unit
Changing a model version is rarely just replacing a weights file. Four things usually change together:
| What changes together | Where it lives |
|---|---|
| Model weights | Model Registry |
| Inference runtime version | AIOPS01 |
| Prompt templates | AIGW01 |
| Retrieval parameters and chunking strategy | AIOPS01 |
If the new version underperforms, all four have to return to their previous state at the same time. VM checkpoints preserve that consistency. Rolling components back individually is how you end up running a new model against an old prompt template: a combination that was never tested, and one that fails in ways nobody can reproduce.
5. Resources can be resized as each phase demands
In preparation phase, LOG01 receives telemetry from roughly 15 devices and 4 GB is sufficient. By next phase, with workstations onboarded, it may need 16 GB. On a virtual machine that is editing a number and rebooting once. On physical hardware it is a maintenance window and a screwdriver.
AI platform growth is harder to forecast than conventional business systems. How large the knowledge base becomes, and how many queries per day the assistant handles, are things you learn by running it rather than by estimating in advance.
Then I think you might have another question: why should the AI server use a separate Hyper-V host? We will explain that in Part 2: Phase Zero Has No AI in It.