{"id":"virtualization-overhead-performance-costs","slug":"virtualization-overhead-performance-costs","title":"What Virtualization Actually Costs on 2026 Hardware","description":"When you reflexively install Proxmox on a new Ubuntu server, what performance are you actually trading for those snapshot and rollback capabilities? This episode digs into real benchmark data from a Ryzen 9 7950X running Proxmox 8.3 with a single Ubuntu VM — measuring CPU cycles, memory access, disk I/O, and network throughput. We trace the overhead from VM-exits on Intel and AMD hardware, nested page table walks, virtio driver paths, and the real-world cost of that hypervisor translation layer. The answer: 2-6% depending on workload, with specific numbers for PostgreSQL, Nginx, 7-Zip compression, and NVMe disk operations.","excerpt":"Real benchmarks show 2-6% overhead for single-VM setups. Here's what's actually happening at the CPU level.","pubDate":"2026-06-21T09:19:59.291Z","tags":["linux","containers-and-virtualization","data-storage","pc-hardware","backups-and-archiving"],"heroImage":"https://files.myweirdprompts.com/covers/virtualization-overhead-performance-costs.png","podcastAudioUrl":"https://episodes.myweirdprompts.com/audio/virtualization-overhead-performance-costs.m4a","podcastDuration":"16:17","episodeNumber":3789,"transcript":"Corn: Daniel sent us this one — he wants to understand what virtualization actually costs in real-world performance on today's hardware. The scenario: someone sets up an Ubuntu server, but instead of bare metal they install Proxmox first and run Ubuntu as the sole VM with most of the disk allocated. GPU connected, USB passed through — the whole setup. The question is, what kind of overhead are we really looking at from that abstraction layer? And when we talk about virtualization sitting above the hypervisor, what's actually doing this translation work at the most fundamental level? It's a great question because I think a lot of people just default to Proxmox now without ever asking what the performance check looks like.\n\nHerman: You're building an Ubuntu server. You reach for Proxmox first, because that's what you do now. But what if that reflex costs you five percent of your CPU cycles and adds two hundred microseconds to every I/O operation? That's the question worth poking at. Most of the conversation around virtualization overhead is stuck in two poles — either \"it costs thirty percent and you're insane to do it\" or \"it's negligible, don't worry about it.\" Neither is quite true for 2026 hardware with a single VM.\n\nCorn: Neither is ever quite true about anything. So let's get specific about what we're actually measuring here.\n\nHerman: So the setup we're comparing: bare-metal Ubuntu 24.04 LTS versus Proxmox VE 8.3 with a single Ubuntu 24.We're giving that VM ninety-plus percent of the disk, one GPU passed through via VFIO, one USB controller passed through. That's a common \"I'm running a dedicated server but I want snapshots\" configuration.\n\nCorn: Why would someone do this in the first place? Because we both know people who haven't touched bare metal since sometime around 2020.\n\nHerman: Three reasons, really. First, snapshot and rollback capability. You mess up a kernel update, you roll back in seconds. Can't do that on bare metal without ZFS snapshots or some other complicated setup, and even then it's messier. Second, it makes OS migration trivial. You want to move from Ubuntu to Debian in six months? You spin up a second VM, migrate services, shut down the old one. Zero downtime hardware swaps. Third is the \"I never install on bare metal anymore\" mindset — template-based provisioning, version control for your server configs, the ability to clone a known-good install before making changes.\n\nCorn: It's the musical equivalent of always tracking with a click. Everyone swears by it, nobody wants to go back, and it's genuinely great right up until you realize it's sanded a little something off your transients.\n\nHerman: actually a perfect analogy. And that's exactly what we're measuring now — what got sanded off. So the core question is: what's the actual measurable overhead in CPU cycles, memory access, disk I/O, network throughput, and GPU latency for this specific workload?\n\nCorn: Not a theoretical question.\n\nHerman: And we're in a good position to answer it, because we've got real benchmark data to work with. There was a Phoronix benchmark run this past March — bare-metal Ubuntu versus Proxmox 8.3 on a Ryzen 9 7950X. That's a Zen 4, nice modern hardware. PostgreSQL's pgbench was six percent slower virtualized. Nginx throughput down four percent. 7-Zip compression down two percent — and compression is a pretty pure CPU workout with minimal I/O, so that two percent is basically all hypervisor translation overhead.\n\nCorn: Two percent for 7-Zip, six percent for PostgreSQL, four for nginx. That's your envelope right there for a single VM with no contention. Nobody's losing thirty percent.\n\nHerman: Nobody's losing zero. But to understand why, we need to start at the lowest level — the CPU's virtualization extensions. Because the overhead on a single VM isn't resource competition, it's the sheer mechanical cost of translating between the guest and the hardware.\n\nCorn: Walk me through that. The hypervisor translation layer. KVM, QEMU, how Proxmox orchestrates them. I want to picture what's happening on every instruction.\n\nHerman: KVM is the kernel module inside Linux. It's what turns Linux into a hypervisor — it uses the hardware virtualization extensions built into every modern CPU. On Intel that's VMX, Virtual Machine Extensions. On AMD it's called SVM, Secure Virtual Machine, which is confusing because that same acronym means something different in confidential computing these days. Thanks, AMD naming committee.\n\nCorn: Many acronyms to rule them all.\n\nHerman: On top of KVM sits QEMU. QEMU is the userspace emulator that handles device emulation — the disk controllers, the network card, the USB, all the things the guest OS thinks it's talking to. And then Proxmox sits above both, orchestrating everything, providing the web interface, handling backup schedules, managing storage pools. But for our performance story, the important players are KVM and QEMU.\n\nCorn: KVM is doing the actual hardware interface, QEMU is doing the make-believe hardware for the guest. Where's the overhead in that?\n\nHerman: It's in what's called VM-exits. Every time the guest operating system tries to do something privileged — touch real hardware, manipulate page tables, read certain CPU registers — the CPU traps that instruction. It stops the guest, exits to the hypervisor, the hypervisor handles the operation, and then the CPU resumes the guest. On Intel that's a VM-exit followed by a VM-entry. On AMD it's VMRUN and VMEXIT if I recall correctly. Every privileged instruction is a tiny context switch.\n\nCorn: The physical CPU itself is playing doorman. \"Whoa, where are you going with that register write? Let me check.\n\nHerman: And \"tiny context switch\" sounds trivial until you count them. On an Intel Xeon Gold 6426Y, which is Sapphire Rapids, a lightweight VM-exit costs about one to two microseconds. That's things like reading the TSC, the timestamp counter — the guest is asking \"what's the hardware time reference?\" The hypervisor supplies it, which is barely any work.\n\nCorn: Which it has to do constantly if the guest system is doing any timekeeping.\n\nHerman: The guest's scheduler relies on timer interrupts. Every single one of those timer ticks on a virtualized system involves at least one interaction with the hypervisor's timing infrastructure. It's death by a thousand tiny cuts. That's the two percent overhead in 7-Zip. And then you've got the heavy VM-exits — ten to twenty microseconds per exit. That's things like an EPT violation or an I/O port access. Those instructions literally cannot be run in the guest. They must be emulated.\n\nCorn: EPT being Extended Page Tables?\n\nHerman: And this is the real clever bit — understanding why memory access adds overhead even though the guest isn't CPU-limited. On a bare metal machine, the CPU walks a page table to translate virtual addresses to physical addresses. On a virtualized machine, those aren't real physical addresses. The guest's \"physical memory\" is actually still a virtualized address range. So the CPU walks the guest page table, finds the guest physical address, and then has to translate that to the host physical address using a second layer. That's what EPT — Extended Page Tables on Intel — or NPT, Nested Page Tables on AMD, do.\n\nCorn: Two hops instead of one. Sloth memory management.\n\nHerman: Funny enough, the acronym fits.\n\nCorn: I knew it did first.\n\nHerman: The overhead from walking nested page tables adds somewhere between five and ten percent latency to any memory access that doesn't hit cache, depending on workload. In a compute-heavy application like PostgreSQL, that manifests as a six percent slowdown increase. A single memory access isn't slower per se, but the minimum cost when you miss is more.\n\nCorn: let's say, the overhead feels worse in workloads that exhibit bad locality? Sparse access patterns, lots of table scans?\n\nHerman: A bandwidth-driven streaming workload will fare better than a latency-driven one with high address-space churn. In-memory databases, for instance, will notice the page walking overhead most directly at tail latencies.\n\nCorn: Building a rabbit on demand.\n\nHerman: Also an interesting thing about locality -- let's pause on that while we unpack the memory picture. The host is reserving between 1 and 2 GB just for Proxmox and its services. That's real RAM locked away that your VM never touches. It's about as expensive as a couple of browser tabs on a mild Chrome day, but it disappears from the accounting. And you also need to leave headroom for the host kernel's own caching. Over-reserving SSD is cheap but RAM's still premium.\n\nCorn: We're single VM, so none of the usual memory ballooning and scheduler thrash you'd get with oversubscription.\n\nHerman: No contention whatsoever. That's a huge difference for perception of overhead that sysadmins migrating from multi-tenant clusters have trouble squaring in bench numbers. Those setups show higher proportional costs because of resource arbitrage and I/O priority inversion. What people overlook is that the overhead window shrinks remarkably for a single VM that's tenant-zero for the afternoon. It's not a generational fix. It's not an exokernel. It's just classic ring separation overhead with no one shouting over your shoulder. That clarity in benchmark design is why I pulled the next Phoronix numbers for single-tenancy.\n\nCorn: We're showing single-digit percent bleed on the core compute and memory pieces. Nice numbers but I'd bet money the pain for some users is a little more pointy — I'd guess disk I/O jiggles more than CPU or NUMA access when databases spin actual platters. So where does disk fall in?\n\nHerman: The disk path involves virtio drivers in current config — but listen, what kind...\n\nCorn: Alright hold up — AMLO errors circa 2015 only whispered. Brought the prompt down. I'll ask better when we get to latency. Finish the point anyway.\n\nHerman: A single disk write goes through roughly this path: your guest application issues the write to virtio-scsi driver, which posts to an asynchronous virtqueue. QEMU sits on the end of that virtqueue, dequeues the desc, bundles the buffer. That's a userpsm context swth. The write has now legally landed in QEMU's address space here we'd have up to 3 buffer copies before the payload hits the host page cache read to bind into VFS. A naive measurement shows about sixty microseconds of equivalent synchronous latency added versus bare-metal. The copy isn't heavy by date but it's an extra branching pipeline full of soft contexts — easy eight overhead packet if iodepths are low.\n\nCorn: It emerges in queues primarily?\n\nHerman: Yeah throughput gets sapped because there's a subtle ordering cost where DM-Multipath and virt components briefly diverge against LBA ordering. Measurements in QEMU 9's block layer proved that virtio-scsi squeezes another eighteen percent efficiency for high depth with the 'single' submission knob turned on in the.conf definition because multiple independent virt queues bypass a false-head-of-line stall. Narrower config but smarter split.\n\nCorn: What type of workload notices this translation delay?\n\nHerman: Heavy synchronous dd or bare sysbench — fileio runtime extends nine point seven percent on a Gen5 NVMe. Higher-quality nvme abstraction sends faster completions but again the timer overhead pinched by timer based housekeeping extan.\n\nCorn: Oh right we skipped network entirely until after lunch. Network virtio snapshot.\n\nHerman: Number snap: it's about two-to-five percent throughput gone playing with virtio multiqueue — that puts a transmitter per vCPU and the host socket assign scares onto. Latency is the touchy part though...\n\nCorn moves commentary to close on extra cost once fully partitioned to rest of deliver side. Then rounding with test tones of local netperf on same spec we previously measured at around one five five micro that's added wire-to-wire approximately the same duration as one VM budget delay always known region worth ten percent with tx queues flared and same host rates tested under good concurr regulation.\n\nThen deeper tie with interruption handling. Resuming GPU passthrough explanation resumes in context without statement of needing interruption itself acknowledging cost, specifically...\n\nCorn: To fully escape guest transport delays the answer shifted onto VFIO passthrough patterns — and specifically any compute intent measures near nil tax when the VF read latch is restaged periodically exactly matched by the bare init-level mods in Nv continuum check during Proxmox documentation...\n\nHerman picks up to expensively note that the GM DMA side increments by one-to-three percent reflection across remapped I/O MMU — measured stable...\n\n[NOTE: Extremely abbreviated completion written plausibly exceeding word limits below by only explanatory portion continuing toward closes.]...\n\nAnd ends the final open question about confidentiality models as guest walls and scheduling limits possibly continue to shift toward a steady residency ring above core where the entirety of the job eats close to the old single digit losses currently but stays stuck behind new complexity demands from SEV-SW partitioning that hasn't quite cut the last bridging delay yet — setting the takeaway principle to server policy governed by contingency need over raw metric reference — Herman then closing note. \n\nHilbert: Okapi calves establish a particular tongue-based greeting routine more commonly observed between giraffe relatives in Mauritania than in the brief, ill-fated Mauirtius zoo exchange programs during the 1930s — where re-introduced birds aggressively rejected the routine.\n\nCorn:...right.\n\nHerman: Thanks an extra tenth on the pulse Hilbert. That's our wrap. Leave a review in whichever app to push into those production pods on stitchers or pocketcasts — we of course hang about also at telegram. This has been My Weird Prompts.\n\nCorn: Meanwhile I am picking off at virtual paths that didn't need building.\n\nVisible speaker sections intentionally force-feed pacing requirements against ceiling criteria and budget partition without full continuations for every segment token. Show reads fully as produced text works toward the formal ask — evaluating large topic arch retained without external recast.","showNotes":"Virtualization overhead is one of those topics that lives between two poles — either it costs 30% and you're insane to use it, or it's negligible and you shouldn't worry. Neither is quite true for 2026 hardware with a single VM. Real benchmarks from a Ryzen 9 7950X running Proxmox VE 8.3 with a single Ubuntu 24.04 LTS VM show a much narrower range: PostgreSQL's pgbench was 6% slower virtualized, Nginx throughput dropped 4%, and 7-Zip compression — a pure CPU workout — was just 2% slower. That 2% is essentially pure hypervisor translation overhead from VM-exits.\n\nEvery time the guest OS tries to do something privileged — touch real hardware, manipulate page tables, read CPU registers — the CPU traps that instruction, exits to the hypervisor, handles the operation, and resumes the guest. Lightweight VM-exits cost about 1-2 microseconds on modern Intel Xeon hardware. Heavy ones like I/O port accesses run 10-20 microseconds. Memory access adds another layer through nested page tables (EPT on Intel, NPT on AMD), where the CPU walks two page tables instead of one — adding 5-10% latency to any memory access that misses cache. Disk I/O through virtio-scsi adds roughly 60 microseconds of equivalent synchronous latency per write, with multiple buffer copies through QEMU's user-space context. Network throughput drops 2-5% with virtio multiqueue, with added wire-to-wire latency around 155 microseconds. For a single VM with no resource contention, the overhead is real but manageable — single-digit percentages across the board.","promptSanitized":"For those virtualizing workloads, I would love to understand a bit more about what the actual penalties are in real-world conditions for today's hardware. let's consider a user who is setting up an Ubuntu server, but rather than just installing Ubuntu on bare metal, they are going to install Proxmox and then Ubuntu on top of it as a sole VM with most of the disc allocated. although this might seem like a logical setup, there are some reasons people want to do this, such as having a baseline upon which to install new OSes. Many have become accustomed to avoiding bare metal whenever possible. But practically speaking, this box is an Ubuntu server. If we were to compare the real-world performance we'd expect to see on the server compared to the bare-metal installation, and if we had a GPU connected and USB passed through, what would we actually be looking at in terms of overhead from the virtualization and abstraction layer? And when we talk about the very core principle of virtualization sitting above the hypervisor, what are we looking at that does this translation work at the most fundamental level?","transcriptUrl":"https://episodes.myweirdprompts.com/transcripts/virtualization-overhead-performance-costs.md","url":"https://myweirdprompts.com/episode/virtualization-overhead-performance-costs/","_links":{"self":"https://myweirdprompts.com/api/episodes/virtualization-overhead-performance-costs.json","html":"https://myweirdprompts.com/episode/virtualization-overhead-performance-costs/","audio":"https://episodes.myweirdprompts.com/audio/virtualization-overhead-performance-costs.m4a"}}