When Daniel daisy-chained three switches around his house, he hit a question that haunts anyone who's ever stacked network gear: surely extra hops add lag, right? The plumbing analogy makes every junction feel like a bottleneck. But the real story is how dramatically the internet breaks that analogy.
Three home switches store-and-forward a 1500-byte frame in about 36 microseconds total—roughly a hundredth of what the human eye can perceive. Your cable modem, on the other hand, adds 5 to 15 milliseconds just from DOCSIS scheduling. That single hop is roughly 200 times the latency of all three switches combined. The home network optimization people obsess over gets vaporized by the ISP's first contention-based traffic management system.
Tracing the packet further reveals even more hidden complexity. From the ONT to the OLT, from aggregation routers to border routers running BGP, the real path accumulates visible and invisible hops. MPLS label-switched paths mean traceroute undercounts the physical routers involved. The core insight: your home switches aren't just negligible—they exist in a different unit of measurement entirely.
#3808: Tracing a Packet: 3 Home Switches vs the Internet Backbone
Three home switches add 36 microseconds. Your cable modem adds 5-15 milliseconds. Let's follow a packet from phone to Google News.
Episode Details
- Episode ID
- MWP-3987
- Published
- Duration
- 26:20
- Audio
- Direct link
- Pipeline
- V5
- TTS Engine
-
chatterbox-regular - Script Writing Agent
- deepseek-v4-pro
AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.
Downloads
Transcript (TXT)
Plain text transcript file
Transcript (PDF)
Formatted PDF with styling
You Might Also Like
Never miss an episode
New episodes drop daily — subscribe on your favorite platform
New to the show? Start here#3808: Tracing a Packet: 3 Home Switches vs the Internet Backbone
Daniel sent us this one — he was setting up switches around his house and hit that nagging question that probably visits everyone who's ever daisy-chained a bunch of network gear together. If my packet has to hop through three switches just getting from the bedroom to the ISP router, surely that adds lag, right? The plumbing analogy makes it feel like every extra junction slows things down. And then came the real insight — wait, if it's three switches just inside my house, how many hops are there OUTSIDE my house between me and a website? The answer is where all the interesting stuff lives. He wants us to trace a packet from a phone on a bedside table out to Google News and back, count the hops, and actually understand what's going on in those intermediate stops. So, Herman Poppleberry — let's follow a packet.
This is the episode I've been waiting for, because it touches almost everything about how the internet actually routes traffic, and most people never see below the surface. The plumbing analogy breaks immediately, but it actually tells us where to dig. In a pipe, water has mass. Every bend, every coupling, every valve creates turbulence and pressure drop. The water slows down measurably. Electrons in copper move at roughly two-thirds the speed of light. A switch stores an entire frame, looks at a MAC address, and forwards it — and at gigabit speeds, that whole store-and-forward dance takes about twelve microseconds per switch for a fifteen hundred byte frame. Three switches in your house adds thirty six microseconds. Your eye can't perceive anything below about ten milliseconds. Thirty six microseconds is literally a hundredth of your reaction time threshold.
It's so far below perception it's in the realm of leaf medicine.
I'm ignoring that. But the interesting thing is where the real latency lives. The instant your packet leaves that third switch and hits your ISP's modem, the game changes entirely. If you're on DOCSIS three point one cable internet, the modem alone adds five to fifteen milliseconds. That's your first real speed bump. And that single hop — just the modem — is roughly two hundred times the latency of all three of your home switches combined.
The home switches are collectively a fly sneezing in a hurricane. But I want to pause on that modem number for a second because five to fifteen milliseconds is a surprisingly wide range. What's actually happening inside that box to create such variability?
Great question, and it gets at why cable internet feels different from fiber in ways people notice but can't articulate. DOCSIS — that's Data Over Cable Service Interface Specification — doesn't give your modem a dedicated line back to the ISP. It's a shared medium, like old-school Ethernet hubs but managed. Your modem has to request a transmission time slot from the CMTS at the headend. It's literally asking permission to speak. If the cable segment is busy — neighbors streaming, gaming, whatever — your modem waits in a scheduling queue. That's where the five to fifteen millisecond spread comes from. At 3 AM with no contention, you're at the low end. At 8 PM on a Tuesday with every house on the block watching something, you're pushing the high end. And that's before your packet even reaches a router.
The modem isn't just translating signals — it's participating in a traffic management system that's fundamentally contention-based. That explains why my ping times to the first ISP hop look like a seismograph during peak hours.
And this is where the home switch conversation becomes almost comical. People will spend hours optimizing their internal network — buying managed switches, setting up VLANs, running iPerf tests between bedrooms — and then plug it all into a cable modem that's playing a polite game of musical chairs with forty other houses on the same coax segment. The thirty six microseconds you saved with careful switch placement gets absolutely vaporized by a single DOCSIS scheduling cycle.
To be clear, when Daniel's prompt frames the question, the instinct is to think of the home side as the potential bottleneck you're introducing yourself — like stacking too many extension cords and wondering about voltage drop. But you're rearranging dust while the ISP is doling out boulders. Let's get that packet out of the ISP and onto the actual backbone.
Right, so at this point our packet has gone phone, switch one, switch two, switch three, ONT — let's say we're on GPON fiber, so about two milliseconds invested for that modem-equivalent hop — and it now hits the ISP's first aggregation router. This is usually in a local exchange building, somewhere in your city or town, and it's a rack of big Juniper or Cisco gear, fan trays screaming, fiber patches everywhere. This is the demarcation point that genuinely matters because it's the boundary between your local loop and the ISP's core network. From here, your ISP will route the packet to a border router that peers with the rest of the world. And this is where the hop count starts adding up in ways that actually register on a traceroute.
Before we go further, let's define what a "hop" actually means at this stage, because I think there's a mental model problem. When I run traceroute and see twelve lines of output, my brain wants to picture twelve physical boxes connected by twelve physical cables in a nice straight line across a map. But that's not what those lines represent, is it?
Not at all, and this is probably the single most important thing to understand about reading traceroute output. Each line in a traceroute is a router interface that responded with a TTL-exceeded message. But multiple interfaces can belong to the same physical router. You might see hops three, four, and five all belonging to the same service provider in the same city — those could be three different line cards in one chassis, or they could be three separate routers in a ring topology within a single facility. And then there's MPLS, which we should talk about, where the entire middle of your traceroute might show label-switched paths that don't correspond to IP hops at all. The traceroute is a useful tool but it's a sketch, not a blueprint.
Okay so MPLS — Multiprotocol Label Switching — this is where the ISP wraps your packet in their own internal routing label and suddenly your traceroute shows a bunch of intermediate hops that seem to belong to the ISP but don't actually decrement your IP TTL. Can you unpack that?
Think of MPLS as the ISP building a private highway inside their network. Normally, every router along the path has to open your IP packet, read the destination address, do a longest-prefix-match lookup in a routing table that might have eight hundred thousand entries, and then forward it. That lookup takes time — not a lot, maybe half a millisecond on good hardware, but it adds up. With MPLS, the first router does that lookup once, slaps a label on the packet, and every subsequent router just reads the label and swaps it for the next label in a tiny lookup table. It's like the difference between reading a full street address at every intersection versus following colored lines painted on the road. The MPLS routers in the middle don't touch your IP header at all — they don't decrement TTL, they don't appear in traceroute unless the ISP has configured them to. So your twelve-hop traceroute might actually be traversing eighteen or twenty physical routers, and you'd never know.
That's fascinating because it means the traceroute is undercounting, not overcounting. The real path is potentially more complex than what we see. So when we say a typical path is twelve to eighteen hops, we're talking about visible layer-three hops, and the actual number of devices that touched the packet could be significantly higher.
And this is where the home switch conversation gets its final burial. We were worried about three switches adding thirty six microseconds. In the ISP core, a single MPLS label-switched router might add fifty microseconds of processing — still tiny, still not human-perceptible — but there could be a dozen of them you don't even see. The point isn't that any individual hop is slow. The point is that the entire architecture is built on a scale where your three home switches are not just negligible — they're not even on the same number line. They exist in a different unit of measurement.
We've established that the home switches don't matter for latency. But let's actually build that traceroute. I want to walk through a realistic path from a residential connection to Google News and count the real hops, visible and invisible.
Let's do it with a fiber connection because it's cleaner. Picture a phone on a bedside table at seven in the morning, and you open Google News. The app generates an HTTP request — probably HTTP/3 over QUIC these days, but let's keep it simple and say it's a TCP SYN to port 443. Your phone wraps it in an IP packet with a destination address that DNS resolved to something in Google's network, and sends it out the Wi-Fi interface. Wi-Fi adds one to four milliseconds right there due to airtime contention and frame aggregation. But let's say for this trace we use a phone plugged into a USB-C to Ethernet adapter, hardwired into switch one, for a clean measurement. First hop, switch one — a little eight-port gigabit Netgear in the home office. Switch two in the hallway, another twelve. Switch three where the fiber ONT connects, twelve more. Total home switching latency: thirty six microseconds.
That's it. That's the entire contribution of Daniel's three switches. Thirty six millionths of a second.
Now the ONT converts the electrical signal to optical and transmits it over the passive optical network. On GPON, the upstream is time-division multiple access, so similar to DOCSIS there's a scheduling component, but it's much tighter — typically one to three milliseconds total for the ONT-to-OLT hop. The OLT is the Optical Line Terminal at the ISP's local exchange. That's our first visible hop if the ISP configures it to respond to traceroute. From there, the packet hits an aggregation router — this is a big box, probably something like a Juniper MX series, serving your entire neighborhood or even your whole town. This router does the first real IP lookup and either routes the packet toward the ISP's core or slaps an MPLS label on it. Let's call this hop three or four depending on how the ISP counts.
Already we're at maybe three milliseconds total, which is a hundred times the home switch contribution. We haven't even left town.
Next, the packet traverses the ISP's regional core. This might be one or two more routers within the same metro area, connected by fiber rings that circle the city. Each of these adds maybe half a millisecond of processing plus whatever propagation delay the fiber introduces — light in fiber moves at about two-thirds of light in vacuum, so figure roughly five microseconds per kilometer. If the regional core spans fifty kilometers, that's two hundred fifty microseconds of propagation delay. Still sub-millisecond. The processing delay of the routers themselves is the bigger factor here.
Within the ISP's own network, we're accumulating maybe five to seven hops before we reach a border router. And each hop is adding somewhere between point-five and two milliseconds depending on load and hardware generation.
The border router is where things get interesting. This is the router that connects your ISP to the rest of the world. It's running BGP and it has a decision to make: which upstream provider or peering partner gets this packet? For a residential ISP, the border router probably has multiple transit providers — maybe Cogent, maybe Lumen, maybe NTT — and a bunch of peering relationships at local internet exchanges. The BGP table on this router has somewhere around nine hundred thousand to a million routes. The router's forwarding engine has to find the best match for Google's IP prefix among all of those.
Here's where I want to understand BGP path selection because this is the part that contradicts physical intuition. I used to picture the internet like a road system where you'd just take the geographically shortest path, maybe a few long-haul fiber routes straight to wherever Google's nearest data center lives. But that's not how it works at all, is it?
Not even close, and this is my favorite part. BGP — the border gateway protocol — is the glue that knits all those autonomous systems together, but it's a policy routing protocol, not a shortest-physical-path protocol. The internet backbone doesn't optimize for geographic distance — it optimizes for business relationships. The first filter is: does the ISP have a direct peering relationship with Google? If they peer at a local internet exchange, the packet might go straight from the ISP's border router to Google's edge router in the same building. That's ideal — maybe one hop, sub-millisecond latency between the two networks. But direct peering requires both parties to agree to exchange traffic without payment, and that only happens when the traffic volumes are roughly symmetric. If your ISP is small, Google might not bother peering with them directly.
If there's no direct peering, what happens?
Then the ISP pays a transit provider — a Tier 1 network like Lumen or NTT or Cogent — to carry the traffic. The transit provider has peering relationships with everyone, including Google. So the packet goes from your ISP's border router to the transit provider's network, traverses their backbone, and gets handed off to Google at some interconnection point. The BGP decision process is: first, highest local preference — which is a manually configured value that encodes business policy. Then, shortest AS path length — fewest autonomous systems to traverse. Then, lowest multi-exit discriminator if there are multiple entry points to the same next AS. Then, and only then, does it consider the IGP metric to the BGP next-hop, which is the closest thing to a geographic distance metric. Physical distance is fourth in line behind business, topology, and policy.
BGP as toll avoidance mechanism.
It prioritizes routes with the shortest AS path — fewer networks crossed — and if two routes have equal AS path length, it then goes to local preference, multi-exit discriminator, then lowest path to the next BGP next-hop. Physical distance isn't a property BGP measures at all. To map a real traceroute, what we want now is to walk a packet through the big sausage of the cloud, the intermediate part that massages billions of packets into their target networks. A representative path from a residential fiber connection to Google dot com as of twenty twenty-six typically looks like this: your first ISP hop at the aggregation router is usually an MPLS-capable edge node serving your whole neighborhood. That's hub two. Then a regional core router, in the same metro area probably, that aggregates whole suburban rings — hub three. From there, an IP handoff to a border router that exchanges routes with upstream transit providers — hub four or five in your ISP's network alone.
We're already five hops deep and we haven't even left our ISP. What's next?
The border router hands off to a transit provider. Let's say it's Lumen, because they're one of the big Tier 1 networks. The packet enters Lumen's network at their nearest point of presence — maybe a carrier hotel in your city. Lumen's internal network is a mesh of high-capacity fiber routes crisscrossing the continent. The packet might traverse three or four Lumen routers, each adding half a millisecond of processing, plus the propagation delay of the fiber spans between them. If the packet goes from, say, Denver to Ashburn, Virginia — which is a major interconnection hub — that's roughly twenty-four hundred kilometers. Light in fiber covers that in about twelve milliseconds. Add the router processing delays, and the transit across Lumen's backbone might total fifteen milliseconds.
Ashburn is where Google has a major presence?
Ashburn is basically the internet's Grand Central Station. Something like seventy percent of the world's internet traffic passes through Ashburn at some point. Google has massive data center capacity there. So the packet arrives at Lumen's border router in Ashburn, which peers directly with Google's network. BGP on the Lumen router sees that Google's prefix is reachable via the direct peering session, and forwards the packet to Google's edge router. That handoff is a single hop, probably in the same building, over a cross-connect fiber that's maybe fifty meters long. Propagation delay: effectively zero. Router processing: maybe half a millisecond.
Then within Google's network?
Google's internal network is a thing of beauty and it's worth a brief tangent. They don't run standard BGP internally the way an ISP does. They built their own software-defined networking stack called Jupiter, and their data centers are connected by a private backbone called B4 that carries something like forty percent of all internet traffic worldwide. When the packet hits Google's edge router in Ashburn, it might not stay in Ashburn. Google's global load balancer decides which data center is best positioned to serve your request based on current load, capacity, and the specific content you're requesting. For Google News, the content is heavily cached. There might be a cache node right there in Ashburn that can serve the entire page without ever touching a backend database. In that case, the packet traverses maybe two or three internal Google routers, the server processes the request in a few milliseconds, and the response starts its journey back.
Let's tally the hops. We had three home switches, the ONT, the ISP aggregation router, maybe two regional core routers, the ISP border router, three or four transit provider routers, the Google edge router, and two or three internal Google routers. That's somewhere between thirteen and seventeen hops end to end.
That's a realistic count for a well-connected residential fiber path to a major content provider. If you're on cable with DOCSIS, add the CMTS as an extra hop and more variable latency. If your ISP doesn't have good peering, you might traverse two transit providers instead of one, adding three or four more hops. The total visible hop count on a traceroute typically lands between twelve and twenty-two for most residential connections to most major websites. And the round-trip time — the number you actually feel as a user — is the sum of all those hops in both directions, plus server processing time. For our fiber scenario, we're probably looking at twenty-five to forty-five milliseconds round-trip, depending on geographic distance and peering arrangements.
Our three home switches contributed thirty six microseconds in each direction. Seventy-two microseconds total out of a forty-millisecond round trip. That's zero point one eight percent. If the round trip were a mile, the home switches contributed about ten feet.
That's the optimistic case where we're measuring the switches in isolation. In reality, the application latency — the time from when you tap the Google News icon to when the page is fully rendered — includes DNS resolution, TCP handshake, TLS negotiation if it's HTTPS, the HTTP request itself, server processing time, and then rendering time on the phone. DNS alone can add ten to fifty milliseconds if the record isn't cached. TCP handshake is one round trip. 3 adds one round trip for the handshake. HTTP/3 over QUIC collapses some of that, but you're still looking at multiple round trips before the first byte of content arrives. The total page load time might be two hundred to five hundred milliseconds. Your three switches' seventy-two microseconds is now zero point zero three six percent of the user experience.
We've thoroughly demolished the idea that home switches matter for latency. But I want to circle back to something you mentioned earlier because I think it's the real practical takeaway for someone like Daniel. You said Wi-Fi is the biggest offender in home network latency. If someone is actually trying to improve their perceived internet speed, where should they focus?
Wi-Fi is absolutely the elephant in the room. A typical Wi-Fi 6 connection in a residential environment with a few walls between the access point and the device adds one to four milliseconds of latency just for the wireless hop. But that's the average. The tail latency — the worst-case scenarios — can be much worse. Wi-Fi uses carrier-sense multiple access with collision avoidance, which means every device has to listen before transmitting and back off if the channel is busy. In an apartment building with thirty visible Wi-Fi networks, your phone might have to wait for a clear channel, and that wait can spike to ten or twenty milliseconds. Then there's retransmission — if a frame gets corrupted by interference from a microwave or a neighbor's baby monitor, the access point has to request a retransmit, which adds another few milliseconds. A single Wi-Fi hop can easily introduce more latency than the entire rest of the internet path combined under bad conditions.
If Daniel wants to actually reduce his latency, he should be looking at his Wi-Fi setup, not his switch count.
A hundred percent. The hierarchy of home latency improvements goes like this. Number one: use Ethernet instead of Wi-Fi whenever possible. A wired connection eliminates the entire class of wireless latency problems. Number two: if you must use Wi-Fi, position your access point centrally, use 5 GHz or 6 GHz bands which have less interference, and consider a mesh system with wired backhaul so each node has a clean connection to the network. Number three: replace any ancient hardware. An old 802.11n access point from 2010 is going to perform dramatically worse than a modern Wi-Fi 6E or Wi-Fi 7 device. Number four: if you're on cable internet, check your modem's signal levels. A modem with poor signal-to-noise ratio will constantly renegotiate its modulation profile, and that can add tens of milliseconds of jitter. Number five: way down the list, if you have a truly pathological home network with fifteen daisy-chained switches in a loop — which would cause spanning-tree problems anyway — maybe clean that up. But three switches? Don't even think about it.
There's one more thing I want to touch on before we close. Daniel's original question was about counting hops, and we've done that. But I think there's a deeper curiosity there that we should honor. He noticed that three switches felt like a lot, and then he zoomed out and realized there are orders of magnitude more hops outside his house than inside. That's a genuine moment of perspective shift. The internet is physically enormous in a way that's invisible to us. We tap a screen and a page loads, and the sheer number of devices that touched that request — routers in windowless buildings, fiber spans under cornfields, cross-connects in carrier hotels, load balancers in data centers — is staggering.
It really is. And I think there's something almost poetic about it. Every time you load a web page, you're launching a tiny expedition. Your packet is a digital message in a bottle that gets handed from device to device, each one making a sub-millisecond decision about where to send it next, across town, across the country, maybe across an ocean, and back again, all in less time than it takes you to blink. The three switches in your house aren't a bottleneck — they're the first three steps of a journey that involves hundreds of steps you never see. And the fact that we can build systems that do this reliably, billions of times per second, is one of the great engineering achievements of our species.
The home switches? They're not the problem. They're barely even participants. They're like the first three inches of a marathon.
So Daniel, if you're listening: keep your switches. Add more if you need ports. Don't lose a minute of sleep over the latency they add, because it's literally less than the time it took for the light from your screen to reach your eyes while you were reading the traceroute output. The real optimization work is elsewhere — in your Wi-Fi, in your ISP choice, in the peering relationships you can't control. The switches are fine. Let them switch.
That's a good place to land. Daniel, thanks for the question — it took us on a journey through switching fabric, DOCSIS scheduling, MPLS label-swapping, BGP policy routing, and carrier hotels in Ashburn, all to confirm that your three Netgear switches are not the problem. Sometimes the best engineering insight is knowing what not to optimize.
Sometimes the best podcast episodes come from someone looking at three blinking boxes and thinking, wait, how many more of these are out there? The answer is: a lot. And now we know what they're all doing.
This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities.