Section III: Distributed Systems Fundamentals
April 9, 2026
In our last lesson, we saw the high-level P2P architectures:
But this raises the real engineering questions:
- How does a peer gossip? How does it find other peers to talk to?
- How do you build an “address book” for a P2P network without a central server?
- What happens when peers constantly join and leave?
In this lesson, we dive into the “plumbing” of P2P by exploring the two main types of overlay networks and the critical security problems they face.
A logical map built on top of the physical internet. It’s the “rules of the road” that peers use to find each other and share data, without a central server.
Physical topology is transparent to overlay nodes
Numbers show how many copies each peer receives. Flooding keeps going until every peer that learns the message has forwarded it, so outer peers are effectively in a race to rebroadcast first.
The sender does not know how to get to the final destination. Each intermediary just forwards the message to the closest next destination it knows.
The two P2P designs (Unstructured and Structured) are tools for different jobs. Choosing the right one depends on what you need to do.

Why P2P Networks Get Clogged
Unstructured (Flooding)
Structured (DHTs)
1. The Big Idea: “The Clock”
50 would be stored on the first peer with an ID greater than 50 (e.g., Node 55).2. The Problem: The Slow Walk
Node 1 and need file 50, you could just “walk around the clock” asking every peer… but that would be incredibly slow on a network with millions of peers.3. The Solution: “Shortcuts”
Chord gives each peer a “finger table,” which is just a list of shortcuts to peers on the other side of the clock.
Instead of walking, you “jump” across the clock, getting closer with each hop.
This “jump” method lets you find any peer in a massive network in just a few hops.
The big idea is to organize the entire network into a logical ring, like a clock.
Node 8 Finger Table
| Node | IP Address |
|---|---|
Node 12 |
10.0.0.12 |
Node 17 |
10.0.0.17 |
Node 24 |
10.0.0.24 |
Node 42 |
10.0.0.42 |
Node 42 Finger Table
| Node | IP Address |
|---|---|
Node 51 |
10.0.0.51 |
Node 51 |
10.0.0.51 |
Node 56 |
10.0.0.56 |
Node 8 |
10.0.0.8 |
Node 51 Finger Table
| Node | IP Address |
|---|---|
Node 56 |
10.0.0.56 |
Node 56 |
10.0.0.56 |
Node 8 |
10.0.0.8 |
Node 12 |
10.0.0.12 |
Lookup result: File 54 is stored at successor Node 56.
CAN (Content-Addressable Network) is another “structured overlay” design. Instead of a circle, it uses a logical map or grid.
0-7 or 8-F.0-3, 4-7, 8-B, C-F.The Routing: “Greedy Walk”
N1 and find file D4Pastry is a third “structured overlay” design. It routes queries by matching the target’s ID one digit at a time, like looking up a phone number.
The Big Idea: “The Phone Number”
1A2B.1...1A...1A2...1A216B16B -> 1C4 matches the first digit 11C4 -> 1A7 matches the prefix 1A1A7 -> 1A2 reaches the responsible neighborhoodRouting table at 16B |
next hop |
|---|---|
prefix 1... |
1C4 |
prefix 2... |
2F1 |
prefix A... |
A90 |
Routing table at 1C4 |
next hop |
|---|---|
prefix 1A. |
1A7 |
prefix 1B. |
1B9 |
prefix 1F. |
1F2 |
Leaf set at 1A7 |
peers |
|---|---|
| lower IDs | 196, 19F |
| higher IDs | 1A2, 1AD |
This “prefix-matching” design is extremely fast and resilient, making it a popular choice for building the “address book” for decentralized applications.
All P2P networks face a “fundamental property”: peers are not reliable. They constantly join, leave, and crash. This is called churn.
The Problem of “Churn”
What does churn break?
In Unstructured Networks (Gossip):
In Structured Networks (DHTs):
Node 55 (storing file 50) suddenly leaves, file 50 is now gone.Node 45 and Node 55 is broken.Node 55 is now pointing at an empty space.Peers are constantly running maintenance tasks in the background:
File 50 isn’t just on Node 55; it’s also copied to the next few peers (e.g., Node 60, Node 65).All P2P networks can break. A “network partition” is when the network splits into two or more “islands” that can’t talk to each other.
How Networks Break
To survive in the real world, P2P networks combine redundancy with maintenance, but they do it in different ways.
Structured (DHT)
stabilize() and fix_fingers() to repair broken successor links and shortcuts.Unstructured (Gossip)
Partitions are a “Code Red”
These P2P networks have a fatal security flaw. We’ve assumed peers are honest. What if they’re not?
The Sybil Attack
Let’s review the core P2P concepts before we move on.
stabilize()). Unstructured networks “heal” by gossiping peer lists (like CYCLON).
Peer-to-Peer Design — Army Cyber Institute — April 9, 2026