Posted: 2026-01-20

🔗 Peer-to-Peer: Round Robin Disk Queues

Here's a really straightforward optimisation for sending files across networks to many endpoints at once. This concept was developed for use in local, high-speed networks, but because we live in the future the Internet is increasingly approaching those in throughput anyway. Many existing P2P protocols treat bandwidth as the scarce resource, but that is decreasingly true in the real world.

I thought of it probably 10-15 years ago, but this is still something nobody seems to be paying attention to — even though it is likely relevant to the behaviour of many mass data transit systems, not just P2P.

🔗 Prior Art

The traditional way that DC++ handled multiple transfers at once was to have a fixed number of global "slots" for uploads/downloads. The primary rationale was that spinning-platter hard disks would suffer from degraded performance as a result of constantly seeking from doing multiple transfers at once. Networks are near-infinitely divisible, but hard disks work much better when they're doing one thing at a time.

This obviously doesn't apply for SSDs, but in any case the slot system doesn't distinguish: slots are not connected to which disk the data is coming from or what kind of disk it is. We could easily end up with degraded performance from every slot pulling from one disk, while other disks sit idle. Peers may wait minutes or hours to access a slot so they can download a single file on an idle disk very quickly.

This is, in short, a terrible solution.

🔗 A Hopefully Better Way

• Each disk has a separate queue.
• Each entry in the queue is for a personal identity (not a connecting machine).
• An SSD simply streams data to each person in the queue simultaneously.
• A mechanical hard drive reads data to one person's local buffer at a time, rotating to the next person every couple of seconds. (there'll have to be detection for slow uploads to allow the host to switch early when the buffer's full)
• The network connection continuously pipes the read data to the recipients.
• Similar logic is applied for writing.

One of the catches with this approach is that each personal buffer takes up a small amount of space in memory. It is critical that these buffers grow or shrink according to free memory, because as soon as you start paging them the whole thing falls apart.

Its fairness is also partly predicated on having an identity layer, though you can implement it per-machine and still get a reasonable result.

🔗 Theoretical Outcomes

There are many possible benefits to this approach:

• Hard disks no longer have to interleave multiple physical reads to service multiple transfers.
• SSDs are appropriately separated and privileged.
• Nobody has to wait hours in a queue to receive a small file.
• Contention over slots simply doesn't exist.

There are some edge cases regarding backpressure, gaming the system, and identity locality but those are likely best discussed after an actual implementation is done.

BitTorrent and other contemporary platforms are structurally indifferent to this topic: they're overwhelmingly worried about the health of the swarm and data availability, rather than the mechanics of raw individual throughput.

Checkin

Version: 1

Written: 2026-01-20

Written on: 7.5mg olanzapine since 2025-11-11

Mental health was: poor - estimate 25% brain