Usenet Spool Software: Diablo, Cyclone, INN, and How Servers Store Articles

Spool software is the server-side application that stores, indexes, and serves Usenet articles. When you connect to a Usenet provider and request an article, the spool software finds it on disk and delivers it over NNTP. When a peer sends new articles, the spool software decides whether to accept them, writes them to disk, and updates its index.

Spool software runs on the back-end servers of a Usenet provider. It is the core technology that determines how efficiently a provider can store petabytes of articles, serve thousands of simultaneous readers, and ingest hundreds of terabytes of new content per day. Most users never think about it, but it is the foundation everything else sits on.

Diablo

Diablo is a commercial Usenet server developed specifically for high-volume Usenet operations. It was designed from the ground up to handle the scale of modern Usenet: billions of stored articles, terabytes of daily ingest, and thousands of concurrent reader connections. Diablo uses a custom storage layout optimized for the access patterns of Usenet traffic, where recent articles are read frequently and older articles are accessed less often but must remain available.

Diablo is the spool software used by several major backbone operators. Its architecture separates the reader (front-end) and feeder (peering) functions, allowing operators to scale each independently.

Cyclone

Cyclone is another high-performance NNTP server designed for large-scale Usenet deployments. It handles both reading and feeding, and is designed to operate on commodity hardware at scale. Cyclone uses a distributed storage approach that spreads articles across multiple disk volumes, which helps with both performance and fault tolerance.

INN (InterNetNews)

INN is the oldest and most widely known open-source Usenet server. Originally developed by Rich Salz in 1991, INN has been the reference implementation of NNTP for decades. It is free, well-documented, and runs on most Unix-like systems.

INN works well for small to mid-size deployments: universities, hobbyist servers, and organizations that carry a subset of newsgroups. For large commercial providers handling the full Usenet feed (hundreds of terabytes per day), INN does not scale as well as Diablo or Cyclone without significant customization. Most major providers outgrew INN years ago, but it remains important as the software that keeps smaller parts of the Usenet network alive.

DNews

DNews is a commercial NNTP server that was popular with ISPs in the late 1990s and 2000s. It was designed to be easy to set up and manage, making it a good fit for ISPs that wanted to offer Usenet access without dedicating a full-time administrator to it. DNews is less common today as most ISPs have dropped Usenet service, but some smaller providers still run it.

Scale of storage

A full Usenet spool with multi-year retention stores petabytes of data across thousands of disks. The spool software must track the location of billions of individual articles and retrieve any one of them in milliseconds when a user requests it. This is a harder problem than it sounds, because the articles arrive in no particular order and get accessed in no particular pattern.

Ingest rate

A Tier-1 backbone ingests 400-600TB of new articles per day from multiple peering connections simultaneously. The spool software must accept, deduplicate, index, and write all of this data to disk without falling behind. If ingest falls behind peering, articles get dropped and completion rate suffers.

Concurrent reads

While ingesting new content, the spool software simultaneously serves articles to thousands of reader connections. Recent articles (on NVMe hot spool) are fast to serve. Older articles on spinning disk require more careful I/O scheduling to avoid bottlenecks.

Expiration and cleanup

Articles beyond the retention window must be expired and their disk space reclaimed. This has to happen continuously without impacting read or write performance. Poor expiration handling is a common source of performance problems on Usenet servers.

You will never interact with spool software directly. But the quality of the spool software your provider runs directly affects the service you receive: how fast articles are served, how reliably new content is ingested, how deep the retention goes, and how high the completion rate is.

Providers that operate their own independent infrastructure make their own choices about spool software, storage hardware, and operational practices. Resellers have no control over any of this because they connect to someone else back-end.

NewsDemon operates its own spool infrastructure across three server regions, ingesting approximately 500TB of new articles daily with 5,695+ days of retention and 99%+ completion.