Damon Kohler

Tags

Tags

Project Nessie: A P2P Platform in Twisted

4 min read

Nessie was my attempt to build a peer-to-peer application platform in Python using Twisted. It ran from late 2006 through 2008, cycling through architectural rewrites, framework battles, and the kind of progress-in-reverse that teaches you more than shipping ever does.

The Idea

Nessie was meant to be a P2P framework — a platform for building distributed applications where peers could discover each other, authenticate, and exchange data without a central server. Think of it as a toolkit for building things like distributed chat, file sharing, or collaborative apps on top of a peer-to-peer network.

The initial plan was to use Twisted’s Perspective Broker (PB) for network communication. I started with “oldpb” because “newpb” didn’t support third-party references in a P2P-friendly way — it required direct connections to third parties, which defeats the purpose of decentralized routing.

The Cred Refactor

The first major refactoring was migrating to Twisted Cred for authentication. The idea was clean: information flows from the authenticator to the authenticated. For a server to get data from a client, the server authenticates with the client too — mutual authentication.

This turned out to be a much larger change than expected. The existing code had a single root peer object that shared information bidirectionally, and the Cred model required a fundamentally different data flow. Along the way, I discovered that my dynamic mixin/services approach — where peer capabilities were added at runtime by modifying class bases — was mutating all instances of a class, not just the target instance. A failing test caught it, and I still don’t know how it ever passed in the first place.

Progress stalled. I committed code with all tests set to skip, in the spirit of “open development.” The tests started almost passing. Then a hard drive failure and a demanding work project pushed Nessie to the back burner.

The Architect

After coding myself into two consecutive corners, I stepped back and drew an architecture diagram. The system had gotten too complex to keep straight, and I’d been trying to get tests to pass by hacking at code instead of understanding the structure.

The diagram helped, but it also revealed how much complexity had accumulated. Nessie needed avatars and peers and perspectives and services, and the interactions between them were growing faster than I could reason about.

Revival

A year later, I threw it all out and started over. The rewrite was drastically simpler: OpenDHT for peer discovery (instead of custom routing), XML-RPC for communication (instead of Perspective Broker), and no routing at all — just direct connections over HTTP. SSL encryption would come later, and nodes could proxy for firewalled peers eventually, but the core was stripped to the minimum.

The new Nessie could do chat through a Python interpreter in a single day’s work. That told me something about how over-engineered the original had been.

I added a wxPython GUI with a GridBagSizer wrapper I wrote to make layout less painful, and built a small web framework called GSD on the side as a simpler way to build Nessie’s web interface.

What I Learned

Nessie never shipped as a product, but it was one of the most educational projects I worked on:

  • Twisted’s learning curve is real. Perspective Broker, Cred, Deferreds, the reactor loop — each is well-designed, but composing them for a novel architecture is hard. The framework fights you when your mental model doesn’t match its assumptions.
  • Start with the simplest network protocol that works. XML-RPC over HTTP was boring and perfect. Perspective Broker was powerful and a constant source of architectural friction.
  • Dynamic mixins are a footgun. Modifying class bases at runtime to add capabilities sounds clever until you realize it mutates every instance. Composition over inheritance, especially at runtime.
  • Draw the architecture before you code yourself into a corner. The diagram came too late for the first version, but it was the first thing I did for the rewrite.

The Python skills from Nessie led directly to PyRobot and eventually to SL4A — the same instinct to build platforms that let people build things on top of them.

Graph View