The Quilt Project

writings on data, communication, and computation

The Story So Far

I’ve only been talking about Quilt publicly for a couple of weeks, but there is already a lot of interest and enthusiasm out there. Many thanks for your encouragement and kind words.
There’s so much to say about what Quilt is, the problems I am trying to address by building it, and what its full shape will be.  I’ll be digging into all of that in due time in this space, but I wanted to first make an “official” announcement regarding an upcoming event that is related to Quilt, and provide some links to the places where I’ve been talking and writing about Quilt over the last few weeks.
So, in reverse-chronological order, starting with the yet-to-happen event:

May 15, 2014: Talking about CRDTs at Papers We Love NYC

I’m incredibly excited to be presenting on A Comprehensive Study of Convergent and Commutative Replicated Data Types by Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski at the May 15th meeting of Papers We Love (NYC).
From my talk abstract:
CRDTs (more generally called conflict-free replicated data types, whether they are the convergent or commutative variety) are a formalism for providing practical data and programming primitives for use in distributed systems applications without necessitating expensive (and sometimes impractical) consensus mechanisms.  Their key characteristic is that they provide conflict-free “merging” of distributed concurrent updates given only the weak guarantees of eventual consistency.
The reason why I’m so excited is because CRDTs are a foundational concept in Quilt, its treatment of data, and how computational services and their clients interact. While the body of CRDT literature is growing, and data structures that can be characterized as CRDTs predated it, the Shapiro et al. paper was the first to name them as such and provide a comprehensive description of their semantics and operational characteristics (and an understandable one at that!). I am certain that Quilt would not exist were it not for that paper.
That said, this talk will absolutely focus on CRDTs generally, and the characterization of them as given by Shapiro et al., not Quilt’s particular treatment and implementation. Of course, if you corner me afterwards and want to talk about Quilt, I’ll happily oblige! 😁
If you’re in the NYC metro area and are either interested in Quilt or distributed systems in general, be sure to stop by! I’d strongly recommend that you RSVP ahead of time if you want to attend.

Theorizing the Web, an experience

Last week, I was lucky enough to attend Theorizing the Web (TtW), an
inter- and non-disciplinary annual conference that brings together scholars, journalists, artists, activists, and commentators to ask big questions about the interrelationships between the Web and society.
Earlier this week, I wrote up that experience. Insofar as I think of Quilt as addressing social and otherwise non-technical problems as much as (or more than) software and other technical issues, I’ve known for some time that I need to focus on understanding the mechanisms underlying those problems just as much as I might research algorithms and data structures.  That’s been ongoing via various means, but TtW was a wonderfully immersive way to survey various modes of inquiry around the Web, the internet as a whole, and the ways in which we interact with technology socially and impact that technology via our socialization.
Further, I met a number of people there that have already done a great deal of work in investigating the subtle and not-so-subtle ways we are affected by the centralized nature of how we mediate data and organize our computation and communication. References and initial help from them and their presentations have already helped me greatly to better understand such dynamics — a convenient confluence to some of the concerns that motivate me to build Quilt, which I’ll of course be writing about in specific detail at a later date.
So, check out my TtW write-up. I think it’s all relevant to Quilt in ways big and small, but you can skip to the last section for the tl;dr…and tweets, don’t forget the tweets!
If you have an idea, and it seems crazy or too big, remember: everything wonderful about our world started that way. Don’t be afraid.

PhillyETE talk: Distributed Systems and the End of the API

Also last week (it was a busy week!), I attended and spoke at PhillyETE on Distributed Systems and the End of the API. The premise of the talk is that the fundamental building block of how the software we build communicates and coordinates computation — the API — is an anachronism from a non-distributed programming context that has been poorly reused as the metaphor for the distributed systems we are building today. This is not just an argument about terminology: the API concept both constrains the sorts of communication we can usefully talk about when programming in a distributed context, and does not constrain us enough in certain important ways that make building and understanding systems that operate over a network much, much more difficult and error-prone than necessary. I conclude with a brief introduction to one possible alternative to APIs: semilattices, CRDTs that use their formalism to ensure properties critical to operating in a hostile network environment, and a sketch of how the particular use case of APIs — synchronous, point-to-point, request/response communication between two parties — can be lifted to use CRDTs as a substrate.
The talk was recorded, but is not yet online. I’m currently working on a written distillation of its contents that will appear in this space next week.

Thoughtbot / Mostly Lazy interview podcast

Earlier in April, I sat down with Ben Orenstein of Thoughtbot to record an episode of their Giant Robots Smashing into Other Giant Robots podcast.  We talked about a lot of things, but one of the topics was the inevitable “what are you working?” question, which I took as an opportunity to talk about Quilt (obliquely, but there it is).
You can catch the recording in Mostly Lazy #12 (the podcast that I host), which syndicated the episode that was first published as Giant Robots Smashing into Other Giant Robots #95.


Thus concludes the roundup of Quilt stuff so far. If you read this far, perhaps you were interested or intrigued by some of all of the above; awesome! You can keep up to date with what’s happening with the Quilt Project by following @QuiltProject on Twitter.