Back at the WeAreDevelopers World Congress in July, Douglas Crockford took the stage to tell the JSON story and what's next. You can watch the video or read all about it here.
Introduction
Hello dear friends! Today, we delve into the world of data interchange with none other than Douglas Crockford, the creator of JSON. Whether you're an avid programmer or just a tech enthusiast, you'll find this tale of invention, disruption, and innovation utterly captivating.
The Birth of JSON: Jason Who?
Who among us hasn't heard of JSON (pronounced "Jason" by most) by now? JSON, or JavaScript Object Notation, has become the world's best-loved data interchange format, an unsung hero of web development that enables the seamless exchange of data across the internet.
A Need for Communication
In 2001, Douglas Crockford developed JSON. He had started a company to create what we now call single-page web applications. These applications were multi-user, functioning well before the web had figured this out.
"It was multi-user, which is something the web hasn't figured out yet."
Back then, there was an urgent need for a way to communicate data between the server (written in Java) and the client (JavaScript running in a browser). Faced with this challenge, Crockford conceived JSON.
The Minimalist Approach
Contrary to popular advice to use XML, Crockford opted for a minimalist approach. His guiding principles were straightforward:
- Minimal: It had to be as simple as possible.
- Textual: Since they were delivering it in HTML pages, JSON needed to be text.
- JavaScript Subset: At the time, they used the JavaScript compiler to parse JSON.
These design tenets kept JSON small and efficient, preventing what he called "feature creep."
"These were my design rules in developing JSON: that it'd be minimal, that it’d be textual, and that it had to be a subset of JavaScript."
The Community's Reaction
Not everyone was thrilled with Crockford’s innovation. The XML community, which had prided itself on disrupting pre-existing systems like ASN.1 and CORBA, was less than pleased to be on the receiving end of disruption. Nonetheless, Crockford pushed forward, confident in JSON's potential.
"The good thing about reinventing the wheel is that you can get a round one."
The Steady Rise of JSON
Over the years, JSON steadily gained traction. Unlike XML, which experienced a rapid peak and decline in interest, JSON’s journey was a slow, steady ascent.
Stability: JSON's Core Strength
One of JSON's paramount virtues is its stability. There is no version number; JSON has remained unchanged since its inception. This unchanging nature is invaluable in a world where technologies constantly evolve.
"Everything's changing except the JSON layer. The JSON layer will be the way it is to the end of time."
Crockford emphasizes the importance of not breaking JSON. Any new standard should offer substantial improvements rather than superficial tweaks, avoiding unnecessary fragmentation and incompatibility.
An Evolutionary Peak
Crockford never envisioned JSON as the last data interchange format. Instead, he sees it as an adaptable tool suitable for multiple environments. Recently, he's been advising new data format designers, urging them to build upon JSON's successes without breaking it.
Moving Beyond JSON: The Rise of Misty
Crockford's latest project, Misty, emerges from his admission that JSON, while excellent, does not meet all his current needs. Misty is an ambitious undertaking, an actor-based system allowing processes to be distributed across many machines with a secure networking layer.
A New Standard: Noda
To facilitate Misty's needs, Crockford introduces Noda (which stands for Notation). Noda is a JSON-like binary encoding format with a focus on large binary objects (blobs) and private process addresses.
"Noda does what JSON does, it's not intended to replace JSON. It is not human-readable, but it is much more compact and much easier to encode and decode."
Kim: Keep it Minimal
Crockford also introduces Kim, a minimalistic encoding scheme that improves upon UTF-8 by using variable byte encoding. This makes it much more compact and easier to process, particularly for specific scripts and applications.
"My favorite design principle, keep it minimal."
Technical Deep Dive: Understanding Noda
Blobs and Process Addresses
Noda utilizes blobs and process addresses within its binary encoding framework. A blob (binary large object) is encoded in a manner similar to Kim, using a prefix to indicate its type and length, followed by a series of data bytes.
Example Encoding
To encode an elliptic curve key (521 bits long), Noda uses the blob prefix:
- Continuation bit set for additional bytes
- Bits of the key filled in
- Followed by data bits
Text and Arrays
Text and arrays in Noda follow similar encoding principles. Text is succeeded by Kim characters, and arrays are followed by data elements or other Noda components.
Records and Floating Points
Records, known as objects in JSON, are called records in Noda, aligning more closely with classical programming terminology. Floating points are represented using an integer coefficient and an integer exponent, akin to scientific notation without decimal points.
"We represent a number with an integer coefficient and an integer exponent so that you know the value of your number is the coefficient times ten to the exponent."
Symbols
Noda supports symbols, allocating space for 16 different types, currently utilizing four. One notable symbol is private, indicating a record containing a private process address.
Conclusion: A Glimpse into the Future
Noda is poised to play a crucial role in Misty, providing a low-level messaging framework while JSON continues to offer a bridge to other systems. This evolution highlights the dynamic nature of technology and the continual drive for better, more efficient solutions.
Crockford's journey from JSON to Misty embodies the spirit of innovation. His advice remains ever relevant: "Please don't make bugs."
"Please don't make bugs. Thank you and good night."
And with that, we bid farewell to an enlightening exploration of data interchange formats and look forward to the exciting developments on the horizon.
Further Reading
For those eager to dive more deeply, here are some recommended resources:
