.NET’s new fast, compact Web Service endpoint: The JSV Format

April 10th, 2010 by mythz Leave a reply »

New! Benchmarks graphs are now available to better visualize the performance of ServiceStack’s JSON and JSV text serializers.

Service Stack’s Git repo is still hot from the fresh check-in that has just added TypeSerializer’s text-serialization format as a first class Web Service endpoint.

JSV Format (i.e. JSON-like Separated Values) is a JSON inspired format that uses CSV-style escaping for the least overhead and optimal performance.

Service Stack’s emphasis has always been on creating high-performance, cross-platform web services with the least amount of effort. In order to maximize performance, our web services are effectively raw text/byte streams over light-weight IHttpHandler’s (Note: SOAP endpoints still use WCF bindings). This approach coupled with extensive use of cached-delegates (eliminating any runtime reflection) has proven to provide superior performance in itself.

So why the new format?

Well up until now the de/serialization for all web service endpoints were done using the DataContract serializers in the .NET BCL. The XML DataContractSerializer looks to be a well written library providing good performance for serialization of XML. Unfortunately for reasons articulated in my previous post on the history of web services, XML although great for interoperability, does not make a good ‘programmatic fit’ for integrating with many programming language models – e.g. for AJAX applications JSON is a much more suitable format. The verbosity and strict-extensibility of XML also does not make it the ideal format in performance-critical, or bandwidth constrained environments.

The problem with JSON in .NET is that although being 2x more compact than XML it is 1.5x slower (based on Northwind model benchmarks). Unfortunately that is the fastest JSON implementation in the BCL there are others like JavaScriptSerializer which are over 40x slower still. The other blocker I encountered was that although the JSON implementation in .NET was slow, the equivalent one in MONO just doesn’t work for anything but the most simple models. Effectively Mono users had no choice but to use the XML endpoints, which is clearly not a good story for bandwidth-constrained environments as found in iPhone/MonoTouch apps.

Quite simply if I want a fast, compact, cross-platform serialization format that’s ideal to use in bandwidth-constrained, performance-critical environments as found in iPhone networked applications I had to code it myself. Drawing on years of experience in handling different serialization formats I had a fair idea on what I thought the ideal text-format should be. Ultimately the core goals of being fast and compact is the major influence in the choice of syntax. It is based on the familiar JSON format but as it is white-space significant, does not require quotes for normal values, which made it the most compact text-format that was still lexically parseable.

Other key goals was that it should be non-invasive to work with any POCO-type. Due to the success of schema-less designs in supporting versioning by being resilient to schema-changes, it is a greedy format that tries to de-serialize as much as possible without error. Other features that sets it apart from existing formats makes it the best choice for serializing any .NET POCO object.

  • Fastest and most compact text-serializer for .NET (5.3x quicker than JSON, 2.6x smaller than XML)
  • Human readable and writeable, self-describing text format
  • Non-invasive and configuration-free
  • Resilient to schema changes (focused on deserializing as much as possible without error)
  • Serializes / De-serializes any .NET data type (by convention)
    • Supports custom, compact serialization of structs by overriding ToString() and static T Parse(string) methods
    • Can serialize inherited, interface or ‘late-bound objects’ data types
    • Respects opt-in DataMember custom serialization for DataContract DTO types.

For these reasons it is the preferred choice to transparently store complex POCO types for OrmLite (in RDBMS text blobs), POCO objects with ServiceStacks’ C# Redis Client or the optimal serialization format in .NET to .NET web services.

Simple API

Like most of the interfaces in Service Stack, the API is simple and descriptive. In most cases these are the only methods that you would commonly use:


string TypeSerializer.SerializeToString<T>(T value);
void TypeSerializer.SerializeToWriter<T>(T value, TextWriter writer);

T TypeSerializer.DeserializeFromString<T>(string value);
T TypeSerializer.DeserializeFromReader<T>(TextReader reader);

Where T can be any .NET POCO type. That’s all there is – the API was intentionally left simple :)

By convention only public properties are serialized, unless the POCO is a DataContract in which case only DataMember properties will be serialized. Structs can provide custom (e.g. more compact) serialization value by overloading the ToString() instance method and providing a static TStruct.Parse(string).

The JSV Web Service Endpoint

The home page for TypeSerializer on code.google.com goes into more detail on the actual text-format. You can get a visual flavour of it with the screen shots below

Note: the results have been ‘pretty-formatted’ for readability, the actual format is white-space significant.

In comparison here is the equivalent data formatted in XML (under a nice syntax highlighter):

View JSV live Web Services

One of the major features of Service Stack is that because JSV is a supported out of the box endpoint, it doesn’t require any code for all your web services to take advantage of it. You can access all your web services with the JSV endpoint by simply changing the base URL. Below are live web service examples from the Service Stack’s Examples project:

GetNorthwindCustomerOrders XML JSON JSV | debug
GetFactorial?ForNumber=3 XML JSON JSV | debug
GetFibonacciNumbers?Skip=5&Take=10 XML JSON JSV | debug

*Live webservices hosted on CentOS / Nginx / Mono FastCGI

You can view all web services available by going to Service Stack’s web service Metadata page:

http://www.servicestack.net/ServiceStack.Examples.Host.Web/ServiceStack/Metadata

Download TypeSerializer for your own projects

The JSV-Format is provided by the TypeSerializer class in the ServiceStack.Text namespace. It is perfect for anywhere you want to serialize a .NET type, ideal for storing complex types as text blobs in a RDBMS. Like the rest of Service Stack it is Open Source, released under the New BSD Licence:

Advertisement

18 comments

  1. chadbr says:

    This is a darn good idea. Have you tested it in Silverlight?

  2. chadbr says:

    Thanks – I’ll give it a go

  3. Would sending a json object to a servicestack service cause two serializations then? One from JSON to POCO, then from POCO to JSV?

  4. mythz says:

    Hi,

    Nope the Json/Jsv/Xml/etc are completely different endpoints for the same service. So if you call via the JSON webservice endpoint it will be just JSON => C# DTO. Calling the JSV endpoint will be just JSV => C# DTO, so no double de-serialization.

    Although calling the GET REST urls for any of these service endpoints by passes any serialization (since its just the url and no JSON/JSV to serialize) and just populates the C# DTO using the query string.

  5. Mike says:

    The TypeSerializer is great. One question though: Is there a setting for branding the POCO type in the actual content? For example, Json.NET has a $type prefix on the serialized string so you can deserialize without knowing the POCO type beforehand. This would be very useful on consuming services who may not know what they are deserializing for consumption.

  6. mythz says:

    Hi Mike,

    Glad you like it. I’m leaving out serializing type information as a design decision to keep the payload as ‘data only’ and largely schema-free. Although having said this, there is nothing to stop you from doing it when its required.

    Since this topic has come up a few times in a few emails I thought it was about time to write a post highlighting what’s possible here:
    http://www.servicestack.net/mythz_blog/?p=314

  7. Mike says:

    I figured that.
    My workaround was just to prepend the type onto the serialized string before writing to the stream.
    Thanks for the post and your work on the project.

  8. Nick says:

    So is there a objective-c library for iPhone applications to deserialize and consume JSV services, to take advantage of the benefits you highlight? As you mention, mobile apps would be a great fit for a format like this.

    • mythz says:

      At the moment the only way to consume it on an iPhone is with MonoTouch.
      The unmanaged and static nature of Objective-C and its lack of reflection capabilities makes it a poor candidate for web services.
      You either need to provide a loosely typed collection of NSMutableArray / NSDictionary (which isn’t ideal) or a code-generated static proxy (which takes effort).
      Unfortunately I haven’t done either. It’s on the TODO list however I don’t envisage a lot of free time in the future – in the meantime I’m happy to accept pull-requests :)

  9. Mike says:

    quote: “The unmanaged and static nature of Objective-C and its lack of reflection capabilities makes it a poor candidate for web services.”

    I think you may have some incorrect information about Obj-C. It is actually dynamically typed and has at it’s core the ability to introspect objects and their properties in a much, much more efficient way than System.Reflection.

    • mythz says:

      I’m aware of its dynamic message dispatches but don’t see any reflective capabilities any more than that. I’m guessing there’s going to be no Eval, Emit or expression support?
      Do you have a link showing its reflective capabilities?

  10. Josh says:

    Did you take a look at Google Protocol Buffers and Facebook Thrift, before writing this? These formats are not self-descriptive, but they are tens of times faster in serialization/deserialization than XML in most of the supported platforms (Java/C++/Python on Linux and C# on .NET).

  11. aytharn says:

    @mytz:
    For the Reflective capabilities: please take a look at objc/runtime.h
    Anyway: Objective-C is simply NOT CAPABLE of creating static objects at all

Leave a Reply