ServiceStack eclipses jquip!

February 24th, 2013 by admin No comments »

This weekend I’m happy to find we passed an important internal milestone:

Where for the first time since its release, the 4+ year ServiceStack has finally surpassed my 3 week hobby js project jquip in followers!

What’s jquip?

For those that don’t know, jquip is a modularized version of jQuery optimized to have the lightest footprint possible whilst retaining the most useful parts of jQuery. It’s useful for bandwidth constrained environments where you don’t want an external jQuery reference (e.g. widgets, offline stand-alone web app) but still want to code against a pleasant jQuery-like API. We use it in the Careers 2.0 Apply Button widget that can be hosted on external sites and provides faster start times by being able to execute straight after the in-lined jquip. Another area jquip has proven useful is to power ServiceStack’s built-in MiniProfiler Web UI allowing it to work in offline environments without internet access. Embedding the full version of jQuery would’ve significantly increased the size of ServiceStack.dll by more than 100kb. This lets you develop your own lightweight offline single page apps that have access /ssr-jquip.all.js using nothing other than a ServiceStack.dll :)

Why is this important?

This is a major milestone for us as one of the reasons to get into Open Source is for your creations have a positive impact on the world and the number of downloads and watchers your project has are tangible metrics that can be used to measure the reach your software has. These metrics re-affirms that contributions continue to be useful to a wide audience motivating the continued dev efforts.

I’ve always thought going Open Source immortalizes your code and as a developer provides the best chance for your creations to go on living long after you’ve left this planet. It’s also immune from Market threats where not even your commercial competitors can kill it. Although both projects now have an equal number of followers, ServiceStack has received an order of magnitude more effort with more than 2,000+ commits since first moving to GitHub. The unfortunate reality is the OSS culture is relatively weak in .NET whose market share is primarily ruled by commercial entities. I attribute this to the deep roots the .NET culture has in the enterprise and the fear and stigma attached from straying from the prescribed default .NET stack into the many quality OSS alternatives. Unless we see a significant shift in .NET culture I expect the adoption for .NET OSS projects to continue to remain lack-lustre and see the top leading talent of web developers and start-ups continue to erode to the more popular JavaScript, Ruby and Python platforms and their thriving OSS ecosystems. Which is justifiable at this stage, if your motivation for contributing to Open Source is to make an impact and maximize reach and utility of your software, it’s hard to ignore the leading platforms which sees a considerable more OSS activity and code-sharing of OSS libraries and frameworks.

.NET is a quality platform, and C# / F# are semantically world-class designed languages with the fewest warts and a rich balance of intuitive features. They’re not always the most productive option (e.g. slow iteration times of ASP.NET) but they do have the richest IDE and tooling experience. I see the biggest threat to the platform is its waning adoption and network community effects of the strengthening alternative web platforms. The outlook is not all grim, e.g. xamarin.com is bucking the trend and pushing C# in uncharted territories as a growing platform for use in cross-platform mobile development allowing the same shared C# code-base and libraries to run on the leading iOS / Android mobile platforms.

How can you help?

Essentially the goal is to increase .NET OSS activity and anything you can contribute to this effort will have a net-positive effect, some examples of this might include:

  • Create more .NET OSS projects: Consider Open Sourcing your in-house internal libraries and frameworks (which if you think of the benefits of OSS makes more sense to be OSS). One of the few benefits of having a weak ecosystem is the opportunity to fill in the missing features! :) .NET could do with more love around NoSQL solutions, MQ’s, web tooling and libraries for alternative cloud providers, etc. These are great places to start.
  • Contribute to existing .NET OSS projects: Start with the ones you’re already using otherwise here’s the leading list of C# projects on GitHub you can work through.
  • Evaluate if .NET existing projects are a better fit: If you have the opportunity for your next project, consider spending some time to evaluate if more viable OSS alternatives exist that may provide a better fit for your current default closed-source stack.
  • Increase the collective knowledge around existing projects: i.e. blog, submit talks, help out on Stack Overflow, participate in mailing lists or JabbR. If you or your team has spent a lot of time implementing a solution for a popular use-case consider blogging about it, providing insight to others following in a similar path.
Currently C# stands as the 12th and F# the 43rd most popular languages on GitHub (the home of OSS), it would be great to see if we can all beat these numbers next year!

ServiceStack is on InfoQ! TL;DR and Links

January 8th, 2013 by admin No comments »

Adding to ServiceStack’s great run in 2012 I was approached by InfoQ to find some details about ServiceStack vs WCF, why we created it, it’s advantages over WCF and the philosophical differences in development styles and approach between ServiceStack and high-level Microsoft libraries and application frameworks. The resulting interview contains my thoughts on the best development and remote services practices, WCF’s architecture, its use of SOAP/WS-*, why heavy abstractions and artificial server-side programming models are not ideal, how pragmatic and simpler serialization formats are better suited for services, why message-based designs are preferred by most other frameworks, the state of .NET Open Source and much more.

The entire interview is quite long which ran to 8,000 words that as a result was split up and published in two articles. To make it easier to find and navigate the content, I’ve published a TL;DR summary, including related questions, table of contents and all embedded links used below:

Interview With Demis Bellot, Project Lead of ServiceStack – Part 1:

InfoQ: What problems do you see in Microsoft’s approach to Services and how does ServiceStack try to tackle it?

Interview With Demis Bellot, Project Lead of ServiceStack – Part 2

InfoQ: What exactly is a message-based Web service?
InfoQ: You recently introduced a razor engine making ServiceStack a more complete web framework than just a web services framework – what was the motivation behind that?
InfoQ: Are there any scenarios where you think WCF/Web API/MVC might be better suited than ServiceStack?
  • MVC better at large write-heavy server-generated websites
  • ServiceStack is optimizing for SPA’s like Backbone.js, AngularJS, Dart Web Components, etc
  • WebApi better choice for building server-side driven systems with HATEOS Restrictions
  • ServiceStack better choice for maximum re-use and utility of your services
  • MVP and Microsoft Gold Partners will want to continue to choose the prescribed Microsoft stacks
  • ServiceStack will optimize for alternative platforms (e.g. Mono), Clouds (EC2, Google CE), RDBMS (Sqlite, MySql, Pgsql, etc), Redis/NoSQL
InfoQ: Microsoft has collaborated with open source projects in the past (JQuery, NuGet for e.g.) and MS folks like Scott Hanselman seem quite open about adopting good open source solutions into the MS default stack wherever possible – do you foresee any such collaborations that could benefit the .NET community in general?
InfoQ: You made a comment recently on one of the forums – “I’m hoping next year to be even better, as I have a few things planned that should pull us away from the other frameworks” – would you elaborate what features you have in mind for the future?
ServiceStack set to make exciting new product announcements in 2013. Current road map includes:
  • Merging the Async branch and its async pipeline
  • Create new fast Async TCP Endpoints
    • Enable fast, native adapters for node.js and Dart server processes
  • Enable integration with more MQ Endpoints (i.e. RabbitMQ and ZeroMQ)
  • VS.NET Integration and our improved solution for WCF’s ‘Add Service Reference’
  • Integrated Development workflow and story for Mono/Linux
    • Enable automated deployment story to Amazon EC2 and Google Compute Engine clouds
  • Signed and Long-term stable commercial packages
  • Starter templates around popular Single Page App Stacks: Backbone.js, AngularJS, Yeoman and Dart
  • Starter templates for creating CRM and SharePoint-enabled support systems
  • Website re-design and improved documentation

If you have the time I recommend reading the entire interview, otherwise hopefully these bulleted points can help you find the bits you’re interested in sooner.

After 480 days uptime, lights out for old servicestack.net server

August 30th, 2012 by admin No comments »

From today, after having served its 484th day of uptime we’re now relenquishing the old servicestack.net server into a well-earned retirement.

Over 480 days uptime and 10M+ commands processed

Since its last restart it has processed more than 10+ Million redis operations resulting from ServiceStack’s Redis-powered live demos and our other websites we had that were using Redis for front-level caching.

It was a CentOS/Mono powered 2GB RAM dedicated server from leaseweb which we put to work to host the servicestack.net domain (inc. all the live demos) as well as the other side projects I had running e.g: ajaxstack.com, freelamusica.com.

Unfortunately time became its ultimate undoing, where newer spec’ed hardware and profileration of in-expensive hosting providers have now diluted its once great value.

The big move

Fortunately moving between Linux servers is easy and we were able to transfer to a different hosting provider in 1 weekend. Configuring Nginx and Apache has gotten a lot better where both now have “no touch” virtual host configuration and simple sym-links to decide which sites should be enabled.

Transferring the GB’s of website content was done with a single rsync command that completed in minutes, thanks to both servers having blindingly fast bandwidth.

WordPress migrations still painful on Linux

Most of the time ended up being spent on migrating this WordPress blog, where I had to manually create the new database, configure host settings, export and import the blog content and as well as hunt down and enable the website template and all the existing plugins. Its funny that despite being developed on the LAMP stack, it’s still easier to install WordPress on Windows than Linux.

Final Memento

I’ll share details of the new server in a future post. But for now I’ll leave this final screenshot of the RedisAdmin UI info page as reminder of the little work horse that was, captured in its final hours – showing its impressive redis-server stats :)

In cloud speak, that’s more than 99.99999% uptime!

Async, Cached Twitter API Proxy in F#

October 3rd, 2011 by mythz No comments »

Following on from showing how easy it was to create web services in F# with Service Stack, it’s now a good time to step it up a bit and do something useful to showcase some F#’s async goodness!

The final solution is available as a stand-alone HttpListener or an ASP.NET host which I just happen to have deployed earlier on a Linux server at http://www.servicestack.net/ftweetstack/. Here’s a couple example urls to see the stats of all my friends and followers:

http://www.servicestack.net/ftweetstack/followers/demisbellot
http://www.servicestack.net/ftweetstack/friends/demisbellot

Edit Screenshots added:

As I’m now getting rate limited by twitter, the live services have stopped working so I’ve included screenshots of the web service responses, reminiscent of happier times :)

[format=html]

[format=csv]

[format=json]

» Read more: Async, Cached Twitter API Proxy in F#

F# Web Services on any platform in and out of a web server!

September 28th, 2011 by mythz 1 comment »

Following on from my previous post where I took a first look at F# through my C# tinted eyes, I looked into some of the advantages of what this versatile language brings to a C# devs table. One of the major benefits I mentioned in passing but failed to explain in detail was it’s exceptionally good async story, which in many respects provides arguably the best async programming development experience to date on any platform.

F# and .NET Web Services

If F# only had to interface with other code as its inputs and outputs, the async world of F# is pristine and beautiful. Such is rarely the case so we’re stuck with having to interface with existing frameworks to provide these additional user facing services. Unfortunately the Web Services story for .NET isn’t pretty, where if for whatever reason you’re forced to use Microsoft-only technologies, you’re left with either having to deal with the WCF problem or extend MVC’s framework and roll your own REST-ful framework-in-a-framework yourself – which although less capable, I’ve witnessed to be a viable alternative providing a better iterative experience than WCF in the wild. Another web framework worth considering along similar lines, but developed in true Open Source spirit is NancyFX.

However if like me you like Web Services and still want to use a framework optimized for the task but consider WCF to be an anti-pattern promoting, over architected abstraction I’m happy to highlight a more elegant option in Service Stack which does the same job as WCF in allowing you to expose XML, JSON and SOAP services (as well as CSV, JSV & HTML formats) but does so cleanly, out-of-the-box, without any configuration required! It’s typed, DTO-first development model gets out of your way providing a friction-free dev experience that encourages the development of clean, best-practice web services – easily consumable from any client.

I should note the latest Web Service offering coming out of Redmond: WCF Web API is not be confused with WCF (even if the naming tries) does a much better job of ‘exposing HTTP and REST’ rather than hiding it like the original WCF framework upon which it is built. This may work better for you, although my preference for Open source solutions favouring light, DRY, typed, performance-focused API’s with the ability to run cross platform keeps me using and continue to recommend and maintain Service Stack for the foreseeable future.

Hello, F# World!

To get started we just need the Service Stack dlls that can be downloaded from GitHub. In the unzipped folder you can spark up your favourite text editor and write your first Hello, World web service which looks like [gist]:

open System
open ServiceStack.ServiceHost
open ServiceStack.WebHost.Endpoints

type Hello = { mutable Name: string; }
type HelloResponse = { mutable Result: string; }
type HelloService() =
    interface IService<Hello> with
        member this.Execute (req:Hello) = { Result = "Hello, " + req.Name } :> Object

//Define the Web Services AppHost
type AppHost =
    inherit AppHostHttpListenerBase
    new() = { inherit AppHostHttpListenerBase("Hello F# Services", typeof<HelloService>.Assembly) }
    override this.Configure container =
        base.Routes
            .Add<Hello>("/hello")
            .Add<Hello>("/hello/{Name}") |> ignore

//Run it!
[<EntryPoint>]
let main args =
    let host = if args.Length = 0 then "http://*:8080/" else args.[0]
    printfn "listening on %s ..." host
    let appHost = new AppHost()
    appHost.Init()
    appHost.Start host
    Console.ReadLine() |> ignore
    0

The rest of the code except for the 5 lines of code needed to create the actual service need only be written once which is just used to setup the AppHost and run it, in this case as a Console Application.

Great, once you copy all that in, and hit save using the filename Hello.fs – you can compile with the 1-liner:

fsharpc -r:ServiceStack.Common.dll -r:ServiceStack.Interfaces.dll -r:ServiceStack.Text.dll -r:ServiceStack.dll Hello.fs

Which cuts you a nice Hello.exe that you can double-click to run on Windows, otherwise run on OSX or Linux with:

sudo mono Hello.exe

And Voila! your stand alone F# self-hosting Web Service is ready to serve you at: http://localhost:8080/hello/FSharp!

Convert into an ASP.NET Web Service

To convert this into an ASP.NET application we just need to do some minor tweaks add the appropriate files to deal with the nuances of having an ASP.NET application. For this we need to venture back to a time before VS.NET templates existed that did all this for us.
Note: All files including HelloAsp.fs, Global.asax and Web.Config files used in the walkthrough below are available on this [gist].

First lets take a copy of the file and call it HelloAsp.fs

1. As we want to create a .NET assembly we wrap the entire script in a namespace adding the declaration on top and indenting the rest of the file

namespace HelloFSharp

2. Change the AppHost from a Http Listener to an ASP.NET AppHost by removing the word HttpListener from AppHostHttpListenerBase e.g:

    type AppHost =
        inherit AppHostBase
        new() = { inherit AppHostBase("Hello F# Services", typeof<HelloService>.Assembly) }
        override this.Configure container =
            base.Routes
                .Add<Hello>("/hello")
                .Add<Hello>("/hello/{Name}") |> ignore

3. Replace starting the AppHost from the command line with initialising it from HttpApplication Application_Start() event, i.e:

    type Global =
        inherit System.Web.HttpApplication
        new() = { }
        member x.Application_Start() =
            let appHost = new AppHost()
            appHost.Init()

4. Create the Global.asax stub to point it to the Global type, i.e:

echo "<%@ Application Inherits="HelloFSharp.Global" %>" >> Global.asax

5. Compile HelloAsp.fs into a library since its now an ASP.NET app instead of a stand-alone executable:

fsharpc -r:ServiceStack.Common.dll -r:ServiceStack.Interfaces.dll -r:ServiceStack.Text.dll -r:ServiceStack.dll --target:library HelloAsp.fs

6. make a bin directory and copy all the dlls into it:

mkdir bin && cp *.dll bin

7. Copy the desired Web.config mapping from Hello World Tutorial

<?xml version="1.0"?>
<configuration>
<system.web>
  <httpHandlers>
    <add path="*" type="ServiceStack.WebHost.Endpoints.ServiceStackHttpHandlerFactory, ServiceStack" verb="*"/>
  </httpHandlers>
</system.web>
<!-- Required for IIS 7.0 -->
<system.webServer>
  <handlers>
    <add path="*" name="ServiceStack.Factory" type="ServiceStack.WebHost.Endpoints.ServiceStackHttpHandlerFactory, ServiceStack" verb="*" preCondition="integratedMode" resourceType="Unspecified" allowPathInfo="true" />
  </handlers>
</system.webServer>
</configuration>

8. Run the .NET 4.0 version of xsp in the current directory

xsp4

If all went well you should see the following output:

Listening on address: 0.0.0.0
Root directory: /Users/mythz/src/fsharp/fsharpstack
Listening on port: 8080 (non-secure)
Hit Return to stop the server.

and you’ll be able to hit the service again using the same url at http://localhost:8080/hello/FSharp!

You now have a fully-fledged ServiceStack ASP.NET application that can be hosted in any ASP.NET compatible web server e.g:

  • IIS
  • Apache + mod_mono
  • Nginx + MonoFastCGI
  • XSP
  • WebDevServer.exe

Now that you know how to create functional F# web services, you can read more about the goodies you get for free when using Service Stack, complete with live links to all those juicy XML, JSON, SOAP, HTML, CSV and JSV Formats check out: http://www.servicestack.net/ServiceStack.Hello/

Next – Async Goodness!

With the basics out of the way tomorrow we’ll tackle a more complex service taking advantage of F#’s async and parallel good ness to provide nice high-level aggregate features off twitters public API.

Stay Tuned!

A Look at F# from C#’s corner

September 27th, 2011 by mythz 1 comment »

For a while now I’ve been hearing many great things that have been coming out of Microsoft Research’s popular .NET language F#. For the unfamiliar, F# is a strongly-typed, functional-based language for .NET – originally created by Don Syme (better known to most as the father of generics in .NET) it has now become a fully supported language in .NET with soon to be first-class support in VS.NET.

Despite being deeply inspired by its functional roots, it stands out from other languages in that it also supports imperative and OOP paradigms as well. Boasting both interactive and compile modes, it’s been holding over C# lately, sporting a more succinct syntax and already comes complete with features like  async workflows and an interactive mode we wont see in C# until V.Next.

The announcement of F# 3.0 pushes the envelope even further where the new Type Providers promises to be even more productive by allowing you to build against a strong-typed API (with intelli-sense) against a runtime datasource.

So not wanting to fall too far behind a good thing I’ve made F# on this years must-learn-list. Before looking at F# I have to admit I thought C# was close to the perfect server-side language with its biggest weaknesses just being the lack of string interpolation and Go’s-like interfaces. Given that, I was quite surprised to find how much more elegant F# is in a number of ways:

Type Inference to the Max

Where some C# developers are still reluctant to use var in their works, F# takes type inference to the extreme where you can effectively omit type declarations whenever the type is not ambiguous and can be safely inferred.

Less code, the better

I’m a firm believer a smaller code base is a good one and we should be striving for a DRY, tight code-base where the solution permits. Having less code means there is less to maintain and less chance for something to go wrong, where a high signal-to-noise ratio is generally more readable as you’re able to focus more on the solution and less on the needless abstractions that get in the way.

In addition to type inference, F# has many features to tackle code boat including:

Out of all its features I believe what promotes the least code bloat is the quality of F#’s community (see: The Python Paradox) and its preference to simple, elegant composable solutions.  This is contrast to C#’s abstraction fetish it copied from Java and its relentless need to cater for the lowest common Drag n’ Drop developer and impress them with 5 minute demos. Although it’s a subject for another post, this rarely leaves us with quality frameworks or APIs.

Functional vs Imperative

Even though F# allows for programming in procedural, Object Orientated and functional paradigms, its roots and optimized syntax lends itself towards functional-style of programming. In many ways functional programming provides more elegant and robust solutions. Luca Bolognese explains this best in his excellent Intro to F#, where the instincts of a C# programmer attempts to solve a problem imperatively resulting in mutable variables and disjointed logic allowing for moving parts to go wrong.  In his example Luca uses the ‘sum of squares’ as an example where the budding C# developer would approach it into something like this:

public int Square(int num)
{
	return num * num;
}

public int SumOfSquares(IEnumerable<int> list)
{
	var sum = 0;
	foreach (var num in list)
	{
		sum += Square(num);
	}
	return sum;
}

var nums  = Enumerable.Range(1,100);
Console.WriteLine("Sum of squares of 1-100: {0}", SumOfSquares(nums));

The F# functional approach would lend it more to the original question, i.e. square the numbers, then sum them:

let squares x = x * x
let sumOfSquares nums =
	nums
	|> Seq.map squares
	|> Seq.sum

printfn "Sum of squares of 1-100: %d" (sumOfSquares [1..100])

Although oddly enough despite its already succinctness, it can be even further reduced to:

let sumOfSquares nums = nums |> Seq.sumBy squares

I recommend watching the rest of Luca’s video as he goes on to show how the F# solution lends itself to easy parallelization, without interrupting the existing flow of logic.

In a real world problems closer to home, @ayende recently posted one of his interview questions online asking for example solutions to calculate israels tax. As expected most C# solutions were similarly written the same way complete with mutable variables, tightly coupled solutions – many of them having custom types and other artefacts.  Comparatively the few F# solutions posted had significantly less code, yet was easier to read and maintain.

My attempt at solving this problem in a purely functional style resulted in this [gist]:

let taxOf salary taxRates =
	((0m,0)::taxRates, taxRates)
		||> Seq.zip
		 |> Seq.map(fun ((_, prevBand),(rate, band)) -> (prevBand, rate, band))
		 |> Seq.sumBy(fun (prevBand, rate, band) ->
			match salary with
				| x when x < prevBand -> 0m
				| x when x > band -> decimal(band - prevBand) * rate
				| x -> decimal(x - prevBand) * rate
		    )

let israelTaxRates = [
	0.10m, 5070;
	0.14m, 8660;
	0.23m, 14070;
	0.30m, 21240;
	0.33m, 40230;
	0.45m, System.Int32.MaxValue]

let taxOfIsrael salary = israelTaxRates |> taxOf salary

//Usage: taxOfIsrael 5800

The nice aspects of this solution was having the tax rates and bands in an easily readable and maintainable collection optimized for the reader (like all other F# solutions) separate from its implementation. The internal logic is neatly coupled together into 3 readable scenarios, making it easy to work out how the tax was calculated.

Another nice feature is being able to easily combine the implementation and input tax rates to create a high order taxOfIsrael function that ultimately performs the task. This in-built ability to curry functions makes functional composition a breeze and after using it for a while I can quickly see how it’s more elegant to OOP programming style in a lot of areas.

Interactive Mode

Inside Visual Studio 2010 (or by using fsi on the command line) is F# Interactive mode which works very much like Ruby’s irb or  Pythons interactive mode, it’s just rare to see this from a typed .NET language, although Boo was likely the first to do this.

F# open and F# everywhere!

Despite F# being the latest creation forged in the deep trenches of Microsoft’s R&D department, F# is surprisingly and arguably the most open of all of Microsofts languages with their entire implementation available on GitHub and free to use, released under the liberal and OSS approved Apache 2.0 licence! This wouldn’t mean much if it didn’t immediately work elsewhere however Don Syme and his team have done a good job actively supporting Mono, going as far as reporting blocking Mono bugs, ensuring it continues to work flawlessly. It’s even being currently distributed with the latest release of Mono on OSX.

Being this open is a treat, we can finally build GTK desktop applications with a typed, fast functional language using Open Source components end-to-end!

I’m actually surprised how well it works where I’m doing all current F# development on an OSX Macbook Air, deploying my finished solutions onto http://www.servicestack.net CentOS linux server for hosting.

Getting Started with F#

Although not mentioned here, it should be noted F# is great for concurrent programming tasks as well. It is also particular good at creating composable asynchronous work flows – which happened to be the source of inspiration for C#’s 5.0 async/await feature.

Getting started with F# is easy where if you have Visual Studio 2010 – you’ve already got it! Just create an F# project and your good to go. For all other platforms follow this link.

For the newbie budding developer, I recommend the following learning resources:

Blogs

Next up: Easy Self Hosted, Cross Platform F# Web Services

Although not a surprise for a .NET language, I’m happy to report F# works flawlessly with Service Stack where its thin API doesn’t get in F#’s way – allowing it to easily create elegant self-hosted solutions. In my next instalment I’ll show how you can easily create an async+parallel cached twitter proxy that works cross platform on Windows/OSX and Linux in =~ 100 LOC.

First day on the job, after 1 crazy week!

August 26th, 2011 by mythz No comments »

Wow what a week! it’s been an eventful one to remember: there are Rebellions overthrowing dictatorships in Libya, Earthquakes in Washington with tremors being felt in New York and hurricane Irene swirling around keen on paying us a visit anytime now. All this just setting the stage for the sad news I hoped not to hear for many more years with Mr. Apple (aka Steve Jobs) bowing out on top, relinquishing command as king of the tech world.

Meanwhile all this is happening I’ve undergone some radical lifestyle changes as well with just having just moved to the city of New York after a 2 month hiatus from programming, travelling the wrong way round the world from London. I’m now currently settling into my new apartment in Brooklyn whilst just having finished the first day at my new job after having joined the talented team at: http://stackexchange.com!

My new, New York life is largely thanks to the opportunity I received from the eloquent Joel Spolsky. Which was pretty surreal in itself , given I’ve been following and enjoying his entertaining articles on his world famous blog since the dawn of my computing career >10 years ago!

My first call to action at StackExchange will be helping “Stand Up Matt” @clipperhouse and “Mad Typist” @JasonPunyon (my first impressions – sorry guys! :) ) enhance the new StackOverflow Careers 2.0 site – Something I’ll be looking forward to as It’ll have the potential to help and reach a very large audience. This is in stark contrast to the standard enterprise developer positions I’ve found myself in over the years – developing custom backend solutions that are usually only destined to support a handful of people.

Day 1

With having survived Day 1 (and not having done much), the biggest change I’ve noticed so far is the view. Where back in London I had a quiet desk next to a peaceful, lifeless cemetery where on a winters day I’m greeted with a ghostly calming atmosphere:

In New York that image is a distant memory where sitting next to a window on the 26th floor, neighbouring skyscrapers hit you in the face:

Although this changes very quickly with the bit of rain today, you still get a ghostly picture but instead of snow tipped tombstones – it’s the city of New Jersey!

That’s about it for my first day, I’m told my second will be even better with some in-house Team StackOverflow beer festival on Friday to close off the week!

Dark Pastel Visual Studio Theme

May 26th, 2011 by mythz No comments »

Since I’m pretty happy with the Dark Visual Studio Theme I’ve been rolling for a long time with I thought I’d share it with the wider .NET world so others can bask in the glow of an eye friendly dark theme when they’re doing their midnight code runs!

I’ve had it for so long that I’m no longer sure where it was originally based, but looking at Scott Hanslemans VS.NET theme gallery it looks like it’s a mix of Rob Conery’s TextMate and John Lam’s Vibrant Ink.
Over the years where I’ve felt the contrast could be improved, I’ve spiced it up with my own unique blend of colours from the pastel famiglia palette.

And since this theme isn’t complete without a name I’m calling it….

Dark Pastel

Here’s a code screenshot:

C# code screenshot

Some Web.config love:

Web.config screenshot

A splash of HTML:

HTML screenshot

And the all important download.

.

Enjoy!

REST Media Types, SOAP, Heavy frameworks and Literate Programming

April 17th, 2011 by mythz No comments »

Obviously the title of this post doesn’t make sense to be all in same article, yet I’ve still managed to carry an underlying theme interlinking the above topics : -). Effectively I’m hoping to encourage the pursuit of end user value over and above all else.

Disadvantages of Custom Mediatypes

Although I’m going to focus on Custom Media Types in this instance (since it reflects my most recent public discussion), the essence of my argument can be applied to many other areas, protocols, libraries, patterns, etc being advocated in software development today.

The main problem I have with Custom Media Types, is the same I have with any other technologies asking me to devote more effort to produce to what I ultimately conceive as a less valueable output. Where the only time when it proves to be beneficial over raw canonical data forms is when the consuming client has built-in support for the custom media format saving the integration point since the client already understands how to process the response.

Although this post may contain a critique on certain facets of REST, I’m by no means advocating against it as overall it’s a far superior more elegant approach to designing services than the SOAP and WS-* stack (which I’m happy to see now starting to slowly fade out of existence). In this light I instead consider myself more a devils advocate providing critique on parts which provide minimal value yet require significant more effort to implement. So consider this a public service announcement as due to the taboo nature of REST you’re unlikely to find many posts like this critiquing it which is partly my motivation for writing it.

There are times when custom mediatypes are useful however what’s not being discussed in these cases (and what I continue to have issue with) is the amount of airtime given to what I consider a small % of applicability and real-world use. This has a tendency to result in a significant body of knowledge flowing around new developers information streams and if they didn’t know any better would think that this is the right way to go about developing web services. What we’re going to end up with are solutions looking for problems and a swath of invented custom formats – more then there deserves to be. Given the unhealthy amount of coverage given to REST frameworks used to flex different parts of REST theory, I fully expect solutions developed just because developers want to obtain full-compliance REST reputation badges implementing non-trivial CONNEG + Custom Media Types multi-hop solutions when far simpler, more direct solutions can elegantly solve the same problem space. Of course I fully expect people to argue that a superior solution can be achieved even with full-compliance of REST HATEOS restrictions, and I simply want to re-iterate for developers to consider, to understand and weigh all options, and that Getting Things Done in the minimum LOC and effort possible is also a worthy goal to strive for, as when we get older, time becomes more precious and being able to hone this skill will ultimately pay dividends later : -)

Web Services should be optimized for Data Formats

In my 10+ years as a backend systems developer, I can count on one hand the number of times when creating custom media types has proven beneficial or was a customer requirement. The majority of my time is spent developing systems with a primary purpose of servicing clients, thick, thin or otherwise. In my web services world view, not all media types are created equal and its the popular data formats XML, JSON, SOAP, CSV, etc  that are ultimately exposed by web services and consumed by its clients. It’s these formats I believe Web Service Frameworks should be optimized for as they provide the most utility, re-usability and value.

As expected my real-world experience is also the primary influence on http://www.servicestack.net where the above data formats are built-in and are configured to work with all your services automatically. Your C# logic remains DRY and is fused with the expected HTTP result providing a free logic block to be able to return any C# POCO or throw any C# Exception. At the same time making all services available on all above data formats. These are the traits I’ve found to be the most productive and ultimately provide a friction-free development experience.

The other notable mediatypes which are actually more prominent than web services data formats are the media types understood by web browsers. By this I mean HTML/CSS/JavaScript /img/vids/etc. The development of web sites have long been the focus for most development platforms where websites have now become the most important deployment target for most new applications. These web formats are first class formats where entire web frameworks are built around and optimized for, which you should be using instead of Web Service Frameworks.

Custom formats are still important, just not of primary importance

Despite my criticisms, I still think the production of custom formats is important when it’s required, and ServiceStack actually has very good support for it with one of the DRY-est API’s you’re likely to see in accomplishing this task. I just don’t personally give prominence to this functionality since it’s not where I believe web service developers should be speding their time in implementing solutions with. Although for the interested this is how ServiceStack implements the VCard Format. Where its non-invasive layered API makes it trivial to support any custom media format that is well supported since its the same integrated pipeline the newer built-in HTML and CSV formats are using.

Note: at the lowest HTTP API level in every web framework (e.g. IHttpHandler, HttpServletRequest, etc) this has always been possible, so all web service frameworks are showcasing its high-level API to achieve this end. Some frameworks make this functionality available using APIs mapping directly to REST concepts, ServiceStack however prefers to make it as DRY, typed and expressive as possible.

Data formats the preferred choice in SOA systems

A key objective in SOA systems and web services today is to servicify and free your data making it as universally available and re-usable as possible. Given these objectives it is common to see SOA systems today make use of HTTP for their transport protocol and utilizing either XML or JSON for their data formats. This is largely because the data formats contain the essence of your data where should they need to at a later date, your clients can use this canonical form of your data and create the custom media types themselves. The same is not true the other way around where if the only format your service provided were custom media formats (e.g. the VCard format) it not only takes much more effort to extract the raw data, but the specific media type will only contain the subset of your contact data as supported by the format. So if your developers only have an option to deliver 1 format for your resource, you’re best to make it a raw data format since the development of other formats can still be achieved by your clients long after the developers have moved on.

In summary you should only be accessing the specific VCard format (or other specific mediatype) only if your client already has built-in support for it. Since it requires more effort on the client and server to implement and access otherwise.

It is therefore important to remain vigilant and keep the 80/20 Rule in mind (or in the case of custom media formats the 95/5 rule : -) and treat custom media types as complimentary formats that you should only add in addition to your data formats. When development of your SOA services have finished you want to ensure it’s data is as accessible and re-usable as possible in the formats that provide the most utility to your current and future consumers of your services, i.e. the data formats.

The guiding value light

With the current demise of SOAPREST is making it’s way back into the forefront of IT minds.
The value of REST has always been available in HTTP/1.1 since its inception but between the marketing and committees pushing their own agendas, the Enterprise development world was largely oblivious to its possibilities as they were taught if your data didn’t come in a XML decorated envelope it’s not a real web service.

Its come-back can largely be attributed to developers and companies ignoring the blind advice of big iron Company’s and instead discovered that they are better able to provide value by embracing the inherent properties in HTTP rather than the friction-encumbered dev experience of building SOAP web services.

Displacing big iron-backed monolithic standards is hard, it takes courageous companies willing to discard current enterprise teachings who are able to re-think the essence of what they’re trying to achieve.

Focusing on max value and min effort

It’s therefore important we don’t repeat the same mistakes and don’t lose sight of what the value proposition on the technologies we ultimately choose to adopt, can provide. I’m constantly seeing blind embracing of software process, approach or technology where at the most extreme levels its treated akin to a religion, instead of what it should be: productive tools serving a means to an end.

HATEOS

REST at its extreme conformance imposes some hefty HATEOS restrictions which coupled with most Hyper media driven applications fixation on XML, does not leave you with a very productive window to develop in. If you see real-world value in conforming to these restrictions then by all means adopt them, I personally don’t, and since collecting stamps of approvals from the REST community is not my #1 priority, I routinely discard they exist.

JSON vs XML

XML is another one of those funny technologies handed down from standard committees as the holy grail data format.  Unfortunately this only holds true when you don’t have to develop with it, as it’s a poor programattic fit, making it harder to work with than JSON, that’s both more verbose and slower as well. A good reference that explores these differences in more detail is Douglas Crockford’s entertaining talk on Heresy & Heretical Open Source.

XML as a interoperable structured data format has its place, at the same time I’m finding cases where it would be the ideal data format harder to find. I no longer store it in databases, make use of it for efficient messaging or let it go near my mobile or web apps.

Choosing the right technology

The way I defend myself from adopting the wrong technology is to take a cynical view against all promises, hype and marketing materials. I usually form a picture of the easiest way to develop something and use that as a benchmark to compare other solutions against. The onus then becomes on other frameworks to prove their value by either reducing the amount of effort or code required or in someway enhancing re-usability, performance or promote better maintainability by reducing coupling. If it does not provide any realizable real-world value I don’t spend too much time researching it and continue what I was doing before this Hot Tech Tip landed on my lap.

Basically don’t take any advice at face value, understand the benefits and amount of effort to achieve the desired outcome of each approach and choose the one that offers the maximum end user utility given the least effort to implement. If you don’t understand the full value that your architecture/approach/framework is providing, you won’t be able to maximize its benefits and will soon find you’ve inadvertently subscribed to a solution looking for a problem.

So if you’re spending a lot of time fighting with your framework just to try to get it to prescribe to some artificial standard or religion it might be time to explore other approaches/technologies. When evaluating technologies think about the concept of Last principles where you first look at the ultimate end-user result you want to deliver, then look backwards at the tools, technoligies and frameworks that will best meet this end with the least effort and maximum efficiency.

Heavy weight libraries and frameworks

I personally believe that the Java/.NET worlds are plagued with leaky, heavy abstractions that introduce more friction an inertia that they’re trying to solve. The pain in configuring these monstrosities can be attributed to:

“So much complexity in software comes from trying to make one thing do two things.”

– Ryan Singer

Whilst the un-intended, unknown or magic behaviour I frequently see at runtime whilst developing in said frameworks I’m attributing to its poor, leaky abstractions.

Prefer thin abstractions promoting DRY code

In recent years I’ve been more active in proactively shunning heavy frameworks and look to libraries allowing me to write less code and configuration. Configuration is actually a form of inferior code which happens to be more awkward to define object graphs with than your general purpose programming language. What’s left is a fragile, un-typed mess that you can usually only determine correctness at runtime.

I have many reasons to prefer shorter code-bases which follow my general mantra of “You never have to maintain any line of code you didn’t write”. However here a couple of my favourite quotes I’ve collected on the subject that put this more eloquently than I:

“I happen to hold a hard-won minority opinion about code bases. In particular I believe, quite staunchly I might add, that the worst thing that can happen to a code base is size.”

Steve Yegge

“A programmer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.”

– Antoine St Exupery

With that said, my general preference is for light-weight development libraries and frameworks with thin abstractions allowing me to capture my logic’s intent as DRY as possible. Usually this means these libraries have a low artificial construct / concept count whilst remaining expressive at its core.

The rise of Ruby and Literate Programming

Despite its poor runtime performance and no sight of any corporate backing, Ruby has been making deep inroads into developers mindshare simply because it allows developers to be more productive. Although the language itself is quite expressive, I’m attributing it’s large productivity gains on the communities focus on Simplicity, Convention, DRY-ness and it’s ultimate goal of its ability to compress logic into pseudo code that runs.
In this goal, it seems either by accident or in a way that was cleverly conceived by the languages author it appears that much of Rubys libraries follow a deep symmetry to Donald Knuth’s Literate programming which enables programmers to develop programs in the order demanded by the logic and flow of their thoughts.

Code as literature

“Meaning belongs to a text or to a program to the extent that it acts upon intelligence in a particular way.”

We say that a textual work is “literary” if it contains a high density of meaning.

The above quotes are from @jashkenas enlightening talk at Ruby’s SunnyConf 2010 exploring the subject.

Literate programming is an intriguing topic that not only looks to enhance developer productivity but at the same time increases developer enjoyment and satisfaction. I can definitely see its appeal as it seems like a breadth of fresh air especially if you spend your time fighting heavy frameworks in your day job. Having said that Ruby’s lack of good performance characteristics which I consider important keeps me from using the language in any capacity.

JavaScript’s star new language – CoffeeScript

Luckily CoffeeScript is a hot new language that looks to embrace this vision and since its built on JavaScript, can take advantage of the arms race going on amongst browser vendors in recent years trying to make JavaScript as fast as possible.

Unlike other JavaScript frameworks tackling this domain, it’s not interpreted at runtime and simply compiles 1:1 with efficient JavaScript (in some cases more efficient than a developer would write).

This gives it some unique qualities where it is able to run in a browser without any plugins and thanks to node.js the same CoffeeScript code can run very efficiently on servers as well.

We’re now starting to see this powerfull and expressive combination powering new servers like Ruby’s new Pow Rack server for OSX.
Its recent inclusion in the default Rails install is causing the biggest rift I recall in recent times for a single line of config change.

So while everyone may not be in love with it yet, they at least have an opinion on it :)

What’s yours?

The App Store bubble: How much does an App earn today?

September 15th, 2010 by mythz 15 comments »

The rise and fall of Internet riches

Within the last 15 years on the rise of the Internet we have seen many a phenomena gracing technology circles promising great riches to early adopters and investors. It first started out with a gold rush land grab for domain names where in this era a 3-letter domain name like sex.com was able to earn millions a year just by providing a holding page hosting adult-related advertising banners and was eventually sold for 14M.  Then came the dotcom boom where anything Internet related were the most sought after prize on the stock market where traffic and the size of your user base was the hard currency of the day and was rewarded with over-inflated valuations never mind that many had no visible path to profitability. The preferred sport of many startups at the time was who had the highest burn-rate which were fueled by expensive Aeron chairs and Sparc servers.

The Bigger They Come, The Harder They Fall

But just as quick as the riches came to dotcom companies, they vanished again almost entirely within one weekend (which this author remembers quite well as an eager investor on the friday before black monday). Eventually the market corrected itself and only businesses with viable business models were the focus of cautious investors whose fingers were still smarting from the bursting of their dotcom investments.

In time faith was restored in the online advertising market spurred largely by Google’s foray into the area of targeted advertising. Confidence was restored in Internet companies and a smaller bubble is starting to emerge that many casual observers are labelling Web 2.0.

Cometh the App Store, Cometh the riches again

Out of the all the events to show up on the technology radar recently, none has been as exciting and show as much promise as the introduction of the App Store by Apple. With the introduction of their revolutionary smart phone, Apple for the first time put the power of a desktop operating system in the palms of technophiles (with above-average discretionary incomes). Although its potential wasn’t realized until Apple put the first version of the iPhone SDK in developers hands and opened the much publicized App Store on July 10, 2008. Within months of its introduction stories or riches started to be told of hobbyist programmers earning $600,000 in just 1-month enough to quit their full-time job to pursue a lucrative career in developing mobile apps for the App store. Soon enough a new economy was created and in pretty short order a growing number of developers made it to the App store millionaire club. This brings us to where we are today where as of September 1, 2010 there were 250,000 third-party applications with over 6.5 billion total downloads generating its developers over USD $1 billion in revenue.

The reports of App Stores death are greatly exaggerated

With now over 250,000 applications available for sale and many of them free many people have chimed in with their opinions some proclaiming the app bubble is bursting and the path to easy riches is over while others maintain the App store maintains a fruitful future. Unfortunately the ubiquity of the Internet has made opinions cheap and plentiful where it looks like anyone with a keyboard can pluck 2 numbers out of the air to show a downward trend.

An App Store Developers story…

As a developer on the App Store I wanted to weigh in on the state of the App store with some empirical evidence of my own as it evolves. My story dates back 18 months ago when it took me 3 weeks to learn enough Objective-C to submit my first app. Within a couple of weeks, inspired by the success of iFart apps I put aside my morals and pursuit of software excellence and ended up submitting my own cookie cutter apps. Although my own fart apps didn’t do nearly as well as notable others my translator app earned a noticeable side income where it managed to pay off its development effort many times over. Unfortunately around this time I joined my first startup and between the all nighters and 12 hour work days my free time was sapped and my once App making machine days were over. That is until my recent resignation which saw the return of my free time and I was finally able to open xcode again after an 18 month hiatus. After tying up some loose ends on my open source project I got to work on a new App.

How much does an App built today earn?

So that Non-App developers can get an idea of what they can expect when developing an App today, I’m going to be completely open and transparent about the development and the financial results of my latest App that has just been approved today (14th Sept UK). I will keep this blog updated with live financial results as I get them.

Coming up with an idea for an App

Developing a successful app in this age can be a tricky balance to get right where the subject and appeal of your app is every much as important as its features and quality. Because of this, I’m not a true believer of investing a lot of time in developing an App before I see some ROI. So I’m always on the lookout for small, quick wins with short development cycles. It just so happen that at the time I was seeking inspiration for my new app, I also spent a large portion of my free time playing Words with friends – an enjoyable, network-enabled SCRABBLE® clone. Although many would see my poor scrabbling skills the result of a long absence from participating in any academic curriculum, I saw it as an opportunity to build an App to improve my scrabble game! (read: cheat) After a quick sanity check to make sure there weren’t a hundred free apps already available on the App store that did the same thing: Word Mate was born – a collection of word finding utilities that help you play SCRABBLE® like a computer :-)

Development efforts and total cost

Word Mate took me roughly a week full-time to develop which included downloading all the latest SDK’s and tools, a bit of reading designed to re-ignite my love with the Objective-C language and its Cocoa inspired frameworks as well as creating all the App and website graphics. As the website is freely hosted on Google App Engine and the only 3rd party icons used were free, no costs were incurred into making this app other than the $99/year iOS Developer Program fee and my own time.

App Store submission process

Although the language and tools haven’t improved much, the app submission and code-signing process has improved significantly. When it was first released, code-signing your app so it can be submitted to the App store was considered a black-art which thanks to my careful literal following of every word in an online tutorial caused me to be stuck with the embarrassing title of ‘iPhone Developer: Demis Bellot’ for the name of my developer profile. Anyway I submitted my app on the 1st September and it was only approved at midnight on the 13th September. Where the QA staff that Mr Jobs says approves most apps within 7 days are? I don’t know – I’ve never had an any of my apps approved within 2 weeks.

Regardless today will be my first full day on the App Store market. Like all my apps, Word Mate will be free for the first week. I like to do this as it gives a chance for my friends and anyone else who wants one a chance to download my app for free. Making an App for free and then charging for it will also give you an idea of the Free vs Premium download ratio. From the experience of my first app the difference was about 6-9% where for every 100 free apps that were downloaded (when it was free) only 6-9 were paid for (when I started charging USD $0.99 for it). It will be interesting to see if that trends continues with Word Mate.

Live Financial Results

I’m going to maintain a live list of results on the table below, so you can check back on this post to see how it develops. iTunes Connect (where we login to get our sales reports) doesn’t publish daily results until 12pm GMT the next day. You can follow @demisbellot on twitter to find out as soon as the latest results have been added.

Period Ending Cost Qty Sold Earnings
14th Sept $0.00 118 $0.00
15th Sept $0.00 118 $0.00
16th Sept $0.00 41 $0.00
17th Sept $0.00 28 $0.00
18th Sept $0.00 24 $0.00
19th Sept $0.00 21 $0.00
20th Sept $0.99 15 free/1 $0.99
21st Sept $0.99 1 $0.99
22nd Sept $0.99 0 $0.00
23rd Sept $0.99 2 $1.98
20-26th Sept $0.99 4 $3.96
27 Sept-3rd Oct $0.99 5 $4.95
4th-10th Oct $0.99 2 $1.98
11th-17th Oct $0.99 3 $2.97
18th-24th Oct $0.99 2 $1.98
25th-31st Oct $0.99 5 $4.95
1st-28th Nov $0.99 5 $4.95

*NOTE: Word Mate will be free until 20 Sept 2010, so if you think you might find this app useful now would be a good time to get it for free :-)

Follow the conversation

A discussion thread has started in hacker news which you can get involved in at:

SCRABBLE® is a registered trademark of Hasbro, Inc. in the United States and Canada, and Mattel, inc. elsewhere. I’m not affiliated with either company.

Update 23/09/210

It’s just come to my attention that I may have stuffed up the SEO of my app in the App Store as Word Mate can’t be found with the words ‘Word Finder’ or ‘Word Cheat’  2 of the most popular searches for finding like apps. Unfortunately Apple wont let you change it after you’ve added it so I’m going to have to submit a new app. Unfortunately SEO is just as important as having a quality app as if users can’t find it, they can’t buy it.