Highlights from Yow! 2014 - Day 1

A couple of weeks ago I was lucky enough to attend the Yow! Developer Conference in Melbourne. Here's a summary of my favourite talks from the first day.

'How Did We End Up Here?' - Martin Thompson and Todd L. Montgomery

This was the opening keynote, and it was a bit of a rant on why so many software projects fail, despite all the developments in software engineering over the last few decades. I took a tonne of notes from this talk, but here are my favourite bits:

  • If selling software isn't your core business, open source your code - I really like this concept. The idea is that you won't be losing anything, and you can make gains on maintenance (by having others contribute features and fixes), and recruitment (by talented devs seeing you involved in OSS). However, you won't get it for free just by pushing everything to github tomorrow. The code and docs need to be good enough that people will be interested in it. Although, that added motivator of quality is in itself a good thing!

  • People in IT shouldn't refer to 'the business' like some separate entity - IT needs to be deeply integrated with its stakeholders, rather than segregated from it, and the language that we use affects our behaviour.

  • Synchronous communication ruins distributed systems - Thinking asynchronously is hard, but doing so frees you from the mess of timeouts and retries. Immutable events and eventual consistency are your friends.

  • Development cost is always less than the opportunity cost of not developing, or developing slowly - So optimise your business for speed of development, and you'll also come out on top in terms of cost.

'Types vs Tests : An Epic Battle?' - Amanda Laucher

Amanda resisted the temptation of flamebait (mostly), giving a very balanced analysis of the pros and cons of type systems and testing. The two key points that I took away from it were:

  • A strong type system makes certain types of tests redundant - If a function takes an integer type that can only hold values from 1 to 10, then you don't need to test what happens when you pass it 11. The compiler makes illegal states unrepresentable.

  • TDD is still helpful for driving out your design - Amanda said that while she definitely leans towards the type end of the spectrum, she still finds that a test-first approach consistently produces better architectures.

The logical conclusion from these two points?

  1. Use TDD to drive out your design
  2. Lock things down with solid typing
  3. Delete all of the tests that are no longer needed. Test code is still code, and it needs to be maintained like app code, so the fewer unnecessary tests you have, the better.

For me, this was one of the most eye-opening ideas from the whole conference.

'5 Years of Metrics and Monitoring' - Lindsay Holmwood

Most of what I got from this was a list of monitoring tools that I hadn't heard of yet, but there was one concept that I think is worth mentioning: Operations should provide monitoring infrastructure as a service to developers. Devs can then check in some sort of monitoring spec alongside their application code, and get the results that they need.

'Implementation of MicroService Architecture' - Fred George

This talk assumed you already had a rough idea of what microservices are, and largely focussed on the communications side of things. Following on from the morning's keynote, Fred advocated strongly for asynchronous communications and immutable events, although he acknowledged that there are other very smart people who advocate for synchronous microservices, mostly because it lowers the barrier to entry a great deal.

Fred's particular implementation was built around a high-capacity bus for communication between services. All services, whenever they have something to say, send an event onto the bus, and any interested services can subscribe to the events that they're interested in. Subscribers can be organised hierarchically, so that events gradually filter down a tree of services to the relevant listeners. If a service wants to respond to an event, it simply puts its reply on to the bus, and the originating service can listen for it. Given that events are the key part of the system, the services should be designed around these events, rather than traditional entities.

Fred gave the example of a travel website that needs to present special offers to a user:

  1. A customer arrival event appears on the bus
  2. The offer service needs an offer to show, so it puts a request on the bus for offers relevant to this customer
  3. Multiple offer providers may respond with 1 or more offers that they think are relevant to the customer, each of which goes on to the bus:
    • A generic offer provider might offer 5% off as part of a site-wide sale
    • A location-specific offer provider might offer 10% off if the customer has selected a certain destination
    • A loyalty offer provider might offer 20% off, because it's the customer's 3rd trip this month
  4. The offer service collects all of the possible solutions from the bus, and determines the best one(s) to show to the customer

In addition to services providing solutions, Fred identified some other possible responses:

  • Services that don't offer solutions at all, but contribute relevant data to the bus. For example, a spending habits service might point out that the customer almost always books a hire car with their flight, which might prove useful somewhere...
  • Services that provide 'negative' solutions. For example, a service might pick up the above data, and provide an anti-solution that says not to offer a discount like "Save X% by adding a car to your purchase", because the customer was almost certainly going to book the car anyway, so the offer would only lose money.

I found this talk really interesting, especially as Fred provided concrete examples for how microservices can help you to build a living, dynamic system that's as modular as possible. One of the key benefits of this approach is that it provides a great platform for innovation and experimentation. If someone has an idea for a new kind of offer to improve customer conversion rates, it's very fast and easy to build a new service, put it into production, and see what actual business impact it has. If it works, it stays; if not, it goes.

I do have one outstanding question: If a service doesn't know how many responses it will receive for a given request, how does it know when to stop listening for answers and pick the best one? I think Fred mentioned that the offer service would just listen for a fixed duration (say, 300ms) before picking its favourite solution, but it seems like there must be a more deterministic way to ensure that services respond as quickly as possible without missing anything important.

That's it!

Unfortunately I had to skip the last session of the day, as I had been asked to give a talk a local meetup group (alongside fellow ThoughtWorker Karl Chu) entitled 'DevOps 101'. Incidentally, if anyone out there is interested in that topic, I'd be happy to reprise the talk.

That's all for now, I'll write up Day 2 of Yow! as soon as I get a chance, hopefully quicker than it took to get this post up!