Ping Conference, Play Edition, Day 2

pingconf-scifi

“We want to go crazy about technology today”

Continuing with the second day of the awesome Ping Conference, with no further ado, I’ll shortly summarize the talks. Here we go!

Talks

Keynote – Building composable, streaming, testable Play apps – Stream, Code, Slides

Yevgeniy ‘Jim’ Brikman (@brikis98) started the second day of the conference with his talk about how Play is used at LinkedIn – which it is a lot. He gave a quick overview over the vast array of services that are written in Play. Apparently they are also migrating existing services from SpringMVC and JSP.

He explained how to write composable controller actions using various techniques1, and also showed this in code.

He also gave an easy to follow introduction to Enumerators, paving the way to explain their approach to streaming various parts of a page to the Browser to increase (perceived) load times, inspired by Facebook BigPipe. A first stab at implementing this was to write a custom template type (HtmlStream) which streams its content as soon as it’s available. This had the advantage that the basic HTML page would load immediatly (header, and parts of the body until a (slow) service was to be included). This however still meant that including several snippets (or pagelets, as Jim called them) could mean that a slow service could block the rendering of the page.

To take it to the next level, he only included placeholder divs on the page which would load the pagelets via JavaScript in parallel. This of course makes the application more complex, because it requires you to develop multiple strategies to avoid flickering on the page, and has testing and SEO issues2.

Talk 1 – Play is for Performance – Talk, Slides

James Roper (@jroper), tech lead of the Play framework at Typesafe, talked about performance optimization in Play. James really adheres to eat your own dog food, his presentation was actually a running Play app.

First off, he cleared a common misconception: Making your code asynchronous does not make it faster!3 Making your code asynchronous however makes your code more resourceful, and is currently the best way to scale.

Another misconception is that wrapping blocking calls in a Future {...} block makes the code asynchronous. Actually you have to be careful on which execution context you put your code to avoid blocking the web server thread. You can achieve this by putting them on a separate, isolated context. This can be achieved easily in Play by using Akka’s dispatcher.4

He also shared some internally used execution contexts like an immediate context (in contrast to the default one, it runs tasks immediately, but can cause stackoverflows) and a trampoline context (like the immediate one, but prevents stackoverflows). These are not available for public use, however. If you copy them, make sure not to put any blocking operation on their queues.

The key take away was that you need to carefully think about execution contexts and what code you execute on which one.5

Talk 2 – Play2, Scalaz-Stream & SciFi – Stream, Slides

Julien Tournay (@skaalf) and Pascal Voitot (@mandubian) continued their quest of uniting crazy topics with Play development.

They shortly summarized some of Iteratee’s shortcomings and introduced the audience to Scalaz Streams. Its base abstraction is the Process, a stream of values, defined as Process[F[_], O], where F is an input container and O the output. F is usually effectful, (in Play’s case I/O; if the output were effects, O would be Unit), but it can be anything and even vary for a given process.

Process is modeled as a three-state finite-state machine, the states being Await (input, computes next step), Emit (outputs O using head and tail semantics), and Halt (processing finished, or error).6

Similiar to libraries like Scalding, a Process can be both source and sink. It is implemented as a Free Monad, which according to Pascal takes "one, two, three talks, mabye one conference" to explain. In summary "let’s just say that this is cool".

After giving some examples how to use Process, Julien and Pascal even went so far as to reimagine Play’s architecture with Process instead of Iteratees. Actions, BodyParser and Route would become a chain of several types of Process. It was certainly an interesting approach, but still looked a little complex to use. So far it is only considered research and not a near-future option.

In the discussion afterwards, Pascal also explained how it is easier to manage backpressure (a mechanism when a consumer cannot process any more incoming data and needs to pause the producer) with Process than with Iteratees.

Talk 3 – Making the case for Play – Stream, Slides

Adam Evans (@ajevans85) and Asher Glynn (@asherglynn), two software engineers responsible for BBC’s children’s website, presented their migration path from PHP and Java to Play.

They were pretty honest about the many challenges they faced when making such a fundamental switch. In the end however they were successful, and even management was happy with the results. Developer productivity increased, leading to shorter cycles times (i.e. features got delivered faster, and which manager wouldn’t like that?).

Interestingly, the project also generated a lot of hype internally, and many other developers were eager to join the team. It also had a positive impact on hiring, because in general Scala seems to attract more curious developers.7

To approach Scala slowly, they started by writing Scala templates. The controllers were also being written in Scala, however rather as a Java without semicolons. I think this is a lot more courageous than the typical approach of only writing tests in Scala.

They also talked about their architecture, ending up with two practices one keeps seeing with Play:

  1. Split your app rigorously into sub-modules
  2. Keep it simple, don’t over-architect

Why would I mention a sponsor pitch? Because they didn’t pitch their products, but quickly talked about their architecture which is running on Scala, Play, and Hadoop.

Talk 4 – Play at the Gurdian (untitled) – Stream

Another publishing company that has moved to Play is the Guardian. Represented by Grant Klopper (@grantklopper) in a very engaging speaking-style (he would talk first, and then show supporting slides/code/pictures, and make funny remarks).

The Guradian has recently open-sourced their entire mobile/responsive front-end stack, written in Play of course, and Grant introduced us to their thinking behind it, the architecture and how it is able to handle around 900 requests/second with ease (CDN + Nginx cover a lot of load though).

Grant added some extra-spice when after explaining their advanced build processes, he live-deployed a humans.txt to www.theguardian.com/?view=mobile, which in turn would motivate the Gawker/Kinja guys to also live-deploy on Kinja (also during his talk).8

Talk 5 – Behavioural Abstractions in Play – Stream

Tobias Neef (@tobnee) talked about abstractions in Play, drawing from his experience as a consultant on Play projects.

After a short introduction to abstractions in general, he pointed out that actions vary in their type (Result vs. Future vs. Iteratee), and the problems this poses when trying to abstract over actions for reusability and composability.

A common mistake in Play projects, according to Tobias, is that when controller actions are composed using identical actions all over the place, which violates the DRY principle. He suggested using WrappedRequest and ActionBuilder (as a replacement for Play 2.1’s action composition) to define common top-level actions and use these instead of ad-hoc composition. Referring to Erik Meijer he recommended to factor out the error path and only make the happy path visible in controllers.

He also reminded us to consider using filters for cross-cutting concerns instead of wrapping all actions with the same surrounding action.

Talk 6 – Async: Reacting instead of waiting for better times – Stream, Slides, Code

Johan Andrén (@apnylle) had a tough job – the final talk, after two long, intense conference days. But he managed well to keep the crowd awake and engaged, first by some physical exercise, then by really beautiful slides explaining in great detail a topic that would have been a great introduction to some of the other talks.

That topic was one of the best aspects in Play (vs. some other callback-hellish platform), that is the fact that asynchronous computations are built on top of Futures and Promises. What I really liked (and probably will steal for future explanations) was how he explained these two (and combinators operating on them):

Promise: "I promise that I will give you a kitten when you are old enough."

Future: "When I eventually get a kitten, I’m going to play with it all day."

He also explained Iteratees, ExecutionContexts, and even Akka, and all the codes examples were given in both Scala and Java.

Wrap up

Some other stuff I picked up at the conference:

  • The move to Spray.io is definite, and there already exists an experimental branch. In the beginning, Play will support both Netty and Spray. When the move to Spray is complete, controllers will become Actors.
  • Java 8 APIs will be adopted in a seperate branch to make some things available to Java users that are currently not (e.g. Iteratees), and also make the API nicer to use.

That’s it, hope to see you there next year!


  1. To my surprise LinkedIn does this at the Result level, not via action composition.

  2. A useful tip for testing was to use the -N (or --no-buffer) option with curl to avoid buffering of streamed data, so every byte is shown as soon as it is streamed.

  3. Shocker! Of course it makes things slower, context switches and friends.

  4. Also, you should never use Scala’s default execution context (scala.concurrent.ExecutionContext.Implicits.global) in Play apps, but rather Play’s default one (play.api.libs.concurrent.Execution.Implicits.defaultContext) which is managed by Akka

  5. This will also be documented more carefully in the future.

  6. Pascal also blogged about it.

  7. Unfortunately, this seems only true in the English-speaking world, in Europe especially London. Companies over there tend to move a lot faster towards new technologies.

  8. Typesafe couldn’t resist either, and published a special PingConf-build of Play 2.2.2-RC2, which Christopher announced at the end of the day.

Posted in Scala & Playframwork
0 comments on “Ping Conference, Play Edition, Day 2
1 Pings/Trackbacks for "Ping Conference, Play Edition, Day 2"

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Time limit is exhausted. Please reload CAPTCHA.