Inside sbt-web: Part I – Using sbt-web
sbt-web is an sbt-based web infrastructure for handling client-side files. It serves as the new foundation for the Playframework and potentially any other sbt-based web project. sbt-web defines a basic directory structure (overriden in Play), and it offers an API for processing sources, static files, and reporting errors.
Why do we need a new web infrastructure, why was the old one not good enough?
When Play 2.0 was conceived in 2011, the developers chose to integrate several client-side technologies: CoffeeScript, LESS, Google Closure, and later RequireJS. These were tightly integrated into Play, so everyone who wanted to use something different (e.g. SASS or Dust) had to first learn sbt, write a plugin, and somehow deal with integrating that framework into Play’s build mechanism. This obviously wasn’t a flexible enough solution.
It became more and more difficult to catch up. At some point it wasn’t even possible to update Twitter Bootstrap because of newer LESS features that weren’t supported by Play. And the Play team simply didn’t have the resources to update client-side dependencies every couple of weeks.
So with Play 2.3 the decision was made to factor out any client-side concerns into a new independent project called sbt-web. sbt-web introduces the notion of assets to sbt. Taken from the Rails world, assets are simply files that are served to and displayed or otherwise processed by the web browser.
sbt-web is organized via sbt plugins that use sbt-web’s infrastructure to make integrating and updating the latest tools easier.
The basic project structure2 defined by sbt-web adapts to the standard sbt (or Maven) conventions and essentially adds the
assets folder to the mix.
The input folder is
src/main/public is for static assets. When preparing for production-readiness (staging), the two will be merged and processed.
Please note that the subfolder organization below
assets is only an example, sbt-web doesn’t require you to use a specific naming scheme (or any subfolder at all for that matter).
There is also a
src/main/test/assets folder indicating that sbt-web also supports testing client-side code.
+ src --+ main ----+ assets ------+ js ----+ public ------+ css ------+ images ------+ js --+ test ----+ assets ------+ js ----+ public ------+ css ------+ images ------+ js
target folder contains all sbt output files, sbt-web’s output is organized under
web. No matter where the input comes from, it will be stored in
public/main plus their original relative path. The subfolders are again only examples.
+ target --+ web ----+ public ------+ main --------+ css --------+ images --------+ js ------+ test --------+ css --------+ images --------+ js
And finally the
stage directory that contains the complete, distributable version of your assets when staging:
+ target --+ web ----+ stage
Note that Play overrides
Assets can be sources that need processing or analysis by a source task. All assets, no matter if converted sources or static, will run through the asset pipeline.
Source tasks process files in
Some plugins in this category will convert source files to a file that can be processed by a web browser. The output goes to
Plugins in the analysis category don’t produce any output but only check your source files and report errors or other results, for example static code analysis such as JSHint, or server-side testing such as Mocha.
Try out source tasks by running
sbt web-assets:assets and check the
target/web folder. Errors should be displayed in the console.
The asset pipeline is also a concept from Rails. In Rails however, it describes all asset processing. In sbt-web, it only describes processing for staging output, that is when you want to deploy your app to a server. The asset pipeline consists of stages that are processed in sequential order. Each stage takes the previous stage’s set of files3 and passes it to the next stage, possibly adding generated files or removing no longer needed ones.
An asset pipeline plugin typically provides one stage. You have to explicitly add it to the pipeline in your build file. Examples are gzipping your assets, or adding a hash of the file to its name. As you can imagine, order matters here. You should hash first, then gzip.
To try the pipeline out, run
sbt web-pipeline and check the
target/web folder. To prepare for staging, run
sbt web-stage and check
target/web/stage/public. The latter should contain all your assets while the former only contains products from source tasks and the asset pipeline.
On top of sbt-web there are two additional frameworks.
Both will be covered in future blog posts.
Client-Side Dependency Management
Not a part of sbt-web, but deeply integrated, are WebJars. WebJars are client-side dependencies packaged in JARs so you can declare them as normal dependencies in your build file. sbt-web extracts WebJars to
target/web/web-modules/main/webjars/lib. WebJars are also used to export assets from sub-projects.
In Play you can refer to them via
lib. For example
Work is also underway to resolve dependencies from Node Package Manager (npm)
Using sbt-web in Play
So far Play is the only framework using sbt-web, but in theory you could also use it to manage your personal homepage. sbt-web provides a couple of plug-ins for various web languages, preprocessors, and frameworks.
If you haven’t seen sbt-web in action yet, please watch this excellent and comprehensive talk by James Ward.
sbt-web and Play use sbt’s auto-plugin architecture, so as soon as you add an sbt-web plug-in to
plugins.sbt, it is enabled automatically.
Here are the official source plugins:
- sbt-stylus CSS preprocessor
- sbt-less CSS preprocessor
And the official asset pipeline plugins:
- sbt-digest Fingerprinting (adds a MD5 hash to filename to avoid browser caching issues)
- sbt-gzip Gzip assets (compresses assets using Gzip to reduce load times)
Just add them to your
addSbtPlugin("com.typesafe.sbt" % sbt-<plugin>" % "1.0.x"). Asset pipeline plugins also have to be added to
pipelineStages in your build.sbt:
pipelineStages := Seq(rjs, digest, gzip)
sbt-web gives us flexible and powerful asset handling in Play or any other sbt project. It is now easier than before to integrate modern tooling without resorting to seperate build systems for front-end concerns. The community now depends less on the framework’s developers’ time. Since it is easier to contribute plugins, and sbt-web can also be used in other frameworks, the community can potentially become larger.
Actually mappings of base directory to relative path, but we’ll cover that in the next post in more detail.↩
For performance reasons it is recommended to use Node. Java 8’s implementation is also very fast, but not yet supported.↩