Stay on Target

Stories of Adopting Alternative JVM Languages in Java Projects in Better Software Magazine

Groovy, JRuby, Scala, and Clojure are some of the most widely used alternative Java Virtual Machine (JVM) languages today.  I spoke with several developers who adopted one of these languages in a Java-based project.  This article ("Alternative JVM Languages for Java Projects") tells their stories - which language they picked, why they picked it, and how they integrated it with their existing Java code base or infrastructure.

This article includes stories from the following people:

• Moss Collum: Gradually migrating code from Java to Groovy
• Kris Nuttycombe: Starting to use Scala to write tests for a Java project
• Najati Imam: Seamlessly deploying JRuby on Rails in an existing Java infrastructure
• Bob McWhirter: Combining some of the best parts of Java with Ruby language to build something new: TorqueBox
• Dan Chamberlain: Using Clojure to more naturally and succinctly model the business problem domain

 

Download PDF

 

This article appears in the September/October 2011 issue of Better Software Magazine.

Thanks to Heather Shanholtzer, Lee Copeland, Jean Middleton, and Cheryl Burke at Better Software Magazine for many things - the opportunity to write this article, top-notch editing, and valuable feedback.  Thanks to my friends Alex Baranosky, Moss Collum, Paul Infield-Harm, Joe Leo, Jason Reid and colleagues at Cyrus Innovation for their formative suggestions for the article.

An Introduction to Scala article in November/December 2010 Better Software Magazine

Scala is a programming language that blends functional and object-oriented language features. Scala programs run on the Java Virtual Machine and can easily interact with Java code. Learn how Scala can yield concise, safe, and compatible code, and how you can start learning Scala on your own.

My article "An Introduction to Scala" is now in print in the November/December 2010 issue of Better Software Magazine.

Download PDF

See the online Sticky Notes and Further Reading sections for footnotes and further references.

Special thanks to Daniel Spiewak for the technical review and edits. Thanks to Paul Blair, Najati Imam, Eric Stolten, and Joe Leo for their reviews and feedback.  Thanks to Lee Copeland, Heather Shanholtzer, Cheryl Burke, and the publication team at Better Software Magazine for their support and top-notch editing.

Addendum

In the article I wrote "Scala is developed at a university—not funded commercially by a big company like Java and Sun/Oracle—so its resources are more limited."  Since the time of writing, Martin Odersky and colleagues have formed Scala Solutions, a company that provides commercial support, tools, and training for Scala.

Introducing Bane, a Test Harness for Server Connections

Bane is a tool that makes it easy to test your application in scenarios where any third-party server dependencies behave in unexpected ways.  This post explains why you might want to use Bane, how to get started, and the design motivations behind Bane. 

Problem: The World is Unsafe (For an Application)

If you are building an application, you may depend on third-party servers or other external sources for your data.  You may use any of the publicly available web services from companies like Google, Yahoo!, or Twitter.  Or perhaps you use web services in an internal business application to obtain data from authoritative sources.  Most of the time these services are reliable and work as you expect.

Yet however reliable these services claim to be, at some point they’ll behave in an unusual manner.  The servers may be busy and accept your connection but never respond, they might reply in an unexpected response format, or there might be network problems that require endless retransmits of the data - and that's just a few of the ways that things can go wrong.  It’s up to you to program defensively so that if the remote service fails, your application doesn’t fail as well.  Michael Nygard’s excellent book “Release It!” is full of patterns and anti-patterns for building applications that can survive in these situations; I can’t recommend it highly enough.

To make sure your application isn’t susceptible to external service failures, you should test your application under various hostile conditions.  However, getting your external services to behave in the desired fashion can be a real challenge.  Instead, Nygard recommends building a tool that can simulate various bad behaviors, as described in the chapter “Test Harness.”  

Introducing Bane

Bane is an open source implementation of the test harness described in “Release It!”  Bane is written in Ruby and is available as a Ruby gem.  To install it, you’ll need a working installation of Ruby.  Install Bane with “gem install bane” which makes an executable “bane” available from the command line.  To make sure Bane is installed correctly, type “bane” with no arguments from the command line.  You should see a usage message like this:

$ bane

Usage: bane port_number <servers>

All behaviors:

 - CloseAfterPause

 - CloseImmediately

 - DelugeResponse

 - DelugeResponseForEachLine

 - FixedResponse

 - FixedResponseForEachLine

 - HttpRefuseAllCredentials

 - NeverRespond

 - NewlineResponse

 - NewlineResponseForEachLine

 - RandomResponse

 - RandomResponseForEachLine

 - SlowResponse

 - SlowResponseForEachLine

Bane requires you specify a port number on which to start listening and a behavior to use to respond on that port.  A behavior is one of the several nefarious ways in which Bane can respond, such as never responding, sending random data, closing the connection immediately, etc.  Behaviors are named in CamelCase (e.g. NeverRespond, NewlineResponseForEachLine) because they correspond to Ruby classes.

A Usage Example

Imagine you’ve built an application to track financial investment portfolios.  All stock quotes are obtained via a third-party web service.  You’d like to see how your application behaves when the web service accepts your request but never sends a byte of data in response.  Let’s pretend that your stock quote service listens on port 8080.

Start up the NeverRespond behavior on port 8080, which accepts a connection but then never sends a response.

$ bane 8080 NeverRespond

[Tue Sep 28 10:49:25 2010] NeverRespond 127.0.0.1:8080 start

Point your application at port 8080 and see how your application behaves.   Notice that Bane logs the incoming connections:

[Tue Sep 28 10:50:05 2010] NeverRespond 127.0.0.1:8080 client:61459 127.0.0.1<127.0.0.1> connect

You may need to add timeouts or other failure strategies to keep your application functioning under this scenario.  Consult “Release It!” for more information and other alternatives.

Bane comes with several bad behaviors you can use to test out your application in the manner described above.  See the README and wiki pages for more information.

The Design Motivations Behind Bane

Before I created Bane, I used to write little one-off scripts to create these badly behaving servers.  They aren’t much work; Ruby’s standard library includes a GServer class that makes it easy to quickly create a multithreaded TCP/IP server.  In fact, if you look at the source code, you’ll see that Bane uses GServer internally.  So why not just create these little servers each time by hand?

It turns out that each time I had to create one, I’d take about five minutes to bring up the RDoc and get all the parameters right.  So I created Bane to have all these common bad behaviors available in just a few seconds.  Right now Bane only supports two response protocols: send a response immediately upon connection and respond after each newline sent by the client.  I haven’t needed other response protocols on my projects yet, but it should be simple to create variations that respond after every n bytes, etc.  

However, if it turns out there needs to be some advanced protocol (e.g. respond after certain keywords, preserve state, etc.) then you may be better off spending the five to ten minutes to write your own server in Ruby using GServer or Sinatra, or some other language of your choice.  My goal is to keep Bane easy to use for common cases, to get a bad behavior up and running in a few seconds rather than five minutes.

Manual or Automated?

I use Bane primarily for manual exploratory tests, and write unit tests around my applications to create fast, repeatable automated tests.   Typically I use Bane in my consulting work to test out an existing application and see how it behaves, find the failure modes, write automated unit tests to expose the behavior, and then fix the problem by applying the patterns described in “Release It!”  Many of these unit tests use test doubles to throw exceptions that would be created by the I/O library in use.

That said, Bane provides an API through the Bane::Launcher class for automation in Ruby.  Take a look at Bane’s integration tests and the examples folder in the source distribution for an example of this API in use.

How to Contribute

Bane is open source and I welcome your feedback or contributions.  Here are a few ways you can contribute:

Provide Feedback: Please take Bane for a spin on your project and let me know if it helps, where it didn’t, and any suggestions for enhancements.

Write new behaviors: Fork Bane on GitHub and contribute useful bad behaviors or any other items described in “Release It!” which are not yet implemented (see the README for a list).  In particular, Bane does not yet support any of the low-level protocol errors in “Release It!”, such as the remote end endlessly sending TCP/IP resend packets.  To my knowledge, this low-level behavior is not possible with Ruby’s I/O library and requires a C extension.

Code Reviews: If you know Ruby and have some ideas about how to improve the code, please fork Bane and suggest some changes.

For more information

• Bane at GitHub
• Bane’s FAQ
• E-mail me at dan at danielwellman dot com

More Thoughts on Scala's Option Class

After reading a post by Cédric Beust that linked to an earlier article I wrote about Scala's Option class, I’ve decided to write about what I’ve learned about Option in the past two years.  Cédric brought up some good points about how that article’s pattern matching example using maps doesn’t feel drastically different than checking for nulls in Java.   

The code examples in the previous blog post demonstrate using Option with pattern matching.  I found this to be an easy way to learn about pattern matching, however after reading and writing more Scala code, I’ve learned that there are more efficient ways to use Option:

1. Use the getOrElse() function for simple this-or-that cases.  The Map getOrElse(key, default) function will return value contained in the Some() container or the specified default otherwise.  Using the map from the example above, you could write something like this:

scala> val map = Map("Hi" -> "Dan", "Hello" -> null)
map: scala.collection.immutable.Map[java.lang.String,java.lang.String] = Map((Hi,Dan), (Hello,null))

scala> map.getOrElse("Hi", "No key with that name!")
res1: java.lang.String = Dan

scala> val result = map.getOrElse("Gutentag", "No key with that name!")
res2: java.lang.String = No key with that name!

In our example, since “Gutentag” doesn’t exist in the map, result will be the “No key with that name!” message.  (Technically this is not the same as the original pattern matching example, as this code doesn’t append the “Found value” string before a found key, but for this example the important part is “or else” functionality.)  This code feels shorter and more concise to me.

Note that the Option class also has a getOrElse(): 

Some(“foo”).getOrElse(“Nothing!”) // returns “foo” 

2. Use Options and for comprehensions to simplify a series of calls which may return a result or no value.   Daniel Spiewak’s comment on Cédric’s blog post provides a quick example of this; I’ll post the code here, but please do read his comment for a detailed explanation (and some more thoughts on Option in Scala):

val value = for {
 row <- database fetchRowById 42
 key <- row get "port_key"
 value <- myMap get key
} yield value
val result = value getOrElse defaultValue

This code will set result to either a Some value if there’s a database record with the given row id, that row has a “port_key” value, and myMap holds an entry with the referenced key or None if any of those lookups fail.  The getOrElse call will either return the value stored in myMap or else some default value.

The concise for comprehension syntax works well when you can accept a default value in the event of failure (an “all or nothing” design approach), but if you need to handle an error at each individual step, you’ll need to do something else.  Lift’s Box class is similar to Option but has hooks for error handling.  

3. Option is about programmer intent. JSR-305 has proposed @NonNull and @Null annotations for Java to allow programmers to indicate that a method may or may not return null.  Scala’s Option class provides a similar indication to programmers; this function may return a value (Some) or it may not (None.)  Java JSR-305-aware compilers can provide warnings at compile-time if a null value might be dereferenced.  Scala’s compiler can also provide a warning if you use pattern matching against an Option class and don’t check for both Some and None cases.

However, Scala’s Option class gives you one thing Java’s null value does not – behavior.  Option defines methods that you may safely call (like getOrElse, map, etc.) whether you have a result (Some) or not (None.)  Using these methods along with for comprehensions seems to yield fairly concise code.

"A Testing Tip" Lightning Talk Notes from the Java Posse Roundup 2010

Last night I presented a lightning talk entitled "A Testing Tip" at the Java Posse Roundup 2010.  Here are some links to the items I mentioned:

• Generated Value Pattern which is useful when specific literal values aren't important for a test
• Make It Easy library for expressive test data builders in Java
• Book: Growing Object-Oriented Software Guided By Tests by Steve Freeman and Nat Pryce
• Book: xUnit Test Patterns by Gerard Meszaros

And here are the slides (PDF), though the links above are more likely to be useful if you didn't see the talk.

Flexible Argument Matching with Mocha

If your Ruby unit tests are failing because you change the signature of a method and all your mock expectations fail, you might have a case of Overspecified Software.  To fix this problem, make your test require only the parameters that are significant.  To quote Gerard Meszaros, "If a value is not important to a test, it's important that it not be in the test."

For example, imagine you had a test which took an options Hash as a parameter.  You might have an expectation like this:

processor.expects(:order).with(:payment => :credit_card, :delivery => :express, :group => :separate)

Unless your test is verifying that all of these parameters are set correctly, it might be better to change your tests so that each test verifies only the parameters that are important.  In this post, I'll describe how to do this with Mocha, but Flex Mock also provides similar features.

Mocha provides a flexible syntax to match parameters of expectations.  For example, if you wanted to check that a given key was present, and ignore the rest of the arguments, you could say:

processor.expects(:order).with(has_key(:payment))

If you are interested in both the key and value, you could use has_entry, or the alternate has_entries:

processor.expects(:order).with(has_entry(:payment => :credit_card))

If you just wanted to make sure something was NOT present, you can use the Not operation to invert the condition:

processor.expects(:order).with(Not(has_key(:emergency_order)))

Or specify any number of conditions using the any_of condition.  

processor.expects(:order).with(any_of(includes(:delivery), has_key(:shipping)))

Mocha provides many more variations, see the Mocha::ParameterMatchers RDoc for more possibilities.

By combining these parameter matchers, you can make tests that are more expressive about the arguments they care about and avoid unrelated test failures when you make a harmless change to your code.

Text File Processing with Ruby from the Command Line

My colleague David showed me some useful ways to use Ruby for simple text processing from the command line.  I'm blogging about it here to remind myself how it works.  (Thanks for the tip, David!)

The Ruby interpreter can be run in a mode which executes a one-line script against a text file.  You can choose to print the results of the script to standard out or to replace the file.  In this way, Ruby becomes a text processing tool like awk, sed, and grep.

For example, if you've got a file with a list of recipe names (in "recipe_index.txt") and you'd like it to contain only those recipes that have 'banana' in the title, you'd do something like this:

ruby -ni.bak -e 'print if /banana/i' recipe_index.txt

-n feeds every line of input_file into your script, simulating sed's text file processing

-i applies changes to recipe_index.txt in-place, otherwise prints the results to standard out.  If you specify any text after -i, such as "-i.bak", Ruby will first make a copy of your input file and append the extension.  In this example, you'd end up with recipe_index.txt.bak

-e specifies a one-line Ruby script to execute, which prints the line only if it matches the case-insensitive regular express 'banana'

You can also use -p if you want to print out every line of the input file as well as your script's result.  For example, if you wanted to prepend the number of characters in each line to the beginning of the line, you'd say:

ruby -pe 'print $_.size' input_file"

Note the magic variable $_ (dollar sign underscore) which holds the contents of the last line matched by gets().

BuildBunny - Nabaztag notifications for TeamCity

If you're like me, you might be looking for more ways to get feedback about your software's continuous integration build process.  Here's one of my favorites: the BuildBunny plug-in for TeamCity will send build status announcements to a Nabaztag rabbit. 

Version 0.3 has just been posted (look for it on the Downloads page).  This release include several little enhancements to the plug-in, including a patch I contributed that enables you to specify a different Text-To-Speech voice to use for the announcements.  To me, the U.S. voices are the lowest fidelity, while some of the U.K. voices are much easier to understand.  I like "UK-Leonard" and "UK-Penelope".

If you want to find a list of all the voices supported by your rabbit, you can send it a message using its API.  Check for action 9 - "Get a list of all supported languages/voices".

A few months ago I read that Violet, the Nabaztag creator, had filed for bankruptcy.  Anyone know what's going on with them?

Five Things "This is Spinal Tap" Taught Me About Software Development

There’s a lot of Spinal Tap activity these days, which is no surprise given that the rock “mockumentary” This is Spinal Tap celebrates its 25^th anniversary this year.  The recent “Unwigged and Unplugged” tour featuring Michael McKean, Christopher Guest, and Harry Shearer playing Spinal Tap and other songs has just been released on DVD.  Spinal Tap re-recorded several of their songs with several celebrity guests and released the “Back from the Dead” album. 

While listening to the Keith Emerson-featured “Heavy Duty”, I started thinking about some lessons we can learn from Tap in the software world:

 

1. What the customer asks for isn’t always what they want. (a.k.a. “the Stonehenge principle”)  As famously demonstrated when the band provided the specifications to the stage prop for their epic Stonehenge, the customer wasn’t happy when they got exactly what they asked for.  When building software, we need to listen to the customer describe what they want, then deliver frequently and get feedback to make sure they get what they need.

 

2. Experiment, but avoid the Jazz Odyssey.  Experimentation is a good thing; it leads to innovation, productivity improvements, and insight into difficult problems.  But self-indulgent experimentation without focus is counterproductive.  Here I’m thinking of technical design and refactoring experiments/spikes – make sure you have an objective so you’ll finish having learned something.

 

3. The performance is best when the band is together.  Nigel Tufnel and David St. Hubbins fought frequently, but productively resolving the conflict and channeling that energy led the band to write and perform at their best.  Derek Smalls acted as the facilitator (“lukewarm water” to Nigel’s fire and David’s ice), a role that is needed to help software teams stay together.  As Esther Derby and Diana Larsen taught me at the Secrets of Agile Teamwork workshop, this facilitator role doesn’t always have to be one person – it can rotate, as can many of the other roles of Agile Leadership.

 

4. Good tools go to eleven.  Spinal Tap required special amplifiers to create their distinctive sound that gave them the title of “England’s Loudest Band”.  While the movie example is a gag, I can identify with the reverence that Nigel displays when describing his favorite amp.  For me, I get that same feeling when using IntelliJ IDEA – it continues to delight me after all these years of day-to-day use.  The lesson is simple – good tools make us more productive.

 

5. “Have a good time, all the time”.  This rock & roll philosophy offered by keyboardist Viv Savage is important to remember.  Many of us got into software development because it was fun, but it’s easy to forget when we get stressed out by some deadline or difficult bug.  Brian Marick mentions joy as one of the key values of great teams, and I agree – I’ve done my best work when I was having a good time.

 

What about you?  Have you got any favorite Spinal Tap-isms that apply to software development?

Resources for the "Agile AJAX: The GWT Experience" session at Agile 2009

Here are the resource and reference links for the Agile 2009 talk "Agile AJAX: The Google Web Toolkit Experience" that Paul Infield-Harm and I are presenting.

Sample Code from the Demo

Source code for the Book Stack demo presented in the talk is available on github:
http://github.com/pinfieldharm/Agile-2009-GWT-Demo/tree/master

Introduction to GWT

"Google Web Toolkit: Your Shortcut to Ajax Web Applications" by Daniel Wellman
An overview of GWT, appeared in Better Software magazine in October 2008.

"Reveling in Constraints" by Bruce Johnson, GWT Tech Lead.  acmqueue, Vol. 7 No. 6 - July 2009
Explains why the GWT team chose Java and how GWT solves common AJAX development problems.

The Tutorial on the Google Web Toolkit project page is an excellent way to tour the framework

Comparison of GWT to other AJAX Frameworks

Matt Raible's (of AppFuse fame) comparison of GWT, Dojo, YUI, and Ext JS

A balanced discussion at Stack Overflow about GWT pros and cons

Readings on Model-View-Presenter and Testable UIs

"GUI Architectures" by Martin Fowler, which include the Supervising Controller and Passive View variations.

"The Humble Dialog Box", by Michael Feathers is one of the original papers on designing testable GUIs.

"Presenter First", by Atomic Objects is a pattern similar to MVP used for building GUIs test-first.

Testing and GWT

"Google Web Toolkit: Writing Ajax Applications Test-First" by Daniel Wellman
Explains the different ways to test GWT applications, including an example of using Model-View-Presenter.  This article was also adapted and included in the GWT documentation as Testing Methodologies Using Google Web Toolkit.

Matt Raible's blog "Testing GWT Applications" lists many online articles for GWT testing.

My blog "Mocking GWT Widgets with GWTMockUtilities" explains how to write plain ol' JUnit TestCases when you need to mock GWT widgets, instead of requiring a GWTTestCase.

Google Testing on the Toilet: Testing GWT without GWTTestCase from August 8, 2009 provides more information on MVP in GWT.

Other GWT Resources

My other blog posts on GWT

onGWT is a news site with frequent updates

Google I/O 2009 Session Videos contain a lot of useful tips about GWT

• "GWT Architecture: Best Practices for Architecting Your GWT App" by Ray Ryan includes discussion of MVP, testing, using an event bus, and the command pattern.
• "Measure in Milliseconds: Performance Tips for Google Web Toolkit" by Kelly Norton has lots of tips for making your GWT application faster.
• "GWT Can Do What?!?!  A Preview of Google Web Toolkit 2.0" by Bruce Johnson
• "Google Wave: Powered by GWT" by Adam Schuck talks about why the Wave team picked GWT

Google Gin provides Guice-style dependency injection for client-side GWT code.