Scala Testing Presentation Outline

Here's the outline for my upcoming presentation on Scala and Testing. It's a bit ambitious, and I know I won't get through it all, but that's okay. If there's anything glaringly missing, now is the last chance to let me know. It also seems like this could be the start of something bigger, like a short book called something like, "Scala Testing for Java Developers"...maybe. However, the talk is only geared toward Java developers for the first few minutes or so.

Without further ado:

1) The prereqs ... Just Java code
        a) "Production" code
          - A simple List interface
          - An ugly LinkedList implementation
        b) Testing Java with Java
          - JUnit
          - Testng

2) The basics ... Testing Java code with Scala
        a) JUnit with Scala
         - JUnit3 (test, setup/teardown)
         - JUnit4 Annotations
        b) TestNG with Scala
         - @Test
         - @DateProviders
         - All Annotations available

3) The basics ... More testing Java with Scala - Intro to ScalaTest
        a) Suite
         - test methods
         - assert with ===
         - expect(x){}
         - intercept
        b) BeforeAndAfter
        c) TimedTest (displays time information for each class and test ran)
        d) TestNGSuite
         - decorate original TestNG test from step 3, "with TestNGSuite"
         - run in both runners

4) Scala extending/implementing Java classes/interfaces
        a) ScalaLinkedList (simple extention of Java LinkedList class)
        b) ScalaMutableList (ugly impementation of simple List interface)

5) Testing Scala with Scala
        a) Briefly redo #4 - but by testing the new Scala code. 
        b) minor intermission - Can we backtrack and test the new Scala code with Java?

6) A nicer, Functional List implemention in Scala

7) Testing Scala with Scala
        a) FunSuite
          -(including HOF's for those who don't know...maybe?)
          -explain how tests methods work

8) Advanced ScalaTest
        a) ScalaTest matchers (in a FunSuite)
        b) Overview of OOTB matchers
          -String, List, Hash, Option matchers, etc.
        c) Combining matchers with and/or
        d) Extending ScalaTest matchers - Creating your own matchers

9) Specs
        a) Specifications
        b) Matchers
        c) Extending Specs matchers - Creating your own matchers

10) Mocking in Scala (via Specs)
        a) Mockito
        b) JMock

11) ScalaCheck (via ScalaTest Checkers)
        a) Properties
        b) Generators
        c) Arbitrary Magic

12) Ant
        a) ScalaTest
        b) JUnit Scala and Java run together
        c) Specs runs as JUnit
        d) TestNG Scala and Java run together

13) Scala Frameworks Interoperability
        a) ScalaTest and Specs
        b) ScalaTest and ScalaCheck
        b) Specs and ScalaCheck

IntelliJ Now Supports ScalaTest

IntelliJ 8 now has support for ScalaTest (at least as of build 9805). I believe it comes in via the Scala plugin. It works, but, when I right clicked on my test class I expected to see a run option for ScalaTest and I didn't, which stinks. Not seeing it also made me think it just didn't work. It does.

To run a ScalaTest test in IntelliJ, you have to add a run configuration. Most Intellij users probably know how to do this, but for those who don't, here are the steps.

1. Click "Edit Configurations" in the Run menu dropdown (right next to the green play arrow).


2. Click the + button to add a new run configuration. This brings up a dropdown containing different runners.


3. ScalaTest is in the list, choose it.


4. Fill in the classname of your test by typing it, or selecting the "..." button to bring up the class finder dialog.


5. Click ok.


6. Run!

Thanks IntelliJ Scala guys! Now if you can only get that right click thing working...

Quotes

Quote from Gilad Bracha

"Tangent: Yes, I did spend ten years at Sun. A lot of it was spent arguing that Java was not the final step in the evolution of programming languages. I guess I’m just not very persuasive."

Gilad is one of my heroes. The work he's doing on Newspeak seems very, very promising.

Sun totally failed in ignoring him. I remember when he left he posted an unflattering blog about Sun, about how he felt that he couldn't get anything done there. He was very frustrated. I'm very glad to feel like I have something in common with such a smart guy.

Quote from Anders Hejlsberg

From Masterminds of Programming, which is (so far) an absolutely fabulous book:

"I think to be really good at something, you have to be passionate about it too. That's something you can't learn. That's just something that you have, I think. I got into programming not because I wanted to make lots of money or because someone told me to. I got into it because I just got totally absorbed by it. You just could not stop me. I had to write programs. It was the only thing I wanted to do. I was very, very passionate about it.

You have to have that passion to get really good at something, because that makes you put in the hours, and the hours are the real key. You need to put in a lot of work."

Anyone who knows me well knows that this describes me exactly. I particularly like, "You just could not stop me. I had to write programs, It was the only thing I wanted to do." I guess I have an important quality in common with another of my heroes. For some childish reason, I'm very excited about this.

Quote from Bill Venners

"I charge $1 for each paren, and $598 for knowing where to put them!"

'Nuff said!

Tests and Booleans and Matchers

I was working with the new ScalaTest matchers API recently when I came across something that made me think (I usually don't think). I had something simple like:

Nil.empty must be(true)

This seems harmless enough, but when I made it fail intentionally the error message I got was something reasonably similar to:

Expected true but got false.

This obviously isn't a very good error message, and I was thinking at first the maybe we should provide the user API to provide their own error message. Something like so:

Nil.empty must be(true, "expected Nil to be empty, but it wasn't!")

This also seems harmless enough. However, I talked this over with Bill Venners and in this case there is a way to do it using symbols and reflection that gives better error messages:

Nil must be('empty)

If this fails, it will give a nice error message like:

Nil was not empty

This is ok, but type safety people will scream, and they'd probably be right. And, does this work in all cases and can we always get good error messages? I think no. Let me explain why.

What if we have an object that has a method that takes an argument and returns a boolean like this theoretical surroundedBy method that could exist on String:

def surroundedBy( s: String ) = startsWith(s) && endsWith(s)

We can't use the symbol form here because the method takes a parameter. So what can we do? If we go back to the original form then we are back to our original bad error message.

"htth".surroundedBy( "h" ) must be (true)
"http".surroundedBy( "h" ) must be (false)

Do we have any other options? Yes (with caveats as well). But first lets think about what we'd really like to write here. If we were writing pure English, it would be this:

"htth" must be surrounded by "h"
"http" must not be surrounded by "h"

Matchers

ScalaTest 0.9.5 came out with a new Matchers API that I'll touch on briefly here, and point you to documentation for more. Using matchers, we can get our client code pretty close to this. We'll have to define our custom matcher first:

case class surroundedBy(r:String) extends BeMatcher[String] {
  def apply(l: String) =
    MatchResult(
      l.surroundedBy(r),
      l + " was not surrounded by " + r,
      l + " was surrounded by " + r
    )
}

And our client test code turns out to be pretty close to the English:

"htth" must be(surroundedBy("h"))
"http" must not be surroundedBy("h")

I like this a lot, but, only if its going to be used a lot. Otherwise, if it's a one off test, I could just kick it back way old school and use an assert with an error message:

assert( "htth".surroundedBy("h") === true,
     "expected http to be surrounded by h, but it wasnt." )
assert( "http".surroundedBy("h") === false,
     "expected http not to be surrounded by h, but it was!!!" )

This is less readable code, but it is less code - about 6 lines less when you include the custom matcher. You get more or less the same error message in this case as well. It is quite ugly though. In this case I'd much rather have an API that just gives me what I originally wanted:

"htth".surroundedBy("h") must be(true,
 "expected htth to be surrounded by h, but it wasnt.")

or better:

"htth" must be('surroundedBy, "h")

I could see this one existing easily. It's really nice, but not type safe.

Anyway, maybe there already are better alternatives like the ones I've explained I simply don't know about. Maybe I haven't done my homework and I should be scolded. I really hope not, and I'll happily write up (and more importantly - use) anything I find that helps make this code more readable. Until alternatives surface, I guess I'll somewhat reluctantly stay with these rules:

When testing methods that return booleans you can

• Use the symbol style ('empty) if the method takes no arguments and you aren't terrible concerned about type safety.


• Use old school assertions when the method takes arguments but you won't have a lot of repetitive tests for the method.


• Use a custom matcher otherwise

You'll also find that ScalaTest (and Specs which is unfortunately not covered here) provides many matchers that can use for many common types.

In my next post I'll provide a much deeper overview of how to write your own custom matchers in ScalaTest, starting by explaining what that surroundedBy case class means.

In the meantime, you can look at ScalaTest Matchers and Specs Matchers.

Here's all the custom matcher code:

package org.scalatest.examples
 
import org.scalatest.matchers.MustMatchers._  
import org.scalatest.matchers._


class MatcherTest extends FunSuite with MustMatchers{
 
  implicit def surroundable[T](l:String) = new {
    def surroundedBy(r:String) = l.startsWith(r) && l.endsWith(r)
  }
  
  case class surroundedBy(r:String) extends BeMatcher[String] {
    def apply(l: String) = 
      MatchResult(
        l.surroundedBy(r), 
        l + " was not surrounded by " + r, 
        l + " was surrounded by " + r
      )
  }
  
  test("weve got the place surrounded"){
    "htth" must be(surroundedBy("h"))
    "http" must not be surroundedBy("t")
  }
}

ScalaTest IDE Support coming soon.

I've written something that allows me to write my tests in ScalaTest as normal (don't have to use any TestNG annotations in my tests), but also run them in all the IDE's, piggybacking off the TestNG IDE plugins. The code isn't in ScalaTest yet, but I'm guessing it likely will get in soon.

Until then, anyone using FunSuite can borrow this code and temporarily use MiniFunSuite instead. This all seems to work quite well.

Here's the client/test code:

class FunSuiteExample extends FunSuiteTestNGAdapter{
  test("hey"){ println("hey") }
  test("fail"){ assert( 1 === 2 ) }
}

And here's the library code:

import org.scalatest.testng.TestNGSuite
import org.testng.annotations.{DataProvider, Test}

trait MiniFunSuite{
  var testMap = Map[String, Function0[Unit]]()
  def test(name:String)(f: => Unit) = testMap += (name -> f _)
}

trait FunSuiteTestNGAdapter extends MiniFunSuite with TestNGSuite{

  @DataProvider{ val name="tests" }
  def tests = testMap.map{ case (s,f) => Array(s,f) }.toList.toArray

  @Test{ val dataProvider = "tests" }
  def scalaTest(testName:String, f: => Unit) = { f }
}

Twitter/Ruby/Scala/Blah

For some reason I feel compelled to chime in on this debate. I like both languages. I like Scala more, but Ruby is fun.

(Reader Beware: This post was written in one pass with little to no editing.)

I don't like how people get all steamed about their languages, as if it's part of them. But, if used properly, this energy can be channeled into creating better languages, so I guess its a necessary evil. The problem is, it not often used properly. Developers seem to just get mad at each other instead of talking through things civilly, and thinking critically and objectively. This is nothing new, and nothing that hasn't been said a thousand times I'm sure, but I think serves well as a nice introduction.

Maybe the problem is that so many of the elements of programming languages are based on feel, as opposed to simple fact. I wish we'd all think more logically and have nice intelligent debates, and I wish we could support our arguments more with fact.

Now for a debate. I was tweeting with Daniel Spiewak who said,

"Obie has once again decided to completely miss the point. Twitter's legacy code is bad *because* of Ruby, not in spite. The experiences described by @stevej and @al3x precisely mirror my own: it isn't impossible to write large, maintainable Ruby...just hard."

I asked him if he could elaborate on "because of", and to that he replied,

"Ruby makes it *hard* to architect a large infrastructure and to keep it maintainable. I think that dynamic typing is part of the problem, but not all of it. Possibly the way the module system works? It isn't impossible to do large apps in Ruby (as Obie and Ola remind us), but it takes more effort and more discipline. Java and Scala both make this sort of large scale code base much easier to create and maintain."

Now, I don't necessarily disagree with that (and recall that I do like Ruby). But I'd like to point out that I've been on a lot of Java projects that were completely unmaintainable. You could easily call them absolute disasters. In fact, the majority of the Java projects that I've been on have been disasters (yes, yes get your laughs in and tell me that its my fault...but its not. I did a lot to help those failing/flailing projects). Maybe that's because I've picked bad projects, and haven't many changes to work with great teams on applications built correctly from the start.

But, I think that emphasizes my point a little. Who has? How many projects are built correctly from the ground up, by talented, disciplined developers? I'd say not many. Most developers are horrible, and the language wouldn't make a shred of difference. And for those developers who are really good, the language probably doesn't make that much difference either. A group of solid developers could easily build something far more maintainable in Ruby than the garbage developers who are writing in Java.

But can those same developers build something more maintainable in Java or Scala? I think it depends on a whole number of things (listed at the end of the post). If there's a good chance of turnover on the project, then you probably want something with type information in the code because it serves so well as documentation. But if there's little chance of turnover it probably doesn't matter.

That said, even with turnover, because Ruby is inherently higher level that Java its probably more easy to maintain. Java is just not a good language anymore, it has not stood the test of time. It came about to fix some problems with C++, and had a syntax that appealed to those developers, but that was 1994. It has not evolved. Ruby is so much more expressive than Java that the lack of type information probably doesn't mean much at all.

Scala on the other hand, probably no contest for large projects. I think Twitter made the right decision. They get the type safety, the type documentation, and the high level expressiveness all together. I do like Ruby a lot, but Scala just feels better to me, and feel is very important, even if its not measurable.

But all that could be wrong. How many large projects have actually been built in Scala? Few. I said in the beginning that I want people to think critically and objectively. It would be nice if we could base some of these debates on fact. Are there any facts out there that large Java projects are more maintainable than Ruby? There may well be, and if so, we should get that information out there. Then again, there may be documentation that supports the opposite.

So I'm not really calling for comments here, though they are always welcome. What I am calling for is experiments, evidence, observations, or facts that support that one language is more maintainable than the other for any sized project.

Here are some questions whose answers would be helpful (in no particular order):

• What language was used?


• What was the size of the project:
  
  
  
  1. LOC? Production and Test code.
  
  
  2. Number of developers?
  
  
  3. Number of testers?


• How long did the project take?


• What was the amount turnover?


• What was the experience level of the developers?


• The amount of time the team has been together?


• The amount of time the developers have spent with the language used?


• Type of project? (Obviously if the language chosen was a horrible choice for the project, it makes a difference)

This information is probably pretty difficult to come by. The lack of it shouldn't impede intelligent debate, but the presence of it could help us as a community.

Random Thoughts

I haven't written in a while, but that doesn't mean that a lot hasn't been going on, just nothing I really felt the need to write about unfortunately. I do have some random thoughts about some things though, and so here they are.

Little Things

I've noticed in some of the comments on my posts that people like to point out that some of the things I mention are no big deal - like not having to put a dot in a method call:

p ! t

vs.

p.!(t) or in Ruby: p.! t

I'd like to point out that sure, they are little things. But, little things add up. The more and more of the little things that a language gets right, the more of a joy it is to program in. Scala seems to get a lot of these things right.

Java 7

I remember months ago (at least 4) I asked, "will Java 7 ever come out?" I did a brief google search and see no evidence that its any closer. Why is this? It's insane.

I think the main reason is (so many people have realized and stated this before) that programming language by committee is BS. There must be a dictator running the show. I hope this is a road that Scala doesn't go down.

There are plenty more reasons of course. Holding on for dear life to backwards compatibility is a bad idea IMO. If companies don't want to move forward to a new version because something has changed, then well, you aren't really at risk of losing them as users anyway. If they want to stay on the old version, let them. As a language designer you should not sacrifice the language. You should always do your best to put out the best language possible. If the language is the best, then when the company finally decides to move off the old version, they will move to the latest version. If your language isn't the best, they won't. No big deal.

ScalaTest

I'm finally getting back into ScalaTest development. I'm really excited about it. It's been a while. I'll be giving a presentation at the next Scala meetup on testing in Scala. It won't be specifically to ScalaTest - I'll be including specs, matchers, mocking, ScalaCheck, lots of cool stuff.

Ruby

Ruby is definitely rubbing off on me. I read an interesting quite by Matz that said something along the lines of, "It really comes down to just how you feel writing in a language that matters". I feel good writing Ruby. I feel even better writing Scala (though I do sometimes end up in fights with the type system and get angry). I feel HORRIBLE writing Java.

camelcase vs. underscores

I've decided that camelcase is much harder to read than underscores, and wish the Java and Scala communities would make the switch. WhenIReadCamelCaseAndEverythingIsBunchedUp, its_just_harder_to_read_than_underscores. I think this is especially true as the statement gets longer. TwoWords is not so_bad. ButAfterThat I think underscores_win_out.

Management

I can't begin to explain how good it feels to have good management again. I had a great one a few years back by the name of Eric Golin. He was so incredibly smart and knew how to speak to me and how to get the best out of all times. This is how I feel at my current job. Management has worked extremely hard to get things running smoothly. It just feels great.

I've had so many terrible, terrible, terrible managers in the past - almost all of them. Its an absolute drag. I need to do better to make sure it doesn't happen again.

Here's one tip - if anyone in the management chain wears sunglasses indoors, run like hell.

C++?

I don't have much experience in C++ at all, but, in what little time that I've spent with it, I like it. And I like it a hell of a lot more than Java. Now, I know Java's appeal was that it fixed so many problems C++ developers had to face, but...for the most part I think it just appealed to the 90% of developers who aren't any good.

But, Java probably was a good step back in order to fix some of those problems, in order to allow really nice things like Scala to come along. You know, I used to really like Java, until Scala came along. Now seeing Java code makes me throw up in my mouth a little.

Algorithms

I need to get better at explaining fundamental algorithms. Its not that I don't understand the concepts, I just don't do a good job of explaining my understanding of them. I will improve this.

School

School is a perfect place for me. I have to go back. I change jobs often because I lose interest. I need to be challenged, always. I need new ideas. I need really smart people I can talk to all the time. Very few jobs can offer even half of what I need. I get all these things and more at school.

If I need a challenge, I just pick a challenging class. Almost everyone in my program will be really smart, and interested in exactly the same things as me. Fantastic. And, I wont have to deal with sunglasses indoors.