Scala over Ruby

In this post, I'll be maintaining a list of things I like more about Scala, over Ruby. I am also maintaining a similar, opposite list - Ruby over Scala. I plan on maintaining this list here and adding more to it over time, as opposed to several posts. I'm not exactly sure how that works with feeds and things, but oh well.

Also, if at any point I'm totally wrong (which is very possible because I'm new to Ruby), please let me know! It's likely that I just didn't know I could do something in Ruby. I'm eager to learn, and for new ideas, and I'll be more than happy to update the post. Anyway, Onward...

1. Scala doesn't make you use the "." operator when calling methods.

This one I'll try to explain using some simple examples. Here is some Ruby code:

x = 5
y = 6
z = x + y
z2 = x.+y

class ClassWithPlusMethod
  def +(other)
   # do something, doesnt matter what...
   return "whatever"
  end
end

x = ClassWithPlusMethod.new
y = ClassWithPlusMethod.new
z = x + y
z2 = x.+y

class SomeOtherClass
  def plus(other)
   # do something, doesnt matter what...
   return "whatever"
  end
end

x = SomeOtherClass.new
y = SomeOtherClass.new
z = x plus y

(irb):25: warning: parenthesize argument(s) for future version
NoMethodError: undefined method `plus' for main:Object
from (irb):25

z2 = x.plus y # this is ok!

As you can see, Ruby has magic handling of + mathematical "operator" (and others). "+" is a simple method call, but since it's name is +, Ruby doesn't require the dot, its interpreter handles it differently. This magic is built into the language, and does not extend to methods with regular names, as shown above with "plus".

In Scala, the . operator is not mandatory on any method call. This may seem trivial, but it is not. It helps in creating cleaner DSL's. Take for example, my post on Equalizer.

In Scala, I was able to add the mustBe method to Any, and call it very nicely. But, in Ruby, I had to put in the dot. Contrast these examples:

x mustBe 5

vs

x.must_be 5

2. Method overloading is good.

I use method overloading a lot. An awful lot. I love it. I don't think I need to explain it however.

3. Scala's apply method.

Scala's apply method is something that I'm pretty sure just doesn't exist at all in Ruby, and I'm not sure it can be done easily. For those who don't know what it is, I'll explain, and maybe some Rubyists can show some similar examples. Given that I'm still not a Ruby expert, I could be totally wrong. If so, I would like to know.

For any method called apply, ".apply" can be omitted from the method call. For example given the following definition of the class Array:

class Array{
  def get(index:Int) = { ...some code to get from the array... }  
  def apply(index:Int) = get(index)
}

And an instance of that class:

val a = new Array(whatever)

Then the following calls are essentially equivalent:

a.get(7) // though only because apply calls get
a.apply(7)
a(7)

This is really useful in cleaning up syntax. However, the real beauty of it is hidden below the surface, and will be the subject of a future post.

4. Ruby faking keyword parameters to methods

Calling methods with keyword params is nice for client code readability. For example:

c = httpConnection( :host => "google.com", :port => 80 )

However, in Ruby the niceness ends at the client code. The library code to support this is abysmal. The reason being, for the method using those parameters, its not at all obvious what the method requires. You have to dig down into the implementation to find out what it actually pulls out of params. This is not ok with me. When I look at a method signature, I should be able to understand what dependencies the method has. Example:

def httpConnection(params)
  ...
  ...
  port = params[:port]
  ...
  ...
  host = params[:host]
end

5. 1 vs. 3 line method defs.

I run into this one a lot, and find that Ruby code ends up being a lot larger than Scala code for this reason. In Ruby, I can't define a method on one line without it looking simply terrible. Example:

def add7(x)
  x + 7
end

or

def add7(x); x + 7; end

The first version is 3 lines when it doesn't need to be. The second version is one line littered with noise.

Here's the much simpler Scala version:

def add7(x:Int) = x + 7

While this entire point might sound trivial, its not. Functional style encourages us to write many small methods. Ruby code grows larger than Scala code quickly. The problem also becomes bad when using nested functions. Ruby:

def add7(x)
  def add3
    x + 3
  end
  add3 + 4
end

Scala:

def add7(x:Int) = {
  def add3 = x + 3
  add3 + 4
}

All of this might seem rather trivial, but in practice it definitely adds up.

Coming Soon

For people that care (probably just myself, but I hold out hope), I'm going to spew out a list of the things I'm working on, posts that can be expected in the near future, and other random crap.

1. The project I was working on was indeed cancelled, as predicted. In hindsight, I'm coming out looking pretty smart. I expect to write some more of my true feelings on this situation. One sad thing of course is that my good friends and old coworkers may soon have to look for new jobs. I hope not.
2. Ruby is nice. I won't say that it's not as nice as Scala, as I may be attacked by the very vocal Ruby community. But I will say that I enjoy writing Scala much more (I do plan to expand on why, but this isn't the right time). However, some of the dynamic things you can do with Ruby are fascinating.
   
3. Rails is nice as well, though I think it has its flaws. I hope to learn Lift very soon and do a nice comparison. We'll see if I have enough time.
4. Today I started reading about building Facebook apps in Rails. This is something I'll likely dedicate a lot of time to.
   
5. Yesterday I learned a bunch about iPhone development. I haven't written anything yet, just had a big walk-through of a pretty popular application. At first glance it seemed fantastic as Apple has made creating views quite simple. This is great because I'm terrible at any visual stuff. Anyway, this is the thing that I will probably end up dedicating the most time to; mostly because I still hold out hope for entrepreneurship.
   
6. I am loving my new job. Leaving NYSE has turned out to be a fantastic decision. My new team is great. I'm re-energized. I'm challenged. I'm actually learning at work, instead of just at home. Learning is encouraged, not shunned. I love it.
   
7. Lastly, I told myself about a year ago that I wouldn't be getting back into web development. This has changed for a few reasons:
   
   
   1. I will still never ever ever try to make things look nice.
   2. It seems like the frameworks available today eliminate most of the rest of the crap that I didn't want to do before.
   3. Having immediate access to millions of users on Facebook seems really exciting, eliminating more crap that I was never any good at - getting people to come to my site.
   4. Most importantly, I have an opportunity to learn new languages, and obviously that's what I really care about

foldLeft in Scala, Little Schemer style

I'm going to write up a Scala version of foldLeft in the style of The Little Schemer.

What is 0 + (1 + 2 + 3)?

>6

What is 0 + (1 + 2 + 3)?

>0 + 1 + (2 + 3)

What is 0 + 1?

>1

What is 1 + (2 + 3) ?

>1 + 2 + (3)

What is 1 + 2?

>3

What is 3 + (3)?

>3 + 3

What is 3 + 3?

>6

Are we done?

>No, we still haven't found out what foldLeft is.

What is foldLeft?

def foldLeft[A, B](as: List[A], z: B)(f: (B, A) => B): B = {
    as match {
      case Nil => z
      case x :: xs => foldLeft( xs, f(z, x) )( f )
    }
  }

But that doesn't tell us much does it? Let's walk through an example.

What is List( 1, 2, 3 )?

>List( 1, 2, 3 )

What is foldLeft(a, 0){ (x,y) => x + y } when a is List( 1, 2, 3 )

>6 ... but why? Let's step through it together.

What is the first question foldLeft asks about the list?

>case Nil

Is a Nil?

>No, it's List( 1, 2, 3 ), so we ask the next question.

What is the next question?

>case x :: xs

What is the value of case x :: xs?

>true, because the list is not Nil, it contains one element x which is 1, followed by a list xs which is List(2,3).

So what is the next step?

> foldLeft( xs, f(z, x) )( f )

What is f( z, x )?

> Well, remember, we called foldLeft like so: foldLeft(a, 0){ (x,y) => x + y }

And what is f?

> { (x,y) => x + y }

Again, what is f( z, x )?

> f( 0, 1 )
> (0,1) => 0 + 1
> 1

So what has foldLeft( xs, f(z, x) )( f ) become?

> foldLeft( List(2,3), 1 )( f )

Now we recur into foldLeft. What is the first question foldLeft asks about the list?

>case Nil

Is a Nil?

>No, it's List( 2, 3 ), so we ask the next question.

What is the next question?

>case x :: xs

What is the value of case x :: xs?

>true, because the list is not Nil, it contains one element x which is 2, followed by a list xs which is List(3).

So what is the next step?

> foldLeft( xs, f(z, x) )( f )

What is f( z, x )? Remember that z is now 1.

> f( 1, 2 )
> (1,2) => 1 + 3
> 3

So what has foldLeft( xs, f(z, x) )( f ) become?

> foldLeft( List(3), 3 )( f )

Now we recur into foldLeft. What is the first question foldLeft asks about the list?

>case Nil

Is a Nil?

>No, it's List(3), so we ask the next question.

What is the next question?

>case x :: xs

What is the value of case x :: xs?

>true, because the list is not Nil, it contains one element x which is 3, followed by a list xs which is Nil.

So what is the next step?

> foldLeft( xs, f(z, x) )( f )

What is f( z, x )? Remember that z is now 3.

> f( 3, 3 )
> (3,3) => 3 + 3
> 6

So what has foldLeft( xs, f(z, x) )( f ) become?

> foldLeft( Nil, 6 )( f )

Now we recur into foldLeft. What is the first question foldLeft asks about the list?

>case Nil

Is a Nil?

>Yes

So what do we do?

> case Nil => z

And what is z?

6!

So what is foldLeft(a, 0){ (x,y) => x + y } when a is List( 1, 2, 3 )

6!

Are we done?

Yes! Now foldLeft a taco right into your mouth.

Refactoring Imperative Code To Functional Code

I've been refactoring Scala literally for days. It's fantastic how much I've learned over the last year. I knew a little bit about functional programming from doing Lisp in college, but a year and a half ago I couldn't have given you the definition of it.

I decided to tackle some of the most obvious (and ugly) imperative/procedural code in my CPU simulator and turn it into an elegant, functional style.

I'll paste the original code here, then explain it a bit, then show the refactorings one step at a time. In explaining, I will assume that you know at least a bit about how anonymous functions work in Scala.

class EightBitAdder( a: EightBitNumber, 
                     b: EightBitNumber, 
                     carryIn: PowerSource ) {

  private val fullAdders: Array[FullAdder] = createFullAdders( a, b, carryIn )
  private val output: EightBitNumber = createOutput()
  
  def getOutput: EightBitNumber = output
  def getCarryOut: PowerSource = fullAdders(7).getCarryOut
  
  
  private def createFullAdders( a: EightBitNumber, 
                                b: EightBitNumber, 
                                carryIn: PowerSource ): Array[FullAdder] = {
    val fullAdders = new Array[FullAdder](8)
    fullAdders(0) = new FullAdder( a(0), b(0), carryIn );
    for( i <- 1 until 8 ){
      fullAdders(i) = new FullAdder(a(i),b(i), fullAdders(i-1).getCarryOut);
    }
    fullAdders
  }
  
  private def createOutput(): EightBitNumber = {
    val out = new Array[PowerSource](8)
    
    var count = 0;
    fullAdders.foreach( p => {  
      out(count) = p.getSumOut 
      count = count + 1
      } 
    )
    new EightBitNumber(out)
  }
}

This code was designed to build an 8 bit adder out of two 8 bit numbers (numbers in this particular case aren't much more than arrays of bits) and a carry in bit. This is your typical, ordinary, everyday 8 bit adder that you much have seen in 1950. The adders job is simple, output the result of the two input numbers added up. As with any 8 bit adder, there are 9 outputs, 8 standard output bits, and the carry out/overflow bit.

The adder accomplishes addition in a standard fashion, by chaining 8 full adders (one bit adders) together. The first full adder in the chain uses the given carry in bit as its carry in bit, and subsequent adders in the chain use the carry out of the previous full adder as the carry in. The carry out of the entire adder is simply the carry out of the last full adder in the chain. Here is a picture of one:



Now that we have all the background out of the way, I'll start the refactorings. I'm actually going to go a little bit in reverse order, refactoring the createOutput() method first, because it is substantially easier to refactor than the createFullAdders method.

Simple Refactoring

Lets take another look at createOutput:

  private def createOutput(): EightBitNumber = {
    val out = new Array[PowerSource](8)
    
    var count = 0;
    fullAdders.foreach( p => {  
      out(count) = p.getSumOut 
      count = count + 1
      } 
    )
    new EightBitNumber(out)
  }

This method is returning an EightBitNumber, which is really just an array holding the 8 output bits. The code is terribly imperative, and terribly terrible. The sad thing is, I actually wrote this code, and I'm not just making up a bad example :( Anyway, its overall strategy is pretty clear.

1. Create an empty, length 8 array
   
2. Create a counter object for indexing into the array
   
3. Loop over all the full adders (those are already create by the time this method gets called, and well see that in a bit)
   
4. For each adder: Put each full adders sum out into the array, and increment the counter.
   
5. Finally, create an 8 bit number object using the array.
   

The first 4 steps above are there simply to create the array to pass to the EightBitNumber object. Now lets take a look at the refactored code:

private val output = new EightBitNumber(fullAdders.map(fa => fa.getSumOut))

Wow! That looks a lot easier - 12 lines down to 1! But...some people might not know what it does, so I'll do my best to explain. The map method, which is a method on all Seq objects (short for Sequence; Array is a subclass), "Returns the list resulting from applying the given function f to each element of this list." An example should help.

Given a=List(1,2,3,4,5) then a.map( i => i * 10 ) returns List(10,20,30,40,50). i * 10 was applied to each element "i" in the list.

In the cpu simulator code above, the call to map has built an Array containing the sumOut of each full adder using the function fa => fa.getSumOut.

Slightly More Difficult Refactoring

With the easier part out of the way, I tackled the more difficult createFullAdders. Let's review the original implementation again.

  private def createFullAdders( a: EightBitNumber, 
                                b: EightBitNumber, 
                                carryIn: PowerSource ): Array[FullAdder] = {
    val fullAdders = new Array[FullAdder](8)
    fullAdders(0) = new FullAdder( a(0), b(0), carryIn );
    for( i <- 1 until 8 ){
      fullAdders(i) = new FullAdder(a(i),b(i), fullAdders(i-1).getCarryOut);
    }
    fullAdders
  }

This method is responsible for creating all the full adders, and chaining them together. The createOutputs method was only responsible for getting all the outputs off of the full adders created here.

Similar to the last method, this method uses an imperative style, creating an array, populating it, and finally returning it. It's quite a bit trickier though because of the chaining. You can't simply use the map function because there's no context in map. Here, each new full adder needs to know about the preceding full adder. This guy is going to be a bear to explain, but let me just go ahead and dump the code on you:

val (fullAdders, carryOut) =
    (as zip bs).foldLeft((List[FullAdder](),carryIn)){
      case ((current, carry), (a, b)) =>
        val adder = new FullAdder(a, b, carry)
        (current ::: List(adder),adder.carryOut)
    }

With James Iry's help, we've made this code about as simple as possible. With the first pass, I wasn't sure if the functional code was more readable than the imperative, but after he helped me clean it up, I'm positive. Now, if you aren't familiar with some of the concepts contained in that code, you might be thinking, "What are you $%^&ing nuts?" But, I'm convinced that after you get used to reading this, it's so much easier to read, and so much less error prone, and so much more natural, that you'll never go back. Honestly, after leaving this project alone for almost a year and coming back to it and finding the imperative code, I almost threw up in my mouth a little.

Okay, now I'll try to explain these concepts, and most likely fail miserably.

1. First, and simplest, is zip. This one is pretty easy. Taken right from the Scaladoc - zip:

   "Returns a list formed from this list and the specified list 'that' by associating each element of the former with the element at the same position in the latter. If one of the two lists is longer than the other, its remaining elements are ignored."

   
   I think a few examples will explain perfectly.
   
   Given a=List(1,2,3) and b=List(a,b,c) then a.zip(b) will return List((1,a), (2,b), (3,c)).
   Given a=List(1,2,3) and b=List(a,b,c,x,y,x) then a.zip(b) will return List((1,a), (2,b), (3,c)), as the remaining elements in b are ignored.
   
   The code in the cpu simulator zips as and bs, which associates the appropriate input bits together. Take a quick look back at the picture to see that this returns ((a0,b0),(a1,b1),(a2,b2),(a3,b3),(a4,b4),(a5,b5),(a6,b6),(a7,b7)).
   


2. Next is foldLeft, which is a bit more complicated. Once again, from the Scaladoc - foldLeft:
   

   Combines the elements of this list together using the binary function f, from left to right, and starting with the value z.

   
   
   This one I've written up separately, because it was so long. You can find it at http://jackcoughonsoftware.blogspot.com/2008/11/foldleft-in-scala-little-schemer-style.html
   


3. 
   Next is pattern matching, but I have to go to bed again! At least I've made some progress :)
   

Revised Code

Here is the finshed product, which no longer uses 8, but instead creates adder chains of N, depending on the length of the inputs. Overall, I think it's a vast improvement over the original.

class AdderChain(as: Number, bs: Number, carryIn: PowerSource) {

  if( as.size != bs.size ) error("numbers must be the same size")
  
  val (fullAdders, carryOut) =
    (as zip bs).foldLeft((List[FullAdder](),carryIn)){
      case ((current, carry), (a, b)) =>
        val adder = new FullAdder(a, b, carry)
        (current ::: List(adder),adder.carryOut)
    }

  val output = new Number(fullAdders.map(fa => fa.sumOut))
}

Using Scala Implicits to Replace Mindless Delegation

I'm still refactoring my Scala CPU Simulator code, as I keep finding ways to just lop off piles of unneeded code. In this latest example, I was using pretty standard Java style delegation - having my class implement an interface, having a member variable of that interface, and delegating all method calls on that interface to the member variable.

There are several problems with this:

1. The main problem: if I add a method to the interface I then have to add it to everyone implementing the interface. But, what If I'm not actually writing the implementation of the interface? Client code won't compile. Adding an implicit doesn't remove this problem entirely, but it certainly works for plain old delegates.


2. It's error prone. Maybe the method was a void, and my IDE added the signature for the method, so the code compiles, but I forgot to actually delegate to the member.
   
3. It's just plain wordy and ugly.
   

To remove the boilerplate, all I needed to do was add an implicit conversion from my class to the interface, using the member variable. I'll show this below.

First, here is an example of the old, more wordy style:

trait PowerSource{
  def connect( p: PowerSource ): Unit
  def disconnect( p: PowerSource ): Unit
  def reconnect( p: PowerSource ): Unit
}

class DelegateToPowerSource( in: PowerSource ) extends PowerSource {
  def connect( p: PowerSource ) = in connect p
  def disconnect( p: PowerSource ) = in disconnect p
  def reconnect( p: PowerSource ) = in reconnect p
}

DelegateToPowerSource has a member, "p", and implements the PowerSource interface by delegating to that member for each of the methods on the PowerSource interface. The more methods that PowerSource has, the longer DelegateToPowerSource gets, and yet the code is just boilerplate. Even if the IDE does this for you, its still wordy and potentially error prone.

Now for the new version:

trait PowerSource{
  def connect( p: PowerSource ): Unit
  def disconnect( p: PowerSource ): Unit
  def reconnect( p: PowerSource ): Unit
}

object DelegateToPowerSource{
  implicit def delegateToPowerSource( d: DelegateToPowerSource ) = d.p
}

class DelegateToPowerSource( p: PowerSource )

That's it. Now, any time I add a method to the PowerSource interface (I should probably start calling it trait), DelegateToPowerSource simply has that method; via the conversion. I never have to change DelegateToPowerSource because of a change to PowerSource. DelegateToPowerSource can simply have whatever code in it that it was originally intended to have, obviously augmenting PowerSource in some way.


For completeness, I'll post the actual code where I did exactly this. But, it's pretty much the same, so if you get the point, no need to keep reading.

trait LogicGate extends PowerSource{
  val inputA: PowerSource
  val inputB: PowerSource
  val output: PowerSource
}

abstract class BaseLogicGate(val inputA: PowerSource, inputB: PowerSource) extends LogicGate {

  def state = output.state
  
  def -->( p: PowerSource ): PowerSource = output --> p 
  def <--( p: PowerSource ): PowerSource = output <-- p 
  
  def disconnectFrom( p: PowerSource ): PowerSource = output disconnectFrom p 
  def disconnectedFrom( p: PowerSource ): PowerSource = output disconnectedFrom p 

  def handleStateChanged( p: PowerSource ) = {}
  def notifyConnections = {} 
}

class AndGate(a: PowerSource, b: PowerSource) 
  extends BaseLogicGate(a: PowerSource, b: PowerSource){
  val output = new Relay(a, new Relay(b))
}

Of course there were several other LogicGates (or, nor, nand, etc). All of this was condensed down to:

object LogicGate{
  implicit def logicGateToPowerSource( lg: LogicGate ): PowerSource = lg.output
}

trait LogicGate{
  val inputA: PowerSource
  val inputB: PowerSource
  val output: PowerSource
}

class AndGate(val inputA: PowerSource, val inputB: PowerSource) extends LogicGate{
  val output = new Relay(inputA, new Relay(inputB))
}

BaseLogicGate is now removed entirely. This is great because BaseLogicGate was really weird, it didn't even use its inputs. The only reason it was there was to decouple the delegation logic from the actual trait.

Now things have become MUCH more clear. AndGate is a LogicGate with inputs A and B, and one output, made from Relays.

Equalizer

I wrote a cool Equalizer class in Scala that allows me to do assertions that are more readable than traditional assertions.

Quick side note: (Equalizer is really an idea shamelessly stolen from Bill Venners Equalizer in ScalaTest. Hopefully we'll add my methods in as well.)

My Equalizer lets me do things like:

val x = 5
x mustBe 5

This replaces the more common assertEquals( x, 5 ), and I think it does so nicely.

You can also do a few more sophisticated things with it, like so:

val x = 5
x mustBe ( 3 or 4 or 5 )

I probably could have used "in" here, such as x mustBe in( 3, 4, 5 ) ... what do you think?

Additionally:

val x = 5
x cantBe 6

For whatever this one is worth (I've actually found use cases for it):

val x = false
x canBeNullOr false

val y = 6
y canBeNullOr ( 5 or 6 )

This works by implicitly converting Any to Equalizer, which contains the methods above. The code can be found below.

import org.testng.Assert._

case object Equalizer {
  implicit def anyToCompare(a: Any) = new Equalizer(a)
}

class Equalizer(a: Any) {

  def mustBe(bs: Any*): Unit = {
    bs(0) match {
      case x:MyTuple => x mustBe a
      case _ => {
        val message = "In Equalizer: expecting one of=" + bs + "\nIn Equalizer: actual  =" + a
        println(message)
        bs size match {
          case 1 => assertEquals(a, bs(0), message)
          case _ => assertTrue(bs.contains(a), message)
        }
      }
    }
  }

  def canBeNullOr(bs: Any*) = {
    if (a != null) mustBe(bs: _*)
    else println("In Equalizer: expecting one of=" + bs + " or null\nIn Equalizer: actual  =" + a)
  }

  def cantBe(b: Any) = assertFalse(a == b)

  def is(b: Any) = a equals b

  def or(b: Any) = {
    a match {
      case x:MyTuple => x + b
      case _ => MyTuple2(a, b)
    }
  }

  import Equalizer._

  trait MyTuple{
    def +(b: Any): MyTuple
    def mustBe(a: Any): Unit
  }
  case class MyTuple2(y: Any, z: Any) extends MyTuple{
    def +(b: Any): MyTuple = MyTuple3(y, z, b)
    def mustBe(a: Any): Unit = a mustBe (y,z)
  }
  case class MyTuple3(x: Any, y: Any, z: Any) extends MyTuple{
    def +(b: Any): MyTuple = MyTuple4(x, y, z, b)
    def mustBe(a: Any): Unit = a mustBe (x,y,z)
  }
  case class MyTuple4(w: Any, x: Any, y: Any, z: Any) extends MyTuple{
    def +(b: Any): MyTuple = MyTuple5(w, x, y, z, b)
    def mustBe(a: Any): Unit = a mustBe (w,x,y,z)
  }
  case class MyTuple5(v: Any, w: Any, x: Any, y: Any, z: Any) extends MyTuple{
    def +(b: Any): MyTuple = MyTuple6(v, w, x, y, z, b)
    def mustBe(a: Any): Unit = a mustBe (v,w,x,y,z)
  }
  case class MyTuple6(u: Any, v: Any, w: Any, x: Any, y: Any, z: Any) extends MyTuple{
    def +(b: Any): MyTuple = throw new IllegalArgumentException("too many ors")
    def mustBe(a: Any): Unit = a mustBe (u,v,w,x,y,z)
  }
}

Sorry about MyTuple...I wanted people to be able to use Tuples as arguments in their mustBe statements, so I had to make sure I didn't pass in a Tuple into mustBe, via the "or" method. Anyway, Tuples don't even appear to be working so....ugh... If anyone can help me clean this up, that would be awesome.

Regardless, what do you think? I really like using the code in tests, even if the Equalizer class itself is a bit hairy.

Refactoring Scala: My CPU Simulator Revisited

I'm preparing to learn Ruby (though I've already tinkered with it some), and to do so, I'm going to revisit my CPU Simulator project that helped me learn Scala so well. I'm going to write it again in Ruby.

So today, I took up revisiting it, with the goal in mind of writing some Ruby. Turned out that didn't actually happen. What did happen was a lot of refactoring of the original Scala code. I hadn't touched that code in several months, and in between then and now I've written a LOT of Scala. Here's some of the things I've learned (and refactored).

1. I used to use way too much redundant type information. This was a natural habit from writing Java for so long (in fact, pretty much all my bad habits stem from too much Java). I rigorously removed ALL the obviously redundant type declarations. I did find that in some cases it helped to have it there, that the code wasn't exactly clear without it, and so I left it. I'm very interested to see what happens here when I slide over to Ruby.
2. Out of habit, I used a lot of unneeded semi-colons. I removed all of them. I hate semi-colons. It's official.
3. I got a lot better at functional programming. There were a lot of areas where I should have used functions, and I refactored the code to do so. My line of code cound is down considerably.
4. Finally, I removed all the Hamcrest matchers. This one I'll explain in more detail in my post tomorrow. I'm basically using my own assert library that allows me to say things like: x mustBe 5 I prefer this style to any other assertions I've come across.
   

Back

For the past several months I've been giving my life to a cause that I now believe will fail miserably. I've decided to leave the NYSE. I wasn't being challenged technically, and management was simply not up to snuff. They actually asked me to, "just drink the kool-aid". I swear.

I haven't been writing, and I haven't even been coding much, all to work on this Death March project. Basically, its been terrible, and I feel like I went into a coma and I've just come out. I'm going to start writing again passionately.

I've accepted a new position where I'll be writing IPhone applications, Facebook apps with Rails, other web work with Rails, etc. I'm very excited about it. I get to learn two languages that I have little experience with (Ruby and Objective C). I get to work with a team that is much more in line with my way of thinking - adopting new technology, open-minded, agile, lots of tests, curious. It feels like a giant weight has been lifted off of my back. I'm very excited.

Additionally, I won't have to work in Java anymore. I'm convinced at this point that java is a dying language. I heard Neal Gafter left Google to work on languages at Microsoft.

I feel his situation parallels mine. He worked tirelessly on closures for Java 7, which he truly believed (as do many) were right for the language. But, the JCP is so damned conservative, that not only will his closures probably not make it into Java 7, but come on...will Java 7 ever actually come out?

I worked tirelessly for NYSE, trying to introduce new technology and exciting ideas. I tried to bring in Scala, and was shot down. I tried to bring in a test first attitude and stress that we really needed time to work on the testing framework there in general, only to be shot down at every suggestion. I felt the same as I imagine Neal felt, "This is impossible".

I believe NYSE ATS software projects will continue to fail until management is replaced.

This was a bit of a rant, but it was a long time coming.

Language Feature Request

Maybe this feature exists in some language, I'm not sure. My inexperience is letting me down. :( Here is what I want, as demonstrated in Scala.

I want to be able to use the name of my variable programmatically. So, instead of having to give my objects names like this:

case class Server( name: String )
val server = Server("Altair")
println(server)

Which yields: Server("Altair")


or like this:

class Server( name: String ){ override def toString = name }
val server = new Server("Altair")
println(server)

Which yields simply: Altair


I would like something like this:

case class Server extends VariableName
val Altair = new Server
println(Altair)

Which yields simply: Altair

That example shows it as simply a library. It probably can't be done as a library, so it seems like it would have to be a language feature. Something like this.

varname case class Server
val Altair = new Server
println(Altair)

Which yields simply: Altair


Does anything like this exist? Would it be terribly difficult to build into a compiler? Things would probably be tricky if you said something like...

varname case class Server
val Altair = new Server
val Moxy = Altair
println(Moxy)

Would you want to get Altair, or Moxy?

Can anyone give any opinions on this at all?

Using Scala Actors

I'm assuming (maybe incorrectly) that most of the Actors that will be written will simply want to react to messages, forever. Something like this:

val reactor = new Actor(){ 
  def act() { 
    loop{ 
      react{
        case msg => ...
      } 
    } 
  } 
}

Given that assumption, it seems like it would be nice just to define whats in the react portion of the code. The rest is redundant. This is pretty simple. First, I created a simple little factory called Actors.

object Actors{
  def newReactor( f: PartialFunction[Any,Unit] ): Actor = { 
    new Actor(){ def act() { loop{ react(f) } } }
  }
}

and then I simply used it:

val reactor = Actors.newReactor {
  case msg => println( "Got something: " + msg )
}

This consolidated the code outside the actual reaction from 4 lines (and 4 right brackets) to a single line (and a single right bracket).

You can also do some other interesting which allow you to keep your reaction code separate from the actual actors (you can do this using the original approach as well). You can define partial functions, and pass them into the newReactor method like this:

def normalReaction : PartialFunction[Any,Unit] = {
  case x: int => println( "Got int: " + x )
  case msg => println( "Got something else: " + msg )
}
  
def abnormalReaction : PartialFunction[Any,Unit] = {
  case msg => println("eruhewiurhqweihu!!!!")
}

val normalReactor = Actors.newReactor { normalReaction } 
val abnormalReactor = Actors.newReactor { abnormalReaction } 

I know this is all rather simple and trivial, but I'm crazy about reducing redundancy and improving readability.

Scala and Enums

Scala doesn't have language level support for enumerations, but I think its fairly easy to argue that its a good thing. First, something isn't quite right about Java enums. Sometime soon I'll post more about that. Scala is such a nice language that you can do things cleanly without needing built in support for extra things like enum. Extra features in a language clutter it up.

As an example, I lovingly ripped off the Planets example from the Java tutorial itself, and implemented it in Scala. Here is the Scala code.

case object MERCURY extends Planet(3.303e+23, 2.4397e6)
case object VENUS extends Planet(4.869e+24, 6.0518e6)
case object EARTH extends Planet(5.976e+24, 6.37814e6)
case object MARS extends Planet(6.421e+23, 3.3972e6)
case object JUPITER extends Planet(1.9e+27,   7.1492e7)
case object SATURN extends Planet(5.688e+26, 6.0268e7)
case object URANUS extends Planet(8.686e+25, 2.5559e7)
case object NEPTUNE extends Planet(1.024e+26, 2.4746e7)
case object PLUTO extends Planet(1.27e+22,  1.137e6)

// mass in kilograms, radius in meters
sealed case class Planet( mass: double, radius: double ){
    // universal gravitational constant  (m3 kg-1 s-2)
    val G = 6.67300E-11      
    def surfaceGravity = G * mass / (radius * radius)
    def surfaceWeight(otherMass: double) = otherMass * surfaceGravity
}

And here is the original Java code.

public enum Planet {
    MERCURY (3.303e+23, 2.4397e6),
    VENUS   (4.869e+24, 6.0518e6),
    EARTH   (5.976e+24, 6.37814e6),
    MARS    (6.421e+23, 3.3972e6),
    JUPITER (1.9e+27,   7.1492e7),
    SATURN  (5.688e+26, 6.0268e7),
    URANUS  (8.686e+25, 2.5559e7),
    NEPTUNE (1.024e+26, 2.4746e7),
    PLUTO   (1.27e+22,  1.137e6);

    private final double mass;   // in kilograms
    private final double radius; // in meters
    Planet(double mass, double radius) {
        this.mass = mass;
        this.radius = radius;
    }
    public double mass()   { return mass; }
    public double radius() { return radius; }

    // universal gravitational constant  (m3 kg-1 s-2)
    public static final double G = 6.67300E-11;

    public double surfaceGravity() {
        return G * mass / (radius * radius);
    }
    public double surfaceWeight(double otherMass) {
        return otherMass * surfaceGravity();
    }
}

The Scala code is nicer, though its unfortunate that you have to say "extends Planet" for each Planet. Each planet is defined as a Scala "object" which is really nothing more than a singleton, which is what enum values are in Java.

I could and should go into more detail on all of this, but I'm mostly posting it for my own reference.

When To Call a Constructor Part 1

I've it said before (and people a lot smarter than me like Gilad Bracha), and I'll say it again: Constructors are Evil. Unfortunately, in most common situations, they are impossible to avoid. Rather than beating a dead horse, I'm going to take a slightly different approach. In this post I'll focus on when it is ok to call a constructor, and how to do so effectively. This information can be applied to any number of OO languages.

(Note: It is of course sometimes possible to get away from calling constructors by using a DIF like Guice. Sometimes its just not possible. For example, you have a legacy code base that you are maintaining/extending. It may be possible to switch it over to a DIF, but its unlikely to happen all at once and you probably don't want to end up with a code base that is somewhere halfway between. That can make code even more difficult to reason about. Regardless, even if you are using a DIF you still want to call new sometimes, which I'll explain later.)

I'll start with a simple example.

public class PersonCache {

  private final Map<Name, Person> storage;
  
  public PersonCache(){
    storage = new HashMap<Name, Person>();
  }
  
  public void addPerson(Person p){
    storage.put(p.getName(), p);
  }

  public Person getPerson(Name name) {
    return storage.get(name);
  }
  
  public void removePerson(Person p){
    storage.remove(p.getName());
  }
}

This class looks reasonable, and in fact it is. But as you'll see later, simple classes like this lull developers into a false sense of security with the "new" statement. Here, calling new on HashMap is ok because:

1. It is a trusted/tested class
2. It has no significant dependencies
1. It doesn't reference any static state
2. It doesn't do any IO

Theres a bit of a theme here. If you are going to call a constructor, you need to have a solid understanding of the class you're instantiating. Because it's so well documented, we know HashMap is safe to instantiate. Unfortunately, most code isn't so well documented. Most developers don't understand each class they instantiate.

So why isn't it ok to instantiate an unknown, untrusted class, or a class with dependencies?

To answer that lets first briefly look at the design forces. There are at least four (and probably more) design forces at play here.

• Encapsulation
• Readability
• Static Dependencies
• Testability

The forces certainly push and pull on each other a bit. As encapsulation goes up so do readability and static dependencies, while and testability goes down. While we can't always have the best of each, it's important to understand when to choose one over the other.

Consider the following example.

public class PersonCacheWithDatabase {

  private final Database storage;
  
  public PersonCacheWithDatabase(){
    storage = new Database();
  }
  
  public void addPerson(Person p){
    storage.put(p.getName(), p);
  }

  public Person getPerson(Name name) {
    return storage.get(name);
  }
  
  public void removePerson(Person p){
    storage.remove(p.getName());
  }  
}

This certainly doesn't look a whole lot different than the first example. But, its infinitely worse. Doubly do because it cleverly tricks you into thinking that its ok by looking so similar.

First, DatabasePersonCache may appear to be encapsulated, but in reality its not. It forces you to know about the database whether you like it or not. If you're going to call this constructor, you had better have a database set up somewhere. Otherwise, try to use it and you're going to get exceptions left and right. For the exact same reason, its difficult to read and test this code. Reading it alone is simply not enough. You need to understand the database class as well. Additionally, this class is forever statically bound to the Database class. If you somehow want to store your people in a more convenient way, well, you just can't. If you want to test a class that uses PersonCache, good luck.

There is a way around this - pass in a StorageStrategy into PersonCache.

public interface StorageStrategy {

  public void put(Name name, Person p);
 
  public Person get(Name name);
 
  public void remove(Name name);
}


public class PersonCacheWithStorageStrategy {

  private final StorageStrategy storage;
  
  public PersonCacheWithStorageStrategy(StorageStrategy storage){
    this.storage = storage;
  }
  
  public void addPerson(Person p){
    storage.put(p.getName(), p);
  }

  public Person getPerson(Name name) {
    return storage.get(name);
  }
  
  public void removePerson(Person p){
    storage.remove(p.getName());
  }
}

PersonCacheWithStorageStrategy is much better than the PersonCacheWithDatabase. As long as StorageStrategy is an interface, PersonCache is now nice and reusable, it can be used with a HashMap, a Database, anything. It isn't statically bound to any implementation. It's definitely more readable as you are safe assume that the StorageStrategy passed in works fine. It's far more testable on the whole - you don't have to set up a database to test it.

However, even though its better than the second example, it does suffer problems that the original PersonCacheWithHashMap does not suffer - poor encapsulation. You have to know something about StorageStrategy in order to use it. What if you only ever want to use this as a quick in-memory helper object? The first example would be far better. What if you only ever needed the in-memory storage capability while using PersonCache? A client is still forced to create a StorageStrategy. Ick.

This is the point where it would be really nice to have a forth example and say this is how to do it. Unfortunately there isn't one magic scenario that solves every problem. Developers need to understand the design forces and the code objects they are clients of in order to make reasonable decisions about their code. If its not entirely safe to call new, based on the rules above, then you must pass your dependencies in. You trade some encapsulation for another design force: sanity.

Now, you may be thinking, well great, in the example 3 you just deflected the problem of calling new upward, but that doesn't help me much, since I still have to call new in the clients of PersonCache. You would be correct. I haven't addressed that issue just yet. But for that you'll have to stay tuned for part two of this mini series.

My First Scala Presentation

I gave my first talk on Scala today, to my team at NYSE. It was an entirely informal, BYOL (Bring your own lunch) talk that I hadn't prepared for at all (I was hoping someone else would speak, but since no one else ever does, its always me, prepared or not). Anyway, there are some lessons learned from the talk.

The talk didn't really go over that well, and mostly because I didn't hit them hard with a great example up front. Next time I will. I finally won them over when I showed a List example, which I'll show here.

Say you want to create a List of integers containing the elements 1, 2, and 3. In Java there are a few ways to go about it, none of them very easy. I'll start with the most common example.

List<Integer> ints = new ArrayList<Integer>();
ints.add(1);
ints.add(2);
ints.add(3);

Like I said, there may be easier ways to do this, but I don't think many people will argue that this would be by far the most commonly used approach. There are several things wrong with it.

• It's about a billion characters long.
• The redundant type information in the first line is so frustrating.
  
• The next three lines of code are almost identical.
• The semi colons are pretty much pointless.
  

Here's how you do the same thing in Scala.

val ints = List(1,2,3)

Thats it.

• Its 20 characters total (not including the unneeded spaces). 20 characters vs. A Billion! I pick 20.
• There is no need whatsoever to put an absurd amount of type information. The compiler is perfectly capable of figuring that out thank you. As are humans; any second year college student could tell that thats a list of integers. Heck, any 7 year old could too.
• There is absolutely no redundant code here.
  
• There are no semi colons.

This example has probably been posted on the internet about a million times by now, and its not the point of this post. The point is this: If you want to give a talk on a language, hit the audience hard with a solid example immediately. Don't dilly-dally and give examples that are only slightly different than their current language and then give them the good stuff. You'll meet too much opposition up front. I thought I was doing them a favor by easing them into Scala but what really happened was quite the opposite. For some terrible reason Java developers are quite territorial. I was providing fuel for them to say, "I'll stick with Java."

Next time, I give the good example up front, then transition to the easy stuff once I've peaked their interest, and then make sure to finish up with a solid example too. And of course next time I'll be quite a bit more prepared.


Ok. Thats the gist of what I was wanting to talk about, now some sideline commentary.

One particularly odd complaint IMO was that most of this was just syntactic sugar. First off, I whole-heartedly disagree, but I can see why some people could incorrectly think that way. My colleague happily responded:

If you think it's just syntactic sugar, then I have a perfect language for you. It only contains 2 characters, 1, and 0. Using anything else, well thats just syntactic sugar.

Of course anyone can think that higher level languages are just prettier syntax, but they would be entirely missing the point. The point is to not have our primitive human minds bombarded with useless information so that we can better and more easily understand the meaning of each line of code. Thats not sugar, thats evolution, baby.

ScalaTest and TestNG

THIS IS AN INCOMPLETE DRAFT, POSTED FOR REVIEW. I UNDERSTAND SOME SECTIONS NEED WORK AND SOME SECTIONS ARE EMPTY OR MISSING ENTIRELY, AND THAT THE FORMAT MIGHT BE MESSED UP.

ScalaTest has two important goals.

1. Allow tests to be written in Scala easily, and concisely.
   
2. Allow Java developers to transition to ScalaTest with minimal effort.

With these two goals in mind I'm happy to announce ScalaTest's integration with TestNG. This integration offers two features that meet the goals of ScalaTest.

1. TestNG tests can be written in Scala, and run in both ScalaTest and TestNG runners.
   
2. Existing TestNG tests can be run in ScalaTest.

By being able to write new TestNG tests in Scala, a developer doesn't have the overhead of learning a new test framework and a new language at once. And more importantly, by being able to run existing test suites in ScalaTest developers can feel confident that all their code is working without the overhead of running two test frameworks at once.

This article will show you how to use ScalaTest to do both, with the help of an example (available for download from the ScalaTest Subversion repository). The example has been tested against Scala 2.7.0, and TestNG 5.7. To use the example you'll also need to download the latest version of ScalaTest.

After downloading everything, you'll need to place the jars into the lib directories of the example. Place the TestNG jar into the java/lib, and place the Scala related jars into the scala/lib folder. You should end up with the following directory structure (as shown here in Eclipse).

About the Example

The example contains what we'll refer to as "existing" code (Java), and "new" code (Scala). The thinking here is that you are working on an existing project in Java, you have a Java code base complete with unit tests (you do have unit tests, don't you?), and that you're interested in exploring Scala. If you're Java code isn't covered with tests the content here is still relevant; you can learn how to write TestNG tests in Scala.

The existing code lives in the java folder where you'll find:

• VolumeKnob.java - An interface for volume knobs
• BoringVolumeKnob.java - A boring implementation of VolumeKnob
• BoringVolumeKnobTest.java - A boring test for BoringVolumeKnob
• volume-tests.xml - A TestNG XML suite to run BoringVolumeKnobTest
• build.xml - Ant file that builds the code and runs the tests
  

The new code is in the scala folder, and it builds upon the existing Java code. In the scala folder you'll find:

• AwesomeVolumeKnob.java - A totally awesome implementation of VolumeKnob
• AwesomeVolumeKnobTest.java - An awesome test for AwesomeVolumeKnob
• build.xml - Ant file that builds the code and has targets to
  
• Run just the existing Java tests in ScalaTest
  
• Run just the Scala tests in ScalaTest
• Run both the Java and Scala tests in ScalaTest
  

Quick Look At What Needs To Be Tested

In the next section we're going to learn how to write TestNG tests in ScalaTest. But before we do, lets take a quick look at what we're going to test. Recall in the java folder the interface VolumeKnob. It's very simple:

public interface VolumeKnob {
    public abstract int currentVolume();
    public abstract int maxVolume();
    public abstract void turnUp();
    public abstract void turnDown();
}

VolumeKnob has two implementations that need testing. The first is a rather boring Java implementation - BoringVolumeKnob. It's too boring to show here, and it can't be turned up beyond 10. It has a corresponding TestNG test class also written in Java - BoringVolumeKnobTest. That won't be shown here either, since we'll assume you know TestNG.

There is also a Scala implementation of that interface, AwesomeVolumeKnob. AwesomeVolumeKnob's have three amazing qualities:

• They always go to at least 11 (of course)
• They can never be turned down
• They can always be turned up, regardless of the max volume.

import org.scalatest.legacy.VolumeKnob

class AwesomeVolumeKnob( val maxVolume: int ) extends VolumeKnob {
  if( maxVolume < 11 ) 
    throw new IllegalArgumentException("...These go to eleven.");
  
  var currentVolume = maxVolume;
  
  def turnDown = 
    throw new IllegalAccessError("AwesomeVolumeKnobs cannot be turned down");
  
  // AwesomeVolumeKnobs don't care about max volume
  def turnUp = currentVolume = currentVolume + 1;
}

AwesomeVolumeKnob is accompanied by AwesomeVolumeKnobTest, which is a TestNG test written in Scala. We'll cover that next.

Writing TestNG tests in ScalaTest

Because Scala allows you to use Java's annotations, TestNG tests can be written in Scala at least as easily as they can in Java. Here is a quick example:

import org.scalatest.legacy.VolumeKnob

class AwesomeVolumeKnobTest{
  @Test
  def awesomeVolumeKnobsCanBeTurnedUpReallyHigh(){
    val v = new AwesomeVolumeKnob(10000)
    for( i <- 1 to 1000 ) v.turnUp
  }
}

There's really nothing to it. This class can be compiled by the Scala compiler and run in any TestNG runner.

To enable your test to be run in ScalaTest, simply extend the ScalaTest trait org.scalatest.testng.TestNGSuite. That's it. Here is the full implementation of AwesomeVolumeKnobTest in all its glory.

import org.scalatest.testng.TestNGSuite
import org.testng.annotations._

class AwesomeVolumeKnobTest extends TestNGSuite{

  @Test{ val description=
         "create AVK's with max volume < 11 and ensure IllegalArg is thrown" 
         val dataProvider="low volumes", 
         val expectedExceptions = Array( classOf[IllegalArgumentException] )}
  def awesomeVolumeKnobsAlwaysGoToAtLeastEleven(maxVolume: int){
    new AwesomeVolumeKnob(maxVolume);  
  }  
  
  @Test{ val description=
         "try to turn down some AVK's and ensure IllegalAccess is thrown"
         val dataProvider="high volumes", 
         val expectedExceptions = Array( classOf[IllegalAccessError] )}
  def awesomeVolumeKnobsCanNeverBeTurnedDown(maxVolume: int){
    new AwesomeVolumeKnob(maxVolume).turnDown();
  } 
  
  @Test{ val description="crank it up" }
  def awesomeVolumeKnobsCanBeTurnedUpReallyHigh(){
    val v = new AwesomeVolumeKnob(10000)
    for( i <- 1 to 1000 ) v.turnUp
  }
  
  @DataProvider{val name="high volumes"}
  def goodVolumes = 
    Array(v(11), v(20),v(30),v(40),v(50),v(60),v(70),v(80),v(90),v(100))
  
  @DataProvider{val name="low volumes"}
  def lowVolumes = 
    Array(v(1), v(2),v(3),v(4),v(5),v(6),v(7),v(8),v(9),v(10))
  
  def v( i: Integer ) = Array(i)
}

There are a few things to notice with this implementation.

Running Scala TestNG Tests in ScalaTest

Running TestNGSuite's in ScalaTest is no different than running any other ScalaTest Suite - simply use the Ant task. In scala/build.xml in the example, you'll find a target called "test-scala-only":

 <target name="test-scala-only" depends="compile">
  <taskdef name="scalatest" classname="org.scalatest.tools.ScalaTestTask" 
   classpathref="test.classpath"/>
  
  <scalatest>
   <runpath>
    <pathelement path="test.classpath"/>
    <pathelement location="${test.jar.file}"/>
   </runpath>
   
   <suite classname="org.scalatest.testng.AwesomeVolumeKnobTest"/>
   
   <reporter type="stdout" />
   <reporter type="graphic" />
   
  </scalatest>
 </target>

In this case we have just one Suite to run:

<suite classname="org.scalatest.testng.AwesomeVolumeKnobTest"/>

Running this task brings up the ScalaTest UI:

Running the Java Tests

If you're a TestNG user, you're likely to be familiar with running test suites from the IDE. There are a couple of different ways to do it, and whichever you choose, you're sure to end up like something like this (as shown in Eclipse):

While this is great, and familiar, and comfortable, it would be a pain to have to run your Java tests through TestNG's UI and then have to switch over to ScalaTests UI to run your Scala tests.

Running Java Tests in ScalaTest

ScalaTest provides a simple way to run TestNG xml suites in its Ant task. In scala/build.xml in the example, you'll find a target called "test-java-only":

 <target name="test-java-only" depends="compile">
  <taskdef name="scalatest" classname="org.scalatest.tools.ScalaTestTask" 
   classpathref="test.classpath"/>
  
  <scalatest>
   <runpath>
    <pathelement path="test.classpath"/>
    <pathelement location="${test.jar.file}"/>
   </runpath>
   
   <testNGSuites>
    <pathelement location="${java.dir}/src/test/volume-tests.xml"/>
   </testNGSuites>
   
   <reporter type="stdout" />
   <reporter type="graphic" />
   
  </scalatest>
 </target>

Inside the testNGSuites block, simply put the location of the xml suite.

  <testNGSuites>
    <pathelement location="${java.dir}/src/test/volume-tests.xml"/>
  </testNGSuites>

While in this case there's one xml suite only, ScalaTest supports multiple xml suites. All suites get run in the same TestNG instance. Running ScalaTest via Ant with the graphic reporter option brings up the ScalaTest UI:

As you can see, ScalaTest reported the exact same results as the TestNG Eclipse plugin. (But...notice that our green bar is much brighter!)

Running All The Tests Together

Finally, for the moment of truth...though being so easy, it's likely a bit of a letdown. To run all the tests together, simply use both options in the Ant task, as in the "test" target in scala/build.xml.

 <target name="test-java-only" depends="compile">
  <taskdef name="scalatest" classname="org.scalatest.tools.ScalaTestTask" 
   classpathref="test.classpath"/>
  
  <scalatest>
   <runpath>
    <pathelement path="test.classpath"/>
    <pathelement location="${test.jar.file}"/>
   </runpath>
   
   <suite classname="org.scalatest.testng.AwesomeVolumeKnobTest"/>

   <testNGSuites>
    <pathelement location="${java.dir}/src/test/volume-tests.xml"/>
   </testNGSuites>
   
   <reporter type="stdout" />
   <reporter type="graphic" />
   
  </scalatest>
 </target>

Running this task brings up the ScalaTest UI:

Problems

Summary

ScalaTest and Mocking

I've added in some mocking into ScalaTest thanks to specs integration. I've done it in a BDD style, kind of like rspec's given/when/then. rspec is much further along and I still need to learn a lot more about it, but thats okay. I'm hoping just to get some peoples opinions on readability and such. Here are a couple very quick examples.

 mockTest( "Report should be generated for each invocation" ){
       
   val reporter = mock(classOf[Reporter])

   expecting( "reporter gets 10 passing reports because invocationCount=10" ) {
     nTestsToPass( 10, reporter )
   }

   when ( "run the suite with method that has invocationCount=10" ){
     new TestNGSuiteWithInvocationCount().runTestNG(reporter)
   }
 }
     
 mockTest( "Reporter should be notified when test is skipped" ){
       
   val reporter = mock(classOf[Reporter])

   expecting ( "a single test should fail, followed by a single test being skipped" ){ 
     one(reporter).runStarting(0) 
     one(reporter).testStarting(any(classOf[Report])) 
     one(reporter).testFailed(any(classOf[Report])) 
     one(reporter).testIgnored(any(classOf[Report]))
     one(reporter).runCompleted()
   }

   when ( "run the suite with a test that should fail and a test that should be skipped" ){
     new SuiteWithSkippedTest().runTestNG(reporter)
   }
 }

As you can see, I've added expecting and when blocks, which optionally take a String (that is currently just used for readability). The simple idea here is that I have grouped chunks of my test into places that make sense, hopefully making it easier for someone reading the test. Do you like it? Do you hate it? Any other opinions that could help?

Which Style Is More Readable?

I've been writing ScalaTest tests for the ScalaTest TestNG integration (say that five times fast). I used a couple of different styles and I was hoping to get some input on which style people thought was more readable, more clear. Both styles are functional, one merely masks it a bit while the other flaunts it.

The first style I'll show is the discrete style. Here I call a test method which takes a function and executes it.

     test( "Reporter Should Be Notified When Test Passes" ){
    
       val testReporter = new TestReporter

       // when
       new SuccessTestNGSuite().runTestNG(testReporter)

       // then
       assert( testReporter.successCount === 1 )
     }
  

     test( "Reporter Should Be Notified When Test Fails" ){
    
       val testReporter = new TestReporter

       // when
       new FailureTestNGSuite().runTestNG(testReporter)

       // then
       assert( testReporter.failureCount === 1 )
     }

The I'm showing only two examples of calling the test function but in actuality I have many more tests. Notice that in each test the first line is always the same:

       val testReporter = new TestReporter

There are benefits to this. All the code is right there and you can read the test method without looking anywhere else. There are also some negatives as well. I have the same line of code in several places.

Now I'll show the even more functional style. It takes a little more explaining. Here I declare a withFixture method that accepts a function that takes a TestReporter as its input. The withFixture method creates the TestReporter (so each test call doesn't have to) and calls the input function passing it the TestReporter. To create tests I call the testWithFixture function which takes a function and passes it to the withFixture function I just wrote.

     override def withFixture(f: (TestReporter) => Unit): Unit = {
       f(new TestReporter) 
     }
     
     
     testWithFixture( "Reporter Should Be Notified When Test Passes" ){ 
       t: TestReporter =>

       // when
       new SuccessTestNGSuite().runTestNG(t)
       // then
       assertThat( t.successCount, is(1) )
     }
  

     testWithFixture( "Reporter Should Be Notified When Test Fails" ){ 
       t: TestReporter =>

       // when
       new FailureTestNGSuite().runTestNG(t)
       // then
       assert( t.failureCount === 1 )
     }

I'm realizing as I'm writing this that all the extra explaining I had to do is because the code is more complicated. There are definite negatives here. Code is being passed around to other code, you might now be sure how things are actually running, you might not be sure where the TestReporter object is coming from. On the upside the methods are slightly shorter and I've removed the most obvious duplication by moving it to the withFixture function. There's still plenty of duplication (in each example) that could be removed but you can definitely push duplication removal too far. Especially in tests, you must balance duplication and indirection. They are in direct opposition.

Does this code push it too far?

Is this just a setup method in disguise? I'm of the opinion that setup methods are bad, but at least here everything is immutable.

If people were to get used to this style would it become more readable in the long run?

Please please let me know your thoughts.

Scala and TestNG in Far Greater Detail

I wrote before on running TestNG in Scala and due to popular demand I'm going to go into much greater detail. My goal is to show that running TestNG in Scala is as easy as it is in Java. I've thrown in Hamcrest to show that that integrates seamlessly as well. Hopefully along the way you'll learn a couple of Scala nuggets too. And, be warned I'm assuming you know a bit about TestNG so I'm not going to explain it much. If you don't...www.testng.org

I created a new Scala class for testing my AndGate class. The idea is that I want to make sure that my AndGate is on or off according to the standard And boolean logic table:

x y | output
------------
0 0 |   0
0 1 |   0
1 0 |   0
1 1 |   1

The testing class is called ScalaTestNGExampleTest, and it looks like this:

import org.testng.annotations._
import org.hamcrest.MatcherAssert._
import org.hamcrest.Matchers._;
import org.testng.annotations.DataProvider;
import com.joshcough.cpu.gates._

class ScalaTestNGExample {
  
  @DataProvider{val name="generators"}
  def createGenerators = { 
    val gens = Array(off, on)
    for( x <- gens; y <- gens ) yield Array(x,y)
  }
  
  private def on = new Generator(true)
  private def off = new Generator(false)
 
 
  @Test{ val dataProvider="generators" }
  def testAndGateStates(genA: Generator, genB: Generator){
    val and: AndGate = new AndGate(genA, genB);
    val whatItShouldBe = genA.on && genB.on
    assertThat( and.on, is(whatItShouldBe) );
    println( and.on + "==" + genA.on + "&&" + genB.on )
  }  
  
  @BeforeMethod def printLineBefore = println("------entering test------")
  @AfterMethod  def printLineAfter  = println("------exiting  test------")
  
}

If you aren't familiar with Scala that might look a bit like magic, so I'll explain one step at a time.

Data Provider

The first thing I did was set up a data provider for my logic table:

1. The annotation declaration is obviously different than in Java. Instead of parentheses, you have to use curly brackets. Instead of simply name/value pairs, you have to declare vars. I could explain why, but instead you could just go to http://www.scala-lang.org/intro/annotations.html.
   
   
2. DataProvider methods are supposed to return Object[][], but...what the heck is this one returning? Well, before I explain what that funky for statement is actually doing I'll just announce that this method is actually returning Array[Array[Generator]]. I could have made it more explicit by saying def createGenerators(): Array[Array[Generator]] but Scala's type inference lets me get away with leaving it off. Does leaving it off hamper readability? In most leaving it off is just eliminating some redundancy. Maybe in this case I should have left it on, but I wanted to show type inference a little bit.
   
   
3. Wait a second...Array[Array[Generator]] isn't Object[][]...or is it? Actually, yes. Scala's typed array class (Array[T]) actually compiles down to Java arrays. In this case, Array[Array[Generator]] compiles down to Generator[][] in Java.
   
   
4. What is Array(on, off)? The type of Array(on, off) is Array[Generator] and it contains two elements, Generator(true) and Generator(false) which are returned from the on and off methods respectively. It may be confusing for a Java programmer to see simply "on" with no parens. In most cases (for reasons far beyond the scope of this post) Scala doesn't force you to use parens on method calls with no arguments.
   
   
5. Ok finally, what is that funky for loop looking thing doing? Rather than explain, why don't I just give the equivalent code in Java for the createGenerators method?

public Generator[][] createGenerators() {

Generator[] onAndOff = new Generator[]{ on(), off() };

Generator[][] gensToReturn = new Generator[4][2];
for( int i = 0; i<onAndOff.length; i++ ){
for( int j = 0; i<onAndOff.length; j++ ){
 gensToReturn[i+j] = new Generator[]{ onAndOff[i], onAndOff[j] };
}
}
return gensToReturn;
}

Honestly? Those three lines of code are doing all of that...? Yes. Honestly. Rather than me trying to explain it though, James Iry does an excellent job in part 2 of his four part series on monads called Monads are Elephants. That is the link to part two, but I recommend reading all four.

So now we have a data provider and we have a reasonable idea how it works. But quickly before I move on, heres another way I could have done it which is arguably more readable but not nearly as much fun. Once again, the idea here is that this data provider is essentially creating the And logic table for us.

def createGenerators = {
Array(Array(on, on),Array(on, off),Array(off, on),Array(off, off))
}

Test Method

At this point I think everything else is really straight forward. Despite that, I'll go over the test method in detail.

1. The @Test annotation is defined in the same fashion as I described above for @DataProvider. In this case I define which data provider to use. Done.
   


2. The method takes two Generator parameters which come from createGenerators method.
   


3. The first line simply instantiates an AndGate object using the two Generator parameters.
   


4. The second and third lines simply assert that the AndGate is what it should be! It should be on only when both generators are on, according to the logic table. The third line uses Hamcrest matchers for asserting. I'm not going to bother explaining them here.
   


5. Finally I throw in a print statement.
   

BeforeMethod And AfterMethod Annotations

Before each test method is called, TestNG will call any methods annotated with @BeforeMethod. In my case before each method I just print a nice message. The same goes for @AfterMethod, but after each test method, of course.

Results

Here is my output from the console:

 [testng] -----------entering test-----------
 [testng] true==true&&true
 [testng] -----------exiting  test-----------
 [testng] -----------entering test-----------
 [testng] false==true&&false
 [testng] -----------exiting  test-----------
 [testng] -----------entering test-----------
 [testng] false==false&&true
 [testng] -----------exiting  test-----------
 [testng] -----------entering test-----------
 [testng] false==false&&false
 [testng] -----------exiting  test-----------

Problems

I did run into a few problems.

1. FIXED: Unfortunately, I couldn't use @Test{expectedExceptions = {SomeException.class}} because Scala doesn't you say Anything.class.
2. Running the tests through Eclipse is not as easy as it is in Java, and needs some work. I ended up mostly running through Ant, but sometimes through Eclipse.

Conclusion

I hope I've convinced you that running TestNG in Scala is simple. I didn't test all the features, but most of what I have tested works great. I plan to use it for most of my Scala development. I personally think its pretty far ahead of the pack, but if you want to see for yourself they are: ScalaTest, ScUnit, Rehersal, JUnit, and specs. I've tried the last four, and of those I thought specs was really nice. It seems like a reasonable alternative.

I am very open to hear ideas on why I should switch to an xUnit test framework built in Scala. Are there any advantages? What are they?

My Small Problem with the Scala Actor Model

Something seems wrong to with the whole send/! idea. The Scala guys say ! apparently means "send", but it really means "add message to actors mailbox", or "put". Take this example:

actor ! message
actor send message

If ! means "send", then it certainly seems like the actor is sending the message. Of course, you have no idea who its sending it to, so by that logic it must be getting the message, but it still just seems confusing. I think the API might be more readable if it used one of the following:

actor <-- message
message --> actor

I've demonstrated that you can use --> and <-- as method names already, so, why not use them here?

Scala: Fold Left Question

I recently realized that I could make a piece of code I posted in a recent blog much cleaner using foldLeft. I have a bit of a problem however, the code is in a very performance sensitive area and I should break out of the fold whenever I encounter a certain condition. To the best of my knowledge foldLeft has no way to break out. It just folds all the way and you get your result at the finish. This is a bit unfortunate. Is there a way to break out and still keep the code clean?

Here is my example, old code first. A PowerSource is on if any of its incoming power sources are on.

 private def updateOnOff = {
   def calculateOnOff: boolean = {
     incomingPowerSources.foreach( p => {if( p.isOn ) return true; })
     return false;
   }
   val newOnOff = calculateOnOff
   if( cachedOnOffBoolean != newOnOff ){
     cachedOnOffBoolean = newOnOff
     notifyConnections;
   }
 }

Here is the new code using foldLeft.

  private def updateOnOff = {
    val newOnOff = incomingPowerSources.foldLeft( false )(_||_.isOn);
    if( cachedOnOffBoolean != newOnOff ){
      cachedOnOffBoolean = newOnOff
      notifyConnections;
    }
  }

The new code is far nicer than the old code. I've decided I'm going to go with it. In most cases PowerSources have only one direct input. Some have two, like OrGate, but for now I'm going to see what kind of performance hit I get.

Now, if only I had a performance testing framework in Scala. Does anyone know of one?

Closures Non Local Return - Scala And Java

I've heard a lot of people bickering about Java Closures. One of the many reasons they are fighting with each other is that in the BGGA proposal changes the semantics of return. In BGGA there can be local and non local returns (I'll explain those in a second). Bloch says it going to be too prone to bugs. Gafter says we need to do it before the language becomes a dinosaur. I tend to go with Neal on this one, maybe thats because I'm more adventurous and not quite as worried about bugs since I do extensive unit testing.

Anyway, I wanted to demonstrate the problem in Scala. It did bite me today a little bit. I caught it quickly in my tests. The following two bits of code have different meanings:

private def updateOnOff = {
 def calculateOnOff: boolean = {
   incomingPowerSources.foreach( p => {if( p.isOn ) true; })
   return false;
 }
 cachedOnOffBoolean = calculateOnOff;
}

private def updateOnOff = {
 def calculateOnOff: boolean = {
   incomingPowerSources.foreach( p => {if( p.isOn ) return true; })
   return false;
 }
 cachedOnOffBoolean = calculateOnOff;
}

The purpose of the updateOnOff function is to simply set the cachedOnOffBoolean to true if any incoming power sources are on. If none of them are on, the boolean is set to false. The inner function, calculateOnOff is responsible for looping through the incoming power sources to see if any are on. If one is on the function should return true to the outer function which sets the boolean.

To be honest, I'm not sure which one is the local and which one isn't. I'm in the process of trying to figure it out. It seems natural to me though to assume that the first one is the local return. If anyone knows better, please let me know.

The first example is an example of local return. Local return returns from the most local block of code, and in this case its the closure function itself, not the calculateOnOff function. The closure here is what's inside the foreach call:

( p => {if( p.isOn ) true; } )


This is the function that will be returned from. It simply returns back to the loop, which goes on to the next item. Finally, when the loop is finshed, false is returned. So, in the first example false will always be returned! This is certainly not correct.

The second example is and example of non local return. Specifying "return" before false means that you intend to return from the enclosing method, not just the closure itself. You intend to break out of the loop. You've found what you were looking for, and you were done. The return returns true from the calculateOnOff, and the boolean is set to true, just as you wanted.


Its pretty straight forward when you know it but I'm sure Bloch is right, there are going to be lots of bugs. Developers who don't use unit testing religiously are going to get it. A simply copy past from a refactoring, and boom.

So what does this tell us? Well, at the least it says be careful, and write lots of unit tests. At the most it might mean that lots of idiots just stay in Java and the cool kids move on to Scala, and thats all right now baby, yeah.