Friday, November 13, 2009

Introducing Sweet

 
In this post I introduce Sweet, the fully complete test framework (even with IDE support) in only 99 lines of code. Sweet is compiled against the new Scala-2.8.0.Beta1-RC1, and works great with simple build tool 0.6.3. Sweet lives at: http://github.com/joshcough/Sweet.

Your First Sweet



Sweets are just like ScalaTest's FunSuite, using the test("name"){ ... } style. The syntax is exactly the same:

class HelloWorldSweet extends Sweet {

test("hello, world!"){
"hello, world!" mustBe "hello, world!"
}
}

Just this (and one minor sbt configuration) will get you fully up and running with simple build tool. Your tests will be discovered and run automatically by sbt, just as they are with the other Scala test frameworks.

Assertions



Sweet currently offers only two assertions, in matchers style: mustBe and mustNotBe. Let's extend the hello world example to show usages of each:

class HelloWorldSweet2 extends Sweet {

test("hello, world! with mustBe"){
"hello, world!" mustBe "hello, world!"
}
test("hello, world! with mustNotBe"){
"hello, world!" mustNotBe "goodbye, world!"
}
}

And while were at it, let's how a quick boolean example:

class BooleanAssertionExampleSweet extends Sweet {

test("hello, world!"){
val b = imagaryCallThatReturnsABoolean()
b mustBe true
}
}

I know it's not much, but it probably covers 90% of common assertions, which are normally of the form:

assertEquals( 1, 1 )
assertTrue( something )

Could there, should there, will there be more matchers and different types of assertions in the near future? My Magic 8 Ball says, "Rely on it".

Running in the IDE



Sweet runs in the IDE by piggybacking on the TestNG IDE plugins. Currently, you all you need to do is mix in the SweetTestNGAdapter trait into your test to get full IDE capability. Here, I modify the Hello World example to provide IDE support:

class HelloWorldSweet extends Sweet with SweetTestNGAdapter {

test("hello, world!"){
"hello, world!" mustBe "hello, world!"
}
}

That's it!

Note: There is a chance that I simple add this capability right into Sweet so that you don't have to mix in this trait at all, but there are some disadvantages. First, it would require that all users of Sweet also depend on TestNG. Secondly, the community is currently working on a common set of test interfaces that all test frameworks would implement. The hope that eventually all the IDEs (and other tools) would use these interfaces, and that any test framework implementing the API could run in the IDE. For now, I'll leave SweetTestNGAdapter as a trait, and hold out hope.

What's next for Sweet?



I plan to use Sweet somewhat as an experimental framework for ideas that are a bit too radical for ScalaTest. A staging grounds if you will. Here are some of the things in the works currently, or planned.


  • Making use of my scala-parallel project, I've already added a Sweet that allows all tests to be run in parallel.
  • I've added my concurrent programming test API in, so multi-threaded code can be testing very easily.
  • I plan eventually to add Actor testing API here.
  • Certain more as well.


Why Sweet?



This is really two questions. Why did I write Sweet, and why should you use Sweet?

As I explained, I want to use Sweet for experimental ideas.

You should use Sweet if you're looking for something that is always up to date with the latest Scala builds, and tools. I understand that there's only a handful of people using the nightly builds, and with beta coming out, it won't really matter. But for now, I plan to build Sweet nightly. Also, it's Sweet, Dood. Come on!

Implementation



Given that it's only 99 lines, I might as well just list the entire implementation here. However, you can also find it at github. Below, I had to make some of the lines shorter so that they would fit in my blog, but I assure you, the actual implementation is 99 lines.

Sweet.scala

package sweet

trait Sweet extends Assertions {

private[sweet] var tests = List[TestCase]()

case class TestCase(name: String, f: () => Unit) {
def apply(reporter: SweetReporter) {
try{
reporter(TestStarting(name))
f()
reporter(TestSucceeded(name))
}catch {
case t: SourAssertionException => {
reporter(TestFailed(name, t))
}
case t: Throwable => {
reporter(TestErrored(name, t))
}
}
}
override def toString = name
}

def test(name: String)(f: => Unit) {
if (tests.map(_.name).contains(name)) println("duplicate test name: " + name)
tests = tests ::: List(TestCase(name, f _))
}

def run(reporter: SweetReporter) {
tests.foreach(_(reporter))
}
}

Assertions.scala

package sweet

trait Assertions {

case class Equalizer(a:Any){

def mustBe(b:Any){
if( ! a.equals(b) )toss(a + " did not equal " + b + " but should have")
}

def mustNotBe(b:Any) {
if( a.equals(b) ) toss(a + " must not equal " + b + ", but did")
}

def toss(message: String){ throw new SourAssertionException(message) }
}

implicit def Any2Equalizer(a: Any) = Equalizer(a)
}

SourAssertionException.scala

package sweet

class SourAssertionException(message: String) extends RuntimeException(message)

SweetFramework.scala (implementation of Framework from test-interfaces)

package sweet

import org.scalatools.testing._

class SweetFramework extends Framework {
def name = "Sweet"
def tests = Array(new TestFingerprint {
def superClassName = "sweet.Sweet"; def isModule = false
})
def testRunner(testLoader: ClassLoader, loggers: Array[Logger]) = {
new SweetRunner(testLoader, loggers)
}
}

SweetRunner.scala (implementation of Runnerfrom test-interfaces)

package sweet

import org.scalatools.testing._

class SweetRunner(val classLoader: ClassLoader,
loggers: Array[Logger]) extends Runner {

def run(testClassName: String, fingerprint: TestFingerprint,
eventHandler: EventHandler, args: Array[String]){
val testClass = Class.forName(testClassName,
true, classLoader).asSubclass(classOf[Sweet])
val sweet = testClass.newInstance
val reporter = new MySweetReporter(eventHandler)
sweet.run(reporter)
}

class MySweetReporter(eventHandler: EventHandler) extends
SweetReporter with NotNull{

def newEvent(tn: String, r: Result, e: Option[Throwable]) {
class MyEvent(val testName:String,
val description:String,
val result:Result, val error:Throwable) extends Event
eventHandler.handle(new MyEvent(tn, tn, r, e getOrElse null))
}

def apply(event: SweetEvent) {
event match {
case t: TestStarting =>
loggers.foreach(_ info "Test Starting: " + t.testName)
case t: TestFailed =>
newEvent(t.testName, Result.Failure, Some(t.reason))
case t: TestErrored =>
newEvent(t.testName, Result.Failure, Some(t.reason))
case t: TestSucceeded =>
newEvent(t.testName, Result.Success, None)
}
}
}
}

SweetReporter.scala

package sweet

trait SweetReporter {
def apply(event: SweetEvent)
}

SweetEvent.scala

package sweet

trait SweetEvent

case class TestStarting(testName: String) extends SweetEvent
case class TestFailed(testName: String,
reason:SourAssertionException) extends SweetEvent
case class TestErrored(testName: String,
reason:Throwable) extends SweetEvent
case class TestSucceeded(testName: String) extends SweetEvent


Proof



Here's proof that it's really just 99 lines:

$ wc -l Assertions.scala \
SourAssertionException.scala \
Sweet.scala SweetEvent.scala \
SweetFramework.scala \
SweetReporter.scala \
SweetRunner.scala \
SweetTestNGAdapter.scala
14 Assertions.scala
2 SourAssertionException.scala
28 Sweet.scala
7 SweetEvent.scala
8 SweetFramework.scala
4 SweetReporter.scala
29 SweetRunner.scala
7 SweetTestNGAdapter.scala
99 total

6 comments:

  1. 99 lines of Sweet in my code,
    99 lines of Sweet...
    Take one down, pass it around,

    199 lines of Sweet in my code.... (That's the over-engineering in process....)


    Sweet!

    ReplyDelete
  2. > 199 lines of Sweet in my code... (No, that's the Java version)

    Sour!

    ReplyDelete
  3. Why "tests = tests ::: List(TestCase(name, f _))"? That's O(nlogn). It would be better to use "::" and just reverse before iterating through it, or use a ListBuffer instead.

    ReplyDelete
  4. No real good reason. I don't like reversing simply because I want to type fewer characters. I'm not particularly concerned with performance since I doubt anyone will be writing a Sweet with thousands of tests. I probably should have used ListBuffer, and can always change it. I really do just want to say: tests = tests + Testcase(name, f _), so ListBuffer would probably be the way to go right?

    ReplyDelete
  5. This comment has been removed by a blog administrator.

    ReplyDelete
  6. This comment has been removed by a blog administrator.

    ReplyDelete