I reviewed a few Scala unit testing frameworks today: SCUnit, and Rehersal. When all was said and done I determined something that I found rather remarkable - TestNG is the best unit testing framework for Scala. For an example click here. To listen to my mind numbing justification of that radical statement, read on...
Why aren't new Unit Testing frameworks up to snuff?
The problem with any new unit testing framework in a new language is that it doesn't have IDE support. This is a huge pain in the ass for someone who relies on their IDE for everything (like me). TestNG already has IDE support, and you can use Java Annotations in Scala, so you can use TestNG just fine in Scala as well.
Does that mean the test frameworks themselves are bad? No, they are fine. But to be entirely honest, its probably not all that worthwhile for someone to build a xUnit framework in Scala other than the fact that its probably really fun. And I say all this knowing full well that could be horse shit - I don't enough about Scala. There may be some nice syntactic stuff that makes testing in Scala just better, more readable, easier to write - I'm not sure. Its my gut feeling. Anyway...
There is still a fundamental underlying problem here.
The main problem with this whole thing is (well there actually might be a couple, but one at a time) that if you want to write an xUnit framework for a language you have write the IDE support for it too. You shouldn't have to do this.
So what can we do to fix this?
IDE's should provide xUnit framework plugin points. All you have to do is plug in your particular framework, and boom instant IDE suppport for your xUnit framework. I've been saying this for a while about languages themselves. All you need to do is plug-in the compiler and boom, you have everything. Maybe thats too much to bite off...but I don't think plugging in your own xUnit is.
The JUnit and TestNG Eclipse plug-ins are nearly identical. They both look damn near the same as the test runner for NUnit. The patterns are clearly obvious. I haven't looked at the implementations at all, so I don't yet have any concrete ideas on how to make this happen, but its so worth it. Someone should do it. It could be me.
Finally...
This could be the first step towards instant IDE support for new languages. Maybe we identify several areas like this that are similar across languages and provide plug-in points for each. What would those features be? I'm not sure I still need to think. And, maybe more languages should just use the support of Java like Scala does.
But, I have to say this was a great day. In my opinion realizing this was a real breakthrough in what I know about well, everything. Additionally, I think I am finally saying things that I haven't really heard other people saying. Maybe they are and I haven't read it, but I feel like I might actually be making some progress.
Speed of adoption for new technologies is always hampered by the lack of tool support, especially when the expectation level has already been set. I agree that running tests in your ide/editor of choice should be a one click no brainer, otherwise it's difficult to get any traction with a new tool. Plus I am lazy and never like going backwards when it comes to tool support.
ReplyDeleteI like you idea of a pluggable xUnit plug-in. You should write it. ;-)
but that would require effort.
ReplyDeleteand brains.
One possibility is that since Scala now has Maven support (http://scala-tools.org/mvnsites/maven-scala-plugin/index.html), and since when you use Maven inside of NetBeans all of the IDE's testing functions are delegated to Maven, it should be fairly trivial to get things working that way.
ReplyDeleteOf course, I wish that the surefire plugin was a bit better about being able to delegate to something other than JUnit or TestNG.