Wednesday, January 23, 2008

Scala and TestNG in Far Greater Detail

I wrote before on running TestNG in Scala and due to popular demand I'm going to go into much greater detail. My goal is to show that running TestNG in Scala is as easy as it is in Java. I've thrown in Hamcrest to show that that integrates seamlessly as well. Hopefully along the way you'll learn a couple of Scala nuggets too. And, be warned I'm assuming you know a bit about TestNG so I'm not going to explain it much. If you don't...www.testng.org

I created a new Scala class for testing my AndGate class. The idea is that I want to make sure that my AndGate is on or off according to the standard And boolean logic table:

x y | output
------------
0 0 | 0
0 1 | 0
1 0 | 0
1 1 | 1

The testing class is called ScalaTestNGExampleTest, and it looks like this:


import org.testng.annotations._
import org.hamcrest.MatcherAssert._
import org.hamcrest.Matchers._;
import org.testng.annotations.DataProvider;
import com.joshcough.cpu.gates._

class ScalaTestNGExample {

@DataProvider{val name="generators"}
def createGenerators = {
val gens = Array(off, on)
for( x <- gens; y <- gens ) yield Array(x,y)
}

private def on = new Generator(true)
private def off = new Generator(false)


@Test{ val dataProvider="generators" }
def testAndGateStates(genA: Generator, genB: Generator){
val and: AndGate = new AndGate(genA, genB);
val whatItShouldBe = genA.on && genB.on
assertThat( and.on, is(whatItShouldBe) );
println( and.on + "==" + genA.on + "&&" + genB.on )
}

@BeforeMethod def printLineBefore = println("------entering test------")
@AfterMethod def printLineAfter = println("------exiting test------")

}

If you aren't familiar with Scala that might look a bit like magic, so I'll explain one step at a time.

Data Provider

The first thing I did was set up a data provider for my logic table:

  1. The annotation declaration is obviously different than in Java. Instead of parentheses, you have to use curly brackets. Instead of simply name/value pairs, you have to declare vars. I could explain why, but instead you could just go to http://www.scala-lang.org/intro/annotations.html.

  2. DataProvider methods are supposed to return Object[][], but...what the heck is this one returning? Well, before I explain what that funky for statement is actually doing I'll just announce that this method is actually returning Array[Array[Generator]]. I could have made it more explicit by saying def createGenerators(): Array[Array[Generator]] but Scala's type inference lets me get away with leaving it off. Does leaving it off hamper readability? In most leaving it off is just eliminating some redundancy. Maybe in this case I should have left it on, but I wanted to show type inference a little bit.

  3. Wait a second...Array[Array[Generator]] isn't Object[][]...or is it? Actually, yes. Scala's typed array class (Array[T]) actually compiles down to Java arrays. In this case, Array[Array[Generator]] compiles down to Generator[][] in Java.

  4. What is Array(on, off)? The type of Array(on, off) is Array[Generator] and it contains two elements, Generator(true) and Generator(false) which are returned from the on and off methods respectively. It may be confusing for a Java programmer to see simply "on" with no parens. In most cases (for reasons far beyond the scope of this post) Scala doesn't force you to use parens on method calls with no arguments.

  5. Ok finally, what is that funky for loop looking thing doing? Rather than explain, why don't I just give the equivalent code in Java for the createGenerators method?


public Generator[][] createGenerators() {

Generator[] onAndOff = new Generator[]{ on(), off() };

Generator[][] gensToReturn = new Generator[4][2];
for( int i = 0; i<onAndOff.length; i++ ){
for( int j = 0; i<onAndOff.length; j++ ){
gensToReturn[i+j] = new Generator[]{ onAndOff[i], onAndOff[j] };
}
}
return gensToReturn;
}

Honestly? Those three lines of code are doing all of that...? Yes. Honestly. Rather than me trying to explain it though, James Iry does an excellent job in part 2 of his four part series on monads called Monads are Elephants. That is the link to part two, but I recommend reading all four.

So now we have a data provider and we have a reasonable idea how it works. But quickly before I move on, heres another way I could have done it which is arguably more readable but not nearly as much fun. Once again, the idea here is that this data provider is essentially creating the And logic table for us.


def createGenerators = {
Array(Array(on, on),Array(on, off),Array(off, on),Array(off, off))
}

Test Method

At this point I think everything else is really straight forward. Despite that, I'll go over the test method in detail.
  1. The @Test annotation is defined in the same fashion as I described above for @DataProvider. In this case I define which data provider to use. Done.

  2. The method takes two Generator parameters which come from createGenerators method.

  3. The first line simply instantiates an AndGate object using the two Generator parameters.

  4. The second and third lines simply assert that the AndGate is what it should be! It should be on only when both generators are on, according to the logic table. The third line uses Hamcrest matchers for asserting. I'm not going to bother explaining them here.

  5. Finally I throw in a print statement.


BeforeMethod And AfterMethod Annotations

Before each test method is called, TestNG will call any methods annotated with @BeforeMethod. In my case before each method I just print a nice message. The same goes for @AfterMethod, but after each test method, of course.

Results

Here is my output from the console:

[testng] -----------entering test-----------
[testng] true==true&&true
[testng] -----------exiting test-----------
[testng] -----------entering test-----------
[testng] false==true&&false
[testng] -----------exiting test-----------
[testng] -----------entering test-----------
[testng] false==false&&true
[testng] -----------exiting test-----------
[testng] -----------entering test-----------
[testng] false==false&&false
[testng] -----------exiting test-----------

Problems

I did run into a few problems.

  1. FIXED: Unfortunately, I couldn't use @Test{expectedExceptions = {SomeException.class}} because Scala doesn't you say Anything.class.
  2. Running the tests through Eclipse is not as easy as it is in Java, and needs some work. I ended up mostly running through Ant, but sometimes through Eclipse.

Conclusion

I hope I've convinced you that running TestNG in Scala is simple. I didn't test all the features, but most of what I have tested works great. I plan to use it for most of my Scala development. I personally think its pretty far ahead of the pack, but if you want to see for yourself they are: ScalaTest, ScUnit, Rehersal, JUnit, and specs. I've tried the last four, and of those I thought specs was really nice. It seems like a reasonable alternative.

I am very open to hear ideas on why I should switch to an xUnit test framework built in Scala. Are there any advantages? What are they?

Sunday, January 20, 2008

My Small Problem with the Scala Actor Model

Something seems wrong to with the whole send/! idea. The Scala guys say ! apparently means "send", but it really means "add message to actors mailbox", or "put". Take this example:

actor ! message
actor send message

If ! means "send", then it certainly seems like the actor is sending the message. Of course, you have no idea who its sending it to, so by that logic it must be getting the message, but it still just seems confusing. I think the API might be more readable if it used one of the following:

actor <-- message
message --> actor

I've demonstrated that you can use --> and <-- as method names already, so, why not use them here?

Scala: Fold Left Question

I recently realized that I could make a piece of code I posted in a recent blog much cleaner using foldLeft. I have a bit of a problem however, the code is in a very performance sensitive area and I should break out of the fold whenever I encounter a certain condition. To the best of my knowledge foldLeft has no way to break out. It just folds all the way and you get your result at the finish. This is a bit unfortunate. Is there a way to break out and still keep the code clean?

Here is my example, old code first. A PowerSource is on if any of its incoming power sources are on.


private def updateOnOff = {
def calculateOnOff: boolean = {
incomingPowerSources.foreach( p => {if( p.isOn ) return true; })
return false;
}
val newOnOff = calculateOnOff
if( cachedOnOffBoolean != newOnOff ){
cachedOnOffBoolean = newOnOff
notifyConnections;
}
}


Here is the new code using foldLeft.


private def updateOnOff = {
val newOnOff = incomingPowerSources.foldLeft( false )(_||_.isOn);
if( cachedOnOffBoolean != newOnOff ){
cachedOnOffBoolean = newOnOff
notifyConnections;
}
}


The new code is far nicer than the old code. I've decided I'm going to go with it. In most cases PowerSources have only one direct input. Some have two, like OrGate, but for now I'm going to see what kind of performance hit I get.

Now, if only I had a performance testing framework in Scala. Does anyone know of one?

Sunday, December 30, 2007

Closures Non Local Return - Scala And Java

I've heard a lot of people bickering about Java Closures. One of the many reasons they are fighting with each other is that in the BGGA proposal changes the semantics of return. In BGGA there can be local and non local returns (I'll explain those in a second). Bloch says it going to be too prone to bugs. Gafter says we need to do it before the language becomes a dinosaur. I tend to go with Neal on this one, maybe thats because I'm more adventurous and not quite as worried about bugs since I do extensive unit testing.

Anyway, I wanted to demonstrate the problem in Scala. It did bite me today a little bit. I caught it quickly in my tests. The following two bits of code have different meanings:


private def updateOnOff = {
def calculateOnOff: boolean = {
incomingPowerSources.foreach( p => {if( p.isOn ) true; })
return false;
}
cachedOnOffBoolean = calculateOnOff;
}

private def updateOnOff = {
def calculateOnOff: boolean = {
incomingPowerSources.foreach( p => {if( p.isOn ) return true; })
return false;
}
cachedOnOffBoolean = calculateOnOff;
}


The purpose of the updateOnOff function is to simply set the cachedOnOffBoolean to true if any incoming power sources are on. If none of them are on, the boolean is set to false. The inner function, calculateOnOff is responsible for looping through the incoming power sources to see if any are on. If one is on the function should return true to the outer function which sets the boolean.

To be honest, I'm not sure which one is the local and which one isn't. I'm in the process of trying to figure it out. It seems natural to me though to assume that the first one is the local return. If anyone knows better, please let me know.

The first example is an example of local return. Local return returns from the most local block of code, and in this case its the closure function itself, not the calculateOnOff function. The closure here is what's inside the foreach call:

( p => {if( p.isOn ) true; } )


This is the function that will be returned from. It simply returns back to the loop, which goes on to the next item. Finally, when the loop is finshed, false is returned. So, in the first example false will always be returned! This is certainly not correct.

The second example is and example of non local return. Specifying "return" before false means that you intend to return from the enclosing method, not just the closure itself. You intend to break out of the loop. You've found what you were looking for, and you were done. The return returns true from the calculateOnOff, and the boolean is set to true, just as you wanted.


Its pretty straight forward when you know it but I'm sure Bloch is right, there are going to be lots of bugs. Developers who don't use unit testing religiously are going to get it. A simply copy past from a refactoring, and boom.

So what does this tell us? Well, at the least it says be careful, and write lots of unit tests. At the most it might mean that lots of idiots just stay in Java and the cool kids move on to Scala, and thats all right now baby, yeah.

Coolest Code I've Ever Written

Because Scala lets you name methods pretty much whatever you want, I got the chain wires together like this:



@Test
def lastPowerSourceInChainShouldBeNotifiedWhenFirstInChainIsTurnedOff() = {
//given
val first = new Generator
val secondToLast = new Wire

first-->new Wire-->new Wire-->new Wire-->secondToLast

val last = createMockWithConnectionTo(secondToLast)

// when
first.turnOff

// then
verify(Array(last))
}

private def createMockWithConnectionTo( p: PowerSource ): PowerSource = {
val mockPowerSource: PowerSource = createStrictMock(classOf[PowerSource]).asInstanceOf[PowerSource];
expect( mockPowerSource <-- p ).andReturn( p ) expect( mockPowerSource.handleStateChanged( p ) ) replay(Array(mockPowerSource)) p --> mockPowerSource
mockPowerSource
}


The -->'s are actually methods on any PowerSource that connects the two together. So, the code can actually look like a chain of wires connected together. This is totally the coolest thing ever.

Friday, December 28, 2007

Using TestNG in Scala: By Example

Here is a very, very simple example of some TestNG stuff that I wrote the other day in Scala (it can also be found here):

package com.joshcough.cpu.electric;

import org.testng.annotations._
import org.testng.Assert_


package com.joshcough.cpu.electric;

import org.testng.annotations._
import org.testng.Assert._

class GeneratorTest {

@Test
def expectNewGeneratorToBeOn() = {
val gen: Generator = new Generator
assertTrue(gen.isOn);
}

@Test
def expectTurnedOffGeneratorIsOff() = {
val gen: Generator = new Generator
gen.turnOff
assertTrue(gen.isOff);
}
}


This might not be the prettiest code in the world, but:
  • It works (which is more than I could accomplish with others)
  • I can run it in my IDE
  • I can run it in Ant
  • I only have to do exactly what I ever did with Java
  • Its still way better than JUnit

There were a few things I had to do to make this work:
  • When compiling in Ant, I had to add this to my scalac call:
    • target="jvm-1.5" ...
  • When compiling in Eclipse I had to do this:
    • Project -> Properties -> Scala Compiler Properties -> target = jvm-1.5

I could give a more detailed example, but I haven't tried it yet. I'm pretty sure that things like @BeforeMethod, @AfterClass, @DataProvider and what not all just work the same. I'll try to come up with a better example though.

Reading Scala

After working a bunch in Scala the other day, and continuing to obsess over it, I've realized something.

Reading the code Scala language source code is perfect code reading material.

I had said that I needed some good code to read. Scala is perfect for this. Here are the reasons its good for me.

  • It's really good code
  • Its easy to read (I guess these two go together nomally)
  • It has a really nice mix of OO and Functional Programming (which I need to get better at anyway)
  • Its got a compiler written in Scala compiling to Java bytecode. :)
  • Its a langauge for god sakes, and I want to do language design so I should look at more language implementations
But, are those the same reasons its good for someone else to read? Maybe not, but I think Scala is still great for other people to read, beginners, professionals, and Ph.d's alike.

  • The first two points hold true: Its really good code, and its easy to read.
  • Beginners could easily read the data structures code in Scala collections. They could learn their data structures, learn FP, and learn OOP better all at once.
  • Theres a wealth of stuff for more advanced people to read.
    • The Actor Model for instance, which is Scalas concurrency stuff built on top of Doug Lea's fork join.
    • Langauge Designers could read it and learn a bit.
    • Compiler writers could read it and learn a bit.
    • Software Engineers could read it, and read the examples to learn a few things:
      • How to do things better in their own language
      • What they are missing stuck in an ancient language.

As I said before, It's probably helpful to read java.util.concurrent and the java collections stuff, and I'm sure theres a whole lot of other code thats great to read too. But, I'm getting worn out when it comes to Java. I'm ready to move on. Scala is new and fresh and exciting, and, I'll learn more by doing it.

Wednesday, December 26, 2007

Unit Testing in Scala and in IDE's in General

I reviewed a few Scala unit testing frameworks today: SCUnit, and Rehersal. When all was said and done I determined something that I found rather remarkable - TestNG is the best unit testing framework for Scala. For an example click here. To listen to my mind numbing justification of that radical statement, read on...

Why aren't new Unit Testing frameworks up to snuff?

The problem with any new unit testing framework in a new language is that it doesn't have IDE support. This is a huge pain in the ass for someone who relies on their IDE for everything (like me). TestNG already has IDE support, and you can use Java Annotations in Scala, so you can use TestNG just fine in Scala as well.

Does that mean the test frameworks themselves are bad? No, they are fine. But to be entirely honest, its probably not all that worthwhile for someone to build a xUnit framework in Scala other than the fact that its probably really fun. And I say all this knowing full well that could be horse shit - I don't enough about Scala. There may be some nice syntactic stuff that makes testing in Scala just better, more readable, easier to write - I'm not sure. Its my gut feeling. Anyway...

There is still a fundamental underlying problem here.

The main problem with this whole thing is (well there actually might be a couple, but one at a time) that if you want to write an xUnit framework for a language you have write the IDE support for it too. You shouldn't have to do this.

So what can we do to fix this?

IDE's should provide xUnit framework plugin points. All you have to do is plug in your particular framework, and boom instant IDE suppport for your xUnit framework. I've been saying this for a while about languages themselves. All you need to do is plug-in the compiler and boom, you have everything. Maybe thats too much to bite off...but I don't think plugging in your own xUnit is.

The JUnit and TestNG Eclipse plug-ins are nearly identical. They both look damn near the same as the test runner for NUnit. The patterns are clearly obvious. I haven't looked at the implementations at all, so I don't yet have any concrete ideas on how to make this happen, but its so worth it. Someone should do it. It could be me.

Finally...

This could be the first step towards instant IDE support for new languages. Maybe we identify several areas like this that are similar across languages and provide plug-in points for each. What would those features be? I'm not sure I still need to think. And, maybe more languages should just use the support of Java like Scala does.

But, I have to say this was a great day. In my opinion realizing this was a real breakthrough in what I know about well, everything. Additionally, I think I am finally saying things that I haven't really heard other people saying. Maybe they are and I haven't read it, but I feel like I might actually be making some progress.

Things are coming together (and Scala is awesome)

I did a ton of reading on Scala today, and I did a lot of writing code and using Scala today. This makes me really happy. The best idea that I had today was to throw out any Java code on the CPU Simulator project, and do it all again in Scala. Why? Because if I write it in Java I won't learn as much. So, that is started, and a bunch of the new code is checked in.

Oh man I have a lot to talk about with Scala. I wonder if I should break it into many posts... I think I will.

But let me just say that I what I did today ties together everything I've been working on over the last four months. Doing the CPU Simulator in Scala helps me with languages, language design, IDE's, unit testing, compilers, reading code, preparing to teach classes, everything...

I was sick today, yet able to do work. I didn't have much energy, but sitting here was fine. Mostly I didn't want to get other people sick so I didn't go to work. Anyway, I did some of my best work today, and I was able to come up with a lot of really good ideas and I want to write them down before they are gone forever.

Wednesday, December 12, 2007

More on Reading Code

Let's revisit reading code and bring in some extra twists. (Hint: Debuggers are WRONG)

First, I know for a fact that I've read a lot of really really awful code. So based on what I said the other day, does this mean that I'm going to write really awful code? I like to think not. I'd like to think that it helps me understand what bad code is, how to read it and how NOT to write it.

Does this help me write great code? Most definitely not. To do that you need to read great code.

But, let's go back to that point about understanding the bad code. I can read bad code pretty well. I've been doing it for years. Sometimes I can actually get into the head of the developer (believe it or not, I used to write some pretty bad stuff, haven't we all?). Anyway, since I can read it well, this is just another reason I don't need to use the debugger.

By actually reading the damn code I can figure out whats going on. A well placed assertion in a unit test can confirm my beliefs. Sure, I could have used a well place break statement, but then I might be tempted to start stepping though code and wasting hours. Well thought out assertions in unit tests not only make the debugger basically useless to me, but give me regression testing going forward.

This would all be for naught if I couldn't read bad code. I read it, I understand it, I don't need the damn debugger to tell me what I already know, I write tests, I improve code.

I'm betting this: people spending hours using the debugger just need to learn how to read code better.

Sunday, December 09, 2007

CPU Simulator on Google Code

No one might actually contribute, except myself, but I put my CPU Simulator project up on http://code.google.com/p/cpusimulator.

This wasn't just for the purpose of generating interest in this project, which is likely uninteresting to most people, but it was my first venture into creating/hosting my own project somewhere in the world. This is something that any self respecting developer should do at some point.

Overall I think Google Code is fabulous. I had my project up in minutes, with my code in Subversion. You get a wiki, downloads, administration, and a number of other things automatically.

Also, and awesomely, this helps me solve a major problem. I've been bitching that I have no way to easily link to code that I've written. Well, now, see this: http://cpusimulator.googlecode.com/svn/trunk/src/main/com/joshcough/cpu/gates/NorGate.java. Totally awesome.

I was disappointed that you only get 10 projects lifetime, but I guess I'll have to choose them wisely. Maybe if I get to be a teacher, and I'm using several projects for teaching, they will lift that limit.

Anyway, I'm really happy. Great day.

Saturday, December 08, 2007

Reading Code

I haven't written and songs in a while. Way too long. Like, maybe a year. Terrible. It dawned on me just now (though I'm sure I knew it a long time ago) - I write more songs, and better songs, when I'm playing other peoples music. This is especially true when the music is difficult for me to play in some way (maybe the guitar part is hard, maybe the vocals are hard, maybe the rhythm is weird).

Notice I said "playing" other peoples music and not "listening" to. However, when I'm listening to more music, I still write more than I do when I'm not. Its just that playing is even better.

I not sure if I tie playing and listening to music to two different actions in code reading. I might try here, but the real point is that I need to read more code. And, it has to be good code of course. I need to read good code, and a lot of it. If I do my, code will become better and better.

I suppose listening to music is like glancing over code once. You want to see it to know what it does, but you don't really care to understand the thought process behind it. I guess you don't have to do that when you are playing someone else's song, but I usually do. So playing someone else's music is like, reading code really hard. Getting into the head of the coder, like getting into the head of the song writer. You can just glance at code, and just listen to a song. But, if you want to get better, you have to really concentrate on it. The better the song, the better the code, the more work put in, the better the results.

At Oswego I don't think they did enough to say that we should always be reading good code. I think most schools probably don't stress this enough unfortunately. And, even more unfortunately, most of my professional career has been filled with terrible code. Because of those two things, I didn't read much code that was any good. I didn't even know that I should.

When I struggled for a few years after college, it wasn't really obvious what the problem was. I was far and away the most talented guy in my class (I'm not even ashamed to act arrogant about this anymore, its just plain true), yet it didn't seem to translate to professional success. Finally I started reading books. Not so much reading code, but lots and lots of books. Things started looking up.

I still haven't read good code though. Well, not that much of it. I need to read more. I'm certain that when I do my code will improve, maybe even dramatically. So, I plan to attempt to find some books on code reading, potentially this one: http://www.spinellis.gr/codereading/. But, thats obviously not enough. I need choose some good code to read.

What are some good OSS projects to start reading? I read through CruiseControl Java a bit, and it was ok. But it was a little bit boring. I suppose it helps to pick something I'm familiar with. I think the java.util.concurrent that Dr. Lea wrote is the best place to start. It might be Java, which I'm getting sick of, but I'm certain its some of the most well written code there is.

But here is something else important. I'd like to attempt to put together a bunch of code, somehow ordered by increasing difficulty. Why? So I could develop my own college course in code reading. It would be so fun to have a bunch of stuff in a bunch of different languages that gets more and more difficult to read, and have the students have to tell you what it does. And, add features to it. Oh man, what a good course. Its so practical too, thats what most of my development has been (though I wish it wasn't that way). Its been mostly - here is this code base, and I need you to add this feature to it.

I think this is an essential class probably missing from most CS or SE programs. Maybe I'm wrong and just talking out of my ass, but I know it was missing at Oswego, and I consider Oswego to be one of the finest CS schools.

Wednesday, November 21, 2007

Testability vs. Encapsulation

I'm always on the side of Testability, but I'd like to hear some other peoples opinions on the subject. I think (in most situations) its totally OK to relax encapsulation in favor of testability, but some people at my work do not. Let me give a rather long example of something I came across that bothered me a bit.

I was trying to test a class - well call it G, with a Logger - LOG , which was declared like this:

static final Logger LOG = Logger.instance();

Later on in G, I found a method shutdown, which called LOG.logAndExit() and logAndExit was final and called System.exit().

G:
public void shutdown(){ LOG.logAndExit( "shutting down" ); }


Logger:
public final void logAndExit( String message ){

System.out.println(message);
System.exit(0);
}

If my tests wanted to test the shutdown method on G, the entire JVM would shutdown which simply shutdown my tests. Brilliant. I had to find some way to Mock or override the LOG variable. But there were several problems with that.

  1. It was private
  2. It was final
  3. It was static
  4. The logAndExit method was final

All of these things make difficult testing. I tried to get around the "private" by writing my PrivateFieldHelper Class. But, as it turns out, you cannot change the accessibility of static final variables at Runtime. It only works for instance fields of all types, and non final static variables. So I was stuck with the LOG object that I had.

Unless of course I relaxed encapsulation and removed the final from LOG. Then I could use my PrivateFieldHelper to set it to a new Logger of some kind. BUT...I still had a problem.
The logAndExit method was final. So even if I extended our Logger class and tried to override logAndExit so that it wouldn't call System.exit(), I still could not do so. Even when I created a Mock Logger object using JMock I had the same problem. It appears that even mock objects can't override final methods.

So once again I decided it was best to relax encapsulation. I removed the final keyword from Logger logAndExit, and created a new class - SafeLog - that overrode the logAndExit method. I used my PrivateFieldHelper to set the (now only static private) LOG field on G, and I was able to safely call the shutdown method from my test code.

What a pain.


I understand that you can go way too far on this. Some people say you should never relax encapsulation for testability, because in doing so you relax intent and readability which later creates more of a maintenance problem. In some ways I do agree with this. On public API's and Libraries you most certainly will have higher maintenance costs. But, I do think removing a final here and a final there is ok, especially if its documented. I also think removing private in favor of default (package private) is ok.

What do you think? Does anyone know of any good articles or books explaining the trade-offs?

God this post is about to get long....

Some say you shouldn't relax private for package private, and all testing should be done through public methods. This is another one I just don't agree with. Lets say you have a reasonably complicated class that only exposes one public method. People reading the class later on might not know what inputs are valid for and what outputs are expected for each of the private methods in the class. You certainly can, and should get 100% code coverage through testing public methods, but it still might not be immediately obvious to someone reading the code later on what those private methods are doing.

Testing private (or package private) methods extensively should make it immediately obvious. Once again, What do you think? Does anyone know of any good articles or books explaining the trade-offs?

I think there can and should be things built into the languages themselves to expose hidden members to testing. Something like a test keyword. Or like JSR 294, superpackages, which define what classes in a package are accessible, and to whom. If it things like this were built right into the language, exposing members to testing, then we wouldn't even be having these debates.

How would it work? Maybe a lot like Generics. All the type information is removed at compile time, and so could any test availability information. You could turn this off. You could say, produce a jar for testing, and produce a jar for delivery. Its not that hard. Just ideas...You have any?

Friday, November 16, 2007

CPU Simulator Library in Progress

I've been working on a lot of stuff lately and not writing. For that I apologize. I want to keep everyone updated on what I'm doing, but mostly I want to keep myself update for tracking progress and what not.

I'm still wanting to design languages, write compilers, write IDE thingies, and all of that. If you recall, I decided that in order to do that I should become a compiler master. In reading a lot about compilers I decided I should take yet another step back to refresh my memory on Computer Architecture.

So I've been reading about that a LOT. Really low level, as low as possible. When I say as low as possible I even mean Quantum Physics. But, I don't intend to become a master in Quantum Physics. Its so fascinating but it also really almost drove me to insanity about 6 years ago. A simple refresher was all I needed.

After that I read a bunch about the history of computers, mixed in with some history of philosophy, and mathematics. All really great stuff. Once again I don't intend to become an expert in it but I want solid foundations in all this stuff, which I feel I don't quite have. I'm close.

So heres where the fun part begins. I started writing this cool little CPU simulator in Java. I don't have much yet but I'll keep the world informed. What I plan to do is - write classes that represent pretty much everything you'll find in a computer hardware. So far I have a MemoryModule class and things like that. You can imagine this pretty easily.

We can start with a Computer class which has CPU, Memory, IO, and a Bus. We can then break the CPU down into ControlUnit, ALU, Registers, and maybe Connections. The ControlUnit can be broken down further into SequencingLogic, Registers, Decoders, ControlMemory, etc...

This process of drilling down can probably go down to the atomic level. I'm not sure how low I'd go, where I'd draw the line. But the point is by doing all this, by actually writing the code, ensures that I understand each part. I could take some parts down further than others, and in doing so I'd learn more about that component. This is fine. I probably don't need to know things down to the atomic level on say, video. But going down to FullAdders and HalfAdders in the ALU, or even down to LogicGates is probably a useful thing to do.

Maybe I can invent my own instruction set for my "Hardware", or reuse an existing one. Either way, once all of this is done it would be nice to write a little OperatingSystem class that makes use of that instruction set. I would learn a lot in doing so I imagine. Then once my OperatingSystem is in a reasonable state (I don't have any idea what that would look like yet), I could attempt to write a little compiler that compiles some random language down to an Assembly Language that runs on my hardware.

This is a long way off but I think its a damn good goal. Yes its reproducing things that people have done 50 years ago in a way that most certainly won't be reusable, but whatever, it will get me towards my goal of language writing.

PrivateFieldHelper Class

Heres a little class I wrote to set private fields. I plan to use it a little for testing, but only when I'm backed into a corner, and can't change code otherwise.



/**
*
* @author Josh Cough
*
*/
public class PrivateFieldHelperImpl implements PrivateFieldHelper {

Class clazz;

/**
*
* @param c
*/
public PrivateFieldHelperImpl(Class c) { this.clazz = c; }

/*
* (non-Javadoc)
* @see com.joshcough.reflect.PrivateFieldHelper#setStaticFieldValue(java.lang.String, java.lang.Object)
*/
public void setStaticFieldValue(String fieldName, Object newValue)
throws IllegalArgumentException, NoSuchFieldException {
setPrivateFieldValue(findPrivateStaticField(fieldName), null, newValue);
}

/*
* (non-Javadoc)
* @see com.joshcough.reflect.PrivateFieldHelper#getStaticFieldValue(java.lang.String)
*/
public Object getStaticFieldValue(String fieldName)
throws NoSuchFieldException, IllegalArgumentException {
return getPrivateFieldValue(null, findPrivateStaticField(fieldName));
}

/*
* (non-Javadoc)
* @see com.joshcough.reflect.PrivateFieldHelper#getInstanceFieldValue(java.lang.Object, java.lang.String)
*/
public Object getInstanceFieldValue(Object instance, String fieldName)
throws NoSuchFieldException, IllegalArgumentException {
return getPrivateFieldValue(instance, findPrivateInstanceField(fieldName));
}

/*
* (non-Javadoc)
* @see com.joshcough.reflect.PrivateFieldHelper#setInstanceFieldValue(java.lang.Object, java.lang.String, java.lang.Object)
*/
public void setInstanceFieldValue(Object instance, String fieldName, Object newValue)
throws IllegalArgumentException, NoSuchFieldException {
setPrivateFieldValue(findPrivateInstanceField(fieldName), instance, newValue);
}

/**
*
* @param f
* @return
* @throws
*/
private Object getPrivateFieldValue(Object instance, Field f) {
f.setAccessible(true);
Object o;
try {
o = f.get(instance);
} catch (IllegalAccessException e) {
throw new PrivateFieldException(e);
}
f.setAccessible(false);
return o;
}

/**
*
* @param f
* @param newValue
* @throws
*/
private void setPrivateFieldValue(Field f, Object instance, Object newValue){
f.setAccessible(true);
try {
f.set(instance, newValue);
} catch (IllegalAccessException e) {
throw new PrivateFieldException(e);
}
f.setAccessible(false);
}

/**
*
* @param fieldName
* @return
* @throws NoSuchFieldException
*/
private Field findPrivateStaticField(String fieldName) throws NoSuchFieldException {
for (Field f : clazz.getDeclaredFields()) {
if (f.getName().equals(fieldName)) {
if (Modifier.isStatic(f.getModifiers()))
return f;
}
}
throw new NoSuchFieldException();
}

/**
*
* @param fieldName
* @return
* @throws NoSuchFieldException
*/
private Field findPrivateInstanceField(String fieldName) throws NoSuchFieldException {
for (Field f : clazz.getDeclaredFields()) {
if (f.getName().equals(fieldName)) {
if (! Modifier.isStatic(f.getModifiers()))
return f;
}
}
throw new NoSuchFieldException();
}

}

Cruise Control Remote Management API

I've never contributed to an OSS project before. Why that is I'm not quite sure. Looking through the CruiseControl Java code its pretty clear to me that I have the skills. I understand the code and I understand ways that I could refactor and improve it. I understand that I probably had those skills years ago. So why I haven't is beyond me; maybe I just never had the confidence.

Anyway, I've finally submitted something. I wrote a CruiseControl Remote Management library that wraps the exposed JMX attributes and operations for a server and its projects. You can do nice things like getting a reference to a server, getting all its projects, force builds on projects, set labels on projects, lots of nice stuff. Here's a simple example that forces a build on all projects:

CruiseServer server = new Server("localhost");
List projects = server.getProjects();
for( CruiseProject p: projects ){ p.forceBuild(); }



You could do nice things with Build Pipelining like trigger builds in other CruiseControl instances on different servers after a project on your server builds. Indeed, thats actually what I've done.

I think that this code could be used to clean up a lot of the code in their current tree. Maybe I'm wrong, or maybe they already have something like this, but I didn't see it.

Hopefully they will take this and add it in. Maybe they will ask me to add it in. Maybe I'll get to refactor a bunch of their Dashboard code. I'm almost certain it would be cleaner using this.

We'll see what happens.

Tuesday, October 30, 2007

Closures

Can anyone tell me how closures are implemented?

Scala has closures, and it compiles to Java bytecode, but, Java doesn't have closures. This is not to say that it can't be done, of course. I'm sure the implementation is probably not that hard. I'm just really curious to know how its done... I could go look at the source code for the Scala compiler of course. But I have been working on a number of other things.

I wrote a library for remote management of CruiseControl that seems to be much easier to use than the one they are using in the new Dashboard. I'm going to try to get that submitted here shortly. I'm starting to feel pretty confident in my development skills. Finally. I'm also starting to feel confident in my code reading skills too. I was able to look through the CC Java code with ease. Of course it helps that its mostly nice clean code, but still, I'm doing well.

Well enough, I suppose, that I could figure out how closures are implemented. And, since no one reads my blog anyway, I think I'm on my own.

Thursday, October 18, 2007

But Thats How It Was When I Got Here

Companies typically let their build system go to shit.

Why is that? Does anyone have an answer to this? The first thing I ever do on new projects is make sure I have a consistent build process. ThoughtWorks was the same way. The first thing I look at when joining a new company is the build process, and how to fix it. Why don't most companies do this themselves? I have some ideas, here they are:
  • They don't know how.
  • They somehow (wrongly) think its not as valuable as developing the next features
  • They get comfortable with the fact that it takes an hour to build, because it works
  • They suffer from "But Thats How It Was When I Got Here" syndrome
All of these are BAD BAD BAD reasons, and all make me very offended. Those might not be the only reasons, and they might not be the best reasons, but regardless, I'm going to tear them apart. My intention is not to make my coworkers feel bad, but to wake them up to the reality that theres a lot of new technology out there that they need to be learning and leveraging.


They don't know how.

If you don't know how to do something entirely modern, then you need to start learning. Everyone knows this, but so few people do it. Why? (Topic for a whole new blog posting). If you don't do it you're going to be passed by. This will occur first on the individual level, people will start to pass you making more money and you'll wonder why. Well, knowledge is why.

Worse though, if this is somehow your corporate culture, not learning new technology, eventually your whole company is going to stagnate. I refuse to let this happen at my group. Fortunately we have some great people who are eager to learn.


They somehow (wrongly) think its not as valuable as developing the next features

If you think its not valuable, you're dead wrong. If you don't have a repeatable build process someone will end up making a mistake and you'll deliver something broken to a client. Maybe that never happens, but you WILL end up slowly adding to your awful build process until you get something that is a total pile of nonsense. With this pile, any change takes days or weeks to figure out. You have to figure out all the side effects, you have to somehow verify that a updated build process produces the same results as the old process.


They get comfortable with the fact that it takes an hour to build, because it works

Believe it or not, if you are comfortable with your build process this is actually a sign that something might be wrong. I am never comfortable with my build process. I'm always tweaking it, trying to make it run faster, trying to remove duplication from it, trying this trying that...I think a good build should be less than 5 minutes. Some people say ten. I don't agree. If you have an hour build, you're likely doing a bunch of manual steps and will run into the problems I pointed out in the last step. If you have an hour long build and its completely automated then you have other issues that aren't quite as serious, but are still very bad.


But Thats How It Was When I Got Here

This one is troublesome to me. I HATE legacy software. I feel like. I don't know, some kind of Vigilante on a mission to KILL it. Unfortunately, even I find myself saying, "But thats how it was when I got here" from time to time. This is not a good excuse. If you're saying that now, you'll probably go on saying this until you end up in a situation like the last two items. This is almost an excuse for all the other items rolled up into one. Its like saying, "Yeah I know its bad, but what can you do?" And on top of that, its like, a way out, a way of saying, "I'm not going to deal with that problem." Well, guess what? You are going to deal with it, the hard way.



So is this all avoidable? Of course. "But how?" One simple way, READ. DAMN YOU. READ MORE. Thats it. Really. Just read what people are writing and you'll learn how to do things right. Now, if only we could get everyone to read. I smell a post.

Agitar Examples

I finally have a chance to post some examples of code generated by Agitar. I promised them in comments to the original post, but it turns out this is easier. First, I'll give a little background on the code that I wanted to test, and then I'll show the tests generated by Agitar. This example is designed to show the readability of Agitar. If a test fails after a change and its difficult to read, what approach should a developer take - read the test and see why it failed, or regenerate the test?

I wrote a simple interface for returning all the files under a directory:

(By the way I lost all my formatting and I apologize. Formatting on Blogger is a PITA.)

public interface ListFilesStrategy {
public List listFiles(File dir);
}


I have a two classes implementing this interface, RecursiveListFilesStrategy and NonRecursiveListFilesStrategy, which both extend AbstractListFilesStrategy.Here are the listings for AbstractListFilesStrategy and NonRecursiveListFilesStrategy:

public abstract class AbstractListFilesStrategy {

public void assertFileIsDirectory(File dir) {
if( dir == null )
throw new IllegalArgumentException("directory is null");

if( ! dir.isDirectory() )
throw new IllegalArgumentException("file is not directory");
}

}

public class NonRecursiveListFilesStrategy extends AbstractListFilesStrategy
implements ListFilesStrategy{

public List listFiles(File dir){
assertFileIsDirectory(dir);

List files = new ArrayList();

List directories = new ArrayList();
directories.add(dir);

while(! directories.isEmpty()){
File currentDir = directories.remove(0);
for( File f: currentDir.listFiles()){
if (f.isFile()) files.add(f);
else directories.add(f);
}
}
return files;
}
}

Now the test Agitar generated that was difficult to read. This test tested the listFiles method on my NonRecursiveListFilesStrategy:

public void testListFilesWithAggressiveMocks1() throws Throwable {
NonRecursiveListFilesStrategy nonRecursiveListFilesStrategy = new NonRecursiveListFilesStrategy();
File file = (File) Mockingbird.getProxyObject(File.class);
File file2 = (File) Mockingbird.getProxyObject(File.class);
File[] files = new File[0];
File file3 = (File) Mockingbird.getProxyObject(File.class);
File[] files2 = new File[2];
File file4 = (File) Mockingbird.getProxyObject(File.class);
File file5 = (File) Mockingbird.getProxyObject(File.class);
files2[0] = file4;
files2[1] = file5;
Mockingbird.enterRecordingMode();
Boolean boolean2 = Boolean.TRUE;
Mockingbird.setReturnValue(false, file, "isDirectory", "()boolean", new Object[] {}, boolean2, 1);
ArrayList arrayList = (ArrayList) Mockingbird.getProxyObject(ArrayList.class);
Mockingbird.replaceObjectForRecording(ArrayList.class, "()", arrayList);
ArrayList arrayList2 = (ArrayList) Mockingbird.getProxyObject(ArrayList.class);
Mockingbird.replaceObjectForRecording(ArrayList.class, "()", arrayList2);
Boolean boolean3 = Boolean.FALSE;
Mockingbird.setReturnValue(false, arrayList2, "add", "(java.lang.Object)boolean", new Object[] {file}, boolean3, 1);
Mockingbird.setReturnValue(arrayList2.isEmpty(), false);
Mockingbird.setReturnValue(arrayList2.remove(0), file2);
Mockingbird.setReturnValue(false, file2, "listFiles", "()java.io.File[]", new Object[] {}, files, 1);
Mockingbird.setReturnValue(arrayList2.isEmpty(), false);
Mockingbird.setReturnValue(arrayList2.remove(0), file3);
Mockingbird.setReturnValue(false, file3, "listFiles", "()java.io.File[]", new Object[] {}, files2, 1);
Mockingbird.setReturnValue(false, file4, "isFile", "()boolean", boolean2, 1);
Mockingbird.setReturnValue(false, arrayList, "add", "(java.lang.Object)boolean", boolean3, 1);
Mockingbird.setReturnValue(false, file5, "isFile", "()boolean", boolean3, 1);
Mockingbird.setReturnValue(false, arrayList2, "add", "(java.lang.Object)boolean", boolean3, 1);
Mockingbird.setReturnValue(arrayList2.isEmpty(), true);
Mockingbird.enterTestMode(NonRecursiveListFilesStrategy.class);
List result = nonRecursiveListFilesStrategy.listFiles(file);
assertNotNull("result", result);
}


This is certainly difficult to read. I'm pretty sure I could explain what its doing, but I'm more well read in testing and mocks than most developers. There are a few things they could do to clean it up however. They could try to be more in line with Behavior Driven Development and they have a few options in doing so. The method doesn't have an intent revealing name, I know what method its testing, but I don't know exactly what its doing. They could put in "given, when, then comments". The could use the extract method refactoring to seperate out the setup, the action, and the assertions, but maybe at least the setup portion.

All these things could be done, but even so there is a lot of setup happening here, and Most developers aren't ready, or aren't willing to read through it, and will likely start to ignore Agitar errors.

Now, I was told by Barry at Agitar that I could put in a test helper class, which would basically provide the setup portion of the test, and then it would generate more meaningful results. Here is an example:

public class FileTestHelper implements ScopedTestHelper {

public static File createFile() {
return new File("./src/main");
}
}


Should result in something like this …

public void testListFiles() throws Throwable {
ArrayList result = (ArrayList) new NonRecursiveListFilesStrategy().listFiles(FileTestHelper.createFile());
assertEquals("result.size()", 5, result.size());
}

Indeed, I could do something like this. But, there is a big problem with this. We have 5000+ classes that we want to generate tests for! How do we know which tests are meaningful, which ones need test helpers, yada yada?

I do promise more examples of simple methods that don't seem to be giving meaninful results. Mostly just checking to make sure methods catch NullPointerException. I do think the product will work wonderfully for green development, but will likely be ignored during legacy developement. Please let me know your thoughts.

Sunday, October 14, 2007

IDE's vs vi

I just told my friend that instant IDE support for new languages is the place to be (I truly believe this can happen). In response he told me, "I love vi."

I know vi, and I can get around in it OK, but I'm not a hardcore vi guy. I know there are some things you can do in vi that are really nice and allow you to do some things very quickly and powerfully. But, I don't know what they are. The real question here is, can vi do things that modern IDE's cant do? Can someone explain to me this?

I do think that vi is ancient technology, and I'm sure thats old news anyway. I don't know very many people who use vi or emacs anymore. I'm sure a lot of people would look at my friend funny for that comment. But I'm a little different. I'm curious.

What features do retired editors have that modern IDE's lack?