I spent some time recently with Sue Kunz of Solidware Technologies. Sue and her co-founder Char Devich were both high-level managers with Sun, and left in order to start Solidware in October of 2004. Ive heard Sue described as a firecracker and someone you dont want to bet against, so I was curious to meet her and see what she was up to.
Solidware is developing a very interesting software quality engineering platform called Splat (cool name). Current SQA tools such as Worksoft, Segue and Mercury perform traditional functional (black box) testing while those such as Agitar, Klocwork, Fortify, and Coverity address structural (white box) testing. Some systems, such as IBM Rational address both types.
With Splat, Solidware is attempting to introduce a new software quality paradigm called analytical testing. This type of testing is commonplace in other engineering disciplines, but doesnt happen today in software.
I took a look at a pre-beta version of Splat 2.0 and fed it about 80,000 lines of my own Java code. I figured that if my code cant break it then nobodys can. Splat crunched my code for a minute or so and then spit out a beautiful interactive flash view of my code. Heres a sample screen shot so you get the idea.
Splat lets you quickly visualize your code to identify hot spots (in red) that carry unusual proportions of risk. You can keep diving down into the riskier areas of code, and Splat continues to help you visually isolate your risk. The Splat analytics engine uses some 40+ metrics with cool names like cyclomatic complexity to figure this stuff out.
The code I used to put Splat through it’s paces was developed by a team of 4 people. It consisted of one rock star (no, not me!), and 3 other developers. I found it interesting that in general Splat found the riskiest code to be the code developed by the best developer.
When I discussed this result with the Solidware team, they told me that this is actually fairly typical. The superstar of the team is typically doing the hardest and most complex work relating to the architecture of the system or the trickiest aspects of the implementation. I had to think back to my computer science classes (wow, that was a long time ago) to recall that risk is not bad. Unmanaged risk and risks not fully understood are bad.
Splat also helped me see that one of the three remaining developers on the project had been writing unnecessarily complex code. Upon reflection, I realized that we were getting a disproportionate number of bug reports and logged errors from the parts of the system that Splat identified.
All of this was a neat validation for me that the product really seems to work. If this thing was integrated into my IDE and was analyzing my code in real time as I was developing it, I would use it. I hope thats where Solidware takes it.
You can play around with the Splat 1.0 prototype right now, but it only works with C code and takes some effort to get it going. Its easier to check out the animated demo. The upcoming 2.0 version handles Java and appears to me to be a major jump forward in many ways for the product.
What do you think? Is analytical testing the way of the future?
