Editor’s note: Tim Macer, managing director of U.K. consulting firm meaning ltd., writes as an independent software analyst and advisor.

We always try to present something new in these software reviews, but this time, we think we are onto something that could break the mold: Ruby, a new tabulation software package from an Australian producer, Red Centre Software, that leaves most of the existing choices looking decidedly dated. It’s refreshing, because for a while, most efforts in market research software seem to have gone into improving data collection and making it work across an ever-broadening spectrum of research channels. Innovation at the back end seems to have focused on presentation, and has often left research companies and data processing operations with a mishmash of technology and a few lash-ups along the way to transform survey data into the range of deliverables that research clients expect today.

Ruby could easily be mistaken for yet another end-user tabulation tool like Confirmit’s Pulsar Web or SPSS’s Desktop Reporter, with its graphical user interface and drag-and-drop menus. The reality is that it is a full-fledged tabulation and reporting system aimed squarely at the data processing professional. If you are looking for a Quantum replacement, this program deserves a test-drive.

No limits

There appear to be no limits on the data you could use. It will import data from most MR data formats, including Quantum, Triple S and SPSS. Internally, it works with flat ASCII files, but it is blisteringly fast, even when handling massive files. It will handle hierarchical data of any complexity and offers the tools to analyze multi-level data throughout, which is something modern analysis tools often ignore.

It is equally at home dealing with textual data. The producers provided me with a series of charts and tables they had produced from analyzing Emily Brontë’s Wuthering Heights by treating the text as a data file. The same could be done for blogs, RSS feeds and the mass of other Web 2.0 content that many researchers feel is still beyond their grasp.

More conventionally, Ruby contains a broad range of tools specifically for handing trackers, so that you are not left having to automate the reconciliation of differences between waves due to variations in the question set and answer lists.

Ruby is very intelligent when it comes to processing the data. The data in the tables reported or charted in MR has often gone through a long chain of transformations, and in the old tools, there could be yards of “spaghetti code” supporting these transformations. Trying to work out why a particular row on a table is showing zeroes when it shouldn’t can take forever in the old tools, as you trace back through this tangle of code, but Ruby will help you track back through the chain of definitions in seconds, and even let you see the values as you go. It is the kind of diagnostic tool that DP professionals deserve but rarely get.

In Ruby, you will probably make most of these data combinations and transformations visually, though it does also allow you to write your own syntax, or export the syntax, fiddle with it and import it again (the combination that DP experts often find gives them the best of both worlds). However, Ruby keeps track of the provenance of every variable, and at any point you can click on a variable and see exactly where the data came from and even see the values at each stage.

Broad range of expressions

The array of options for tabulation and data processing is immense, with a broad range of expressions that can be used to manipulate your data or columns and rows in tables. There is complete flexibility over percentaging and indexing values off other values, or basing one table on another, so it is great for producing all of those really difficult tables where every line seems to have a different definition.

With charting, Ruby gives you the choice of using its own proprietary charting engine or pushing the data out to PowerPoint or Excel charts. The native Ruby charts are a treat to work with, as the developers seem to have gone out of their way to redress the inadequacies of Excel and PowerPoint charts. For time-series charts, concepts such as smoothing and rolling periods are built-in. You can add trend lines and arbitrary annotations very easily. Charts can be astonishingly complex and can contain thousands of data points or periods, if you have the data. Yet it will always present the data clearly and without labels or points clashing, as so often happens in Excel.

Excel and PowerPoint charts are also dynamic, and the Ruby data source will be embedded in the chart so that the charts can be refreshed and updated if the underlying data changes.

Charting is a particular strength of Ruby, which handles very large amounts of data swiftly and intelligently. This chart, for example, contains 200 weeks of smoothed data.

This is what Ruby describes as the specification. There is no syntax in sight, but hidden around this window are hundreds of options that can be applied to transform the variables that make up the current table or chart.

Completely flexible

Amy Lee is data processing manager at Inside Story, a market research and business insights consultancy based in Sydney, Australia. She has been using Ruby for two years, alongside five other researchers and analysts, to analyze custom quantitative projects and a number of large-scale trackers.

“We were able to move to Ruby a couple of years ago, and it is now the main program we use, because it can do everything we need to do. I find it is an extremely powerful and flexible tool. Other tools can be quite restrictive, but Ruby is very powerful and completely flexible,” Lee says.

The program goes beyond what more traditional DP crosstab tools enable her to do, she says. “Compared with other programs I have used, Ruby allows me to filter and drill down into the data much more than I could with them. It’s especially good at exporting live charts and tables into documents.

“Once they are in PowerPoint or Word, trend charts can be opened up and adjusted as necessary. When it is a live chart, it means you can update the data. Instead of having to go back to Ruby, open it up and try to find the chart and then read the data, you can just double-click it inside PowerPoint and you see all the figures change. And there is even an undo feature, which is good for any unintentional errors.”

Lee freely admits that this is not a program you can feel your way into using without having some training and time to understand it. “It is really designed for a technical DP person. If you have someone with several years’ experience with another program they will have no problem picking this up, as everything will be very familiar to them. But we also had a client who wanted to use it, someone with a research rather than a DP background, and they found it a bit overwhelming, because it can do so much. It looks complex, but once you get the hang of it, you can do what you need very quickly.”

Among the other distinguishing features Lee points to are the speed of the software, which is very fast to process large amounts of data and produce numerous of tables and charts; its built-in handling of time-series, allowing you to combine or suppress periods very easily; and the range of charts offered, in particular the perceptual maps.

So far behind

Some of the research companies I speak with are becoming uneasy that the legacy data processing tools they depend on have fallen so far behind and are, in some cases, in danger of becoming dead products. They have endured because the GUI-based “replacements” bundled in with the more modern data collection tools just don’t cover the breadth of functionality that is needed. Even though the sheer range of functionality it offers is bewildering to the newcomer, you can’t criticize Ruby for lacking breadth or depth in what it lets you do.