Visualization Tool Data Formats - A Constant Frustration

I was just looking at some java script and flash visualization tools: MooWheel, JavaScript Information Visualization ToolKit JIT, Open Flash Chart.

And there it is again, that frustration about data formats. I wanted to try the tools with my own data, just to realize that each and every tool had another input requirements. None of them takes simple CSV input! They want arrays:

var data = [{
             id: 'joeb',
             text: 'Joe B',
             connections: ['ryank', 'charliec']

My favorite - how can I be surprised - JSON. It had to happen. I hate the Web 2.0 people for this. Sorry.

var json = [
	"id": "aUniqueIdentifier",
	"name": "usually a nodes name",
	"data": [
	    {key:"some key",       value: "some value"},
		{key:"some other key", value: "some other value"}

All of these formats are just absolutely horrible to generate. I have CSV data, or at least I can generate that easily! Will I really have to write converters for all of this?

It's all about the interface.

What it comes down to is selecting an interface between orthogonal processes.

Somehow, somewhere, some-when, each of the two partner processes need to know what is being sent and received.

They can know that statically or dynamically.

It can be coded into the interface, ala json et all, or it can be defined external to the functional interface, ala csv.

With json and the others, what each parameter is can be encoded into the actual interface by the selection of the parameter names so any coder can pick up the interface definition and build against it.

But, that interface essentially becomes carved in stone and its apparent applicability instantly limited.

With csv, the contents/format of the csv file/string needs to be known a` priori before it can be used in the interface.

It can be thought of as "late binding" or "Run time binding".

If you use a more flexible interface definition separating function from format, the interface essentially becomes composed of two elements, one parameter to communicate the data structure implemented in a csv, or other similar formatted string and the string itself.

In this way, the actual data format of the data interface is defined external to the function interface.

Both ways have their advantages and disadvantages.

My personal preference is the more flexible and extensible method because a point the author of the article doesn't mention but is almost implied, data lives longer than the apparent lifetime of any function implemented to work on that data.

For example, I have log lines from years ago but the analysis I performed on them is not the same as the analysis I perform on log data now. You could say that the basic "interfaces" haven't changed because I still go through the same types of steps in the analysis but, the implementation of and the data used by each step is different now than it was then and different now than it will be next month or next year.

What it really comes down to is whether or not you take a data centric view or a process centric view.

I'm not sure why you are so

I'm not sure why you are so frustrated. Any modern scripting language (i.e Python, Ruby) should allow you to dump your hashes, arrays, etc into JSON or convert back/forth between CSV and JSON.

JSON output from scripts

Not sure that'll work. JSON is not JSON. Meaning that I can come up with any arbitrary schema for JSON. Each visualization tool is using its own schema. Some use an ID field, others use children, others use hierarchies, etc. etc. It's a mess. How would you write that code?
Printing out the hash from Ruby or Python might work. However, I have to write code. Think about how many people would like to use the tools but don't know how to convert their data? I think this is one of the biggest reasons that visualization is still not used enough.

Try Tableau Software

LINK deleted by moderator...

Why using a commercial tool

Why would I use a commercial visualization tool to achieve my goal of converting data from CSV to JSON or something? I don't need a visualization tool. I need JavaScript code that I can adopt and use to visualize my own data.