Thursday, February 3, 2011

Graph visualization with Neo4J and Arbor.js


As you might have gathered from my previous post, I have been tinkering with Neo4J lately and I am beginning to appreciate the power and elegance of graphs.
I also came across Arbor.js and was bowled over by it. Loved the force directed layouts and the fact that if you are patient, you will watch your nodes traveling far and wide to settle in their final destination. No need to drag, drop or re-organize the graph- what emerges is a visual pattern, obvious to the eye. I can think of tons of usages for such a visualization, but thought I'd start with something simple just to get the feel of it.


I decided to model a simple organization chart or graph. So off I went to create that model in Neo4J, fed data into it, and my graph was ready very quickly.
On to Arbor.js- I must admit, I was rather stumped, even with the reference doc they publish. I did not know where to start and where to end- of course, my lack of knowledge around jQuery syntax contributed a lot.


So, I did the next best thing which was to get the sample project working locally. Tweak main.js to play with the layout and understand what's really happening. Scroll right down to the $(document).ready function to see the 'ParticleSystem' getting initialized. The sample project initializes with three parameters: repulsion, stiffness and friction. There are more available like frames per second and precision, but I have not tried them yet. Play with those values to see how they affect the graph. I settled on: var sys = arbor.ParticleSystem(500, 600, 0.5)
I also left gravity=true since I definitely want the settling graph.


You can then add edges and nodes to the ParticleSystem object you just initialized.
Adding a node is as simple as sys.addNode(name,data).
The name you supply functions as an identifier of the node and the data is an object you can use to associate any set of parameters with this node. These are especially useful when you need additional information about the node to render it- for example, set its color.
Adding an edge is just as simple: sys.addEdge(source,target,data). Source and target nodes are identified by their node names, and data again is an object of parameters to associate with the edge.


Now the sample project displays little black boxes for nodes. I was looking for something more like circular nodes with their labels in them- much like the arbor.js home page.


Just adding nodes and edges does not do anything- the ParticleSystem needs a renderer to well, render the graph. The arbor.js reference will tell you that you need to create an object with two methods- init and redraw, and set your ParticleSystem renderer to this object.

If you look at
main.js, this is exactly what it does (viewport is the canvas object in which to render the graph)-
sys.renderer = Renderer("#viewport")

The init method in this case simply sets up screen dimensions.
The redraw method is the one responsible for painting nodes and edges. You will see those in the
particleSystem.eachEdge and particleSystem.eachNode loops.

So then this is the place to tweak the actual representation of nodes and edges.
Borrowing from the source of the arbor.js home page, I changed the code to:
particleSystem.eachNode(function(node, pt){

var w = Math.max(20, 20+gfx.textWidth(node.name) )
if (node.data.alpha===0) return
if (node.data.shape=='dot'){
gfx.oval(pt.x-w/2, pt.y-w/2, w, w, {fill:node.dat
a.color, alpha:node.data.alpha})
gfx.text(node.name, pt.x, pt.y+7, {color:"white", align:"center", font:"Arial", size:12})
gfx.text(node.name, pt.x, pt.y+7, {color:"white", align:"center", font:"Arial", size:12})
}else{
gfx.rect(pt.x-w/2, pt.y-8, w, 20, 4, {fill:node.data.color, alpha:node.data.alpha})
gfx.text(node.name, pt.x, pt.y+9, {color:"white", align:"center", font:"Arial", size:12})
gfx.text(node.name, pt.x, pt.y+9, {color:"white", align:"center", font:"Arial", size:12})
}

})
You will note that the color of the node is pulled from the data object attached to the node.
The font, size, width etc. can all be parametrized by setting them in the data object when you create the node.
This example uses the graphics library, so be sure to include graphics.js in your html.


For the edges, I changed the color and also parametrized the thickness of the connector line by the width parameter.
particleSystem.eachEdge(function(edge, pt1, pt2){
if (edge.source.data.alpha * edge.target.data.alpha == 0) return
gfx.line(pt1, pt2, {stroke:"#151B8D", width:(edge.data.weight/2), alpha:edge.target.data.alpha})

})


That is sufficient for a basic graph representation. Now to get the data out of Neo4J and into the graph.
The ParticleSystem has a graft method which accepts an object in the form
{nodes:{}, edges:{}}.
These nodes and
edges are added to the particle system.
I navigated through my Neo4J graph and spat out the data in this json like format.

Then, in the JS, you simply do a
var data = $.getJSON("orgChart.json",function(data){
sys.graft({nodes:data.nodes, edges:data.edges})
})

and you're done.
Just make sure you enclose each attribute in quotes- for example, color and shape in the code below.
And it also freaks if you supply a .8 instead of a 0.8 for the length attribute in this example.


The final graph rendered in my browser looks like this(click to enlarge)-

This really just scratches the surface of what you can do with arbor.js
Add to it the various types of graphs or models you can extract out of Neo4J, and you could build some great stuff!


orgChart.json contains the following-
{"nodes":{
"Aidan":{"color":"green", "shape":"dot", "alpha":1},
"Sofia":{"color":"green", "shape":"dot", "alpha":1},
"Liam":{"color":"GoldenRod", "shape":"dot", "alpha":1},
"Charlotte":{"color":"green", "shape":"dot", "alpha":1},
"Noah":{"color":"GoldenRod", "shape":"dot", "alpha":1},
"Ava":{"color":"green", "shape":"dot", "alpha":1},
"Jackson":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Addison":{"color":"green", "shape":"dot", "alpha":1},
"Ethan":{"color":"green", "shape":"dot", "alpha":1},
"Olivia":{"color":"green", "shape":"dot", "alpha":1},
"Kayden":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Amelia":{"color":"green", "shape":"dot", "alpha":1},
"Mason":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Lily":{"color":"green", "shape":"dot", "alpha":1},
"Logan":{"color":"green", "shape":"dot", "alpha":1},
"Isabella":{"color":"green", "shape":"dot", "alpha":1},
"Jacob":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Emma":{"color":"green", "shape":"dot", "alpha":1},
"Benjamin":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Abigail":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Caleb":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Scarlet":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Owen":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Audrey":{"color":"GoldenRod", "shape":"dot", "alpha":1},
"Gavin":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Grace":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Connor":{"color":"green", "shape":"dot", "alpha":1},
"Claire":{"color":"GoldenRod", "shape":"dot", "alpha":1},
"Riley":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Emily":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Nolan":{"color":"lightblue", "shape":"rectangle", "alpha":1},
"Quinn":{"color":"red", "shape":"dot", "alpha":1}
},
"edges":{
"Quinn":{"Liam":{"length":2.5,"weight":2},
"Audrey":{"length":2.5,"weight":2},
"Noah":{"length":2.5,"weight":2},
"Claire":{"length":2.5,"weight":2}},
"Liam":{"Sofia":{"length":2.5,"weight":2},
"Ethan":{"length":2.5,"weight":2},
"Amelia":{"length":2.5,"weight":2}},
"Audrey":{"Aidan":{"length":2.5,"weight":2},
"Ava":{"length":2.5,"weight":2},
"Jackson":{"length":2.5,"weight":2}},
"Claire":{"Charlotte":{"length":2.5,"weight":2},
"Olivia":{"length":2.5,"weight":2}},
"Noah":{"Addison":{"length":2.5,"weight":2},
"Kayden":{"length":2.5,"weight":2}},
"Olivia":{"Mason":{"length":2.5,"weight":2},
"Lily":{"length":2.5,"weight":2}},
"Ethan":{"Logan":{"length":2.5,"weight":2},
"Isabella":{"length":2.5,"weight":2}},
"Addison":{"Jacob":{"length":2.5,"weight":2},
"Emma":{"length":2.5,"weight":2},
"Benjamin":{"length":2.5,"weight":2}},
"Emma":{"Abigail":{"length":2.5,"weight":2},
"Caleb":{"length":2.5,"weight":2}},
"Logan":{"Scarlet":{"length":2.5,"weight":2},
"Owen":{"length":2.5,"weight":2},
"Gavin":{"length":2.5,"weight":2}},
"Lily":{"Connor":{"length":2.5,"weight":2},
"Grace":{"length":2.5,"weight":2},
"Nolan":{"length":2.5,"weight":2}},
"Connor":{"Riley":{"length":2.5,"weight":2},
"Emily":{"length":2.5,"weight":2}},
}
}




23 comments:

damariri said...

Thank you for the info! I'm currently starting to learn arborjs and your comment was quite useful, It seems that the documentation is not very thorough, I have a question:
¿where do I have to put the $.getJSON in my javascript...is it inside the variable "that" (the one where init, redraw and other functions are included) or outside? Does it have to be part of another function? That is the way I've seen it in atlas and the other examples.

Thanks a lot,

Dámaris

Aldrin & Luanne Misquitta said...

Hi, the getJSON goes into your ready function.
$(document).ready(function(){
var sys...
sys.renderer...
var data = $.getJSON("xyz.json",function(data){
sys.graft({nodes:data.nodes, edges:data.edges})
})
})

Hope this helps!

Dámaris said...

Thanks, but It's not working, I have this:

$(document).ready(function(){
var sys = arbor.ParticleSystem(1000, 600, 0.5) sys.parameters({gravity:true}) sys.renderer = Renderer("#viewport")
var data = $.getJSON("prueba.json",function(data){sys.graft({nodes=data.nodes,edges=data.edges})})
})
})(this.jQuery)

And in the same folder, my json file is like this:

{"id":"prueba","nodes":{"Pau":{"lenght":20},"NBA":{"lenght":20},"Lakers":{"lenght":20},"España":{"lenght":20}},
"edges":{"Pau":{"NBA":{"length":2.5,"weight":2},"Lakers":{"length":2.5,"weight":2},"España":{"length":2.5,"weight":2}}}

It's very basic, and only to get my head round this library. Do you see any obvious mistake? I'm puzzled
=)

Thanks

Dámaris.

Dámaris said...

Thanks a lot, I still have the same problem though, and don't know what I'm doing wrong, this is what I have at the end of my .js:

$(document).ready(function(){
var sys = arbor.ParticleSystem(1000, 600, 0.5)
sys.parameters({gravity:true}) sys.renderer = Renderer("#viewport")
var data = $.getJSON("prueba.json",function(data){sys.graft({nodes=data.nodes,edges=data.edges})})
})

})(this.jQuery)

My file "prueba.json" is in the same folder as "main.js" and it look like this:
{"id":"prueba",
"nodes":{"Pau":{"lenght":20},"NBA":{"lenght":20},"Lakers":{"lenght":20},"España":{"lenght":20}},
"edges":{"Pau":{"NBA":{"length":2.5,"weight":2},"Lakers":{"length":2.5,"weight":2},"España":{"length":2.5,"weight":2}

I'm trying to get my head round it, but nothing comes to mind.Do you by any chance see an obvious mistake? I'm puzzled =)

Thanks,

Dámaris.

Dámaris said...

By the way, I'm not using Neo4J to get my json, I'm just using this library...hope that's ok

Aldrin & Luanne Misquitta said...

Your JSON structure looks odd. You seem to have an "id" in it. I guess it should look like:
{"nodes":{"Pau":{"lenght":20},"NBA":{"lenght":20},"Lakers":{"lenght":20},"España":{"lenght":20}},
"edges":{"Pau":{"NBA":{"length":2.5,"weight":2},"Lakers":{"length":2.5,"weight":2},"España":{"length":2.5,"weight":2}}
}}

Dámaris said...

I deleted id, and I have the same problem...I just realised I had wrong spelled "length", but again that's not the problem.

I even used another json file called "Africa" from the examples that the library arborjs has, but nothing changed.

Thanks for the help though

Dámaris.

Aldrin & Luanne Misquitta said...

That's really strange, cos I tried out your JSON in my js and it worked. Do you get any particular JavaScript error?

Dámaris said...

I don't know how to check for errors there, I put a log.console(data) after the instruction getJSON but nothing happens.
The way I have it in my folders is:
In my webapps I have 2 folders, lib and src, which I think are important, from arborjs. Inside webapps I also have a folder called "my-ejemplo", and inside it, the .js,.json,.html and .css.

Do you think I should have any other libraries, or if the path for them is not what should be?

Thanks a lot,

Dámaris.

Aldrin & Luanne Misquitta said...

If you're running it in Firefox, can you check if there are any errors on the error console (Tools->Error Console)?

damariri said...

It says:

Error: missing : after property id
Source File:
http://localhost:8080/mi-ejemplo/main.js
Line: 141, Column: 70
Source Code:
var data = $.getJSON("prueba.json",function(data){sys.graft({nodes=data.nodes,edges=data.edges})})

And there is an arrow that goes to the "=" character between nodes and data.

I don't know why it refers to id, my json file is like this:

{"nodes":{"Pau":{"length":20},"NBA":{"length":20},"Lakers":{"length":20},"España":{"length":20}},"edges":{"Pau":{"NBA":{"length":2.5,"weight":2},"Lakers":{"length":2.5,"weight":2},"España":{"length":2.5,"weight":2}}}

Thanks for your help, any ideas?

Dámaris.

Dámaris said...
This comment has been removed by the author.
Dámaris said...

I just realised I was using "=" nodes=data.nodes instead of ":". Sorry for the blunder, nevertheless I still cannot load the json file.

Getting there though,

Many thanks

Dámaris.

damariri said...

I got it =) Something I must have done wrong in my json, not sure what, I just made a smaller example and it works =)

Thanks for your help, I couldn't have got here without that

Dámaris.

HUK said...

umm... stupid question... i can't get any of the demos for arbor working locally(all i get is a blank screen). i am directly opening the html file in my browser(chrome). any idea what i'm doing wrong?

Anonymous said...

Hello!
I already tried the code myself (even before I found your article) but I couldn't get it up and running.

Now my question is: Do I have to declare gfx or something else? Because there's the variable gfx in the code but it hasn't been declared anywhere and I think that's the problem with my code.

With kind regards,
Margaretha

Michael said...

Margareta,

you have to declare

< script src="arbor-graphics.js" >< / script >

as part of the script libraries to load. Then, use

var gfx = arbor.Graphics(canvas)

like this:

var Renderer = function(canvas){
var canvas = $(canvas).get(0)
var ctx = canvas.getContext("2d")
var gfx = arbor.Graphics(canvas)
var sys = null

Then the gfx functions operate.

Aldrin, could you post your complete code here? I do not get an output like your at all, but I am fairly new to JQuery and arborjs. It would be great if I could compare where I went wrong... thanks!

Michael

JavaLang said...

Hello,

i want the graph to be settled down instead of continously moving,in th arbor sample they have mentioned the following for the same

// use center-gravity to make the graph settle nicely (ymmv)
sys.parameters({gravity:true})

I have tried different options for the same but was not able to achieve the same.

Thankz for the info.

Shireesh.

lluis said...

Thanks for the article, it works in FF but not in Chrome.

Something happens with workers.js (Chrome do not allow web workers) and with the JSON data (FF complains about not well-formed but loads it nevertheless, while Chrome do not load for security reasons).

Santhosh said...

Hello,

thanks a lot for posting an understanable version of ajborjs...

buti still cant get it to work...

there are no errors in firefox or in chrome...

i have also added graphics.js in my HTML...and the json file contains ur exact json code...PLS HELP !

Santhosh said...

here is the ready fn in my js :


$(document).ready(function(){
var sys = arbor.ParticleSystem(1000, 600, 0.5) // create the system with sensible repulsion/stiffness/friction
sys.parameters({gravity:true}) // use center-gravity to make the graph settle nicely (ymmv)
sys.renderer = Renderer("#viewport") // our newly created renderer will have its .init() method called shortly by sys...


var data = $.getJSON("orgChart.json",function(data){
sys.graft({nodes:data.nodes, edges:data.edges})
})


})

})(this.jQuery)

Anonymous said...

This is a great article! Is it at all possible for you to share the demo code?

Ananda Khamkula said...

Your first problem is your JSON file, the last comma (Conner -> Emily) must be removed

from :

"Connor":{"Riley":{"length":2.5,"weight":2},
"Emily":{"length":2.5,"weight":2}},
}

To:
"Connor":{"Riley":{"length":2.5,"weight":2},
"Emily":{"length":2.5,"weight":2}}
}

The next problem are

particleSystem.eachNode & particleSystem.eachEdge

not working, let's remove