netjsongraph.js – Google Summer of Code (GSoC) 2017 summary
Link on medium (opens in a new tab)
Throughout the last three months, I was quite fortunate to work for Freifunk (opens in a new tab) on netjsongraph.js (opens in a new tab) under the guidance of my mentor Federico Capoano (opens in a new tab). Thanks for this invaluable experience that I learned a lot of knowledge and use them in a practical project. Here is a summary of the work I have done during the Google Summer of Code (GSoC) 2017.
Goal
netjsongraph.js (opens in a new tab) is a visualization library for NetJSON (opens in a new tab), a network topology data format. The main goal of netjsongraph.js (opens in a new tab) may be concluded in below three lines (more details you can see in GSoC 2017-netjsongraph.js: visualization of NetJSON data (opens in a new tab)):
- Apply the modern front-end development tools and add tests workflow (#1 (opens in a new tab), #45 (opens in a new tab))
- Rewrite it with WebGL (#11 (opens in a new tab), #29 (opens in a new tab), #39 (opens in a new tab), #42 (opens in a new tab), #47 (opens in a new tab))
- Improve the performance (#41 (opens in a new tab), #44 (opens in a new tab), #46 (opens in a new tab))
Achievement
Github Repository : netjsongraph.js (opens in a new tab)
Examples on GitHub pages: netjson (opens in a new tab)
You can browse all examples on GitHub pages. Some screen shots of the application:
The force-directed layout is usually used to visualize network data. It offers insights on the relationships between nodes and links. The previous version of netjsongraph.js (opens in a new tab) is implemented by d3 (opens in a new tab) and it's rendered using SVG. It would be very slow if there were thousands or ten of thousands nodes or links. So I have to embrace the WebGL speeded up by GPU to have a better performance.
I have recorded my work in the blog every milestone:
- GSoC 2017-netjsongraph.js milestone 1 (opens in a new tab)
- GSoC 2017-netjsongraph.js milestone 2 (opens in a new tab)
BTW, It's a great management method to make members submit weekly reports and blog posts in Freifunk (opens in a new tab).
During the three months, there have been 116 commits from me. I created a big Pull Request include them:
Almost all goals have achieved:
- Published a minor version
- Improved development workflow
- Tests Added
- Refactored visualization by Three.js (opens in a new tab) and d3-force (opens in a new tab)
- Added more interaction like hover (show nodes tooltips), click (show nodes or links information panel), pan and zoom
- Improved performance
Especially on performance aspect, it runs efficiently on Chrome reached 60FPS under 5k nodes and 10k links. And if you don't wanna animation, you can choose the static rendering layout.
Challenges
I also encounter some challenges I never met before.
Event binding and handling
As you know, WebGL renders all objects in one canvas tag. How to bind events on every geometry? You should use the Ray casting (opens in a new tab). Raycasting is used for mouse picking (working out what objects in the 3d space the mouse is over) amongst other things. So you can know which geometry your mouse over and add some interaction effect.
There are thousands of objects and every object has several events you should handle, I had to develop an event controller to manage it.
Performance
The bottleneck in this visualizer is performance(#41 (opens in a new tab)). I tried many methods to improve it include:
Reuse geometry and material
However, the color of every node is different and the one link should highlight itself when it hovered, so the material should be independent and can not use in common.
Combine the mesh
Same problem with above. It's not flexible to combine them to one mesh, different nodes and links should have different positions.
Static rendering
Make calculation before rendering, so there is no animation and repaint.
Using Web Worker
Web Workers is a simple means for web content to run scripts in background threads. The worker thread can perform tasks without interfering with the user interface. So put static layout calculation into it will be efficient.
Force-directed algorithm
There are different complexity and cost in the different force-directed algorithm. The Force-Atlas2 algorithm (opens in a new tab) has some benefits over the force layout implemented in d3-force (opens in a new tab). So current version may be refactored by an advanced algorithm in the future.
What is left to be done
- Add optional geographic map (#40 (opens in a new tab))
- Using Force-Atlas2 algorithm
More interactions and features should be added, and performance may be optimized by using new algorithm. I'd like to continue developing this project after GSoC.
In the end, thanks for the great patience and guidance from my mentors. Thanks for Google to provide me with this rare chance to contribute to an open source community together with awesome members from all over the world. I really appreciate this invaluable experience accumulated this summer and I believe it will have the profound impact on my career and life.
