Monday, June 10, 2013

An HTML5 fancy user interface project

Visualizing disk space with a sunburst graph
Sunburt graph
I recently finished a new project that uses HTML5 to display and graph file system data and I thought I would write a blog post about it.
HTML has come a long way from the early days when I was working on Lynx. The latest and greatest features are pushing graphics and user interfaces that are approaching native code levels of complexity and performance. I'm particularly fascinated by these features because they represent the realization of a vision that we had in the early days of Netscape where HTML could take over as a complete application programing interface.

WWW applications?
Using the Web just like a computer program seems pretty obvious these days, we use it that way on many if not most web sites, but in the beginning it was really just a document publishing platform. Hypertext, the basis for the web, was designed as a way to make documents more powerful by linking them together with references that could be accessed incredibly easily.  The idea of putting powerful application logic behind a hypertext interface was pretty far fetched in the early '90s.  I happened to have a need to do just that due to a need at my university. I came up with some ideas and put forth an idea to the other developers I knew that were working on the web. This idea soon sparked the addition of form elements to the web. Later while at Netscape, we took the next step by adding a scripting language, Javascript, to add even more client side capabilities and responsiveness.

Full GUI support
One of the visions we had at Netscape was to create a platform that had a fully complete and functional set of graphical elements so that other developers could create any application imaginable. The foundational elements are reasonably well known and are included in all modern operating systems and major graphics libraries.  The difficulty in delivering GUI primitives to the Web was figuring out a way to efficiently add them to HTML and to make them massively cross platform.    When I say "massively cross platform" I don't mean just Windows and Mac. We were thinking about operating systems, screen sizes, resolution, and a whole bunch of other things.  The web was designed from the ground up to try to be device neutral and that makes the implementation very hard.  CSS is one technology that supports multiple displays by providing different layout and styling for different displays as required.  Netscape as a company ran out of life before it was able to complete its roadmap for a full complement of GUI primitives, but it did make a good effort.  A technology called XUL (pronounced Zool) was introduced that allowed a large variety of GUI to be expressed, unfortunately, XUL didn't move forward fast enough and was essentially dead on arrival.     Simultaneously Microsoft smothered the market with Internet Explorer and tried its best to keep the Web from moving forward with HTML.
Visualizing disk space with a tree graph
Tree graph

Enter HTML5
Fast forward several years from the Netscape era and the world has changed substantially.    Firefox, Chrome and a variety of Webkit based browsers have wrestled away the strangle hold that IE6 held on the market and began to make substantial improvements to the abilities of HTML.  HTML5 was soon born and with it a whole bunch of new graphical technologies that enable many new types of GUI applications.    SVG, advanced CSS, Canvas and Javascript now combine to create an ability to create very rich interfaces that are entirely driven within the Web browser.

My experiment  
A while back I came across a group that was creating a set of tools that harnessed the new power of HTML5.  The project is called D3 for Data Driven Documents.  I saw what D3 could do graphically and wanted to try it out and do something personally useful with the technology.   Since my current company is in the cloud storage space I decided to write a tool to visualize the size of the data on a computer hard drive.  One of the difficult issues with a tool to analyze disk space is that it needs access to all of the data on a drive and that data is sensitive and private.  I needed to figure out a way to scan the data and not violate a user's privacy. Standard HTML applications run in the cloud and couldn't do something like this without sending data to a server, but the new world of HTML5 can make everything happen locally in the browser.

Some complications
At this point in my project I had an idea of what I wanted to do, and a fancy new graphical library for graphing, but I still needed a way to collect the disk space data to be graphed.  Javascript has chosen to stay away from local file access to simplify the security model, but Java long ago added the ability to ask the user for local disk access.  I ended up writing a Java beans integrated module that spawns a Java application to scan the local disk and return the data to Javascript running in the browser for analysis.  The beauty of the Java/Javascript integration is that none of the data ever leaves the local computer that it is running on. The downside is that Java is often broken on user machines, or is disabled due to security concerns.  I hope at some point Javascript figures out a workable security model to allow some forms of local disk access.
Visualizing disk space with a tree map graph
Tree map graph

The finished project
Please check out the finished project if you are interested. I have made a video demo and a walk through to explain the use and features.   There is also sample data to play around with that skips the scanning process.  The code has been published as open source with a BSD license.    This project just might come in handy for you the next time you run out of disk space and need to know what is using it all up.