The Need for Smart Data Visualization

Just because you can do data visualization, should you?

Big Data is a big topic! It is one of the most popular buzzwords in the tech world today. From finance and banking, genomics and healthcare, to marketing and communications, nearly all industries want to utilize data to drive business decisions. Advances in communications, social networking, and information technology have fueled a tsunami of Big Data paving the way for development of interactive data visualization tools.

Traditionally, data visualization tools were static, non-interactive graphs and tables that were a staple in board rooms. They provided a visual representation of the data, but required more time to analyze and understand the data. Further, the traditional data visualization tools could be error prone and often required in-depth knowledge of the application in use. The increased interest and advancements we are experiencing with interactive data visualization can be attributed to:
• Advances in computational power, data analysis, and graphics which have enabled widespread access to data visualization products
• Generation and availability of large amounts of data which cannot be easily analyzed by traditional methods
• A need for rapid analysis and decisions on the large amount of data that is generated within an organization

Interactive tools…the answer or a step in the right direction?Interactive tools afford a better understanding of relationships and trends in data sets and allow a quick drill down of data to the smallest unit. These tools were initially developed as ad hoc solutions by organizations to address a specific question within a specific set of data and have gained tremendous popularity. Consequently, companies (regardless of the size) are racing to develop better, faster data visualization tools and in turn, fueling an almost irrational expectation that data visualization is the magic-bullet for tackling Big Data. While these expectations may be warranted based on some of the success stories, it is imperative that data analysts and programmers ensure they are asking the right questions and utilizing the right methods in order to generate valuable analyses of the data. Immediate, narrowly focused answers will never provide the desired big picture solution to Big Data.

Quality interactive tools – key considerations for big picture solutions…We have all heard about the challenges related to the volume of semi-structured and unstructured data that is being driven by the popularity and ease of use of mobile devices and platforms like Twitter, Facebook, Tumblr, etc. Currently, a number of standalone products are available to analyze and consume the semi-structured and unstructured data. Going forward, these solutions should be incorporated and offered as part of a comprehensive data visualization solutions suite.

Technology advancements have positively impacted the user friendliness of data visualization tools as they no longer require the data managers/analysts to be computer scientists. While this is a positive enhancement, we cannot neglect to recognize that it is becoming increasingly important that the consumers of data visualization tools become savvy and comfortable with using the tools. These consumers need to become data scientists – in addition to analyzing the data; they have to look for patterns, hypotheses, outliers, and unusual trends to draw inferences.

Human visual perception capabilities are often overlooked by data visualization vendors. The human visual system is endowed with tremendous abilities to see patterns and make decisions (the animal kingdom uses a similar behavior in prey detection; refer to the Search Image Theory exhibit¹). These abilities are governed by certain rules with regard to size, shape, color, and proximity of the objects. There has been an extensive amount of research conducted and there is a large amount of data available in the field of human visual systems and cognition. Incorporating this research and these concepts into the design and development of data visualization tools will only strengthen the capabilities of the tools.

In an ideal world, data visualization tools should not only provide information on what is expected but also help to decipher what is not expected. The tools should be a means to identify outliers and unusual trends, account for various types of data (i.e. structured vs. unstructured), utilize the appropriate analysis methodology (statistical understanding), and incorporate human visual perception. Then and only then will the data visualization tools help with decision support and lead to better management by exception.

Our experiences with data visualization in an adverse event management system highlighted a number of the issues discussed above and prompted us to address them. In doing so, we were able to provide our client with tools to attain greater levels of efficiency through a cost-effective, low-risk solution.