Category Archives: infovis

July 2009 Visualisation … 7 Key Challenges we face

The Digital Humanities Observatory is running a week long series of workshops on TEI, XSLT, Data Modelling and Data Visulisation from Monday the 13th to Friday the 17th of July. As part of this I was invited to present a lecture to all the workshop participants. I entitled my talk “Visualisation as an analytical tool, from networks to data streams. 7 Key Challenges we face.” Michael Maguire gave a very flattering description of my talk on his blog (thanks).


As I said during my talk, I normally give such a talk as an introductory session to information visualisation or visual analytics. However, this time I structured my talk around what I see as the 7 key challenges we (or anyone interested in visualising data) face. This blog post is a summary of the 110+ slides I presented (sans examples and mathematics!).

The ideas I presented are my view on the world of information visulisation and visual analytics. The key challenges were not presented in order of importance (as their relative importance is problem or domain dependent). There are also a number of challenges I personally feel (including multi-device and small screen visulisation) are crucial but I realise are not as pressing as the mainstream issues people face.

My ideas are informed by my ongoing research in InfoVis and from keynotes, lectures, online talks, toolkits and blogs that I’ve read or seen. Useful (and insightful) sources include, the visualizeit blog, the infosethetics blog by Andrew in the University of Sydney, the keynote Peter Eades gave at InfoVis 2006, the keynote Christian Chabot gave at the IEEE VAST 2008 and the ideas I could glean online from the VisWeek 2008 Panel on Grand Challenges for Information Visualization. If I’ve missed anything you feel is important do let me know!

So the 7 key challenges I see include:

  1. Empower: We must ensure the person using visualisation to understand data is empowered to gain insight or save time etc. To achieve this focus (long and hard) on identifying the questions that you need to answer with your visulisation. Do not just think about the data. If you think you have tool, method or technique to help empower a person (yourself or another) to gain insight or save time, can you validate this? What validation methods can you employ to ensure you are not just toying with pretty pictures?
  2. Connect: Ensure, based on the question at hand, you help the person using the visualisation build a connection between the data and any processing/analysis and the visual form presented. The question at hand and hence data drives what is an appropriate visualisation. Also, if you are using a particular visual form (eg. maps) how far can you stretch the metaphor or connection between data and display, before it breaks?
  3. Volume: Ensure if the data needed to help answer the question at hand has many elements that your visualisation method, tool or technique can support this. Voluminous datasets can break many desktop tools simply due to the time/memory/bandwidth needed to “load” the dataset. There are many sources of data with numerous individual elements to consider, 304,059,724 people in the USA (sources US Census Bureau) data on age, gender, ethnicity, household make up, home structure, income, farms, business and sales available. In July 2008 Google found 1 trillion (1,000,000,000,000) unique URLs on the web at once. This is ever increasing with user generated and automatically created content. One of our recent studies on extracting social networks from non-social network data started with 9,468,460 one-way flight passenger records. Clearly there are large datasets one might be faced with. Another problem (often overstated) is the dimensionality of the data (each element having multiple attributes to consider).
  4. Heterogeneity: Ensure if the data needed to help answer the question at hand consists of heterogenous data from multiple different sources or of “variant types” that your visualisation method, tool or technique can support this. If you need to consider a heterogenous data space then ensure the data-sets interlock so coupled or co-ordinated views are meaningful (and possible to display).
  5. Audience:Suit the word (display) to the audience. Ensure you match the visualisations to your questions and your audience. Know your user and don’t explore visualisation questions in a bubble. Engage and explore! Some methods, tools and techniques do not suit particular audiences. “You haven’t made impact with visual analytics until you help people with their own data” and I would add to this “in the particular sociotechnical context where they will use your tools, 
methods or techniques”.
  6. Dynamism: Data isn’t static. Ensure if the data needed to help answer the question at hand is a live source or the display is expected highlight changes over time that your visualisation method, tool or technique can support this.
  7. Discovery: Discover the new world once!: Ensure that your tools can store and capture and automate the process of pattern identification for subsequent data exploration. Convert identified patterns into “alerts” or stepwise mining, analysis, query and refinement into workflow.

As this was a masterclass I went on to point out the 10hr – 100,000hr guidelines to move from Trainee to Mastering visualisation. Gladwell spoke of the 10,000hr rule in his book Outliers which is important to consider when being introduced to a new topic like this. I pointed this out so people could help benchmark their own knowledge and skill level. My talk contained an introduction to cartography and GIS, multi-dimensional visualisation, parallel coordinates, paired parallel coordinates, graph drawing, force directed layouts and treemaps. As I’m currently learning iPhone application development myself, I know how dangerous and presumptive one can become moving from “Trainee” to “Apprentice” levels of knowledge. As they say, a little knowledge is a dangerous thing! I very much enjoyed giving this lecture to this DHO masterclass. I also gained some great insights with the director of the DHO Susan Schreibman on her experiences being a director there. Thanks to Shawn Day and Paolo Battino for inviting me to come along.

June 2009 Master Class on Visualisation

I’m giving a Master Classes called “Visualisation as an analytical tool, from networks to data streams” at the DHO Summer School 2009 in July. Thanks to Shawn and Paolo for inviting me. I’m looking forward to outlining the 7 key research challenges our field faces in light of the ever increasing torrent of both local and remote data sources.

Lecture: Visualisation as an analytical tool, from networks to data streams.

7 Key Challenges we face. Aaron Quigley (University College, Dublin)

Societies continued reliance on information and communications technologies has resulted in organizations generating, gathering, and storing “raw data” at a rate growing each year. The ability for even a mid-sized organization to store tens to hundreds of terabytes of data is already within reach. Massive storage technologies are rapidly outstripping our ability to effectively analyse, explore, and understand such voluminous data. While research in other fields such as data mining, machine learning and knowledge management are also attempting to aid in the analysis of such voluminous data, there is a realisation that the “human-in-the-loop” affords a visual analysis not possible through automation alone.

As such, the area of visual analytics extends the fields of scientific and information visualization by incorporating techniques from knowledge management, statistical analysis, cognitive science and decision science.

This talk will outline how voluminous data is modeled, managed, mined and hence visually presented for exploration. Several large scale data and information visualisation methods will be described and discussed along with the 7 key challenges we face as researchers and developers in using visualisation in an attempt to present information.

Details:

DHO Summer School 2009

Date: 13 – 17 July 2009

Venue: Royal Irish Academy and Trinity College Dublin

The Digital Humanities Observatory in conjunction with NINES and 18th Connect are delighted to offer a week-long workshop to allow scholars undertaking digital projects to develop their skills, share interests, and work towards common goals. Workshop strands, master classes and lectures will focus on the theoretical, technical, administrative, and institutional issues relevant to the needs of digital humanities projects.

The summer school will offer participants four week-long workshop strands to choose from:

* Introduction to the Text Encoding Initiative: Theory and Practice;

* Data Modelling and Databases for Humanities Research;

* Data Visualisation for the Humanities;

* Text Transformations with XSLT.

In addition the Summer School will feature lectures and master classes by leading experts and theorists in digital humanities.

June 2009 Paper Accepted to SNMABA 2009

Congratulations to Mike Farrugia on having our new paper entitled “Enhancing airline customer relationship management data by inferring ties between passengers” accepted as a regular paper at the International Workshop on Social Networks Mining and Analysis for Business Applications (SNMABA2009). This workshop will be held in conjunction with the 2009 IEEE International Conference on Social Computing in Vancouver, Canada. Accepted papers will be published in the workshop section of the main conference proceedings of SocialComp-09.

“Abstract—In the airline industry, as in many other industries, customer relationship management data is predominantly based on quantitative data. In this paper we explore the possibility of augmenting this quantitative data with relational data by inferring ties between passengers. Different methods of inferring relationships are proposed and discussed, along with the business
benefits such relational data adds to current customer information. We also explain some visualisation approaches to facilitate the exploration of this data by business analysts in marketing, sales and customer loyalty sections.”

Feb 2009 Clique Strategic Research Cluster funded: €3.56 million

I am one of the six principal investigators for Clique which was announced by the Irish Tánaiste as an Science Foundation Ireland funded (SFI) Strategic Research Cluster. Our industry partners, in this joint initiative are IBM, Idiro Technologies and Norkom Technologies. Our academic partners are UCD and DERI in Galway. Clique runs from 2009 – 2013 and has a focus on the analysis and visualisation of large graphs and networks, specifically social and biological networks. This is a very exciting development for my research as it will allow me to hire a number of postdocs and postgraduates but more importantly to work in a research eco-system with domain experts, industry partners, rich data sources and collaborators interested in various aspects of the end-to-end problems in visual analytics.


Many of the Clique team will be based in the UCD CASL

While these companies are providing matching funding and resources the SFI is providing €3.56 million over the course of 5 years. In total with industry and SFI funding the program has funding in excess of €5 million. The academic principal investigators involved are, Prof. Pádraig Cunningham, Prof. Denis Shields, Prof. Brendan Murphy, Dr. Aaron Quigley, Dr. Neil Hurley and Dr. Conor Hayes. The funding will be used to hire postdoctoral researchers and pay postgraduate scholarships.

The development of this research cluster has been a long time in the making. I’ve been an IBM visiting scientist since 2005 and others have been collaborating with Idiro and Norkom over a number of years. This cluster was first proposed over 1 year ago when 40 similar clusters were proposed to the SFI. The process to select these five from the forty has included a preliminary expression of interest (then review), full detailed proposal (then reviews), site visit (international panel), SFI review and board review and final the official announcement by the government! It’s a very rigourous process aimed to ensure the best research is funded, the value for the Irish tax payer is strong and the potential for industry is high.

I look forward to blogging about new team members, new research ideas, papers, outputs and commercilisation in the years to come!

[ Read SFI News Release ]

“SFI Strategic Research Clusters (SRCs) will help link scientists and engineers in partnerships across academia and industry to address crucial research questions, foster the development of new and existing Irish-based technology companies, and grow partnerships with industry that could make an important contribution to Ireland and its economy.

The SRC programme has been designed to facilitate the clustering of outstanding researchers to carry out joint research activities in areas of strategic importance to Ireland (in ICT and/or BioTech sectors), while also giving the time and resources to attract and cultivate strong industry partnerships that can inform and enhance their research programmes.”

Feb 2009 CHI Workshop and Surface User Interfaces

I just blogged about Surface User Interfaces on the “Evaluating new interactions in healthcare” blog. We have a paper [1] on “Design Patterns” at this workshop during CHI which this blog is being used to support. While I don’t think I can attend the workshop myself as I have trips to the USA, Sweden, Germany and Australia in the next two months my colleague Julie Doyle will attend. My PhD student Ross Shannon will also attend to present our paper on “Time Sequences” during the work in progress at CHI in Boston.

Figure 1: SharePic photo sharing system [3]

[1] Doyle J., Quigley A. and Nixon P., “Do Pattern Languages help us Structure Evaluations in Healthcare Technologies?” proceedings of the CHI 2009 Workshop on Evaluating New Interactions in Healthcare: Challenges and Approaches, Boston USA, April 2009.

[2] Shannon R., Quigley A. and Nixon P. (2009). Time Sequences. In CHI ’09 Extended Abstracts on Human Factors in Computing Systems (Boston, Massachusetts, April 04 – 09, 2009). CHI ’09. ACM, New York, NY. (in press)

[3] Apted, T., Kay, J., and Quigley, A. 2006. Tabletop sharing of digital photographs for the elderly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 – 27, 2006). R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, and G. Olson, Eds. CHI ’06. ACM, New York, NY, 781-790.

Feb 2009 Open posts for Clique Research Cluster

Clique LogoI am one of the principal investigators for the Clique Research Cluster in Graph, Network Analysis and Visualisation based at University College Dublin and the National University of Ireland, Galway.

We are inviting applications for the following posts and funded PhD positions:

  • Senior postdoctoral researcher (UCD ref 003756)
  • Postdoctoral researcher in probablistic network modelling (UCD ref
    003755)
  • Postdoctoral researcher in analysis of information diffusion in social networks (NUIG-Clique-02)
  • 6-8 funded PhD studentships to address research challenges in
    • anomaly detection
    • biological network analysis
    • computational techniques in network analysis
    • information visualisation
    • models of information flow
    • probabilistic network models.

Please note, the closing date for this round of applications is 8th
March 2009. Please see the Clique Cluster vacancies page for more details.

Feb 2009 CHI 2009 Work In Progress paper

Congrats to Ross on having his work in progress paper accepted to CHI 2009 to be published as an extended abstract.

Shannon R., Quigley A. and Nixon P. (2009). Time Sequences. In CHI ’09 Extended Abstracts on Human Factors in Computing Systems (Boston, Massachusetts, April 04 – 09, 2009). CHI ’09. ACM, New York, NY. (in press)

Jan 2009 VDA 2009 San Jose

VDA 2009

I presented our paper on the Visual data exploration of temporal graph data at the Conference on Visualization and Data Analysis 2009 which was Part of IS&T/SPIE’s International Symposium on Electronic Imaging 2009. This event was held on the 19-20 January 2009 in San Jose, California, USA. The paper was co-authored by Mike Farrugia and myself and is based on Mike’s masters research and his submission to the IEEE VAST 2008 contest (which he won a Cell phone Mini Challenge award).

The event itself was quite interesting due to the co-location with a number of other conferences on Graphics, Imaging and Visualisation. I also got to meet a number of interesting researchers including Katy Börner who presented a inspiring paper on “Teaching children the structure of science”. Katy is the director for the Cyberinfrastructure for Network Science Center at the School of Library and Information Science, Indiana University. I enjoyed our discussions and hearing more about the Network Workbench which will be of great interest for an upcoming project we are starting. I also enjoyed discussing research infrastructures with the highly energetic Russell Duhon. He discussed EpiC: A Computational Infrastructure for Epidemics Research with me and gave a talk on Creating Marketplaces for Science.

This is worth watching too.

Jan 2009 New Translational Research Project Dviz & Senior Software Architect post

12 HorsesIADT logoUCD crestNDRC logo

Representatives of UCD, IADT and Twelve Horses in late December 2008 signed a two year contract with the NDRC for the Dviz digital media technologies project. I am the UCD PI, Hilary Kenna in the IADT PI and Gabrielle Stafford leads the project with Twelve Horses Ltd.

The Dviz project is a collaborative digital technology research project between Twelve Horses, IADT and UCD, and is funded by the NDRC. This project brings together a diverse skill set from both its commercial partner and university research teams to realise an innovative visualisation platform with strong commercial potential.

Dviz itself is an online platform to facilitate the contextualisation of statistical data in realtime using dynamic visualisation technology. The ultimate objective of this project is to unlock the value of data by creating a platform that will enable users to visualise, manipulate and track data in a way that is meaningful for them. The resultant dynamic visualisations can then be used to investigate causal relationships to influence decision and policy making with evidence based conclusions.

Jan 6, 2009: The Dviz team is hiring a Senior Software Architect Link: http://tinyurl.com/7lr7lt

Draft Advert: Senior Software Architect – Dviz – (check Twelve Horses link for full and up to date advert).

The Senior Architect will be tasked with conceiving, designing, presenting and realising software and architectural approaches for the DViz platform and applications.

The ideal candidate must be fluent in modern enterprise and open source technologies, and must be able to apply these to a distributed system environment. A solid understanding of Web application technologies, databases, and performance parameters is a must. The ability to work independently as well as having excellent written and verbal communication skills is required.

The Senior Software Architect will be expected to technically review and manage the software development lifecycle, ensuring best practise process and techniques are deployed throughout design, implementation and testing, and appropriate metrics utilised to control and monitor the integrity and maintainability of the technical solution. Both conventional and Agile development lifecycle models will be used where appropriate, and full configuration management of all versions and variants will be maintained.

The Senior Architect will be expected to lead the following tasks:

* analysis of the requirements
* ensuring that the architectural design of the solution is consistent with the component architecture, and that all requirements are met
* providing guidance and refinements on the modelling and design process
* ensuring that the architecture fully covers all requirements, and has full traceability to requirements
* identification of any study, risk reduction, or prototyping stages that need to be planned into the development

Qualifications and/or Skills

* Minimum of 7 years Experience as Software Engineer
* Bachelor’s Degree Computer Science or related field (Advanced degree preferred)
* Experience as technical lead in designing and architecting scalable enterprise solutions
* Strong programming skills in Java and J2EE required
* Web Services, GUI, SOA, JSP, and XML desired.
* Extensive knowledge of Object Oriented (OO) Analysis and Design

Dec 2008 Two Conference Papers Accepted.

Myself and three of my PhD students have recently had two papers accepted at leading international conferences. Both will be published in upcoming volumes on the Springer Lecture Notes in Computer Science Series.

Firstly, myself Ross, Tom, along with our colleagues Adrian, Simon and Paddy had “Situvis: a visual tool for modeling a user’s behaviour patterns in a pervasive environment” accepted at the Seventh International Conference on Pervasive Computing in Nara Japan. This year the conference had a very low acceptance rate of 18.4% which makes this all the move satisfying personally. The back story to Sitvis is a very interesting one and is a great testament to our new structured PhD program in UCD. Tom developed the core Situvis visualisation framework as part of a project he developed in my InfoVis course in 2007. He worked with Ross on developing it into a graph drawing system by using coupled layouts. We then further developed the idea when Adrian came with the situation and sensor problems and proposed Situvis which we all worked on together. The ebb and flow of ideas in and out of the students areas of core interest goes to show what great outcomes we can have with structured learning.

Secondly, myself and Umer Rashid had a paper accepted at the HCI International Conference 2009 on “Interaction Techniques for Binding Smart Phones: A Desirability Evaluation“. It will be published by Springer in a multi-volume set in the Lecture Notes in Computer Science (LNCS) and Lecture Notes in Artificial Intelligence (LNAI) series. This conference will be held on 19-24 July 09 in San Diego, CA, USA.

Japan and the USA in 2009.. here we come for these and other papers to come…!