Moderator:

Panelists:

Description (10/30/03)

For some time now the practitioner community has taken issue with the research activities of their academic counterparts. Likewise, the academic community in HCI has often felt a level of frustration associated with knowing whether or not their research findings generalize to the real world, are timely enough, or "cutting edge" enough, to benefit the practitioner community. Additionally, transferring research technology or techniques to the practitioner can be problematic, since the two communities might not attend the same conferences or read the same materials. In fact, the practitioner community may not have the time to attend conferences or publish how they perform their daily tasks, further exacerbating the flow of knowledge between the two areas of discipline.

What follows are a series of position statements from the panelists. We will have 3 academic panelists and 2 industrial participants. Of our industrial participants, one is an active user researcher in the midst of real product design, while the other is an industrial researcher who works closely with product teams and has worked in product development in the past. Our format will be as follows:

The practitioners will outline what they feel their community would like to see more of or need from their academic counterparts. The academic researchers will describe what they think is of interest to academic researchers in HCI in the short and long run. Practitioners will then respond to this research agenda in terms of whether or not they think these topics are of interest to practice, whether or not they are too esoteric, timely, and whether or not the practitioner community could provide resources to help with the technology transfer. After 45 minutes or so of debate on these topics, we will open the panel to audience participation.

MARY CZERWINSKI

I have worked as a usability engineer on product teams, as an adjunct professor at universities and as an industrial research scientist doing applied research. Because of these experiences, I have come to have a keen understanding of how difficult it is to transfer HCI research knowledge, especially from within an academic setting, to the practitioner community. As academics, we tend to think very long-term, and often more theoretically and systematically, than our practitioner counterparts have the luxury to afford. As practitioners, we tend to need to have a myriad of tools and techniques at our fingertips, ready for application quickly as our product development cycle dictates. Practitioners rarely have the time to perform research necessary for refining or iterating on a problem or an aspect of their craft, much less publish methods or techniques that they have developed to solve a practical problem. In my opinion, academic HCI researchers need to partner with their practitioner counterparts. The academic researcher needs access to real user scenarios, and real data or artifacts, and real design challenges for their research to have the proper scope of influence. By product teams partnering with academics or their students, everyone wins. The product team benefits from the perspective and technology that the researcher can bring to bear, in addition to the benefit of theoretical or systematic research findings. They also get access to technological resources that they might not have had otherwise. The academic quickly comes to realize which parts of their research program do or don't make sense in a real world context, given real world time pressures and resources.

IZAK BENBASAT

Academics and practitioners are sometimes characterized as the two solitudes though neither party desires such a state of affairs. As my fellow panelists describe in more detail below there is a strong and genuine desire to cooperate, but very often the realities, rewards structures, and constraints imposed by our separate environments make it difficult to do so. In the information systems academic literature there has been interest in recent years in exploring the means for cooperation and knowledge transfer, including a commentary that I wrote with Bob Zmud (Benbasat and Zmud, 1999). However, the papers written have put forth the views of academics only. This panel will give academics an opportunity to hear the opinions of the practitioners, and allow us to adjust our thinking and tactics to fit their needs and constraints, and will do the same for practitioners. It is my view that though cooperation is a desired goal the means to achieve it is not easy. Hence, I hope that we will be able to come up with a few but concrete means of achieving cooperation, and measure our success in doing so in follow-up panel discussions that will take place at ICIS in future years.

JULIE RATNER

After working in industry for nearly a decade, I perceive my years doing academic research and longitudinal government research through a different lens with a more strategic business perspective. Today, I interpret academic research results I read with keen interest and notice that I usually yearn for timely reporting and a focus on practical details and less theory.

Since I work with engineers and designers on wireless applications with 1-10 week definition to delivery timeframe; the key to successful collaboration between InfoSpace Mobile and academia is efficacy and flexibility. To use a common metaphor, HCI results are relevant to my product teams "when the rubber hits the road;" when they impact the bottom line, before applications launch. Success is measured by initial user experience; if a mobile user's first experience is intuitive, user adoption of wireless applications is likely to increase.

I have had a few successful collaborative research projects with academia since I have worked in the software industry. When I worked at on the east coast, we sponsored semester long research projects each year. The reason they were successful is that the graduate and undergraduate students stayed in budget, delivered what they promised, and listened to and answered the product teams' questions about users. The value of working with these students was multi-faceted; we were able to delegate 6-month field studies that the company did not have the resources to conduct and we frequently hired exemplary students as interns once the semester ended. The students in turn gained practical experience about the value of research in industry and a few even received offers of employment with the company.

Depending on how collaborative projects are structured in the academic settings, my reaction is initially mixed, not because I don't value and appreciate academic research (which I do) and not because I don't see the benefit of partnering with academia (which I totally support), but because I know for a fact that our time-frames are out of sync. In one academic semester for example, our business goals typically shift repeatedly and oftentimes the HCI research that would have been priceless in January is not relevant by May.

RADHIKA SANTHANAM

Though I have worked in the industry, it was not related to HCI work and I consider myself to be primarily an academic researcher. Therefore, my views may seem a little radical to the practioner panelists, and I do welcome them to convince me otherwise. While I think it is important for academic research to be relevant to practice, I also feel that we will and must continue to have a certain areas of research space that is distinctly different, and which will seem somewhat irrelevant to the other group. In fact, I feel that if we did similar kinds of research and chased the same specific problems, we will not have much to offer to the other group. I clarify this premise up-front so that we can better discuss what knowledge we academics need to transfer to practice and vice-versa. It will also help us identify those intersecting areas of interest.

First, the research goals of our two communities are fundamentally different. Our goal as academic researchers in HCI is to understand underlying, and (hopefully) enduring, principles of human behavior that come into play when interacting with computers. We focus on building a cumulative body of knowledge. As I see it, HCI practioner researchers are also interested in understanding these underlying principles, but want it in a form that they can readily apply to system design and product development. They usually do not have the time to investigate and develop underlying principles. Therefore, one way I see for academics to communicate this knowledge is to get together every couple of years in a workshop with the sole goal of "Knowledge Transfer between Academics and Practioners". In such a workshop, an expert in specific areas of HCI, (e.g. on the topic of visualization, decision making or training) will present all the key findings/ideas that have been generated in academic research in the last few years. The expert will also indicate how these ideas could be applied to practice. This will enable the transfer of knowledge discovered in academia by eschewing the theory, the complicated statistics, the obfuscating language and all other things that practioners do not want to sift through. Practioners could ask for clarifications, quiz and maybe even disparage these findings! But this interactive process will provide good feedback to academic researchers on what aspects of research was useful to practice, and practioners will hopefully obtain nuggets of knowledge in a speedier fashion.

As an academic researcher, what I want to know from practioners is about repetitive problems that they have faced, and about issues they have not been able to solve. I emphasize repetitive because a user problem in one specific application is something that academic researchers should not worry about. Using the wireless applications as an example, if there are persistent problems about displays, or issues relating to user learning/adoption that are perplexing, then it should be brought to our attention. We as academics can search for some underlying issues that could perhaps explain these persistent problems. Once again, I think the workshop setting is an avenue where this can occur. I think an important way by which practioners can help transfer and also help develop knowledge that is useful to practice, is to share data on these problems.

While I like the idea of collaborative projects, I think knowledge transfer has to occur at a higher level of abstraction than single projects. These projects do have value to the extent that each group can get to know members of the other group and understand their perspectives. But I am afraid that too many such projects will lead to a situation where academic researchers are also huffing and puffing about product development cycles and delivery schedules. Furthermore, if we academics also start to focus on immediate problems and specific products, I fear that in the long run, we will become even more irrelevant to practice.

PETER TODD

Should we build bridges between academic research and the practice of HCI? Most of us would agree this is a laudable goal. A motherhood and apple pie agenda. But as Professor Santhanam notes above, such a goal may have unintended consequences. As academics are neither trained nor motivated to examine issues in the short term or to provide rapid results. As a consequence by following the needs of practice we risk making academic research, which is narrow, focused, long term and cumulative in nature less relevant as we try to meet the needs of practice, to provide rapid results to immediate issues with bottom-line impact. And to do it with fewer resources, with less sense of the market and ultimately less well than do our colleagues addressing the same issues in practice.

Those absorbed in practical issues of systems design and implementation are likewise not well-attuned or motivated to the possibility of taking our narrow theoretical notions and applying them to their practical efforts. In this context the chasm between our research abstractions and the immediate needs of practice appear to be nearly insurmountable. What then can we do?

My colleague Izak Benbasat suggests the way, we academics, can get practical. Not practical in our substance, but practical in our approach. We need to look for the few things we can practically do that will help to build bridges. Our colleagues in practice can also become more open to the importance ideas that evolve over the long term. In addition we can all be a little more patient.

Lets start with patience. Recently I was preparing a graduate class on decision-making in our executive-format Master of IT Management Program. As is often the case for these classes I turn to sources such as the Harvard Business Review to find coverage that will be accessible and acceptable to them. In this particular instance one of the articles I chose was:

Delusions of Success: How Optimism Undermines Executives' Decisions by Dan Lovallo and Daniel Kahneman (HBR July 2003).

Kahneman, of course, was the recipient of the Nobel Prize in economics in in 2002 for his landmark work with the late Amos Tversky (who also received the award). Their initial work dates to the early 1970's and formed the basis for the HBR article. Thirty years from theory to practice. Lets learn to be patient.

While we are waiting there are few other things we can think about.

We should learn to talk. I have often found it is possible to have interesting and productive conversations with practitioners about theory and research results. We should learn to listen. Practice is a great source of interesting questions. Often not the question that is being posed but higher level questions that really are enduring. The issues and questions do not change as quickly as they are made out to. We should learn to cooperate. Unlike almost any other area of research the HCI field has a remarkable opportunity to collaborate to collect information that can lead to important theoretical insights and inform practice.

Our panel discussion should provide us with an important opportunity to examine these and other issues. One thing we can be sure of we will all be optimistic about the possibilities, pessimistic about the ability to act on those opportunities and impatient for results.

REFERENCES

Benbasat, I. And R. W. Zmud, "Empirical Research in Information Systems: The Practice of Relevance", Management Information Systems Quarterly, March 1999, pp. 3-16.