Introduction

This paper compares two studies that employ grounded theory to investigate information systems development tools. These tools facilitate information systems analysis, design, development and maintenance activities in ways that improve productivity. Tools that may improve productivity are those that use automation to affect the timing, cost and quality of products and services delivered by the information systems function (Henderson & Cooprider, 1990).

These studies respond to the expressed need for broader research methods in information systems (see Hirschheim & Klein, 1989; Kaplan & Duchon, 1988; Lee & Kim, 1992). Primarily, research on information systems development tools has focused on the tools themselves and not on the use of the tools in an organizational context (Wynekoop & Conger, 1990). This comparison of two studies is of interest to researchers using qualitative methods to investigate information systems development tool usage in the context of an organization or a development team. The comparison illustrates ways in which grounded theory can be used to analyze and understand interview data.

Studying the context in which information systems development tools are meant to function requires an appropriate research methodology. The two studies illustrate the use of such a methodology. In these studies grounded theory, an ethnographic approach, is used to capture information needed to explicate the interactions between teams and information systems development tools, and organizations and information systems development tools. According to Glaser and Strauss (1967) theories are either deduced from logical assumptions or generated from observation. Grounded theory is a qualitative approach that generates theory from observation. It provides the structure often lacking in other qualitative approaches without sacrificing flexibility or rigor. The resulting theory is an explanation of categories, their properties, and the relationships among them. The results lead to an evolutionary body of knowledge that is grounded in data.

The Studies

This comparison illustrates ways in which grounded theory can be used to analyze and understand interview data. The first study assesses the use of a software engineering tool, dialog charts, by systems designers (Calloway, 1988). The second study investigates organizations that use Integrated Computer-Aided Software Engineering (ICASE) tools (Knapp, 1995). We discuss the similarities and differences in the methods of information generation, data collection, data analysis and reduction, and data synthesis.

Data Generation

The two studies differ in the way information was generated. The first study is based on a field experiment with teams of designers who used the dialog charts while developing interactive systems. In a field experiment a controlled setting is used to simulate natural conditions. In the second study data was generated by ICASE tool users in various natural settings in organizations. In a field study data emerges naturally in an uncontrolled setting.

Data Collection

Semi-structured interviews were used in both studies. Teams of designers were interviewed in the first study, and the interviewer disguised the purpose of the study and the tool of interest from the respondents.

ICASE users from all levels of the organizations were interviewed in the second study. These interviews were used to determine dimensions of the organization as a whole in relation to ICASE implementation. The researcher's interest in ICASE tools was clear to the respondents. The interviewees were chosen to reflect a user profile constructed from an analysis of responses to a mailed questionnaire.

Data Analysis and Reduction

A critical data reduction decision in qualitative studies is to determine the unit of analysis. The first study used keyword analysis to extract sequences of words about the subject of interest along with their contexts. These sequences were called "mentions" and were subsequently coded into categories. In the ICASE study coding was based on a qualitative evaluation of each sentence of each interview.

In grounded theory studies, data analysis and the later stages of data reduction operate iteratively. Coding is a process of simultaneously reducing the data by dividing it into units of analysis and coding each unit. The first study used a multiple coding protocol, where mentions could represent more than one concept category. The nature of responses from teams of up to four people during the interview was more realistically captured by a multiple coding protocol. In the second study data were coded as belonging to one category only, since the interviews were one-on-one. Therefore, there was no interleaving of comments.

Synthesis

After the categories are integrated and synthesized into a core set of categories, a narrative is developed that explains the properties and dimensions of the categories, and the circumstances under which they are connected. This explanation of the phenomena under investigation is the theory developed based on the data. The studies did not differ markedly in the analysis and synthesis.

Results of the Study Comparison

A comparison of the studies resulted in four findings. These findings relate to the granularity of focus, the coding method, the independence of the method of data generation, and the cultural dimension of the interaction of users and tools. These findings appear to be significant since both of these studies were the basis of extensive research projects and each resulted in significant research findings which are published elsewhere (e.g., Calloway and Ariav, 1995; Knapp, 1995). There is no indication that the differences in methodology affected the emergence of theory. The emergence of theory appears to transcend the specifics of methodological difference, since theory can be detected regardless of methodological differences. The methodology therefore appears to be transparent.

Granularity of Focus

A comparison of these two studies suggests that grounded theory is useful in interpreting interview information regardless of the granularity of focus. The first study investigated design teams using a software engineering tool. The second study investigated ICASE technology implementation and dimensions of the organization as a whole.

Coding Methods

All grounded theory studies use a data coding scheme. Qualitative methods use codes to categorize data rather than to quantify it. Therefore, the number of times an individual comment is categorized is less relevant. Different methods of coding are effective in different contexts. The first study used a multiple coding protocol to capture the interactions among the various team members. Since the comments of different team members relating to different categories were interleaved, single coding these comments would have sacrificed accuracy. However, because the interviews for the second study were primarily one-on-one, single coding was sufficient to indicate the underlying categories.

Method of Data Generation

Grounded theory can be used regardless of the way the data is generated. These two studies generated data differently. The first study generated data using a field experiment. The second study was conducted in a natural setting.

Cultural Dimensions

Organizations are often spoken of as having cultures, even though the degree to which organizational culture is the same as societal culture is a matter of debate. However this debate is resolved, it is certainly true that organizations can be treated "as if" they were cultures for the purpose of ethnographic analysis (Morey, 1986). Grounded theory is an appropriate tool for studying organizational cultures (Glaser & Strauss, 1967; Martin & Turner, 1986; Orlikowski, 1993; Strauss & Corbin, 1990; Turner, 1983). Cultural dimensions of the interaction of users and tools are revealed by the two studies. For example, the first study reveals that designers develop strong attachments to their tools that they express in highly emotional language. These attitudes, attitude patterns and beliefs that accompany tool usage and systems design are learned. The second study found that ICASE implementation success relies on the interaction between management's understanding of information technology and the information systems development environment. Information systems development complexity also influences success. For example, the degree to which an organization adhered to a systems development methodology prior to the introduction of ICASE emerged as the single most critical factor influencing the implementation of ICASE.

Conclusions and Suggestions for Future Research

As more researchers use qualitative methods and grapple with the problems of interpreting interview data, the more important methods for systematic analysis of symbolic information become. This comparison shows the applicability of using grounded theory to analyze and interpret interview data. Grounded theory is useful regardless of the granularity of analytical focus, the coding method, or the method of data generation. The grounded theory approach also allows the cultural dimension of the interaction of users and tools to emerge.

This comparison shows that grounded theory approaches are rich and robust because differences in application can be accommodated. Although these results are promising, further investigation is needed to understand the extent of both the limits and the applicability of grounded theory.

References

Calloway, L. J. and Ariav, G. (1995) Designing with Dialog Charts: A qualitative content analysis of enduser designers' with a software engineering design tool. Journal of Information Systems, 5 (2), in press.

Calloway, L. J. (1988) An approach for assessing tools for designing dialog structures: a study of the dialog charts. Doctoral dissertation, New York University.

Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine Publishing Company.

Henderson, J. C., & Cooprider, J. G. (1990). Dimensions of I/S planning and design aids: a functional model of CASE technology. Information Systems Research, 1(3), 227-254.

Hirschheim, R., & Klein, H. (1989). Four paradigms of information system development. Communications of the ACM, 32, 1199-1216.

Kaplan, B., & Duchon, D. (1988). Combining qualitative and quantitative methods in information systems research: a case study. MIS Quarterly, 12, 571-588.

Knapp, C. A. (1995) An investigation into the organizational and technological factors that contribute to the successful implementation of CASE technology. Doctoral dissertation, City University of New York Graduate Center.

Lee, J., & Kim, S. (1992). The relationship between procedural formalization in MIS development and MIS success. Information & Management, 22(2), 89-11.

Martin, P. Y., & Turner, B. A. (1986). Grounded theory and organizational research. The Journal of Applied Behavioral Science, 22(2), 141-157.

Morey, N. C. (1986) The Process and Product of Organizational Ethnography: An Anthropological Approach. Doctoral dissertation, The University of Nebraska, Lincoln.

Orlikowski, W. J. (1993). CASE tools as organizational change: investigating incremental and radical changes in systems development. MIS Quarterly, 17(3), 309-340.

Strauss, A., & Corbin, J. (1990). Basics of Qualitative Research: Grounded Theory, Procedures and Techniques. Newbury Park, Ca: Sage Publications.

Turner, B. A. (1983). The use of grounded theory for the qualitative analysis of organizational behaviour. Journal of Management Studies, 20(3), 333-348.

Wynekoop, J. L., & Conger, S. A. (1990). A review of computer aided software engineering research methods. In H-E. Nissen, H. K. Klein, & R. Hirschheim (Ed.), Information systems research: Contemporary approaches and emergent traditions, (pp. 129-154). Copenhagen, Denmark: Elsevier Science Publishers, B.V.