Measuring User Involvement:
A Diffusion of Innovation Perspective
Leon A. Kappelman
Associate Professor, Business Computer
Information Systems
Associate Director, Center for Quality and Productivity
College of Business Administration
University of North Texas
P.O. Box 305249, Denton, Texas 76203
Phone: (940) 565-3110
Facsimile: (940) 565-4935
E-mail: kapp@unt.edu
Website: www.year2000.unt.edu/kappelma/
Completed April 24, 1995
Published in the special issue on adoption, diffusion, and implementation of information technology of The DATA BASE for Advances in Information Systems, 26(2&3), (May/August, 1995), 65-86, a quarterly publication of the Association for Computing Machinery's Special Interest Group on Management Information Systems (SIGMIS). An earlier version of this manuscript was completed May 18, 1994 and appeared as Kappelman, L. (1995). "Measuring User Involvement with Information Systems and with Their Development," Journal of Computer Information Systems, (Summer), XXV(4), 66-76.
© 1995 Leon A. Kappelman. All rights reserved.
Dr. Leon A. Kappelman, is a researcher, writer, teacher, speaker, facilitator, and consultant dedicated to helping organizations better manage their information assets. Currently he is directing much of his attention to helping enterprises of all kinds accept and solve their year 2000 computer date problems. He is an Associate Professor of Business Computer Information Systems in the College of Business Administration at the University of North Texas, Associate Director of the Center for Quality and Productivity, and Co-Chair of the Society for Information Management's (SIM) Year 2000 Working Group. After a successful career in industry, he received his Ph.D. (1990) in Management Information Systems (MIS) from Georgia State University. He has conducted research and consulting projects in banking, insurance, aerospace, defense, telecommunications, retail, municipal government, educational, non-profit, sales, marketing, distribution, electric utility, petrochemical, and other organizations. In addition to year 2000, his professional expertise includes the management of information assets, information systems development and maintenance, change management and technology transfer, project management, and information systems assessment and benchmarking. He has published dozens of journal articles and his work has appeared in the Communications of the ACM, Journal of Management Information Systems, InformationWeek, Project Management Journal, the DATA BASE for Advances in Information Systems, National Productivity Review, Industrial Management, Logistics Information Management, the Journal of Systems Management, the Journal of Computer Information Systems, as well as other journals and conference proceedings. He authored Information Systems for Managers, McGraw-Hill (1993) and recently completed Solving the Year 2000 Computer Date Problem: A Guide and Resource Directory, SIM (1996). He can be reached at Box 305249, Denton, Texas 76203; phone (940) 565-3110; facsimile (940) 565-4935; email kapp@unt.edu; website www.coba.unt.edu/bcis/faculty/kappelma
Measuring User Involvement: A Diffusion of Innovation Perspective
Leon A. Kappelman
Abstract
User involvement is a need-based motivational attitude toward
information systems and their development. As such, it has
important implications to the successful creation and deployment
of information systems in organizations. This paper reports on
the development and validation of an instrument to determine if
the distinction between a user's involvement in the process of
information system diffusion can be measured independently of
that user's involvement with the information system innovation
itself. Utilizing previously-validated instruments from consumer
behavior research, these two object-based categories of user
involvement were operationalized. A longitudinal field study was
conducted of users in a large financial institution during the
implementation phase, in particular the later activities of the
adaptation stage, of the information system diffusion process.
During adaptation, the information system product becomes
available for use in the organization. Late adaptation stage
activities include hardware installation, system conversion, and
training. The instruments were pre-tested and assessed as to
their content validity, internal consistency, convergent
validity, unidimensionality, temporal stability, discriminant
validity, predictive validity, and factorial validity. The
evidence indicates that the measurement scales are reliable and
valid. The primary question of scale independence was examined by
discriminant validity. The empirical evidence supports the
theoretical distinction between user process involvement and user
system involvement. The implications of these finding to research
and practice are discussed.
Keywords: Adaptation, attitude, diffusion of innovation, implementation, instrument validation, measurement, research frameworks, research methods, user engagement, user involvement, user participation, user process involvement, user satisfaction, user system involvement.
ACM Keyword Codes:
© 1995 Leon A. Kappelman. All rights reserved.
MEASURING USER INVOLVEMENT:
A DIFFUSION OF INNOVATION PERSPECTIVE
INTRODUCTION
Innovation diffusion theory (Rogers, 1983) provides a general
explanation for the manner in which new things and ideas
disseminate through social systems over time. In the diffusion of
innovation literature, an "innovation" is "an idea
or behavior that is new to the organization adopting it"
(Swanson, 1994, p. 1070). The theory has a communication-oriented
view of innovation-based change with a focus at the individual
level of the process. Information system (IS) studies utilizing
the theory have therefore considered individual characteristics
and perceptions, as well as other theory elements such as social
norms, communication channels, opinion leaders, technology
champions, the time factor, and the characteristics of the
technology being implemented (e.g., Brancheau & Wetherbe,
1990; Moore & Benbasat, 1991; Hoffer & Alexander, 1992;
Borton & Brancheau, 1993; Swanson). Roger's theory appears to
be quite applicable to the implementation of information
technologies in organizations, albeit imperfectly (Brancheau
& Wetherbe; Attewell, 1992).
An important consideration in studies that utilize innovation diffusion theory is how potential adopters' perceptions of the innovation influence the diffusion process (Moore & Benbasat, 1991). This paper reports on the development and validation of an instrument with which to operationalize one such attitude, user involvement, in the context of the diffusion of an information system innovation. In particular, a study was conducted to determine if the distinction drawn by Cooper and Zmud (1990), between a user's involvement in the process of diffusion and a user's involvement with the information system innovation itself, can be operationalized with independent measures. [For simplicity, and because it was not necessary for the purposes of this research, no formal distinction is made in this paper among perceptions, attitudes, feelings, beliefs, expectations, and other invisible mental and/or psychological states ( cf ., Galletta & Lederer, 1989).]
User Involvement
Drawing on the experience of researchers in psychology and
marketing, the distinction between the behavioral and
psychological engagement of information system users with
information systems and their development was proposed by Barki
and Hartwick (1989). They suggested the term
"participation" to refer to the behavioral
engagement of users in information system development activities;
and, the term "involvement" to refer to the psychological
engagement of users with the resultant information system product
of that development process. Behaviors are visible actions,
unlike psychological states, which are invisible to the eye.
Empirical evidence that participation and involvement are, in
fact, independent constructs has been provided by use of
discriminant validity in two studies utilizing different
operationalizations (Jarvenpaa & Ives, 1991; Barki &
Hartwick, 1994). Moreover, several studies have provided
empirical evidence that both participation and
involvement are important in understanding information system
implementation success (Kappelman, 1990; Barki & Hartwick,
1991, 1994; Jarvenpaa & Ives, 1991; Kappelman & McLean,
1991, 1992; and Guimaraes & McKeen, 1993).
Although there is no universal definition for the involvement construct in any field (e.g., Blau, 1985; Zaichowsky, 1986; Bearden, Netemeyer, & Mobley, 1993), a psychological state of involvement is generally said to be engendered by an objectwhen it is of importance, significance, and/or relevance to the individual (e.g., Sherif & Hovland, 1961; Sherif & Sherif, 1967; Apsler & Sears, 1968; Kanungo, Gorn, & Dauderis, 1976; Kanungo, 1979, 1982; Zaichowsky, 1985; 1986). Involvement is conceptualized as a need-based cognitive (or belief) state of psychological identification with some object. Such a state depends upon (1) one's salient needs and (2) one's perception about the need-satisfying potentialities of some object or situation (Kanungo, 1979; Zaichowsky, 1986). Since human motivation is about the satisfaction of needs (Maslow, 1954; Herzberg, 1968), a psychological state of involvement is a result of the perceived (and/or actually experienced) motivational potentialities of some object. Involvement and motivation are closely related, and sometimes synonymous, phenomena (Cook, Hepworth, Wall, & Warr, 1981; Price & Mueller, 1986). Semantic subtleties aside, a motivational state of involvement toward an innovation could markedly affect the outcomes of its diffusion.
The consumer behavior literature makes a distinction between the task and product objects of one's involvement (e.g., Bloch & Richins, 1983). This distinction is analogous to the differentiation between attitudes toward behaviors and attitudes toward things (Fishbein & Ajzen, 1974). Utilizing this differentiation, a user's "attitude toward a[n information] system would be considered an attitude toward an object[; ... whereas,] an attitude concerning system use would be considered an attitude concerning a behavior" (Hartwick & Barki, 1994, p. 443). Such a distinction is important because it parallels the two dominant "sets of information systems activities ...: first, recognizing and assessing information technology innovations; and second, facilitating the diffusion of appropriate technologies into an organization's work units" (Kwon & Zmud, 1987, p. 231). Correspondingly, Cooper and Zmud (1990) distinguish between "process" and "product" related concerns at every stage of their "model" of information system implementation "viewed from a technological diffusion perspective" (p. 124).
Making this distinction between the task and product objects of user involvement was proposed by Kappelman and McLean (1993, 1994). Using the nomenclature of their "user engagement taxonomy" (depicted in Table 1), they suggested that the psychological identification of users with the process of information system development be termed "user process involvement" (i.e., their subjective attitude toward the IS development task). In addition, they proposed "user system involvement" as the term used in reference to the psychological identification of users with respect to the information system itself (i.e., their subjective attitude toward the product of development). Correspondingly, they noted, this process-system dichotomy also applies to the behavioral component of a user's engagement. Thus, as also denoted in Table 1, "participation" is the term applied to the behavioral engagement of users in the process of information system development and "use" is the term employed to designate the behavioral engagement of users with an information system. [ In reference to the total process of conceiving, building, deploying, and operating information systems in organizations, the terms "development" and "diffusion" are used synonymously here to refer to that entire process. ]
Examining the distinctions depicted in Table 1, it seems self evident that there is, in fact, a difference between user participation in the information system development process and their use of the resultant information system product of that process. Such behaviors are easily observable and are observably different. It seems plausible, therefore, that the conceptual distinction between the psychological involvement of users in the process of information system development may, in fact, be different from their psychological involvement with the resultant information system product of that process, i.e., with the innovation itself. But can this distinction between process involvement and system involvement be empirically demonstrated?
Table 1: A User Engagement Taxonomy
+---------- COMPONENTS OF ENGAGEMENT -----------+
| |
| BEHAVIORAL | ATTITUDINAL |
| | |
User Activities | User Involvement
+------ +-----------------------+-----------------------+
| | | |
| | | |
*ISD | PROCESS | PROCESS |
O E | PARTICIPATION | INVOLVEMENT |
B N PROCESS | | |
J G |(task-related behavior)| (task involvement) |
E A (Task) | | |
C G | | |
T E --------+-----------------------+-----------------------+
S M | | |
E | | |
O N *ISD | SYSTEM | SYSTEM |
F T | USE | INVOLVEMENT |
PRODUCT | | |
| | (IS-related behavior) | (product involvement) |
| (The IS)| | |
| | | |
+------ +-----------------------+-----------------------+
*ISD = information system development
(Kappelman & McLean, 1993,1994: Used with permission)
The primary research question therefore asks: Is there an empirically demonstrable difference between the psychological involvement of users in the process of information system diffusion, and their psychological involvement with the information system innovation itself? This distinction is pictured in Figure 1 which differentiates the "user involvement" construct into the two separate and distinctive constructs of user process involvement and user system involvement, respectively, as these concepts were depicted in Table 1. Their independence, of course, would not preclude their also being related; much like, participation and involvement have been shown to be independent and also related phenomena (Jarvenpaa & Ives, 1991; Barki & Hartwick, 1994).
Figure 1: Independence of User Process Involvement and User System Involvement
+----------------+ +----------------+ | User | | User | | Process | | System | | Involvement | | Involvement | +----------------+ +----------------+
It was the purpose of this study to determine if this
distinction between these two object-types of user involvement
can be empirically operationalized. Thus, Figure 1 can be said to
represent a "theory" about the independence of these
two involvement constructs. This theory can be tested through the
analysis of two measurements, one for each of the two constructs
of interest; assuming, of course, the existence of such
measurements. Specifically, it is hypothesized that,
H1: User process involvement and user system involvement are
independent phenomena.
THE CONDUCT OF THE STUDY
A field study was conducted in fifty-two (52) branch offices of a
twenty-eight billion dollar ($28,000,000,000) regional interstate
bank-holding company. All of the branches were located in a large
metropolitan city situated in the southeast United States. These
branches were all recently acquired from several small local bank
companies and were now being converted to the holding company's
organization-wide information system. In terms of the information
system diffusion process, this places these branches into the
"implementation" phase as per Swanson (1994, p. 1071),
and specifically into the "adaptation" stage as per
Cooper and Zmud (1990, p.124). During adaptation, the information
system "product ... [becomes] available for use in the
organization" as the "process ... [sees to it that the]
application is developed, installed, and maintained, ...
procedures are revised and developed[, and] organizational
members are trained" (p. 124).
The information system under study supported all of the bank's activities and linked the bank's branches by satellite, across five southeastern states, to a central data center. Although this information system had been operational for over five years in over 600 of the bank's more than 700 branches, it was a completely new innovation to these 52 branches. Because software development was already successfully completed, this study focuses on the later activities of adaptation, in particular hardware installation, system conversion, and training. The fact that this was a well-established and proven information system facilitated the purposes of the study as it minimized the possibility of confounding due to technological or design failure: This information system had already demonstrated that it did in fact meet the needs (i.e., technological and information requirements) of the organization, and presumably of these branches. This high degree of task-technology congruence (Cooper & Zmud, 1990) facilitated a more concise and valid examination of the variables of interest.
The study consisted of two main phases, which not only differed in time and purpose, but also with respect to the sample and the data they employed. Named for these differences, the terms "pre-test" and "primary" are used to distinguish these two phases and the data collected during them. A questionnaire was developed, pre-tested, and used to collect the relevant data. The regional vice-president of operations provided sponsorship letters which were sent with both the pre-test and primary questionnaires. Complete confidentiality was guaranteed to the respondents by the researcher and by the bank. Sealable envelopes were provided for the return of the questionnaires. The bank's inter-office mail system was used for distribution and collection of questionnaires, but complete information for direct response to the researchers was also provided.
The pre-test was conducted during a period of time which began approximately two weeks before cutover to the new information system. Hardware was being installed and tested, and training was underway. A total of 311 questionnaires were sent out. The sampling methodology used for this pre-test was stratified (Cochran, 1977) in order to facilitate equal representation of each of the 52 branches in the sample. The subjects were given less than two weeks in which to respond and no follow-up letter was used. A total of 103 usable questionnaires were returned.
Five weeks after cutover, which took place over the course of a three-day weekend, the primary research questionnaires were distributed. Training was ongoing during this period, basic bank transaction processing was fully supported by the new system, but some system capabilities were yet to be made available to the branches. Although there are many similarities between the pre-test and the primary data collections, there were three important differences motivated by the researchers' desire to increase the size of the actual response in order to increase the validity of certain statistical techniques, particularly factor analysis (Blau, 1985), the use of which was anticipated: (1) The entire population of 512 users was polled in the primary data collection; (2) the respondents were given more than twice as much time to respond to the primary questionnaire, i.e., nearly four full weeks; and (3) a follow-up postcard was distributed at the beginning of the third week in order to remind the subjects to respond. A total of 146 usable questionnaires were returned. These response rates of 33.1% (103/311) and 28.5% (146/512) are considered acceptable for research of this kind (Cochran, 1977; Dillman, 1978). No consideration was made in this study for non-response bias (Cochran).
The Users and Their Participation in the
Innovation Diffusion Process
Nearly 89% of the branch banks (46 of 52) were represented in the
sample of 146 subjects responding to the primary data collection
questionnaire. More than 73% of the branches (38/52) were
represented by two or more respondents. The subjects(of 131
responding) ranged in age from 18 to 66 years. Twenty-four
percent were younger than 25, 47% were 25 to 40, and nearly 30%
indicated that they were 41 or older. Nearly 87% of the 143
subjects responding to the gender question were female. More than
99% of the subjects were high school graduates, 43% (63/145) had
some college, and 22% (32/145) held four year college degrees.
More than 15% (22/146) of the respondents reported that they were
still attending college.
In order to facilitate predictive and nomological validation of the user involvement measurements, data were also collected concerning the behavioral participation of these users in information system implementation activities. More than 88% of the subjects responding to the primary data collection questionnaire indicated that they had participated in activities typically associated with the adaptation stage in the diffusion of this information system (Cooper & Zmud, 1990). These ten user-participation-in-adaptation questions, and their Likert-type response scale, are shown in Appendix 1. Since this information system was already extant, these 10 questions, adapted from Kappelman (1990), were selected because of their focus on training and installation activities. Participation in software development activities was not possible for these users; however, some of these users did participate in system development activities like hardware installation and testing, file conversion and verification, as well as project management and training-related activities.
Training was the primary form of adaptation-stage participation engaged in by these subject employees, with nearly 73% of the responding subjects (105 of 144) indicating their participation as a trainee. [. User training is a visible behavior that users do engage in during the development and implementation of an information system and is therefore a form of user participation (Kappelman & McLean, 1991, 1992; Barki & Hartwick, 1994).] Scheduling their own training was the next most common form of participation indicated among the respondents, with nearly 56% (80 of 144) responding positively. Thirty-three percent (48/144) indicated their participation as a trainer, more that 31% (45/144) indicated their participation in evaluating the performance of the installed system, and nearly 28% scheduled the training of others. Seventeen to 20 percent of the respondents perceived that they had participated either in planning (17.5%) or scheduling (19%) the conversion, in actual conversion and installation (17.5% and 19%), and in testing the new system (19%).
It is noteworthy, although not directly related to the primary research question of this study, that all of these subjects were involuntary adopters and non-discretionary users. The technology diffusion phases of initiation and adoption took place at the corporate level, and no branch or individual adoption decisions took place. Moreover, as employees of the corporation, these users were required to use this information system to do their jobs. Nevertheless, these subjects were participants in the diffusion of this innovation, they were affected by (and had some effect on) the process, and they were affected by the information system. Moreover, even small effects on employee attitudes, especially motivation-related ones, can have significant economic consequences to the organization (Zedeck & Cascio, 1984; Schneider, 1985), as well as significant implications to the overall success of the innovation diffusion process. Furthermore, if the distinction between process involvement and system involvement can be operationalized for these users, then the measurements used to do so will be applicable to: (1) other types of users, including the voluntary user more typically examined in the diffusion of innovation literature; (2) other stages in the process of diffusion; and, (3) other types information systems (and perhaps, with some modifications, even to other types of innovations).
INSTRUMENT DEVELOPMENT
Since the user process involvement and user product involvement
constructs are actually variations of the larger concept called
user involvement, differing primarily in their respective objects
of involvement, they were measured using minor variations of the
same instrument. The user involvement construct was
operationalized by Zaichowsky's (1985) "Personal Involvement
Inventory" which was developed "to measure a person's
involvement with products" in purchase decisions (p. 349).
There is "strong evidence of reliability and validity"
for this instrument (Barki & Hartwick, 1989), and it has been
used in information system research (e.g., Kappelman, 1990;
Kappelman & McLean, 1991, 1992; Barki & Hartwick, 1994).
Moreover, like Kanungo's (1982) job involvement definition, this
is also a need-based construct. Zaichowsky's "definition of
involvement used for the purposes of scale development was a
person's perceived relevance of the object based on inherent
needs, values, and interests" (p. 342, emphasis
added).
As shown in Appendix 2, the instrument consists of an object statement followed by 20 bipolar adjective-paired items and a seven-choice response scale situated between them. Half of the items are reverse (i.e., negative-positive) scored. Zaichowsky's (1985) instructions were very complete (as can be seen in Appendix 2). Since the same instrument was used to operationalize both user process involvement and user system involvement, the only modification to the content of the measure was in the object name and the instructions as they related to that object name. The sequential ordering of the 20 individual items was also changed in the two versions of the instrument.
User process involvement, an attitude toward a behavior
(Fishbein & Ajzen, 1974), was operationalized as user
participation involvement, i.e., a user's involvement with their
own participation in the information system diffusion process. As
Kappelman & McLean (1994) pointed out:
Since "user involvement" refers to the set of all such
user subjective attitudes toward, or psychological
identifications with, information systems and their development,
... distinctions could also be made among many other
object-of-involvement-based sub-categories within this
task-product dichotomy. For example, one's state of
psychological identification with their own participation in
information system development activities could be termed their
participation involvement" (or, perhaps more lucidly,
their "involvement in their participation") which is a
type of user process involvement (p. 515, italics added).
ANALYSIS AND RESULTS
As stated above, the theory embodied in the primary research
question (as well as its associated research hypothesis and
Figure 1) can be tested through an analysis of two measurements.
Give the existence of these two measurements, the primary
research question is essentially a question of instrument
validation, particularly discriminant validity. Thus, the
analysis which follows is largely a description of the validation
of these two instruments. The approach is fashioned after the
strategy employed by Barki and Hartwick (1994) in assessing the
"construct validity" (i.e., "the extent to which a
scale measures a theoretical variable of interest" p. 69) of
their measurements. Specifically, in making such an assessment
here the following are considered: (1) content validity, (2)
internal consistency, (3) convergent validity, (4) temporal
stability, (5) discriminant validity, (6) predictive validity,
(7) nomological validity, and (8) (uni)dimensionality (i.e.,
factorial validity).
In the following narrative, abbreviations are used when referring to the various scales and sub-scales which are discussed. The number in parenthesis after a construct abbreviation indicates that a reference is being made to a scale consisting of that number of items, e.g., "UPI(20)" stands for the full 20-item user process involvement scale. Consistent with prior use of the scale by its developer (Zaichowsky, 1985), the individual items comprising these two scales were not individually numbered. In order to simplify references to the individual items in these scales, the items which comprise these two user involvement scales are referred to by their sequence number in the scale. Thus, UPI-1 refers to the first item in the UPI(20) scale, UPI-2 the second, and so on. Similarly, USI-1 refers to the first item in the USI(20) 20-item user system involvement scale and USI-20 refers to the last.
Content Validity
Content validity considers how representative and comprehensive
the individual items in a scale are. It is assessed through an
examination of the process by which scale items are generated
(Nunnally, 1978; Straub, 1989). As described above, all of the
items in both scales were taken from Zaichowsky's (1985) 20-item
Personal Involvement Inventory. Her item generation process and
content validation of the scale items had four main stages. It
began with a two-phase process, using two panels of experts, 168
word pairs, and resulted in a 30-item pool of word pairs. Then,
data were collected with these 30 items and an assessment of
internal scale reliability (via item-to-total correlations,
Cronbach's alpha, and factorial validity) resulted in 6 items
being dropped. Using another sample, data were collected with the
remaining 24 items and their test-retest correlations examined:
Four items were dropped and internal consistency reliability
assessed for the remaining 20 items. Finally, using a combination
of an additional data collection and a panel of expert judges, a
second content validation phase was conducted.
Internal Consistency, Unidimensionality, and
Convergent Validity
Reliability in terms of internal consistency, important because
the "items on an opinion scale will be summed in deriving a
total score" (Crano & Brewer, 1973, p. 234), was
assessed by means of Cronbach's (1951) internal consistency
reliability coefficient alpha "probably the best estimate of
internal consistency" (Crano & Brewer, p. 230). The
results are shown in Table 2 which also reports the sample size
actually used in the calculation. Based on the greater than .80
rule-of-thumb (Crano & Brewer, 1973; Nunnally, 1978; Blau,
1988), these calculated coefficients indicate that both the
process and system involvement scales appear to have high
internal consistency. These results are comparable to
Zaichowsky's (1985).
Table 2: Internal Consistency Coefficients of 20-Item User Involvement Scales
Data Set/Instrument Cronbach's Alpha Pre-Test Data (n = 96) User process involvement .943 User system involvement .955 Primary Data (n = 146) User process involvement .966 User system involvement .942
Although Cronbach's (1951) alpha is a widely-used method for assessing internal consistency (e.g, Straub, 1989; Barki & Hartwick, 1994), it is not the only method. Alpha is based on an evaluation of inter-item correlation, another method of assessing internal consistency is by an evaluation of inter-item variability to determine whether the items "share only one common focus" (Crano & Brewer, 1973, p. 231). The purpose of this is to examine the "unidimensionality" of the scale. "This property states that a single construct underlies a set of scale items" (Segars, 1994, p. 2, emphasis of original). Zaichowsky's instrument, as originally developed and examined, is believed to be unidimensional (1985; Bearden, Netemeyer, & Mobley, 1993); however, there is some evidence to the contrary (Zaichowsky; Munson & McQuarrie, 1987; Kappelman & Seitz, 1991; Seitz, Kappelman, & Massey, 1993; Barki & Hartwick, 1994). Based on the literature and an analysis of the data collected in this study, the assumption of a unidimensional scale was made. Nevertheless, in an effort to reconcile some of the conflicting evidence, this topic is more thoroughly examined in the last part of this analysis and results section.
Convergent validity is concerned with determining whether multiple measures of the same construct agree. Other measures of user involvement were not used in this study; and as such, convergent validity was not assessed. Zaichowsky (1985), however, did assess "criterion-related validity ... by comparing the scores from the developed instrument with one or more external variables that provide a direct measure of the variable in question" (p. 345). Her focus was on the level of involvement (low to high) across different product categories, and her findings were in agreement with other studies.
Temporal Stability
Reliability in terms of temporal stability was evaluated here by
means of a test/retest reliability coefficient (Galletta &
Lederer, 1989, p. 424). Zaichowsky (1985) utilized this technique
to assess the temporal stability of each item (in determining its
suitability for inclusion) and then of the entire 20-item scale.
Since some of the pre-test subjects were also primary data
collection subjects, a test/retest reliability coefficient was
calculated for the two user involvement instruments. These
Pearson correlation coefficients between the two administrations
of the same instrument, paired by subject, are reported in Table
3. These represent the correlations between the linear sums
(Blau, 1985) of all of the items in each scale. Both of these
user involvement scales have high test/retest reliability, and
therefore, appear to be temporally stable.
Table 3: Test/Retest Correlation Coefficients (n = 42)
Test/Retest Number Sample
Instrument Correlation of Items Size#
User process involvement .534 + 20 39
User system involvement .691 * 20 40
P-values: * < .0001 + < .0005
# Subjects were eliminated because of missing values.
Discriminant Validity: The Primary Research
Question
The primary research question addressed by this study asks: Is
there a difference between the psychological involvement of users
in the process of information system diffusion and their
psychological involvement with the information system innovation
itself? In order to answer this question, it was necessary to
determine if the relationship depicted in Figure 1 could be
operationalized. Re-stated this way, the research question
became: Can two measurements be found that are operationally
distinct, i.e., independent, such that one measures a user's
involvement in the information system development process and the
other measures a user's involvement with the information system?
This is a question of instrument validation, particularly, the
ability of n instruments (n = 2 in this study), when used
together (e.g., in the same questionnaire), to validly
discriminate between n (n = 2 here) constructs. An assessment of
this ability of instruments to so discriminate is the domain of
discriminant validity (i.e., measurement scale independence) and
is assessed here utilizing principle components and factor
analysis (Saleh & Hosek, 1976; Nunnally, 1978; Kanungo, 1982;
Blau, 1985, 1987, 1988; Jarvenpaa & Ives, 1991; Barki &
Hartwick, 1994); although, other techniques are extant (e.g.,
Bagozzi,1980; Joreskog & Sorbom, 1981, Pedhauzer, 1982).
Specifically, it was hypothesized that:
H1: User process involvement and user system involvement are
independent phenomena.
Since, "to demonstrate discriminant validity, ...
[process involvement and system involvement] items should load on
different factors" (Blau, 1987, p. 248), hypothesis H1 was
restated in terms of the individual items in a scale. Thus
emerged the hypothesis actually tested to determine discriminant
validity, hypothesis H1a.
H1a: In a two-factor, orthogonal solution of the 40 items from
the UPI(20) and USI(20) scales, each item will load predominately
on its respective factor.
The user process involvement and user system involvement scales, UPI(20) and USI(20) respectively, were subjected to a factor analysis which forced an orthogonal two-factor solution of the 40 items. The SAS PROC FACTOR procedure was utilized (SAS, 1990). Criteria were established for determining whether an item loaded predominantly on its respective factor and did not load on the other. Nunnally (1978) propounded the minimum-factor-loading-of-.30 criteria as a guideline for considering an item to be part of a factor. This criteria was used by Blau (1985, 1987, 1988) in order to assess the independence of scale items for various psychometric constructs. Straub (1989) used a minimum factor loading of .50 in his research, as did Ives, Olson, and Baroudi (1983) and Barki and Hartwick (1994). "On the basis of such information, [the researcher] might decide to retain only a certain subset of items" (Crano & Brewer, 1973, p. 232). This "item-dropping" technique is widely used in instrument development and validation (e.g., Blau, 1985; Zaichowsky, 1985; Barki & Hartwick, 1994; Segars, 1994). The criteria used in this research to operationalize "load predominately on its respective factor" were as follows. Each item was subjected to both decision rules.
The two-principal-component solution of the 40 items which comprised the UPI(20) and USI(20) scales accounted for 57.3% of the variance in the scales. A varimax (orthogonal) rotation of these two components resulted in a two-factor solution in which each individual item had two factor loadings, one on each of these two orthogonal factors. These loadings represent the correlation of the item with the factor. The larger of these two factor loadings for each of the twenty items in the UPI(20) scale were on the same factor, and the larger of these two factor loadings for each of the twenty items in the USI(20) scale were on the other factor. The larger factor loading of each of the UPI(20) items ranged from .48 to .90 on one factor, and the larger factor loading of each of the USI(20) items ranged from .47 to .82 on the other factor. This suggested that these two factors could reasonably be named "user process involvement" and "user system involvement" respectively. But the smaller factor loading of six of the items from the UPI(20) scale and four of the items from the USI(20) scale were .30 or larger.
Taking this into account, and including the two USI(20) items with small factor loadings of .29, it was concluded that twelve of the items in the two scales were not indicative of independent measures. Six items from each scale seemed to be also measuring, to some extent, whatever it was the other scale was measuring. This indicated a lack of independence. In other words, using Nunnally's (1978) minimum factor loading of .30 as guidance, each of those 12 items individually failed the test, which resulted in the rejection of hypothesis H1a insofar as that particular item was concerned, and thereby disconfirmed the theory that this item was a valid discriminator. Based on these forty individual hypothesis tests, so to speak, it was decided that these twelve items would be eliminated from the scales, in order to reduce this lack of independence between the two scales.
Thus, items UPI-9, UPI-13, UPI-14, UPI-15, UPI-16, UPI-18,
USI-1, USI-4, USI-6, USI-7, USI-9, and USI-15 were eliminated
from the analysis. An additional item from each scale had a
loading under .50 on its hypothesized factor, and thus failed to
meet the other criteria of independence: Therefore, UPI-2 and
USI-2 were also eliminated. [. It is worth noting that 3 of these
7 eliminated items were shared by the two instruments. These 3
items were UPI-9:USI-9 (matters to me -- doesn't matter to me),
UPI-14:USI-4 (unexciting -- exciting), and UPI-2:USI-15 (of no
concern -- of concern to me). The significance of this is unclear
and may be a rewarding topic for future research. ] The remaining
26 items, 13 for each scale, were subjected to a principal
components factor analysis which forced a two-factor solution.
These "pruned" versions of the UPI(20) and USI(20)
scales are referred to as the UPI(13) and USI(13) scales.
Cronbach's (1951) coefficient alphas of .96 (n = 135) and .93 (n
= 143) were calculated for the UPI(13) and USI(13) scales
respectively. This was similar to the alpha values calculated for
the UPI(20) and USI(20) scales and reported in Table 2.
Hypothesis H1a evolved into hypothesis H1b:
H1b: In a two-factor, orthogonal solution of the
twenty-six(26) items from the UPI(13) and USI(13) scales, each
item will load predominately on its respective factor.
The individual items in these scales are referred to by their sequence in the original 20-item scale. The results of this analysis are reported in Table 4. This two-factor solution of the UPI(13) and USI(13) scales accounted for 62.4% of the variance in the scales. Using the two decision rules described above for hypothesis testing, the factor loadings of each of these 26 items indicated that it loaded cleanly only on its hypothesized factor. These two factors, as seen in Table 4, were named for the constructs they were intended to measure: user process involvement and user system involvement. Although half of the adjective pairs in each of these instruments were in reversed (i.e., negative-positive) order, in Table 4 the positive adjective is listed first: This is representative of the way the data were analyzed. Moreover, all 146 observations were used in the analysis of both the 20-item and 13-item scales; however, SAS omitted 19 from the former and 17 from the latter because of missing values, leaving effective sample sizes of 127 and 129 respectively. [Due to space constraints, some of the empirical evidence could not be included here. Please contact the author for additional information such as: (1) factor loadings (before and after rotation) and covariance matrices of the 40-item, 26- item, 18-item, 13-item, and 9-item involvement analyses; (2) individual item descriptive statistics (mean, standard deviation, sum, minimum, maximum); and (3) item-to-item and item-to-total correlations; etc.]
Table 4: Varimax-Rotated Factor Loadings (x100) of 13-Item User Involvement Scales
==== User Involvement =====
Sequence # Process System
Adjective Pair in Scale Scale:UPI(13) Scale:USI(13)
------------------------------------------------------------------------
vital-superfluous UPI-12 90 04
valuable: worthless UPI-6 89 19
significant: insignificant UPI-11 88 10
essential: nonessential UPI-17 87 17
beneficial: not beneficial UPI-8 85 23
relevant: irrelevant UPI-3 84 06
useful: useless UPI-5 82 20
needed: not needed UPI-20 82 22
wanted: unwanted UPI-19 80 23
important: unimportant UPI-1 79 06
fundamental: trivial UPI-7 78 20
means a lot: means nothing UPI-4 76 12
interested: uninterested UPI-10 72 23
important: unimportant USI-14 16 82
valuable: worthless USI-13 10 82
appealing: unappealing USI-5 16 79
desirable: undesirable USI-20 07 77
essential: nonessential USI-19 17 76
relevant: irrelevant USI-16 17 74
needed: not needed USI-12 00 72
wanted: unwanted USI-11 14 72
beneficial: not beneficial USI-3 10 69
vital: superfluous USI-8 21 65
interesting: boring USI-10 21 64
fascinating: mundane USI-18 17 58
fundamental: trivial USI-17 10 56
---------- ---------
Percent of 2-factor variance accounted for: 56.4% 43.6%
Table 4 is the orthogonal, two-factor solution of the 26 items of these two pruned user involvement scales. The higher an item loaded on its respective factor and the lower it loaded on the other, is an indication of how well that item discriminates the construct of interest. The factor loadings shown in Table 4 indicate that each of these 26 items has withstood the test of hypothesis H1b (Nunnally, 1978). Therefore, an affirmative response to the primary research question is suggested. In other words, it appears that independent empirical measurements do exist to operationalize the distinctive difference between the involvement of users in the process of information system implementation (specifically adaptation) and their involvement with the information system innovation itself. The indication being that user process involvement and user system involvement are independent phenomena.
Predictive Validity and Nomological Validity
Predictive and nomological validity differ only by degree since
both are concerned with the theory-based ability of measures to
predict measures of other constructs (Bagozzi, Davis, &
Warshaw, 1992). In the case of the latter, the other constructs
are part of a theoretical network of relationships. Space and
study limitations preclude a more thorough analysis; however,
correlations are used to provide some indications as to
congruence with theoretical expectations.
Participation -- Involvement: In the
innovation literature, participation is generally positively
associated with attitudes toward change (Kwon & Zmud, 1987).
User participation was hypothesized (Barki & Hartwick, 1989)
and confirmed (Kappelman & McLean, 1991, 1992) as an
antecedent of user system involvement. It follows that
participation is also an antecedent of process involvement.
Causality aside, it was hypothesized that:
H2-a: User participation is positively associated with user
system involvement.
H2-b: User participation is positively associated with user
process involvement.
The correlations between the linear sums of the 10-item user participation scale (Appendix 1) and the two 13-item user involvement scales were .29 (p < .0006, n = 134) for process involvement and .20 (p < .0155, n = 142) for system involvement. Both hypotheses are confirmed.
Process Involvement -- System Involvement:
In situations when process participation actually precedes actual
system contact, it would seem logical that process involvement is
an antecedent of system involvement. It is posited that this is
similar to the associations depicted in the "technology
acceptance model" (Davis, Bagozzi, & Warshaw, 1989),
where related attitudes gradually become more directed toward the
information system as a whole. Issues of antecedence aside, it is
hypothesized that:
H2-c: User process involvement and user system involvement are
positively associated.
The correlation between the linear sum of the two 20-item scales was .52 (p < .0001, n = 137) and the correlation between the linear sum of the two 13-item scales was .38 (p < .0001, n = 133). Although smaller in size, it is believed that this latter number represents a more valid and accurate reflection of the true association between process involvement and system involvement in this study. The hypothesis is confirmed.
Participation -- Involvement -- Satisfaction:
There is evidence that involvement mediates the
participation-satisfaction relationship; and as such, both user
participation and user involvement are antecedents of user
satisfaction (Kappelman & McLean, 1991, 1992). Causal models
aside, it was hypothesized that:
H2-d: User participation is positively associated with user
satisfaction.
H2-e: User process involvement is positively associated with
user satisfaction.
H2-f: User system involvement is positively associated with
user satisfaction
The single overall user satisfaction item (shown in Appendix 1) was used in this analysis. It was taken from (Kappelman & McLean, 1991, 1992) and such single-item overall satisfaction measures have been shown to be reliable and valid (Galletta & Lederer, 1989). The correlations with this single item were calculated to be .15 (p < .0821, n = 132) for process involvement, .33 (p < .0001, n = 139) for system involvement, and .22 (p < .0095, n = 140) for participation. These findings are comparable to Kappelman and McLean (1991, 1992), except they did not examine process involvement. The hypotheses are confirmed; although, the evidence of an association between process involvement and satisfaction is weak.
On the Dimensionality of the User
Involvement Construct
Concern for the unidimensionality of a scale, sometimes discussed
in terms of the "homogeneity" among a set of scale
items, is substantiated by the work of many researchers (e.g.,
Allen & Yen, 1979; Scarpello & Campbell, 1983; Galletta
& Lederer, 1989, Segars, 1994). This concern is founded on
"cautions against combining measures of separate personal
qualities into composite variables in the hope of tapping a
deeper theoretical construct" (Bynner, 1988, p. 403).
Moreover, Cronbach's (1951) alpha assumes item homogeneity
(Galletta & Lederer). Factorial validity, sometimes
considered to be an indicator of both the reliability and
validity of research measurements (e.g., Blau, 1985; Straub,
1989; cf., Crano & Brewer, 1973; Bynner, 1988), was
used to assess the dimensionality of the two user involvement
scales.
"Factorial validity helps to confirm that a certain set of measures do or do not reflect latent constructs" (Straub, 1989, p. 160) and that the individual items which make-up an instrument "share only one common focus" (Crano & Brewer, 1973, p. 231). The factorial validity of the two 13-item scales was assessed using primary data by means of principal components and factor analysis using SAS PROC FACTOR (SAS, 1990); although, other techniques to assess unidimensionality are sometimes used (e.g., Segars, 1994). Regardless of technique, however, those who wish to interpret factors or other statistical evidence as "real dimensions must shoulder a substantial burden of proof" cautioned Cronbach and Meehl (1955, quoted in Bynner, 1988, p. 391). Theory, logic, and common sense are also important in this decision process (e.g., Bynner; Galletta & Lederer, 1989). Nevertheless, some guidelines have been developed for interpreting the statistical indications.
Kaiser (1960) proposed the eigenvalue-greater-than-one criteria as evidence of a component or factor indicating an underlying construct. Cattell (1966) proffered the use of the graphical screen test as evidence of the number of underlying constructs. Gorsuch (1974) posited the use of both these guidelines. Another important consideration is the amount of overall variance accounted for by a principal component, measured in terms of the eigenvalue of that component. Another "generally accepted criteria for unidimensionality" in a set of scale items is for the first principal component to account for at least six times the variance of the scale accounted for by the second principal component (Bynner, 1988, p. 397); although, it would seem that instrument size may be a consideration in using this rule-of-thumb. Minimum factor loading guidelines for considering an item to be part of a factor (e.g., Nunnally, 1978; Straub, 1989) are also applied, and item dropping used in order to achieve a unidimensional scale (e.g., Crano & Brewer, 1973; Blau, 1985; Zaichowsky, 1985; Kappelman & McLean, 1991, 1992; Barki & Hartwick, 1994; Segars, 1994).
The UPI(13) scale had only one principal component with eigenvalues greater than one, a range of factor loadings from 75 to 91 on the first principal component that accounted for 70.4% of the variance in the scale and was 12.5 times larger than the second principal component. By all guidelines, the evidence suggested that this process involvement scale was unidimensional. On the other hand, mixed indications were calculated for the USI(13) scale which had three principal components with eigenvalues greater than one, a range of factor loadings from 60 to 85 on the first principal component that accounted for 54.7% of the variance in the scale and was 5.0 times larger than the second principal component.
As mentioned above, this was not the first evidence questioning the unidimensionality of Zaichowsky's (1985) scale (e.g., Munson & McQuarrie, 1987; Kappelman & Seitz, 1991; Seitz, Kappelman, & Massey, 1993; Barki & Hartwick, 1994). Zaichowsky found a varying number of greater-than-one eigenvalues depending on the object of involvement, although one component consistently accounted for about 70% of the variation in the data. McQuarrie and Munson (1986) suggested that Zaichowsky's scale was contaminated with attitudinal items and proposed a modified two-dimensional 16-item version (Munson & McQuarrie, 1987) of the scale, but the two dimensions (attitudinal and arousal involvement) were by no means independent. Another version (McQuarrie & Munson, 1991) had two dimensions named perceived importance and interest. Similarly, Barki & Hartwick (1994) derived a 13-item three-dimensional instrument from Zaichowsky's scale. None of the factor patterns of individual items shared among these instruments is consistent: Neither do the same items consistently load together, nor do the factor patterns appear to be stable across objects or over time.
As an example, Zaichowsky's (1985) item "relevant/irrelevant" loaded cleanly on Barki and Hartwick's (1994) "personal relevance" factor; it also loaded on their "importance" and "attitude" factors in their pre-development data set. Similarly, Zaichowsky's "important/unimportant" item loaded cleanly on Barki and Hartwick's "importance" factor; it also loaded strongly on their "relevance" and "attitude" factors in their pre-development data set. They concluded that "involvement and attitude ... were not distinguishable ... in the pre-development sample" (p. 67). But both of these items are part of McQuarrie and Munson's (1991) "importance" factor, and both of them loaded on both factors in their earlier scale (1987). Moreover, their two-factor pattern appeared in just slightly more that half their samples (1987). And this is only the tip of the iceberg: There is a plethora of additional involvement instruments, versions of involvement instruments, and proposed dimensions of involvement (e.g., Cook, Hepworth, Wall, & Warr, 1981; Price & Mueller, 1986; Blau, 1985; Zaichowsky, 1986; Bruner &. Hensel, 1992; Bearden, Netemeyer, & Mobley, 1993).
There is no simple answer: The situation is in transition, under development, and being studied. Even the possibility that perhaps there is dimensional stability over object types, in particular information system objects of involvement, seems faint: A replication (using the data set of this study) of the two-factor, varimax rotation of the 9 system-involvement items from Zaichowsky (1985) that Barki and Hartwick (1994) used, only confirmed their findings for two items from each of their two dimensions. Neither the important/unimportant nor the relevant/irrelevant item loaded cleanly on a single factor, and they both loaded more significantly on the same factor. A similar pattern was evident for the 9-item user process involvement scale. Moreover, an 18-item, 2-factor, orthogonal solution of these two instruments did not exhibit discriminant validity and had 5 items that failed to meet the two decision rules described above.5 Perhaps Zaichowsky was right when she said "the assumption is that no individual item is sufficient, and that it is the scale taken as a whole that tends to measure the involvement construct" (1985, p. 344). She (1990) too, however, derived an allegedly unidimensional, 10-item version of her original 20-item scale, for the purposes of measuring involvement toward advertising, that included items from all three of Barki and Hartwick's (1994) dimensions.
There is neither sound theoretical nor empirical evidence for concluding that Zaichowsky's (1985) Personal Involvement Inventory is anything other than a unidimensional scale for measuring "a person's perceived relevance of the object based on inherent needs, values, and interests" (p. 342). There is evidence, however, for concluding that (1) the factor pattern of the scale items varies by the object of involvement and over time, and (2) that further research is clearly needed. Nevertheless, none of this diminishes the facts that (1) significant improvements have been made recently in the development of measurements for important user-related behavioral and attitudinal variables (e.g., Kappelman & McLean, 1991, 1992; Barki & Hartwick, 1994), and (2) that user process involvement and user system involvement have been distinctly operationalized here. These facts have important implications to those who practice, study, and teach information systems.
SUMMARY, IMPLICATIONS, AND CONCLUSION
Kappelman & McLean (1993, 1994) categorized the associations
of users with information systems and their development as forms
of "user engagement" along two dimensions: On the one
hand, a distinction is made between the behavioral and
attitudinal components of such engagements; and on the other, a
distinction is made between the process and product objects of
such engagements. These distinctions result in the four
dimensions of user engagement depicted in Table 1. Empirical
evidence in support of the validity and importance of some of
these distinctions was already available (Kappelman, 1990; Barki
& Hartwick, 1991, 1994; Jarvenpaa & Ives, 1991; Kappelman
& McLean, 1991, 1992; and Guimaraes & McKeen, 1993).
Moreover, the distinction between attitudes toward things and
attitudes toward behaviors was well established (e.g., Fishbein
& Ajzen, 1974). Nevertheless, little was known about the
theorized distinction between the psychological involvement of
users with information systems and their psychological
involvement with the process of developing and implementing such
systems. This study endeavored to remedy that situation with a
focus on the adaptation stage in the diffusion of an information
system innovation.
Utilizing previously-validated instruments from consumer behavior research, these two object-based categories of user involvement were operationalized. A longitudinal field study was conducted of users during information system conversion in a large financial institution. The instruments were pre-tested and assessed as to their content validity, internal consistency, convergent validity, unidimensionality, temporal stability, discriminant validity, predictive validity, and factorial validity. The evidence indicated that the scales were reliable and valid. The primary question of scale independence was examined by discriminant validity. The empirical evidence supports the concepts embodied in the user engagement taxonomy: The theoretical distinction between user process involvement and user system involvement has been empirically confirmed.
There are important implications, both for research and practice, of this newly-found ability to discriminate between process involvement and system involvement; because, this distinction parallels the two dominant sets of information systems activities: (1) recognizing and assessing information-system innovations and (2) facilitating their diffusion (Kwon & Zmud, 1987). Moreover, the ability to measure these important motivational constructs is not diminished by uncertainty over which items or which dimensions. That is not to say that these yet-to-be-understood considerations should be forgotten, in fact, they should be rigorously examined. But these unknowns should not stop us from using assessments of these motivational outcomes in order to help us more effectively study, and more successfully implement and manage, information systems in organizations.
Until more is known about the dimensionality of the user involvement construct, it is recommended that the instruments examined here be used in their full 20-item form. Adjustments can easily be made for poorly discriminating items following the procedures described in this paper. Since all of the data examined here were collected only during the adaptation stage in the diffusion of this information system, it raises questions about how these instruments might perform during other phases and/or with other types of information systems and/or with other types of users (e.g., discretionary or voluntary). For example, it may be that user involvement, since it is a motivational construct, may help in the early identification of different types of adopters, champions, and change agents (e.g., Rogers, 1983; Beath, 1991; Kappelman, 1995). Moreover, considerations of repeated-use and long-term temporalstability should be examined, since innovation diffusion theory is itself a longitudinal theory and since IS-based change is a longitudinal phenomenon.
Future research should determine if some kind of "attitude formation" phenomena has been identified with regard to user involvement (Barki & Hartwick, 1994). Is it possible that the strong evidence for a unidimensional process-involvement scale presented in this study was a function of the fact that the process of adaptation was largely behind these users and their involvement with it had stabilized? Concomitantly, is the weak evidence of unidimensionality for the system-involvement scale a sign that the "jury was still out" because the system innovation itself was still too new? The lower test/retest score for process involvement suggests that more change occurred in this measure. Longitudinal studies would certainly be one potentially fruitful avenue to take in examining these issues. So would the use of other statistical techniques, in particular structural equation modeling (e.g., Bagozzi, 1980; Joreskog & Sorbom, 1981, Pedhauzer, 1982; Segars, 1994). This technique would be particularly valuable not only for purposes of scale development and validation, but also to further our understanding of the nomological networks and causal models which may include these user involvement constructs.
Previous research has established that both user involvement (i.e., attitudinal engagement) and user participation (i.e., behavioral engagement) are important in understanding (and achieving) information system implementation success (e.g., Kappelman, 1990; Barki & Hartwick, 1991, 1994; Jarvenpaa & Ives, 1991; Kappelman & McLean, 1991, 1992). Previous evidence suggests that participation induces system involvement, which intervenes (i.e., mediates) in the participation-satisfaction relationship (Kappelman; Kappelman & McLean). The correlation analysis conducted in this study in assessing predictive validity suggests the hypothesis that participation induces process involvement, which intervenes in the relationship between participation and system involvement. Given the potential importance of these need-based, motivational involvement constructs in understanding user behaviors, additional research could prove worthwhile. Moreover, these involvement constructs must be examined in relationship to other user attitudes that have already been examined in a diffusion-of-innovation context. It would seem that these motivational states of involvement may help us to better understand (and thereby manage) important behavioral constructs like use, adoption, and acceptance (especially in the context of voluntary users). One fertile research direction may be to examine user process and system involvement in the context of some of the behavioral-attitudinal work already conducted in these areas (e.g., Davis, 1989; Davis, Bagozzi, & Warshaw, 1989; Moore & Benbasat, 1991).
The user involvement instruments suggested by this study may provide information system researchers and practitioners with the ability to better understand, and thereby manage, these critically important psychological components of users. Information system vendors, practitioners, researchers, and academics would be well served to know what kinds of user assistance and support services, opportunities for behavioral and attitudinal engagement, and system diffusion and implementation strategies produce the largest payoff in various situations.
REFERENCES
Allen, M. J. and Yen, W. M., (1979). Introduction to Measurement Theory, Brooks/Cole, Monterey, California.
Apsler, R., and Sears, D. O. (1968). "Warning, Personal Involvement, and Attitude Change." Journal of Personality and Social Psychology, 9(2), 162-166.
Attewell, P. (1992). "Technology Diffusion and Organizational Learning: The Case of Business Computing," Organization Science, 3(1), February, 1-19.
Bagozzi, R. (1980). Causal Models in Marketing. New York: Wiley.
Bagozzi, R., Davis, F., and Warshaw, P. (1992). "Development and Test of a Theory of Technological Learning and Usage," Human Relations, 45(7), 659-686.
Barki, H., and Hartwick, J. (1989). "Rethinking The Concept Of User Involvement." MIS Quarterly, 13(1), 53-64.
Barki, H., and Hartwick, J. (1991). "User Participation And User Involvement In Information System Development," Proceedings of the Hawaiian International Conference on the System Sciences, 487-492.
Barki, H., and Hartwick, J. (1994). "Measuring User Participation, User Involvement, and User Attitude." MIS Quarterly, 18(1), 59-82.
Bearden, W., Netemeyer, R., and Mobley, M. (1993). Handbook of Marketing Scales. Newbury Park: Sage.
Beath, C. M. (1991). "Supporting the Information Technology Champion," MIS Quarterly, 15(3), 355-372.
Bloch, P. H., and Richins, M. (1983). "A Theoretical Model For The Study of Product Importance," Journal of Marketing, 47 (Summer), 69-81.
Blau, G. J., (1985) "A Multiple Study Investigation of the Dimensionality of Job Involvement," Journal of Vocational Behavior, Volume 27, 19-36.
Blau, G. J. (1987). "Using A Person-Environment Fit Model To Predict Job Involvement And Organizational Commitment." Journal of Vocational Behavior, 30(3), 240-257. (From PsycLIT Database, 1987, Abstract No. 74-33079).
Blau, G. J. (1988). "Further Exploring The Meaning And Measurement Of Career Commitment." Journal of Vocational Behavior, 32, 284-297.
Borton, J., and Brancheau, J. (1993). "Does and Effective Information Technology Implementation Process Guarantee Success?," in Diffusion, Transfer, and Implementation of Information Technology, L. Levine (Ed.), Elsevier Science B.V. (North-Holland), IFIP Transactions A-45, 159-178.
Brancheau, J. and Wetherbe, J. (1990). "The Adoption of Spreadsheet Software: Testing Innovation Diffusion Theory in the Context of End-User Computing," Information System Research, 1(2), 115-143.
Bynner, J. (1988). "Factor Analysis and the Construct Indicator Relationship," Human Relations, 41(5), 389-405.
Bruner II, G., and Hensel, P. (1992), Marketing Scales Handbook: A Compilation of Multi-Item Measures, Chicago, American Marketing Association, .
Cattell, R. (1966). Handbook of Multivariate Experimental Psychology, Rand McNally, Chicago.
Chaiken, S. (1980). "Heuristic Versus Systematic Information Processing and the Use of Source Versus Message Cues in Persuasion." Journal of Personality and Social Psychology, 39(5), 752-766.
Cochran, W. G. (1977). Sampling Techniques. (3rd edition). New York: John Wiley & Sons.
Cook, J. D., Hepworth, S. J., Wall, T. D., and Warr, P. B. (1981). The Experience Of Work. New York: Academic Press.
Cooper, R. and Zmud, R. (1990). "Information Technology Implementation Research: A Technological Diffusion Approach," Management Science, 36(2), 123-139
Crano, W. D. and Brewer, M. B. (1973). Principles of Research in Social Psychology, McGraw-Hill, New York.
Cronbach, L. J. (1951). "Coefficient Alpha and the Internal Structure of Tests," Psychometrika, 16, 297-334.
Cronbach, L. J., and Meehl, P. (1955). "Construct Validity in Psychological Tests," Psychological Bulletin, 21, 281-302.
Davis, F. (1989). "Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology," MIS Quarterly, 13(3), 319-340.
Davis, F., Bagozzi, R., and Warshaw, P. (1989). "User Acceptance of Computer Technology: A Comparison of Two Theoretical Models," Management Science, 35(8), August, 982-1003.
Dillman, D. A. (1978). Mail and Telephone Surveys: The Total Design Method, John Wiley, New York.
Fishbein, M. and Ajzen, I. (1974). "Attitudes Toward Objects as Predictors of Single and Multiple Behavior Criteria," Psychological Review, 81(1), 59-74.
Galletta, D. F. and Lederer, A. L., "Some Cautions on the Measurement of User Information Satisfaction," Decision Sciences Journal, Volume 20, Number 3, 1989, pp. 419-438.
Gorsuch, R., Factor Analysis, Saunders, Philadelphia, 1974.
Guimaraes, T., and McKeen, J. D. (1993). "User Participation in Information System Development: Moderation in All Things." Proceedings of the IFIP Working Group 8.2 Conference on Information Systems Development: Human, Social, and Organizational Aspects, Noordwijkerhout, The Netherlands, May, 171-191.
Hartwick, J., and Barki, H. (1994). "Explaining the Role of User Participation in Information System Use," Management Science, 40(4), 440-465.
Herzberg, F. (1968). "One More Time: How Do You Motivate Employees?," Harvard Business Review, (January-February).
Hoffer, J. A. and M. B. Alexander (1992). "The Diffusion of Database Machines", DataBase, 23(2) 13-20.
Ives, B., Olson, M. H. and Baroudi, J. J. (1983). "The Measurement of User Information Satisfaction," Communications of the ACM, 26(10), 785-793.
Jarvenpaa, S. L., and Ives, B. (1991). "Executive Involvement and Participation in the Management of Information Technology." MIS Quarterly, June, 204-227.
Joreskog, K., and Sorbom, D. (1981). LISREL V: Analysis of Linear Structural Relations by the Method of Maximum Likelihood. Chicago: International Educational Services.
Kaiser, H., (1960). "The Application of Electronic Computers to Factor Analysis," Educational and Psychological Measurement, 20, 141-151.
Kanungo, R. N., Gorn, G. J., and Dauderis, H. J. (1976). "Motivational Orientation of Canadian Anglophone and Francophone Managers." Canadian Journal of Behavioral Science, 8, 107-121.
Kanungo, R. (1979). "The Concepts of Alienation and Involvement Revisited." Psychological Bulletin, 86(1), 119-138.
Kanungo, R. (1982). "Work Alienation: An Integrative Approach." New York: Praeger.
Kappelman, L. A. (1990). The Implementation of Computer-Based Information Systems: The Respective Roles Of Participation And Involvement In Information System Success. Unpublished Doctoral Dissertation, Georgia State University.
Kappelman, L. A. (1995). "Standards: The New 800-Pound Gorilla," InformationWeek, (April 10), 96.
Kappelman, L. A., and McLean, E. R. (1991). "The Respective Roles of User Participation and User Involvement in Information System Implementation Success." Proceedings of the Thirteenth International Conference on Information Systems, 339-349.
Kappelman, L. A., and McLean, E. R. (1992). "Promoting Information System Success: The Respective Roles of User Participation and User Involvement." Journal of Information Technology Management, 3(1), 1-12.
Kappelman, L., and McLean, E. (1993). "User Engagement in Information System Development, Implementation, and Use: Towards Conceptual Clarity," in Diffusion, Transfer, and Implementation of Information Technology, L. Levine (Ed.), Elsevier Science B.V. (North-Holland), IFIP Transactions A-45, 199-214.
Kappelman, L., and McLean, E. (1994). "User Engagement in the Development, Implementation, and Use of Information Technologies," Proceedings of the 27th Hawaiian International Conference on the System Sciences, Maui, January, Volume IV, 512-521.
Kappelman, L. and Seitz, V. (1991). "Some Cautions on the Measurement of Involvement," Proceedings of the Decision Sciences Institute, Volume I, Miami (November 24-26), 229-231.
Kwon, T. H., and Zmud, R. W. (1987). "Unifying The Fragmented Models Of Information Systems Implementation." In R. J. Boland & R. A. Hirschheim (Eds.), Critical Issues In Information Systems Research. (227-251). New York: John Wiley & Sons, Ltd.
Maslow, A. H. (1954). Motivation And Personality. New York: Harper & Row.
McQuarrie, E. and Munson, J. M. (1991). "A Revised Product Involvement Inventory: Improved Usability and Validity," in Advances in Consumer Research, Vol. 19, J Sherry and B. Sternthal (eds.) Provo, UT: Association for Consumer Research.
McQuarrie, E. and Munson, J. M. (1986). "The Zaichowsky's Personal Involvement Inventory: Modification and Extension," in Advances in Consumer Research, Vol. 14, P. Anderson and M. Wallendorf (eds.),, Provo, UT: Association for Consumer Research, 36-40.
Moore, G and Benbasat, I. (1991). Development of an Instrument to Measure the Perceptions of Adopting an Information Technology innovation," Information System Research, 2(3), 192-222.
Munson, J. M., and McQuarrie, E. (1987). "The Factorial and Predictive Validates of a Revised Measure of Zaichowsky's Personal Involvement Inventory," Educational and Psychological Measurement, 47, 773-782.
Nunnally, J., Psychometric Theory, 2nd ed., McGraw-Hill, New York, 1978.
Pedhauzer, E. J. (1982). Multiple Regression In Behavioral Research. (2nd edition). New York: Holt, Reinehart, & Winston.
Price, J. L., and Mueller, C. W. (1986). Handbook Of Organizational Measurement. Marshfield, MA: Pitman Publishing Inc.
Rogers, E. (1983). Diffusion of Innovations (3rd edition), Free Press, New York.
Saleh, S., and Hosek, J. (1976). "Job Involvement: Concepts and Measurements," Academy of Management Journal, 19, 265-288.
SAS Institute (1990). SAS/STAT User's Guide, Version 6 Fourth Edition, SAS Institute, Cary, North Carolina.
Scarpello, V. and Campbell, J. P., "Job Satisfaction: Are All the Parts There?," Personnel Psychology, Volume 36, 1983, pp. 577-600.
Schneider, B. (1985). Organizational behavior. Annual Review of Psychology, Vol. 36, pp. 573-611.
Seitz, V., Kappelman, L., and Massey, T. (1993). "Measuring Involvement: Another Look," Proceedings of the American Marketing Association Winter Conference, Volume 4: Marketing Theory and Applications, 292-297.
Segars, A. (1994). "Scale Development in Information Systems Research: A Paradigm Incorporating Unidimensionality and its Assessment," Proceedings of the 15th International Conference on Information Systems, Vancouver.
Sherif, M., and Hovland, C. (1961). Social Judgement. New Haven, Connecticut: Yale University Press.
Sherif, C., and Sherif, M. (Eds.) (1967). Attitude, Ego-Involvement, and Change. New York: John Wiley & Sons, Inc.
Straub, D. W., (1989). "Validating Measurements in MIS Research," MIS Quarterly, 13(2 ), 147-169.
Swanson, E. B., (1994). "Information Systems Innovation Among Organizations," Management Science, 40(9), 1069-1092.
Zaichowsky, J. (1985). "Measuring The Involvement Construct." Journal of Consumer Research, 12, 341-352.
Zaichowsky, J. (1986). "Conceptualizing Involvement." Journal of Advertising, 15(2), 4-14.
Zaichowsky, J. (1990). The Personal Involvement Inventory: Reduction and Application to Advertising, Unpublished Working Paper, Burnaby, British Columbia: Simon Fraser University. Reported in Bearden, W., Netemeyer, R., and Mobley, M. (1993). Handbook of Marketing Scales. Newbury Park: Sage.
Zedeck, S. and Cascio, W. (1984). Psychological issues in personnel decisions. Annual Review of Psychology, Vol. 35, 461-518.
APPENDIX 1
QUESTIONNAIRE ITEMS FOR USER PARTICIPATION AND USER SATISFACTION
Adaptation Stage User Participation Questions:
Regarding the NEW SYSTEM, I participated ...
Adaptation Stage User Participation Answer Scale:
| 0 | 1 | 2 | 3 | 4 | 5 |
| Not Applicable | Very Little | A Little | Moderately | Much | Very Much |
Overall User Satisfaction Question:
Overall, I am very satisfied with the new system.
Overall User Satisfaction Answer Scale:
| 1 | 2 | 3 | 4 | 5 |
| Strongly Disagree |
Disagree | Neither Agree or Disagree |
Agree | Strongly Agree |
APPENDIX 2: User Involvement Questionnaires
INSTRUCTIONS: The purpose of the following questions is to measure a person's involvement or interest in the PROCESS OF IMPLEMENTING a new computer-based information system.
PLEASE NOTE:"Implementation" refers to the activities listed in the previous questions about your participation. Part of implementation is the actual "conversion" to the new system.
To take this measurement, we need you to judge your participation in the new system implementation process against a series of descriptive scales, according to how YOU perceive it. Here is how to use these scales:
If you feel that your participation in the process of
implementation was very closely related to one end of
the scale, you should place your mark as follows:
important X_:__:__:__:__:__:__
unimportant or important __:__:__:__:__:__:_X unimportant
If you feel that your participation was closely related to one or the other end of the scale (but not extremely), you should place your mark as follows: appealing __:X_:__: or :__:_X:__ unappealing
If you feel that your participation seems only slightly related to one end of the scale (but not really neutral), you should place your mark as follows: uninterested __:__:X_: or :_X:__:__ interested
If you feel that your participation was equally related to either end of the scale (that is, neutral), you should place your mark as follows: essential __:__:__:_X_:__:__:__ nonessential
IMPORTANT:
MY PARTICIPATION IN THE new system IMPLEMENTATION PROCESS (is/was) ...
important __:__:__:__:__:__:__ unimportant
of no concern __:__:__:__:__:__:__ of concern to me
irrelevant __:__:__:__:__:__:__ relevant
means a lot to me __:__:__:__:__:__:__ means nothing to me
useless __:__:__:__:__:__:__ useful
valuable __:__:__:__:__:__:__ worthless
trivial __:__:__:__:__:__:__ fundamental
beneficial __:__:__:__:__:__:__ not beneficial
matters to me __:__:__:__:__:__:__ doesn't matter to me
uninterested __:__:__:__:__:__:__ interested
significant __:__:__:__:__:__:__ insignificant
vital __:__:__:__:__:__:__ superfluous
boring __:__:__:__:__:__:__ interesting
unexciting __:__:__:__:__:__:__ exciting
appealing __:__:__:__:__:__:__ unappealing
mundane __:__:__:__:__:__:__ fascinating
essential __:__:__:__:__:__:__ nonessential
undesirable __:__:__:__:__:__:__ desirable
wanted __:__:__:__:__:__:__ unwanted
not needed __:__:__:__:__:__:__ needed
INSTRUCTIONS:The purpose of the following questions is to measure your involvement or interest in the new computer-based information system itself. The instructions are the same as in the section you just completed, except that now you are to judge the NEW COMPUTER SYSTEM against a series of descriptive scales according to how YOU perceive it.
Please, only one mark for each scale.
Very closely related to one end: _X:__:__: or :__:__:X_
Closely related to one end: __:_X __: or :__:X_:__
Slightly related to one end: __:__:_X: or :X_:__:__
Equally related to either end: __:__:__:_X_:__:__:__
THE NEW COMPUTER SYSTEM (is/was) ...
means a lot to me __:__:__:__:__:__:__ means nothing to me
useless __:__:__:__:__:__:__ useful
beneficial __:__:__:__:__:__:__ not beneficial
unexciting __:__:__:__:__:__:__ exciting
appealing __:__:__:__:__:__:__ unappealing
uninterested __:__:__:__:__:__:__ interested
significant __:__:__:__:__:__:__ insignificant
vital __:__:__:__:__:__:__ superfluous
matters to me __:__:__:__:__:__:__ doesn't matter to me
boring __:__:__:__:__:__:__ interesting
wanted __:__:__:__:__:__:__ unwanted
not needed __:__:__:__:__:__:__ needed
valuable __:__:__:__:__:__:__ worthless
important __:__:__:__:__:__:__ unimportant
of no concern __:__:__:__:__:__:__ of concern to me
irrelevant __:__:__:__:__:__:__ relevant
trivial __:__:__:__:__:__:__ fundamental
mundane __:__:__:__:__:__:__ fascinating
essential __:__:__:__:__:__:__ nonessential
undesirable __:__:__:__:__:__:__ desirable
Last Modified: Thursday, 06-Feb-1997 07:00:00 p.m.