Agreement Between Self and Other Ratings in Multi-Rater Tools: Performance, Alternative Measures, and Importance. (open access)

Agreement Between Self and Other Ratings in Multi-Rater Tools: Performance, Alternative Measures, and Importance.

Multi-rater tools also referred to as 360-degree feedback tools, are frequently used in addition to traditional supervisory appraisals due to sources (i.e., supervisor, peer, direct report) unique perspectives and opportunities to view different aspects of job performance. Research has found that the differences among sources are most prevalent between self and other ratings, and the direction of agreement is related to overall job performance. Research has typically focused on one form of agreement, the direction of an individual's self-ratings compared to others' ratings. The current study expanded on past research on rater agreement using a data set (n = 215) consisting of multi-rater data for professionals participating in a leadership development process. The study examined the ability to predict job performance with three different measures of self-other agreement (i.e., difference between overall mean scores (difference), mean absolute difference across items (difference), and mean correlation across items (similarity)). The study also examined how the relationships may differ across performance dimensions. The final purpose was to explore how the importance of the performance dimensions, as rated by the participant, may moderate the relationship between self-other agreement and job performance. Partial support for study's hypotheses was found. The direction and difference measures of …
Date: August 2008
Creator: Grahek, Myranda
System: The UNT Digital Library
The Angoff Method and Rater Analysis: Enhancing Cutoff Score Reliability and Accuracy (open access)

The Angoff Method and Rater Analysis: Enhancing Cutoff Score Reliability and Accuracy

At times called a philosophy and other times called a process, cutting score methodology is an issue routinely encountered by Industrial/Organizational (I/0) psychologists. Published literature on cutting score methodology appears much more frequently in academic settings than it does in personnel settings where the potential for lawsuits typically occurs more often. With the passage of the 1991 Civil Rights Act, it is no longer legal to use within-group scoring. It has now become necessary for personnel psychologists to develop more acceptable selection methods that fall within established guidelines. Designating cutoff scores with the Angoff method appears to suit many requirements of personnel departments. Several procedures have evolved that suggest enhancing the accuracy and reliability of the Angoff method is possible. The current experiment investigated several such procedures, and found that rater accuracy methods significantly enhance cutoff score reliability and accuracy.
Date: December 1993
Creator: Baker, Charles E., 1957-
System: The UNT Digital Library
Antecedents of Commitment to and Support of a Proposed Change Initiative in a Southern Baptist Congregation. (open access)

Antecedents of Commitment to and Support of a Proposed Change Initiative in a Southern Baptist Congregation.

This study extends research findings directed at a micro-focus of change by assessing individual organizational members' perspectives and psychological constructs influencing change efforts by an organization. The change initiative in question regards the construction of a new facility and subsequent relocation to said facility. Moral commitment to the organization (negative), change initiative's fit with organizational vision, and social influence significantly contributed to variance in members' affective commitment to change. Trust in leadership and normative commitment to the organization (NCO) significantly contributed to variance in members' normative commitment to change. Continuance commitment to the organization and participation (negative) significantly contributed to variance in members' continuance commitment to change. NCO, change initiative's fit with organizational vision, and participation significantly contributed to variance in support of the proposed change initiative. Affective commitment to the organization (negative), NCO (negative), trust in leadership (negative), and disruption of influence significantly contributed to variance in members' intent to leave the organization.
Date: August 2009
Creator: Lee, Audra
System: The UNT Digital Library
Assessing Measurement Equivalence of the English and Spanish Versions on an Employee Attitude Survey Using Multigroup Analysis in Structural Equation Modeling. (open access)

Assessing Measurement Equivalence of the English and Spanish Versions on an Employee Attitude Survey Using Multigroup Analysis in Structural Equation Modeling.

The study utilized the covariance structure comparison methodology - Multigroup Analysis in Structural Equation Modeling - evaluating measurement equivalence of English and Spanish versions of an employee opinion survey. The concept of measurement equivalence was defined as consisting of four components: sample equivalence, semantic equivalence, conceptual equivalence and scalar equivalence. The results revealed that the two language versions of the survey exhibited acceptable measurement equivalence across five survey dimensions Communications, Supervision, Leadership, Job Content & Satisfaction and Company Image & Commitment. Contrary to the study second hypothesis, there was no meaningful difference in opinion scores between English-speaking and Spanish-speaking respondents on the latent construct of Job Content & Satisfaction.
Date: August 2003
Creator: Koulikov, Mikhail
System: The UNT Digital Library
Commitment as an Indicator of Turnover in First Line Manufacturing Supervision (open access)

Commitment as an Indicator of Turnover in First Line Manufacturing Supervision

Organizational commitment is most commonly defined as a measure of an employee's commitment to the company or larger organization. In a longitudinal study, the Organizational Commitment Questionnaire was administered to 123 first line manufacturing supervisors in a defense contracting firm. After a one year check, subjects were grouped into categories of voluntary and involuntary turnover. The results suggest that significant relationships exist among the variables of departmental commitment, turnover and tenure. However, the study failed to show any relationship between organizational commitment and turnover.
Date: August 1994
Creator: Tuggle, Tamara K. (Tamara Kay)
System: The UNT Digital Library
A Comparative Analysis of Three Forms of Evaluating Management Training Programs (open access)

A Comparative Analysis of Three Forms of Evaluating Management Training Programs

The practice of training evaluation has not kept pace with prescription, and evaluations being being done are frequently negligent of appropriate controls needed to draw valid conclusions. A comparison was made of training outcomes contrasting results obtained using carefully controlled scientific approaches with those from a more popular less scientific approach. The research design involved the collection and analysis of data from a single organizations managerial training program. , Three different methods of training evaluation were studied: an "immediate reaction" rating sheet, a self-report participant survey, and a similar survey completed by t he participants' subordinates. Bo th surv ey r e sul t s showed no signif icant c hang es in on-the-job behavior six weeks after training. In contrast the "immediate reaction" ratings were positive, implying the training program was a "success." Conclusions w ere drawn concerning the validity of methods compared.
Date: December 1985
Creator: Hale, John P.
System: The UNT Digital Library
A Comparison of Training Needs in the Public and Private Sectors (open access)

A Comparison of Training Needs in the Public and Private Sectors

The training needs of managers in the public and private sectors were investigated and compared. Future trends in training that are foreseen by these managers were also researched. Forty-four public sector managers and 34 private sector managers completed a questionnaire covering such topics as: current training needs, current training efficiency, and future trends in training. Topics covered included an overview of the problem, identification and explanation of current trends in topical literature, results of the research, and conclusions drawn from the findings. The results indicated a small difference in current training needs of the two sectors. Recommendations for future studies included a larger sample population and a follow-up study of the private sector managers.
Date: August 1994
Creator: Delfeld, Lauri A.
System: The UNT Digital Library
A Concurrent Validation Study of a Paper and Pencil Test Battery for a Sales Position (open access)

A Concurrent Validation Study of a Paper and Pencil Test Battery for a Sales Position

Participating in this study were 251 decorator consultants. The decorator consultant position is a direct sales position. The primary objective of this study was to demonstrate that a relationship existed between decorators' selection test scores and their job performance. The SRA Verbal Form, the EAS Numerical Ability Test, the EAS Space Visualization Test, and the Sales Attitude Checklist were evaluated as potential selection tests. Behavioral criteria and managerial ratings were used to assess job performance. Correlational analyses revealed that all the tests but the SRA Verbal Form were significantly correlated with two or more criteria.
Date: May 1990
Creator: Irons, Deedra Kim
System: The UNT Digital Library
Coping with Job Loss and Career Stress: Effectiveness of Stress Management Training with Outplaced Employees (open access)

Coping with Job Loss and Career Stress: Effectiveness of Stress Management Training with Outplaced Employees

A stress management intervention was developed and integrated into the services provided by one of the nation's largest outplacement consultants. The purpose of the intervention was to aid outplacement clients in the management and resolution of stress associated with job loss and career transition. Based on the results of this study, the intervention had the effect of helping treatment participants maintain their levels of effective coping when compared to nonparticipants. This study supports the hypothesis that stress management training can be helpful for unemployed individuals and may impact their emotional well-being as they go through the job search process.
Date: August 1995
Creator: Maysent, Anne Miya
System: The UNT Digital Library
Correlates of a Past Behavior Interview for the Business Unit Leader: Experience, Motivation, Personality and Cognitive Ability (open access)

Correlates of a Past Behavior Interview for the Business Unit Leader: Experience, Motivation, Personality and Cognitive Ability

This research evaluates the relationship between various individual differences constructs and performance on a past behavior interview (PBI)-one of the most popular forms of personnel selection interviews used today-within a sample of business unit leader level incumbents and applicants from organizations across the United States. Correlation analysis is conducted on the relationship between overall performance on a PBI and four work-related constructs: Experience, Motivation, Personality, and Cognitive Ability. The existing literature on PBIs and the four independent variables is critically reviewed. As limited research has been conducted on the influence of Experience and Motivation on PBI performance, this study makes unique contributions to the literature regarding impact of these two constructs. The major hypotheses stated that Experience and Motivation would yield significant, positive correlations with PBI performance while Personality and Cognitive Ability would not be significantly correlated with PBIs. Results partially supported the hypotheses-Experience, Motivation, and Personality were significantly related to overall PBI score, while Cognitive Ability was not. Implications for the findings as well as suggestions for future research are discussed.
Date: August 2008
Creator: Conner, Lane A.
System: The UNT Digital Library
Cultural Implications of Self-Other Agreement in Multisource Feedback: Comparing Samples from US, China, and Globally Dispersed Teams. (open access)

Cultural Implications of Self-Other Agreement in Multisource Feedback: Comparing Samples from US, China, and Globally Dispersed Teams.

Application of multisource feedback (MSF) increased dramatically and became widespread globally in the past two decades, but there was little conceptual work regarding self-other agreement and few empirical studies investigated self-other agreement in other cultural settings. This study developed a new conceptual framework of self-other agreement and used three samples to illustrate how national culture affected self-other agreement. These three samples included 428 participants from China, 818 participants from the US, and 871 participants from globally dispersed teams (GDTs). An EQS procedure and a polynomial regression procedure were used to examine whether the covariance matrices were equal across samples and whether the relationships between self-other agreement and performance would be different across cultures, respectively. The results indicated MSF could be applied to China and GDTs, but the pattern of relationships between self-other agreement and performance was different across samples, suggesting that the results found in the U.S. sample were the exception rather than rule. Demographics also affected self-other agreement disparately across perspectives and cultures, indicating self-concept was susceptible to cultural influences. The proposed framework only received partial support but showed great promise to guide future studies. This study contributed to the literature by: (a) developing a new framework of self-other …
Date: August 2007
Creator: Lin, Yue
System: The UNT Digital Library
Design and Empirical Analysis of a Model of Empowering Leadership. (open access)

Design and Empirical Analysis of a Model of Empowering Leadership.

Mid-level leaders are often expected to implement employee empowerment initiatives, yet many do not have a clear understanding of how to empower employees. To address this issue, a model of empowering leadership was developed. The model presents specific, actionable behaviors that a leader should perform in order to empower employees. The model comprises 13 factors built around the areas of ability, accountability, and authority. First, leaders must ensure employees have the ability to be empowered. To do so, they must (a) build employee organizational knowledge, (b) provide access to pertinent information, (c) assure employees have the necessary skill set, and (d) identify and provide needed resources. Second, leaders must create systems of accountability for employee outcomes by (e) setting a standard of continuous improvement, (f) recognizing and rewarding good work, (g) regularly evaluating employee efforts, and (h) providing continuous feedback on employee efforts. Third, leaders should provide employees with the authority to be empowered by (i) serving as advocates of employee efforts, (j) providing an environment that is conducive to empowerment, (k) setting a clear and consistent direction to guide employee efforts, and (l) building systems and structures to support employee empowerment. The thirteenth factor of the model is a …
Date: May 2005
Creator: Bodner, Sarah L.
System: The UNT Digital Library
Development and Validation of a Two Factor Model of Adult Career Orientation (open access)

Development and Validation of a Two Factor Model of Adult Career Orientation

Subjects in this study were 5,523 respondents from a survey which was sent to households throughout the United States. The purpose of this study was to examine two basic components of career orientation: career indecision and career insight. Correlational analyses found relationships between career indecision and average job tenure, industry leaving intentions, industry staying intentions, and job satisfaction. Correlational analyses found relationships between career insight, industry staying intentions and job satisfaction. Multiple regression analyses were run using both career indecision and career insight as independent variables found that they had useful levels of incremental validity in predicting industry leaving intentions and job satisfaction. Potential uses of the two-dimensional career indecision - career insight model are discussed.
Date: August 1998
Creator: Toofanian, Maryam
System: The UNT Digital Library
Development of a Model of Leadership for Self-Managed Teams in a Greenfield Environment (open access)

Development of a Model of Leadership for Self-Managed Teams in a Greenfield Environment

This study identified and defined leader behaviors with two levels of leadership in a self-managed team organization. Job analysis methodology was used. A comparison of task importance values was made within groups and between hierarchical levels in the organization. Identified leader behaviors were compared with effective, traditional leader performance. Qualitative data collected throughout the investigation clarified an integrative model for effective organizations developed from the literature. The model included leader characteristics and team member behaviors when using self-managed teams.
Date: May 1992
Creator: Burress, Mary Ann
System: The UNT Digital Library
Development of Biographical Predictors of Cashier Turnover at a Convenience Store Chain (open access)

Development of Biographical Predictors of Cashier Turnover at a Convenience Store Chain

Subjects, 432 convenience store cashiers, were divided into long-tenure and short-tenure groups. Chi-square analysis of application blank information for a weighting sample drawn from both groups revealed two items which significantly (p < .05) differentiated between the long tenure and short-tenure groups: number of previous jobs and full-time/part-time preference. Response weights were computed for these two items and used to calculate composite scores for the remaining holdout sample. A significant reduction in turnover would have occurred at the highest composite score level, if used as a hiring cut off. Results were tempered by several considerations, including a high percentage of false negatives and an insignificant linear relationship between composite scores and tenure.
Date: May 1989
Creator: Huffcutt, Allen Ivan
System: The UNT Digital Library
Dimensional Assessment of Empowerment in Organizations (open access)

Dimensional Assessment of Empowerment in Organizations

This research project was inspired by a survey that was designed to help an organization determine how well it was doing in its efforts to empower associates and achieve a goal of continuous improvement. Initial review of the survey created suspicion that the survey was not built around the appropriate dimensions to accurately measure the level of empowerment in organizations. As such, the survey was psychometrically analyzed to determine the validity of the instrument as a measure of empowerment. Additionally, an extensive review of the literature was performed to determine new dimensions that would most accurately measure empowerment. Eight dimensions (culture, trust, accountability, leadership, ability, commitment, responsibility, and communication) were put forth as the most appropriate to measure empowerment. Subject matter experts with knowledge and experience in the area of organizational empowerment reviewed the new dimensions for accuracy and fit with the original survey items.
Date: May 2003
Creator: Bodner, Sarah L.
System: The UNT Digital Library
The Effect of Presence of Support Systems and Level of Agreement on the Performance of Work Groups (open access)

The Effect of Presence of Support Systems and Level of Agreement on the Performance of Work Groups

In the study of team-based organizations most of the research has focus on the internal make-up and structure of teams. Recently there has been more interest in the effects that environment has on teams. With this new focus Support Systems in organizations have become an area of interest. Examining the perceptions of workers with respect to support systems of organizations could give insight into performance. This study specifically examines the interaction between a team's shared perception of the support in their environment and the level of support in their environment. The interaction between the two does seem to have a strong relationship with perceived performance. How do the two concepts interact, and what does this mean for organizational designers? Both questions are discussed.
Date: May 1999
Creator: Adcock, John R. (John Roger)
System: The UNT Digital Library
The Effect of Type A and Type B Personality and Leadership Style on Absenteeism (open access)

The Effect of Type A and Type B Personality and Leadership Style on Absenteeism

This study explored the relationship of Type A/B personality and leadership style to absenteeism. Absenteeism data were gathered for 243 male fire fighters and fire engineers. Each subject was administered the Jenkins Activity Scale to measure his Type A characteristics and the Leader Behavior Description Questionnaire to measure his perception of his supervisor's leadership style. The results, though non-significant, revealed that: a) Type A's had less absenteeism than type B's; b) Subjects who perceived their supervisors as being low on consideration had less absenteeism than those who perceived their supervisors as being high on this dimension; c) Type A's absenteeism was low and Type B's was high when working under a leader perceived as low on structure. Finally, a weak but significant three-way interaction effect revealed that the highest amount of absenteeism occurred when Type B' s worked under supervisors who were high in consideration and low in structure. The least amount of absenteeism occurred when Type A's worked under supervisors who were high in structure and low in consideration.
Date: August 1987
Creator: Nichols, Judith Ann, 1957-
System: The UNT Digital Library
Effects of Implementing a Competency-Based Performance Management System on Measures of Sales Performance (open access)

Effects of Implementing a Competency-Based Performance Management System on Measures of Sales Performance

Use of competency models has exploded in recent years. Unfortunately, the empirical research to validate such systems is scarce. This study explores the relationship between Competency-based Performance Management Systems and sales performance to determine whether the use of these systems increases performance. Performance data from sales representatives in a medical products company were examined to determine changes in performance following the introduction of the Competency-based Performance Management System (N=64). Correlations with performance were obtained for each competency dimension to determine if any factors were highly correlated with performance and if state-factors were more positively correlated with performance than trait-factors (N=66). The study found no significant relationship between implementation of a Competency-based Performance Management System and sales performance. Also state-factors were not more positively correlated with sales performance than trait-factors.
Date: May 2000
Creator: Lynch, Ronald
System: The UNT Digital Library
Effects of Performance Levels of Subject Matter Experts on Job Analysis Outcomes (open access)

Effects of Performance Levels of Subject Matter Experts on Job Analysis Outcomes

Much research has been undertaken to determine how Subject Matter Expert characteristics affect job analysis outcomes. The current study seeks to discover if performance levels are related to current incumbents ratings of their positions. A group of 114 corporate associates, from two administrative positions, served as Subject Matter Experts (SME) for this study. Separate job analyses for each position were conducted using the Job Analysis Task Checklist. The results for each job were analyzed to determine if SME performance levels affected job analysis outcomes. The results for both jobs showed that there were very few differences in job analysis results as a function of SME performance levels.
Date: December 1997
Creator: Boyd, Charlotte Friedersdorff
System: The UNT Digital Library
The Effects of Realistic Job Previews on Turnover in a Financial Services Organization (open access)

The Effects of Realistic Job Previews on Turnover in a Financial Services Organization

Realistic Job Previews have been shown to impact newcomers to jobs through ircreased self-selection, reduced turnover, eased adjustment, improved performance and increased job satisfaction. To address a turnover problem, Realistic Job Previews were implemented in hiring for two entry level positions in half of 539 branch offices of a large financial services organization. Subjects consisted of 122 Service Representatives and 98 Financial Representatives. Eight months after implementation, turnover rates were compared for control and experimental groups. There was no significant difference between turnover among Service Representatives. Financial Representatives in the experimental group had lower turnover rates (p < .10), with the difference increasing over time. Comparing the turnover rates between three and six months tenure resulted in a statistically significant difference (p < .05).
Date: August 1987
Creator: Goerz, Marilyn J.
System: The UNT Digital Library
An Empirical Investigation of Criterion Development and the Multiple Criteria Versus Composite Criterion Debate (open access)

An Empirical Investigation of Criterion Development and the Multiple Criteria Versus Composite Criterion Debate

The purpose of this study was to empirically examine two empirically examine two main areas of concern in selecting criteria for validation studies: the development of the criterion and the multiple criteria versus composite criterion debate. Evidence was found for the ability of the various weighting schemes used to generate composites that were statistically and conceptually different from one another. Knowledge of the nature of each composite, along with the multiple criteria approach, proved essential to understanding the composite criterion to the validation process. Selection and treatment of the criterion apparently consist of judgment and individual estimations.
Date: August 1990
Creator: Dailey, Patrick A. (Patrick August)
System: The UNT Digital Library
Employee Engagement: The Impact of Spiritual, Mental, Emotional and Physical Elements on the Relationship Between Employee Engagement and Behavioral Outcomes (open access)

Employee Engagement: The Impact of Spiritual, Mental, Emotional and Physical Elements on the Relationship Between Employee Engagement and Behavioral Outcomes

Data were collected by an external company for a healthcare service firm interested in learning the job-related attitudes of their employees. Thus, archival data from 1,287 employees were collected for a different purpose. The survey consisted of 117 questions covering a broad range of constructs. Consequently, the items were used to derive effective measures of employee engagement and behavioral outcomes, as well as the emotional, mental, spiritual and physical dimensions. Exploratory and confirmatory factor analysis procedures were used to create scales reflecting these four factors. Interestingly, six scales emerged, logically linking to and further specifying the initial dimensions. These were Organizational Linkage, Manager Relationship, Job Fit, Job Clarity, Work Pressure, and Meaningful Work. To test the hypotheses, six separate regression equations were calculated, which statistically supported modification by each of the dimensions. However, statistical significance of the interactions resulted from having a large sample, given the actual association was too small to be meaningful (e.g., a contribution of 0.6% of the variance). As each of the dimensions had a main effect on the behavioral measure during hypothesis testing, exploratory regression equations were calculated to further understand the interrelationships. Of most interest was finding that in the presence of Engagement, the …
Date: August 2014
Creator: Zobal, Cheryl
System: The UNT Digital Library
An Employee Participation Change Project and Its Impact on the Organization: a Case Study (open access)

An Employee Participation Change Project and Its Impact on the Organization: a Case Study

The purpose of this study was to document and assess the consequences of implementing employee involvement in a manufacturing setting. Using a quasi-experimental design, the study utilized information from various sources of data including archives, interview, and questionnaire data for a three to four year period. Time series comparisons were used. The results indicated that production increased initially, but then dropped back to original level. Quality of products increased and continued to improve gradually. The highest rate of improvement was observed in safety. An attempt was made to measure current level of commitment at the plant but was unsuccessful due to a low return rate of questionnaires. Overall, data collected partially support the hypotheses. Implications for further research and practice are discussed.
Date: August 1990
Creator: Roustaei, Simin
System: The UNT Digital Library