Evaluating dynamic performance: The influence of salient Gestalt characteristics on performance ratings.

It is well recognized that performance changes over time. However, the effect of these changes on overall assessments of performance is largely unknown. In a laboratory experiment, we examined the influence of salient Gestalt characteristics of a dynamic performance profile on supervisory ratings. We manipulated performance trend (flat, linear-improving, linear-deteriorating, U-shaped, and ∩-shaped), performance variation (small, large), and performance mean (negative, zero, positive) within subjects and display format (graphic, tabular) between subjects. Participants received and evaluated information about the weekly performance of different employees over a simulated 26-week period. Results showed strong main effects on performance ratings of both performance mean and performance trend, as well as interactions with display format. Theoretical and practical implications of the results are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sociometric ratings as predictors of academic performance.

An experiment to determine the degree of accuracy with which sociometric ratings can be used to predict academic performance. A 14-item sociometric questionnaire was administered to 86 students before any academic grades were available. 2 rating measures, one emphasizing future job performance and the other dealing with personal adjustment, were found to be significantly related to academic performance (r = .40, p     

The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: Unconfounding construct-level convergence and rating difficulty.

Meta-analysis was used to cumulate the correlations between supervisor and peer ratings for different dimensions of job performance. The general pattern in the data suggested that raters from the same organizational level disagree as much as raters from different levels. Methods were used to separate the effects of rating difficulty and lack of construct-level convergence on the correlation between supervisor and peer ratings. The authors found complete construct-level convergence for ratings of overall job performance, productivity, effort, job knowledge, quality, and leadership but not for ratings of administrative competence, interpersonal competence, and compliance or acceptance of authority. Higher rating difficulty was more strongly associated with lower mean observed peer-supervisor correlations than were construct-level disagreements between peers and supervisors. Implications for research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The roles of rater goals and ratee performance levels in the distortion of performance ratings.

The goal-directed perspective of performance appraisal suggests that raters with different goals will give different ratings. Considering the performance level as an important contextual factor, we conducted 2 studies in a peer rating context and in a nonpeer rating context and found that raters do use different rating tactics to achieve specific goals. Raters inflated their peer ratings under the harmony, fairness, and motivating goal conditions (Study 1, N = 103). More important, raters inflated their ratings more for low performers than for high and medium performers. In a nonpeer rating context, raters deflated ratings for high performers to achieve the fairness goal, and they inflated ratings for low performers to motivate them (Study 2, N = 120). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Does a structured free recall intervention reduce the effect of stereotypes on performance ratings and by what cognitive mechanism?

The purpose of this article was to extend previous work on the effect of racial biases on performance ratings. The 1st of 2 studies examined whether a structured free recall intervention decreased the influence of negative racial biases on the performance ratings of Black men. Results indicated that without the intervention, raters who endorsed a negative stereotype of Black men as managers evaluated Black men more negatively. However, the structured free recall intervention successfully reduced these effects. The second study examined in more detail the cognitive mechanisms underlying the success of the intervention. Results are consistent with the assumption that the reduction of the influence of racial biases under structured free recall conditions is a consequence of a modified strength threshold for retrieval of behaviors from memory. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach.

A review of research on job performance suggests 3 broad components: task, citizenship, and counterproductive performance. This study examined the relative importance of each component to ratings of overall performance by using an experimental policy-capturing design. Managers in 5 jobs read hypothetical profiles describing employees' task, citizenship, and counterproductive performance and provided global ratings of performance. Within-subjects regression analyses indicated that the weights given to the 3 performance components varied across raters. Hierarchical cluster analyses indicated that raters' policies could be grouped into 3 homogeneous clusters: (a) task performance weighted highest, (b) counterproductive performance weighted highest, and (c) equal and large weights given to task and counterproductive performance. Hierarchical linear modeling indicated that demographic variables were not related to raters' weights. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influence of performance evaluation rating segmentation on motivation and fairness perceptions.

This study investigated the impact of rating segmentation (i.e., the number of alternative appraisal categories available for rating employee performance) on motivation and perceptions of fairness. Participants were 305 student volunteers who performed a catalog search task, A 2 X 4 factorial design with 2 performance rating systems--low segmentation (3 categories) and moderate segmentation (5 categories)--and 4 performance levels was used. Overall, the results indicated that the 5-category system resulted in higher self-efficacy regarding participants' ability to reach the next higher rating category, higher goals for rating improvement, and higher rating improvements than the 3-category system. The effects of rating system and performance rating on rating improvement were partially mediated by self-efficacy and personal goals. The rating system and the performance rating affected perceptions of distributive and procedural justice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Presentation modality and indirect performance information: Effects on ratings, reactions, and memory.

The authors investigated whether raters integrate indirect (second-hand) information from an employee's co-worker with their direct observations when completing performance evaluations. Performance levels of direct and indirect information and presentation modality (auditory vs. textual) were manipulated (N=220). Results showed that indirect information was perceived to be of highest utility when the performance levels of the direct and indirect information were consistent. Confidence in performance ratings was lowest when the indirect source delivered negative performance feedback that was contrary to the rater's own positive observations. Indirect information was only reflected in the performance ratings when direct observations were positive. There was a significant 3-way interaction between performance level of the direct information, performance level of the indirect information, and presentation modality on memory for performance incidents. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Adaptability measures as predictors of performance ratings.

How well do measures of temperamental and motivational characteristics in training predict success of performance of officers as indicated by Officer Effectiveness Ratings? S were selected from 666 cadets at one base and 69 at another. Variables used were 3 global indicies of adaptability and 10 variables representing the common factors. "(a) measures of adaptability to training and Air Force Life are more highly related to later officer performance than are measures of aptitude or ability; (b) assessments of a man's functioning involving personal judgments of peers, superiors, and experts are predictive of later performance as an Air Force officer." (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Effects of rater goals on rating patterns: Evidence from an experimental field study.

The goal-based perspective of performance appraisals suggests that raters who pursue different goals give different performance ratings. Yet previous studies have not provided strong empirical evidence that there are different impacts of different goals on mean rating and discriminability, nor have they provided evidence of a goal-rating causality. The authors extend the literature by manipulating rater goals in the context of peer evaluations of graded group projects with a sample of 104 undergraduate students. They find that (a) pursuing a harmony goal increased mean rating and decreased discriminability, and (b) pursuing a fairness goal increased mean rating and decreased discriminability when the group projects had not ended and increased mean rating but did not change discriminability when the group projects had ended. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The interactive effect of leader-member exchange and communication frequency on performance ratings.

The authors tested the hypothesis that communication frequency moderates the relationship between leader-member exchange (LMX) and job-performance ratings. In a study of 188 private sector workers, they found that LMX was more strongly related to job-performance ratings among individuals reporting frequent communication with the supervisor than among those reporting infrequent communication. At high levels of LMX, workers reporting frequent communication with the supervisor received more favorable job-performance ratings than did workers reporting infrequent communication. In contrast, at low levels of LMX, workers reporting frequent communication with the supervisor received less favorable job-performance ratings than workers reporting infrequent communication. The authors conducted a 2nd study of 153 public sector workers to provide a constructive replication and found similar results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The five-factor model of personality and managerial performance: Validity gains through the use of 360 degree performance ratings.

This study investigated the usefulness of the five-factor model (FFM) of personality in predicting two aspects of managerial performance (task vs. contextual) assessed by utilizing the 360 degree performance rating system. The authors speculated that one reason for the low validity of the FFM might be the failure of single-source (e.g., supervisor) ratings to comprehensively capture the construct of managerial performance. The operational validity of personality was found to increase substantially (50%–74%) across all of the FFM personality traits when both peer and subordinate ratings were added to supervisor ratings according to the multitrait–multimethod approach. Furthermore, the authors responded to the recent calls to validate tests via a multivariate (e.g., multitrait–multimethod) approach by decomposing overall managerial performance into task and contextual performance criteria and by using multiple rating perspectives (sources). Overall, this study contributes to the evidence that personality may be even more useful in predicting managerial performance if the performance criteria are less deficient. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Accuracy of performance ratings as affected by rater training and perceived purpose of rating.

Investigated the effects of perceived purpose for rating and training type on the following dependent variables: accuracy, leniency/severity, and illusory halo. The purpose factor comprised 3 levels: a hiring purpose, a feedback purpose, and a research-only purpose. The training factor comprised 4 levels: rater error (RE) training, frame-of-reference (FOR) training, the combination of both methods, and no training. With both factors crossed, 164 undergraduates were randomly assigned to 1 of 12 conditions and viewed videotapes of lectures given by bogus graduate assistants. Heterogeneity of variance made it necessary to apply a conservative analytical strategy. Training significantly affected 2 measures of accuracy and halo such that a training condition that contained an FOR component did better than RE or no training. The conservativeness of the conservative analytic strategy made effects for the purpose factor on correlation accuracy, leniency/severity, and halo only tentative; it dissipated the 1 interaction effect of the 2 factors on distance accuracy. Discussion centers on (a) comparison of the results with those of S. Zedeck and W. Cascio (see record 1983-09102-001), (b) potential reasons for the heteroscedasticity, and (c) implications for the development of student evaluations of university instructors. (32 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Predicting ratings of sales success with objective performance information.

What information about salesmen best predicts success in sales work as indicated by sales managers' appraisals of sales performance? To find out, 40 salesmen were studied for 6 months. "In all, 21 objective variables were intercorrelated with 19 appraisal variables… . the best predictors of appraisal factors: Spot Orders, Spot Orders vs. Par, Shop Orders/Shop Calls, New Business Orders, and New Business Orders vs. Par." The procedures were considered very promising for trial by others. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluating frame-of-reference rater training effectiveness using performance schema accuracy.

Frame-of-reference training has been shown to be an effective intervention for improving the accuracy of performance ratings (e.g., Woehr & Huffcutt, 1994). Despite evidence in support of the effectiveness of frame-of-reference training, few studies have empirically addressed the ultimate goal of such training, which is to teach raters to share a common conceptualization of performance (Athey & McIntyre, 1987; Woehr, 1994). The present study tested the hypothesis that, following training, frame-of-reference–trained raters would possess schemas of performance that are more similar to a referent schema, as compared with control-trained raters. Schema accuracy was also hypothesized to be positively related to rating accuracy. Results supported these hypotheses. Implications for frame-of-reference training research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of feedback, casual attributions, and self-efficacy in goal revision.

An important issue in work motivation is how, when, and why individuals revise their goals up or down over time. In the current study, the authors examine feedback, causal attributions, and self-efficacy in this process. Although self-efficacy has frequently been suggested as a key explanatory variable for goal revision, its role has yet to be directly evaluated. Additionally, although attributions have been shown to influence goal revision following failure, the extent to which attributions influence goal revision following success remains unclear. In the current study, the authors address these issues by experimentally manipulating goal progress via performance feedback and tracking the resulting changes in self-efficacy and goal revision over time. In so doing, the authors also address several interpretive ambiguities present in the existing research. Results support the hypothesized model, finding that performance feedback and attributions interactively influenced self-efficacy, which in turn influenced goal revision. These results suggest that interventions targeting attributions, and self-efficacy more directly, may have meaningful influences on goal setting and pursuit, particularly following feedback. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Improving peer supervisor ratings of therapist performance in dialectical behavior therapy: An Internet-based training system.

The present study involved the development of an Internet-based training system (ITS) to help train peer supervisors. The system, which was piloted with Dialectical Behavior Therapy (Linehan, 1993) using mock sessions, demonstrates how Internet-based technology can facilitate training protocols to support the ongoing training and supervision of therapists efficiently. Participant evaluation of the system was very positive, with over 90% of respondents reporting that they believed that the ITS was very useful or extremely useful for therapist training. Possible uses of the system include: (a) helping therapists to learn to discriminate more effective from less effective interventions to provide better feedback to supervisees and peers on their sessions, (b) helping therapists to improve their own ability to monitor and deliver a treatment effectively, and (c) helping to structure therapist training and supervision activities. The system could also be used to help with real-world supervision of therapists, assuming that legal and ethical issues associated with the use of the Internet for clinical supervision are addressed adequately. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Concordance of self- and informant ratings of adults' current and childhood attention-deficit/hyperactivity disorder symptoms.

Self-informant rating concordance for attention-deficit/hyperactivity disorder (ADHD) symptoms was assessed in 281 adults at the subscale (Inattention, Hyperactivity-Impulsivity) and individual symptom levels. Potential demographic, diagnostic, and informant identity moderators were also investigated. Concordance levels were similar for current and childhood symptoms. Although moderate positive correlations were found between self- and informant ratings on both subscales, informants endorsed more significant inattentive symptom severity. Kappa coefficients were variable, suggesting low concordance for certain symptoms. Sex and ADHD diagnosis moderated concordance, although effect sizes were small. These results have implications for the use of behavior rating scales in diagnosing ADHD, raise questions about the validity of self- and informant ratings, and support the need to investigate individual differences variables that may impact concordance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)