Reporting Valid and Reliable Overall Scores and Domain Scores

In educational assessment, overall scores obtained by simply averaging a number of domain scores are sometimes reported. However, simply averaging the domain scores ignores the fact that different domains have different score points, that scores from those domains are related, and that at different score points the relationship between overall score and domain score may be different. To report reliable and valid overall scores and domain scores, I investigated the performance of four methods using both real and simulation data: (a) the unidimensional IRT model; (b) the higher-order IRT model, which simultaneously estimates the overall ability and domain abilities; (c) the multidimensional IRT (MIRT) model, which estimates domain abilities and uses the maximum information method to obtain the overall ability; and (d) the bifactor general model. My findings suggest that the MIRT model not only provides reliable domain scores, but also produces reliable overall scores. The overall score from the MIRT maximum information method has the smallest standard error of measurement. In addition, unlike the other models, there is no linear relationship assumed between overall score and domain scores. Recommendations for sizes of correlations between domains and the number of items needed for reporting purposes are provided.     

Interpreting NCE Scores

In recent years a new score system for standardized tests has entered the scene. The new scores are called Normal Curve Equivalents (NCEs). They were named and introduced by G. Kasten Tallmadge. NCEs are used in reporting results to federal funding agencies. Some schools may use them internally in order to deal with only one kind of score from standardized tests. Several kinds of confusion are possible with NCE scores, and the following quiz and its explanations may help you avoid some of the pitfalls. The correct answers and brief explanations appear on page 3 1. This "test" may be reproduced without written permission. Please cite Educational Measurement: Issues and Practice as the source.     

Variability in Reading Scores on a Given Level of Intelligence Test Scores

ABSTRACT

Previous studies have shown that several key variables influence student achievement in geometry, but no research has been conducted to determine how these variables interact. A model of achievement in geometry was tested on a sample of 102 high school students. Structural equation modeling was used to test hypothesized relationships among variables linked to successful problem solving in geometry. These variables, including motivation, achievement emotions, pictorial representation, and categorization skills, were examined for their influence on geometry achievement. Results indicated that the model fit well. Achievement emotions, specifically boredom and enjoyment, had a significant influence on student motivation. Student motivation influenced students’ use of pictorial representations and achievement. Pictorial representation also directly influenced achievement. Categorization skills had a significant influence on pictorial representations and student achievement. The implications of these findings for geometry instruction and for future research are discussed.     

Independence and Non-Independence of True Scores and Error Scores in Mental Tests

A comparison of animism in college males and females was made. The test instrument was the Crowell-Dole Information Scale, a self-report questionnaire of common objects. A total of 59. 8 percent of all Ss indicated animistic tendencies. Chi-square analysis of the raw data indicated no significant difference in incidents of animism for males and females. No significant difference was found between those students having one or more college biology courses and those with no formal training in biology.     

Estimating Average Domain Scores

A simulation study was performed to determine whether a group's average percent correct in a content domain could be accurately estimated for groups taking a single test form and not the entire domain of items. Six Item Response Theory based domain score estimation methods were evaluated, under conditions of few items per content area perform taken, small domains, and small group sizes. The methods used item responses to a single form taken to estimate examinee or group ability; domain scores were then computed using the ability estimates and domain item characteristics. The IRT-based domain score estimates typically showed greater accuracy and greater consistency across forms taken than observed performance on the form taken. For the smallest group size and least number of items taken, the accuracy of most IRT-based estimates was questionable; however, a procedure that operates on an estimated distribution of group ability showed promise under most conditions.     

非标准化试题的智能评分

通过对函数S-粗集和动态规划算法的研究,提出了相似度和可信度概念,给出了非标准化试题实现评分的方案和步骤,其中关键步骤是迁移处理和计算最长公共子序列长度。主要阐述了基于函数S-粗集的迁移处理,并分析了计算最长公共子序列长度解的结构和计算方法,最后分别给出了迁移函数和计算最长公共子序列长度函数的源程序。     

Interpreting Scores from Standardized Tests

This essay seeks to establish a metaphor of the professional practice of teaching to the attributes and training of an offensive lineman in the game of American football. Effective classroom instruction does not rely exclusively on a rare set of talents but rather rests on the commitment to the work of teaching. Like the position of offensive lineman, the profession of teaching is one of service. And more, it is one in which the person's performance can blossom through intense determination. An invitation is offered to serve as an effective teacher.     

高校实验课成绩考核与评定探讨

经济管理专业实验成绩考核方法的革新是深化实验教学的重要内容,传统实验成绩考核方法存在很多弊端。分析了传统的实验成绩考核方法,并且根据经济管理专业常见实验方式与实验人数的不同,提出了针对分组型与个体型实验两种不同实验类型的多元考核模式、实验成绩考核方法和评定参考指标。而后针对两种类型实验分别总结出了不同类型的实验课程成绩考核参考公式,该成绩考核方式能够更加准确与清晰的反应学生的真实成绩。最后以管理决策模拟实验课程为例进行了实验成绩考核分析。借此实验成绩的考核的思路和途径,更精确评定学生的实验成绩,激发学生实验课程的积极性,同时为实验教学改革提供参考。     

IRT Estimation of Domain Scores

In classical test theory, a test is regarded as a sample of items from a domain defined by generating rules or by content, process, and format specifications, l f the items are a random sample of the domain, then the percent-correct score on the test estimates the domain score, that is, the expected percent correct for all items in the domain. When the domain is represented by a large set of calibrated items, as in item banking applications, item response theory (IRT) provides an alternative estimator of the domain score by transformation of the IRT scale score on the test. This estimator has the advantage of not requiring the test items to be a random sample of the domain, and of having a simple standard error. We present here resampling results in real data demonstrating for uni- and multidimensional models that the IRT estimator is also a more accurate predictor of the domain score than is the classical percent-correct score. These results have implications for reporting outcomes of educational qualification testing and assessment.     

Longitudinal Consistency of Elaboration Scores

Sixty kindergarten, sixty second grade, and sixty fourth grade students performed several memory tasks under one of six conditions. The conditions differed as to the method of presentation of information. The study focused on developmental changes in children’s use of verbal, nonverbal, and spatial-positional cues for memory. The results, in general, showed consistent trends suggestive of a developmental change in representational ability, such that younger children tended to rely on visual cues and older children tended to rely on verbal cues to retain information. Children in all grades performed better when both visual and verbal cues were available and demonstrated an ability to utilize spatial-positional cues for retention.     

高校学生成绩的微机管理

为配合燕山大学目前所实施的学年学分制,采用 Foxbase~ 语言开发了一套学生成绩管理系统。经试用,获得较满意的效果。