非标准化试题的智能评分

通过对函数S-粗集和动态规划算法的研究,提出了相似度和可信度概念,给出了非标准化试题实现评分的方案和步骤,其中关键步骤是迁移处理和计算最长公共子序列长度。主要阐述了基于函数S-粗集的迁移处理,并分析了计算最长公共子序列长度解的结构和计算方法,最后分别给出了迁移函数和计算最长公共子序列长度函数的源程序。     

Variability in Reading Scores on a Given Level of Intelligence Test Scores

ABSTRACT

Previous studies have shown that several key variables influence student achievement in geometry, but no research has been conducted to determine how these variables interact. A model of achievement in geometry was tested on a sample of 102 high school students. Structural equation modeling was used to test hypothesized relationships among variables linked to successful problem solving in geometry. These variables, including motivation, achievement emotions, pictorial representation, and categorization skills, were examined for their influence on geometry achievement. Results indicated that the model fit well. Achievement emotions, specifically boredom and enjoyment, had a significant influence on student motivation. Student motivation influenced students’ use of pictorial representations and achievement. Pictorial representation also directly influenced achievement. Categorization skills had a significant influence on pictorial representations and student achievement. The implications of these findings for geometry instruction and for future research are discussed.     

Uses and Abuses of Achievement Test Scores

Are variations in test-preparation practices from school to school undermining the meaningfulness of achievement test results? Is there pressure to raise achievement test scores by the use of educationally unsound practices? What uses of achievement test scores are most common? Do teachers and administrators have reasonably accurate views of test score uses?     

Schooling and the Norming of Intelligence Test Scores

How does schooling affect the development of intelligence in children? How should the amount of schooling be considered when developing norms for turning intelligence test performance into IQ scores?     

Regression to the Mean in Average Test Scores

A group's average test score is often used to evaluate different educational approaches, curricula, teachers, and schools. Studies of group test scores over time often try to measure "value-added" by holding constant certain student characteristics such as race, parents' education, or socioeconomic status; however, the important statistical phenomenon of regression to the mean is often ignored. There is a substantial literature on the importance of regression to the mean in a variety of contexts, including individual test scores. Here, we look at regression to the mean in group averages. If this regression is not taken into account, changes in a group's average test score over time may be misinterpreted as changes in the group's average ability rather than natural and expected fluctuations in scores about ability. California Academic Performance Index scores are used to illustrate this argument.     

考试分数的强化与评价视域的窄化——有关考试与评价问题的几点辨析

在应试教育的背景下,考试分数的作用被无限夸大。考试分数的强化窄化了评价视域,简化了课程目标,进而异化了基础教育。异化的教育又将考试分数的虚高价值进一步推向极致,最终形成教育的怪圈。本文在对考试分数的不当使用进行案例分析的基础上,提出正确理解与把握评价的几对关系,以期对走出教育的怪圈有所启示。     

What Happened to Test Scores,and Why?

What has happened in recent decades to the test scores of American students? What can be said about the causes of trends in scores? Do these trends provide an indictment of the effectiveness of American schools? How should test scores be used to inform the debate about education policy?     

Grades and Test Scores: Accounting for Observed Differences

Why do grades and test scores often differ? A framework of possible differences is proposed in this article. An approximation of the framework was tested with data on 8,454 high school seniors from the National Education Longitudinal Study. Individual and group differences in grade versus test performance were substantially reduced by focusing the two measures on similar academic subjects, correcting for grading variations and unreliability, and adding teacher ratings and other information about students. Concurrent prediction of high school average was thus increased from 0.62 to 0.90; differential prediction in eight subgroups was reduced to 0.02 letter‐grades. Grading variation was a major source of discrepancy between grades and test scores. Other major sources were teacher ratings and Scholastic Engagement, a promising organizing principle for understanding student achievement. Engagement was defined by three types of observable behavior: employing school skills, demonstrating initiative, and avoiding competing activities. While groups varied in average achievement, group performance was generally similar on grades and tests. Major factors in achievement were similarly constituted and similarly related from group to group. Differences between grades and tests give these measures complementary strengths in high‐stakes assessment. If artifactual differences between the two measures are not corrected, common statistical estimates of validity and fairness are unduly conservative.     

Maintaining Equivalent Cut Scores for Small Sample Test Forms

This study examines the effectiveness of three approaches for maintaining equivalent performance standards across test forms with small samples: (1) common‐item equating, (2) resetting the standard, and (3) rescaling the standard. Rescaling the standard (i.e., applying common‐item equating methodology to standard setting ratings to account for systematic differences between standard setting panels) has received almost no attention in the literature. Identity equating was also examined to provide context. Data from a standard setting form of a large national certification test (N examinees = 4,397; N panelists = 13) were split into content‐equivalent subforms with common items, and resampling methodology was used to investigate the error introduced by each approach. Common‐item equating (circle‐arc and nominal weights mean) was evaluated at samples of size 10, 25, 50, and 100. The standard setting approaches (resetting and rescaling the standard) were evaluated by resampling (N = 8) and by simulating panelists (N = 8, 13, and 20). Results were inconclusive regarding the relative effectiveness of resetting and rescaling the standard. Small‐sample equating, however, consistently produced new form cut scores that were less biased and less prone to random error than new form cut scores based on resetting or rescaling the standard.     

以考试成绩分析为案例诊断与改进教学质量

考试是教学质量监控中十分重要的环节,是实现学校教学评价和教育教学目标的一种重要手段。考试不仅是对学生学习效果的检测,也是对教师教学质量的检查,还是对学校管理的监控,通过对学生考试成绩的统计分析,可以帮助教师及时发现教学中存在的问题及薄弱环节,及时调整教学内容,改进教学方法;可以对教学管理制度、教学运行体制进行有效得当的检验、监控和纠偏的作用。     

The Differential Impact of Curriculum on Aptitude Test Scores

A sample of 22, 923 students who had taken the SAT and the GRE General Test was classified by the four general undergraduate fields of study and by sex. The authors performed several analyses to determine the degree of differential impact that sex and field of study might have on GRE-Verbal, GRE-Quantitative, and GRE-Analytical scores after controlling on SAT-Verbal and SAT-Mathematical scores. They found, first, that the correlations of SAT-Verbal with GRE-Verbal scores and SAT-Mathematical with GRE-Quantitative scores were extremely high, .86 in the total sample and ranging from the low to middle .80s in the eight subgroups. The impact of curriculum and sex, after controlling on SAT scores, was found to be low on GRE- Verbal scores but relatively high on GRE-Quantitative scores, with students in heavily quantitative fields enjoying an advantage over their peers in less quantitative fields of study. The impact was moderate on GRE-Analytical scores. Further studies designed to "purify" the fields of study and include only clearly verbal fields and clearly mathematical fields showed small additional impact. An additional study indicated a generally slight effect of the institution attended on GRE-Quantitative scores after controlling for sex, major field of study, and initial ability.     

Combining Multiple-Choice and Constructed-Response Test Scores: Toward a Marxist Theory of Test Construction

Assessment instruments of the future will probably be composed of a combination of different types of questions. Even though different kinds of questions require different scoring procedures, there may be a need to have those different scores combined as a composite. In this article, we describe how mixtures of such scores may be efficaciously combined. Also, if no post hoc adjustment is desired, we provide two characterizations of measurement effectiveness to aid in making unadjusted score combinations efficient. In addition, we explore the implications for test construction of some typical findings.     

关于测验成绩正态分布与偏态分布的思考

教育评价过程中,成绩的分布形态是由测验的目的、功能和评定标准,学生的能力素质和知识基础,教师和学生的积极能动性,学生群体规模的大小及学生的学习时间和效率等因素决定的。必须客观、全面地评价正态分布与偏态分布的利弊,在此基础上对两者加以有效整合,以提高测验的真实性和实用性,促成全体师生的共同努力,促进全体学生的共同发展。     

Using Kernel Equating to Assess Item Order Effects on Test Scores

This study explored the use of kernel equating for integrating and extending two procedures proposed for assessing item order effects in test forms that have been administered to randomly equivalent groups. When these procedures are used together, they can provide complementary information about the extent to which item order effects impact test scores, in overall score distributions and also at specific test scores. In addition to detecting item order effects, the integrated procedures also suggest the equating function that most adequately adjusts the scores to mitigate the effects. To demonstrate, the statistical equivalences of alternate versions of two large-volume advanced placement exams were assessed.     

The Impact of Exam Environments on Student Test Scores in Online Courses

While enrollment in online courses has tripled in ten years, little is known about the impact of different exam-taking environments on learning. A recent study of economics students found that online students taking un-proctored exams scored one letter grade higher than online students taking proctored exams. However, there were no apparent systematic safeguards against cheating in the un-proctored section. This study adds to the literature by comparing student’s performance in online classes with proctored exams to the performance of online students in a carefully controlled online testing environment (Respondus Lockdown Browser [RLB]). Methods: Data refer to 287 students enrolled in a criminology course at a Carnegie research-extensive university. The experimental group consisted of students in online sections who were called back to campus to take exams in a proctored environment. The control group took exams off campus using RLB. All sections were taught by the same instructor. The dependent variable is the score on the standardized final examination. The central independent variable is the type of exam environment: proctored vs. the RLB environment. Results: Controlling for other constructs, there was no significant difference between student exam scores in the proctored sections and the sections employing RLB. Conclusion: The results suggest that the judicious use of RLB can level the playing field between exam scores in proctored vs. nonproctored online sections. Technology such as RLB, which minimizes opportunities for cheating online, may provide tools for fairness in grading. Future work is needed for other courses, other fields, and other types of academic institutions.