Optimizing Worked‐Example Instruction in Electrical Engineering: The Role of Fading and Feedback during Problem‐Solving Practice

How can we help college students develop problem‐solving skills in engineering? To answer this question, we asked a group of engineering freshmen to learn about electrical circuit analysis with an instructional program that presented different problem‐solving practice and feedback methods. Three findings are of interest. First, students who practiced by solving all problem steps and those who practiced by solving a gradually increasing number of steps starting with the first step first (forward‐fading practice) produced higher near‐transfer scores than those who were asked to solve a gradually increasing number of steps but starting with the last step first (backward‐fading practice). Second, students who received feedback immediately after attempting each problem‐solving step outperformed those who received total feedback on near transfer. Finally, students who learned with backward‐fading practice produced higher near‐ and far‐transfer scores when feedback included the solution of a similar worked‐out problem. The theoretical and practical implications for engineering education are discussed.     

Developing a Comprehensive Assessment Program for Engineering Education*

This paper provides an overview of one institution's efforts to establish a comprehensive assessment program for continuous improvement of engineering education. A five step systematic process to develop an integrated assessment program from identifying educational objectives to applying measurement methods is explained in detail. Activities to encourage faculty participation and commitment are outlined. Four integrated assessment processes used by both faculty and students to assess and provide performance feedback are described. The focus of these assessment methods is on the measurement, development, and improvement of student learning outcomes aligned with ABET Engineering Criteria 2000. Preliminary results and lessons learned from the overall experience are highlighted.     

An Assessment Analysis Methodology and Its Application to an Advanced Engineering Communications Program

An assessment of a discipline‐specific advanced engineering communications program initiated over a decade ago and those assessment strategies that best measure the success of the program are described. Novel ideas for the visualization and interpretation of the data are presented. These techniques are conducive to an “assess‐revise‐assess” strategy for curriculum improvement since they can efficiently assist in defining an appropriate and rapid response to program needs and constituency expectations. Based in part on the assessment results, additions and extensions to the original program have been made. These include instruction in interpersonal communications, teamwork, engineering research and professional ethics, management and professional development skills, critical and creative thinking, and engineering design and are described briefly to place the current program in proper context for assessment. Positive correlations show that the program continues to be highly regarded by students, faculty, the college administration, alumni, and industry.     

EC2000 and Measurement: How Much Precision is Enough?

As engineering faculty engage in the process of developing assessment plans to implement continuous quality improvement and satisfy the requirement of Engineering Criteria 2000 (EC2000), there is a concern about what measures are adequate to provide evidence that an engineering program is meeting its stated objectives. Some engineering programs are looking at using course grades as evidence that students are meeting the learning outcomes mandated by Criterion 3 of EC2000. After all, if there is a course in engineering design, why shouldn't grades be used to demonstrate that students are acquiring the skills necessary to meet the required outcome? The question remains as to whether or not course grades are adequate and/or efficient as a means to evaluate program effectiveness. This paper will define what is meant by educational inputs, processes, outputs, and outcomes in order to clarify the focus of the new “outcomes assessment” model of engineering education accreditation. A framework will be presented to clarify the meaning and scope of assessment activities needed to meet the information needs of academic programs and institutions. Models for course assessment and program assessment will be presented and the similarities and differences discussed.     

Assessing K‐12 Pre‐Engineering Outreach Programs*

Motivated by a desire to excite K‐12 students about the joys of engineering and spark their interest in pre‐engineering subjects, the Integrated Teaching and Learning (ITL) Program at the University of Colorado at Boulder has developed a pre‐engineering outreach program targeted at K‐12 teachers and students. To supplement anecdotal success indicators, ITL developed several assessment tools to measure the impact of these programs. Assessment strategies consist of three key components: 1) assessment of workshop participant feedback (teachers and students), 2) assessment of long‐term outcomes (teachers), and 3) assessment tools developed for the teachers' classroom use (i.e., embedded assessment). This paper reviews the process used to develop the assessment plans and tools. Examples of the tools used to assess participant feedback and preliminary outcomes are provided. Additionally, the process used to develop embedded assessment tools is described, including development of performance criteria and assessment tools that are linked to the learning goals, objectives, and K‐12 State educational standards.     

Students' Conceptions of Tutor and Automated Feedback in Professional Writing

Background Professional writing is an essential outcome for engineering graduates and hence a vital part of engineering education. To provide a successful learning experience for students engaged in writing activities, timely feedback is necessary. Providing this feedback to increasing numbers of students poses a major challenge for instructors. New automated systems work towards providing both timely and appropriate writing feedback, but students' views on automated feedback, and feedback in general, are not well understood. Purpose (Hypothesis ) To contribute to a deeper understanding of students' conceptions of feedback from tutors and an automated system called Glosser, and how these conceptions are related to achievement. Design /Method Students in an engineering course worked in pairs to write an engineering report on e‐business. The design of the study involves in‐depth interviews and the analysis employs an approach in which student conceptions of automated feedback are investigated in relation to related feedback from their tutor, perceptions of automated feedback in general, and their academic achievement. Results Students' conceptions of feedback vary and can be grouped into cohesive and fragmented, which is consistent with other theoretical models. Close associations were found between more cohesive conceptions of feedback and better academic performance. Conclusions A student's conception of traditional and automated feedback is similar, being either cohesive or fragmented. Changing one may change the other. Deep learners see feedback as a way of learning about the topic whereas shallow learners see them as a way to improve the communication aspects of writing. Design considerations based on these results are discussed.     

Developing and Assessing Statewide Competencies for Engineering Design

An assessment system was developed and piloted in Washington state to evaluate the engineering design competence of community college transfer students and continuing students at Washington State University (WSU) and the University of Washington (UW). A multiple measured approach was employed consisting of a multiple-choice assessment, a team design performance assessment, and an essay, each administered to junior level students at WSU and UW. These assessments covered important design and design-related outcomes valued by the engineering community in Washington and expected of junior level engineering students. Scoring criteria were developed by engineering faculty for the team design and essay components. The assessment results provide faculty and other decision makers at community colleges and four-year institutions with data they need to determine the extent to which students are meeting design competency expectations. Moreover, the approach described in this paper illustrates how institutions can productively address the ABET Engineering Criteria 2000 requirements by developing the assessment support that engineering educators need to make informed programmatic decisions and achieve continuous quality improvement.     

The Effect of an Entrepreneurship Program on GPA and Retention*

There is a small but growing body of evidence that entrepreneurship programs add value to students, the degree programs in which they are housed, and the institutions that host them. The Engineering Entrepreneurs Program at North Carolina State University, a program in which undergraduate students participate in design teams formed around technology start‐up company themes, was started with funding from the NSF‐sponsored SUCCEED (Southeastern Universities and Colleges Coalition for Engineering Education) Coalition primarily to improve the confidence and retention of engineering students. Multiple assessment approaches including surveys, focus groups, interviews, longitudinal assessment of retention and academic performance, and anecdotal evidence triangulate on the success of this program at meeting its primary objectives and others. Particularly, the longitudinal study revealed that program participants had higher engineering retention rates (70 percent vs. 51 percent) and GPAs (3.08 vs. 2.83) than a matched set of non‐participants. The program and its rigorous assessment serve as models for the engineering entrepreneurship community.     

Using Design Projects for Program Assessment

Engineering education is undergoing an assessment of its own. Organizations responsible for engineering education are looking at themselves in an attempt to improve their total quality as well as the quality of the programs they assess. The reasons for these changes include issues relating to cost, efficiency and quality of engineering education programs, and the increased demand for accountability by constituencies (state legislatures, students, employers). The ability to work in a group and to acquire and assimilate new knowledge effectively is another key requirement of graduate engineers and is difficult to assess. This article describes the national and local forces for more definitive outcome assessment and how we have responded to those forces. The article in particular describes how we have used senior design projects as an important part of undergraduate program assessment.     

An Intervention to Address Gender Issues in a Course on Design,Engineering, and Technology for Science Educators

A course on design, engineering, and technology based on Bandura's theory of self‐efficacy was taught to nine science education graduate students who were also practicing teachers. The interpretive analysis method was used to code and analyze qualitative data from focus groups, weekly reflections on classes and readings, and pre‐, post‐, and delayed‐post course questions. The improvement in tinkering and technical self‐efficacies for five males was limited because of initially higher self‐efficacies while that for four females was moderate to high, especially when working in same‐sex teams in a non‐competitive environment. All students also increased their understanding of the societal relevance of engineering and their ability to transfer engineering concepts to pre‐college classrooms. Implementing the principles employed in this intervention in pre‐college science and university engineering classrooms could help recruit students into engineering as well as help retain both male and female undergraduate engineering students.     

Improving Learning in First‐Year Engineering Courses through Interdisciplinary Collaborative Assessment

This paper describes a feedback process that assessed first‐year engineering student learning using a mastery exam. The results were used to improve learning and teaching in first‐year courses. To design the initial exam, basic knowledge and concepts were identified by instructors from each of the host departments (Chemistry, Math, Physics and Computer Science). In 2004, the 45‐item exam was administered to 191 second‐year engineering students, and in September 2005, the revised exam was administered to the next class of second‐year engineering students. The exam was analyzed using Item Response Theory (IRT) to determine student abilities in each subject area tested. Between exam administrations, workshops were conducted with the four department instructor groups to present exam results and discuss teaching issues. The exam provided a learning assessment mechanism that can be used to engage faculty in science, mathematics, and engineering in productive linkages for continual improvement to curriculum.     

New Curriculum Reforms in a Geological Engineering Program

Recent curriculum revisions to the geological engineering program at Queen's University at Kingston in Canada have led to a more streamlined program incorporating modern engineering education practices. Following a carefully designed program philosophy, the emphasis in the core curriculum changes through the entire four‐year program in three progressive stages, from the acquisition of knowledge, to integration and analysis, and finally to synthesis and design. This is reflected in an increased concentration of mathematics and basic science courses in first and second year, engineering science courses in third year, and engineering design courses (capstone courses) in fourth year. Two tools which concisely illustrate the course curriculum and curriculum content are: (1) the flow sheet, which can contain a wealth of information, such as showing linkages between courses (e.g. how upper‐level courses can build on lower‐level courses through course prerequisites), the timing of various courses, courses taught within the home department (vs. other departments), and courses taught by professional engineers; and (2) the ternary phase diagram, which is a quantitative method of displaying engineering content within individual courses or an entire program and can clearly show patterns and trends in curriculum content with time. Such tools are useful for academic engineering programs which may have to undergo an accreditation review and are readily adapted to any other engineering fields of study. Other engineering elements woven throughout the program include strong interactions with professional engineering faculty, the use of student teams, enhanced communication skills, and exposure to important aspects of professional engineering practice such as engineering ethics and law. To ensure that the curriculum is kept current and relevant, formative evaluation instruments such as questionnaires are used in all years of study, and are also sent to recent graduates of the program. External reviews of the revised program have been positive, indicating that the program goals are being achieved.     

Course Assessment Plan: A Tool for Integrated Curriculum Management

As we enter the 21^st Century in engineering education, a common desire exists to improve curriculum structure, integration and assessment. Much has been written and discussed concerning the process for assessing and/or revising a program curriculum. Studies are beginning to show the positive effects of well‐integrated curricula where assessment methods are applied consistently. There has also been much written to support individual course assessment and revision. What is missing in many instances is a credible link between program‐level curriculum management and course assessment. At the United States Military Academy (USMA) at West Point, an integrating tool within the academy's assessment model, called a Course Assessment Plan, has been developed and refined. The course assessment process and the resulting written documentation provide an essential link between a program curriculum and its constituent courses. The plan's process, content, and an example outcome are the major focus of this paper.     

Designing and Teaching Courses to Satisfy the ABET Engineering Criteria

Since the new ABET accreditation system was first introduced to American engineering education in the middle 1990s as Engineering Criteria 2000, most discussion in the literature has focused on how to assess Outcomes 3a‐3k and relatively little has concerned how to equip students with the skills and attitudes specified in those outcomes. This paper seeks to fill this gap. Its goals are to (1) overview the accreditation process and clarify the confusing array of terms associated with it (objectives, outcomes, outcome indicators, etc.); (2) provide guidance on the formulation of course learning objectives and assessment methods that address Outcomes 3a‐3k; (3) identify and describe instructional techniques that should effectively prepare students to achieve those outcomes by the time they graduate; and (4) propose a strategy for integrating program‐level and course‐level activities when designing an instructional program to meet the requirements of the ABET engineering criteria.     

Development of Engineering Education as a Rigorous Discipline: A Study of the Publication Patterns of Four Coalitions

A combination of publication analysis and faculty interviews was employed to study four NSF‐sponsored engineering education coalitions as a case study of the recent history of engineering education. Current calls within the engineering education community for increased rigor can be understood in terms of the ways similar disciplines have emerged. In science education, for example, time was needed to develop consensus on important research questions, accepted methods, and standards of rigor. The abstracts of 700 publications listed on active engineering education coalition Web sites were analyzed over time by type of intervention, population of focus, and product. A picture consistent with other reports of coalition contributions emerged. Early focus was on freshman courses and integrating across disciplines, with teamwork, design and other active learning activities. Students and course improvement remained the dominant focus, but efforts increased over time in assessment, faculty development, and research. Interviews with coalition leaders and leading authors supplement the publication analysis and describe how coalition work helped lay the foundation for more rigorous engineering education research.     

Diffusion of Engineering Education Innovations: A Survey of Awareness and Adoption Rates in U.S. Engineering Departments

Background Despite decades of effort focused on improvement of engineering education, many recent advances have not resulted in systemic change. Diffusion of innovations theory is used to better understand this phenomenon. Purpose (Hypothesis ) Research questions include: How widespread is awareness and adoption of established engineering education innovations? Are there differences by discipline or institutional type? How do engineering department chairs find out about engineering education innovations? What factors do engineering department chairs cite as important in adoption decisions? Design /Method U.S. engineering department chairs were surveyed regarding their awareness and department use of seven engineering education innovations. One hundred ninety‐seven usable responses are presented primarily as categorical data with Chi square tests where relevant. Results Overall, the awareness rate was 82 percent, while the adoption rate was 47 percent. Eighty‐two percent of engineering departments employ student‐active pedagogies (the highest). Mechanical and civil engineering had the highest rates, in part due to many design‐related innovations in the survey. Few differences by institution type were evident. In the past, word of mouth and presentations were far more effective than publications in alerting department chairs to the innovations. Department chairs cited financial resources, faculty time and attitudes, and student satisfaction and learning as major considerations in adoption decisions. Conclusions The importance of disciplinary networks was evident during survey administration and in the results. Specific recommendations are offered to employ these networks and the engineering professional societies for future engineering education improvement efforts.     

Engineering Design Thinking,Teaching, and Learning

This paper is based on the premises that the purpose of engineering education is to graduate engineers who can design, and that design thinking is complex. The paper begins by briefly reviewing the history and role of design in the engineering curriculum. Several dimensions of design thinking are then detailed, explaining why design is hard to learn and harder still to teach, and outlining the research available on how well design thinking skills are learned. The currently most‐favored pedagogical model for teaching design, project‐based learning (PBL), is explored next, along with available assessment data on its success. Two contexts for PBL are emphasized: first‐year cornerstone courses and globally dispersed PBL courses. Finally, the paper lists some of the open research questions that must be answered to identify the best pedagogical practices of improving design learning, after which it closes by making recommendations for research aimed at enhancing design learning.     

Constructive Alignment of Interdisciplinary Graduate Curriculum in Engineering and Science: An Analysis of Successful IGERT Proposals

Background Interdisciplinary approaches are critical to solving the most pressing technological challenges. Despite the proliferation of graduate programs to fill this need, there is little archival literature identifying learning outcomes, learning experiences, or benchmarks for evaluating interdisciplinary graduate student learning. Purpose (Hypothesis ) The purpose of this study is to understand how engineering and science academics conceptualize interdisciplinary graduate education in order to identify common practices and recommend improvements. Questions generated by an instructional design framework guided the analysis: what desired outcomes, evidence, and learning experiences are currently associated with interdisciplinary graduate education? To what extent are these components constructively aligned with each other? Design /Method Content analysis was performed on 130 funded proposals from the U.S. National Science Foundation's Integrative Graduate Education and Research Traineeship (IGERT) program. Results Four desired student learning outcomes were identified: contributions to the technical area, broad perspective, teamwork, and interdisciplinary communication skills. Student requirements (educational plans) addressed these outcomes to some extent, but assessment/evidence sections generally targeted program level goals—as opposed to student learning. This lack of constructive alignment between components is a major weakness of graduate curriculum. Conclusions Current practices are promising. Further clarification of interdisciplinary learning outcomes, coupled with closer alignment of outcomes, evidence, and learning experiences will continue to improve interdisciplinary graduate education in engineering and science. Specific recommendations for engineering and science faculty members are: define clear learning objectives, enlist assessment/evaluation expertise, and constructively align all aspects of the curriculum.     

An Assessment Matrix for Evaluating Engineering Programs

In this paper we describe the use of an assessment matrix to help faculty develop an assessment plan for their engineering program. Use of the matrix assures that each of the key steps in an effective assessment plan is addressed: setting goals and objectives; selecting performance criteria; planning an implementation strategy; choosing appropriate measures; setting a timeline; and providing timely feedback. The matrix has been used successfully to provide an assessment framework for engineering curricula, individual courses, and educational research projects.     

Manufacturing: A Strategic Opportunity for Engineering Education

This paper examines the importance of the manufacturing enterprise and the need for manufacturing education. The objective is to present a case for the expansion of manufacturing‐related education as a strategic opportunity for engineering education. A brief history of engineering education is presented, as well as an exploration of the current ABET criteria for various engineering disciplines. Approaches for achieving manufacturing‐related education are presented noting that Mechanical Engineering and Industrial Engineering are often most closely associated with manufacturing. Surveys of industry reveal the need for manufacturing education and identify preferred approaches. If manufacturing is to be included as part of a mechanical engineering program, there are a number of possible approaches. Of all the new technologies that will impact engineering education, none is larger than the Internet. The number of manufacturing educational programs in the United States is growing substantially. New manufacturing programs are encouraged along with review of educational content in traditional engineering disciplines‐especially the related discipline of mechanical engineering. Analysis leads us to believe that manufacturing represents a strategic direction and opportunity for engineering education to pursue.