2000  Senior, Alumni and Manager/Supervisor Survey Report

 

Produced by College of Engineering

 

Committee on Outcomes Assessment for Undergraduate Engineering Programs

 

 September 1, 2000

 

 

Background

 

            This report is intended to be a college level summary of outcomes assessment surveys, administered during the 1999-2000 academic year, by the College of Engineering on behalf of its undergraduate engineering programs.  This is the second year of use of a survey of 2nd, 6th and 15th year alumni.  Results are reported in Part 1 of this document.  This year a Senior Exit Survey developed by EBI Corporation was used in place of a locally designed survey used in 98-99.  Results are reported in Part 2 of this document. Generally the data in this report is an aggregate of all programs.   Since differences in responses between programs can exist and programs have additional inputs from other sources, this information needs to be used carefully by each program as part of its on-going outcomes assessment and continuous quality improvement.  Program by program summaries of the basic data can be found at http://www.cse.ohio-state.edu/~neelam/COE/SURVEYRESULTS/.              

 

Where feasible, observations about the data and comparisons to the previous year will be included to enhance understanding.  Copies of the college-common part of each survey and the EBI Survey are appended.  Demographic Data describing the alumni populations being surveyed and the respondents is also appended.  

 

Recommendations for follow-up activity and future survey development will need to be developed by the Outcomes Assessment for Undergraduate Engineering Programs Committee  (OAC) and others in the Autumn of 2000. 

 

PART 1 – OSU SURVEY RESULTS

 

A.  Educational Outcomes Summary

 

            Since the same twenty-five questions were asked, a summary of Educational Outcomes across alumni year groups can be presented.  A tabulated form of the data for 1999-2000 by alumni year is in Appendix 2-B.   Since only a few questions showed obvious variation or trends across alumni years, data are summarized in the following Tables and Figures for the full college, across all programs and all alumni years.  The full text for each question can be found in the appropriate Appendix 1 segment.

 

 

Weighting of the scale is:

Importance:    Not Important = 1, Somewhat Important = 2, Important = 3, Very Important = 4, Extremely Important = 5

Ability on the Job or Preparation:  Not prepared = 1, Somewhat Prepared = 2, Prepared = 3, Well Prepared = 4, Very Well Prepared = 5

Difference: Ability/Preparation minus Importance

 

Table 1. Educational Outcomes Summary Across All Alumni Years and Programs by Survey Year

 

Topic

Importance

Ability/Preparation

Difference

 

98-99

99-00

98-99

99-00

98-99

99-00

An understanding of and ability to apply knowledge of:

Math

3.19

3.07

3.83

3.75

0.64

0.68

Chemistry

2.26

2.25

3.03

2.72

0.77

0.47

Physics

3.22

3.11

3.51

3.45

0.29

0.34

Eng. Science

3.66

3.47

3.65

3.54

0.01

0.07

Comp. Sci

3.74

3.37

3.85

3.43

-0.44

-0.42

Humanities

2.54

2.63

3.07

2.79

0.53

0.16

Business

3.38

3.32

2.68

2.60

-0.70

-0.72

An understanding and ability to:

Experiment

3.42

3.48

3.39

3.42

-0.03

-0.06

Analyze Data

3.84

3.81

3.65

3.55

-0.19

-0.26

Design System

4.05

4.01

3.44

3.38

-0.61

-0.63

Multi-D Teams

4.28

4.18

3.42

3.49

-0.86

-0.69

Solve Problems

4.20

4.17

3.77

3.80

-0.43

-0.37

Ethics

4.02

3.90

3.34

3.31

-0.68

-0.59

Comm Orally

4.36

4.29

3.20

3.33

-1.16

-0.96

Comm in Wrtng

4.30

4.26

3.45

3.48

-.085

-0.78

Stay Current

4.00

3.86

3.25

3.25

-0.75

-0.61

Skills-Practice

3.99

3.78

3.58

3.44

-0.41

-0.34

An understanding of and ability to:

Diverse Env.

3.51

3.29

3.35

3.33

-0.16

0.04

Computer-Commun.

3.95

3.97

3.30

3.65

-0.65

-0.32

Computer-Design

3.89

3.90

3.32

3.58

-0.57

-0.32

Integrate

3.73

3.54

3.19

3.17

-0.54

-0.37

Apparatus

2.57

2.70

2.91

2.72

0.34

0.02

An understanding of:

Env. Aspects

3.02

2.80

2.57

2.57

-0.45

-0.23

Global Scale

2.78

2.61

2.51

2.35

-0.27

-0.26

Rel to Society

2.58

2.35

2.49

2.31

-0.09

-0.04

Being a Licensed Professional Engineer/Surveyor-0.54

 

2.41

2.07

2.91

2.58

0.50

0.51

 

 

The Educational Outcomes data from the 1999-2000 Surveys is plotted in the following figures for easier study.

 

Figure 1. (1999-2000 data only)

 

 

Figure 2  (1999-2000 data only)

 

Figure 3  (1999-2000 data only)

 

 

 

 

Figure 4  (1999-2000 data only)

 

Suggested Priorities for Action

 

            Looking at those items with the largest differences between Importance versus Ability/Preparation, in its 1999 report the OAC choose six area for highest priority consideration by the College.  The following compares results of the most recent survey to those of 98-99 for these six areas.  Although considerable activity has occurred relative to these issues in the past year, it is of course too early to see the results of those efforts in this data.  However, highest difference did occur in the same areas confirming the choice of these as action areas.   

 

1.  Communications

           

            Reference Question(s)                                                  Ability/Prep. - Importance

An understanding of and ability to:                                                         98-99       99-00

            - communicate orally: informal and prepared talks                      -1.17        -0.96           

            - communicate in writing: letters, technical reports, etc. -0.85        -0.78

            - use computing technology in communications              -0.59        -0.32

 

2.  Teamwork

 

            Reference Question(s)                                                  Ability/Prep. - Importance

 

An understanding of and ability to:                                                         98-99       99-00          

            - function on multi-disciplinary or cross-functional teams            -0.86        -0.69

 

 

3. Basic Sciences

 

Reference Question(s)                                                  Ability/Prep. - Importance

 

An understanding of and ability to apply knowledge of:              98-99       99-00

- chemistry                                                                               0.77         0.47

- mathematics                                                                           0.64         0.68

 

 

4.  Business

 

Reference Question(s)                                                  Ability/Prep. - Importance

 

An understanding of and ability to apply knowledge of:              98-99       99-00

- business/finance                                                                      -0.70        -0.72

 

 

 

 

 

5.  Stay Current

 

            Reference Question(s)                                                  Ability/Prep. - Importance

 

An understanding of and ability to:                                                         98-99       99-00

            - stay current technically and professionally                                -0.76        -0.61

 

6.  Ethics

 

Reference Question(s)                                                  Ability/Prep. - Importance

 

An understanding of and ability to:                                                         98-99       99-00

- recognize professional and ethical responsibility                       -0.68        -0.59

 

 

B.  Educational Experience at Ohio State

 

In this segment, exiting seniors, alumni were asked to evaluate selected aspects of their experience at OSU.  Data by alumni year are presented in Appendix 2C.  The following Table presents the mean of  all alumni years across all programs for each year of the survey.

 

Table 2.  Summary of Educational Experience at Ohio State

 

 

98-99

99-00

 

 

98-99

99-00

Quality of instruction provided in MAJOR

Faculty

4.13

4.26

 

T.A.’s

3.56

3.58

Quality of Instruction provided by faculty OUTSIDE major:

CIS

3.47

3.36

 

Eng Grphs

3.78

3.90

Eng Mech

3.74

3.76

 

Elec Eng

3.27

3.27

Ind Eng

3.88

3.90

 

Mats Sci

3.75

3.65

Mech Eng

3.68

3.66

 

 

 

 

Quality of instruction provided by faculty in NON-ENGINEERING courses:

Chemistry

3.44

3.33

 

Humanities

3.67

3.74

Math

3.62

3.62

 

Physics

3.72

3.84

Quality of Advising with respect to:

Academic

3.41

3.47

 

Career Pln

2.90

3.10

Grad Ed

2.47

2.95

 

 

 

 

Physical Quality of Facilities

Comp Labs

3.69

3.76

 

Classr’ms

3.42

3.60

Sci Labs

3.52

3.64

 

Eng. Labs

3.50

3.55

S & E Lib

4.03

4.42

 

 

 

 

Overall Preparation:

Be an Eng

3.99

3.97

 

Obtain Job

4.02

4.05

Transition

3.72

3.85

 

Dvlp Prof

3.94

3.97

Contribute

3.98

4.01

 

 

 

 

 

 

It can be observed that improvement in perception of quality for at least one parameter can be correlated to a known event.  In the second survey year (1999-2000) a higher percentage of the population graduated after the new Science and Engineering Library addition.

 

C.  Overview Analysis of Data on Respondents

 

The background data collected from the respondents is tabulated in Appendix 2-D.   Summaries presented are a composite across programs. 

 

D.  Written Comments Summary

 

Written comments from the 2nd and 6th year alumni are summarized by the individual program.  From the 15th Year alumni 44 comments were received regarding graduates and 25 regarding potential improvements in the survey.  All comments regarding the graduates are reviewed by the Outcomes Assessment Committee.  They can be placed in four categories of:  1) alumni need more computer skills beyond CAD (e.g. data bases, spreadsheet), 2) co-op/real world, hands on experience is highly valued and needs to be encouraged, 3) more business related content is needed (e.g. accounting, economics), and 4) better people (team work) and communications (written and oral) skills continue to be desired.  Suggestions regarding the survey itself can be summarized around three items: 1) correct an error in labeling of the rating system on the form, 2) put the survey on line, and 3) make instructions clearer.

 

 

 

PART 2 - 2000 EBI Engineering Student Assessment Study

 

During the academic year 1999-2000 the College decided to participate in a new Engineering Student Assessment Benchmarking Project being developed by EBI Corporation of Springfield, Missouri.  The benchmarking included use of a senior exit survey (See Appendix 1-D for a copy) asking eleven demographic questions followed by seventy-one (71) questions common to all participants.   Each participating school could add an additional 10 institutional specific questions.  The last two sets of questions were all answered on a 1 to 7 scale. 

 

Participation by programs in the College was voluntary.  Surveys (195) were collected by the following ten programs (EBI Categories):  Agricultural (5%), Ceramic (1%), Chemical (4%), Civil/Construction (17%), Computer Science (4%),  Electrical/Electronic (6%), Environmental (3%), Industrial (37%), Materials (5%), Metallurgical (4%), Other (Welding) 14%.

 

Nationally 52 engineering schools participated.  Twenty-one (21) were Research I Universities by the Carnegie Classification.  Participants were allowed to select six other schools for a peer group comparison.  For Ohio State the following six schools were selected; MIT, Penn State University, University of Minnesota, University of California – Berkeley, University of Illinois at Urbana-Champaign, University of Texas Austin.

 

An extensive statistical analysis done by EBI has been shared with each member of the Outcomes Assessment Committee and is available through the Associate Dean for Academic Affairs and Student Services.  The analysis produced by EBI has been shared with each program. Following are highlights from the data.

 

 

Through statistical analysis, EBI reduced the results of the seventy-one questions to 14 factors.  The following table compares Ohio State to the Selected 6 Mean on these 14 factors.

 

Table 3.  T-Test of Factors: Comparison with Select 6 Schools

 

Factor

 

OSU

Selected

6 Mean

1

Satisfaction with Instruction and Interaction in Major Courses

4.82***

4.46

2

Satisfaction with Major Courses (Grades/Accessibility/Responsiveness/Amount of Work/Size/Availability)

5.41***

4.98

3

Satisfaction with the Breadth of Curriculum (Technology/Practical Experience/Professional Interaction)

4.81**

4.56

4

Satisfaction with Extracurricular Activities (Team Experiences/Organization Activities Leadership Opportunities)

5.09*

4.86

5

Satisfaction with Computer Resources (Quality/Availability/Remote Access/Training)

4.90

4.89

6

Satisfaction with Fellow Students (Academic Quality/Team Work/Camaraderie)

5.62**

5.37

7

Satisfaction with Career Services and Job Placement (Assistance/Number of Quality of Companies/Access to Alumni)

5.14

5.03

8

Degree of System Design & Problem Solving (Design/Interpret/Identify/Analyze/Teamwork/Solve Engineering Problems)

5.51

5.40

9

Degree of Understanding Ethical Responsibilities and Global Impact of Engineering Solutions

4.95***

4.53

10

Degree of Tools/Communication/Pilots & Text (use of Modern Tools/Oral and Written Communications Skills/Pilot Tests/Text Materials)

5.15

5.01

11

Degree that major Design Experience Built on Previous Course Work and Skills

5.41

5.25

12

Degree that major Design Experience Addresses Issues (Economic/Environmental/Sustainability/manufacturability/Ehtical/Health and Safety/Political)

4.46**

4.16

13

Degree that Laboratory Facilities Aided in Learning/Student and Faculty Interaction/Use of Modern Engineering Tools

5.07*

4.85

14

Overall Satisfaction with the Engineering Program

5.09

5.13

 

*p<.05, **p<.01,***p<.001 (asterisk appears on the greater mean)

 

 

 

It should be noted that OSU was statistically above the mean on eight (8) of the 14 factors and statistically below the mean on none (0) of the parameters.

 

EBI also presented an analysis of which fourteen factors and demographic parameters were predictors of overall satisfaction.  That analysis is summarized in the following Table.

 

 Table 4.  Factors and Demographic Predictors of Overall Satisfaction (In order of Importance)

 

Ohio State

Select 6 Comparative Group

Major Predictors – Factor (See previous Table for More Detailed Description)

11

Degree that Major Design Experience Builds

3

Satisfaction with Breadth of Curriculum

1

Satisfaction with Instruction & Interaction in Major Courses

1

Satisfaction with Instruction & Interaction in Major Courses

2

Satisfaction with Major Courses

8

Degree of System Design & Problem Solving

6

Satisfaction with Fellow Students

2

Satisfaction with Major Courses

7

Satisfaction with Career Services and Job Placement

7

Satisfaction with Career Services and Job Placement

Minor Impact

 

 

6

Fellow Students

 

 

13

Lab Facilities

 

 

9

Ethical Responsibilities

NO Impact

3

Breadth of Curriculum

4

Extracurricular Activities

4

Extracurricular Activities

5