1999  Senior, Alumni and Manager/Supervisor Survey Report

 

Produced by College of Engineering

 

Committee on Outcomes Assessment for Undergraduate Engineering Programs

 

 August 1999

 

 

Background

 

            This report is intended to be a college level summary of outcomes assessment surveys, administered during the 1998-99 academic year, by the College of Engineering on behalf of its undergraduate engineering programs.  The report will briefly outline the process for survey development and then summarize the data.  Generally the data in this report is an aggregate over all programs of the College.   Since differences in responses between programs can exist and programs have additional inputs from other sources, this information needs to be used carefully by each program as part of their on-going outcomes assessment and continuous quality improvement.  In some cases observations about the data will be included to enhance understanding.  Recommendations for future survey development and follow-up will need to be developed by the Outcomes Assessment for Undergraduate Engineering Programs Committee  (OAC) and the individual programs.  The report closes with preliminary recommendations of the Outcomes Assessment Committee.

                       

            During the 98-99 Academic year the College of Engineering OAC developed a Programs Outcomes Assessment Model for the College.  In the model, collection of data from constituents (e.g. current students, alumni, and employers) is important to both the Program Educational Objectives System (long-term) and the Programs Outcomes System (short-term).  The OAC developed a series of surveys for use in the College. The primary purpose of the surveys developed is to seek data on the perceived importance of outcomes and objectives established for our current engineering programs and assess how well we are doing relative to those.  The surveys are intended to be a integral part of the data collected for program outcomes assessment including items directly relevant to the ABET EC2000 criteria.    However, the results must be combined with other outcomes assessment  information collected by the College and its programs to support a plan of action for continuous improvement at each level.

 

Selection of Survey Groups

 

The OAC decided to focus on three groups:  1) exiting seniors,  2)  recent alumni, and 3) managers/supervisors of our alumni.   Alumni two and six years after graduation were selected with the idea that they would still have a connection to the programs as currently offered.  In our assessment model, employers were a significant constituent group.   In the Committee’s opinion, Supervisors and Managers for our recent alumni would be in a position to comment on both the importance of an outcome/objective to engineering practice and the ability/preparation of our recent graduates.  On a trial basis and as a strategy to approach this supervisor group, the fifteenth year alumni of the College were selected as a population to represent managers and supervisors of the recent alumni.

           

Surveys Developed

 

            Alumni survey

 

            After review of available surveys from other institutions and consultation with faculty from our campus with survey expertise, a five-part alumni survey was developed (See Appendix 1-A).  Each part is described briefly in the following paragraphs.

 

Part I. Educational Outcomes – General

 

This segment focused on the ABET EC2000 Criterion 3 Program Outcomes and Assessment.  Alumni were asked to rate twenty-five skills, abilities and attributes generally expected of an engineering graduate in two ways.  First, they were asked how important each has been relative to their needs and employment experience since graduation.  Second, they were asked to rate each item relative to their abilities.  Response for Importance were divided into a five segment Lickert scale.  For scoring they were weighted; Not Important = 1, Somewhat Important = 2, Important = 3, Very important = 4, Extremely Important = 5.  Similarly Ability on the job was rated and scored; Not Prepared = 1, Somewhat Prepared = 2, Prepared = 3, Well Prepared = 4, Very Well Prepared = 5.  A final question related to being a licensed professional was included.

 

Part II.  Educational Experience at Ohio State

 

This section focused on the quality of instruction received at The Ohio State University.  Alumni were asked to respond to twenty-five questions related to quality of instruction, advising and facilities.  Responses were again scored on a Lickert scale with the following options:  Unsatisfactory = 1, Poor = 2, Fair = 3, Good = 4  Excellent = 5 or no opinion (not included in analysis).  In addition five questions regarding overall preparation were asked using the same available response scale.

 

Part III.  Program Specific Section

 

In this section each program was afforded the opportunity to focus on ABET EC 2000 Criterion 2 Program Educational Objectives and any other program specific information they desired to collect.  Format was specified by the Program.  Data from this section is summarized by the Program.

 

Part IV.  Additional Comments and Suggestions.

 

This segment simply solicited comments regarding the program or educational experiences at Ohio State.  Suggestions for improvement were specifically requested to be forwarded to the individual program.

 

Part V.  Background Information

 

This segment collected twelve items of demographic data about the respondent.

 

            Engineering Supervisor Assessment Survey

 

            A survey for use with fifteenth year alumni, as a surrogate for supervisors, was developed in three parts (See Appendix 1-B).  The first part of the survey simply asked for the primary job functions of the person being surveyed.  The second segment was a slightly modified version of Part I of the Alumni Survey on Skills, Abilities and Attributes.   The respondent was asked to rate importance in regard to a new engineer joining the firm.  Rather than Ability, the respondent was asked to rate preparation of recent Ohio State engineering graduates that they supervise.  Respondents were asked to indicate the program of the person(s) they were basing their evaluation on.  They were asked to do multiple forms if more than one program was represented.  If the respondent was not supervising any recent graduates, they were still asked to respond to the importance questions.  The last element of the survey asked for additional comments regarding the graduates and suggestions for survey improvement. 

 

            Senior Exit Survey

 

            A four part, draft Senior Exit Survey was developed by the OAC (see Appendix 1-C).   The intent was for the survey to be administered to seniors at or nearing graduation.  Part I asked for seven items related to demographics, participation in various programs, and future plans.  Part II Educational Outcomes – General used the same elements and structure as Part I of the Alumni Survey.   Students were asked to rate importance of each item and their current ability.  Part III Educational Experience at Ohio State made use of the same structure as Part II of the Alumni Survey with the addition of two questions related to Coop/Intern and Career Services.   Since the survey form was not finalized until end of winter quarter 1998, generally spring quarter 1999 graduates were the only ones available for this survey.  As with the all surveys, use by each Department/Program was at the discretion of the program.

 

Populations Surveyed and Return Data

 

            For the Alumni and Supervisor/Manager Surveys (Years 2, 6, & 15), available addresses from the OSU Alumni Association were used.  Alumni graduates were based on calendar year;  Year 2 = 1996, Year 6 = 1992, and Year 15 = 1983.  The Table below indicates the number of surveys sent out in January of 1999, and number returned for each program.  For the Senior Exit Survey, seven programs across five Departments used all or a portion of the draft survey.  The number of students surveyed is indicated in the Table below.  Some programs choose to continue using existing survey forms and procedures, rather than adopt the form at this time.

 

Table 1.  Surveys Distributed and Returned

 

Exits

Yr 0 - 99

AlumniYr 2–96

AlumniYr 6-92

Supervisor

Yr 15 – 83

Program

Use(2)

Sent

Retnd

Sent

Retnd

Sent

Retnd

Grad (1)

Aero and Astro

 

27

8

41

7

73

9

4

Agr./FABE

2

20

2

23

6

14

0

0

Aviation

 

12

0

16

0

6

1

1

Ceramic

 

8

0

16

2

31

6

1

Chemical

 

 

1

 

0

119

19

8

Civil

28

120

17

64

20

136

16

7

Civil – Environ.

7

(5)

3

(5)

10

 

 

 

Computer Science

 

102

8

100

10

77

11

3

Electrical

23(4)

145

15

134

13

164

17

7

Eng. Physics

 

7

0

8

4

7

0

0

Industrial

50

94

11

66

22

124

14

6

Materials Science

 

18

0

2

3

 

0

0

Mechanical

24(3)

141

27

131

27

147

19

10

Metallurgical

 

8

4

26

1

23

3

0

Surveying

 

3

0

7

0

8

0

0

Welding

13

38

7

41

9

32

5

2

Total

147

743

102

675

138

961

120

49 (40.8%)

Percent Returned

---------

------

13.7%

------

20.4%

------

12.5%

5.1%

(1) Not all surveys comments on both Importance and Graduate Preparation.  This column gives number responding to graduate preparation.

(2) Not all programs used all elements of survey.

(3) Done Winter 98, all others were done Spring 99.

(4) Completed experience at OSU section only; supplemented with Dept. specific questions.

(5) Included in Civil

 

Overview Analysis of Data on Respondents

 

The following tables summarize the background data collected from the respondents across the college.   (Data files for each program were forwarded to program representatives.)  Summaries presented are for a composite across programs.  Data in terms of number of respondents and distribution of respondents across programs is given in Appendix 2-A.

Table 2.  Demographics (%)

 

 

Yr 0 - 99

Yr 2 - 96

Yr 6 - 92

Gender

Male

81.8

93

81

 

Female

18.2

7

19

Racial/Ethnic Group

African-American

3.4

0

1

 

Asian-American

8.5

2

3

 

Caucasian

61.0

93

91

 

Native American

0.0

1

1

 

Hispanic-American

0.0

0

0

 

Other/Not Given

27.1

4

4

U.S. Citizen while student

Yes

75.0

95

99

Co-op/Intern Exp.

0 qtrs.

15.6

24

23

 

1 – 2 qtrs.

30.2

33

24

 

3 – 4 qtrs.

33.3

22

30

 

> 4 qtrs.

20.8

21

23

 

                               

Table 3.  Employment Description (%)

 

 

Yr 2 - 96

Yr 6 – 92

Employment Status

Full-time

92

97

 

Part-time

0

1

 

Military

3

0

 

Family/Home

0

1

 

Un/Seeking

1

0

 

Un/Not Seeking

0

0

Employed as Engineer/Surveyor

Yes

76

80

Worked Outside US

Yes

10

6

Pri. Job Function

Design

14

17

(Up to 3 selected)*

Sales

2

4

 

Test/Reliability

4

7

 

Purchase/Procurement

0.5

2

 

GIS/LIS Admin.

0

1

 

Manufacturing

11

6

 

Plant Engineer

4

2

 

Computer Applic.

6

8

 

Education

1

1

 

Hydrographic Surv.

0

0

 

Res. & Dev.

8

5

 

Project Eng.

14

14

 

Comp.Sys. Admin.

2

2

 

Construction

1

4

 

Management

12

6

 

Service

1

1

 

Process Eng.

5

5

 

Medical

0

1

 

Boundary Surveyor

.5

1

 

Consultant