AISL > Archived Report Index > Gen Ed Project Analysis - Spring 2009

The following text refers to Gen Ed Project Results - Spring '09:

25 faculty members submitted data for 342 students and a total of 981 scores. Referring to the first tab of the spreadsheet, if we lump all the results together, we see:

  1. Unsatisfactory, 133 scores {11, 020, 020, 063, 19}
  2. Satisfactory, 435 scores {46, 104, 118, 073, 94}
  3. Exemplary, 413 scores {50, 080, 070, 143, 70}
The {N1, N2, N3, N4, N5} numbers in each category above denote the number of scores for General Education Goals (GEG) 1, 2, 3, 4, and 5, respectively. These data are tabulated and graphed on the first tab of the spreadsheet. Note that in every case except GEG4, there are more 2s than 3s and more 3s than 1s; the percentages are almost all ~10%, 50%, 40% for GEG1, GEG2, GEG3, and GEG5. Only GEG4 has the most 3s, fewer 2s, fewest 1s.

I wonder if this is a function of grading bias ... the dominant grade is "middling?"

The number of credit-hours earned for each of the 342 students was associated with each score earned, then the student S-numbers were stripped out for anonymity. The average number of credit-hours earned for each score category were then computed for each 19 GenEd objectives - 1a, 1b, 2a, ... 5c. These are tabulated and graphed on the subsequent 19 tabs of the spreadsheet.

Each tab in this spreadsheet contains a graph of the average number of credit hours earned by students who scored a particular value. The expectation is that more education would tend (like "value added") to increase the score (and thus the higher the score, the higher the number of hours), but the graphs mostly show flat or the opposite trend.

On the other hand, the numbers of data points is sufficiently small in this sample that these statements cannot be made with much confidence. In fact, I ran a bootstrap analysis where I resampled (with replacement) the values and fitted each resampling to a line, computing the overall probability of a positive slope. Every one of the data sets came in with ~50% probability of positive slope (and thus ~50% probability of negative slope - technically non-positive). This tells us that we don't REALLY know that either trend is really occurring.

The only definite thing we can say about the spring '09 GenEd Project data is that it does not show any clear "value added" trend.

Only for 4b, 4c, and 4d did the number of 1s (Unsatisfactory) exceed 2s (Satisfactory). 

What would a negative slope mean here?
	mental  laziness?
	reduced  confidence?
	degrading test-taking performance?
	disregard for the test/exercise/project?
	(This consideration should be eliminated by fully embedding
	 the assessment into the course, i.e. context-appropriate
	 production that counts for part of the student’s grade.
	 The student should not be told that this is something other
	 than a course production related to a course-objective.)