The following text refers to Gen Ed Project Results - Spring '09:
25 faculty members submitted data for 342 students and a total of 981 scores. Referring to the first tab of the spreadsheet, if we lump all the results together, we see:
I wonder if this is a function of grading bias ... the dominant grade is "middling?"
The number of credit-hours earned for each of the 342 students was associated with each score earned, then the student S-numbers were stripped out for anonymity. The average number of credit-hours earned for each score category were then computed for each 19 GenEd objectives - 1a, 1b, 2a, ... 5c. These are tabulated and graphed on the subsequent 19 tabs of the spreadsheet.
Each tab in this spreadsheet contains a graph of the average number of credit hours earned by students who scored a particular value. The expectation is that more education would tend (like "value added") to increase the score (and thus the higher the score, the higher the number of hours), but the graphs mostly show flat or the opposite trend.
On the other hand, the numbers of data points is sufficiently small in this sample that these statements cannot be made with much confidence. In fact, I ran a bootstrap analysis where I resampled (with replacement) the values and fitted each resampling to a line, computing the overall probability of a positive slope. Every one of the data sets came in with ~50% probability of positive slope (and thus ~50% probability of negative slope - technically non-positive). This tells us that we don't REALLY know that either trend is really occurring.
The only definite thing we can say about the spring '09 GenEd Project data is that it does not show any clear "value added" trend.
Only for 4b, 4c, and 4d did the number of 1s (Unsatisfactory) exceed 2s (Satisfactory).
What would a negative slope mean here? mental laziness? reduced confidence? degrading test-taking performance? disregard for the test/exercise/project? (This consideration should be eliminated by fully embedding the assessment into the course, i.e. context-appropriate production that counts for part of the studentís grade. The student should not be told that this is something other than a course production related to a course-objective.)