This is the first time this test has been given and the results will give the math department an insight as to any revisions that need to be made before the test is administered again.
Questions #1, 5, and 7 in particular should be looked at to see if the extremely low scores are a reflection of the wording of the questions, or can be attributed to poor math skills.
This is the first time this test was given.
The first three questions were designed to test the ability of the student to analyze the validity of an argument. Questions #1 and #3 were multiple choice while #2 was a short answer format. The short-answer format allowed more insight into what the student was actually thinking.
Question #4 first gave the results of an experiment and the experimenter's erroneous conclusion; it then asked the student to evaluate the assumptions and errors. No student identified the primary assumption, and only about one third seemed to realize the essential problem with the experimenter's conclusion.
Regarding the instrument itself, each of the questions will evaluated in light of the responses to purify the intent of the question.
Two primary issues seem to be indicated: one, that students do not recognize the concept of argument as a logical progression, and two, that students do not seem to realize what assumptions they are making in their own thinking.
An informal overview of the data reveals that most students made progress in most areas. The average shows the greatest progress being made in sentence fluency and conventions.
Most teachers of composition devote much attention to revision and editing, encouraging students to rewrite papers by incorporating correction noted in peer and teacher conferences and in consultation at the writing center. This accounts for the improvement in most areas.
The negative average for voice has a reasonable explanation. Most first essays were personal essay where voice is exceptionally strong; the second essay, being a research essay demands a greater attention to one's personal voice.
The data suggest that special attention needs to be paid to developing voice in academic research papers. This can be accomplished by close attention to voice in text essays and essays and articles from other sources. In research essay, students tend to rely too much on using quotations rather than paraphrases. They often are reluctant to interpret the research. Emphasis needs to be placed on selection of unbiased resources and using summary and paraphrase. This can be done by spending more time on these skills, calling attention to it in model essays, and attempting to paraphrase sample paragraphs.
Teachers, for the most part continue to use workshop methods in their classes, which accounts for improvement in most areas.
We also want to encourage submission of essays from other disciplines.
We have continued various methods of delivery, including workshop, Picture-tel, and computers. Teachers observe that students with the greatest development are those with regular attendance. Although we attempt to emphasize attendance, we need backup from the administration. Having the students on campus with dorm scholarships demands this accountability.
The data from 73 students had an average(stdev) of 47%(24%) overall. Separating it into the categories: 12 had taken up to Problem Solving, MAT 110, and scored 21%(8%); 14 had taken up to College Algebra, MAT 121, and scored 48%(30%); 21 had taken up to Statistics, MAT 135, and scored 47%(15%); 17 had taken up to Survey of Calculus, MAT 125, and scored 51%(23%); and 10 had taken Calculus, MAT 201 (and 202) and scored 72%(14%).
The trend does show that students do better with more mathematics training.
Comments from MAT135 instructor: "I am surprised again at the low percent of students who answered the first problem correctly. I believe that most students assume they know the answer and answer without thinking about what the problem actually asks. I am pleased that the majority of statistic students answered the probability problem correctly, but don't understand why so many more answered the first part right and missed the second part of the question. I am surprised that so many students missed problem 4a and 4b. This reflects the students' inability to realize when an answer makes sense, and when it does not."
Mathematics faculty feel that students seem to simply memorize techniques often without really understanding what it is they are doing. Several courses already contain mathematics project requirements, but these will be modified in ways that better promote critical thinking in students. One simple change that the calculus instructor is considering is to simple give the project assignments earlier and require drafts so that students do not wait until the last possible moment.
More emphasis needs to be placed on number sense - in particular, realizing when an answer makes sense and when it doesn't. Also, find some critical thinking exercises or examples to stimulate the thinking process.
The instrument was purified after the fall pilot to improve the wording on several problems, the "min/max problem" was eliminated, and a new problem was introduced which attempts to measure the students true understanding of the connection between English and an associated graph plus the difference between linear and exponential growth.
To get at students' effort, a trio of checkboxes was added for each question. These are "I've seen problems like this before," "I've never seen problems like this before but I tried to reason it out," and "I've never seen problems like this, so I guessed."
A general education embedded science assessment was administered. This instrument was a compilation of seven questions written in collaboration with all science faculty.
This was the second time this test was given; it consists of four multiple-choice questions which attempt to measure logical reasoning, a question that measures the student's ability to read an unfamiliar argument with scientific terms, a question which relates to the interpretation of experimental results, and a question which gives the student a chance to synthesize an experimental solution.
The data shows wide spans of scores, probably due somewhat to presentation of the instrument itself (required exam, extra credit, etc.). Unsurprisingly, the science reading question gave the best results, logic a distant second, followed by experimental design and identifying assumptions.
Questions still need to be fine-tuned, for example, a term is used in the design question that causes trouble which has nothing to do with the intent of the question.
Two primary issues seem to be indicated: one, that students do not recognize the concept of argument as a logical progression, and two, that students do not seem to realize what assumptions they are making in their own thinking.
The instrument was modified after the fall pilot to improve the wording on several problems. The response to the reading question was changed so that the student could write a short answer. The last question was added to provede a measure of scientific numeracy skills; it needs fine-tuning.
Students in ART 111 were general able to apply what they had learned in answering the essay questions. Students in HUM 121 were the least able to do so. Students in the origination location of Pictel delivered courses performed better than students at the romote locations.
Overall students averaged 3.0 on a 4.0 scale. Individual courses were as follows: ART110- 2.8, ART111- 3.6, HUM121- 2.6
Spreadsheet summaryThis is the first time this test was given.
There is a need for more visits to remote sites as well as more practice in writting essay questions.
I intend to build comparison/contrast type writting assignments into these courses on a regular basis. I also will work on developing facilators at remote sites and making more personal visits to those sites. Another consistant comment on student evaluations, especially from Art Appreciation students (ART 110), was that they would like more hands-on opportunities. I intend to build more activities into the course. This is difficult with the remote sites but by adding a supplies list to the course syllabus I should be able to build in activities.