Student #

Mastery Math Lab Pre-Test Results

Introduction

During the first week of class of the Fall 2005 semester (06F), all students taking a developmental math class in the Math Lab were given an internal diagnostic pre-test to confirm that the students were registered for a class appropriate to their skill level. For all courses except Math 030, the first part of the pre-test consisted of material that was covered in that classes’ pre-requisite course. For example, Math 090 students first took a pre-test covering Math 060 content material to ensure that those students had already mastered Math 060 before entering Math 090. If students answered fewer than 10 of the 15 questions correctly on the pre-requisite content pre-test, they were advised to move down to that lower-level pre-requisite course. Students not wishing to move down were given the option of signing a waiver which would allow them to remain in the course in which they were initially registered; however, they were advised that it was our professional recommendation that they either move down to the lower level course or sign up for tutoring as soon as possible. Students who correctly answered 10 or more questions on the first part of the pre-test were given the option of remaining in the course for which they initially registered, or taking the second part of the pre-test which contained content material for the course in which they were currently enrolled. A high score on this second part of the pre-test would allow the student to move up to the next math class, skipping the class in which they were initially enrolled.

For specific information for each individual course, please see the spreadsheets on the following pages.

Math 106 Pre-Test – 090 Content

(If 9 or fewer answers were correct, student should enroll in Math 090. If 10 or more questions were answered correctly, it was not necessary for the student to sign the waiver in order to be allowed to remain in the class in which he/she was enrolled.)

Student #	# Correct	Highest Accuplacer Score AR EA	Accuplacer Placement Into Course	Took Pre-Requisite	Moved Up/Down	Signed Waiver (Remained in 106)
1	3.5	98 68	X		No	Yes
2	7	33 --		X	No	Yes
3	8.5	90 70	X		No	Yes
4	5	91 70	X		No	Yes
5	9	86 31		X	No	Yes
6	6	44 34		X	No	Yes
7	3	68 63	X		Yes/Down	No
8	8	45 35		X	No	Yes
9	7	50 66	X		No	Yes
10	7	97 62	X		No	Yes
11	7	52 69	X		No	Yes
12	10	32 24		X	No
13	5	82 67	X		Yes/Down	No
14	5	29 78		X	No	Yes
15	6	107 43		X	No	Yes
16	7	84 83	X		No	Yes
17	4	70 68	X		No	Yes
18	5	45 81		X	No	Yes
19	4	-- 69	X		No	Yes
20	4	110 72	X		No	Yes
21	7	34 63		X	No	Yes
22	6	75 40		X	No	Yes
23	6	66 40		X	No	Yes
24	7	69 42		X	No	Yes
25	3	59 61		X	No	Yes
26	3	24 --		X	No	Yes
27	4	68 62	X		Yes/Down	No

All 13 placed into 106 by Accuplacer failed the internal test. Score range = [3,8.5] and mean(std)=5.3(1.7)

Of those who placed into 106 by taking MAT090, 13/14 failed. Score range = [3,10] and mean(std)=6.3(1.9)

This second group (placed by taking pre-requisite) had a mean score of 0.55 standard deviations above the ACCUPLACER group.

Testing the hypothesis that the means are the same (using a z-Test) indicates that the means are probably not different. (Because z =1.36 does not fall within the critical region, 1.96, FAIL TO REJECT H0 (also, alpha=.05 < .17). Thus, these means are probably the same.

z-Test: Two Sample for Means
	ACCUP	Pre-Req
Mean	5.30769231	6.285714286
Known Variance	3.1	3.9
Observations	13	14
Hypothesized Mean Difference	0
z	-1.36
P(Z<=z) one-tail	0.09
z Critical one-tail	1.64
P(Z<=z) two-tail	0.17
z Critical two-tail	1.96

Math 090 Pre-Test – 060 Content

(If 9 or fewer answers were correct, student should enroll in Math 060. If 10 or more questions were answered correctly, it was not necessary for the student to sign the waiver in order to be allowed to remain in the class in which he/she was enrolled.)

Student #	# Correct	Highest Accuplacer Score AR EA	Accuplacer Placement Into Course	Took Pre-Requisite	Moved Up/Down	Signed Waiver (Remained in 090)
1	3	ACT – 19			No	Yes
2	5	43 22		X	No	Yes
3	3	28 38		X	No	Yes
4	9	61 56	X		No	Yes
5	5	23 47	X		No	Yes
6	8	86 48	X		No	Yes
7	7	39 53		X	No	Yes
8	11	94 53	X		No
9	3	23 25		X	No	Yes
10	3	32 56	X		No	Yes
11	5.5	46 33		X	No	Yes
12	7.5	77 46	X		No	Yes
13	6.5	53 23		X	No	Yes
14	6	44 32		X	No	Yes
15	4	49 46	X		No	Yes
16	7	24 29		X	No	Yes
17**	8	110 55	X		Yes/Up -107	No
18	4	61 45	X		No	Yes
19**	8	72 54	X		No	Yes
20**	10	55 37		X	No
21**	12	94 55	X		No
22**	11	100 54	X		No
23**	11	83 34		X	No
24	11.5	63 51	X		No
25	4	38 31			No	Yes
26	4	65 35		X	No	Yes
27	5	40 49	X		No	Yes
28	5.5	85 23			No	Yes
29	10.5	72 60	X		No
30	6	78 58	X		No	Yes
31	4	28 34		X	No	Yes
32	8	99 47	X		No	Yes
33	5	57 31			No	Yes
34	6	73 48	X		No	Yes
35	3.5	24 34			No	No
36	5	No scores			No	Yes
37	10.5	99 47	X		No
38	4	51 47	X		No	Yes
39	4	29 21		X	No	Yes
40	1	0 51	X		No	Yes
41	3	76 36		X	No	Yes

**Student was accidentally allowed to use a calculator

The Accuplacer Arithmetic (AR) score is 48% correlated with the pretest score, but the elementary algebra score (EA) showed a very low correlation (16%).

Of those who placed into 090 by Accuplacer, 6 of 21 passed: Score range=[1,12] and mean(std)=7.3(0.7)

Of those who placed into 090 by taking MAT060, only 2 of 14 passed. Score range=[3,8] and mean(std)=5.6(0.7)

The first group had a mean score of 2.1 standard deviations above the Pre-Requisite placed group. The z score lies in the critical region (z=2.12 > 1.96, so we REJECT H0 – also alpha=.05>.034). Thus the means are probably different; students placing using the ACCUP seem to do better than those who enter by satisfying the pre-requisite.

z-Test: Two Sample for Means
	ACCUP	Pre-Req
Mean	7.286	5.643
Known Variance	3.141	6.286
Observations	21	14
Hypothesized Mean Difference	0
z	2.123
P(Z<=z) one-tail	0.017
z Critical one-tail	1.645
P(Z<=z) two-tail	0.034
z Critical two-tail	1.960

Math 090 Pre-Test – 090 Content

(If 9 or fewer answers were correct, the student should remain in Math 090.)

Student #

# Correct

Highest Accuplacer Score

AR EA

Accuplacer Placement Into Course

Took Pre-Requisite

Moved Up/Down

Signed Waiver (Remained in 090)

100 54

72 60

Math 060 Pre-Test – 030 Content

(If 9 or fewer answers were correct, student should enroll in Math 030. If 10 or more questions were answered correctly, it was not necessary for the student to sign the waiver in order to be allowed to remain in the class in which he/she was enrolled.)

Student #	# Correct	Highest Accuplacer Score AR EA	Accuplacer Placement Into Course	Took Pre-Requisite	Moved Up/Down	Signed Waiver (Remained in 060)
1	5	60 29	X		No	Yes
2*	15	59 35	X		Yes/Up
3	7	83 38	X		No	Yes
4	6	85 31	X		No	Yes
5	11	61 24	X		No
6	12	67 35	X		No
7	7	58 35	X		No	Yes
8	6	76 29	X		No	Yes
9	5.5	36 36		X	No	Yes
10	13	73 37	X		No
11	7	54 34		X	No	Yes
12	11	53 29		X	No
13	5	54 38		X	No	Yes
14	2	47 31			No	Yes
15	5	73 44	X		Yes/Down
16	5	75 24	X		No	Yes
17	6	61 35	X		No	Yes

*Student #2 took the Math 060 – 060 Content Pre-Test as well.

Of those who placed into 060 by Accuplacer, 4 passed and 8 failed the internal test: score range=[5,15] and mean(std)=8.2(1.0)

Off those who placed into 060 by taking MAT030, only 1 of 4 passed. Score range=[5,11] and mean(std)=7.1(1.4)

The Accuplacer group had a mean score of 1.05 standard deviations above the mean score of the Pre-Requisite placed group. Because z=1.05 does not fall within the critical region 1.96, we FAIL TO REJECT H0. Thus the means are probably the same.

z-Test: Two Sample for Means
	ACCUP	Pre-Req
Mean	8.1666667	7.125
Known Variance	3.5633	2.7195
Observations	12	4
Hypothesized Mean Difference	0
z	1.05
P(Z<=z) one-tail	0.15
z Critical one-tail	1.64
P(Z<=z) two-tail	0.29
z Critical two-tail	1.96

Math 060 Pre-Test – 060 Content

(If 9 or fewer answers were correct, the student should remain in Math 060.)

Student #

# Correct

Highest Accuplacer Score

AR EA

Accuplacer Placement Into Course

Took Pre-Requisite

Moved Up/Down

Signed Waiver (Remained in 060)

59 35

Yes/Up

Math 030 Pre-Test – 030 Content

(If 9 or fewer answers were correct, student should remain in Math 030. If 10 or more questions were answered correctly, it was not necessary for the student to sign the waiver in order to be allowed to remain in the class in which he/she was enrolled.)

Student #	# Correct	Highest Accuplacer Score AR EA	Accuplacer Placement Into Course	Moved Up
1	7	49 25	X	No
2	1	24 21	X	No
3	6	31 34	X	No
4	2	26 25	X	No
5	9	53 38	X	No
6	3	29 24	X	No
7	9	48 38	X	No
8	5	24 32	X	No
9	7	46 33	X	No
10	5	32 34	X	No
11	4	24 26	X	No
12	5	48 38	X	No
13	3	42 38	X	No
14	3	27 28	X	No
15	1	37 25	X	No
16	6	22 33		No
17	3	48 24	X	No
18	7	34 33	X	No
19	4	43 33	X	No
20	5	No scores		No
21	7	53 24	X	No
22	2	32 22	X	No
23	0	21 21		No
24	4	44 24	X	No
25	3	21 25		No
26	5	46 34	X	No
27	0	24 22	X	No
28	0	No scores		No
29	6	35 39	X	No
30	1	25 38	X	No
31	7	30 0	X	No

Summary

The Accuplacer seem to be accurate in placing students into Math 030, as the results of our pre-test seemed to agree with those of the Accuplacer. For courses above Math 030, however, there seemed to be a great discrepancy between the Accuplacer’s assessment of a student’s skill level and our pre-test’s assessment of those same skills. In the majority of instances, Accuplacer placed students into a higher level math course than our internal diagnostic pre-test. There are several possible explanations for this phenomenon. First, the Accuplacer is a multiple choice test, and our pre-test is “show your work and answer” math test. Students who are unsure of an answer on the Accuplacer can guess and still have 25% chance of selecting the correct answer; that is virtually impossible on our test. Second, students have been allowed to re-take the Accuplacer as many times as they like, with the highest score being used for their placement, while the Math Lab pre-test was given only once. Finally, students knew that there was a waiver attached to our pre-test that stated that they didn’t have to move down to a lower course if they didn’t want to, so we suspect that some students may not have taken the test seriously since there was no consequence for performing poorly.

Another interesting bit of information shown by this pre-test is the fact that students who took the pre-requisite course fared no better on the pre-test than the students who were placed into the course by their Accuplacer score alone. It appears that students are not retaining what they’ve learned in previous math courses, whether those courses had been taken in high school or the previous semester in college. While we have no control over what they were taught and retained from high school, we can try to improve the retention skills of students taking our math courses. Normally, students in the Math Lab must pass each chapter test with a minimum score of 75% before they are allowed to advance to the next chapter. Students who are behind schedule at the end of the semester are sometimes allowed to keep a score lower than 75% on the last chapter test so that they can continue on to the final exam and complete the course. Perhaps if we discontinue this practice, it will alleviate the problem of students procrastinating and try to rush through the last chapters without truly learning the information contained within them. This is something that will be discussed at spring in-service so that all math lab instructors can offer their input on this subject. The more we can increase student retention, the better our students will fare in future math courses.