Assessment

Based on the 1998-1999 assessment results, the communications division attempted to remedy some of the problems observed, specifically, the problem of getting complete and reliable data. Although the division wanted all students to produce a writing sample for placement as an integral part of the initial placement testing procedure, many students enrolled with no writing sample. Some students reported to their adviser that they had been tested when, in fact, they had not. Many students simply did not report to any advisor or the Learning Center for placement and registered in courses without advisement and thus, without an entrance writing sample.

One English instructor completed research on success rates of students placed by means of the writing sample, students placed solely on the Accuplacer, students placed by SAT or ACT scores, and students placed without any test (see Appendix C). The results clearly show that initial correct placement is critical for student success.

For the current year, English assessments consist of development of a portfolio for every student taking an English course. The portfolio is made up of the first essay of each student plus an essay requiring research, usually given toward the end of the semester. This way, every student who registered for an English course was evaluated for the assessment effort. Additionally, this put the entire assessment effort within the control of the English faculty, assuring uniform control of the process. The department has kept portfolios for all students who have completed a writing sample since 1997. Two English instructors evaluated the two selected essays using the 6-trait writing assessment: ideas, organization, word choice, sentence fluency, voice and conventions with a top score of 30.

The speech department assessed student learning by videotaping both the first and the last speech of all students taking Speech 115. The speech instructor implemented the plan during the spring semester of 2000. Students take one speech course, SPE 115, usually prior to their final semester. Since the assessment implementation covered only the spring semester -- and the spring semester is the lowest enrollment semester -- only two individuals from the cohort happened to be in the sample.

Tables 6 and 7 show the English portfolio evaluations upon entry and exit from ENG 121 for the two cohorts.

Of the four from the 1998 cohort, three showed improvement; one did not. The average improvement was 2.25 points, or 7.5% improvement. Of the fourteen from the 1999 cohort, twelve showed improvement. The average improvement was 3.04 points, or 10%.

Table 8 shows the data from the first implementation of the speech course evaluations.

Only two students from the spring semester were part of the selected cohort, so very little information can be reliably inferred. Both students showed significant improvement. The overall data set shows similar results.

Conclusions and Recommendations

The size of the data samples suggests that some data might be missing. Perhaps students seeking associate degrees, who must take at least one English class, are putting these classes off until their second year. The division continues to stress the importance of having students take their composition courses as early in the college experience as possible. The division chair plans to reiterate this request to all advisors during the fall in-service in August 2000. Additionally, the division will solicit the help of all staff to see that all students complete a writing sample prior to placement.

Students in all disciplines are encouraged to use the writing and learning centers. In some classes, students are required to submit papers to the writing center. While no policy will be made regarding this, all faculty will be reminded of the opportunities presented by the writing center regarding assisting their students’ writing.

Both full-time and adjunct English faculty will be required to collect samples from their writing classes. All of these faculty members will also be encouraged to attend communications division meetings, whether teaching for the communications division or for continuing education. This should improve uniformity and consistency within the division. Additionally, the data sampling should be more complete as a result.

Clearly, the speech department’s selected assessment tool requires more students in the cohort in order to become more statistically significant. The department plans to implement the plan for every student in every speech class taken during the fall and spring semesters. Every student who earns an AA or AS degree is required to take speech, so the sample size should be significantly greater in future cohorts. One significant obstacle will be to ensure consistent collection of data by adjunct faculty. The division is considering this issue.

The literature faculty members are developing an assessment tool that will be used for the first time during the next academic year. Also, Spanish was added to the communications division during the 1999-2000 academic year. There is one part-time faculty member, a native speaker of Spanish, who will develop and administer a pre- and post-test to Spanish classes next year.

Changes as a result of assessment

Together with the new coordinator of the writing center, a number of changes and improvements to the Writing Center have been made. Principally, the English faculty members have maintained close contact with Learning and Writing Center personnel and the tutors. While policies and procedures within the Writing Center are not strictly under the English faculty, their input has been, and will continue to be, given top priority by the Learning Center Director and the Writing Center Coordinator. The director and coordinator communicate regularly through e-mail, meetings, and by telephone. The most important changes made in the Writing Center were:

• hiring of a full-time Writing Center Coordinator
• producing a Writing Center mission statement,
• formally presenting the Writing Center and its mission to all faculty at in-service,
• organizing a standard orientation session for visiting classes,
• developing ongoing tutor training sessions.

See Appendix D for a complete list of activities related to the Learning and Writing Centers.

Several English courses use the computer-based writing program, Daedalus. This program facilitates the writing process by providing a platform for students and professors to review papers together at their computers for editing and revision. The program has increased the retention rate for developmental students enrolled in composition classes (see Appendix C).

Assessment

The faculty in the Humanities, Social and Behavioral Sciences decided to assess those program areas using pre- and post-tests. Each instructor has developed (and in some cases has yet to develop) a unique test to use for their course - the same instrument will be used for both pre-test and post-test.

Results

Table 9 summarizes the pre-test and post-test score results for the students in the 1999 cohort who took humanities or social-behavioral science courses in the spring of 2000. Two students in the cohort took only the pre-test.

Conclusions and Recommendations

The assessment results indicated substantial improvement by the few students from the cohort in those courses that participated in the study. Some of the faculty teaching courses in the division did not participate, largely because of internal communication problems. The division consists of mostly adjunct faculty, who traditionally perceive their appointment as primarily teaching and not program development. To increase motivation to collect this data, the division should consider additional compensation for the added work of developing and implementing assessment tools and interpreting results.

The program areas in the Humanities, Social and Behavioral Sciences Division need strong leadership, particularly because of the high number of adjunct faculty. The division is working toward the development of a comprehensive and unified assessment plan for the whole division. The division also plans to incorporate more structured training for all faculty, both full- and part-time.

Assessment

The Occupational Education Division chose to utilize pre- and post-tests as their primary assessment tool. Each program area developed its own instrument. The same instrument is used for both the pre-test at the beginning of the program and for the post-test at the end of the program.

In addition to the pre- and post-test data summarized in Table 10, the Occupational Education Division conducted a 55 question survey of all Occupational Education students currently enrolled to determine their level of satisfaction with their experience at the college. Ten (10) of the questions relate to the student assessment effort and so the results have been included and are summarized in Table 11.

Results

These data show significant improvements by nearly all students, with an average improvement rate of 27%. The Associate Dean for Occupational Education pays careful attention to the details of these data and generally visits with the faculty from each program area to discuss the results for their data set. Finally, most Occupational Education programs require that a certain percentage of their students pass with a 70% level or better. Of the twelve students in this cohort, 58% passed with 70% or better.

A study of the data (adding Very Satisfied and Satisfied together) suggests that students are most satisfied with those items that address basic skills: critical thinking (92.7), speaking (91.1), and problem solving (91.2). Among the lowest values, which are still all rated very high, are the questions dealing with social sciences (82.9 and 84.0) and the arts (82.9). These seem to track well with the fact that very few occupational education students take humanities and social science courses.

Conclusions and Recommendations

The pre- and post-test data indicated that most students were advancing in knowledge of their program area.

Changes as a result of assessment

In an effort to maintain the emphasis on basic skills, the division has pursued a strong integration program by involving faculty from the academic areas in planning and teaching occupational courses. For example, the building trades program is planning to integrate writing, science, and math with building trades by having students develop "environmental impact" type reports as part of the curriculum. The Math/Science Division faculty will provide expert math and scientific advice while the Writing Center and English faculty will be available for writing assistance. The Occupational Education Division continues to work with both the Math/Science Division and the Communications Division to build academic skills into all occupational programs.

One concrete change that was made to incorporate basic skills into Occupational Education programs was to rewrite the AAS general education component. This rewritten specification includes more hours of courses that directly teach important basic skills.

Finally, the Occupational Education Division worked with a member of Student Support Services to develop a workshop and centralize resources for writing cover letters and resumes. The workshop was presented to interested students during the spring of 2000.

Assessment

The Math/Science Division faculty completely revamped the student assessment evaluation from the previous year. The portfolio method was abandoned because the portfolio "productions" yielded so little information relevant to program area improvement. Plus, the additional requirement of such productions was artificial for several courses in the division. Last year's Math/Science Division Student Assessment Report states that "Two of the main objectives of the Math/Science programs – particularly for AA and AS degrees – are that students successfully transfer to a baccalaureate institution and that they succeed in their degree plans." This is the direction the division went with the assessment.

Very few students outside of those in AS degree programs take more than one science and one math course. Also, most non-AS students who take a math or science course are AA students (see Table 12) and nearly all of these students transfer to four-year institutions.

Table 12: Sampling from Spring 2000 Math and Science Courses

The most significant measurable objectives useful for program assessment were determined to be:

• Successful transfer to senior institution and successful "competition" with native students
• Successful attainment of or retention of a job
• Success in personal goals

The instrument used follow-up data of all graduates via telephone or e-mail interviews with the following standardized questionnaire:

If you transferred to another college:

1. Did all of your courses transfer? If not, which ones did not and why?
2. Did your work at TSJC prepare you for your work at your current school?
3. Please make any additional comments regarding the effect of your work at TSJC.

or, if you did not transfer to another college:

1. Did you get a job?
2. If so, is it related to your field of study at TSJC?
3. Please make any additional comments regarding the impact of your work at TSJC.

or, if neither of these apply

1. Did your work at TSJC help you succeed at your educational goals?
2. Please make any additional comments regarding the impact of your work at TSJC.

Table 13: 1999 TSJC Graduates

The raw data indicated that most AA and AS students continued their post-secondary education beyond the two year level. For AS students, at least two-thirds transferred; note that 100% of those contacted transferred successfully to a four-year college. Of those, 100% said either that all of their credits transferred or the only credits that did not, were not expected to transfer (e.g., an earned D grade or certain electives). For AA students, 80% (41/51) of those contacted transferred successfully to four-year schools. Of the AA students contacted, 100% either transferred or were gainfully employed. All of the AS graduates contacted said that TSJC prepared them to successfully "compete" at their current college.

Regarding general comments, several students made remarks indicating that they were happier with the instruction they received at TSJC than at their current, four-year, school. One student said, "instruction at TSJC is better than the instruction at (my current college) - especially in Math and in particular, Mr. Leone." Another student said, "I've attended colleges in other states and found TSJC better because the faculty is excellent and you can actually talk to professors. Also, the small class size is great!" Finally, several students remarked that they were very glad to have had the opportunity to do undergraduate research during the first two years of their college career at TSJC. One student said that she was not fully prepared for her computer science program at the four-year institution because she had never used an Integrated Development Environment (IDE) tool before. Her receiving institution used an IDE, which they expected her to have mastered prior to her junior year and she felt somewhat handicapped by this deficiency. One student said that he had "no idea if we prepared him for the four-year schools" yet because it is too early to make the determination.

While not specifically pertaining to math/science programs, some other feedback from students was that their credits transferred, but they need so many credits in their major that it will still take them an extra year to graduate even if they do not change majors. Also, that "TSJC well out-ranks (my current college) in cooperation with students throughout the different departments - advising, financial aid, etc."

Other data collected

Conclusions and Recommendations:

The Math/Science Division Student Assessment Plan attempted to indirectly measure critical thinking, writing, scientific, mathematical, and study skills through subsequent contact with TSJC graduates. Toward these ends, the division continues to encourage instructors to make the most of the Writing Center to develop writing skills (specifically technical writing skills). In an effort to reduce failure and increase retention, the division participates actively with the Learning Center staff to develop tutoring services and Supplemental Instruction efforts. This is accomplished by recommending top-performing students as tutors and SI facilitators. Doing so has the added benefit of enhancing the experience for these students.

The Math/Science Division’s most significant strength continues to be the excellent and enthusiastic faculty. Despite tight budgetary constraints, faculty members do a remarkable job. Biology students are guided through original research projects, several of which culminate in presentations at regional and national conferences. The Math/Science Division plans to sponsor the Annual Tri-Beta Regional Conference in April 2001. This will provide terrific possibilities for all math and science students, but particularly biology students. Chemistry students are involved in an environmentally sound "small-scale" approach, which uses minimal quantities of chemicals. In addition to the existing Math Lab format, which allows for individualized instruction, math students in certain courses now have the option for computer-aided-instruction delivery. Budget constraints and low enrollments continue to dominate as the significant weaknesses of the department.

Preparation for success of graduates at the baccalaureate level may be a weakness. One of the most immediate ways this is being addressed is to increase both the support for and the number of Supplemental Instruction (SI) sessions offered for science and mathematics courses. The SI program began in the spring semester of 1999 with SI sessions for general college chemistry, statistics, and college algebra. In the fall of 1999, there were five courses using SI sessions (chemistry, physics, calculus, statistics, and anatomy) and in the spring of 2000, chemistry, calculus, and statistics were available with SI. For the fall semester of 2000, SI sessions will be available for those courses, plus Calculus-Based Physics.

In terms of improving the Math/Science Division Student Assessment Plan, more work is required to better integrate it with the VE-135 report (so those graduates do not get two phone calls). This effort will concentrate on maximizing the data-return and quality of that data. Additionally, the Math/Science Plan will attempt to address the issue of students who took classes from the TSJC math or science areas but did not graduate. The Math/Science Division plans to do two specific things: one, all math and science instructors will attempt to obtain all of their student's e-mail addresses; and two, the VE-135 data collection effort will be more closely integrated through interdepartmental planning with the Math/Science Student Assessment Plan effort.

Changes as a result of assessment

The Computer Science department, and the Computer Information Systems department, have purchased Microsoft’s Visual Studio 6.0, an integrated development environment (IDE) for developing software in Visual Basic, Visual C++, and Visual J++. This provides both occupational education students’ and engineering students’ access to state-of-the-art programming and software development tools.

In an attempt to be more responsive to different learning styles, plus the need for distance delivery methods, the Math Lab experimented with a computer-based math program during the fall of 1999. A full-scale pilot was run during the spring semester of 2000 with about two dozen students. This program, provided by Academic Systems, completely replaces the textbook. Computers serve as the primary source of instruction. No home computer or previous computer experience is necessary. Students are guided through the material with computer graphics, animation, and video. Course material is interactive and reinforced with immediate feedback. Frequent self-checks are given for mastery. Both the instructor and student can check the student’s progress. The program allows a student to work as rapidly or slowly as necessary, thus providing great flexibility. The data from the spring semester is encouraging and the program will continue into the next academic year. The hope is to increase retention rates, student success, and interest in mathematics.

The Assessment and Improvement of Student Learning Committee was formed just under three years ago and has spearheaded the effort to produce a student assessment of learning plan for all divisions at all campuses of Trinidad State Junior College. The plan has had institution-wide input from both administration and faculty. The committee has met regularly, developed and met specific attainable goals, and designed schedules and procedures. Most importantly, it has developed a complete feedback cycle to collect and then use assessment data for making recommendations toward upgrading collegiate programs.

The assessment plan is now through one complete cycle and a number of strengths and weaknesses are apparent. The plan is reasonably complete, with an established feedback loop for continuous improvement aimed at producing and maintaining the best programs possible that are consistent with the institutional goals encapsulated in the statement of philosophy.

As with any plan of this magnitude, however, there are a number of weaknesses. Central among these is that the plan lacks solidarity. This stems from at least two things: one, academic and vocational areas were historically separate, a separation that has lessened significantly over the past decade; and two, the institution’s division structure was used to build the plan, i.e., each division developed its own assessment plan from its own perspective. To remedy this problem, the committee plans to focus on broadening ownership of the general education philosophy statement, the assessment plan itself, as well as the implementation phase of the plan. We also plan to continue to build a sense of community and shared goals among all areas of the college.

Another significant problem is that, while many divisions do great work collecting data, too few adequately analyze that data. Additionally, the important aspect of linking student assessment to institutional planning and budgeting needs to be refined. This is an aspect that the committee will develop intensively during the next few years by working to establish accountability of divisions’ faculty and administration to the committee and to the Vice President for Instruction. The committee also plans to better integrate the many divisions’ separate contributions by looking carefully at the existing data and separate recommendations made by each division.