Assessment and Improvement of Student Learning

Annual Report 2001

The AISL Committee published the report for the October 2000 North Central Focus visit and posted it on the TSJC web site (www.trinidadstate.edu.edu/AISL/NCAreport.htm). It details assessment efforts prior to Summer 2000. In part based on items identified in the report, the committee made several goals for the 2000-2001 academic year, specifically:

* to facilitate the assessment piece of the North Central Focus visit in October 2000;

* to develop a specific set of regular, annual, assessment activities;

The committee developed a yearly calendar at the beginning of the fall semester 2000 and began implementing the specific items in the calendar. Specifically, the pre- and post-test data that most departments generate were given formal submission dates. This data has been collected with the specific request to include analysis. A summary of these data follows in the departmental sections below.

The NCA Consultant/Evaluators made the following observations related to assessment:

1. "There remains considerable work to be done in the assessment of general education at the college."
2. "The program to assess student learning should: (a) emerge from and be sustained by a faculty and administrative commitment to excellent teaching and effective learning; (b) provide explicit and public statements regarding the institution's expectations for student learning; (c) use the information gained from the systematic collection and examination of assessment data both to document and to improve student learning."

 

3. "The [college] should add more faculty to the assessment committee."
4. "Reported data samples [are] clearly insufficient."
5. "[The assessment] program [is] stalled."
6. "Most changes reported were within the realm of student support services."
7. "This report [submitted to NCA] does not identify a single pedagogical or curricular change made as result of assessment."
8. "[college faculty] lack of professional development in assessment [and so should] attend annual NCA meeting."

Item 1 reiterates what the AISL Committee had identified and made a priority. Toward this end, specific, measurable, general education objectives have been written and assessment measures for some are now in place. Additionally, several faculty are building a pilot project for implementation in Fall 2001. These efforts are elaborated below under General Education.

This report plus efforts by the AISL Committee to publish work on the TSJC internet (www.trinidadstate.edu/AISL) directly address Item 2.

In response to item 3, the NCA team did not seem to understand that many of the regular members of the committee (listed below) are faculty with some non-teaching duties. Consequently, the team seemed to think that the committee had too few faculty. This faculty/staff mix will be carefully monitored as the committee composition evolves.

Regular Members:

Francis Cuckow Counselor in Student Advising

Deb Haverfield EMT Faculty at San Luis Valley Center

Craig Larson Communications Faculty at TSJC

Robert Philbin Math/Science Faculty and Committee Chair

Bev Stimson Communications Faculty at TSJC

Sandy Veltri Business Faculty at TSJC

Ex-Officio Members:

Frank Armijo Vice President of Instruction

Carol Cimino Dean of Continuing Education

John Giron Dean of Student Services

Carol Rankin Director of Student Support Services
Paul Montera Dean of Occupational Education

Jim Dawson Dean at San Luis Valley Center

Item 4 was addressed at the first AISL Committee meeting in August 2000 wherein the sample population was identified as all of the first-time, full-time, freshman.

Item 5 requires administrative and faculty buy-in to the value of the assessment effort above simply "making NCA happy." This solidarity can only develop as faculty and staff see real change occur due to results of the assessment effort. This is the principal reason the AISL Committee began to stress Course Level Assessment at the mid-year mark. As a result, this report is dominated by course level assessment reports.

Items 6 and 7 stem from the fact that most of the changes listed as a result of assessment were changes made in the student services area, an area integrally related to course mechanics at TSJC. This comment refers to the Supplemental Instruction (SI) program, tutoring, and the Writing Center all having direct relations to course mechanics in many courses on campus.

Item 8 has been addressed in a number of ways. First, the AISL Chair gives a presentation at each in-service for faculty. Second, the 2000-2001 AISL Chair, Robert Philbin, and Dr. Armijo, the Vice-President for Instruction, attended the annual North Central Association meeting in Chicago in April 2001. Finally, Bev Stimson, Jim Dawson, and Donna Meyerholz attended the AAHE Assessment Conference in late June, 2001.

One major concern voiced by the North Central Association consultants was the lack of a comprehensive assessment plan for the general education areas.

Following is the general education philosophy of Trinidad State Junior College:

From this philosophy statement, measurable objectives were generated in four areas: communication, critical thinking, problem solving, and intellectual maturity. The draft copies were circulated via e-mail to all faculty several times; the final list is:

1. Communication Objectives:
1. Read and comprehend college level work
2. Explain and defend ideas orally and in writing
2. Critical Thinking
1. Argue from cases to general principles
2. Argue from principles to specific cases
3. Identify unstated assumptions
4. Judge credibility of statements
5. Interpret evidence
3. Problem Solving
1. Identify the most important aspects of a problem
2. Propose and defend solutions to qualitative problems
3. Solve quantitative problems using mathematics and computer tools
4. Recognize, identify, and explain interrelationships between specialized and general modes of thought and fields of study.
4. Intellectual Maturity
1. Demonstrate openness to new ideas
2. Demonstrate desire to learn
3. Practice community service
4. Demonstrate civility - responsible use of knowledge

The communications objectives are well covered by compiling the portfolio-based data from the Communications Division assessment in English 121 and Speech 115, courses all students who complete a degree must pass. Some critical thinking and problem solving skills are assessed within the competency-based Mastery Math program. While students in certificate programs do not take separate general education courses, their general education competencies are measured within the course-level assessments.

In January 2001, at the in-service training, the AISL Chair presented a variety of options open to faculty for assessing the remaining objectives. A great deal of discussion followed, mostly via e-mail, where various plans were floated - portfolios, exit exams, capstone courses and capstone projects. Portfolios are cumbersome, even in digital form; exit exams suffer from the fact that students have little, or artificial, motivation to do their best on them; and capstone productions suffer from a variety of logistics problems. Finally, the idea of embedding general education questions at random into existing courses seems to answer these criticisms best.

A small group of faculty are working on questions which address the remaining general education objectives. These questions will be embedded into regular tests in various courses, chosen at random, which are part of the general education core. Students should be well motivated to do well on the tests because they are a natural part of these courses. The questions are designed to be discipline independent, so results ought to be inter-comparable across sources. The students’ student identification numbers will be linked to their scores on these tests so that the AISL Committee will be able to track how well students perform at different points in their progression through the general education curriculum. Finally, this approach should answer the problems associated with "churn," the phenomena common at community colleges wherein students often do not progress through courses in the correct sequence or uniformly from semester to semester. This program is scheduled to be piloted beginning Fall 2001.

One goal for 2001 was, and is, to have regular assessment efforts in every single course offered for credit. In the Spring of 2001, the AISL Committee requested that faculty experiment with at least one course and submit results of assessment for those courses to their respective division chairperson plus the AISL Chair. Roughly half of all full time faculty did this and several part-time faculty were involved in this effort. Report summaries follow in the departmental sections below.

The Supplemental Instruction program, administered by Student Support Services, continues to grow in terms of number of courses using it and number of students participating. There were eight courses with SI labs during Spring 2001, Statistics (Mat 135), English Composition (Eng 121 - IT1 and IT2), Psychology (Psy 101), Speech (SPE 115), two math courses (Mat 110 and 030). The participation in SI labs over the past several semesters is as follows:

Since the program began Spring '99, SI has served 236 students in college level, academically rigorous courses. SI serves academically under-prepared students in the ITOP mentoring program, students at distant sites and students on the home campus. SI is not a remedial program, but targets historically difficult courses and offers peer led academic support.

AAS degree programs assess students by emphasis area through exit exams, demonstrations, and in some cases such as nursing, state board exams. The AA, AS, and AGS degree programs plan to implement assessment of student learning through a compilation of course level assessments (a portfolio of assessment work which follows each student) as a holistic view of student's progress through each emphasis area.

Each division will be responsible for the identification of competencies and outcomes for their area. It will also be the responsibility of each area to identify how gain is to be measured and assessed. A follow-up report will be written yearly to the college administration identifying the standards, measures, and assessment tools from each division with assessment results. This report will contain findings, interpretation, and recommendations from the respective faculty, division/department, or program to the administration for budgetary, staffing, and program issues.

CSU and USC send data regarding TSJC transfer students

Trinidad State Junior College programs are organized in a division structure, each division overseen by a division chairperson. These division chairs were responsible for compiling the assessment reports from the program areas under them. All faculty were requested to summarize their assessment efforts, with an emphasis on analysis and recommendations for improvement in their respective program area(s), and submit those summaries to their division chair for compilation. This requirement was, apparently, not uniformly understood and so the reporting here is incomplete.

Mimi Zappanti, Program Head

General Education Diploma graduates were listed with their accomplishments. Most of the GED graduates enrolled in TSJC programs, both certificate and degree programs. About one third of the graduates obtained jobs at local businesses.

Pat Huhn, Division Chair

Composition

From the portfolios selected, many had only one of the essays. This might be explained in several ways: the students dropped out from the course or from the college; the teachers did not submit the required essays. Twelve student portfolios were ultimately included in the study.

Four members of the division participated in the evaluation. These instructors decided that this year’s assessment would focus and make its recommendation based on the individual traits. The before and after scores are reflected on the accompanying chart. These scores were totaled to note the areas needing the most attention and the strongest areas of improvement.

As the table shows, the greatest improvement lies in the areas of Ideas and Sentence Fluency. The area needing most improvement is Organization. More attention has to be given to Word Choice and Voice.

Recommendations regarding Method

The division has recommended to the coordinator of assessment that the cohort from which samples are drawn be the list of the current year’s graduates. That way the sample would be assured of having the two essays included. All teachers of composition will be notified in writing of the essays to be turned in for assessment. Notification heretofore has been orally.

A portion of division meeting will be devoted to discussion of improvement of organization skills. Attention will be called to areas of the textbook and handbook that focus on these skills. Members of the division will also explore the areas of Ideas and Sentence Fluency and the reasons for their improvement.

Speech

John Gilmore, speech instructor, tape-recorded the first and final speeches from each of these classes. He also submitted to the division his evaluation sheet (attached) for the speeches. Two members of the division used this evaluation sheet to evaluate these two speeches from students in the sample.

Frank Armijo, Division Chair and V.P. Instruction

Early Childhood Education

Pre and post test scores for the Early Childhood Program courses are summarized in table below.

Regarding the ECP 215 pre/post-test, I have decided to revamp this test since it involves too much instructor evaluation of the answers. With the exception of my special education student, all the students made acceptable gains in their knowledge of the subject matter as tested by the test

Regarding the ECP 111/112 test, the most frequently missed question was question 48. It is apparent that I need to cover Piaget and his stages of cognitive development more carefully. I will prepare a more visual handout covering Piaget for the next time I teach this class.

Literature

One section of Lit 115 gave a pre/post test with the following results: 17 pre-tests averaged 43%, 3 post-tests averaged 74%. No analysis. The PictureTel section of Lit 115 reported the following:

This was a picture-tel class and, in addition to college students included high school students in the five-year program. The class included two college students at the TSJC site, two students from Trinidad High School at the TSJC site, two students at Primero High School , two from La Vita High School, four from Hoehne High School, and nine college students from the Valley Campus. I observed the following:

1. The nature of a literature course does not lend itself to the Picture-tel. While it is supposed to be interactive, that is, picture activated by the speaker, it doesn’t always work that way.
2. The high school students have not taken a college composition course; most were unable to produce a college-level essay, although I did discuss the writing of the essay. Their tests, for the most part, reflected the same problem since they were primarily essay.
3. The high school students frequently missed classes for school activities: participation in sports, Knowledge Bowl practice, assemblies, etc. None ever sought help for classes they missed.
4. The face-to-face interaction was missing. The college students at the Valley Campus did very well in responding; the others did not.
5. Over all, the pace of the class was slowed, and we were not able to cover the material we should have.
6. The only thing that saved some of their grades was their drama presentation, the last assignment. This seemed to lend itself to the enjoyment of the class and most did very well. Most presentations reflected an understanding of human nature reflected in the play they had chosen.
7. Because of the above difficulties, many readings and material usually covered in this class were not covered.

Philosophy

Philosophy 111, 112, and 113 were assessed with a pre/post-test both semesters; the results are summarized below.

The philosophy professor, Dr. Durland, summarized: "The data tells me that the scores on the post test this year are low compared to last year. Last year, I included the post-test as part of the final examination. Students studied harder and got higher post test scores. This year, I told them the post test did not count as part of their grade. They took the test as an after-thought, either before or after their final test during exam week and the results show that the test is not valid due to this and other problems. Many students begin classes later and don’t take the pre-test so I hold off the pre-test for them until after I give a preview of the whole course. Some students therefor take a pre-test after the course begins. I am looking for another method for next year. The most accurate measurement remains my four major test, written papers, attendance and discussion." Regarding the fall data, summarized below, he wrote "Pre-test, Post-test scores accuracy is affected by the following variables: (1) the test cannot be given after the first class period because students will have begun the study of the subject by that time, (2) however students are notorious for joining classes at TSJC even after 2 weeks have passed, (3) technological problems eliminated the chance of distance learning students to participate: 3 student at Alamosa in PHI 113 were unable to participate due to a breakdown in e-mail communication methods. All in all 9 students were not available on time or because of technical problems to take the pre-test."

Physical Education

The bowling course was assessed using a final exam, which covered rules and terminology. The scores ranged from 20% to 85%. No analysis.

Psychology

Several sections of Psychology 101 were given two pre/post tests. 53 students took pre-test #1 for an average of 36%; 40 students took this test as a post-test for an average of 56%, a 20% gain. On test #2, 35 took the pre-test yielding an average score of 0.5 and 34 took the post-test for an average score of 7.5 (no maximum possible was reported for this test but the top four scores were 15, 19, 19, and 20).

Sociology

Sociology 101 and 102 were assessed with a pre/post-test with the following results:

Note that the "a" and "b" refer to the instructors, Hicks and Kaplowitz, respectively. No analysis.

Robert Philbin, Division Chair

The Math/Science Division faculty had agreed to use the state mandated VE-135 post-graduate study to indirectly evaluate student success and thus departmental goals. The NCA focused visit team, however, indicated that this was insufficient because such assessment is indirect and NCA requires direct assessment measures of student learning. Consequently, the Math/Science faculty decided to spend this past year focusing on assessing at the course level.

VE-135 Study

The Math/Science division continued involvement in the VE-135 study and contacted all of the Associate of Science (AS) graduates from Spring 2000. The comments were, as last year, very positive regarding faculty in the division plus all of the students interviewed said that they felt they received at least adequate, if not excellent, preparation for work at their four-year transfer institution.

Mathematics Assessment

All degree seeking students, plus some other students, take mathematics courses, many of which are taught in the Mastery Math Lab. The Math Lab is based on the competency-based concept of mastering each level at the 75% level or better before moving to the next level. This year, since almost all Associate of Science students plus many Associate of Arts students take College Algebra, the math faculty began analyzing College Algebra, Math 121. Additionally, Math 110, Problem Solving in College Math, which is taken by nearly all Associate of Applied Science (AAS) students, was assessed using a competency based final exam. The department’s plan is to expand this effort to Statistics, MAT 135, a course taken principally by Associate of Arts (AA) students.

College Algebra, like all Math Lab courses, is a competency based course with common tests used by all sections, both math lab sections and lecture sections. The table below summarizes the number of times students attempted each of the required tests in all sections of MAT 121 in spring 2001. Note that students must pass a quiz at the 75% level or better before proceeding to the next quiz, so . The huge drop-off in number of student attempts may indicate that about half the students enrolling in MAT 121 are not prepared. 64 students started the course, 24 dropped, 16 withdrew, and 24 received a grade for the course. Their grades consists of an average of these eight (8) scores plus a homework score. The final exam, which counts twice as much as each quiz, is not a mastery test; students take it once. The average score on the final exam was 52%.

The MAT 121 faculty tallied the correctness frequency for each problem on each of the eight quizzes and final exam. The problems where more than 40% of the students attempting the problem missed were flagged for faculty review. Those quiz questions were, for the most part, not surprising; difficult concepts and multi-concept questions consistently generated the highest error frequencies. For example, questions involving exponents, particularly negative exponents, consistently generated error rates from 43% to 69%. Other problems which suffered from high error rates were factoring polynomials, simplifying complex fractions, rationalizing denominators, solving problems that involve rational expressions, finding functional inverses, logarithms, word problems, complex number problems, and problems which require multiple steps. The math staff will review all of the quiz questions with higher than 40% failure rate and make specific recommendations for pedagogical changes and quiz changes. The most important problem to be addressed in this course is the drop and withdrawal rate. In the fall of 2001, the math faculty will receive class lists with Accuplacer scores; this will identify students with substandard placement scores and those without placement scores. Those students who have taken, or claim to have taken, prerequisite courses will have to have their records evaluated on a case-by-case basis. This will be done during the first week of classes. Further, the math faculty teaching math mastery courses will strongly encourage students take the first quiz prior to census date.

Math 110, Problem Solving in College Math, used a comprehensive final exam for the assessment instrument. The instructor, Debbie Ulibarri, analyzed each question based on the percentage of students answering correctly. Questions ranged from 14% to 100% with an average of 74% answering correctly. Next, those questions where fewer than 70% of the students answered correctly were scrutinized and comments written. Ulibarri plans to use these comments to improve the course the next time she teaches it. As an example of the comments, the question wherein only 14% of the students answered correctly was a "fraction problem .. quite difficult," a question designed to see which students had gained knowledge beyond the norm. Questions that required multiple steps or complicated procedures were the most-missed problems; this is not unexpected. A large fraction of such questions were annotated with "need to spend more time on this concept," and several were specifically noted as questions that need re-wording.

Science Assessments

The biology and geology programs used comprehensive final examinations as the course assessment tool. Each question was analyzed for the number of students answering it correctly and incorrectly. Trends were identified and noted for instructors to address the next time the course is offered.

The physics program, which consists of the algebra-based sequence for pre-health and other science majors and the calculus-based sequence for engineering majors, were assessed by analyzing the comprehensive final examinations at the close of each semester. The instructor, Robert Philbin, analyzed each question and how well students performed on that question. He then made notes on each question for use when he offers the course again next year. Most of the comments indicate either poorly constructed questions (because the question was a fill in the blank type, students assumed that they should not have to do any additional work but rather just write down the answer) or inadequate time spent learning some critical concept (for example, students performed poorly on "free body diagram" questions and Newton’s Third Law questions). Philbin will refer to these notes (http://www.trinidadstate.edu.edu/MathSci/physics/physAssess.htm) during the course the next time it is offered.

Neither chemistry nor astronomy were formally assessed this year, but will be included in the assessment process next year. The General College Chemistry course sported a very active Supplemental Instruction (SI) program, the results of which continue to indicate that the program helps students achieve at a higher rate than without it.

Conclusions and Recommendations

The Math/Science area has held off one year in determining what, exactly, to do to assess the Associate of Science program level. The most likely idea at present is to use science research reports and evaluate them using a common rubric at each year’s end. The department will continue to expand its formal course-level assessment efforts during the next academic year.

The Math/Science Division most significant strength continues to be the excellent and enthusiastic faculty, both full-time and adjunct. The division continues to encourage instructors to make the most of the Writing Center to develop writing, and specifically technical writing, skills. In an effort to reduce failure and drop-out rates the division participates, and plans to continually enhance participation, with the Learning Center staff to develop tutoring services and Supplemental Instruction efforts. For Fall 2001, S.I. sessions will be available for BIO 201, CHE 111, MAT 135, MAT 201, PHY 111, and PHY 211.

Judie Stickel, Division Chair

PN Program

The Pre-Test/Post-Test Scores for PN students for the 2000-2001 academic year were:

In reviewing the assessment test for the PN program, we determined that there is a fairly heavy weight placed on the fundamentals aspect of the program. Many of our students come in to the program with this basic fundamental knowledge and skill because some of them have worked as Nurse Aides. With this prior knowledge, our pre-test scores are higher than what we expected. The post-test scores are adequate, but the difference between the pre and post scores is minimal.

It is our plan to revise our assessment tool, to have a lighter weight placed on the fundamental aspect, and a heavier weight placed on the more complex information that is taught in the program. This new assessment tool will be in place for the start of the Fall 01 semester.

Supplemental Instruction is an academic support program that uses peer assisted study sessions to help students process course content. SI sessions are led by trained peer leaders who model how to learn with what to learn. Students develop essential study skills using the content and materials from the course being facilitated.

Instructors campus-wide are invited to request SI facilitation for any traditionally difficult course. This semester, SI in Trinidad focused on serving academically under-prepared students placed in college level courses with academic support through the ITOP program.

*These students were placed in ENG 121 with SI as academic support

Students in both sections of Eng 121 tested below college level on Accuplacer assessment, and are functioning effectively in a college level core course. All students in Statistics (Mat 135) who chose to participate in SI this semester were failing prior to beginning facilitation. The Supplemental Instruction program has the potential to make a marked difference for academic success and reduction of attrition rates for students enrolled in PicTel distance delivery courses at satellite locations, at-risk, and for students enrolled in historically difficult courses on the campus at-large.

Mike Minic, Division Chair

No reports were received for Graphic Design or Multimedia Arts.

Accounting

A comprehensive report was received from Sandy Veltri; a summary follows.

There were four courses that contained an assessment component beyond regular testing over material covered in the course. Those courses included:

ACC 121 Principles of Accounting I

ACC 122 Principles of Accounting II

ACC 211 Intermediate Accounting I

ACC 212 Intermediate Accounting II

For several years a pre-test has been administered to the students in the ACC 121 course. This pre-test is multiple choice test with essay questions. After review of the test, the instructors decided that such a test does not really show a gain or an improvement in student learning. The instructor for the last several years has assigned a project to each of her classes and she decided to take the project(s) and build a capstone project for assessment of student improvement and learning, Following is an explanation of each project, copies are attached, and relation to each course syllabi is included.

Course objectives include performing the following tasks with a minimum score of 70% :

1. Provide an understanding of the fundamental principles, terminology, techniques and practices of accounting.

2. Record in the books and accounts, transaction for service, retail and wholesale concerns

3. Prepare, evaluate, and present the following basic financial statements: Balance Sheet, Income Statement, Owner's Equity Statement

4. Provide a basic understanding of accounting information systems and internal control procedures.

5. Evaluate and determine the proper accounting treatment and presentation for cash, receivables, investments, inventory, plant assets and liability accounts.

The project for this course (included as Appendix A) is a mini-practice set that requires the student to complete a month's worth of transactions. The practice set takes the student through the whole accounting cycle. Students must be able to apply basic accounting principles and know accounting terminology ( course objective # 1 ). The students must be able to evaluate and record transactions to the appropriate journal, complete and evaluate the financial statements ( courses objective #2 and #3 ). Students are able to determine the importance of internal control ( course objective #4) and see the relevance of proper accounting treatment and presentation of the accounts (course objective #5).

For the 2000-2001 school year, there were 30 students (1st semester had 21 students and 2nd semester had 9) who completed a mini-practice set. Following is a chart which breaks down their scores:

Of those that scored below 69%, excessive absences by the student is partly attributable to the low score.

It seemed that many of the students in the first semester had difficulty with the concept of adjusting and closing entries. This is the area where quite a few of the students had a problem. They either did not record the adjusting and closing entries or they were incorrectly recorded. There was also difficulty with posting of data from the specialized journals. Some of the students had difficulty knowing when to record a transaction to a specialized journal and they journalized the transaction into the general ledger

The instructor altered her presentation and spent more time on adjusting and closing entries, specialized journals, and posting of specialized journals for the Principles of Accounting 1 second semester students. There was an improvement in student scores as a result. There were not as many errors in those areas and students performed at a higher level. Instructor will change course objective #1 and #4 to read "demonstrate" not "provide"

Course objectives include performing the following tasks with a minimum score of 70% :

basic principles of recording, maintaining, and evaluating transactions which occur in partnerships and corporation

sound financial and ethical practice in presenting working papers for a partnership or corporation, or capital stock transaction. specifically in the areas of dividends and retained earnings

techniques and practical application of accounting for long-term liabilities. investment analysis and strategies.

Below is the project that students are assigned. The students are asked to complete the project as if they would be forming a partnership with a fellow student.

Instructions: You and a partner want to open a business in your hometown. You have already gathered market research and know that the business of your choice should be successful. Your tasks will bc to:

* complete an Income Statement and Balance Sheet for the first year of business, the third and fifth year of

business. (objectives #1.2.3. & 4)

* produce a cost analysis of all products or services you plan on selling (objective #2).

* estimate and justify reasoning for mark-up % you select (objectives # 1. 2.4 ).

* estimate how many clients/products will need to be served/sold on a monthly basis to break-even or make a profit (objective #1).

* prepare a master budget for !he first year of operation (objectives # I. 2. 3. & 4 ). (Some items to include:

Rent/Mortgage Payment, Liability Insurance, Utilities, Telephone. Labor. Materials. Supplies. What else might you include?)

Identify how profits, losses, and liability will be split between each partner (objective #2).

* be prepared to justify your answers to me or any other stakeholders.

In the Fall of 2000, six students were enrolled in Principles of Accounting II and in the Spring of 2001, there were ten students enrolled. Students were graded on each component of the project and a scale of 1-5 (five being the highest) was used to assess the projects.

No student attained the mastery (5) level in either semester. All students scored at or slightly above the proficient + (4) level. Students who are striving to earn an A.A. Degree with an emphasis in Business Administration must enroll in Principles of Accounting I and Principles of Accounting II. Those students who are enrolled as A.A. Degree candidates with an emphasis in Accounting must continue on in accounting courses and are given the opportunity in Intermediate I and Intermediate II to finalize and show an attainment of the mastery (5)." 1 level.

The project required students to research and utilize tools outside of the classroom (i.e., Excel, the Internet, the library, the Wall Street Journal, etc. ..). Instructor expected students to have this knowledge base and unfortunately not all students do. The knowledge base for the accounting concepts is there, but the expectation of research and use of other tools is a poor assumption on the instructor's pal1. Time will need to be set aside in the body of the course to review and instruct students on these tools --

Course objectives state that students will review, be able to explain. and/or evaluate/demonstrate:

importance of financial reporting and the accounting profession.

the conceptual framework of accounting and the development of accounting standards.

the accounting process and review, complete and evaluate the Balance Sheet and Income

Statement more completely.

the time value of money as it related to accounting applications.

the following components of financial statements:

-Cash and Receivables

-Inventories

-Noncurrent Operating Assets

Students are asked to take the project they completed in Principles of Accounting II and refine the project, incorporating instructor's remarks (Cash Flows Statements for the 181, 3rd, and 5111 year are required) and are expected to make corrections accordingly. Student can also decide that they want to incorporate their business. (For those students who had other instructors, they are expected to complete the full project.) The same scale is used. Intermediate Accounting I is only taught in the fall semester and in the tall of2000, there were twelve students enrolled. Three of those students earned a proficient + ( 4) score and two of those students were at the deficient ( 1) level. These two students had excessive absences and one student withdrew from the course. The other student asked for an extension on the project. The extension was granted and the student earned a proficient -(3) score.

As in the Principles of Accounting II class, instructor learned that not all students came with the same knowledge base in use of outside tools. Instructor had to review how to use Excel, the, Internet, and other outside resource tools, so students could utilize them. Time needs to be allowed in the course syllabi to review usage of these tools.

Course objectives state that students will define and demonstrate an understanding of the theoretical background and the practical application of the accounting theory as it relates to the following areas:

Owner's equity

Investments in debt and equity securities

Revenue recognition

Leases

Income taxes

Pensions and other retirement benefits

Accounting changes and error corrections

Earnings per share

Financial reporting

Financial analysis

Students were asked to take the project they have been developing since Principles of Accounting II and to complete a business plan. Instruction was given on the drafting and completion of a business plan with handouts and a sample business plan. Students were asked to conduct a more thorough research of the need for their proposed business and to forecast it's success in the community of their choice. (Appendix B shows a copy of one handout distributed to students, along with a copy of the reviewer's notes. Copies of the business plans are on file in the instructor's office.) The business plans were given to an outsider reader, a local banker in the community, for his assessment and rating. The instructor did not rate or grade the projects. A rubric, just like the one on page four, was given to the rater with instructions. The rater was allowed to rate between the scales; i.e. , 4.35 or 3.75. Students and the instructor met with the banker and each student presented their "business plan" to the banker -just as they would if they were seeking funding from the bank. Four students participated in this capstone project this year. On a scale of 1-5, five being the highest, following are the scores:

**This student did not turn their project in on time to be evaluated by the rater. The student did make a presentation to the banker and she was able to give him a business plan. The rater was able to quickly review her project and advise her on its strengths and weaknesses and gave her suggestions for improvement. This student is not graduating (although she is technically classified as a "sophomore'.) Since the purpose of Assessment is not to penalize the student, but to improve on their learning and understanding of the content area, the instructor and student decided that she would be able to complete the project in Cost Accounting and re-present to the outside rater.

Objective #8 is the only objective that the business plan does not address -especially if the student chooses to form a sole proprietorship or partnership for their final report, as all the students did. All other objectives were demonstrated in the capstone project. The reviewer was very impressed with the projects and commended the students for their work. Again, his comments for each individual plan (Students 1 -Student 3) are in Appendix B.

Civil Engineering Technology

Program Level Courses to be assessed:

The following required courses are to be taken by all Engineering Technology students:

• CIT 110 Freshman Surveying I
• CIT 111 Freshman Surveying II
• CIT 130 Intro to Engineering
• DRT 111 Technical Drafting II
• CIT 251 Statics

The TSJC Engineering Technology Department is committed to quality education and this Assessment Plan is designed to provide a process for quality control and educational effectiveness. The assessment process will measure the effectiveness of the instructional delivery and reinforcement of the objectives previously mentioned as well as measuring the integration of academic and vocational skills within the curriculum. The following narrative demonstrates the processes and instruments used to accomplish these tasks.

During Freshman Surveying I the instruction concentrates on numerous surveying activities using mathematical calculations and processes. This course addresses problems using differential leveling and angular measurement techniques in which basic mathematical skills (addition, subtraction, multiplication and division) are necessary. Each student will be given a series of questions and or problems related to these topics in the pretest and then the identical questions will be given after the unit of instruction is completed and then again during a final examination. An analysis of the results will allow the instructor to determine if mastering of the vocational objectives was accomplished.

During Freshman Surveying II the instruction concentrates on various advanced level vocational and academic skills. This course contains subject matter which deals with advanced angular measurement using various advanced equipment, various legal terms and opinions as they correlate with surveying profession and specific construction surveying techniques utilized within the surveying profession. This course will allow the instructor to evaluate advanced mathematical skills; basic and advanced reading and comprehension skills; basic reading skills as well as basic and advanced vocational skills. Each student will be given a series of questions and or problems related to these topics in the pretest and then the identical questions will be given after the unit of instruction is completed and then again during a final examination. An analysis of the results will allow the instructor to determine if mastering of the vocational objectives was accomplished.

During Introduction to Engineering the instruction concentrates on various basic level vocational and academic skills. This course contains subject matter, which deals various historical facts and achievements within the engineering profession. The instruction is designed to raise and awareness of the engineer’s role in society and the tools such as problems solving and mathematics, which are inherent within the profession. This course will allow the instructor to evaluate basic and advanced reading and comprehension skills; basic reading skills as well as basic and advanced vocational skills. Each student will be given a research project, questions and or problems related to these topics in the pretest and then the identical questions will be given after the unit of instruction is completed and then again during a final examination. An analysis of the results will allow the instructor to determine if mastering of the vocational objectives was accomplished.

During Technical Drafting II the instruction concentrates on various basic level vocational and academic skills. This course contains subject matter, which deals with basic and advanced drafting concepts using both computer-aided drafting and manual board drafting skills utilized within the drafting profession. This course will allow the instructor to evaluate advanced mathematical skills using geometric relationships in two and three dimensions; basic and advanced reading and comprehension skills; basic reading skills as well as basic and advanced vocational skills. Each student will be given a series of drawings and or problems related to these topics in the pretest and then the identical questions will be given after the unit of instruction is completed and then again during a final examination. An analysis of the results will allow the instructor to determine if mastering of the vocational objectives was accomplished.

During Statics the instruction concentrates on various advanced level vocational and academic skills. This course contains subject matter, which deals with advanced trigonometric solutions and advanced problem solving. The course also emphasizes the necessary skill and techniques utilized within the engineering design profession. This course will allow the instructor to evaluate advanced mathematical skills. Each student will be given a series of questions and or problems related to these topics in the pretest and then the identical questions will be given after the unit of instruction is completed and then again during a final examination. The statics pretest scores will be assessed separately from the previously mentioned pretest questions. Engineering Technology students enrolled in the certificate program do not have to enroll in the Statics class. All Associate of Applied Science degree students are required to complete the course. An analysis of the results will allow the instructor to determine if mastering of the specific advanced vocational objectives was accomplished.

Questions from each course will be compiled in an Engineering Technology Pre-Test which will be administered to each of the first semester engineering technology students. Test results will be analyzed and the results fed back into the curriculum to ensure continued academic and vocation success within the Engineering Technology Curriculum.

Computer Information Systems

A comprehensive report was received from Charlene Duran; a summary follows.

In CIS 118, students are introduced to the five software packages that comprise the Microsoft Office Suite. At the beginning of each unit, the workplace scenario, terminology, and expected outcomes are delivered. Through a combination of lecture and hands-on activities, the unit content is covered. The student is then assigned projects to enforce learned concepts.

The textbook is easy to read and follow. Projects have step-by-step instructions and referrals to pages where the concepts were covered. The majority of the students should be able to complete the projects with 70% or better. Most points are lost to poor grammar, poor spelling, and failure to proofread assignments. Some either do not read the directions or do not comprehend what is expected as they complete assignments. Of the 5 students failing, all 5 had more than 10 absences.

In the past, the hands-on quiz was given as a final project, but students complained that it was difficult to remember all five software packages. This semester, the hands-on quiz was broken down into 5 separate quizzes and one was administered at the end of each software package covered. The quizzes request specific information, but do not limit the way it is presented. With the software still fresh in mind, scores were high and students enjoyed the ability to complete projects without much restriction. Again most points were lost to poor grammar, poor spelling, and failure to proofread assignments.

The student grades were maintained separately for course work and the hands-on quizzes. As I review the two grades, they are consistent. It is apparent that students can successfully use the software upon completion of the course to complete a variety of workplace activities. However, much more emphasis needs to be placed on spelling, grammar, reading, comprehension, and basic math. Regular class attendance is also necessary as concepts build upon one another.

The comprehensive project is a very effective method for assessing student learning. It also gives the student confidence in their newly acquired skills. I intend to develop similar projects for each of the courses I teach.

Computer Networking Technology / CISCO

Lige gave lots of tests and here are the results. No analysis.

Graphic Design

Four students completed the pre- post- test battery in Graphic Design program. They scored an average of 34% on the pre-test, 78% on the post-test, for an average gain of 44%. No analysis was given.

Med Prep Program

Students completing the program had a 27% – 36 % gain in their post scores. The average pre-test score was 38 (9 students) and the average post-test score 78 (4 students) . From an evaluation standpoint, I’m not sure what this means. There is no standard indicating how much improvement is expected.

This is the first year for the Med-Prep Program. In review of the Pre-Post exam, there were several questions that were missed by all the students. Perhaps it was because the course material was not covered as completely as expected.

In preparation for the coming year, I will evaluate the test questions and the curriculum to be sure all the questions asked are reflecting material that I expect to cover in class. I also think it may be a good idea to periodically look at the exam questions during the year to make sure the material is being covered.

Office Technology - Trinidad Campus

The Office Technologies Pre/Post Test assesses a students skill and knowledge attained in several areas. The results for this year include seven students with pre-test scores averaging 34% and post-test scores averaging 81% for an average gain of 48%. The Pre/Post Test measures a student's ability and skill to:

• Use word processing software (Word) and master the beginning, intermediate and advanced functions.
• Use spreadsheet software (Excel) and master the various functions and capabilities of the software.
• Use presentation software (PowerPoint) and master the various features of the software.
• Create a keyboarding document using proper format.
• Learn standard indexing rules as established by American Records Management Association (ARMA).

Comparing the pre and post data indicates that these two student retained various skills and knowledge presented in various courses in the Office Technologies Program. However, I believe these students were capable of doing much better, but the post-test is always taken very lightly. I will use the method suggested by Mike Mincic and use the post-test results from required and graded assignments during the semester.

Criminal Justice

The Criminal Justice scores for 2001 were 22.8% for an average pre-test score and 87.5% for the average post-test score (a 64.7% average increase). No analysis.

Dave Covington, Division Chair and Assistant Dean

No report was received for Autobody Repair, Building Trades, CLETA.

Automotive Mechanics

The last several years of Auto Mechanics students were uniformly assessed in AUM 110, 111, 120 & 130. They performed as follows on the pre- post- test assessment (Note that because a number of students dropped from the program, the average pre-test plus average gain does not always equal the average post-test score).

Cosmetology - Trinidad Campus

Cosmetology courses are assessed individually and the results are tabulated below:

Gunsmithing

The pre and post test scores for students who are graduating this year had an average of 22.4% with an average gain of 42.6% to 65.1% on the Occupational test and 11.8% with average gain of 40.6% to 52.4% on the Accountability test. We have evaluated our test and are working toward insuring coordination between pre test, course information, course testing, course final testing, and post testing. as you now this takes a great deal of effort.

Because we have new instructors in the past two years we have had to take time to evaluate exactly what we wanted to do. Our findings are:

1. Review the course timing and put into place a post test evaluation of student progress that will mean something.

2. Allow new instructors to build test questions to be given during the course and at the final of that course.

3. Build a new pre/post test using current instruction evaluation questions.

This has been a long process and is now starting to come together. We recently had approved by the curriculum committee our course restructuring and new Comprehensive skills examination class and the new pre test will be put together and used this up coming fall semester.

The gunsmithing department staff feels this should greatly improve student learning and our post test scores

Occupational Safety and Health

OSHA - I got pretest scores but no post-test scores and no analysis

San Luis Valley Campus Programs

No report has been received.

In terms of the assessment plan itself, the faculty and Assessment Committee made significant progress on both the general education and course level components. Many departments are formally reporting on the assessments in all of their courses. We believe that assessment is occurring in all courses but results are not formally reported. This defect will be addressed in the form of somehow standardizing the reporting while attempting to minimize paperwork. The general education level component should be pilot tested this fall.

While many programs perform and report program level assessment, many areas are not reporting results. The Assessment committee needs to (1) clarify exactly what is required and when and (2) provide administration with information such as this report for departmental accountability. In the academic areas, "program level" continues to elude definition. If "program level" is defined as synonymous with degree programs, then there are essentially two, the Associate of Arts and the Associate of Science programs. This definition forces the program-level assessment to be almost identical to general education assessment. If "program level" means the same as emphasis area, e.g., Art, Computer Science, Journalism, etc., then we face the problem of very small sample sets because there are so few students graduating in many of these emphasis areas. Before program-level assessment can take place in the academic areas, a consensus definition must first emerge.

Presently, the Assessment for Improvement of Student Learning plan at Trinidad State Junior College collects data in many courses and programs. The only changes that came from analysis of assessment data were pedagogical changes, no changes requested funding, resources, or other tangibles. Most faculty and staff continue to perceive "assessment" essentially as an externally imposed exercise with little inherent value. Before the plan can mature into more than just data taking and report writing, the consequences of performing assessment must become real. After collection, assessment data must be analyzed by critical participants, specifically faculty and division chairpersons. Concrete recommendations must naturally flow from these analyses, and finally, these recommendations must result in real, tangible action such as resource allocation change, staffing change, or curricular change. Presently these things change only because of (grant) moneys becoming available or enrollment dynamics. Unless and until the assessment process causes such action it will remain a largely irrelevant activity.