In addition, the program review process is intended to promote and facilitate ongoing assessment and improvement.  Changes in personnel, in pedagogical styles and resources, and in the understanding of our educational goals, suggest that we should periodically conduct such a review in order to maintain a clear vision of our goals and develop the most effective means to attain them. These reviews also afford a formal opportunity to identify needs and argue for institutional support (e.g., new tenure track faculty, lab equipment, library resources). Further, because these reviews are intended as self-evaluations to help us maintain the highest standards of education, full department participation is expected.

The guidelines below present the list of all the areas that should be addressed. The list is intended to be exhaustive; while some items may not be applicable to every program, whenever a question is not addressed, the program should explain why.  To assist the Program Review Committee in its efforts to read the review carefully and consistently, program by program, please follow the exact order of topics listed below.  The PRC considers the use of headings for each section helpful.

I. Introduction

1. Describe the process your program used to prepare your review.
2. Include a brief history of your program.
3. Summarize your progress toward meeting the goals specified in your last program review.

II. Mission

Describe to what extent and in what ways your program's mission is consistent with the mission statement of the College.  For help with this section, you may wish to consult the Office of the Vice President for Mission.

1. The liberal arts mission of the College: "to probe deeply the mystery of existence by cultivating the ways of knowing and the arts of thinking." Possible questions to consider:
   
   1. How does your program "place special importance on fostering the intellectual skills and habits of mind which liberate persons to probe deeply the mystery of existence and live authentically in response to the truths they discover"?
   2. How does your program foster "wonder about the nature of reality," by having students "look twice, ask why, seek not merely facts but fundamental principles, strive for an integration of all knowledge, and express themselves precisely and eloquently."
2. The Catholic mission of the College: "to affirm and foster the Christian understanding of the human person that animates the educational mission of the Catholic Church." Please consult the Bishop Cummins Institute for assistance with this section. Possible questions to consider:
   
   1. How does your program lead students to a "transcendent meaning to creation and human existence" by probing "the mystery that inspires wonder about the nature of existence"?
   2. How does your program connect the "intellectual and spiritual journeys" of the students? How do you promote the "dialogue of faith and reason"? How do you "defend the goodness, dignity, and freedom of each person, and foster sensitivity to social and ethical concerns"?
3. The Lasallian mission of the College: "to create a student-centered educational community whose members support one another with mutual understanding and respect." Possible questions to consider:
   
   1. How does your program enable faculty to grow "spiritually and personally when their work is motivated by faith and zeal"?
   2. How does your program help "students, faculty, administrators and staff from different social, economic, and cultural backgrounds come together to grow in knowledge, wisdom and love"?
   3. How does your program show "awareness of the consequences of economic and social injustice and its commitment to the poor"?
   4. How does your program help students learn "their responsibility to share their goods and their service with those who are in need"?

III. Self-Appraisal with Plans of Action: Assess your program’s strengths and weaknesses for each item and, where needed, indicate action items and recommendations.

1. Curriculum
   
   1. List your program's courses, majors, minors, specializations, and concentrations, with their enrollments over the past five years. Indicate whether any departmental courses are derived from or offered in another department or program.
   2. Describe your program’s learning goals. (Every program is expected to include information literacy as a goal (addendum 1), and we encourage you to consult the guidelines appended to this document (addendum 2)).
   3. Describe your program’s learning outcomes. Explain how each learning outcome supports learning goals described in 1.b. (We encourage you to consult the guidelines appended to this document (addendum 2)).
   4. Provide course descriptions and an explanation of how course sequencing and/or degree requirements best achieve your program’s learning goals and outcomes.
   5. Provide sample syllabi (with stated learning objectives) from at least the introductory course or courses in the program and any “core” courses for majors and minors. Please note that syllabi must contain a statement on Services for Students with Disabilities (addendum 3).
   6. If your curriculum has a capstone course/synthesis project/thesis, explain how courses prior to the capstone build toward it. How does the capstone experience allow students to integrate previous learning? If your program does not have a capstone course/experience, explain how you evaluate the degree to which your graduating majors/minors/graduate students have achieved the educational goals of the program.
   7. Identify and explain methods (other than grades in individual courses) used to evaluate student mastery of the goals and learning outcomes. Provide evidence that the learning outcomes are being achieved. Evidence should include some type of program assessment research.
   8. Outline the long-range program assessment plan and the ongoing process for regularly evaluating the program, especially ways you use to improve both individual courses and overall program effectiveness.
   9. Discuss specifically how your curriculum meets the general educational needs of non-majors
   10. Comment on how the program integrates relevant professional and accreditation standards.
   11. Identify any curricular changes that you have implemented within the last five years and explain your rationale for these changes.
2. Student Outreach, Advising, and Services. Please describe the following:
   
   1. Methods for recruiting students
   2. Effectiveness of student advising
   3. Process for accommodating students with special needs
   4. Support for student clubs
   5. Placement of graduates
   6. Availability of internships
   7. Contact with alumni
   8. Collaboration with other campus groups
3. Faculty Quality
   
   1. Faculty changes that have occurred in the last five years
   2. Number of tenure track faculty, adjunct faculty (5/7 or more) and lecturers (4/7 or less)
   3. Ratio of tenure-track faculty to non-tenure track faculty (adjuncts and lecturers)
   4. Ethnic and gender diversity of faculty
   5. Academic qualifications of your faculty
   6. Identify any limitations in faculty’s disciplinary expertise and intellectual perspectives (e.g., coverage of specific areas in the field)
   7. Faculty accomplishments:
      
      1. Teaching
      2. Scholarship
      3. Service
   8. Participation in Collegiate Seminar
   9. Participation in January Term
   10. Participation in other College programs and activities
4. Support for Faculty
   
   1. Short- and long-range plans for faculty replacement because of retirements, reduced teaching loads, administrative reassignments, leaves, etc.
   2. Issues/problems with faculty recruitment and retention
   3. Faculty evaluation process
   4. Faculty mentoring
   5. Faculty development
5. Program Infrastructure
   
   1. 1. Program decision-making structure (duties of chair, rotation of chair, faculty involvement in program planning and administration, process for assigning courses and reassigned time)
   2. Adequacy of program budget
   3. Adequacy of office space
   4. Adequacy of secretarial, clerical and technical support
   5. Adequacy of instructional materials and equipment
   6. Adequacy of library collections and learning resources (please refer to the program’s review of library resources.)
   7. Adequacy of information technology resources
6. How does the program compare with its peers in the discipline in quantitative difference (e.g., number of majors, course offerings, faculty size, resources, etc.) and qualitative difference (e.g., emphases in the field, approaches to different perspectives emerging from the field, etc.)?
7. External Review Process
   
   1. The Program Review must include an assessment by peers in the discipline external to Saint Mary’s College of California. The Department Chair/Program Director, the Chair of the Program Review Committee, the Dean of the School, and the Provost/Vice Provost will agree, in each case, on the form and procedures involved in the external review. It is common practice to provide assurance that there is no conflict of interest involved in the choice of professors who will conduct the external review; therefore, the Department Chair will initiate the process by providing the Chair of PRC, the Dean of the School, and the Provost/Vice Provost with a list of three schools that are being used in the review as appropriate comparison schools and the names of any professors at those schools who might be particularly appropriate to assist in the external review process. The potential external reviewer(s) will be contacted by the Dean of the School no later than May 15 of the academic year prior to the review. One or more reviewer, as appropriate, will be selected by consensus of the four administrators named above. A written report will be submitted by the external reviewer(s) on or before December 15 to the Department Chair/Program Director. This report should be forwarded to the Dean of the School and should be appended to the Program Review.
   2. Please provide a brief response to the report of the external reviewer.

IV. Summary Plan of Action: please extract the action items and recommendations in Part III, and present each item in prioritized order with a timeline for completion.

V. Appendices

1. Copy of College Catalog entry
2. Brochures and/or advertising materials
3. Review of Library Resources and Information Literacy (service provided by the Library upon request)
4. External Review

Program Review Process Guidelines

1. The Chair of the Program Review Committee (PRC) will distribute a list by December 15 to all departments and programs (hereafter, “programs”) to be reviewed during the following academic year. 
2. The Dean of the School for the program will make arrangements for external review by May 15 of the academic year prior to the review.
3. Unless otherwise arranged with the Chair of the PRC, program reviews are to be submitted by Dec. 15.  Delinquent reviews from previous years are due September 1.
4. The Chair of the PRC will schedule a May workshop for chairs of all programs being reviewed in the following year.
5. All faculty in the program should participate in preparing and revising the written report. The report should contain the names and signatures of all faculty who participated in the review process.
6. Programs may request and should expect timely help from the Registrar’s office and the Office of Institutional Research concerning data pertinent to the review.
7. Program chairs may request to meet with the PRC or the PRC Chair before or after the review process has been completed.
8. Programs that undergo major external review by accrediting agencies may submit that review to the PRC in lieu of the normal Saint Mary's review. They must, however, address any items in the Saint Mary's review guidelines not covered in the external review.
9. Deans are to be given a copy of the review document when it is submitted to PRC.  The appropriate Dean will be notified when a scheduled review is not submitted. 
10. As specified in the Faculty Handbook (1.7.3.6.3 (g)), programs that fail to submit a review as scheduled or submit an inadequate review may be censured by the EPB at the request of the PRC, and by order of the chairperson of the EPB, all board actions concerning the program may be suspended until such time as the review is submitted and accepted as satisfactory.
11. The Dean is asked to provide a written evaluation (1.7.3.6.3 (c)) of the program review before the meeting of the PRC.
12. The PRC will meet with the chair and faculty of the program in order to discuss questions that arise from their review of the program’s materials.
13. The PRC will consider the program’s materials and produce a letter containing its conclusions, which is sent to the program Chair, the appropriate Dean, and the Provost or Vice Provost (for graduate or undergraduate programs, respectively).
14. The PRC recommends that the faculty of the program meet to discuss the PRC letter. They should address, point-by-point, the issues identified by the PRC.  A brief summary of the results of this meeting should be written, including any corrections the program wishes to make, as well as any additional plan of action developed during the discussion. If the program wishes to correct any major errors in fact or interpretation, it is encouraged to write a letter to the PRC in response to the review.  
15. The Provost or the Vice Provost schedules a meeting with the Dean of the School housing the program, the Department/Program Chair and faculty, and the Chair of the Program Review Committee (see Faculty Handbook 1.7.3.6.3).  That meeting should focus on the main points of the program review, especially the proposed plan of action, the PRC’s response letter, and any program needs and future planning that can be identified.  Every effort should be made to find a consensus about the future direction of the program and then to obtain specific commitment of institutional resources sufficient to support progress towards achieving the agreed upon goals. A written record of the specific steps that will be taken to coordinate planning and resources over the next five years must be part of the process.
16. The written record produced by these two meetings specified in 12 and 13 must be filed with the office of the Academic Senate no later than the end of the academic semester following the issuance of the PRC’s letter to the program. The next program review cycle must address this material directly.  Failure to carry out this step may result in the program being censured by the EPB, and, by order of the chairperson of the EPB, all board actions concerning the program may be suspended until these steps are carried out satisfactorily.
17. In each of the years following the program review, the Dean for Academic Development will ask the Department Chair or Program Director to provide an updated report on the achievement of the summary plan of action and the specific goals identified in the meeting with the Dean of the School and the Vice Provost.

Calendar of Major Deadlines

Addendum 1: Information Literacy

Information Literacy Defined

Information literacy is a set of abilities requiring individuals to "recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information." 1 Information literacy also is increasingly important in the contemporary environment of rapid technological change and proliferating information resources. Because of the escalating complexity of this environment, individuals are faced with diverse, abundant information choices--in their academic studies, in the workplace, and in their personal lives. Information is available through libraries, community resources, special interest organizations, media, and the Internet--and increasingly, information comes to individuals in unfiltered formats, raising questions about its authenticity, validity, and reliability. In addition, information is available through multiple media, including graphical, aural, and textual, and these pose new challenges for individuals in evaluating and understanding it. The uncertain quality and expanding quantity of information pose large challenges for society. The sheer abundance of information will not in itself create a more informed citizenry without a complementary cluster of abilities necessary to use information effectively.

Information literacy forms the basis for lifelong learning. It is common to all disciplines, to all learning environments, and to all levels of education. It enables learners to master content and extend their investigations, become more self-directed, and assume greater control over their own learning. An information literate individual is able to:

- Determine the extent of information needed
- Access the needed information effectively and efficiently
- Evaluate information and its sources critically
- Incorporate selected information into one’s knowledge base
- Use information effectively to accomplish a specific purpose
- Understand the economic, legal, and social issues surrounding the use of information, and
access and use information ethically and legally

Excerpt from: Information Literacy Competency Standards for Higher Education. (2002) Association of College and Research Libraries, American Library Association.

http://www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm#ildef

Addendum 2: Learning Goals and Learning Outcomes

A) The Relationship Between Goals and Learning Outcomes

After a program’s mission statement has been established, the next step for faculty is to design goals and the intended outcomes for those goals. Faculty should strive after goals that can easily be converted into learning outcomes. It might be helpful to note that while goals tend to focus on delivery of information (Students will become familiar with the cultures of ancient Greece and Rome), the converted outcomes focus on effect (Students will compare and contrast the political, economic, and religious systems of Greece with that Rome). In order to arrive at outcomes, faculty might consider the following questions as a guide for the learning outcomes discussion:

1. What do we want students in our major to know?
2. What do we want our students to be able to do?
3. What values or attitudes (dispositions) do we want to instill in our students?

The statement of learning outcomes at the program level clarifies for all stakeholders the knowledge, skills, and abilities a student must possess to successfully complete a course or program and thus earn a degree from the College. Learning outcomes have three distinguishing characteristics: the learners’ specified action must be observable, measurable, and performed by the learners. Here are examples of goals followed by their respective outcomes:

• Students will be familiar with the syllogism.
• Students will have an appreciation of cultural diversity in the classroom.
• Students will be familiar with the different kinds of writing genres.

These goals can easily be converted into outcomes by inserting action verbs and in some instances making clear the predicate.

• Students will conduct logical arguments through the use of a syllogism.
• Students will summarize in writing their attitudes about cultural diversity in reflective essays
• Students will submit by graduation the following kinds of writing: essay, creative non-fiction, summaries, paraphrase, and reports.

Since the learner’s performance ought to be observable and measurable, the verb chosen for each outcome statement should result in overt behavior that can then be observed and measured. Sample action verbs are: compile, create, plan, revise, analyze, design, select, utilize, apply demonstrate, prepare, use, compute, discuss, explain, predict, assess, compare, rate, critique. Conversely, some verbs that are unclear subject the learning statement to different interpretations in terms of what action is being specified; such verbs, which obscure the intended behavior, cannot be observed or measured. These types of verbs should be avoided when constructing learning outcomes: know, become aware of, appreciate, learn, understand, become familiar with.

B) How to Establish Goals and Learning Outcomes

The most practical way to start the conversation about goals and learning outcomes is to weigh the needs of the program mission statement against examples of outcomes published through the respective professional organizations or by departments with similar programs at other schools. This process should help generate a comprehensive list of ideas and suggestions for learning outcomes that can then be refined and narrowed. Once the department has generated such a list, the most challenging task is then deciding which goals and outcomes are essential at the program level. Practitioners point out that if the final list of learning outcomes initially reaches beyond five or six, the assessment effort will be onerous, even unmanageable. During discussions, it might be helpful to keep in mind that all outcomes should be measurable. Program goals articulate both measurable and non-measurable expectations for students; outcomes must be expressed in terms of demonstrable characteristics. Outcomes focus on what students in the major can demonstrate rather than on what faculty members teach. The list of learning outcomes can always be reviewed and revised as the need arises or as new developments occur in the discipline. In any event, it important to make sure that students know the program’s learning outcomes. Research has shown that students who are aware of the direction of their education are much more likely to be engaged in the learning process.

Here is a model of learning outcomes for a History department:

1. History majors will demonstrate skill in chronological thinking.
2. History majors will demonstrate historical comprehension.
3. History majors will demonstrate the skills of historical analysis and interpretation.
4. History majors will demonstrate historical research capabilities.
5. As a consequence of these skills, History majors will demonstrate a) a working knowledge of the natural and cultural environment in which humans have developed and live; and b) a sense of the diversity of the human experience influenced by geography, culture, race, ethnicity, gender, and class.
6. History majors will demonstrate an understanding of the career search in appropriate educational and professional markets

C) Use Curriculum Mapping to Confirm Outcomes

Once the preliminary list of outcomes is established, faculty should engage curriculum mapping to see where those outcomes are embedded in current courses. A curriculum map consists of a table with two axes, one listing program outcomes, the other listing courses in the major. The coverage of a learning outcome in each course is shown in the cells of the table (see Table 4.1)

Table 4.1. Curriculum Map Template

Course mapping can provide a view of how individual courses are related to the program learning outcomes and can show the curricular emphasis given to each outcome. For example, Outcome 1 in the above table appears in seven courses and is therefore given greater emphasis than Outcomes 2 and 3. A map can also identify program redundancies or gaps, such as the fact that no course currently takes ownership of Outcome 4.

Curriculum mapping can help facilitate faculty discussion about the extent to which the program currently addresses the list of learning outcomes. A different kind of map, one that indicates levels of intensity, can go even further by displaying the degree to which a course emphasizes a specific outcome. In some cases, it may even be useful to show the number of hours devoted to each outcome in each course; the level of achievement that is expected in each course can be indicated as low, medium, or high or other such descriptors. This would reveal how student achievement is expected to develop during their progress through the curriculum.

*For those who are need further convincing about the importance of outcome-based pedagogy,
read the following noteworthy articles.

Articles:
From Teaching to Learning - A New Paradigm for Undergraduate Education By Robert B. Barr and John Tagg. November/December 1995. Change magazine. Found at:
http://critical.tamucc.edu/%7Eblalock/readings/tch2learn.htm

So, What’s a Learning Outcome Anyway? Learning Outcomes and the Learning Paradigm By Mark Battersby, Department of Philosophy, Capilano College. Found at:
www.pierce.ctc.edu/Library/lstagrant/wp-content/battersby.doc

Reprint of document found on the web:
Daly, P. (2005). Establishing Learning Outcomes. In Assessment of Student Learning: Assessment Handbook [Web]. Indiana University Southeast. Retrieved May 7, 2007, from http://www.ius.edu/assessment/pdf/Handbook/Chapter4.pdf