Evaluation of Descriptive Summary - Due April 13, 2000
A. Institutional Research and Evaluation

A.1 Is Institutional research integrated with and supportive of
institutional planning and evaluation.
Answer:

A.2 Does our institution provide the necessary resources for effective
research and evaluation.
Answer:

A.3 Has our institution developed and implemented the means for
evaluating how well, and in what ways, it accomplishes its mission
and purposes.
Answer:

A.4 Does our institution provide evidence that its program evaluations
lead to improvement of programs and services.
Answer:


Descriptive summary

Between 1995 and 1997, institutional planning, research, and evaluation
at Los Angeles Mission College was managed by a team of faculty,
staff, and administrators under a system known as ICAN (Improvement:
Continuous and Neverending). These responsibilities are now assigned
to three committees of the Mission President's Coordinating Council
(MPCC): the Master Planning Committee (MPC), the Accreditation
Committee and the Program Evaluation Committees. Prior to this
reorganization, two formal master plans were developed. The
"Comprehensive Master Plan 1990-1995" was written before the
college moved to its permanent location. In 1996, "Los Angeles Mission
College: Oddysey 2001" was developed. Few members of the college
community are familiar with these plans. They have had limited effect in
guiding the direction of the college as a whole.

The college hired an Associate Dean for Campus Development and
Institutional Research (ADCDIR) in December of 1999. The
infrastructure necessary to perform institutional research is being
developed. This includes establishing specialized research databases
for the college of demographic and student characteristics, course
enrollments, degrees and certificates awarded, and basic skills
assessment and placement. Other databases are being acquired or
established such as data from previous student surveys and census
information and reports to supplement the information. In February,
2000, a data system (IMS-Instructional Measures System) was
established for providing up-to-date information about instructional
staffing and enrollment. This system provides information for program
evaluation.

Using these databases, historical data series of retention, success,
persistence rates, and degrees and certificates awarded are produced.
These and other district and State MIS databases are available for
evaluation purposes to allow the college to assess its progress toward
Partnership for Excellence and other institutional goals.

In addition to establishing the position of ADCDIR, the college provides
physical space, computer equipment, software, and supplies to support
ongoing research. The college supports the participation of research
personnel in district-wide committees and funds conference attendance
related to institutional research and assessment. Also, the college has
institutional membership(?) in the Research and Planning Group (RP
Group) of the California Community Colleges.

The college does not have a process or means of assessing the
adequacy of resources devoted to institutional research and
assessment activities. Since the formal research function is new, the
assessment method has not been established.

The Master Planning Committee (MPC), a shared governance
committee, now has the responsibility for the coordination of college
planning. The primary goal of the committee is to develop a master plan
to integrate educational, financial, facilities and human resource
planning toward achievement of the college mission. Though various
methods have been proposed and discussed over a three-year period,
MPC has not agreed upon a strategy toward the development of a
master plan. However, the administration and MPC facilitated the
creation of a new mission statement and seven college goals, which
were formally adopted by the Faculty Senate, the Community Advisory
Committee, and MPCC in October. The mission statement and goals
are published in the college catalogue.

Most planning and decision-making is driven by the administration, who
takes into consideration influences from community groups, and is held
accountable for the performance of the college by the District Chancellor
and Board. Though chiefly guided by the administration, oversight and
consent of this process is provided by individual, shared-governance
committees. This responsibility is exemplified by several committees.

For example, the Budget Committee (financial resource allocation) and
Campus Development Committee (facilities use, maintenance, and
construction) submit policies and plans directly to MPCC. Priorities for
faculty hiring are established by the Faculty Senate Hiring Committee,
which forwards its recommendations to the president. Requests for
classified workers are submitted by department chairs to one of the
three vice presidents, who presents the request to the Budget
Committee for consideration. A major role of MPCC is to coordinate all
of these functions toward achievement of the stated mission and goals.

Institutional outcomes assessment is guided by two principle factors:
those related to financial health of the college (budgetary); and those
motivated by accreditation self-study and the stated mission of the
college (student education). Those outcomes which the college most
recently made efforts to improve ((Weekly Student Contact Hours
(WSCH ), FTES, FTEE, and WSCH/FTE)) were dictated by recent
district- wide fiscal constraints. The district budget determines the
framework within which the college administration establishes goals for
internal efficiency and enrollment growth. College administrators
communicate these goals in meetings with individual department chairs.
On occasion, disciplines are presented with historical data and other
factors considered in allocation of FTE. Due to budgetary constraints
between spring of 1996 and fall of 1999, an overarching mandate in
college planning and resource allocation has been to increase
WSCH/FTE, so that college income matches, or exceeds expenses.
FTE allocation to each discipline on a semester basis is the
responsibility of the Vice President of Academic Affairs, who discusses
department course offering needs with each chair. Chairs are given
discretion to apply their allotted FTE in a manner that will balance the
educational needs of the students with the efficiency goals of the college
and the district. Current projections reveal that college expenses will not
exceed its District allocation for fiscal year 1999-2000.

The recently revitalized Program Evaluation process is now the vehicle
for oversight of the assessment of student learning outcomes and
overall program review. This process was reorganized in the summer of
1999, and includes the following components (in addition to the fiscal
considerations mentioned above):

1. review of student surveys on effectiveness of programs and courses
2. review of course outlines, certificate requirements, and degree

requirements

3. analysis of instruments used to assess learning outcomes and skill

acquisition

4. description of the unique characteristics of the discipline

5. listing of major accomplishments

6. identification of major changes in academic/vocational area

7. assessment of facilities, financial, and human resource needs

8. establishment of specific goals and objectives

A schedule has been developed for each discipline/program of the
college to participate in the program evaluation process. To date, two
disciplines have completed the new program review, six are in process,
and another seven will commence in spring 2000.

Each course has a defined series of learning objectives that are written
in the course outline, a document that must be approved by the
Curriculum Committee. Student achievement of these learning
objectives is measured primarily by individual instructors and
departments using traditional methods, such as evaluation of
assignments and testing. The English and Math departments are
currently discussing standardized evaluation devices that will assess
"satisfactory" achievement of learning objectives in introductory
courses. The campus requires all students entering English and math
courses to complete assessment tests prior to placement into the
appropriate course. Likewise, course prerequisites must be validated
with appropriate skill/knowledge justifications and outcomes data to
support them. Prior to the establishment of the permanent research
position, part-time campus researchers played a significant role in the
validation of placement exams and prerequisites, and have assisted in
data collection for program evaluation.

Measurement and validation of student achievement above the level of
the individual instructor or discipline has been limited. The program
evaluation process attempts to examine the outcomes of students who
are no longer enrolled at the college. Statistics on certificates and
degrees awarded by the college is consolidated annually in a report
generated by the District. Transfer student tracking has been performed
primarily using reports from the state chancellor's office and the National
Student Loan Clearinghouse. Other than anecdotal evidence
established by correspondence between faculty and former students, no
campus-wide process exists for measuring student job placement or job
advancement.

To evaluate student services and overall student satisfaction with the
educational services provided by the college, the Vice President of
Student Services conducted Noel-Levitz Student Satisfaction Surveys in
1995 and 1999. Likewise, individuals report using their own surveys and
district data to gather pertinent information in order to assist in program
development, planning, and grant writing. However, coordination of
research with planning, program evaluation and resource allocation at
an institutional level has occurred in a non-systematic, ad-hoc fashion,
as information needs arise.

Evaluation - Standard 3
Responses to Questions - Group B (Institutional Planning)
April 7, 2000

B.1 Does the institution define and publish its planning processes and involve appropriate segments of the college community in the development of institutional plans?

The college does not define and publish its planning processes in an independent format. Rather, it is defined and published within the context of the entire governance structure of the college. The degree to which the planning processes are known and understood by the college community is equivalent to the level at which the governance structure of the college is known and understood (see Standard 10). In general, the college community is aware that the individual committees are the deliberative bodies for their respective areas: Budget Committee (financial planning and resource allocation), Campus Development Committee (facilities/property planning and use), Faculty Senate Hiring Committee (prioritization for full-time faculty hiring), Curriculum Committee (course outlines and prerequisites), and Master Planning Committee (development of institutional mission statement, goals, and master plan). Each of these committees submit recommendations to the MPCC, advising the president in the establishment of policies and strategies to achieve the stated mission of the college.

The committees having responsibility for institutional planning and resource allocation vary in their composition, and reflect different segments of the college community.

1. The Budget Committee is a shared governance committee and consists of ……….
2. The Campus Development committee is also a shared governance committee and consists of ……..
3. The Faculty Senate Hiring Committee consists of ………….
4. The Curriculum Committee is ………. And consists of………
5. The Master Planning Committee is a shared governance committee and consists of ……

A Classified Hiring Committee consists of ………… It has not met for two years, and classified hiring is primarily decided by the three Vice Presidents as personnel needs arise. Student representation in the planning process is limited to …………….

Classified personnel and students are underrepresented in the planning process. It should be noted that adequate representation of appropriate segments of the college community in the planning process is dependent upon active participation by all committee members. This requires appropriate announcements and scheduling of committee meetings.

B. 2 Does the institution define and integrate its evaluation and planning processes to identify priorities for improvement?

Evaluation and planning are not integrated at the institutional level. Although evaluation of educational programs and student services does occur, there is presently no strategy for linking the information-gathering processes with institutional planning and resource allocation. For example, there is no formal policy or procedure for committees involved in the planning processes (see above) to receive and analyze program/discipline reviews. A major difficulty in identifying priorities for improvement is a lack of specific and measurable institutional outcomes. Although new college goals were approved in the fall of 1999, the college has yet to define measurable outcomes related to these goals. This should be a major priority for the college, not only at an institutional level, but at the level of each educational program and student service. Until a baseline for performance is established, the college will not be able to identify where resources must be applied to achieve maximum benefit toward achievement of its mission.

B.3 Does the institution engage in systematic and integrated educational, financial, physical, and human resources planning and implement changes to improve programs and services?

Although several attempts have been made over the last five years, there is currently no comprehensive master plan to integrate educational, financial, physical, and human resources planning in order to improve programs and services. Planning and review in each of these individual areas occurs at regular meetings of the responsible committees, which forward their recommendations to MPCC. Currently, it is MPCC and the administration of the college that is solely responsible for their integration. The college did approve new goals in the fall of 1999. However, without a master plan and measurable outcomes which are formulated and examined through systematic institutional evaluation and research, the MPCC and its supporting committees can only make decisions based upon contemporary circumstances at the college. A primary task of the college should be to establish a comprehensive master plan that is integrated with program/discipline evaluation, with measurable outcomes that can be assessed in an objective and systematic fashion. Only if this is achieved can educational, financial, physical, and human resource planning occur in a systematic and empirical manner, in order to improve educational programs and services.

 

outcomes
C. Institutional Outcomes Assessment

C.1 The institution specifies intended institutional outcomes and has clear documentation of their achievement.

There are several levels of institutional outcomes currently operating at the college: (1) the historical budget-driven outcomes, (2) state-drive outcomes such as PFE, and (3) the nascent outcomes developing from the institutional mission statement and goals. It will take considerable commitment and several years for (3) to replace (1) as the driving force of the college which will more than enable the college to meet (2).

In the fall 1999 semester the college adopted a mission statement and seven overarching goals. This is an significant accomplishment as it establishes, for the first time in the college's history, a framework within which to develop planning that springs from the mission of the college rather than the budget of the college. Earlier iterations included a mission statement and hundreds of specific objectives grabbed from thin air to fill up hundreds of pages. They were never revisited to determine the status of accomplishment.

There are ways in which both the mission statement and the overarching goals can be improved. The mission statement is extremely vague and nonspecific. There is a need to augment the statement with specific institutional XXXXXX John has an example which would be good to consider XXXXXX. The overarching goals are also vague and should be viewed as pathways to follow. As such there is need to develop possible measurements through which the college can determine if it is making progress in each of the seven goals. As of late spring 2000 the effort to determine measurements of achievement has only just begun.

It is not at all clear that administrators, faculty, or staff, both full-time or part-time, when asked the question "what does the college do?" would be able to provide anything beyond the most generalized response. There is either a striking air of unconcern or a bevy of rationales offered when it is brought up that the average length of time to complete an two-year degree or certificate of completion is 6.6 years. The situation is much the same when it is pointed out that of all the students who enroll in the college during a particular year (roughly 10,000) only 183 will transfer and only 306 will earn an AA degree (figures for 1998/1999).

It is unclear that any member of the college could tell you the college's enrollment in a particular semester or the enrollment goal for that semester. The same will be true for the budget of the college and its allocation to different functions at the college and whether or not the college has a yearly profit or loss.

Further it is unclear that any member of the college could enumerate actions the college is taking to meet PFE objectives.

There are mounds of documents produced yearly on different aspects of institutional performance. However, there is no one document which summarizes the outcomes anticipated, the actual accomplishment, and the outcomes for the coming year. Such a publication, formulated to support the overarching goals of the college, would start to focus staff attention on both accomplishments and areas where improvements could be made.

C.2 The institution uses information from its evaluation and planning activities to communicate matters of quality assurance to the public.

It is doubtful that this is accomplished. The college includes the official WASC accreditation statement in its official publications but this is a generalized statement.

This college, as do most, employs a variety of techniques for communicating what it is doing to the local population. We publish catalogs, schedules, fliers, air advertisements on cable television, and the like. Members of the college serve on various community service groups such as Kiwanas, Chambers of Commerce, and the like. The college president has a community advisory group which meets four times each year. Most of these activities are designed to promote the college and to learn the concerns of community members. Some items, such as planning for new buildings, appear to strike a cord in the community and, whether for or against, they readily voice their concerns. But it is doubtful that buildings can be classified as a matter of quality assurance.

It would appear, in the ideal, that advisory committees such as is a requirement for vocational programs could serve as a way to validate the quality of the programs at the college. Changes have occurred in vocational program advisory committee membership in the last year, which has broadened the representation on these committees. However, in college publications and on the college website, only the Multimedia Studies program actually lists the members of its advisory committee and their corporate affiliation and additionally provides information on the background of teaching faculty. Non-vocational disciplines have no requirement for advisory committees.

C.3 The institution systematically reviews and modifies, as appropriate, its institutional research efforts, evaluation processes, institutional plans, and planning processes to determine their ongoing utility for assessing institutional effectiveness.

Prior to December 1999 the college's research efforts were, at best, a mishmash of individualized efforts by interested persons lacking extensive formal preparation in research methodologies on matters of interest to them or of immediate need to the college. Examples of these might be matriculation, recruitment, and course prerequisite verification. Their efforts, while commendable, did not produce the type of research necessary for ongoing institutional assessment. Upon review of this situation over a two-year period, the administration was able to create a permanent full-time position to conduct institutional research. As of this writing the ADCDIR is in the process of developing XXXX

There is a district process for evaluating the college's fiscal performance. There are no other institutional evaluation processes.

There have been no effective efforts to review and modify institutional plans.

The college's planning process has changed based on the comfort and insight level of changing administrative appointees or at the direction of changing district managers. At this time there is a recognition that this is not in the best long-term interest of the college and a concerted effort is being undertaken to establish an effective planning process, tied to the mission statement, overarching goals, and the college's program review process.