9.2.1. Application. Students who can apply concepts and principles in new and different situations are operating at the application level of understanding. When students apply or recognize the application of principles and other generalizations, they have clearly gone beyond simple understanding. Students who can answer theory questions about principles of management, human relations, and other significant topics but are not able to put their understanding into practice are not yet at the application level of learning.

9.2.1.1. Students who solve problems by applying principles and other generalizations and are able to explain why are working at least at the application level. It is likely that any educational experience beyond an introductory course or basic principles course will and should involve the application level of learning. Instructors and course developers should plan lessons to this level with the realization that they require more advanced planning and more effective classroom skills than the lower levels of learning.

9.2.1.2. Figure 9.1. contains typical application-level student-centered instructional objectives.

Apply the concepts of ISD to actual curriculum development situations.
Apply basic human relations skills to situations involving superior-subordinate interpersonal relations.
Apply their understanding of the Russian definition of democracy to the interpretation of real or hypothetical situations.
Apply the principles of war to scenarios in simulated, fictitious theaters of operation.

Figure 9.1. Instructional Objectives at the Application Level.

9.2.2. Analysis. The ability to analyze a problem or situation into its component parts and to determine relationships that exist between them characterizes the student at the analysis level. Students retain and use analytic skills long after they have forgotten the specific subject matter used as a vehicle to develop the skills. "Scientific problem solving" and other examples of management tools for decisionmaking characterize this level of learning.

9.2.2.1. Learning and testing at the analysis level and higher can be very complicated. There are few opportunities for "school solutions," and solutions to problems may not be clearly right or wrong. One student's analysis may be better than another, but both may be correct. Often students will discover relationships between elements of a situation not anticipated by the instructor. Being unable to anticipate and plan for all student interpretations can make teaching at the analysis level very challenging for even the most experienced instructor.

9.2.2.2. Figure 9.2. contains some typical student-centered instructional objectives at the analysis level.

Analyze the communication network of an organization to determine discrepancies between the formal and informal organization chart.
Conduct an analysis of work flow within an organization to determine potential areas of improved productivity.
Analyze several sources of input into an accident investigation to determine fact and uncorroborated assumptions or allegations.
Analyze apparently unrelated government documents to identify principles of strategy or tactics.

Figure 9.2. Instructional Objectives at the Analysis Level.

9.2.3. Synthesis. Students who can rearrange elements of a problem or situation in new and creative ways to form relationships not previously apparent are functioning at the very high level of cognition known as synthesis. Since creativity is so important to this level of understanding, instructors may have to evaluate learning at this level without a model to compare to the student's answer or solution. Previously learned skills of deductive and inductive reasoning play a critical role in the student's ability to synthesize.

9.2.3.1. Learning and planning for learning at the synthesis level are certain to be complex and time consuming. Only experienced instructors should attempt to bring students to such a high level of understanding. Fortunately, or unfortunately depending on your point of view, few right or wrong answers to problems exist at this level. Rather than judge student work against a set of absolute standards, instructors must engage in a shared experience to explore new alternatives with students while providing scholarly guidance and seeking expert opinion.

9.2.3.2. Figure 9.3. contains some typical student-centered instructional objectives at the synthesis level.

Synthesize data from recent armed conflicts to propose new principles of warfare that may affect future conflicts.
Synthesize a variety of inputs from management decision-making techniques to determine a course of action that is highly supportive of the mission.
Prepare an accident prevention plan for a new facility that synthesizes appropriate regulations as well as the most applicable principles of safety engineering.
Synthesize relevant research and expert opinion into the revision of existing USAF instructor manuals and other related literature.

Figure 9.3. Instructional Objectives at the Synthesis Level.

9.2.4. Evaluation. Carefully considered judgments made by students based upon objective standards are characteristic of students at the highest level of learning, evaluation. Informed judgment on the part of students requires them to separate personal feelings from objective analysis and apply a set of quantitative and qualitative criteria to given situations. This level of learning requires understanding so complex that few Air Force schools will include it in their curriculums. Where it is applicable, however, it represents the highest level of cognitive behavior and creative problem solving.

9.2.4.1. Teaching and testing at this level is the most difficult task facing an experienced instructor. ( NOTE: Those instructors not at this level themselves should not attempt to teach these lessons.) The constant search for new and different problems to be evaluated for creative solutions is time consuming and difficult. Lack of time prevents many instructors from attempting higher learning outcomes. Teaching at this level represents the best of what we mean when we refer to the instructor as a real expert or a genuine scholar.

9.2.4.2. Figure 9.4. contains some typical student-centered instructional objectives at the evaluation level.

Evaluate alternative operational plans to determine a justifiable course of action.
Evaluate an original research project including data-gathering techniques, research methodology, validity of findings, and logic of recommendations.
Evaluate a system for the comparative assessment of the performance of subordinates to determine its applicability to a given situation.

Figure 9.4. Instructional Objectives at the Evaluation Level.

9.3. Principles of Teaching and Learning at the Higher Levels. There are several important principles to keep in mind as we plan and teach at all the higher levels. The significance of one principle over another varies from situation to situation. However, they all apply generally to instruction at the higher levels. It is unlikely that instruction can be successful at these levels if any of these principles have been violated. Several of these principles are as follows:

9.3.1. Carefully planned learning experiences at the desired level of learning facilitate the transfer of learning.

9.3.1.1. Students who are attempting to reach the higher level of learning need sufficient opportunity to practice their newly acquired skills. If we take our students through the understanding of theory and then ask them to apply the theory without appropriate guidance, we are violating this critical principle.

9.3.1.2. Students need to be guided through experiences at each level of the taxonomy so they can benefit from the constructive feedback of their instructor and other experts. Giving good feedback in carefully structured learning situations isn't spoon feeding; it's good teaching and critical for learning.

9.3.2. Experience with new, unfamiliar, or substantially different learning and testing situations promotes insight at the higher levels of learning. One of the most demanding elements of learning at the higher levels is the need to come up with many new and different situations for applying, analyzing, synthesizing, or evaluating principles and other generalizations. Essential principles may be the same in apparently diverse situations, but there must be enough newness about the problems that students cannot solve them from rote, memory, or recollection. This critical need for instructors to develop apparently different teaching and learning situations with common principles places great demands on their subject matter expertise.

9.3.3. Effective feedback for learning demands flexibility in evaluating the products of student outputs at the higher levels.

9.3.3.1. Teaching, learning, and testing at the higher taxonomy levels places considerable extra demands on the flexibility of both the instructor and the student. Creativity is characteristic of all the higher levels and should be encouraged! Because instructors and students bring their unique backgrounds with them into the classroom, it is often impossible to anticipate the exact solution any student may offer to a given problem.

9.3.3.2. While it is possible to retain the notion of a "school solution" for some application level learning, most learning at the application level and beyond probably does not lend itself to predetermined model answers. Feedback to the students becomes much more difficult and subjective. Evaluation of the student becomes more complex because a solution or strategy proposed by a student may be 180 degrees in the opposite direction of one proposed by the instructor, and yet may be of equal or greater value.

9.3.3.3. As complex as this situation may be, the instructor should remember that feedback and evaluation must remain objective to be worthwhile. Students are to solve problems and propose alternatives based on solid facts and a thorough grasp of higher principles. Learning at the higher levels is disciplined and scholarly and must not be reduced to expressions of personal preference and unsubstantiated opinion, as is often the case.

9.3.4. The need for supplemental expert evaluation of student outputs increases as the level of learning increases.

9.3.4.1. The increased use of expert judgment in evaluation is closely related to the need for flexibility in evaluating the products of student learning we have just described. By definition, students must produce creative insights or solutions when working at the higher taxonomy levels. Often, this creativity is best judged by an objective evaluation of proposed student evidence of learning by one or more experts in a given subject matter area. The instructor may or may not be one of the expert judges.

9.3.4.2. Since learning activities at the higher levels are often highly individualized, students may be doing much, if not most, of their work outside of class. Often, instructors and students engage in mutual goal setting through negotiation. Research proposals and individual learning contracts are often characteristic of learning at these levels. In these cases, a mutually agreed upon plan is developed so that both the instructor and the student agree to the scope and procedures of the learning activity as well as the goals. Often, such research and individual contracts contain a provision to have them evaluated by third parties. The instructor may also evaluate the student product, but a more qualified expert may be consulted if the project is highly specialized or technical.

9.3.5. The focus of evaluation at the higher levels gradually shifts from a primary concern about the process used to arrive at a decision on the quality of the decision itself.

9.3.5.1. As students function at the higher levels of taxonomy, it becomes less important to predict their exact behavior in the process of insight or problem solving. The product of their efforts eventually becomes more important than the process they go through. Sooner or later the student's work needs to be evaluated on whether or not a proposed solution will work.

9.3.5.2. Eventually the books must balance and the solution must have the potential to relieve a problem. A solution that comes about as a result of applying a scientific problem-solving method but doesn't work is unacceptable at the highest levels of learning. Checklists or models often helpful at the application level may be of little or no value at the evaluation level. In fact, solutions generated by students functioning at this level of learning are characteristically different from one student to another in their approach and process.

9.3.5.3. When giving students feedback on their products, it may be necessary to help them with difficulties they had in their process of problem solving. This feedback should aid them in their next attempt to produce a product.

9.3.5.4. The concept of sampling described in Chapter 4 still has validity but must be approached with more caution at the higher levels. Predictable samples of behavior characterize the knowledge and comprehension levels but become increasingly difficult from that point upward in the taxonomy. Many samples of behavior are broad indicators of student activity which serve as a baseline for evaluation. The samples can still be of value if they sufficiently describe the essential elements of the product at these levels. However, efforts to write criterion objectives at these levels may produce such general objectives that they are meaningless.

Top of Page; Services; Staff; Roadmap; OnLine Enrollment; Catalogs; Contact Us ; Online Class Schedule; Credit for Military Experience; Degrees Offered; Government and Military Education Department of SDCC; Transcripts, Partnerships, and Articulation; Veterans; SDCCD Student Web Services; F.A.Q.; HOME