Overview | Instructional Objective | Learners | Context | Scope | Object of Game | Design Details Competing Products | Motivational Issues | Design Process | References OverviewFlyNtalk is a solo, flight-simulation experience that requires the student pilot to listen to verbal instructions in English while maintaining a vertical and horizontal tracking task on a Cessna-172 cockpit display using a yoke and instrument panel peripheral device designed for use with the Microsoft Flight Simulator desktop. FlyNtalk features a virtual flight instructor who acts as a test administrator and as a guardian angel who flies invisibly alongside the student pilot. As test administrator, the virtual flight instructor informs the student pilot to maintain heading and altitude as well as possible while listening to verbal instructions and answering yes-or-no type questions by pressing yes-no buttons on the yoke. The virtual flight instructor explains that while weather conditions may or may not vary, verbal scenarios will increase in difficulty as the flight proceeds; furthermore that performance on the verbal tasks will determine if a signoff is received for the communications portion of the solo cross-country check-ride. The virtual flight instructor emphasizes that busting the floor of a predetermined baseline tracking-performance score on a single-task tracking trial represents an unacceptable deviation from known piloting skill and will invalidate concurrent communication scores (Gopher and North, 1974). A built-in scorecard and a timer record the correct and incorrect answers and response times. An algorithm calculates the root mean square of deviations from center on the vertical and horizontal tracking tasks during the listening events. Student tracking performance baselines (asymptotes) are captured on 10 one-minute single-task trials, at five levels of tracking difficulty, at the beginning of the game. Each of these tracking performance levels must be passed to continue the evaluation. The tracking asymptotes on the last five scheduled Microsoft Flight Simulator training sessions and the in-flight evaluations determine the range of tracking performance expected of each student by the time they reach the stage in the training cycle timeline when they test on FlyNtalk. FlyNtalk also uses a Case-Based Learning Environment (CaBLE) for a feedback mechanism to make the game meaningful (Feifer, 1994). If the student makes a mistake, the guardian angel plays short video clips of professional airline pilots or NTSB officials telling a brief story of an accident or incident where a similar mistake caused damage to property or resulted in injury or loss of life. The root mean square function is suppressed during presentation of graphical or visual-based scenarios. Instructional ObjectiveIn essence, FlyNtalk is a flight simulation-based assessment of student pilots' listening skills in a dual-task environment. It enables flight instructors to give their students more objective feedback on their listening performance, and to invoke reflective self-evaluation. Thus, the first instructional objective is to help student pilots gain insight into their ability to comprehend verbal instructions in English during routine and non-routine flight events while maintaining standardized air-work parameters (tracking performance). The second instructional objective is for instructor pilots, management and students alike. Helmreich, Wilhelm, Gregorich and Chidester (1990) discovered that flight instructors often claim student proficiency to eliminate risk to themselves through added student training; therefore FlyNtalk serves to facilitate tough decisions based on objective (as opposed to subjective) flight-instructor observations. Training adjustments invoke sensitive issues between instructor-student, instructor-parent, student-parent, and various other dynamic interactions among students, management, instructors and parents. But these adjustments must be made for the welfare of the student, his or her family and for the safety of the public at large. The FlyNtalk process helps flight instructors justify recommendations to management that would assign student pilots more dual training time, more time on ground-based flight simulators with an aviation English instructor or more time in the classroom. It also serves as a buffer to recommend a change in flight instructors or English-language instructors before flight-instructor endorsement in the pilot logbook, which authorizes solo flight. Whereas poor performance outcomes of FlyNtalk serves to justify recommended training adjustments and to help identify listening problems under a dual-task workload, good performance on FlyNtalk can be used to help students build confidence to fly solo. The relationships between the flight instructor and the student and the established curriculum are especially relevant considering the cost and time to train--as training decisions may impact the financial ability of a student to continue flight training. But there is also a systemic reality that comes with the decision to increase the time to train. This is because successful training rates are used to qualify fixed-base flight operators (FBOs) for FAR Part 141 flight school certification, which is beneficial to management because it provides a more stable base of flight instructors and more rigid curriculum standards that allow for reduced training-times to meet FAA pilot certification requirements. Training adjustments negatively impact eligibility requirements for fixed base operators (FBOs) who may be struggling to gain FAR PART 141 flight-school status because of the following FAA regulatory requirement: " ... the applicant [FBO] must have, within the 24 months before the date of application ... trained and recommended for pilot certification ... at least 10 applicants ... for pilot certificates ... at least 8 of the 10 most recent graduates tested by the ... Designated Pilot Examiner passed the test the first time" (U.S. DOT, FAA, 2000, FAR 141.5). Part 141 schools normally have tougher language-proficiency prerequisites than most FBO operations. From this perspective, FlyNtalk can be used by Part 141 flight schools and FBOs to evaluate transfer students who are at the solo point in their flight training for placement purposes in the flight school curriculums of the gaining training facility. This may serve to protect the eligibility status of FBOs for FAR Part 141 status, and it may help existing FAR Part 141 flight schools to maintain Part 141 status. It also tightens the overall aviation safety net without adversely impacting the business side of the equation, depending on how flight training and language training programs are integrated into the business plan/flight curriculum. LearnersThe game is primarily designed for student pilots of any nationality who speak English as a second language and who are at the threshold of qualifying for being signed off (qualified by a fight instructor) as safe to perform a solo, cross-country flight to a distant airport in unfamiliar--and sometimes unfriendly US airspace. Ages of the average male and female learner typically range from 15 to 30; however, there is no restriction on age or who may play. Players without an aviation background may find the game more frustrating than players who are undergoing flight training who have a vested interest and professional incentive to perform well at the game. Context of UseFlyNtalk is primarily used for testing at a flight school training facility where special scenarios are loaded to ensure control of scenario-based events and control of test questions. To make the game plausible in a flight training facility, the game would require a testing lab with computers capable of running Microsoft Flight Simulator and a server large and fast enough to store and transfer large sound files and video files for playing asynchronously with Microsoft Flight Simulator. There must be at least 50 video explanations for each error type within each category of flight (preflight, take off, departure, en route, approach and landing) which are relevant for the cross-country flight that is selected randomly. The flight experience must be a novel flight experience at the routine and non-routine levels. FlyNtalk is designed so learners can play it more than once; but it only comes with a limited number of practice scenarios. Separate sound files and video files are reserved for use on the pre-solo flight-simulation check-rides to eliminate any carryover effect that previous exposure to these materials may have on the actual test. For example, exposure to specific scenarios may result in measuring how well the learner remembers answers to duplicate scenarios rather than the ability of the learner to respond to novel situations. Prior to the game, the learner will have completed his or her FAA written exam, received 20 hours of dual flight training in a Cessna 172 single-engine aircraft, will have successfully completed a dual cross-country flight with an off-wing instructor, and will have been evaluated for a baseline tracking task to ensure it matches tracking-trials on the actual test (within a reasonable range). This will control for purposefully neglecting to track while focusing solely on the listening task. The learner will be briefed on the requirement to focus priority on the tracking task during the tracking pre-qualification trial of FlyNtalk. Upon successful completion of tracking qualification, the student will be briefed to focus priority on the communication task, but to perform the tracking task as well as possible. Failure to retain baseline-tracking performance will invalidate answers on the communication tasks. After the game, the learner will receive a printout of his or her tracking and communication scores and will meet with a flight instructor to determine eligibility to fly solo on a cross-country flight. In summary, FlyNtalk will be used primarily for testing purposes in flight schools in a formal assessment setting; however, it may be used in flight labs and in classrooms equipped with Microsoft Flight Simulator, using a yoke and instrument panel peripheral device designed for use with the Microsoft Flight Simulator desktop program. It will normally be used singularly, but it can also be used in groups to elicit classroom participation. FlyNtalk is local-area-network savvy and is designed with a playback function for review and analysis in the practice mode. A single playing of the game for the intended user takes approximately 30-45 minutes, depending on the frequency of mistakes and video explanations. FlyNtalk will be developed initially in a PC platform with enough memory and speed to process Microsoft Flight Simulator 2004. Programming is required to integrate the CaBLE-based video files and the assessment and score-keeping features. A data sucker, currently available for use with Microsoft Flight Simulator, is loaded before the flight simulation experience to capture horizontal and vertical tracking performance in an excel file. Root mean square is automatically calculated in an excel file and is entered into a scorecard after the completion of each event, along with the answers to "yes/no" questions in English, which are weighted by task difficulty. ScopeFyNtalk will take approximately 30-45 minutes to play in the individual testing mode depending on the skill of the learner. If played as a group in the practice mode, the game may take up to 60-75 minutes depending on group interaction skills and teacher management skill in the classroom. It will contain a minimum of 25 verbal-based scenarios that elicit yes or no answers throughout a cross-country flight that includes the preflight, takeoff, departure, en route approach and landing categories. Ten one-minute trial tracking tasks shall be administered at the beginning of the test and compared to tracking scores on flight tracking performance on basic air work after 20 hours of in-flight experience in actual aircraft. Disproportionate deviations in tracking performance will require careful review of the student's historical tracking performance on the Microsoft Flight Simulator portions of the flight-training syllabus and of basic air work scores on the last three flights in an actual aircraft. Review shall be made by the student's primary flight instructor. Graphics and video scenarios will be played at any stage whenever mistakes are made, but during this instructional timeframe, tracking performance is suppressed due the competition between visual resources, based on the multiple-resource theory of attention (Wickens, 1997). To control for tracking manipulation during FlyNtalk, the mean single-task-tracking asymptote of each student is obtained on multiple trials on Microsoft Flight Simulator as part of the flight-training curriculum. After logging on to FlyNtalk, 10 one-minute vertical and horizontal tracking-task trials are conducted ad the tracking asymptote is obtained. Asymptotes of the pre- and post-FlyNtalk tracking-task trials are compared. To continue with the FlyNtalk experience, the percentage difference scores between the pre- and post- FlyNtalk tracking asymptotes cannot exceed 5 percent. This controls for tracking manipulation during FlyNtalk after the tracking trials are completed. Post-trial tracking performance in FlyNtalk is designed to index listening-ability. This is accomplished by harvesting tracking scores only during dual tasks that consist of a visual and aural component. After the FlyNtalk tracking-trials are completed, and after it is determined that the user is not manipulating tracking performance, aural tasks may or may not be ongoing in the background. This requires the user to attend continuously to the priority task of listening and filtering meaningful semantic tasks intended for his or her aircraft and those intended for other aircraft. Therefore, mean tracking scores will be measured with and without verbal tasks and with varying levels of listening- and tracking-task difficulty. A percentage difference score between baseline tracking scores for single- and dual tasks will be calculated to index the effort required to listen accurately in English at all levels of difficulty. Yes/no answers are only relevant while the user maintains a predefined tracking baseline while attending first and foremost to the priority task. A score of 85% on the yes/no tasks is required for the mean spare capacity (tracking) score to count. A mean spare capacity score of 50% must be achieved to pass the test. Any tracking deviations that exceed established tracking thresholds (remain within 100 feet of assigned altitude and within 10 degrees of assigned heading in smooth air) will result in a failure. Object of the GameThe primary goal of the game is to score as high as possible on communication tasks while retaining a predetermined tracking performance level. The secondary goal is to earn a maximum score on spare capacity to act in an emergency situation under high workload. In the case where tracking is attended to while communication tasks are performed accurately, the percentage difference scores between asymptote single-task tracking performance and the dual tracking performance on these tracking-and-communication tasks infers the spare capacity left to accurately listen and comply with verbal instructions in a non-routine flight event (such as getting lost inside the clouds in mountainous terrain with defective radio navigation equipment, such as a defective course deviation indicator or a defective VOR). In such a scenario, the pilot must respond accurately and efficiently to verbal commands from ATC to avoid controlled flight into the terrain (CFIT). In the testing mode, the End State is to pass the pre-solo simulated flight experience with, as high a score on spare capacity, and on the communication tasks as is possible. In the practice mode, the end state is to score high, but primarily to identify and learn from mistakes. Design DetailsUniversal Elements Instruction and assessment. FlyNtalk is both an assessment in a Cessna 172 aircraft flight-simulation experience and a CaBLE-based learning environment that uses graphics and audio or video to tell a story before giving a navigational task. Control Number. The program opens when the FlyNtalk icon is double clicked. A log-in screen appears inside a dialog box, center screen, with text that instructs the student to enter a control number that matches his or her identity using a keyboard and a mouse. If a wrong number is entered or if more than 1 minute lapses, then a dialog box will appear, instructing the student to get an instructor. Responses and Volume Control. After successfully entering the appropriate log-in information, a dialog box and text appears. The dialog box instructs the student to ensure that the computer volume on the headset is in the off position. A yes-and-no box elicits an answer. If the student clicked on no to the question, "Is the headset volume turned off," then the dialog box instructs the student to ensure that the headset volume is all the way down. If yes, the computer starts playing a sound file, and displays a dialog box with text that instructs the student to place the headset on and to carefully adjust headset volume. Another dialog box appears that asks if the student can hear the audio in the headset and is ready to proceed. If no, the program replies with a dialog box and text instructing the student to seek instructor assistance. If yes, the program informs the student that the remainder of the instructions will be delivered verbally via sound files or video files. Interface Device. The user must execute flight controls with a hands-on yoke, operable throttle and rudder pedals that are operated with the user's feet. A headset is worn to be able to listen to commands from ATC. Visual and Aural Fidelity and Experience Level of the Learner Learner involvement will wane with experience and inappropriate levels of fidelity. Fidelity must be commensurate with the stage of learning (Alessi, 1988). Expert pilots fly advanced aircraft that display complex instruments, electrical switches/panels, and more complex hydraulic and power systems. Moreover, higher levels of team interaction and cockpit resource management are required. Additionally, the expert pilot needs high-workload challenges to enhance learning. High levels of visual and psychological fidelity are provided with full motion mechanical flight simulators and the high visual fidelity present inside and outside cockpit views. Flight engineers, copilots, flight crew and FAA examiners interact with expert pilots while they attend to routine and non-routine tasks. This challenges the expert to improve learning, but may distract the novice pilot. Therefore, FlyNtalk (a desktop flight simulation experience) is appropriate for novice and early intermediate pilot trainees; whereas, full motion flight simulators of the highest visual and aural fidelity are more appropriate for expert pilots. Alessi (1988) characterized the relationship of fidelity to learning for beginning, intermediate and expert level pilots, as illustrated in the following diagram:
Therefore FlyNtalk is inappropriate for the expert because it does not possess the high degree of physical-, visual- and motion-fidelity that is required to motivate expert pilots. Antithetically, too much physical-, visual- and motion-fidelity can overwhelm novice pilots because novice pilots are not as experienced as the experts who can quickly recognize complex scenarios and embedded visual cues. Inter-cockpit aural fidelity, however, is high in FlyNtalk because expert and novice pilots share the same airspace and the same radio frequencies. Most communication occurs close to the terminal areas and in controlled airspace where novice and expert pilots must share information. Degree of Difficulty As pointed out by Alessi (1988) novel situations commensurate with the experience level of the user are needed to evaluate flying and communication skills. But novelty must be meaningful (Flach, Hancock, Card and Vicennte, 1995). In FlyNtalk, attention goes beyond the goal of arousing epistemic curiosity (Berlyne, 1965) that is alluded to in the ARCS model. It is safe to say that FlyNtalk generates a survival curiosity by creating novel challenges in the form of unexpected, non-routine events that the user will experience. For example, the user could encounter an unannounced emergency in flight to which he or she must react to (in an accurate and efficient manner) in order to survive. Therefore, the degree of primary- and secondary-task difficulty (listening and tracking, respectively) in FlyNtalk can be manipulated within each stage of the flight plan (preflight, take off, departure, en route, approach and landing). The two types of events that can be manipulated are routine and non-routine. Routine events are expected and can be planned for. But non-routine events are not expected and they increase pilot workload. So, the type of event and the stage flight are independent variables in FlyNtalk that can be manipulated for the purpose of investigating the ability of the student pilot to listen and comply with instructions in expected and unexpected scenarios. But in order to refine the assessment, tracking and communication tasks must remain challenging enough to require the student pilot to divide his or her attention under high workload; else the performance on the dependent variables of interest (tracking and listening) will be meaningless. Therefore within the routine and non-routine types of scenarios that take place throughout the stages of flight, the frequency of verbal commands or the content complexity of verbal commands may be increased or decreased to place an acceptable level of workload on the student pilot, so that he or she will place priority on the primary task (listening). For the secondary task (tracking), 1) the frequency of weather variables (turbulence) or 2) intensity of weather variables may be increased or decreased to ensure that the task requires a meaningful and challenging experience. Measures of motivation: Affective reaction level (rating scale or semantic differential scale--degree of like or dislike). Motivation in FlyNtalk is sustained by the requirement in the Flight Syllabus to pass the FlyNtalk flight simulation experience before being allowed to proceed to the next stage of flight training. Assessment Phase of FlyNtalk: Users perceptions of their performance on listening tasks during FlyNtalk should be correlated with actual outcomes on the FlyNtalk experience, and with flight instructor evaluations on the pre-solo evaluations; also on flight examiner evaluations on the terminal check-ride for pilot certification. To this end, rating scales should be used as an instructive tool to help users develop insight and ownership of their ability level to communicate in English while flying under different workload conditions. This can be useful during a validation study of FlyNtalk if it is to be formally used as a prerequisite to cross country solo flights into controlled airspace where student pilots are required to communicate with ATC (Ground, Tower, Departure, En route and Approach air-traffic-controllers). The level of fidelity and user proficiency level should be pre-defined. Flyntalk Tracking Error and Listening Trials on Single Tasks Mean single-task tracking error asymptote is obtained during the ten scheduled Microsoft Flight Simulator sessions of the flight curriculum. Simulator effect will have been eliminated after the first two 1-hour simulation practice sessions. Tracking asymptote is harvested on the final five 1-hour Microsoft flight-simulator sessions, which are progressively difficult; and they must be successfully completed prior to the dual-cross country check flight. The dual cross-country check flight is successfully completed prior to the FlyNtalk assessment. Unreasonable deviation from baseline tracking performance on FlyNtalk will result in termination of the assessment. Instructional Phase of FlyNtalk: Motivational Rating scales are more appropriate for the learning phase of FlyNtalk as it relates to the instructive feature in the CaBLE-based videos after the users make mistakes during the FlyNtalk experience. It is important to note that even though minimum standards must be maintained in order to pass the private pilot terminal check-ride for pilot certification, the check-ride itself is also used as a learning experience and the examiner may give the applicant instructive cues throughout the examination. Note in the third cell of the following illustration, that testing is an inherent part of instructional design:
From Simulation Fidelity in Training System Design: Bridging the Gap Between Reality and Training (p. 8), by R. T. Hays and M. J. Singer, 1989, New York: Edwards Brothers. Copyright 1989 by Springer-Verlag. Hygienic Factor: Before an instructive video file or audio file is played in the in-flight portion of FlyNtalk, the student pilot must make a mistake and the guardian angel must take control of the aircraft. The FlyNtalk script then selects a video at random from a cue of videos appropriate for the category of error. It does this by using an if command, a randomizer command, a put command and a play command. In order for an error to occur in flight, the user must exceed predetermined minimum or maximum threshold values that are programmed into FlyNtalk as global variables (e.g. altitude within plus or minus 100 feet, airspeed within plus or minus 5 knots, heading within 10 degrees). Another way an error may occur is when the user fails to respond to a discrete command within a reasonable response time such as responding to a squawk command ("Squawk 5461"), dialing in a radio frequency ("Contact Approach on 118.35”), or initiating a turn in the wrong direction ("Turn left now!”), after a reasonable response time (RT) appropriate for the scenario. A progressive disclosure bar will indicate the reasonable time with which to respond to each command scenario. When errors occur, the FlyNtalk program script automatically categorizes the type of error and enables a list of video-files or sound files that are linked to the category-of-error type. If there are multiple errors at once, the program selects the first category-of-error type. The script activates a randomizer command that selects from video files or audio files from the selected category. Then the put command puts the name of the video-file or the sound file into the play command. To ensure the same video is not "fired"(instantiated) for a repeat error, the number one (1) is entered into a field with the same name as the file that has already been played. An if command restricts the sound or video file from being selected by the randomizer in these cases if a 1 exists in fields with the same name as the video or sound file. A different video file or audio file will be played for repeat error type. Specific Elements Description of the Categorization Scheme used for the Questions The 25 Questions are categorized by type and level between the five progressively difficult, sequential categories of a flight plan. The progressively difficult categories in which the questions are housed are the 1) Preflight, 2) Takeoff, 3) Departure, 4) En Route and, 5) Approach to Land categories of flight. The flight plan takes the user from the parked aircraft at one airport with verbal tasks about common aeronautical knowledge, to the runway and questions that require the user to operate his aircraft on the ground while attending to surrounding aircraft and communications. Then up into the air to the departure pattern where the user must control the airborne aircraft with bumpy wind from trees and buildings while clearing the area for traffic and monitoring the radios and communication tasks. Then to the en route category where turbulence and emergencies will occur; and finally to the execution of an unfamiliar instrument approach in unexpected Instrument Meteorological Conditions (IMC) in congested airspace. Here, the user must very accurately control his aircraft and respond equally efficiently and accurately to time-critical radio communications. Below is a chart that outlines the category, content and task difficulty of questions; task increases from left to right: Task Difficulty Increases from Left to Right
Grouping of Sound files. The sound files are grouped by category and time functions. The answer codes of 1 and 2 (Yes or No) are linked to the answer fields for each sound file. Sound files are triggered by the computer timer and are selected by a randomizer script that has a memory function (remembers what file already played). Background Stories (Context) Graphics with audio files and vide files provide background stories to form the context and levels of difficulty in which the user must operate and respond with yes/no questions to demonstrate understanding of verbal instructions while maintaining baseline tracking skills. CaBLE files. These sub-files are arranged by category and difficulty level of challenge and will only fire when a mistake is made (Code 2 in the answer field). For example, there are five (5) CaBLE files for Altimeter in the Category-and-Task Difficulty Chart, above. The altimeter refers to an altitude error in all categories1 ~ 5, but a category 1 altimeter error will likely show a picture of an altimeter and show how setting it wrong changes field elevation; it would be coded as P1 (Random Cable1~5) or a unique file name like P1R3 after the randomizer chooses an unused file within the Preflight-Cable category. For Category 4 CaBLE files, an altimeter-error movie or CaBLE-category sound file may illustrate a mid-air, so the code would be E1 (Random CaBLE1~5) or a unique file name like E1RC2 after the randomizer picked the number 2 out of the random group selection of (1,2,3,4,5). NOTE: There are 5 CaBLE files for each Category and line number as indicated by (1,2,3,4,5 ...) Instantiation of the Sound Files. The files fire at random within each of the five progressively difficult categories. Sound files that are played are remembered by the program soas not to fire the same sound file twice. Omitting the repeat (say-again) function was tradeoff that reduced ecological validity, but which compensated for the level of difficulty needed to offset the relative ease of answering yes-or-no to questions of increasing task difficulty. Data Collection Rule. When graphics are presented that are not part of the outside of the cockpit view, the recording function of tracking performance is suppressed. This is based on the undesirable competition between similar resources referred to in the Multiple-Resource Model of Attention (Wickens, 1994). Global Variables, Independent Variables and Dependent Variables Independent Variables 1. Tracking Task Difficulty: Turbuence. Levels a) Smooth, b) Moderate 2. Listening Task Difficulty: Complexity of Sound File. Levels a) Identify Description, b) Filter Semantic Noise Dependent Variables 1. Tracking Performance: Root Mean Square Scores. Levels a) RMSv and b) RMSh 2. Language Score: Listening Score on yes no questions Use of Global Variables. Globals timePreflight, timeTakeoff, timeDeparture, timeEnroute, timeApproachtoland, Activating the Right Sound File Category. If timePreflight is true, and if CaBLEplay is false, then go to category preflight soundlist, else if timeTakeoff is true and if CaBLEplay is false, then go to category takeoff soundlist ... etc (There is a sound list and time for each category of flight; eg., preflight = timePreflight (00:01 ~ 02:00); takeoff = timeTakeoff (02:01 ~ 04:00), and so forth. Randomly Selecting Sound Files within a Category. Get the random of the caegory preflight soundlist. If it is not in preflight field alreadyplayed, then Put it into the soundname. If fieldname soundfield is empty, put 1 into it and Play the soundname. At sounddone put "empty" into fieldname soundfield. Wait 8 seconds (allows user to click yes or no and allows computer to record answer and calculate tracking score). Put the soundname into field alreadyplayed. (Program is time-and-category smart). Adjusting time functions for user mistakes. If the user makes a mistake, and the Cable library needs to play a video or graphic-based sound file, then the time settings of all categories are updated by the duration of the Cable-based video or sound file that is played. Multiple true/false fields are used with if commands to control this time adjustment function for all five categories. However, the test will have a 75-minute maximum time limit. Failure to complete all five categories will result in extra time to train in the aircraft Answer Response Time. The student has 8 seconds to click on the correct component. Any click on an incorrect component will result in a score of 0 points for the sound file, and the next sound file will be instantiated. A correct click places a 1 in an answer field with the fieldname of the sound file. The scoring mechanism is programmed to add up the correct answers and to place the name of the incorrectly-answered sound files into a review field that is printed out at the end of the test. For correct answers, points are calculated automatically and retained in the appropriate category answer-field. Mistakes. When the student makes a mistake in the preflight category of FlyNtalk, video files appropriate for the part missed play to show an instructor pointing at the aircraft component explaining its function. The student must earn 100% on the preflight portion of the test to continue; else, the test will terminate and print out the right and wrong answers, as selected by the student. . Sample Questions Within Each Category Showing Progressive Levels of Difficulty The preflight category is the first task category that appears. It shows the unlabeled parts of a Cessna 172 aircraft and then tasks the user (verbally) to click on randomly selected parts of the airplane to indicate he or she hears and understands, and can identify the aircraft part or component that was spoken. As an additional part of the preflight the pilot also needs to understand the weather and the forces of the wind and how to express relative direction. In the picture below, it can be inferred that if the top of the picture is oriented to the North, and if a 27-09 runway stretched towards the east and west, then the active runway will be 27 at night if the only change is the opposite temperatures of the land and the water.
Takeoff Category. After the preflight category, the primary scene is the inside view of a Cessna 172 cockpit with instrument panel. Runway events on the ground will center around listening up for approaching traffic, dialing in appropriate frequencies, taking off on the correct runway and demonstrating accurate understanding of tower instructions. For example, the sound file might say: Sound file: "Cessna Five Niner Four Hotel, Lear jet on short final, cleared for immediate take off, roll it right off." Question: Is the Tower telling you to take off immediately because there is a jet on the runway waiting to take off? Departure
Category. Accurate responsiveness to heading and altitude changes and transponder codes will be challenged during this flight category, but a major part will also address location and the language function of understanding relative position. The instrument panel remains the same, but the out-of-the cockpit views vary depending on the flight plan that the user will fly, his or her geo-spatial location and the scenario that is set by the program. For example, The program may show the student pilot in the following screen shot and ask the student to respond with a yes or no click on the answer button on the yoke to the following statement: "From your view outside the aircraft, your aircraft is beyond the bridge."
En Route Category. During this category, the bulk of the tracking in various degrees of turbulence will take place and where pilots must comply with altitude and heading changes to avoid midairs. This is also where the bulk of the CaBLE videos will be played and where CaBLE adjustments will be made for time constraints that were onset by a high frequency of mistakes in previous categories. Graphics and sound files may respond to altitude errors in the en route category because if the student pilot misunderstands a command to descend to a specific altitude, he or she may fly perfectly into a mountain, or in the case of this seasonal demand, he could cause death and destruction that may impact many people. Consider the following screen shot after the student pilot descends to the wrong altitude:
In more complex scenarios, the student pilot may be given a story via a sound file, while he is flying, and then asked to answer yes or no to a question or statement after viewing a graphic. For example, the scenario may begin by the outside view of the cockpit going grey to simulate flight in the clouds. Then a verbal command will be given with a verbal scenario, and questions, as follows: Approach to Landing Category Command: Maintain five thousand feet and a heading of three zero and answer yes or no to the questions by keying your yoke mike once for a yes and twice for a no. It is possible that voice recognition features could be used with a headset and boom mike, and the computer programmed to place a 1 or a 0 in answer fields to represent a yes or a no answer. Situation: You failed to put the left wing fuel cap back on securely and the low air pressure over the left wing has siphoned most of your fuel. You must fly an instrument approach into Vero Beach Florida using the Vero Beach Runway 11 Right approach. Question 1: Did you fail to secure the right wing fuel cap? Question 2: Must you fly the VOR Runway 11 Left approach? Tracking performance is recorded throughout the aural presentation of Scenario One, including the question and answer session. Scenario Two Situation: The student pilot is presented with an approach plate of the Vero Beach 11 Right approach. Command: Look at the inbound and outbound courses you must fly after station passage
Question: You must fly a heading of 120 after station passage to runway 11 Right. Answer yes or no. Tracking performance is suppressed during the graphical presentation of the approach plate. Scoring and Scoring Tables
Microsoft Flight Simulator Tracking Asymptotes
FlyNtalk Tracking-Qualification Recording Sheet (Primary Task is the tracking task on these 10 trials)
FlyNtalk Listening-Capacity Recording Sheet (Primary Task is the Listening Task)
In-Flight Evaluations by the Flight Instructors Date_______________ Student Control Number _________________________
Note: Degree of firmness. The higher the number, the firmer the assessment for the assigned pass or fail (P or F). For example: F-10 is the worst possible score. P-10 is the best possible score P-0 is a just passed score. F-0 is a just-failed score The lowest to highest scores would be represented as follows from left to right: F10, F9, F8, F7, F6, F5, F4, F3, F2, F1, F0, P0, P1. P2, P3, P4, P5, P6, P7, P8, P9, P10 Technical Elements
Competing ProductsThe closest product that I can find to this product is Microsoft Flight Simulator's practice mode for flying and the verbal prompts it gives throughout the flight. Although it does not have an English communication and spare capacity evaluation component, it does have a feature where a flight can be reviewed for tracking tasks. FlyNtalk is the only game out there that measures spare capacity and the accuracy of the user’s compliance with verbal instructions on a desktop flight simulation experience. Moreover, FlyNtalk gives specific learning scenarios and explanations to develop insight for mistakes in communication. This does not require the presence of an instructor because explanations are given by subject matter experts using short video clips and the CaBLE method of meaningful learning. Motivational IssuesThe motivation of the student-pilot users of FlyNtalk is utilitarian. Passing a cross-country dual-flight stage-check evaluation enables the student-pilot user to be administered the FlyNtalk flight-simulation experience. Passing the FlyNtalk experience enables the user to proceed to the next stage of flight training. This comprises an ends level of motivational relevance and confidence (Raynor, 1974) and also builds goal orientation relevance (Keller, 1979). Unless the user is not recommended to be administered the FlyNtalk experience, little relevance- and confidence-oriented motivational strategy is needed for the assessment portion of FlyNtalk. This is because the user who takes FlyNtalk as an assessment-based experience has already been recommended to do so. In and of itself, this is a confidence builder (Keller and Suzuki, 19--). As for relevance, a successful FlyNtalk outcome is required to obtain the user goal of earning a pilot license. Since the FlyNtalk user interface is the same as the desk-based flight simulations that are built into the flight-training syllabus, no confidence-based motivational enhancements to the FlyNtalk interface will be required. FlyNtalk is administered after completion of a successful cross-country check-ride with an off-wing instructor pilot. That experience allowed the user to make mistakes within reasonable parameters and remain safe; therefore, authorization to be tested using FlyNtalk invokes a level of competence and confidence and sends a message to the student pilot that the end task is doable (White, 1959). Therefore, FlyNtalk allows for reinforcement of perceived levels of confidence attained in past and recent flight-training experiences. Moreover, as a simulated flight experience, it still provides a safety net, which allows the user to focus on communication skills, which is the dependent variable of interest that is being tested. The motivation of flight instructors to administer FlyNtalk is their interest in its usefulness as an objective assessment tool. It serves as a blind observer. This helps eliminate observer bias by providing objective data instead of the flight instructor interpretation of student-pilot behavior (Bordens and Abbott, 1999; p. 155). FlyNtalk also helps eliminate role attitudes and demand attitudes. For example, the pressure placed upon flight instructors by management and upon the student pilot being evaluated could contaminate the evaluation process (Adair, 1973). Finally, FlyNtalk can help flight instructors avoid law suits in cases where injury or fatality occur as a result of flight instructors endorsing students safe for solo, knowing the students are not really qualified (Marko, 1993). Design ProcessI first wondered how some pilots who couldn't speak English well got to fly 757s and larger aircraft without being able to speak and understand English well enough to know which way to turn or what to say to clarify information in sticky situations. Not being bilingual I wanted to understand how to help second language speakers focus on how to react more accurately and efficiently in English in time-critical situations. Certainly that would be a confidence builder. With big accidents like in the Canary Islands or in Cove Island New York, a couple misunderstandings killed a lot of people. So how can we as trainers and teachers assess or help to improve one's ability to cope in English in dynamic domains? Stanley Dornic did studies on spare capacity. His participants performed tasks in English while performing another task requiring a motor skill that did not compete heavily with the language task. He found that there was a cost that could be measured by checking the performance score of the second task as long as the language performance was at the standard performance level--or close to it So why not try it with pilots? Maybe we can really find out how much mental gas is left while pilots are flying that 757 in the gue on a final approach, not knowing that they are looking at the wrong instrument while just a minute away from hitting a crane that is positioned on a closed runway. When the stress of nature and man-made equipment converges, it is harder to concentrate and divide your attention. This is especially true if you are having difficulty understanding a busy controller give you last minute instructions in your weaker language. You get my drift. So that is what got me interested. The process. First I read the assigned articles again. Then I revisited the Secondary Task technique and the writings of Christopher Wickens, Daniel Gopher and Daniel Khaneman. I also revisited Steve Alessi's articles on Fidelity and the stage of the learner. Then I proceeded to look at the variables I wanted to manipulate and those I wanted to measure. Had a rough time with my computer because my Mac power book fell apart and I had to bounce back and forth with FTP sites. Bought a new Mac. I followed the outline that Bernie gave us and tried to connect the dots. I needed pictures as I am mostly right-brained. The pictures were the easiest things to put together, so I started in the middle of the design section. But had I listened to Bernie, I would have done more homework first and followed the script. I started thinking that it would be hard to get foreign pilots to take this test by coming to America since the advent of 9/11, but then it dawned on me that this could be a great instructional tool if we added the CaBLE theory into it. I remember Rick Feifer showing classmates how much better we all seem to learn right after we make effortful mistakes and when there is an expert there to explain what went wrong and why. So that got me going with the assessment/tutorial concept. That motivated me to search the web more for expert videos that are out there, and the NTSB has some. The initial idea I came to reject was to incorporate a speaking component. It is too hard to manage and it requires trained people (teachers), something that costs money and something that is not self-contained with a computer. It was a tradeoff, but I felt good about it. Now I could focus on making situations meaningful and addressing how to connect the CaBLE files with user-mistakes, and try to figure out how to manage the rest of the sound files (dovetail them). I became absorbed in these programming thoughts and the complexities I created for myself. I did not do much on feed back because the design involves a lot of technical data that steers folks to loose interest. I did go on line and buy the ICAO's latest on aviation English testing. The biggest lesson I learned is that I need to simplify my approach and work with a team. Going alone is hard and not too smart a thing--given the time constraints. There are always opportunity costs out there. Time management, teamwork and simplification are what I am going to carry to my next project. ReferencesBooks & Journals
Electronic
Last updated December 28, 2005 |
