Project Plan Strategy Assessment
Federal Aviation Administration
NextGen Advanced Concepts & Technology Development
Human Factors Division (ANG-C1)
Air Traffic Control / Technical Operations
Human Factors Research
Plan and Process
FY 2013
Table of Contents
Acronyms Used In This Document....................................................................... 2
1. Purpose ...................................................................................................................... 3
1.1. HF RE&D Portfolio Goals ................................................................................. 3
2. HF RE&D Portfolio Projects .................................................................................... 4
3. Research Project Lifecycle ....................................................................................... 7
3.1. Stage One: Requirements Definition ................................................................. 7
3.1.1. Requirement Evaluation.............................................................................. 7
3.2. Stage Two: Research Budget Appropriation/Procurement Materials ................ 8
3.3. Stage Three: Research Execution ...................................................................... 8
3.3.1. Agreement Administration.......................................................................... 9
3.3.2. Quality Assurance ....................................................................................... 9
3.3.3. Technical Community Requirements Group (TCRG) ................................ 9
3.4. Knowledge Transfer and Implementation / Research Product Processing ...... 10
4. Project Closeout Process ......................................................................................... 10
5. Goal Reporting ........................................................................................................ 10
References ................................................................................................................... 11
Appendix A: HF RE&D Portfolio Project Descriptions ............................................. 12
Appendix B: FAA Agency Codes Used in this Document…………………............35
Acronyms Used In This Document
The following list defines acronyms used throughout this document.
Acronym Definition
AATS Advanced Air Traffic Systems
ATC Air Traffic Control
ATCOV Air Traffic Color Vision Test
ATCS Air Traffic Control Specialist
AT-CTI Air Traffic Collegiate Training Initiative
ATO Air Traffic Organization
ATOS Advanced Tech Ops Systems
ATS Air Traffic Services
AT-SAT AT Selection and Training Test
DFS Deutsche Flugsicherung
DPG Design Process Guide
EEG Electroencephalography
ERAM En Route Automation Modernization
FAA Federal Aviation Administration
FRMS Fatigue Risk Management System
FY Fiscal Year
HF Human Factors
HSI Human Systems Integration
I&TP Individual and Team Performance
IAD Institute of Ergonomics
NARP National Aviation Research Plan
NAS National Airspace System
OA Operational Assessment
OCC Operations Control Center
OMB Office of Management and Budget
PI Principal Investigator
PMO Program Management Office
PS&T Personnel Selection & Training
R&D Research and Development
RE&D Research, Engineering & Development
RMM Remote Maintenance Monitoring
SHARE System for Human Factors Assessment and Readiness
Evaluation
TCRG Technical Community Requirements Group
TO Technical Operations
TOFRM Technical Operations Fatigue Risk Management
TRACON Terminal Radar Approach Control
TRL Technology Readiness Level
1. Purpose
This document maps FY 2013 HF Research, Engineering & Development (RE&D) Portfolio
research projects to the National Aviation Research Plan (NARP), details the Human Factors
research project lifecycle, and lists ongoing research activities. The Air Traffic Control (ATC) /
Technical Operations (TO) Human Factors (HF) Research and Development (R&D) Plan outlines
fiscal year (FY) 2013 research projects and program processes managed by the Human Factors
Research and Engineering Division (ANG-C1) Program Management Office (PMO) in the Federal
Aviation Administration (FAA).
The NARP describes the FAA’s research portfolio by outlining the agency’s mission, vision, and
goals, which together work to define the scope of research and development within the FAA (FAA,
2011). This R&D plan documents the efforts currently underway by ANG-C1 to realize those
goals.
1.1. HF RE&D Portfolio Goals
The ATC / TO HF program supports NARP R&D goals which are aligned with Flight Plan goals.
These publications highlight a necessity for increased safety and greater capacity in the National
Airspace System (NAS). R&D performers support these goals through the development and
execution of sponsored research projects. Results of these projects address identified knowledge
gaps and reduce the operational shortfalls, thereby supporting FAA strategic goals and initiatives.
ANG-C1 supports these goals through four HF RE&D research portfolios: Advanced Air Traffic
Systems (AATS), Advanced Technical Operations Systems (ATOS), Personnel Selection &
Training (PS&T), and Individual & Team Performance (I&TP). Each portfolio consists of projects
that are generated from research requirements. The following descriptions illustrate the goal and
objective of each portfolio:
• Advanced Air Traffic Systems: AATS research supports proposed technology that provides benefit to controllers and maintainers as well as the development of relevant
system standards. This will help to maximize human and system performance by testing
innovative air traffic concepts and technology.
• Advanced Technical Operations Systems: ATOS research supports efforts to effectively manage the needs of the NAS operational and maintenance infrastructure.
• Personnel Selection & Training: PS&T research is centered on selecting personnel who possess the knowledge, skills, and abilities required for ATC system job performance.
Additionally, this research focuses on defining performance requirements and objectives.
Research concentrated on selection and training will improve the quality of air traffic
service providers and decrease attrition rates.
• Individual & Team Performance: I&TP research focuses on identifying contributing causal factors to human error in air traffic accidents and incidents and reducing human
error as well as the improvement of human performance within the air traffic system.
Mitigation of environmental and individual factors will enhance the ability for air traffic
service providers to perform required tasks with improved safety, accuracy, and
methodology that builds on HF RE&D program goals.
2. HF RE&D Portfolio Projects
ANG-C1 research efforts directly map to the NARP R&D Goal 3 and coordinate with Goals 4 and
8.
• Goal 3: High Quality Teams and Individuals aims to demonstrate improvement in air navigation service provider efficiency and effectiveness through automation and
standardization of operations, procedures, and information by 2016.
• Goal 4: Human-Centered Design will demonstrate that operations, procedures, and information can be standard and predictable for users at all types of airports.
• Goal 8: Situational Awareness focuses on demonstrating common, real-time awareness of ongoing air operations, events, crises, and weather in all phases of flight and at all types of
airports by pilots and controllers.
Following is a table of active research projects. Detailed information can be found for each project
in Appendix A: HF RE&D Portfolio Project Descriptions
Table 1: ANG-C1 Active HF RE&D Research Projects
Project Title Portfolio Goal # BLI Sponsoring Organization Performing Organization FY Start FY End Page #
Design Process Guide AATS 4, 3 A11-i ANG-C1 NIA FY 10 FY 14 12
An Inventory, Analysis, and Design
Standard for Alarms and Alerts in
FAA Airport Traffic Control Towers
AATS 3 A11-i AJM ANG-C1 FY 11 FY 13 13
Further Evaluation of Standard Color
Palette for ATC Displays AATS 4, 3 A11-i ANG-C1 ERAU FY 12 FY 14 14
Air Traffic Control Markings and
Symbology Standard AATS 3 A11-i AJM NIA FY 07 FY 13 15
Update the Human Factors Design
Standard AATS 3 A11-i ANG-C1 Contractor FY 11 FY 15 16
ATC Color Standard AATS 4, 3 A11-i ANG-C1 ERAU FY 11 FY 14 17
Standardization of Graphical User
Interface ATOS 4 A11-i ANG-C1 TASC FY 13 FY 14 18
HF Standard: Abbreviations and
Acronyms for the Tech Ops
Environment
ATOS 4, 3 A11-i AJW Booz Allen Hamilton FY 12 FY 13 19
Tech Ops Symbology Standard ATOS 4, 3 A11-i AJW Jenius FY 09 FY 14 20
HF Standards for Tech Publications
Used by Tech Ops ATOS 4, 3 A11-i AJW TASC FY 12 FY 14 21
Dynamic Comprehension: Time on
Position and Mental Fatigue I&TP 8 A11-i AJI AAM-520 FY 09 Ongoing 22
Organization Development in
Operations Command Centers (OCCs) I&TP 3 A11-i AJW AAM-520 FY 09 FY 14 23
Education/Communication
Development and Evaluation of
Fatigue Interventions in Tech OPS
I&TP 3 A11-i AJI AAM-520 FY 11 FY 14 24
Fatigue Risk Management System
Education, Communication, and
Evaluation with Air Traffic Control
I&TP 3 A11-i AJI AAM-520 FY 11 FY 15 25
Profile Analysis of Basic Work Rest
Cycle during the Controller Duty day I&TP 3 A11-i ANG-C1 AAM-520 FY 11 FY 13 26
Evaluating the Effectiveness of
Schedule Changes for the Air Traffic
Services Providers
I&TP 3 A11-i AJI NASA AMES FY 09 FY 13 27
HF of Causal Relationships between
Periodic and Corrective Maintenance I&TP 3 A11-i AJW Jenius FY 10 FY 15 28
AT-SAT: The Longitudinal Validation
of the Air Traffic Selection and
Training Test Battery
PS&T 3 A11-i AJI AAM-520 FY 06 Ongoing 29
Concurrent Validation of the AT-SAT
for Placement PS&T 3 A11-i AJG AAM-520 FY 07 FY 13 30
Improving ATCS Selection from
Sources other than the General Public PS&T 3 A11-i AHR AAM-520 FY 11 FY 15 31
Selection Tests for Air Traffic
Specialists PS&T 3 A11-i ANG-C1 AAM-520 FY 11 Ongoing 32
Update and Deploy Practical Color
Vision Test for ATCS Applicants PS&T 4, 3 A11-i AAM AAM-520 FY 08 FY 13 33
Assessing the Operational Assessment
Process PS&T 3 A11-i AJG AAM-520 FY 12 FY 15 34
Error! Not a valid link.
3. Research Project Lifecycle
All research projects advance through four stages in their lifecycle. The four stages are:
1. Requirements Definition 2. Research Budget Appropriation 3. Research Execution 4. Knowledge Transfer and Implementation
3.1. Stage One: Requirements Definition
The research project lifecycle begins with the identification of an operational need for research
which warrants investigation. The need for human factors research can be identified by programs,
workgroups such as the Technical Community Requirements Group (TCRG) meetings, technical
reports, technical organizations within the FAA, and other sources. The final product of this stage
is a requirements statement.
A requirements statement consists of five defined parts:
1. Shortfall / Knowledge Gap 2. Benefit 3. Product 4. Schedule 5. Product Use
The identified shortfall / knowledge gap states the need for research. The benefit describes the
advantage in closing the shortfall / knowledge gap. The product is the deliverable fulfilling the
requirement that generated the research. The schedule indicates when the final product is due.
How the product will be used is required to ensure successful implementation into the NAS. A
complete requirements statement is required for authorization and funding of each project.
3.1.1. Requirement Evaluation
Research requirements are evaluated by the Program Managers and are prioritized according to
established criteria. The criteria are as follows:
Support of Flight Plan, NARP, and Program Goals
Flight Plan goals: Increasing system capacity, safety, and organizational excellence through
objectives, strategies and established initiatives.
NARP goals:
• Goal 3: High quality teams and individuals
• Goal 4: Human centered design
• Goal 8: Situational awareness
Technical Feasibility
The required data or resources must be available to complete and deliver the final requested
product. A requirement that demands resources that are unavailable or insufficient may be
deemed infeasible by the Program Manager.
Support of Programmatic Policies
The proposed research must not pose a significant threat to other projects already underway or
in consideration. Additionally, the program’s areas of research concentration must remain
balanced. If the requirement specifies research that would halt other priority work or would
reduce the level of effort in other areas of focus, the requirement may be deemed infeasible by
the Program Manager.
Schedule
Requirements are generated and submitted by Sponsors up to three calendar years in advance.
This ensures adequate funding, technical feasibility, and resource availability. If a requirement
is submitted outside of the planned schedule and budget, it is started and funded based on
availability.
Fiscal Year’s (FY’s) Budget
Research funding requirements must be within the FAA’s yearly budget. Budget submission
and resource availability is determined and balanced by the Program Managers.
3.2. Stage Two: Research Budget Appropriation/Procurement Materials
The second stage of the research project lifecycle involves determining the research budget
appropriation. Annually, the FAA Administrator is required to submit the NARP to Congress with
the President’s budget. The NARP includes applied research and development defined by the
Office of Management and Budget (OMB) Circular A-11 and involves research activities funded
in four appropriations accounts: Research, Engineering and Development, Facilities and
Equipment, Airport Improvement Program, and Operations. R&D funding for the ATC/TO HF
program is noted as budget item A11-i in the NARP appendices (FAA, 2012). Each portfolio
within ANG-C1 denotes specific funding levels for projects. The funding amounts are described
in the Spend Plan, an annually updated internal resource planning document.
3.3. Stage Three: Research Execution
Once a research project has been approved, a principal investigator (PI) is selected to conduct the
research. PIs may be affiliated with universities, non-profit organizations, government agencies,
or consulting firms. PIs are tasked to perform research on the requirement in the form of a grant,
co-operative agreement, intra / inter-agency agreement, or contract. The PI remains in contact with
the project Sponsor and Program Manager to provide required status updates on the project and
deliverables.
The Sponsor is the direct beneficiary of the completed research. Throughout the project lifecycle,
the Sponsor provides resources and coordinates with stakeholders. The Sponsor has a thorough
understanding of how the final deliverable fits into the NAS plan, what the deliverable’s impact is
on the existing system and what the implementation strategy is. The result of stage three is the
implementation of research.
3.3.1. Agreement Administration
The agreement is administered and monitored by the Program Managers. Schedule adherence,
deliverable status, and deliverable quality are reviewed and documented throughout the research
project. Internal administration tools are utilized by the Program Manager to progressively track
schedule adherence, milestones, deliverables and significant achievements. These tools are also
used to report the Flight Plan and NARP goals that projects will satisfy.
3.3.2. Quality Assurance
Semi-annual TCRG meetings, semi-annual program reviews, and stakeholder reviews of products
are quality assurance measures that the Program Managers utilize.
3.3.3. Technical Community Requirements Group (TCRG)
The TCRG is facilitated by the Program Manager and each group is defined by the research being
conducted within it. The TCRG meetings are conducted twice per year, during the first and third
quarter of the fiscal year. The TCRG serves as a forum for discussion and review of research
project progress and requirements.
Each requirements group has its own meeting addressing each of the HF RE&D portfolios and
their projects. The project reviews address progress, milestones, accomplishments, and the need
for additional resources for Program Managers and PIs. During each TCRG, PIs must address their
research requirement as well as the requested product from the Sponsor. Addressing the
requirement directly ensures that the project schedule, milestones and deliverables are achieved.
In addition to evaluating current requirements, participants have the ability to propose and identify
new research requirements. During the first quarter of a fiscal year, participants are encouraged to
propose a requirement three calendar years in advance. For example, if a participant proposes a
requirement during a TCRG meeting in October 2012, that requirement will be fulfilled during
FY15 if it meets the criteria (see: 3.1.1: Requirement Evaluation), or earlier if the resources are
available. Additionally, participants may identify a research requirement within two calendar
years, but that requirement will only be satisfied if the required resources are available. During the
third quarter of a fiscal year, participants have the option to propose a requirement two calendar
years in advance. Participants propose requirements two or three years in advance to ensure that
their projects are included in the NARP. To accept a requirement for consideration, a full
requirements statement must be presented to the Program Management team.
3.4. Stage Four: Knowledge Transfer and Implementation / Research Product Processing
Research products are internally reviewed to confirm requirement fulfillment once they are
completed. The Program Manager reviews the deliverable draft within 90 days. The Program
Manager then allows the Sponsor at least one week to review the draft product. The Program
Manager and Sponsor provide comments and feedback to the PI. Edits are completed and the
publication process begins. Project administration tools are updated by the Program Manager with
the final schedule and deliverable.
After the final deliverable has completed processing and has been approved, it is uploaded to the
Human Factors Library (www.hf.faa.gov) for public search and use. Human factors practitioners
are notified about the completed research via e-mail and the HF Newsletter. To complete the
knowledge transfer, researchers present their results and findings to interested parties.
4. Project Closeout Process
Upon project completion, the Program Manager meets with the Sponsor and PI to provide notice
that all activities of a program or project will be terminated. A plan is developed to transfer the
completed product. If a program has been terminated, notice is provided to the PI and Sponsor that
the project is closed.
The PI is responsible for delivery of all project information to the Program Manager. The project
plan, contract/grant/agreement documentation, materials, and deliverables are all expected by the
Program Manager. The Program Manager then archives all documentation, posts products on the
HF Library, and hands off all products to the project Sponsor.
Finally, the Program Manager terminates the grant/contract/agreement and accepts deliverables.
Resources are released and the project lifecycle is completed. A checklist is available to help
ensure that all steps in the close out process have been completed
5. Goal Reporting
Significant portfolio accomplishments are reported at the conclusion of the fiscal year. The NARP,
Annual Report, and other internal publications (facility quarterly reporting, group documents, etc.)
document the linkage between completed deliverables and agency goals. This linkage is tracked
by the Program Manager for all projects to ensure that they meet the group’s standards for
addressing each goal.
References
Federal Aviation Administration (2012). 2012 National Aviation Research Plan.
Retrieved from: www.faa.gov/.../narp/.../2012%20NARP%20Appendices-WEB.pdf
Federal Aviation Administration. (2012). 2012 Air Traffic Control/ Technical Operations Human
Factors Research Program Plan and Process.