mpf creative, llc

Client:	Large multi-state natural gas provider
Project:	Measurement Technician Training: E-learning (Seven modules) Instructor-led training (Two 15-day sessions) Corrosion Technician Training Program: E-learning (11 modules) Instructor-led training (Two three-day sessions and one 15-day session)
Primary Role:	Senior instructional designer (Eight months) Lead instructional designer (Two months)
Year:	2016

To protect the intellectual property and information security of this client, the project is described in broad terms, and the examples shown here have been recreated with generic information. In addition, in order to present a complete picture of the project, this case study references work performed by multiple contributors. In cases where I was a contributor, the term “we” is used, and where I was primarily responsible for a task or deliverable, the term “I” is used.

Background

A large multi-state natural gas provider engaged a learning consulting firm to redesign and expand critical training programs for corrosion technicians and measurement technicians. Faced with the industry-wide aging workforce crisis, the client knew that capturing the knowledge and experience of long-tenured technicians and developing a medium to transfer their knowledge to the next generation of technicians would be an important factor in solving that crisis.

I was brought onto the project initially as a senior instructional designer. After a mid-timeline project team restructuring, I served as the lead instructional designer for the final development through delivery and acceptance.

Development Model

A number of factors made ADDIE the logical model for the development of these programs:

• Size and scope of the course
• Stability of the content
• Number of subject matter experts (approximately 25 from all regions of the company)
• Regulatory compliance requirements
• Successful implementations of several training programs at this company similar in structure to the proposed program and serving similar populations
• Requirement for a streamlined, well-documented project hand-off to the client at project conclusion for their ongoing maintenance

Due to the length and structure of the course as well as the composition of the development team, we took an iterative approach to ADDIE, conducting the Analysis and Design phases at the course level and then completing all five phases at a more granular level for each lesson.

Analysis:

Once the SOW was signed by the executive sponsor, we began the Analysis phase:

Researching the Challenge

We researched the challenges of getting measurement technicians and corrosion technicians job ready by:

• Interviewing stakeholders including the executive project sponsor, supervisors, current technicians, instructors, and HR representatives
• Visiting job sites and the training center
• Reviewing the enormous quantity of source material described below

Learning About the Learner

We built a profile of the candidate technicians entering the program that would be used in all phases of development—as inputs in the early phases and as checkpoints in later phases:

• Level of education
• Years of industry experience
• Motivation for enrolling in the course
• Interest in the subject matter
• Characteristics and aptitude (e.g., problem solver, analytical, aptitude for mathematics and mechanics, detail oriented)
• Relevant record in their current role (e.g., regarding safety)

Defining Performance Outcomes

We identified the competencies that measurement technician and corrosion technician candidates must master to pass the operator qualification written examinations and skill evaluations for those roles by reviewing corporate policy, industry, and regulatory resources:

• On-the-job training (OJT) checklists
• Knowledge, Skill, and Ability (KSA) Operator Qualification Evaluator Checklists/Forms
• Operator Qualification (OQ) written exams

In addition, we collaborated with subject matter experts (SMEs) to identify performance outcomes related to company principles and industry best practices. Further, we analyzed the difference between operations and maintenance procedures and emergency response procedures for the technician candidates in their current role and in the desired measurement technician or corrosion technician roles.

Identifying Skill, Knowledge, and Attitude Gap

Using this compiled information, we defined the prerequisite knowledge, skills, abilities, and attitudes of technicians entering the program and the desired state after program completion to create a gap analysis. This helped us to define prerequisites for program candidates as well as helped narrow the scope of the training.

Evaluating Training Resources

We researched the available training resources, including classroom and laboratory environments and equipment, and built a composite profile of the instructors enabling us to design learning activities that leverage their strengths and provide additional support in needed areas.

Identifying Constraints

We identified project constraints including the budget, development timeline, resources on both our team and the client team, and regulatory requirements.

Determining the Approach

We applied the compiled data on performance outcomes, performance gap analysis, and research on resources, learners, and instructors to collaboratively determine with the client that the optimum solution to meet their performance and business goals was a blended learning approach:

• ILT with a significant hands-on component and a strong assessment/feedback loop
• E-learning prework component for complex theoretical concepts

Assembling the Content Repository

Using the compiled performance outcomes as the framework, I worked with the lead SME to determine and request the necessary source material and then worked with the project manager to construct and populate a content repository for use during project development that would be handed off to the client at project conclusion.

Design

Both the corrosion curriculum and the natural gas measurement curriculum used blended learning designs to leverage the advantages of both in-person hands-on learning and foundational self-directed learning. The team developed the high level designs (HLDs) and project plans in tandem and then presented both to the client for review and feedback.

Following is the design for each course:

---

---

Design Challenges

(Click each challenge to view the mitigating strategies.)

The skills evaluation at the end of the course assessed performance on extremely complicated procedures using equipment and techniques with which the participants were unfamiliar prior to the course.

Mitigating Strategies:

• We filmed the procedures and added the videos and text versions of the instructions to the participant guide eBooks, enabling participants to practice in the lab with these resources. Two additional benefits of this approach were:
  • The demonstration videos follow the company procedures precisely and provide the additional rationale for the steps in contrast to generic vendor-provided videos
  • The videos were added to a technician reference library enabling access in the field following the training

A typical work day for these program participants is physically active. They are outdoors at several different sites per day, working with their hands to solve complex problems. This differs radically from a conventional training environment that might include sitting in the classroom, listening to speakers, participating in group discussions, taking notes, and taking written assessments. This delta impacts transfer of learning not only from a context of use perspective but also from diminished receptivity of the learner.

Mitigating Strategies:

• One of the many benefits of working with this client was their extensive hands-on equipment training facility which features large indoor and outdoor lab work spaces with much of the same equipment that the candidates see in the field. I leveraged the facility as much as possible to intersperse classroom sessions with hands-on facilitated activities in the indoor and outdoor lab spaces to provide ample opportunities for the learners to apply the concepts just encountered in the classroom and receive immediate affirmative or corrective feedback. In addition to making the activities more engaging, this design boosted knowledge transfer because the environment more closely replicates the workplace environment.
• When feasible and beneficial to the learning outcomes, we incorporated visits to sites such as the monitoring center, stations with particular equipment, and vendors. These experiences connected participants to people and places with whom and which they would be interacting as measurement and corrosion technicians while adding elements to the training week that were more similar to a typical workday.

Many of the foundational concepts such as foundations of electricity and gas laws are complicated to teach in the classroom and require an instructor skilled in teaching abstract scientific concepts. In addition, because participants are traveling to the training site, those lessons would have necessarily been delivered back to back, resulting in cognitive fatigue.

Mitigating Strategy:

We designed the foundational concepts as short e-learning modules and incorporated features such as animations similar to popular YouTube explainer videos to allow for spaced learning. E-learning also provided an interactive, visually engaging platform, and consistency of presentation was assured.

Development

With a large-scale project and a short development timeline, the consulting firm had to maximize available resources. The instructional design team focused on designing the learning components and interfacing with the SMEs. We interviewed subject matter experts, visited the training center and specific job sites, and reviewed the source content, and then we developed detailed storyboards along with specifications for assets to be obtained or created.

The following list describes the instructional design development deliverables for each type of asset:

• Presentations:
  • Storyboards for presentation
  • Shot list for photographs to be captured on site
  • Scripts and storyboards for video to be captured on site or in the studio
  • Storyboards for animations
  • Image requests for illustrations
• E-books for participant guides:
  • Storyboards repurposing and augmenting presentation content into a learner-focused participant e-book including:
    • Tools to support knowledge transfer in the classroom and application in the lab
    • Resources for review and study during breaks and off-time during the course in preparation for exams and evaluations
  • Knowledge checks (including affirmative and corrective feedback) that evaluate mastery of the objectives for the lesson
• E-books for instructor guides:
  • Storyboards repurposing and augmenting presentation content into an instructor-focused e-book including materials to help onboard new instructors:
    • Strategies for presenting content
    • Discussion guides
    • Guidance for best practices
• E-learning modules:
  • Storyboards for e-learning modules:
  • Audio scripts
  • Lesson flow/branching diagrams
  • Storyboards for animations
  • Storyboards for videos
  • Specifications for simulations and other interactions
  • Assessments (including affirmative and corrective feedback)

With multiple rounds of revisions and as well as a large number of SMEs, we needed to add checkpoints throughout the development process to ensure that the deliverables remained aligned to the business goals. To achieve this, we verified the deliverables against the learning objectives, operator qualification examinations, and KSA evaluations to ensure alignment at each major revision. We also used this opportunity to ensure the objectives were covered in the review material and knowledge checks.

Storyboards were copyedited prior to SME reviews, helping the SMEs to focus on content.

Once the storyboards were approved, we handed them off to the production team to create the learning assets. This project structure allowed us to develop the lessons in phases: once hand-off was complete on the first set, the instructional designers could move to the next set of lessons while the production team was building out the lessons.

QA was conducted throughout the process with heavy focus on text in the storyboard phases and the focus shifting to functionality in later phases.

When all assets had been approved, we performed final integration testing—ensuring that all components were in place and sequenced correctly, the presentations, participant guide, and instructor guide were all synchronized, and all components functioned as designed.

Project Development Challenges:

Compressed timeline with team members who were experienced instructional designers but lacked deep experience with extremely technical content

Mitigating strategies:

I served as a mentor to less experienced instructional designers:

• Created job aids such as checklists for each of the development phases and guidelines for hand-offs and feedback between instructional designers and other team members such as graphic designers, motion graphic artists, and audio engineers
• Conducted peer reviews of materials and assisted with subject matter

Large team of subject matter experts. Measurement technician and corrosion technician candidates would be attending the training programs from all regions served by the company. To help ensure accuracy, completeness, and relevance for all participants as well as program adoption and support following program launch, the client wanted to include a large number of subject matter experts (approximately 20 over the course of the project) so that all regions were represented.

• To streamline reviews and resolve conflicting opinions immediately, we conducted SME reviews in a series of summits allowing all revisions to discussed and agreed upon with all SMEs present.
• One to two SMEs were identified as lead SMEs for each topic. This helped to ensure that initial drafts were technically accurate prior to presentation to the larger groups. In addition, the lead SME for a topic served as the advocate during the reviews.
• I served as a subject matter liaison and participated in site visits and hands-on demonstrations whenever possible to deepen my understanding of the subject matter. This in turn bolstered my credibility with the SMEs during discussions and when decisions were required.
• I photographed and recorded videos of procedures being performed to be used as reference during development which helped ensure the content was accurate from the start as well as serving as a reference if needed during review sessions.

Implementation

For this project, the client engaged the consulting team through the development phase and conducted the pilot and subsequent course modifications at a later date. With this in mind, we used earlier phases of the project to prepare for the implementation phase:

Instructor onboarding	The instructors were part of the SME team so that they were familiar with the structure and content and had buy-in when the courses were complete.
Administrative handoff	We developed a comprehensive logistics guide for the courses that included a description of the hands-on activities for each lesson and the tools, equipment, and other resources needed for each activity.
Curriculum development handoff	We delivered a repository of source files and source content to the client’s curriculum development team so that they could update the assets following the pilots.

---

Evaluation

Because we were external consultants, the evaluation phase was more focused on the product and process rather than the outcomes of the training. More traditional questions and metrics that I would have recommended are included in the Learnings and Additional Questions section below.

At the conclusion of the project, we held an after-action review with the client to obtain their feedback and opportunities for improvement. Each of the leads compiled that feedback and brought it forward to the internal after-action review in which the consulting team discussed client feedback as well as “Start, Stop, Continue” feedback from the team. Following that meeting, each team lead developed recommendations for process improvements based on the learnings from the project.

Learnings and Additional Questions

A project of this magnitude should and did produce some significant organizational learnings:

• Relationships are key to success at every level. With the compressed timeline, we relied on strong relationships between the IDs and the SMEs, the IDs and the production team, and the project manager and account manager with the executive team as well as with the IDs and production team. Respect, well-defined roles, and clear communications were critical components of those relationships.
• Flexibility is essential, but so are standards, and it’s important to know when each is necessary. Team members must be able to compromise to get the work done and see issues from multiple perspectives. However, if everything becomes a compromise, it can have massive repercussions on the timeline as well as the quality of the end product.
• Communication and well-managed hand-offs are critical because the time invested in it prior to the hand-off will payoff exponentially in time savings later in the project:
  • Hand-offs to SMEs for review with clear instructions, templates for feedback, timelines, and expectations
  • Hand-offs to junior team members with clear instructions, templates, checklists, standards, and quick reference guides
  • Hand-offs to production team members with clear specifications, examples of similar assets, and timelines
  • Feedback to production team members from SMEs that has been translated from SME-nglish (e.g., “Change the pressure reducing valve symbol to a gate valve symbol in the schematic” to actionable feedback that the team member can implement—possibly with a sketch
  • Two additional helpful strategies with communication:
    • Provide TLDR version near the top including essential information
    • Provide the context for item being created, reviewed, etc. For example, when asking an SME to review a lesson, it’s helpful to include an outline of the course showing where the lesson occurs in the course flow.

As a consultant, access to metrics was not possible. Following are some of the metrics that would have provided valuable insight into the success of the project:

• Lost and unaccounted for gas (LAUF) pre- and post program implementation (LAUF is an industry metric that impacts corporate profitability)
• Number of calls to senior measurement and corrosion technicians to assist with procedures or troubleshooting covered in training pre-program (under the existing program) and post program
• Analysis of job performance reviews for the roles trained in the six months following training contrasted with analysis of job performance reviews prior to the implementation

Result

Technicians completing this training were ready for promotion before more experienced technicians who had completed the previous version of the training.

Client Executive Sponsor

The project was delivered on time and within budget, and the client was pleased with the resulting training. However, the most significant feedback we received was in a hallway conversation with an executive sponsor while I was onsite working on a later project with the same client. He shared that technicians who had completed this training were ready for promotion before more experienced technicians who had been through the previous version of the training.

---