Developing & Measuring Training The Agile Way

In 2012, the leadership team of a Client Learning Organization (CLO) I worked with agreed to adopt Agile as a training development methodology.

This adoption resulted in a 75 percent increase in the number of learning assets delivered to customers, double-digit increases in employee engagement and leadership scores, and world-class customer satisfaction scores. 

In the end, internal customers felt that the learning organization was responsive to their needs, and learning department employees believed:

“We can really make our own decisions.”

This chapter is a chronology of that adoption.

The Announcement

developing and measuring training the agile way

After 3 days at our annual strategic planning retreat, the leadership team of the CLO was of one mind. 

If the learning organization was going to meet the new needs of our internal and external customers, the department needed to adopt an organizational structure that enabled rapid and iterative development, quick reaction to change and closer customer collaboration.

As a result, we decided to restructure the learning team from one where designers, instructors, technicians, and business analysts were organized into units based on technical expertise, into one where cross-functional teams of designers, instructors, technicians, and business analysts supported specific lines of business.

The leadership team also believed that the current “waterfall” approach to learning product development that we were using didn’t allow the team to react as quickly as needed to a rapidly changing business environment. 

This realization resulted in our decision to adopt Agile as a learning development methodology. With this new agreement in place, the date of the reorganization and timing of the Agile training was set.

My direct reports were excited about the opportunity to evolve from roles where they essentially operated as expert technologists or instructional designers that oversaw teams of technologists or designers—into positions that allowed them to become leaders of business units where they were responsible for providing all learning solutions to a portfolio of internal and external clients.

There was, however, some inevitable anxiety about their ability to perform well in these new roles. As the leadership team continued our conversations, staff members began to sense that something was “about to happen” and started asking questions.

The rumor mill hit an all-time high when a meeting request was issued for a mandatory “all hands” meeting. As our employees entered the meeting room, it was apparent everyone was on edge.

I wasted no time getting to the point, explaining to everyone that I had called us together to announce reorganization. I briefly explained the rationale for the impending changes, which included feedback from the individuals in the room.

With that, my assistant handed out a three-page document.

On page one (without staff names) was a graphic of the organization structure we’d operated in for the past several years. I quickly spoke about the positives and negatives of that structure. On page two (also minus any names) was a graphic representing this new structure.

I then spent a few minutes speaking to the benefits of that configuration. Finally, page three presented the new arrangement with employee names assigned to teams. 

Questions started to flow.

Most had to do with workflow and process issues.

I explained that our new approach would be one where each team decided for themselves how to get the work done as opposed to an outdated methodology that was primarily engineered by management and “pushed down” to them.

I could see a visible sigh of relief from the employees who felt constrained by the old processes. The employees who took comfort in the familiar structure those processes provided exhibited nervous looks.

The Q&A continued for the better part of forty-five minutes. At that point, I assured everyone that this change was a “win-win” and that “I had confidence they’d make it work.”

I also invited everyone to contact me in any way they felt comfortable with their questions, concerns, or only wanted to talk.

I returned to my office.

I expected a long line of employees with gripes and plenty of voicemail messages expressing dissatisfaction. I waited for about an hour, and no one stopped by.

Curious, I called one of my direct reports. She informed me that the feedback she received was positive and that the staff was excited about these new possibilities. I made my customary follow-up calls and “drive-by” conversations to gather more feedback.

Those responses echoed what I heard from my direct report.

I checked my e-mail and found several notes from staff thanking me for how the reorganization was handled and expressing excitement about the opportunities it offered. Agile training was scheduled for the following week, and part one of our transition to Agile as a training development methodology was off to a good start.

The Training: Part 1

We commissioned a vendor to serve as our Agile coach. The leadership team spent weeks with the coach describing the culture of the company, the nature of our customers, and the temperament of our staff. 

The last thing CLO leadership wanted was an “off-the-shelf” class on Agile.

We felt that if Agile was going to be successful at the organization, the training had to be tailored to our specific needs. As far as we knew, our adoption of Agile would be the first time the methodology would be applied to the development of learning solutions. 

The leadership team, therefore, insisted that the training contain examples that were specific to what we did and not some vague academic exercise.

The vendor assured us that they understood the nature of our work and that by the end of the three-day session, the team would be well-prepared to start using Agile on current projects. The leadership team was confident because one of our colleagues had already implemented Agile at two other companies and had previously worked with the vendor.

Special care was taken to ensure that team members across all geographic locations had a good, interactive experience. Rooms with multiple HD cameras and high-powered microphones were secured. We had a test walk-through on Friday before training, and we seemed to be set for a successful session.

Then everything went wrong.

We found out that the rooms that we tested (and thought we’d reserved) were actually scheduled for a corporate town hall. This left the team scrambling to secure other rooms and equipment. 

Even worse, just two days before the training event launched, the vendor informed us that the coach who’d spent so much time understanding our organization, culture, and workflow had a family emergency and wouldn’t be available. 

We had to hire a backup trainer, and suddenly, the real-world exercises and examples we’d been expecting from the original trainer were now a thing of the past.

These misfortunes resulted in the team spending three days in suboptimal facilities receiving an off-the-shelf academic lecture on Agile.

As Dwight D. Eisenhower once said:

“Plans are useless… Planning, however, is invaluable.” 

Careful preparations allowed us to avert what might have been a catastrophe. 

The Training: Part 2

The Eisenhower quote was entirely applicable to what the CLO team experienced with Agile training. 

The leadership team spent months carefully planning every aspect of the learning experience, in the end; however, the plans became useless as a result of factors beyond our control: the reserved facilities were no longer available, and the preferred instructor was out.

This exercise in planning ultimately allowed the session to achieve most of our goals. We empowered our leadership team to boldly stop the instructor and request that he provide more concrete examples when necessary. When the attendees at remote locations had a hard time seeing or hearing interactions at the primary site, the participants frequently requested that camera angles be changed or that comments be repeated.

While the overall quality of the training left much to be desired, the team walked away with a basic understanding of some essential tenants of Agile, an appreciation of how it could be used in a learning development environment, and a desire to start.

How Would Agile Work in Learning Organizations?

Ensuing discussions and questions uncovered that Agile changes the learning development process from one that looks like this: 

The trainer spends weeks or months analyzing (analysis) a problem followed by the presentation of those findings to a business partner for approval.  Once approval is obtained, the trainer then creates a design document that is subsequently submitted for yet another round of review and approval.  After the business partner approves the design document, the trainer then creates a prototype (development) that needs to go through the same review and approval process (whew! are you still with me?).

To one that looks like this:

The team gets agreement on a set of features (learning outcomes) to accomplish then divides those features into prioritized work items that allow the team to develop and deliver a working learning solution within two to four weeks!

All this caught the attention of the attendees who were visibly tired of processes that required numerous reviews and approvals resulting in months of work that frequently became a waste of time due to changing requirements.

Now The Hard Part

The team left the training session both excited about using the Agile methodology and concerned that they didn’t have enough concrete examples to get it right. 

The leadership team assured them that they didn’t expect them to “get it right” immediately, only that they continue to perfect the approach. In the following sections, I’ll be sharing the good, the bad, and the ugly of the Learning road to Agile.

Creating a Product Backlog

The first task of the Agile methodology is to derive a list of product features that must be delivered to customers.

In Agile, this list is called a product backlog, and once the CLO Learning staff completed the Agile training, each of our three Learning domains set out to accomplish this task.  

Creating a product backlog is typically handled by first meeting with the product owner or customer(s), having them articulate their vision for the product, translating that vision into features, and then prioritizing those. This, of course, is how it’s articulated in the world of software development.

So while the approach sounds simple and straightforward, accomplishing this in a training and development space proved more difficult. 

Each of the three CLO Learning domains discovered that they faced a common set of challenges when attempting to create their first product backlog, including: 

  • How do we translate a language and approach meant for software development into something that makes sense for learning solution development?
  • What do we do with work already in progress?
  • How do we handle work not specific to any one line of business?
  • How do we engage all the numerous product owners that we need to support?
  • What do we do if a specific skillset is missing from one of the development teams? 

Old habits proved hard to break, and team members (at least initially) continued to look to their managers for answers. To the credit of our leadership team, my direct reports continued to challenge each of their teams to come up with plausible solutions to these challenges. 

My role was to ensure the organization that I did not expect things would be overnight perfect, and that it was absolutely OK (and natural) to make mistakes. I walked the floor more frequently and encouraged individuals at every opportunity.

I began sending weekly video podcasts that highlighted the good things happening across the organization. My direct reports constantly reminded us all that the Agile approach adopted must be unique and specific to the challenges we faced.

Ultimately, each team was able to arrive at solutions to these common challenges, including:

1. How do we translate language and approaches meant for software development into something that makes sense for learning solution development? 

As the teams became more familiar with Agile, translation became a non-issue.

They simply started to substitute language that made sense in a learning development environment instead of software development.

2. What do we do with work already in progress?

The three learning domains were well positioned to address work already in progress.

Since teams were largely comprised of individuals that previously supported the lines of business the domain was aligned with, much of the “in-progress work” immediately moved under the responsibility of the corresponding domain.

Team members agreed that when there were exceptions, the individuals who’d started the work would see those projects through to completion.

3. How do we handle work that isn’t specific to any one line of business?

Team members agreed that as cross-functional requests were entered into the organizational backlog, the team with the greatest capacity at that time would be responsible.

4. How do we engage all the numerous product owners we need to support?

There was no universal approach to this challenge.

Instead, each learning domain adopted what worked best for that domain and the lines of businesses supported. One learning domain set up a series of consecutive meetings one day a week. Another arranged meetings with representatives from groups of businesses.

5. What do we do if a specific skillset is missing from one of the development teams?

Team members from each domain agreed to provide cross-training whenever there was a skill deficit, as well as lend expertise to other domains as needed. Within two weeks, each learning domain had met with their customers, established a prioritized product backlog, established operating procedures, and was prepared to begin their first “sprint.”

What made this effort different and so rewarding was that all of these activities were driven and accomplished by the team, not the manager. If nothing else positive happened as a result of our adaptation of Agile, the fact that teams and individuals were feeling more empowered was enough of a payoff.

The First Sprint

The newly formed CLO training teams began their first sprint with significant anguish. Although they’d been through Agile training and participated in creating a product backlog, none of the employees had ever developed learning products in this fashion.

They felt uncomfortable attempting to recommend solutions without having “all the information.” Most were reluctant to commit to the timeframes required to complete the tasks identified.

Many had problems distilling or breaking down learning solution recommendations into smaller learning assets that could be put into production immediately after the sprint was completed. Some felt that their team lacked the skill set required to deliver on the necessary learning assets.

Anguish notwithstanding, all teams performed admirably on their first sprint.

One team released more learning solutions to customers after the first two-week sprint than the previous entire quarter. Another team committed to delivering 43 learning solutions and succeeded in publishing 36 of those items to production.

What made the Agile sprint even more successful was that team members could self-identify obstacles that were preventing them from delivering 100 percent of the value they’d committed to, as well as develop plans for doing even better in the next sprint.

The basic feedback was an overall sense of empowerment.

Each learning team was energized and anticipated their next sprints. The challenge for the leadership team now became maintaining this energy and analyzing the productivity and quality of the learning solutions that were produced pre- and post-Agile.

Early Results

The early results of the Agile implementation exceeded our expectations. The average number of learning assets delivered to customers increased from 20 to 35 per month, translating into a 15 percent increase in total learning solutions compared to the previous year, same time. If the team had started 2012 at this pace, the CLO would be on target to deliver 180 percent more learning solutions than in 2011.

The feedback from both internal customers and CLO employees has also been glowing. One internal customer praised the “quick turnaround” and “quality of work,” and another related that his post-Agile experience with the CLO had been “nothing short of positive.”

Even more encouraging were the personal team member acknowledgments…

One CLO Learning employee said: “I get excited to come to work every day.”

Another reported: “I can’t believe how much work we’re getting done.”

Looking Ahead

While we’re encouraged by these early results, there have been a few bumps along the way. Each of the newly formed learning teams struggled with its own unique set of challenges, but the fact that CLO team members know and believe that they’re empowered to solve these problems has created a level of engagement we never anticipated. The experience to date is best summed up by the words of one employee:

“We can really make our own decisions.”

download agile readiness template cta

Originally published Sep 24, 2020

Scroll to Top