CIO Organization Memo Paper
John_matt
chapter_22.docxChapter Twenty-Two: Balancing IT's Workload
David Blumhorst
OVERVIEW
Most CIOs struggle with a common problem: the insatiable demand for IT work from other departments. After all, IT is a service department, providing the underlying and often-strategic technologies that enable many companies to thrive. Considering the rapid pace of business today, those technologies need to keep up with constantly changing requirements.
Thus, we are faced with an overwhelming demand for work against our limited capacity to perform the work. Most strategies for dealing with this overload involve work request intake processes and prioritization schemes. Some may take the extra step of allocating their resources, usually against projects. But if you simply look at the problem statement, the path to a solution becomes more obvious. To balance the load, we must match the incoming demand for work against the supply of resources to perform it.
IT WORK COMES IN THROUGH MANY CHAOTIC CHANNELS
Incoming demand into IT is not a simple thing. Work can come in through many channels and in many forms. There are tickets flowing through the help desk by the thousands. Some are simple requests for help, such as changing passwords or finding the power button. But many are too complex for Level 1, and require escalation to Level 2 or up to developers. These requests come from all over the company. If there is a centralized support system, most of the requests should come through as tickets. If not, more than likely problems come into IT via e-mail, phone call, or even drive-by, where users walk over to the cubicle of their favorite IT troubleshooter and ask for help. And as every CIO knows, other executives often turn to them for guidance. I have personally taken more than a few midnight calls from a vice president looking for help with e-mail, VPN connections, or even how to find clip art in PowerPoint. Made me wonder why I had a 24/7 help desk.
Demand for work also comes from within IT, particularly to the operations and infrastructure groups. Here we are looking at routine maintenance of networks, servers, databases, and other core infrastructure pieces. Of course, every new system deployed adds to the ongoing demand for maintenance, and IT is a business department in its own right, requiring systems to help with managing support tickets, networks, data centers, workflow, projects, portfolios, and more.
This demand from within IT can be thought of as base demand—keeping current systems operating in good form (aka keeping the lights on). This type of work is mostly invisible outside of IT—unless, of course, something goes terribly awry. It is, however, the base of the IT workload iceberg, not the tip, and this below-the-surface base keeps growing as IT deploys more technology into more corners of the enterprise.
The more visible tip of the iceberg comes in the form of requests to change existing systems or deploy new ones. The most familiar form is the project request—changes requiring large enough expenditures of funds or resources to require approval and visibility by management. Projects generally require teams of people and follow prescribed methodologies. Since there are approval and visibility requirements, this part of the workload is most understood throughout the company. Projects are usually grouped into portfolios and reported on to the executive team (the subject of another chapter). What is not well understood, however, is that projects are only a minor part of IT's workload. I have worked with many IT departments that track all of their time and have never found projects to consume more than 35 percent. More often projects clock in around 15 to 20 percent of total staff time.
So, what is left? What about the business-requested changes that do not come through the help desk but are too small to be considered projects? Examples include adding a custom field to an order form, adding a new business intelligence report to pull the latest sales data by region, or changing the approval process for purchase orders. Typically, the number of these change requests far outweighs the number of project requests. I have heard them called change orders, micro projects, enhancements, and work orders. The most creative term I have run across is "death by a thousand cuts", and this is often how it feels, as this area tends to be the most ungoverned source of incoming work in IT. While support tickets generally flow through a help desk system and projects often require formalized proposals flowing through an approval process, change orders have a way of falling through the governance cracks.
In many IT departments, this work comes into the organization in an ungoverned fashion. Minimally, support work may come in through a help desk system, but sometimes not. When not flowing through a defined process, projects may come in via e-mail, hallway conversations, or direct requests to technical staff. This subjects all staff members to the dreaded death by a thousand cuts as work comes from all directions to just about anyone in IT.
RESOURCE ALLOCATION MYTHS
Let's turn to the capacity for performing all this work—the supply side of the equation. First, though, we need to debunk a couple of myths about resource planning. There is no better way to debunk these myths than by using data gathered from actual time logs of IT employees. Now, I know this is a sensitive topic in many IT groups. IT staff members are generally opposed to logging their time—they have more important work to do. But if all IT staff members do log all of their time, and you analyze that data, some interesting patterns are uncovered.
First is the myth that any person can be allocated to a given project 100 percent of the time—that a person can actually be expected to work 40 hours per week on a given project. This is a common expectation in project management circles, and understandably so. If resources are dedicated to the project, they should have no other work to do, right? Analysis of time logs that I have seen at PeopleSoft (where I worked) and at other companies reveals that this is almost never the case. Even dedicated programmers will spend some time in general meetings, reviewing e-mail, logging time, and doing other administrative work. These activities generally consume 15 to 30 percent of even the most dedicated developer's time. Next, these same developers are the ones brought in to resolve support tickets that have been escalated to Level 3. Theoretically they may be dedicated to their project, but when push comes to shove, an urgent ticket will get priority. Firefighting always takes precedence over the long-term project work. The idea is that they can catch up on the project work, which would be fine if there were not always another fire to fight. As this "support" work shows up in time logs, it must be accounted for.
Myth 2 is that managers do not do project work. Based on time logs I have analyzed, even IT directors can get involved in project work. Indeed, in some IT groups, line managers take on the role of project managers. As I have noted, I have even seen CIOs get involved in troubleshooting.
Finally, myth 3 is that IT is a project-driven organization. As stated earlier, I have yet to analyze an IT group's time logs where project work rises to over 35 percent of the total effort. Most groups report project time in the 15 to 20 percent range. This, of course, is hugely dependent on how a "project" is defined. Different organizations define different thresholds of budget, effort, and risk to cross that project line. Even so, the data clearly show that IT is not a project-driven organization, and simply looking to balance resources against project demand would miss the vast majority of the workload.
What is the lesson learned from these myths? To understand IT capacity, we need to look at all work done by all IT staff. It is simply not possible to segregate the work by one specific IT area or one type of work, no matter how the department is organized.
Given these facts, let's take care of two easier-to-plan areas before turning to more complex matters.
First is the help desk. Here, measuring the volume of tickets and the average effort to resolve each ticket leads to a quick answer as to how much staff is needed. Doing this requires, however, that both those items are actively measured. I can still remember the conversation with my help desk supervisor asking for more staff. When I asked for justification, he responded, "We're just overwhelmed with work—we need more people or we're going to drown!" When I asked how many, he responded that two or three would be good. "How do you know?" I asked. "How do you know it's two or three, and not one or five?"
I continue to be amazed at how many hiring decisions are taken this way. If Level 1 and 2 support simply log their time spent working tickets, and we know the number of tickets resolved, we have help desk capacity in terms of tickets per staff member. Take the incoming flow of tickets and divide by capacity per person, and we have the staff required to "balance" this area's workload. And do not forget to reduce their overall capacity by their own administrative overhead!
The next group, at least in larger IT organizations, is management and administration. Generally, director-level executives on up do very little "productive" work—just ask them. Generally administrative staff—executive assistants, vendor management, IT finance, and so on—are pretty much dedicated to running IT, not doing the technical work that is the source of demand complexities. So we can safely allocate these IT positions to a management or administration bucket. But they still need to log their time as we want to catch any trends that might show otherwise.
This leaves the rest of the IT organization, whose job is to turn business needs and problems into delivered results via technology. As it turns out, their workload is the most complex.
ORGANIZING DEMAND FOR WORK BY SCALE
One way to deal with this complexity is to break the workload into several groupings by scale. Why by scale? Because each of these groupings is governed in a different fashion, each ideally comes through its own intake process, and therefore each needs to have resource allocations planned differently. The categories generally used are:
· Tickets
· Maintenance
· Enhancements
· Projects
Tickets are the support issues flowing through the help desk. These should all flow through a centralized support system. At smaller firms, this may be a simple Access database or a system based on software as a service (SaaS). The point is, all tickets are logged and tracked so that this stream of work can be measured and balanced. Balancing the ticket load for the support group is a simple matter of knowing the ticket capacity per person and adjusting the staff accordingly. It is also important to anticipate peak loads that be caused by new systems coming online or major projects being deployed. For the rest of IT, the demand coming through the ticket stream needs to be balanced against the rest of the demand streams. We will get to this a bit later.
Maintenance work is all the routine tasks required to keep IT systems up and running. Examples include database maintenance, backups, and network monitoring. Most of this work is performed by technical staff that will also be involved in project deployments and ticket escalations in their areas. So, to balance this work stream, we first need to measure it using time logs and then reserve that percentage of time against capacity.
Enhancements are all those changes coming through the door that are smaller than a project but not a ticket. As these do not usually go through a formal approval process, it is again best to first measure this bucket using time logs and then reserve a percentage of capacity. The next trick is determining which enhancement requests will be fulfilled using that capacity. The most successful method I have seen is for business relationship managers (if the department has them) to keep an ongoing list of requests and work with their business counterparts to prioritize those requests.
Finally, there are projects. To delineate this category from the others, it is important to define what meets the criteria for a project. These criteria usually revolve around the costs, effort, duration, and risk of the project. As an example, at PeopleSoft, an effort needed to be over $50,000 in incremental (noninternal labor) cost, or over 90 person-days of effort, or over 30 days' duration to qualify as a project. Anything less was not considered a project and was not subject to our project management methodology or approval processes. Projects require case-by-case analysis and allocation of capacity, as is described in the last section of this chapter.
For projects, it is also important to plan them into capacity before the initiation phase. Project intake, when properly done, is actually a cyclical portfolio process—not part of the formal project management methodology. In this way, requests are considered as a slate on a monthly or quarterly basis. They can be analyzed based on strategic alignment and viewed against capacity. Since they are not being approved ad hoc, there is less chance of a later, more important, project trumping an already in-flight project, which produces extremely inefficient churn. A good way to look at this is illustrated in Figure 22.1.
Figure 22.1: Aligning Resource Management with the Project Life Cycle
In this diagram, we see that capacity planning is aligned with intake. This is not the end of the resource planning process; it is the beginning. Any given project's resource plan will continue to be refined during its life cycle, and the traditional role allocation and resource assignment exercises still occur. They are just front-ended by high-level capacity planning, thus increasing the chance that projects will successfully find the resources they need.
PLANNING IT LIKE A MANUFACTURING FLOOR
IT is, fundamentally, a service department. Its job is to provide information technology that supports and enables the achievement of business strategies and goals. There is no doubt that IT can be a very strategic player for an organization, and done well it provides a competitive advantage to the firm. Still, the bulk of IT's work is delivered to other business units and external customers.
Further, as we have seen from our analysis of the types of demand, this work can be very diverse. IT is not like finance or human resources, providing a relatively homogenous set of services. Instead, IT uses very different sets of technologies, such as network connectivity, telecommunications, e-mail, Web, and enterprise applications to pursue very different business goals. And these technologies change and evolve very rapidly.
One way to look at this diverse work is as a continual incoming stream of work requests. These requests, whether they are tickets, minor enhancements, or projects, get prioritized and queued up for execution. The way most IT departments work today, when these requests are ready, they go looking for people to perform the work.
Now, if you were running a manufacturing plant and had more incoming sales orders than capacity, would you wait until those orders hit the floor before looking for equipment to produce them? And if you sold many types of products—say, diverse auto parts such as brakes, mufflers, and wheels—would you wait for the orders to hit the floor before deciding which manufacturing line to send them down? Of course not. Yet this is how we treat our requests for diverse technology work all the time.
Like any other department that produces tangible product—in our case, various technologies—a little planning is in order. Like any smooth-running plant, that starts with capacity planning. Capacity planning is the science and art of aligning all those incoming requests with the proper work teams and deciding which ones hit the floor when to make the most efficient use of available capacity.
DIFFERENT TECHNIQUES FOR DIFFERENT-SIZE DEPARTMENTS
The techniques used to perform capacity planning vary greatly by the size of the IT organization.
Smaller organizations up to approximately 50 total IT staff plan by the person. At this level there is no overlap between functions, and indeed many people perform several jobs. The project manager is likely to also be the business analyst. There may be only one network engineer, and he may also be the network administrator and perhaps even the e-mail guru. The Web developer handles design, HTML coding, and Java scripting. So, for a smaller department, it makes no sense to think of resource pools. It is much more practical to plan by individual resource.
Still, some basic planning elements are necessary. Help desk demand, even if the area is made up only of a couple of people, should still come through a centralized system— even if it is a simple ticketing system. It should be queued and prioritized, and the volume should be monitored, and the help desk staffers must log their time to show just how busy they are. This way, when they complain of being overloaded, they will have the evidence to back it up—and will have a much better chance of obtaining some relief.
Small organizations should also collect their demand for enhancements and projects in a central location, even if it is just a spreadsheet or Access database. Demand for development time can then be prioritized and queued, then fed into the various developers based on their availability. Most small organizations that do this successfully keep a spreadsheet or database with rows for each staff member, with detail on which projects/enhancements they are working on spread over columns of time, usually in weekly increments. Some small IT departments have taken to using low-cost SaaS project and portfolio management systems. Again, members should log their time to gather feedback from their planning, which greatly improves accuracy.
Medium-size IT departments of between 50 and 200 staff members differently. They often contain multiple people doing the same, more specialized jobs. Network engineers and network administrators are different people. The programmer does not do database administrator (DBA) work. And usually there are multiple programmers and DBAs.
Further, while in a small department enhancements and projects often are executed by one person or at most a small team, with midsize departments, larger teams (over five people) often are involved.
Because of the greater size, overlap, and complexity of spreadsheets, it no longer makes sense to plan by individual. Midsize IT departments that successfully balance their workloads start using resource pools or teams—groups of people who typically do similar functions and/or work on similar endeavors. For enhancements and projects, the demand still should be collected and centralized, but it will not be governed and planned in the same way.
For these departments, it works best to allocate time at the pool level and to allocate certain percentages of time for different scales of work. So, for example, we might allocate programmers to work 40 percent of the time on enhancements, 30 percent on projects, 20 percent on ticket escalations, and 10 percent on administrative work. Network administrator's allocations would be more weighted to tickets and less to projects. Of course, the help desk would be almost entirely ticket driven.
How the work is planned changes in midsize groups. Ticket work still follows the normal help desk queuing and escalation scenarios. Enhancements are usually prioritized and gated by an IT business relationship manager working in conjunction with a business-side colleague. This work is then fed into the development area by priority but only up to the allocated percentage.
The most successful way to handle project work is to gather the requests cyclically, either monthly or quarterly. The requests are then reviewed and prioritized by a steering committee. This steering committee reviews the upcoming availability of the resource pools and slots requests into time frames based on availability. If there is insufficient capacity to handle all the requests (and there always is), the steering committee decides which requests to approve and which to deny.
The steering committee is an important piece of project governance, of course. Ideally it is a cross-functional body comprising executives from around the company. As representatives of the company, it is their job to make prioritization and approval decisions that best align with corporate objectives. Making these decisions is most decidedly not IT's job. When IT takes on prioritization and approval role, plenty of blame and finger-pointing from the other business units ensues. As these decisions have a profound impact on the company, they should be made by the company's leaders.
Large IT organizations typically contain well over 200 staff members. These organizations are characterized by a high degree of both overlap and specialization. There may be whole departments of Web developers, network engineers, DBAs, and so forth. At this size, even measuring capacity by resource pool breaks down. Why?
Let's look at the type of work coming into IT again. A typical network upgrade project will employ several resource pools. There will be a network project manager (PM), network analyst, network engineer, and operations staff to implement the project into production. There may be seven or eight resource pools involved in this one project. Other projects will also employ several resource pools (see Figure 22.2).
Figure 22.2: Classic Resource Pools
If we are looking to measure capacity for projects, which resource pools do we look at? When looking at slates of 50-plus project requests, it is very difficult to queue them up by resource pool when they all involve multiple pools. Yet simply looking at total IT capacity for, say, 500 people is way too broad. This method could lead to the approval of too many application projects, leaving network engineers twiddling their thumbs, or vice versa. How, then, to look at and plan capacity in a meaningful way?
Just as manufacturing lines use many sets of equipment, IT projects use many sets of resource pools. And, just as those sets of equipment are organized into a manufacturing line, we can organize resource pools into virtual production lines based on project type (see Figure 22.3).
Figure 22.3: Virtual Resource Tracks
Enterprise resource planning (ERP) projects will use the app project manager pool, app programmer pool, and the app DBA pool. A network rollout will use the network project managers, network engineers, and network operations. If we can find an alignment of resource pools that lines up with types of projects, we can measure capacity for each of these virtual production tracks and make capacity-planning decisions based on those tracks.
We can even look for resource pool bottlenecks. Again, the manufacturing floor model is a good analogy. Equipment type 1 might process 100 items/hour, while type 2 processes 200. We therefore need two pieces of type 1 for each of one of type 2. Likewise, we may need more programmers for an enterprise app project than DBAs, and can adjust our staffing levels accordingly.
Of course, analyzing time logs so that we know how much each pool can process is critical to finding these bottlenecks.
KEYS TO SUCCESS
There are four keys to successfully balancing IT's often overwhelming workload:
· Consider all of IT's work and staff, not just projects and programmers. As everyone in IT may get involved in the various types of work IT does, narrowing in on just one aspect will not solve the problem.
· Govern the workload by scale. Tickets are governed by help desk queues, enhancements by targeted percentage policies, and projects by a formal intake funnel.
· Plan capacity early. Just as in manufacturing, capacity planning must be done long before projects are launched. Done properly, planning capacity early reduces the scramble for resources and minimizes conflicting priorities. It also allows more work to be completed without interruption, reducing inefficiencies caused by churn.
Track all time. To truly understand the workload in IT, everyone from the CIO on down must log their time. At a minimum, they must log time to the different scales and types of work and to individual projects. Without this critical feedback, even the best planning process is just guesswork.
CONCLUSION
