Executive coaching has quickly become a nearly two billion dollar industry. Manycompanies now rely on coaching for the development of their leaders. Having a coach, once perceived as an admission of failure, is now perceived as a normal part of managing large, complex organizations. And coaching has also become common for entrepreneurs who need help transitioning from leader of a start-up to manager of a rapidly growing company.

With this acceptance of coaching, CEOs and Boards are beginning to ask if the substantial investment they are making in it is paying off.  Unfortunately, we don’t know much about the impact of Survey responses coaching. Studies of coaching tend to rely on surveys of executives who are being coached. But this method of studying coaching is fraught with problems.

For example, an American Management Association survey of coaching found that respondents from organizations that use coaching more than in the past are also more likely to report two kinds of advantages:

1. They’re more likely to report that their organizations have higher levels of success in the area of coaching.

2. They’re more likely to say that their organizations are performing well in the market, as determined by self-reports in the combined areas of revenue growth, market share, profitability, and customer satisfaction.

The problem with this kind of study of coaching is that it is correlational and correlation does not necessarily mean causation. It is highly likely that high performing organizations choose to use coaching more than low performing organizations. In other words, high performance predicts coaching rather than coaching predicts high performance.

Survey studies of coaching are problematic in other ways, too. One, they lump all coaching into a black box without examining the many variations of coaching. Two, they are based on the assumption that the purpose of coaching is to change the individual when, in many cases, the purpose is to change a team or the organization as a whole. Three, they don’t tell us exactly how the coaching contributed to performance improvement within an environment in which many other factors may have contributed to the change. And four, they rarely look at long term impact, which is what is often the intended purpose of coaching.

To evaluate executive coaching, first we need to define what is meant by "coaching" in the situation being evaluated. Do they mean coaching for greater self-awareness, for developing leadership skills, for making a transition to a new job or role, or for solving a specific problem? Do they mean face-to-face, by phone, or email? Do they mean an hour a day, an hour a week, an hour a month, or as needed. Is the coaching process mostly inquiry or mostly advocacy?  Does the coaching include interviews of supervisors and direct reports? Does it involve observation of work situations and interactions? In other words, what does coaching mean for that executive and that organization?

What we need are studies that tell us what kind of coaching under what circumstances contribute to what kind of results for the organization and over what period of time. Surveys are helpful for knowing the extent of coaching and the general attitude of employees toward coaching. But to understand coaching well enough to be able to replicate effective coaching or improve coaching for an organization, we need to tell the story of the coaching experience. How did the executive get involved in coaching? What were her expectations? How did she work with the coach? What did the coach actually do? How much time did they spend in coaching? What happened? What did the executive learn? What is the executive doing differently? What impact is that having on her team and organization? What evidence is there that this kind of change is occurring and that it is linked to coaching?

When we can answer these questions we will understand what it is about various coaching interventions that have an effect on results.  We will be able to compare coaching to other potential interventions and replicate effective methods. We will know the return on investment. We will know what makes an executive coach qualified to coach. We will know what makes one approach to executive coaching better than another.

Evaluation of coaching is also important to do because it reinforces client learning. When we ask someone questions about what they have learned, it helps to make that learning stick. This is what happens when we ask clients questions about the coaching they received. Their self -awareness is increased, behavior change is strengthened, and long-term follow-through is more likely.