Viewing entries in
evaluation

Comment

Assessing Your Organizational Learning Culture

To what extent does your organization have a learning culture? Before you take your first step on your journey, know where you are starting. Look around your organization. What do you see? What do you hear? What do you feel? What is your current culture?

Using Edgar Schein’s definition of organizational culture, you’ll want to know to what extent:

  • Underlying beliefs and assumptions support learning in your organization
  • Values and principles drive learning in your organization
  • Employees and other stakeholders (suppliers, Board of Directors, customers) see the symbols and artifacts of learning and performance improvement.

Do employees, their teams, and the organization as-a-whole know what they need to learn to be successful? Do they know how to develop these competencies? Do they know how to sustain this learning over time? Do they know how to ensure that learning is applied and makes a difference for the organization?

Tools currently exist that might be helpful in this analysis. Each of these tools is designed according to the Survey images (1) authors’ definition of a learning culture. For example, David Garvin and Amy Edmondson created an assessment tool to look for: a supportive learning environment, concrete learning processes and practices, and leadership behavior that provides reinforcement. I wrote this in a previous blog post about the tool:

As the authors say, the tool should be used for learning, not to judge the quality of an organization. The survey provides feedback for organizational reflection. By collecting the data and then discussing the findings, people must confront critical questions: Does our culture support learning? Do we have every day processes and procedures in place to ensure that learning and change are embedded in the way we work together? Do our top leaders make continuous learning a priority and communicate this throughout the organization? And then, hopefully, leaders will use the answers to these questions to motivate further learning and performance improvement and contribute to developing a learning culture in the process.

Another assessment tool is one created by Marcia Conner which she calls a “learning culture audit”. This tool is used to determine the extent to which an organization is oriented towards learning. She writes:

One way to begin the process of creating a learning culture and to enroll others in the effort is to conduct a learning culture audit. A simple diagnostic can help you assess your organization and your management team’s orientation to learning. An assessment describes the characteristics of cultures that encourage learning and those that block learning.

The dichotomous variables that Conner presents, comparing a “pro-learning culture” to an “anti-learning culture”, are certainly grist for discussion within your organization. For example, one set contrasts the item, “People at all levels ask questions and share stories about successes, failures, and what they have learned”, with the item, “Managers share information on a need-to-know basis…People keep secrets and don’t describe how events really happened.” Responses should stimulate a rich discussion about information sharing in an organization.

Learning to be Great™ has also created a learning-culture assessment tool. This tool is based on a framework that consists of five key elements of learning in organizations: 1) Alignment; 2) Anticipation; 3) Alliance; 4) Application; and 5) Accountability. These are the 5As of a learning culture. Each element is measured by four items in the survey. To fill out the survey and find out how your organization is doing on each “A”, go to the Learning to be Great website.

Whichever assessment tool you use, what’s important is to follow a stakeholder-focused process that results in organizational learning. Jim Stilwell outlines this process in his post titled, “How to Use Feedback from Employee Surveys to Change Organizations.” He writes that the organization survey process..

…works best if senior leadership communicates a compelling need for the survey in advance of sending questionnaires to all or a sample of employees.  As a part of this communication, leaders should clearly define the entire data gathering and feedback process emphasizing the role of employees.  Secondly, the findings from the survey would benefit from validation by key stakeholders – namely those who have provided the survey responses. The data from a survey can always be interpreted in many different ways given organizational circumstances, response rates, respondent demographics, the wording of questions and design of the survey. Thirdly, discussing the findings with groups throughout the organization and at all levels for their interpretation and recommendations leads to a deeper understanding, greater alignment, and vastly better solutions. A fourth important step in the process is for leaders to communicate back to the entire organization what it is that will be done as a result of the insights gained through the survey and all of the group discussions. This communication should clearly define what leaders know now that they didn't know before? What will be changed that will improve the work environment and increase organizational success? Who will be responsible and when will it happen? How will follow-up happen to ensure the success of the changes? 

The data from a survey is only meaningful if it generates a conversation in your organization that contributes to discovering of where you are starting your journey. Any survey of your organization should have as its purpose to learn from each other, to learn about your organization’s strengths and weaknesses, and to learn what will improve performance.

Organizational assessment is the GPS of your journey towards a learning culture. It tells you where you are at a point in time. Then you’ll be able to select the best route to your destination: a work environment that supports and encourages the continuous and collective discovery, sharing, and application of knowledge and skills at the individual, team, and whole organization levels in order to achieve the goals of the organization. 

Comment

Comment

Leaders Learning about Learning

Recently, I conducted a workshop for the leadership team of a company that wants to increase the impact of its training programs. I explained the limitations of formal training and the need for taking an organizational learning perspective. I argued that in order for any kind of learning intervention (training, coaching, mentoring, action learning, etc.) to have a positive impact on achieving the organization’s goals, managers had to take an active role in supporting learning.

This message was well received by these senior leaders. They immediately understood the vital role they play in developing individuals, teams, and the organization as a whole. They wanted to know specifically what they could do to facilitate learning.

I suggested the 5As Framework as a place to start. This is a useful model for ensuring that, regardless of the 5A logo learning intervention, they will achieve performance improvement and organizational success. The 5As are: 1) Alignment – align learning with strategic business goals; 2) Anticipation – expect success; 3) Alliance – form a learning alliance between learner and boss; 4) Application – apply new learning immediately; and 5) Accountability – hold learner and organization accountable for business results.

We discussed what leaders can do to ensure that these elements are addressed. Here are their suggestions:

  • Alignment: explain to employees the importance of a particular training program and how that learning will help them help the company be successful; make this conversation part of employees’ performance reviews.
  • Anticipation: clarify expectations for what you want employees to learn and communicate high but reasonable expectations for their learning; make this part of informal conversations they have about performance on a frequent basis (not only at annual performance review time).
  • Alliance: meet with employees before training to identify learning goals, meet with employees during training (if possible) to discuss progress, and meet with employees after training to identify what was learned, how employees will apply that learning in the workplace, and what additional learning they need
  • Application: provide opportunities for employees to immediately apply learning in their work in a way that will make a difference
  • Accountability: give employees feedback on their application of learning and measure the impact of that learning on their teams and the organization

I was encouraged by the enthusiasm and understanding that these company leaders had for their role in learning. I can only hope that higher level executives and the company's CEO will support this role and recognize and reward what my workshop participants are doing to facilitate continuous learning throughout the organization.

Comment

Comment

2015 - Year of the Learning Culture

The theme that cuts across most of my blog posts from last year is creating and sustaining a learning culture in organizations.

As a way of review, I’ve selected five blog posts about a learning culture from 2015 that have the most 2016 interest for readers. Here is the title of each post with a short excerpt. Click on the title to go to the full post.

  1. Developing a Learning Culture Infographic This infographic explains briefly and concisely the need for a learning culture, barriers to a learning culture, actions of a learning culture, and results from a learning culture.
  2. Training Culture vs. Learning Culture What’s the difference between a “training culture” and a “learning culture”? The answer is, “A great deal.” As the chart shows, in a training culture, responsibility for employee learning resides with instructors and training managers. In that kind of culture the assumption is that trainers (under the direction of a CLO) drive learning. Whereas in a learning culture, responsibility for learning resides with each employee and each team.
  3. Pull, Don’t Push, Employee Learning As the digital revolution continues to fuel the faster rate of change, transforming all aspects of business, from supply chain management to communication, the highest-performing corporations are abandoning traditional “push” training for the “pull” learning model.
  4. PwC Canada Strives for a Learning Culture If you’re looking for examples of companies that are striving to create and sustain a learning culture, PwC (PricewaterhouseCoopers LLP) of Canada should be on your list. I recently had the pleasure of speaking about the importance of a learning culture to the Edmonton meeting of The Conference Board of Canada’s Council for Learning and Leadership Development. Another speaker at the event, Karin Muchall, Development Leader for PwC Canada, impressed me with her description of how PwC is implementing the principles of a learning culture through a program they call Enhanced Working PracticesTM.
  5. 50 Ways to Lever Learning In a learning culture, formal training is just one of many methods used to facilitate employee learning. In a learning culture, we start with the performance goal and then select the mix of methods that will help employees acquire and retain the knowledge, skills, attitudes, and beliefs they need in order to achieve those goals. This is a list of 50 of those methods.

Comment

3 Comments

SCORM and xAPI LMS in a Learning Culture

A learning management system (LMS) is considered to be a necessary tool in most complex organizations today, especially those industries, like nuclear energy, airlines, and healthcare, where monitoring compliance with government-imposed training requirements is essential.

However, I have had my doubts about the value of an LMS for enhancing learning and contributing to a learning culture. Knowing Computer learningwho has taken which course and even knowing how they did on a test, is not evidence of learning and certainly doesn’t indicate one’s ability to apply that learning on the job. And more importantly, an LMS does not tell us the impact of that learning on the organization.

I recently attended an “xAPI Party” at Torrance Learning where the conversation was about the shift from a SCORM-based LMS to an xAPI (experience application programming interface) LMS. xAPI appears to be a huge leap forward in tracking the whole gamut of learning experiences from formal training programs to informal, social, and mobile activities. There was a lot of excitement about this change and the benefits seemed clear.

Steve Foreman, in an article for Learning Solutions Magazine, describes XAPI this way:

The xAPI operates based on activity streams, a model that uses software to track things people do. The idea of tracking activity streams emerged from social networking and is used by sites such as Facebook, Twitter, and Google Plus.

The Experience API makes it possible to track activities that people do using computers such as performing work tasks, producing work outputs, interacting with others using social media, achieving milestones in games and simulations, and just about any other activity that one can observe or record.

Imagine that your LMS can communicate, not just with eLearning courses, but also with knowledge bases, collaboration platforms, document management systems, enterprise resource planning systems, helpdesk systems, portals, talent management, performance management, and other types of systems used in the workplace. The learning management system may track not only attendance, completions, and test scores, but actual work inputs, outputs, deliverables, tasks, and more.

However, I have to ask, “Does xAPI do a better job of facilitating learning?” Any tool is only as good as the person who’s using it and this is true for xAPI, as well. Clearly, xAPI, because it includes a wide range of workplace learning experiences, is superior to SCORM for tracking activity, but it is still about activity, not learning.

To make xAPI effective for enhancing learning, employees will have to consistently collect evidence of experience, constantly input the right data, use that data, through feedback and reflection, to help individuals and teams develop their competencies, continually assess the impact of learning on the organization as a whole, and have conversations about the meaning of those findings. These are behaviors of a learning culture.

Employee learning requires feedback, reflection, and practice. Experiences, whether formal, informal, social, or mobile, do not result in learning unless these other elements are present. If an organization is committed to a learning culture, those elements will be the way the organization functions on a day-to-day basis. Without a learning culture, I’m skeptical that any LMS (SCORM or xAPI) can enhance learning and improve performance. However, in the context of a learning culture, xAPI offers great promise for contributing to organizational learning.

[image from Pixabay]

3 Comments

Industry Report: Too Much Training; Not Enough Learning

Training magazine has published its 34th annual Industry Report that is based on a survey of its database of organizations with 100 or more employees, asking them questions about training expenditures and activities. The 2015 study has 777 respondents which is typically a good size sample from a statistical standpoint but I would question the reliability of this data given that it’s not a random sample. I would expect the margin of error to be high.

In any case, assuming that the findings, in general, describe the state of the industry in 2015, it is disappointing to see the continuing substantial investment in “push” training. “Instructor-led classroom TrainingIndustryReport only” is still reported to be the primary method of employee learning and development. If we include “blended learning”, “virtual classroom/webcast only”, and “online or computer-based methods” in the mix, we see that companies are continuing to invest most in methods of learning that have the least payoff.

We know from previous studies that only about 10% to 50% of learners (depending on the nature of the training and workplace environment) actually apply what they learn in training programs back on the job. If Training magazine’s numbers are accurate and approximately $70 billion was spent on training programs last year, we can estimate that the industry wasted roughly between $35 billion and $63 billion on training.

Organizations might be increasing their investment in “pull” learning through social media, on-the-job training, and action learning. However, we don’t hear about these learning interventions because they don’t fit the more easily observed and measured formal methods.

In any case, I think we need to look at all of the ways employees learn in their organizations and to what extent employees apply that learning on the job. Currently, we have a good picture of the state of formal training. If we are to truly understand learning in organizations, we need a better picture of the informal and "pull" ways in which employees learn.

I welcome a different interpretation of Training magazine’s data. Please comment to either support or to challenge my view of the industry.

1 Comment

Performance Management at Deloitte

Kudos to Deloitte for making a concerted effort to develop a fair, efficient, and useful performance management system for its 65,000 employees. In a blog post for Harvard Business Review titled Reinventing Performance Management, Marcus Buckingham and Ashley Goodall describe Deloitte’s evolving system. They write:

This is where we are today: We’ve defined three objectives at the root of performance management—to recognize, see, and fuel performance. We have three interlocking rituals to support them—the annual compensation decision, the quarterly or per-project performance snapshot, and the weekly check-in. And we’ve shifted from a batched focus on the past to a continual focus on the future, through regular evaluations and frequent check-ins. 

To me, the key to their performance improvement will be the weekly check-in. This is where team leaders talk to team members about past performance, goals for future performance, and what can be done to be successful. This conversation, as I have written in a previous post, is an essential aspect of any leader’s role. As the authors say, “…check-ins are not in addition to the work of a team leader; they are the work of a team leader.”

Also, Deloitte has designed a potentially efficient way to measure performance over time. They intend to ask team leaders four questions at the end of each project or at least once a quarter. The questions are:

1. Given what I know of this person’s performance, and if it were my money, I would award this person the highest possible compensation increase and bonus [measures overall performance and unique value to the organization on a five-point scale from “strongly agree” to “strongly disagree”].

2. Given what I know of this person’s performance, I would always want him or her on my team [measures ability to work well with others on the same five-point scale].

3. This person is at risk for low performance [identifies problems that might harm the customer or the team on a yes-or-no basis].

4. This person is ready for promotion today [measures potential on a yes-or-no basis].

It remains to be seen whether these questions will “fuel” performance in the long-run. Given research that shows manager ratings of employees are more about the person doing the rating than the person Rating-24184__180being rated, Deloitte is trying to make the ratings more objective. However, it still requires rating by managers who are affected by their own attitudes and biases. If this process results in more conversation between team leaders and team members about learning and improving performance, then it will have served its purpose.

What concerns me most about Deloitte’s new performance management system is the apparent lack of alignment between wanting to be strength-based and using variable compensation. A system that rewards the performance of some and not others can be a disincentive for learning and performance improvement. In a strength-based system, everyone is recognized for what they contribute and for what they can become. A variable compensation system is a good way to control behavior but not necessarily a good way to encourage learning and improvement.

I want to be clear. I admire Deloitte’s intent and the willingness to experiment and use evidence to improve their performance management system and I am very curious to hear about results and impact on the organization. My concern is that the recognition and reward aspects of the system will negate the learning that can be achieved from leader feedback aspects of the system. We’ll see.

1 Comment

10 Comments

16 Signs of a Learning Culture

How do you know your organization has a learning culture? What will you see people doing? How will people be learning? While a learning culture is an environment that’s always being developed, certain signs indicate that you are making progress. In a learning culture…

  1. Leaders are communicating the importance of learning (acquiring new knowledge, skills, and Improvement road signcapabilities) and holding managers accountable for learning and applying that learning to making a difference for the organization.
  2. Managers are helping their direct reports create an individualized learning plan linked to strategic goals of organization; managers are monitoring learning progress and providing feedback; they are structuring opportunities to apply learning on the job; and holding direct reports accountable for results.
  3. Managers are coaching; they are partnering with direct reports to develop their capacity to achieve organizational goals.
  4. Leaders are mentoring; they are using their experience to advise new and less experienced employees on how to fulfill the functions of their jobs.
  5. In recruitment, hiring, and onboarding of new employees, learning is part of the conversation; learning is conveyed as a value of the organization; expectations for employee learning and development are discussed during recruitment phase.
  6. Employees can see how their learning is aligned with the strategic goals of the organization; they understand how acquiring certain knowledge and skills will help the organization be successful; the direct link from learning to results is made clear.
  7. A wide range of formal and informal, hi-tech and hi-touch methods are being used to facilitate learning; the method used is determined by the intended outcomes for the organization.
  8. Expectations for employee learning are discussed with employees; employees know what they need to learn and why they need to learn it and what criteria will be used to monitor progress and assess results; high expectations are communicated.
  9. Learning is applied throughout the organization to continuously improve performance and achieve strategic goals.
  10. Employees and their managers are held accountable for learning; measures for evaluating impact of learning on the organization are used and the data is used to make improvements in learning methods and processes.
  11. Each learner and learning team in the organization is recognized and rewarded when the application of learning results in solving problems and achieving goals.
  12. Learning is integral to decision-making; before any significant decision, information is gathered to inform that decision and team members are learning how to make effective decisions and apply that ability to a workplace problem.
  13. Employees are encouraged to take risks as a way of learning; they are applying action-learning methods to trying out solutions and assessing the effectiveness of those solutions for their teams and the organization; if an action is not successful, the focus is on learning, not blame.
  14. Employees  are constantly experimenting with new ideas and programs for the purpose of finding out what works and what doesn’t and learning what they need to do to be successful; experimentation is valued by leadership.
  15. Explicit knowledge is documented in a way that makes knowledge easily accessible throughout the organization; tacit knowledge is surfaced through facilitated experiences with employees; knowledge (e.g., experiments, best practices, new information) is openly shared among organizational units, departments, and divisions.
  16. Stories that make up the lore of the organization are about successes and failures that resulted in individual, team, and whole organization learning; these stories communicate the value that the organization places on risk taking and experimentation.

Like road signs that tell you if you are on the right highway, these sixteen signs of a learning culture tell you if your organization is headed in the right direction. You never arrive; there is always more to do on the journey.

For help assessing your organization's culture, see our workshop, Assessing Your Organization's Learning Culture

10 Comments

Comment

How to Achieve an Organizational Learning Goal

It’s that time of year again, when we pay special attention to our personal and career goals. However, how likely is it that we will actually achieve those goals? I belong to a fitness center at the Alignment local community college and I’m always fascinated by the January upsurge in activity and then the fall off around March and April each year. Hopefully, that means some people have achieved their fitness and weight loss goals, but I’m afraid that for many it means they have given up. These folks probably haven’t established the processes and relationships that will help them achieve their goals.

Jesse Lyn Stoner makes some excellent suggestions for what we can do in terms of processes and relationships to ensure that we achieve our New Year’s resolutions. You can apply these same principles to organizational learning goals. For individuals, teams, and whole organizations to achieve their learning goals, they need to develop supportive processes and relationships. It’s not enough to identify the knowledge and skills that you want to develop. And it’s not enough to select a method for learning. To apply and sustain that learning, you must also establish processes and relationships that support learning and application of that learning.

Listed below are Stoner’s six suggestions and my explanation of how each one helps in the attainment of learning goals.

1. Start with your current goals.

What new knowledge and skills are you trying to acquire? How has that been going and what can you learn about your own process of learning? Decide if you want to continue to acquire that knowledge and those skills or are you ready to move on to something else. For example, maybe you want to learn how to give negative feedback to your direct reports. Take stock of your ability to do that and whether you need to continue working on that skill and then identify additional learning goals that you need to achieve.

2. Connect your goals to a larger purpose.

Align your goals with the strategic goals of the organization. Be clear with yourself, with co-workers, and with your manager about how acquiring certain knowledge and skills will contribute to the organization’s success. Have a “line of sight” from the learning goal to the performance of the organization. For example, be clear about how learning Lean/Six Sigma will help the organization be successful.

3. Goal setting is not always a logical process.

Don’t get frustrated by the lack of a straight-line process. You might set some goals and then, in talking with your boss and after some experience, decide that those goals need to be modified. And given the pace of change, a learning goal that you set today could be irrelevant tomorrow. However, whatever the goal, have some notion of how you and your boss will know that it has been achieved.

4. Write your goals down and put them somewhere visible.

Writing them down will help you commit to achieving your goals. Keeping them visible will remind you that this is your task and also allow you to modify the goals as needed. The adage, “out of sight; out of mind”, applies here.

5. Don’t keep your goals a secret.

Discuss your goals with your boss and co-workers. You need their support. For example, if you are learning how to run a more effective and efficient team meeting, you need the cooperation of your team members and their feedback. Your boss should be able to provide you with opportunities to practice these team management skills and advise you on what you need to learn and how best to learn it.

6. Set up processes and practices that support your goals.

You’re more likely to successfully achieve your learning goals if you hold yourself accountable and if others hold you accountable. Discuss the indicators of successful learning with your boss and co-workers. Arrange times to regularly check in with them to take stock of your progress.

Organizational learning is not something you can do in isolation. You can identify learning goals but you will need the support and involvement of bosses and co-workers to achieve those goals. As Stoner recommends, establish those processes and relationships at the outset and you will be more likely to follow-through and be successful.

Comment

Comment

Learning to Learn Collectively

A major characteristic of a learning culture is that individuals, teams, and whole organizations are constantly learning how to learn. They are learning how to acquire the knowledge and skills that they need to help the organization be successful. The teacher-centered, classroom-focused, right-and-wrong answer, static instructional environment that was the primary modality in the schools they attended does not fit the rapidly changing, technology mediated, on-demand knowledge and skills that are needed in today’s organizations. In this environment, people need to be continually figuring out different ways to learn, whether that be individually using new technology, in teams that are trying to become more effective, or as the whole organization learns how to communicate, how to use resources more efficiently, and how to make better decisions.

In their book, A New Culture of Learning, Douglas Thomas and John Seely Brown describe this dynamic world and what it means to learn how to learn in this kind of environment.  They write:

… Imagine an environment that is constantly changing. Imagine an environment where the participants are building, creating, and participating in a massive network of dozens of databases, CultureofLearning Book hundreds of wikis and websites, and thousands of message forums, literally creating a large-scale knowledge economy. Imagine an environment where participants are constantly measuring and evaluating their own performances, even if that requires them to build new tools to do it. Imagine an environment where user interface dashboards are individually and personally constructed by users to help them make sense of the world and their own performance in it. Imagine an environment where evaluation is based on after-action reviews not to determine rewards but to continually enhance performance. Imagine an environment where learning happens on a continuous basis because the participants are internally motivated to find, share, and filter new information on a near-constant basis.

The environment that Thomas and Seely Brown are describing is one in which people have learned how to learn collectively. This collective learning is already happening in a number of different types of organizations. However, this is not how most of us learned how to learn in school. There we were evaluated on the basis of our individual retention of knowledge. We learned how to acquire information and analyze and synthesize that information on our own. Learning collectively is something that individuals, teams, and organizations today need to learn how to do. This is the work of twenty-first century organizations.

Comment

1 Comment

Does Employee Happiness Cause Business Success?

I believe that Shawn Murphy is right. Employees who are engaged in their work, are satisfied, have a sense of wellbeing, have purpose, feel they are doing meaningful work, and have a belief that they MP900431739 can be successful in their jobs, are more likely to achieve positive business outcomes than employees who are unhappy in their work. The problem is that there doesn’t appear to be evidence to back up this assertion.

All of the well-publicized research on this relationship is correlational and, therefore, we don’t know which is causing which or if other factors (such as leadership, organizational structure, R&D, etc.) are more important than employee happiness. In an opinion piece for the NY Times titled, Why You Hate Work, the authors report results from an employee survey they did with the help of Harvard Business Review. The authors observe:

Engagement — variously defined as “involvement, commitment, passion, enthusiasm, focused effort and energy” — has now been widely correlated with higher corporate performance.  

A 2012 global survey by Towers Watson found a high correlation between “happy” employees and a company’s profitability. Gallup does a survey of companies every few years and consistently reports a …well-established connection between employee engagement and nine performance outcomes: 

  1. customer rating
  2. profitability
  3. productivity 
  4. turnover (for high-turnover and low-turnover organizations)
  5. safety incidents
  6. shrinkage (theft)
  7. absenteeism
  8. patient safety incidents
  9. quality (defects)

These are impressive studies that suggest but do not confirm a causal link. A highly plausible alternative explanation is that doing well on some or all of these performance outcomes causes (or, at least, contributes to) employee engagement and happiness. Like me, I’m sure you know companies that have been productive and profitable while having a high-turnover workforce that is quite unhappy with work conditions. Or you know companies that have what appear to be wonderful work conditions but have failed to be profitable and sustainable.

Let me be clear. It’s logical that high engagement, what Rich Sheridan calls “joy”, helps companies achieve success. Some company stories would seem to support this conclusion. However, at this point, that is a hypothesis that has not been proven.

If you have evidence to the contrary, please let me know in a comment to this post.

1 Comment

Comment

Don't Use SAT Scores for Job Hiring

I was surprised (closer to shocked)to read that McKinsey & Company, Bain & Company, Goldman Sachs and other companies are using SAT scores to screen job applicants. That is a misuse of the test and a disservice to the applicant and to those organizations.

Apparently, some HR managers claim that this practice is okay because they use the score in combination with other factors. However, if they are using the SAT to sort applicants and disqualify some, then it doesn’t matter what other factors that are being used - they are using the SAT inappropriately.

The SAT should not be used in this way for three reasons.  First of all, job applicant screening is not Testthe purpose for which the test was designed. The test was designed to give high schools students and college admissions staff an additional indicator of academic success. In combination with high school grades, the test is probably the best indicator we have for success in the first year of college. That doesn’t mean it’s a good indicator of academic success; it just means it’s better than anything else that can be easily managed by college personnel.

Another reason the SAT should not be used for hiring is that although, as a group, people who score high on the test are statistically more likely to do well in school than people who score low, that doesn’t mean this is true for an individual job applicant. Statistical significance (better than chance) is not the same as practical significance. Someone who scores low on the SAT (or any achievement test) could become a wonderful employee and someone who scores high on the SAT could become a terrible employee.

And the third reason is that test scores are just not good predictors of workplace performance, unless, of course, the job task is to do well on tests. The best predictor of job performance is job performance. Companies like Menlo Innovations have job applicants do the job for a few months before they make a hiring decision. In this way, they can look at culture fit as well as competencies.

David Brooks, in his column in the New York Times titled “The Employer’s Creed”, urges employers to look beyond grades, perfectionism, and projected image of job applicants, and look for traits such as being passionate about something, doing the right thing, showing social courage, being willing to speak the truth, overcoming setbacks, and being honest about themselves. About grades, Brooks writes:

Don’t mindlessly favor people with high G.P.A.s. Students who get straight As have an ability to prudentially master their passions so they can achieve proficiency across a range of subjects. But you probably want employees who are relentlessly dedicated to one subject. In school, those people often got As in subjects they were passionate about but got Bs in subjects that did not arouse their imagination.

I understand the need to efficiently sort through hundreds, maybe thousands of job applications. But using SAT scores is a careless and unprofessional way to do this. Companies that use SAT scores to screen applicants will eliminate from consideration some of the best people in the pool.

Comment

Comment

Year in Review - 2013

Throughout 2013, I used this blog to illuminate important leadership and management issues. The 2013 topics ranged from changing organizational culture to evaluating training programs to making managers responsible for employee learning to creating a learning culture.

As a way of review, I’ve selected five blog posts from the past year that seem to have had the most interest for readers. Here are the links with a short excerpt from each post:

Culture Change is More Than Skin Deep

People act on the basis of tacit knowledge but they are often not aware of how daily behavior is shaped by this knowledge. It’s simply “how we do things here.” Underlying beliefs, assumptions, and values become routines that, over time, go unchallenged. Those routines may or may not serve the best interests of the organization and its customers. Culture change must confront the link between the thinking that drives behavior and the effects of that behavior on organizational success.

Managers and Employee Learning 

Managers of employees have a critical role to play in employee learning. Managers are in the best position to help employees acquire the knowledge and skills they need to be successful. By asking questions, by assigning opportunities to apply knowledge and skills, by modeling expected behavior, by providing timely information, by coaching application of learning, and by giving performance feedback, they can facilitate learning and, therefore, performance improvement of their direct reports. 

Why Learning Is Not Applied

Learning and the retention of learning cannot be left to trainers alone. Organizations as a whole must create an environment that supports the learning process. Leaders must make learning a core value, managers must facilitate and support learning for their direct reports, trainers must provide learning interventions appropriate for the content, and learners must participate enthusiastically in the process. Retention of learning is a systems problem, not an individual employee or individual trainer problem.

Do You Have a Learning Culture? (Parts One & Two)

Culture used to be considered a byproduct of organizational life. Today, many companies are being quite intentional about culture. So, how do you know what kind of culture you have and, if you want GreatWallPicture1 to create a learning culture, how do you know when you have one? Dharmesh Shah, Founder and CTO at HubSpot, gives us a way to think about this. He writes, “The true nature of your company – and its culture – is determined by how you instinctively react.”

Joy to the World of Work

After reading Joy, Inc. and visiting [Menlo Innovations] on several occasions, I believe the key to Menlo’s success is its “learning culture”.  This is an intentional culture a la Edgar Schein's three levels: (1) the deep underlying beliefs and assumptions that are often difficult for insiders to articulate; (2) the values and principles that structure action; and (3) the symbols and artifacts that are visible on the surface for all to see. In the Menlo culture the message to employees, both implicitly and explicitly, is that we are learning together how to best serve our clients while creating an enjoyable and meaningful work environment. 

Comment

2 Comments

Customer Satisfaction: Gaming the System

When it comes to measuring customer satisfaction, managers are gaming the system. They are advising customers to answer survey questions in a way that makes the manager look good, i.e., a rating of 5 on a 5-point scale. Rather than trying to improve the customer experience, they are trying to improve their scores.

I understand their motivation. An executive in a central office or a hired survey company might be collecting this data and drawing conclusions about customer service in a location that they may or may not have visited and probably know little. Sometimes the data folks understand the business and sometimes they don’t. Sometimes they understand what customers mean by a particular rating and sometimes they don’t. Sometimes they ask valid and reliable questions and sometimes they don’t. A phone company service technician once pleaded with me not to send a compliment to his supervisor because the “system” counts every comment from a customer as a complaint and it would go against him.

William Grimes, in a column for the N.Y. Times, writes:

In the auto industry, which tries to measure customer satisfaction at every possible stage, from the first tentative Web search to the last service visit, the assessment ritual can become a kind of performance.

Sales representatives have been known to show pictures of their wives and children as they plead for a favorable review in their dealership’s satisfaction survey. Some show their customers a sample survey already ticked off with top marks in every rating category. Dealers sometimes throw in a free tank of gas or a free oil change as a quid pro quo.

Pressure tactics have crept into other industries as well. Cable technicians, after completing an installation or repair, often call into the head office to report and then hand their cellphone over to the customer for a quick round of questioning about the service, an awkward conversation with the technician standing a few feet away.

Sales clerks who once concluded a transaction with “Have a nice day” now plead with customers to fill out surveys and award good marks because “my job depends on it.”

This is a sign I observed on the checkout counter of a CVS/Pharmacy:

CVS Sign

Clearly, this store is trying to influence ratings, not by improving customer service but my pleading for leniency.

I’ve written before about helpdesk employees who, at the end of a phone inquiry, ask customers to report that they were highly satisfied with the service.

Encouraging a particular rating does not tell you what customers really think. You have to ask yourself, “Do I want high ratings or do I want to know the truth?” Customers might tell you what they think you want to hear and then not buy again and not recommend your product and services to others.

Zingerman’s, which is well known for creating an outstanding customer experience, says in its training video that most customers don’t like to complain. By discouraging negative comments a company only makes customers more uncomfortable and more likely to misrepresent what they really think. Then managers will not hear what they need to hear.

If you want to improve performance, ask customers for feedback in a safe and neutral way. If you don’t care about improving performance, don’t ask for feedback. 

Related articles

Customer Service Run Amok
Ecommerce: One Size Does Not Fit All

2 Comments

3 Comments

Key Elements of a Learning Culture

A “learning culture” is a community of workers continuously and collectively seeking performance improvement through new knowledge, new skills, and new applications of knowledge and skills to Business meeting achieve the goals of the organization. A learning culture is a culture of inquiry; an environment in which employees feel safe asking tough questions about the purpose and quality of what they are doing for customers, themselves, and other stakeholders. In a learning culture, the pursuit of learning is woven into the fabric of organizational life.

This kind of culture puts a value on using a variety of learning methods, including workshops, seminars, online courses, DVDs or online video, games and simulations, coaching, mentoring, action-learning, job-rotation, internships, or any of a dozen other ways to structure learning experiences. The method used depends on what individuals, teams, and whole organizations need to learn.

In addition to these methods of learning and performance improvement, the following social elements are critical to a learning culture:

  1. Building trust– We need to believe that we can rely on leaders to keep their word, that they will do the right thing, and that they will not do anything that is intentionally harmful to us and the organization.
  2. Encouraging risk-taking - If we are going to create new businesses and meet the challenges of mature companies, we must continue to try things that could fail and then we must learn from those failures.
  3. Communicating - Critical communication among leaders, between management and employees, and among departments/units, must be continuous.
  4. Engaging stakeholders – We have to find out what managers want to know and why. Ask them for their thoughts about learning methods. Ask them to help pilot these methods to determine if the methods will produce the learning that is needed. Continue to keep them involved.
  5. Allowing for feedback and reflection – We need an organizational routine of feedback from evaluative inquiry (surveys, interviews, group discussions, etc.) and reflection in which you consider the meaning of the data for learning and performance improvement.
  6. Supporting social learning – We need to create opportunities for people to share information with each other and talk with others about what they have learned and how they intend to apply that learning and how the group can support them in doing so.

What else do you think is key to developing a learning culture in organizations?

3 Comments

Comment

The Next Big Thing: Big Data

Everybody’s talking about “Big Data” these days. But as withany fast moving bandwagon, it’s good to get off once in a while and take stock of the direction in which you are going. So let’s take a look at the pluses and minuses of Big Data. We now have the technology to collect enormous amounts of information about people and to analyze huge datasets that have implications for the economy, education, health care, and social services, as well as enterprises such as agriculture, manufacturing, retail, and much more. Having data is not the problem; the problem is using it effectively and responsibly.

Wally Bock, referring to Johannes Kepler and his discovery of a third law of planetary motion, writes:

Most Big Data operations are the same kind of brute force computation that Kepler did, only much faster. Big Data is the new panacea. It will do just about everything, we're told. But before we start planning for the era of universal peace and an end to disease and hunger which will surely come, it's worth pondering that the computation is only the middle part of the process.

Kepler was using the Big Data of his day, but he had to know what he was looking for. He had to use good, accurate data. And he had to have some idea of how he could use it when he found it. That hasn't changed simply because we can calculate faster.

Making good use of large datasets takes preparation and follow-through. Take health care, for example.  Even when findings from Big Data have obvious disease prevention implications, people do not do what MP900401942 is suggested by the findings. Michael Hickins writes in CIO Journal that Aetna is using Big Data to predict which of its members is likely to develop cardiovascular disease. But even with this information, there is relatively low compliance with the regimen that will likely prevent heart disease. It’s not sufficient to have data. What's important is what we do with that data to change behavior of individuals, teams, organizations, and communities.

Darin McKeever, Deputy Director at the Bill & Melinda Gates Foundation, in a post titled "Moving From Big Data to Big Wisdom", makes the point that Big Data does not necessarily mean that people have the wisdom to improve organizations and society. That data must be turned into something meaningful. Data must become useful information. Useful information must become new knowledge. And it is that new knowledge that becomes the wisdom to bring needed change in new situations.

Getting to wisdom starts before data mining begins. It starts with key stakeholders coming together to decide what questions they want to answer, how the data will be collected and analyzed, what information can be gleaned from the data, and how individual privacy can be protected. It continues with learning from the analysis (knowledge) followed by generating the insights and wisdom needed to sustain change. Big Data, by itself, is not the solution. It’s what people do with that data and how it is turned into wisdom that makes the difference.

Comment

Comment

Invisible Gorilla Strikes Again

This time it’s radiologists. In a study reported by NPR, 83%of radiologists failed to see the image of a MP900178759 gorilla on slides that they were reviewing for signs of cancer.  The conclusion by reporter Alix Spiegel is:

… what we're thinking about — what we're focused on — filters the world around us so aggressively that it literally shapes what we see. So, [the researcher] says, we need to think carefully about the instructions we give to professional searchers like radiologists or people looking for terrorist activity, because what we tell them to look for will in part determine what they see and don't see.

This phenomenon of selective attention permeates our society. Drivers don’t see approaching bicycles and motorcycles. Police can’t always determine who’s the victim and who’s the perpetrator in a violent situation. Soldiers responding in a conflict don’t always see the innocent bystanders in their line of sight. Teachers don’t always notice when a student for whom there are low expectations suddenly does something extraordinary.

When I was a teenager, I along with others observed a socio-drama as part of a lesson in tolerance. In this theatrical event, a crime was committed. We were asked to say who committed the crime. To us it was obvious. When the action was repeated it wasn’t so obvious and we learned about how our prejudices and perceptions influence our judgment, especially in quickly evolving situations.

And I wonder about leaders of large, complex organizations.  Do they sometimes display "selective attention"? Do they look at but not see the signs of a failing business model? Do they look at but not see the signs of a recessionary economy? Do they look at but not see managers who are not doing their jobs effectively? Do they look at but not see customers who are dissatisfied and looking elsewhere. Maybe it’s denial, but it also might be an inability to see something that is unexpected. 

Comment

Comment

What Gets Rewarded Gets Done

To be successful, organizations must align their rewardstructure with their intended goals. Eduardo Porter, in a New York Times article titled, When Public Outperforms Private in Services, writes:

The success or failure of an organization often depends on whether it can clearly identify its goals and align the interests of managers and employees to serve them. Yet whatever reward structure an organization picks can skew incentives in an undesirable way.

A lack of goal clarity and alignment can have detrimental effects. When oil companies reward profit over both safety and protecting the environment, loss of life and environmental disasters are the result. When pharmaceutical companies reward sales people only for the quantity of drugs sold, patients will be prescribed drugs they don’t need. When states judge schools on the basis of test scores, not learning, students graduate unprepared for college and work. When banks reward mortgage agents for quantity rather than quality, a high percentage of loans fail and the entire economy suffers.

I don’t think it’s a question of private vs. public. Some public institutions and government agencies reward the wrong behavior as do some businesses. I remember consulting with a nonprofit health insurance company that rewarded customer service reps for the number of phone inquiries they handled in an hour. The tradeoff was quantity of transactions for customer satisfaction. Treating a customer with respect and solving the customer’s problem were given lip service but not rewarded in practice.

The problem, as Porter says, is tradeoffs. A company that truly values customers, might have to sacrifice MC900438796 some portion of profit in the short term for a satisfied customer who will continue to come back to that business over the long term. A government or nonprofit that truly values its mission, might have to sacrifice some portion of revenue in order to achieve its service goals. Neither type of organization can afford to run at a deficit but neither is sustainable without alignment of rewards with goals.

Comment

Comment

Doing Good Effectively

It’s that giving time of year again.And whether you donate to charities or you are a philanthropist that gives grants to nonprofits, you should care about organizational effectiveness and efficiency.

One commonly used indicator of effective management in organizations that do-good is the MP910221024 percentage of their budgets that goes to administrative costs. The rule of thumb is between 11% and 13%. This is a simple and easily calculated and understandable measure. Unfortunately, it’s not the best indicator. Overhead matters, but effectiveness matters more.

Tina Rosenberg, in a New York Times opinion piece, identifies additional criteria that should be used in judging charitable organizations. She writes that the best organizations…

  • Aim to solve the most serious problems (in the normal calculus, this means that providing bed nets to save children from malaria ranks above helping public radio stations or art museums).
  • Use interventions that work.
  • Employ cost-effective strategies (trachoma surgeries, rather than training guide dogs, to help the blind).
  • Are competent and honest. The percentage of donations spent on overhead is one measure of these qualities.
  • Can make good use of each additional dollar. This is the hardest point to assess, but it asks whether the group has the program on the ground to use your money well, and whether your donation will make something happen that otherwise wouldn’t.

I would also ask these questions:

  • Is the program and the organization sustainable? Assuming the problem being addressed requires a long-term solution, I would want to know if the nonprofit has the leadership, infrastructure, and funding from other sources to stay with the solution as long as is necessary.
  • Does the nonprofit have a culture that is consistent with my values? The leaders might have the best of intentions, but if they treat their staff and volunteers poorly, if they don’t pay their vendors, if they relate to low-income and minorities in a disparaging and condescending way, then I want to know this.
  • Are they using the best interventions available? Maybe they are using interventions that work but maybe there are other interventions that are more effective and less costly. And maybe there are other interventions that should be tried to test their efficacy.

Toby Ord, in the blog Giving What We Can, argues that we should mainly be looking at cost-effectiveness. He believes that we have a moral imperative, especially when it comes to global health, to support programs that get the most bang for the buck. Ord writes:

…merely moving funding from one intervention to a more cost-effective one can produce almost as much benefit as adding an equal amount of additional funding. This is unintuitive since it isn’t the case when one option is merely 10% or 30% better than another. However, when one option is 10 times or 100 times better, as is often the case in global health, redirecting funding is so important that it is almost as good as adding new funding directly towards the superior intervention. In times of global austerity and shrinking budgets, it is good to know how much more can be done within existing ones.

People give to charities and other nonprofits for all kinds of reasons. Some because of a religious calling; some for the prestige; some because of a personal connection to the agency or the cause; some for a tax deduction; some because of peer pressure (for example, they serve on the board of the agency); some to honor a relative by making a donation in that person’s name; and some to make a difference in society. Regardless of the reason, we should be concerned about effectiveness and what it costs to be effective. No one wants to throw their money away and people receiving the money want to be proud of what they do. 

Comment

Comment

Philanthropy, Learning, and Change-Making

The recent U.S. election reminded us on a daily basis of allthe serious problems we face as a nation. Political campaigns made the solutions sound simple, as if one person or one policy could put people back to work, improve our educational system, deliver better health care, clean up the environment, or prevent war. It’s just not that simple. As Baldoni writes in his Forbes blog post, “Post-Election: Now What?”:

Now is the time to put the interests of the whole society first. That will require a commitment – indeed a partnership – of community, business, nonprofit and yes government.

Philanthropic foundations, although not specifically called out in Baldoni’s post, are also part of the solution mix. Steve McCormick, President of the Gordon and Betty Moore Foundation, writes that private foundations should be “game-changers”; they should be “…creating solutions to our most critical social problems.” He writes that philanthropists must do some things differently. One of these is measurement. He admonishes philanthropists to:

Use meaningful measures. It’s difficult to chart progress in the not-for-profit sector. Measurement requires creativity, coupled with discipline, to establish the right indicators—qualitative as well as quantitative. But, as the saying goes:  “if you don’t know where you’re going, you’ll never get there.”

Jeff Raikes, CEO of the Bill & Melinda Gates Foundation, in launching MarketsForGood.org (see below), said that foundations should be continually learning from doing. Conversations about learning do not happen as much as they should in foundations and their grantee organizations. They should be sharing information. Raikes says that the answers exist but it’s too hard to find and apply the information. Bigger, faster impact requires finding ways to get better data to social investors and nonprofits that are doing the work on the ground. He said, “Our work relies on the free flow of quality, accessible data and information…that lead to greater impact and a better world.”  

[youtube https://www.youtube.com/watch?v=9o7JjX-Z1ck?rel=0] 

I agree with McCormick that private foundations can be “game-changers” and that creative, disciplined measures will help make that happen. And I agree with Raikes that much of the data and information already exists to increase impact on society’s most serious problems; that it is more a problem of access to information than it is one of collecting the data.

However, beyond applying measures and accessing data is the challenge of actually using the information to bring about change. This is where many foundations and nonprofits get stuck. They have the information in reports on their shelves or in computer files, but that’s where it stays. That information needs to be liberated, discussed, and translated into action. If they are going to be change-makers, they need to get their organizations engaged in conversations about the implications of the information, plan the action that must be taken to implement what has been learned, and hold themselves accountable for making change.

Comment