My paper co-authored with Yuan Lin, Towards Evidence-Driven Policy Design: Complex Adaptive Systems and Computational Modeling, will appear in The Innovation Journal: The Public Sector Innovation Journal.
My second post on the Harvard Business Review site went live today! The post was written in collaboration with H. James Wilson and is titled, Finally, A Majority of Executives Embrace Experimentation. The post outlines the value proposition of building an experimentation culture within organizations and how executives can support employee experimentation.
The post has been picked up by Bloomberg Businessweek as well.
We would love to hear your comments on the ideas presented.
Along with my doctoral student, Yuan Lin, I have co-authored an article that describes how we might move towards evidence-driven policy design. This article draws from the keynote that I have at the 2010 Computational Social Science Society Conference.
Efforts to design public policies for social systems tend to confront highly complex conditions which have a large number of potentially relevant factors to be considered and rapidly changing conditions where continuous adaptation delays or obscures the effect of policies. Given unresolvable uncertainty in policy outcomes, the optimal solution is difficult, if ever possible, to nail down. It is more reasonable to choose a solution that is robust to as many future scenarios that might ensue from the decision. Arriving at such a solution requires policy makers to actively explore and exploit rich information to support their decision making in a cost-efficient, yet rigorous manner. We name this new working style as evidence-driven policy design and outline the characteristics of favorable evidence. We then argue that computational modeling is a potential tool for implementing evidence-driven policy design. It helps the study and design of solutions by simulating various environments, interventions, and the processes in which certain outcomes emerge from the decisions of policy makers. It allows policy makers to observe both the intended and, equally important, unintended consequences of policy alternatives. It also facilitates communication and consensus-building among policy makers and diverse stakeholders.
I am a footballer first and foremost, and then, an entrepreneur, academic, or whatever else you may categorize me as. Growing up, I had the privilege of playing on some good teams and had aspirations to be a great footballer (for my American readers, I am not referring to American Football, but the rather what you call Soccer). My favorite team has always been Manchester United Football Club (MUFC), and my favorite coaches have always been Sir Alex Ferguson and Jose Mourinho. The recent performance by MUFC has been far from stellar. The team has lost points on matches that they should have won, and they have allowed leads to slip by, most recently against West Bromwich Albion. One of the reasons for this has been the precarious performance of their star performer, Wayne Rooney. Wayne Rooney, if you believe the press reports, has fallen out with his manager, Sir Alex Ferguson. Sir Alex had chosen to rest his superstar for several games, claiming that his player needed time to recover from an ankle injury that he sustained last season. Soon thereafter, after national team duty with England, Rooney spoke to the press and contradicted his manager. He claimed that nothing was wrong with his ankle. Today, speculation abounds that Rooney might be ready to part ways with MUFC!
I am fascinated by these developments for several reasons, and what is surprising, even to me, is that I am excited from the perspective of organizational behavior and management, rather than as a raging Red Devil. Dealing with superstars is never an easy proposition for any manager. As one colleague put it, “I love my superstars as much as I hate managing them!” Last season, Rooney carried MUFC, scoring 34 goals, a sizable proportion of which kept MUFC in the premier league title race. However, since sustaining an injury in a match against FC Bayern Munich in the Champions League, he has never really played his best. He had an awful World Cup showing, and has been insignificant for most of the season. So, what is Sir Alex to do?
In discussing this situation with several executives, I wanted to share some food for thought. First, we need to understand that all employees will have ups and downs. Rooney is now in a slump and his performance has suffered. Sir Alex may have had the right intention in benching him (i.e. giving him a break); however, maybe a better strategy would have been to let him to play. As one executive remarked to me, “the most difficult thing for superstars is to deal with lows…the only way out is to keep at it, and then witness the positive trajectory.”
Second, Sir Alex needs to take a step back and allow the team to manage Rooney. Sir Alex is a beast of manager (now reaching 70); he is probably going to go down in history as the most decorated football manager. Rooney, on the other hand, is a kid (only 24 years old). Sir Alex needs to have a conversation with Rooney to reassure him that he is committed to seeing him blossom at MUFC. Then, he needs to enlist the support of his senior players, most notably Ryan Giggs, who is MUFC and Premier League’s most decorated player, in a coaching role. Giggs, who is closing in 40, is at the tail-end of his career and has expressed an interest in coaching. I suggest that Sir Alex mentor Giggs in the art of coaching, by giving him the ‘Rooney Project.’ No matter how good a manager that he is, innovative and star players, trust other players who they have seen deliver. They trust peers who have expertise and the necessary experience. Sir Alex has at his disposal ‘the most decorated player’ in the Premier League, who could help Rooney get out of the slump. Gigg’s playing days are numbered, but he still has a lot to contribute to the game of football, and young lads still look up to him as a role-model.
Third, in dealing with Rooney, especially his behavior on and off the pitch, Sir Alex needs to carefully consider the message that his actions will send to other players. Sir Alex has been known to have a zero-tolerance policy towards player dissent (e.g. past star players, like David Beckham and Ruud van Nistelrooy, were shown the exit door for not abiding by their manager’s request or dissenting). On a personal level, I think Sir Alex is right to discipline players who are selfish and do not put the club before their own interests; however, in Rooney’s case, a small exception can be made, considering that he carried the club on his shoulders the previous season. Sir Alex needs to take all of this into account in making any determinations on how to proceed. One way to address this is best captured by a colleague, “superstars are individuals, and at the end, they still need a team to perform…so, if the superstar is delivering constantly that means the rest of the team is not! Eventually, the team needs to give the superstar a break and deliver on their own.” Sir Alex needs to give Rooney a break, but establish clear parameters around his behavior and professional conduct. In addition, the rest of the team needs to be scolded and shamed into increasing their level of performance so that they are a ‘team’ and not a ‘one man show.’
What do you think Sir Alex should do?
Reflect on how much of your day is spent asking questions and answering questions. How many questions do you ask a day, what kinds of questions do you ask, and why do you ask the questions you do. Try having a conversation with a colleague, or a friend, without asking questions, how long might you go without asking a question? The simple answer: not too long. Questions, and questioning, make for an integral aspect of our lives. Yet, how many of us take the time to be mindful about the questions we ask and how we answer questions. Within organizations, the asking of, and responding to, questions, makes for a critical mechanism through which we elicit information and construct dialogues. Managers that ask good, and hard, questions of their employees in an efficient, and respectful, manner are respected by their employees. Conversely, managers who ask ‘dumb’ questions (yes, there are things like dumb and stupid questions), and do not following basic etiquette when doing so, are more likely to be dismissed by their employees as being incompetent. Similarly, employees are perceived as smart (or foolish) based on the questions they ask and their ability to respond to questions. For the last few years, I have been fascinated with the concept of questions and the mechanics of questioning.
Questioning plays a critical role in organizational discourse. We often hear statement such as: ask the hard questions, question the status-quo, or there is nothing like a stupid question, among others. These statements give lip service to the concept of questioning. Managers are some of the most poorly trained questioners. Students in disciplines such as psychology, medicine, and law, are explicitly taught how to question. Business students almost never examine the art, and science, of questioning in a thoughtful manner. As a result, one of the most cited reasons for organizational failures (such as corporate scandals or committing to a failed course of action), is the inability for those who were observing the disaster unfold to be courageous enough to ask the right questions (and seek appropriate answers). Just imagine what would happen if a psychologist did not ask questions appropriately or if your physician did not ask the right questions to diagnose ailments. Would we tolerate this level of incompetency? Probably not!
Organizations need to urgently embrace the art and science of questioning. I believe that organizations will be healthier if individuals knew how to ask the ‘right’ questions and how to respond to questions. Being deliberate about the concept of questioning will lead to organizations expelling less effort in achieving their goals and objectives. I am currently beginning to write a few articles on questioning. While most of my writing will be for a business (management) audience, they are relevant to fields such as engineering, new product development, and education, among others. I want to encourage all readers to share their experiences with me on the topic. What do you think about questioning? How do you differentiate a good question from a bad one? What kinds of questions do you ask and why? Do you know of people who ask the ‘right’ questions, if so, why do you think they are successful in asking questions? These are just some of the questions that I would love to get answers to. If you have other reactions to the issue of questioning, please do share them with me.
If you would like me to come to your organization and conduct a workshop on the topic, please do not hesitate to contact me. I guarantee that if your organization gets smarter at the art and science of questioning, it will be a more ‘intelligent’ and ‘mindful’ place.
I will be giving a keynote address at the 2010 Computational Social Science Society Conference (CSSS). CSSS 2010 is hosted by the Center for Social Dynamics and Complexity and the Consortium for Biosocial Complex Systems at. For more information on the conference, please click here [LINK].
From Hunches to Evidence Driven Policy Design: Leveraging Information through Simulation
Constructing public policy, whether at the national or local level, is a complex undertaking. Complexity arises from the number of stakeholders involved, varying agendas and incentives, resource constraints, a multitude of interacting variables, multiple time horizons, and even political climates. Due to these complexities, we too often categorize political and social problems as ‘wicked’ and unanalyzable. The default option is to take a haphazard approach to policy design, most often the outcome of the ego-based debates and negotiations of the decision-makers. In this keynote address, I will argue for a move from hunches (or intuition) to evidence driven policy construction. Today, due to the advancement of computational power and modeling techniques, we can simulate complex scenarios. Simulation gives us an ability to move policy construction from an activity primarily driven by historic case analysis and intuitions, to more of an applied science, where we can actually predict and control phenomenon. Through simulation we can, with reasonable certainty, ascertain the cost, benefit, risk, impact, and value proposition of a given policy. Using examples from simulation projects, such as a project that examined strategic options for dismantling terrorist networks, I will demonstrate how we can move policy design from being an ‘art’ to more of a ‘science.’
Once you have advocated, screened, and funded ideas, the next step is to engage in experimentation. To experiment is to try something new. It allows you to observe the interplay between cause and effects—i.e., it is the application of scientific methods to generate actionable knowledge. Simply put, experimentation can be considered the sum of all activities we engage in to test the feasibility and elasticity of an idea. On the feasibility side, we are normally looking at the cost, benefits, effort, resources, and risks involved in transforming the idea into a viable product and/or service. In addition to feasibility, the experimentation process will also unearth the elasticity of an idea. Idea elasticity focuses on the reach of the product and/or service. Elasticity tries to see how far you can stretch ideas, both in terms of the number of products and/or services that you can develop from them, and also the domains in which you can deploy them.
It is through the process of collecting data through the conduct of experiments that we can find support for the ideas. Data collected during the experimentation process will help us gather the necessary evidence to support decision-making. Today, there is a big movement in medicine called evidence-based medicine (EBM), which demonstrates that a move towards more scientific and data-driven decision-making can prove to be valuable, rather than purely relying on gut instincts. Dr. Dave Sackett, a pioneer in the field states “EBM is saying rather than just rely on tradition, expert opinion, wishful thinking, let's try and find the evidence and apply it.”  To build a culture of experimentation one must focus on the following principles: 1) do not just discard ideas without adequate evidence, 2) do not support or move ideas ahead without adequate evidence, and 3) always look to exploit data from experiments.
While at Amazon from 1997-2002, Greg Linden prototyped a system that would make personal recommendations to customers as they checked out. Linden commented, “I heard the SVP was angry when he discovered I was pushing out a test. But, even for top executives, it was hard to block a test. Measurement is good. The only good argument against testing would be that the negative impact might be so severe that Amazon couldn't afford it, a difficult claim to make.”  Linden’s experiment showed how much the customer liked the feature and it won praise – the end result is that this has become a signature design feature for Amazon, and most online web marketers have introduced a similar concept. This illustrates the value and capabilities of organizations to test incremental ideas and achieve innovation through “continual tiny experiments in such areas as business processes and customer relationships rather than a single, company-transforming idea.” 
Experimentation needs to be made part of every employee’s work and has to move beyond the R&D Labs. The R&D Labs have natural constraints that leave a lot to be desired in terms of experimentation. For example, most of the R&D personnel are detached from the day-to-day running of the business and hence are not the best people to experiment on the problems and solutions of interest for today. In addition, these labs are often physically secluded from the operational centers of the business. This separation leads to difficulty when you try to transport (mobilize) ideas from the lab in order to address problems that are happening on the ground. Finally, you also have a numbers issue. The number of individuals working in an R&D lab is minimal compared to employees who are involved with the day-to-day running of the business. As such, no matter how brilliant your R&D lab personnel are, you will be at a loss if you cannot find ways to tap into the 85-90% of your organization’s employees who do not work in the lab. At the Engaged Enterprise, we had a R&D lab, the Institute for Engaged Business Research (IEBR). IEBR was focused on working on applied management problems that had value propositions to our clients. We determined upfront that simply relegating experimenting and innovation to the labs was not optimal. We needed to find a way to blend the experiences of those working on the consulting side with the R&D side. In addition, we needed to find ways to take knowledge that was being generated on the consulting side (as experiments were conducted “live” while projects were being done – i.e. as we tried to install a new service or strategize with a client – we were in essence engaging in experimentation) and move these into the R&D lab. We did this by having people share their time between consulting and R&D. Also, we had our R&D folks develop a method and handbook that could be used for experimentation which explained the basics of experimentation and how to capture and store results. We also encouraged sharing of results from experimentation efforts so that others might use the results, or provide their reflections on the experiments.
The effort to move experimentation beyond the R&D labs needs to be a conscious one. Both, organizational (management) and employee level interventions need to be in place to promote this concept. Managers should not only encourage their employees to experiment with their ideas, but even go so far as making it a requirement when ideas are being developed and proposed. In addition, employees should take responsibility to engage with the experimentation process, and be aware of methods and practices for conducting experiments.
please stay tuned for my new book or send me a note via email…
 Sackett, Dr. Dave. (October 30, 2009). Interviewed by André Picard. Available at: http://www.theglobeandmail.com/life/health/when-we-began-we-were-almost-pariahs/article1344833/
 McCann, David. (March 15, 2010). Testing, Testing: The New Innovation Game. Available at: http://cfo.com/article.cfm/14482988?f=search