Towards Evidence-Driven Policy Design: Complex Adaptive Systems and Computational Modeling

Along with my doctoral student, Yuan Lin, I have co-authored an article that describes how we might move towards evidence-driven policy design. This article draws from the keynote that I have at the 2010 Computational Social Science Society Conference.

Efforts to design public policies for social systems tend to confront highly complex conditions which have a large number of potentially relevant factors to be considered and rapidly changing conditions where continuous adaptation delays or obscures the effect of policies. Given unresolvable uncertainty in policy outcomes, the optimal solution is difficult, if ever possible, to nail down. It is more reasonable to choose a solution that is robust to as many future scenarios that might ensue from the decision. Arriving at such a solution requires policy makers to actively explore and exploit rich information to support their decision making in a cost-efficient, yet rigorous manner. We name this new working style as evidence-driven policy design and outline the characteristics of favorable evidence. We then argue that computational modeling is a potential tool for implementing evidence-driven policy design. It helps the study and design of solutions by simulating various environments, interventions, and the processes in which certain outcomes emerge from the decisions of policy makers. It allows policy makers to observe both the intended and, equally important, unintended consequences of policy alternatives. It also facilitates communication and consensus-building among policy makers and diverse stakeholders.

Reflections from Slovenia: Designing Public-Private Innovation Partnerships

I returned from Slovenia about a week back. During my visit, I had the opportunity to give a keynote talk at the Center of Excellence for Biosensors, Instrumentation and Process Control as part of the Slovenska visokotehnološka MSP na prepihu inovativne in razvojno tehnološke prebojnosti: Slovenija x.0 ? conference. I met with several executives during the conference and enjoyed exchanging ideas on how to design collaborative innovation platforms that promote private-private and private-public innovation partnerships. A key issue that surfaced is how to design an appropriate governance structure so as to promote knowledge transfer and collaboration among industry players that have a lot to gain (and lose) from collaboration. Alignment of incentives, sharing of risks, and even design of prototype collaborative endeavors are all essential components to build collaborative innovation partnerships.

A Longitudinal Analysis of Stakeholder Sentiments: Business Modernization Project at the IRS

The Internal Revenue Service Business Modernization Project undertaken by the Tax Agency of the US Government has been singled out as an example of a massive failure. As envisioned, the project was intended as an Enterprise-wide intervention that would provide modern services and effective data access to citizenry and several government agencies. After more than a decade and 3 billion dollars later, the results appear to be less than exemplary. Sandeep Purao and I have a paper accepted for presentation at the Enterprise Architecture Research (TEAR2010) Workshop that identifies different stakeholders who participated in the project, and analyzes the sentiments and confidence each expressed regarding the fate of the project. We conclude with lessons learned from our investigation including recognizing the importance of multiple stakeholders for Enterprise-wide initiatives.

Purao, S., and Desouza, K.C. “An Enterprise-wide Intervention at IRS: A Longitudinal Analysis of Stakeholder Sentiments,” In Proceedings of the 5th Trends in Enterprise Architecture Research (TEAR2010) Workshop,  Delft, Netherlands (November 12, 2010).

Looking for Clues to Failures in Large-Scale Public Sector Projects

Sandeep Purao and I have a paper accepted at the 44th Hawaii International Conference on System Sciences in the Electronic Government Track (Development Methods for Electronic Government, Minitrack). The paper analyzes the IRS’s Business Systems Modernization Project using sentiment analysis.

We describe results from historical analysis of a large-scale, public sector effort: the IRS Modernization Project that has already spanned a decade and consumed more than 3 billion dollars. The results focus on analysis of Sentiments and Confidence expressed by different stakeholders, as found in various documents. We explore how such analyses may provide a window on project progress and potential early clues that may contribute to preventing undesirable outcomes in the future.

Reference: Purao, S., and Desouza, K.C. “Looking for Clues to Failures in Large-Scale Public Sector Projects: A Case Study,” In Proceedings of the Forty-Forth Hawaii International Conference on System Sciences (HICSS-44), Los Alamos, CA: IEEE Press, Kauai, HI, (January 4-7, 2011).

Large IT Projects as Interventions in Digital Ecosystems

Sandeep Purao (IST, Penn State University) and I have a paper accepted for presentation at the International ACM Conference on Management of Emergent Digital EcoSystems (MEDES’10) (BangkokThailand).

Abstract: Large IT projects, such as the US Government’s Internal Revenue Service Business Modernization Effort, can take a decade or more and consume billions of dollars. Traditional approaches to the study of such projects emphasize concerns such as requirements monitoring, progress tracking and risk mitigation. We propose an alternative approach guided by a digital ecosystems view instead of a hierarchical, decision-oriented view. We argue that this perspective is more suited to understand how such projects evolve and cause changes in the underlying digital ecosystem characterized by not only the IT infrastructure but also the transactional relationships among stakeholders. We illustrate our arguments by drawing on an archaeological case study of the IRS effort, and discuss implications of the digital ecosystem perspective for the study of large IT projects.

Reference: Purao, S., and Desouza, K.C. “Large IT Projects as Interventions in Digital Ecosystems,” In Proceedings of the International ACM Conference on Management of Emergent Digital EcoSystems (MEDES'10), Bangkok, Thailand (October 26-29. 2010).

Role of Internet-based Information Flows and Technologies in Electoral Revolutions: Ukraine’s Orange Revolution

Volodymyr V. Lysenko and I have co-authored a paper that examines the role played by Internet-based information flows and technologies in electoral revolutions. Recent events have drawn attention to the use of Internet-based information and communication technologies (ICTs) in the political process. For instance, ICTs played an important role during attempts at electoral revolutions in Moldova in April 2009 and Iran in June 2009. Employing a case study approach, we examine the part played by ICTs during the Orange Revolution in Ukraine (2000-2004). Roles and activities of the dissenters, as well as their associates, the incumbent authorities and their allies are analyzed with regard to Internet-based technologies during the electoral revolution in Ukraine. The case of the Orange Revolution is particularly salient, as even though only 1-2 percent of the Ukrainian population had access to the Internet, this was sufficient to mobilize the citizens towards an eventually successful revolution. This paper lays the groundwork for further investigations into use of ICTs by political dissenters. The paper will appear in a forthcoming issue of First Monday.

Keynote Address at the 2010 Computational Social Science Society Conference: From Hunches to Evidence Driven Policy Design

I will be giving a keynote address at the 2010 Computational Social Science Society Conference (CSSS). CSSS 2010 is hosted by the Center for Social Dynamics and Complexity and the Consortium for Biosocial Complex Systems at. For more information on the conference, please click here [LINK].

From Hunches to Evidence Driven Policy Design: Leveraging Information through Simulation

Constructing public policy, whether at the national or local level, is a complex undertaking. Complexity arises from the number of stakeholders involved, varying agendas and incentives, resource constraints, a multitude of interacting variables, multiple time horizons, and even political climates. Due to these complexities, we too often categorize political and social problems as ‘wicked’ and unanalyzable. The default option is to take a haphazard approach to policy design, most often the outcome of the ego-based debates and negotiations of the decision-makers. In this keynote address, I will argue for a move from hunches (or intuition) to evidence driven policy construction. Today, due to the advancement of computational power and modeling techniques, we can simulate complex scenarios. Simulation gives us an ability to move policy construction from an activity primarily driven by historic case analysis and intuitions, to more of an applied science, where we can actually predict and control phenomenon. Through simulation we can, with reasonable certainty, ascertain the cost, benefit, risk, impact, and value proposition of a given policy. Using examples from simulation projects, such as a project that examined strategic options for dismantling terrorist networks, I will demonstrate how we can move policy design from being an ‘art’ to more of a ‘science.’

Conference on Intelligence and Nuclear Proliferation: Threat Identification, Policy Formulation and Decision Making, June 3-5, 2010

I will be speaking at the Conference on Intelligence and Nuclear Proliferation hosted by the Centre for Science and Security Studies (CSSS) at King’s College London in June. Kristen Lau and I have authored a paper that examines how information management failures led to an inability to adequately assess and detect nuclear threats in recent times. Lack of adequate information management capabilities have led to numerous international crises surrounding nuclear non-proliferation. For example, the inability to predict nuclear tests by India in 1998, the colossal failures surrounding assessments of Iraq’s WMD capabilities in early 2000, and today, the challenge of addressing Iran and North Korea.

Intelligence and Nuclear Non-Proliferation Programs: The Achilles Heel?
Intelligence is a critical component of all counter-proliferation activities. It allows us to assess and determine what makes up the current threat environment in terms of the proliferation of nuclear weapons and technology. When informed with an accurate assessment of the situation, policy makers are better suited to counter the proliferation threat. However, success and failure hinge upon how well information is managed during the intelligence process. The intelligence process as it relates to estimating nuclear capabilities or intentions is wrought with many challenges and complications. The denial and deception techniques employed by states running covert weapons programs and the dual-use nature of many weapons components create many difficulties for intelligence organizations. Additionally, illicit transnational networks obscure the situation further by serving as a source, for both nation states and non-state actors, for acquiring dual-use commodities and technologies. These challenges can lead to the miscalculation of a state’s capabilities or intentions. As was seen with the case of Iraq in 2003, western intelligence services grossly overestimated the capabilities of Saddam’s regime. This paper presents a comparative analysis of three cases of nuclear proliferation: India, Pakistan and Iran. Drawing from the analysis, the authors examine the lessons learned and propose recommendations for future counter proliferation policy and strategy.

To read prior papers published on this topic, please see:
• Desouza, K.C., and Lau, K.A.* “Managing the Proliferation of Weapons of Mass Destruction: An Information Management Perspective,” International Journal of Public Administration, 31 (13), 2008, 1457–1512. [LINK]
• Desouza, K.C. “Information and Knowledge Management in Public Sector Networks: The Case of the US Intelligence Community,” International Journal of Public Administration, 32 (14), 2009, 1219–1267. [LINK]

Cyberprotest in Contemporary Russia forthcoming in Technology Forecasting and Social Change

Volodymyr V. Lysenko and I have authored paper that explores the possibilities of the Internet as a tool for supplying information necessary for the organization and mobilization of successful opposition movements, especially under non-democratic regimes. Examples of the roles the Internet plays in the political processes in Russia are discussed in detail. In particular, the recent cyberprotest cases of the website and the movement to release political prisoner Svetlana Bakhmina are investigated. Besides showing the Internet’s significant role in organizing modern protests, these cases also demonstrate that in environments where practically all traditional mass-media are under the authorities’ control, the Internet becomes the major source of alternative information. Our paper offers a look at how deploying technologies can bring about social change, even in some of the most difficult political environments.

The paper will appear in Technology Forecasting and Social Change. Volodymyr and I will present the paper at the Harriman Institute for the Etiology and Ecology of Post-Soviet Media Conference at Columbia University on May 7-9, 2010.

Measuring Agility of Networked Organizational structures via Network Entropy and Mutual Information

Yuan Lin, Sumit Roy, and I have authored a paper that examines the use of network entropy and mutual information to measure the agility of networked organizational structures. The paper will appear in Applied Mathematics and Computation.

While the agility of networked organizational structures is important for organizational performance, studies on how to evaluate it remain scant, probably because the difficulty in measuring network evolution. In this conceptual paper, we propose two measures – network entropy and mutual information – to characterize the agility of networked organizational structure. Rooted in graph theory and information theory, these two measures capture network evolution in a comprehensive and parsimonious way. They indicate the uncertainty (or disorder) at the network level as well as the degree distribution at the individual level. We also propose an algorithm for applying them in the scenario of adding links to a network while holding the number of nodes fixed. Both simulated and real networks are used for demonstration. Implications and areas for future research are discussed in the end.

Lin, Y., Desouza, K.C., and Roy, S. “Measuring Agility of Networked Organizational structures via Network Entropy and Mutual Information,” Applied Mathematics and Computation, Forthcoming.