Posts

Investigating Failures in Large-Scale Public Sector Projects with Sentiment Analysis

I have co-authored a paper with Sandeep Purao and Jonathan Becker, both based at the College of Information Science and Technology at Penn State University, which analyzes the IRS’s Business Systems Modernization Project using sentiment analysis. The paper will appear in a special issue of e-Service Journal.

We describe results from historical analysis of the IRS Business Systems Modernization (BSM) as an example of large-scale, public sector projects. The project has already spanned a decade and consumed more than 3 billion dollars. The paper suggests extracting stakeholder Sentiments and Confidence from documents, with a view to exploring how such measures may offer early indications of project progress and assist managers to prevent undesirable future outcomes. The key contribution of this research is a demonstration of a plausible technique to elicit stakeholder perspectives based on the content in publicly available documents, either complementing any existing methods, or supplanting them in projects where collecting primary data may be infeasible.

A previous version of this paper was presented at the 44th Hawaii International Conference on System Sciences in theElectronic Government Track(Development Methods for Electronic Government, Minitrack).

Welcome Letter from the Director – Metropolitan Institute

Just completed finalizing my first letter as the Director of the Metropolitan Institute at Virginia Tech. We will soon be loading this on the Metropolitan Institute's website and sharing it with faculty, staff, and friends of Virginia Tech.

Welcome Letter from the Director

Greetings,

It is a privilege to introduce myself as the new director of the Metropolitan Institute at Virginia Tech. I look forward to growing the Metropolitan Institute in the rich history of the College of Architecture and Urban Studies, fueled by the dynamism of the School of Public and International Affairs. As we chart a new course for the Institute, I invite members of the Virginia Tech family (students, professors, and administrators) and past/future partner organizations from the Washington, DC metropolitan region and beyond, to visit our web site, review our upcoming work, and contact us as we explore new collaborations.

For the past decade, my intellectual curiosity has focused on designing more resilient and innovative organizations. Critical to this endeavor is understanding how information is managed within and beyond the organization’s borders. As part of my explorations, I have employed numerous research methodologies from behavioral to computational, and have collaborated with industry and academic partners across a range of disciplines from engineering to public administration and management.

A necessary reality of conducting my research was traveling to various parts of the globe to study organizations, the leaders and managers who were responsible for them, and the employees who supported their mission. Over the course of last five years, I have had the privilege of traveling to over 20 cities, including Mumbai, Madrid, Rome, Ljubljana, Johannesburg, Bangkok, London, and Prague. Observing and experiencing the diversity of metropolitan forms led me to consider the design and consequences of how information traversed urban systems across infrastructures, organizations, and social networks.

Urbanization is a major force of change in our world today and will impact the future of the planet on a number of dimensions from resiliency to sustainability and economic vitality. The need to become more effective and aware of the design and implementation of policies has never been more critical. Central to developing new and more effective models of urban policy are the needs to:

  1. Innovate the policy setting process, making it more dynamic, inclusive, cost effective, and timely.
  2. Leverage information through the deployment of computational systems, simulation platforms, and participatory platforms that allow for crowdsourcing of solutions to local problems
  3. Facilitate multi-disciplinary and multi-stakeholder approaches to framing, studying, and solving, the most complex urban problems.


I envision the future of the Metropolitan Institute as an open collaborative platform that brings together diverse expertise to solve complex urban issues. The Metropolitan Institute will fold together the best and brightest in the fields of informatics, urban planning, international affairs, engineering, and public policy to forge a new set of solutions for today’s urbanizing world. The Metropolitan Institute is a space that is open for all of you to connect and collaborate.

In the near-term, I plan to meet with stakeholders across Virginia Tech and also community partners, industry, and government to listen to your ideas. I have been humbled by the warm welcomes I have received from the various scholars and leaders at Virginia Tech. An esteemed university like Virginia Tech has many friends in public and private organizations across the 50 states and the world beyond. I look forward to meeting with the many friends of the University and welcoming them to the Institute. I am excited to begin designing a new future for Metropolitan Institute.

I look forward to meeting with you and hearing your ideas.

Yours sincerely,

Dr. Kevin C. Desouza
Director, Metropolitan Institute
Virginia Tech

Washington Post: Appointments and Promotions for the Week of Aug. 29

Made the list of important appointments and promotions in the Northern Virginia, Washington D.C. and Maryland area. See - Link

Source: "Washington area appointments and promotions for the week of Aug. 29," Washington Post, Saturday, August 27, 2011.

P.S. The Roanoke Times also highlighted my new role with Virginia Tech (Link)

Defining Policy Informatics

I have been busy working on a statement that captures the essence of policy informatics. Here is my first-cut at the definition. I thank all those who have already provided comments on this version, especially Dr. Erik Johnston (Co-Director of the Center for Public Informatics at Arizona State University). Please do send me your comments, both positive and negative, and suggestions for improvement. Thanks.

Policy Informatics

Solving complex public policy problems, dilemmas, and challenges requires deliberate, and sophisticated, information analysis.

Policymakers often are faced with conflicting solutions to complex problems, thus making it necessary for them to test out their assumptions, interventions, and resolutions. Towards this end, it becomes critical for a policymaker to have an information-rich, interactive environment in which complex problems can be modeled, simulated, visualized, and studied.

Relevant information can range from being too abundant to hardly existent. In the former case, we face the challenge of leveraging large quantities of information under severe time and resource constraints. In the latter case, limited or incomplete information has to be used to make decisions on ambiguous solution spaces.

In deliberating, designing, and implementing policies, the policy makers and the public face a number of transactional and collaborative inefficiencies. Some of these inefficiencies arise from the simple reality that information held by both parties is difficult to articulate and even transfer, i.e. information is sticky. This makes it difficult for either party to collaborate as they do not fully empathize with the problems of the other.

In addition, advances in communication and computational technologies enable new pathways to solutions. Rather than trying to solve public problems, governments are able to empower its public to solve their own problems. Crowdsourcing and bottom-up, emergent, problem-solving are desirable as the public have a greater chance of taking charge of their own local problems, voicing their concerns, and arriving at locally relevant solutions. Designing and mobilizing platforms where citizen input is used effectively to solve local problems and collaborative forums improves the results, and therefore the relationships, for both the policy makers and the public.

Policy informatics is the study of how information is leveraged and efforts are coordinated towards solving complex public policy problems. Driven by the need to exploit information to tackle complex policy problems and to ensure efficient and efficient policy setting and implementation platforms, policy informatics seeks to

  • enhance policy analysis and design through visualizing, modeling, and simulating complex policy scenarios,
  • study the role of information systems and information-based governance platforms in policy planning, deliberation, and implementation,
  • advance the management of information systems projects in the public sector,
  • study how information analysis and management influences the design of participatory platforms, and
  • arrive at theoretical and practical frameworks to advance our knowledge of the roles of information analysis in policy setting, the use of computational techniques in policy contexts, and how information-driven policy setting influences the nature of governance and governance platforms.

Policy informatics helps us advance evidence-driven policy design, wherein scientific models and analyses drive decision-making for resolution of complex policy challenges, dilemmas, and problems. Policy informatics is an emerging field of both research and a community of practice focusing on 1) advancing decision-making in the public sector through information-centric analysis of evidence that leverages computational and technological advances, and 2) designing, managing, and evaluating of information systems and infrastructures for policy construction, analysis, and implementation. Policy informatics expands to the multi-disciplinary nature of the public administration discipline by infusing it with the advances of information technology, management of information systems, and computational and informational science perspectives.

 

Complexity, Policy Informatics, and Public Administration

Erik Johnston (Arizona State University), Qian Hu (University of Central Florida), and I have completed a paper for the Governance of Complex Systems: Challenges of Making Public Administration and Complexity Theory Work Workshop to be held in Rotterdam, Netherlands in June 2011.

How the Application of Complexity and Policy Informatics to Public Administration can Change the Questions we Ask and the Solutions we Discover


We argue for introducing complexity theory to public administration because it allows us to exploit new connections, to raise new questions, and to explore innovative approaches to governance and management. To support more regular, effective, and defensible use of complexity as a contribution to policy-making, public administration scholars must continue to build supporting evidence. In this paper, first, we reflect on why existing analysis frameworks create structural blind spots for understanding governance practice. Second, using examples from our own research and professional experience we demonstrate that a complexity approach allows new questions to be asked that directly connect to policy problems and facilitate decision making in a cost effective manner. Third, we explore a number of factors including practical strategies and ethical concerns that may encourage or preclude the use of a complexity framework and method in policy settings. And finally, this paper calls on public administration scholars to be thoughtfully aware of and ethically responsible for the consequences of the use of complexity methods in research and practice.

Johnston, E., Desouza, K.C., & Hu, Q. “How the Application of Complexity and Policy Informatics to Public Administration can Change the Questions we Ask and the Solutions we Discover,” A Paper for Governance of Complex Systems: Challenges of Making Public Administration and Complexity Theory Work, Rotterdam, Netherlands, June 23-25, 2011.

What is Network Resiliency?

I hope to use this post to begin a discussion on this question. Specifically, how do we define network resiliency when examining large-scale public sector networks. These networks span multiple-levels from individuals to organizations and may even involve consortiums. Consider the case of the US intelligence community (USIC). The USIC involves both public sector organizations (e.g. CIA, NSA, FBI, etc) but also collaborates with intelligence agencies in other countries (e.g. MI6, BND) and even private organizations (Xe Services LLC). The USIC must ensure that its network is resilient. Its resiliency is dependent not only how well it plans for, and executes, responses to changes in its internal and external, but also how well its network (which consists of many organizations it does not have formal control, or even influence, over) fairs in times of crises.

Today, I was examining the literature in telecommunication networks for concepts that we could draw on. The engineering literature has a myriad of concepts that we could draw on to build a framework for organizational network resiliency. For example, consider the concept of load-balancing. Load balancing is essential to the design of robust electronic networks. While its primary purpose is to allow us to plan for efficient usage of resources, load balancing also helps with managing against overload on devices. To describe the concept without getting too technical, one might conceptualize load balancing as follows: incoming information requests to a network are distributed to the appropriate device within the network by a load-balancer. The load-balancer is responsible for routing the request to the best available device (different algorithms might be used for this, and we can have different criteria for determining the best device to route a request to). Load balancing can help us design failsafe mechanisms (for e.g., if one node is down then traffic is routed to a backup node).

Should we have load-balancing mechanisms for organizational networks? Absolutely! I actually think that organizational networks do have implicit load-balancers. Some view these as gatekeepers? Gatekeepers play a vital role in determining how information moves within networks. Do you know of any organizations that manage their gatekeepers mindfully? If so, how do they do it? Also, are there other organizational concepts that are similar to load-balancing?

During my visit to the CIS @ LSE, I conducted an inquiry into how ecological models might help us understand robustness of networks, especially terrorist networks. One idea that I worked hard on is how do agents within a network adapt under conditions of duress. For example, assuming you took away a food source from an ecosystem, how might the various entities (species) adapt and create work-a-rounds? Would the nature of competition among the species change? Would the patterns that drive the reorganization of the ecosystem be predictable?

Towards Evidence-Driven Policy Design: Complex Adaptive Systems and Computational Modeling

Along with my doctoral student, Yuan Lin, I have co-authored an article that describes how we might move towards evidence-driven policy design. This article draws from the keynote that I have at the 2010 Computational Social Science Society Conference.

Efforts to design public policies for social systems tend to confront highly complex conditions which have a large number of potentially relevant factors to be considered and rapidly changing conditions where continuous adaptation delays or obscures the effect of policies. Given unresolvable uncertainty in policy outcomes, the optimal solution is difficult, if ever possible, to nail down. It is more reasonable to choose a solution that is robust to as many future scenarios that might ensue from the decision. Arriving at such a solution requires policy makers to actively explore and exploit rich information to support their decision making in a cost-efficient, yet rigorous manner. We name this new working style as evidence-driven policy design and outline the characteristics of favorable evidence. We then argue that computational modeling is a potential tool for implementing evidence-driven policy design. It helps the study and design of solutions by simulating various environments, interventions, and the processes in which certain outcomes emerge from the decisions of policy makers. It allows policy makers to observe both the intended and, equally important, unintended consequences of policy alternatives. It also facilitates communication and consensus-building among policy makers and diverse stakeholders.

A Longitudinal Analysis of Stakeholder Sentiments: Business Modernization Project at the IRS

The Internal Revenue Service Business Modernization Project undertaken by the Tax Agency of the US Government has been singled out as an example of a massive failure. As envisioned, the project was intended as an Enterprise-wide intervention that would provide modern services and effective data access to citizenry and several government agencies. After more than a decade and 3 billion dollars later, the results appear to be less than exemplary. Sandeep Purao and I have a paper accepted for presentation at the Enterprise Architecture Research (TEAR2010) Workshop that identifies different stakeholders who participated in the project, and analyzes the sentiments and confidence each expressed regarding the fate of the project. We conclude with lessons learned from our investigation including recognizing the importance of multiple stakeholders for Enterprise-wide initiatives.

Purao, S., and Desouza, K.C. “An Enterprise-wide Intervention at IRS: A Longitudinal Analysis of Stakeholder Sentiments,” In Proceedings of the 5th Trends in Enterprise Architecture Research (TEAR2010) Workshop,  Delft, Netherlands (November 12, 2010).

Looking for Clues to Failures in Large-Scale Public Sector Projects

Sandeep Purao and I have a paper accepted at the 44th Hawaii International Conference on System Sciences in the Electronic Government Track (Development Methods for Electronic Government, Minitrack). The paper analyzes the IRS’s Business Systems Modernization Project using sentiment analysis.

Abstract
We describe results from historical analysis of a large-scale, public sector effort: the IRS Modernization Project that has already spanned a decade and consumed more than 3 billion dollars. The results focus on analysis of Sentiments and Confidence expressed by different stakeholders, as found in various documents. We explore how such analyses may provide a window on project progress and potential early clues that may contribute to preventing undesirable outcomes in the future.

Reference: Purao, S., and Desouza, K.C. “Looking for Clues to Failures in Large-Scale Public Sector Projects: A Case Study,” In Proceedings of the Forty-Forth Hawaii International Conference on System Sciences (HICSS-44), Los Alamos, CA: IEEE Press, Kauai, HI, (January 4-7, 2011).