Capturing the Wisdom of Crowds – Planning

APAI have an article in the current issue of the American Planning Association's Planning magazine.

Capturing the Wisdom of Crowds

Combining citizen intelligence and online civic platforms.

By Kevin C. Desouza and Kendra L. Smith

Technology platforms for citizen intelligence are springing up quickly. Platforms such as Deliberatorium, DebateGraph, Cohere, YourView, and CoPe_it! all allow for extensive discourse. Each has special features such as multiple ways to contact other users and participate in discussion boards. Additionally, these platforms employ social analytics, discourse analytics, and social network maps. These sites allow users to gather information and debate ideas and solutions to specific community issues.

Users can also add evidence and information to other users' claims, which triggers conversations and sharing. In many U.S. cities, leaders are finding value in citizen intelligence. Online civic platforms tend to fall into four main categories, as one of us has also noted in an upcoming Journal of Urban Technology article. To read the more, please click here.

To read the print version, please click here.

Speaking at the GSA on the Future of Challenges and Contests in the Public Sector

GSAlogoOn April 18, I will deliver a webinar from the General Services Administration on the future of challenges in the public sector. This presentation will draw on my work funded by the IBM Center for the Business of Government on The webinar will also highlight findings from my recent work that is looking at how to leverage collective intelligence on participatory platforms. I will conclude with guidelines on how to manage ideas within public agencies based on my book, Intrapreneurship: Managing Ideas Within Your Organization. To register for the webinar, please click here. The webinar is organized by DigitalGov University.

Baumer Lecture at Ohio State University – Designing Smart Cities

logo-tosusquare-flatI will be visiting The Ohio State University on February 6th, 2013. During my visit, I will deliver a talk as part of the Baumer Lecture Series in Ohio State’s Knowlton School of Architecture. My talk will outline how technologies are changing the face of urbanization. Specifically, I will outline how citizens are leveraging technologies to develop innovations in planning and governance of urban spaces. Due to the democratization of technologies, the availability of open data, and an educated citizenry, we are seeing an unprecedented rate of innovation in the design, planning, and management of our cities today. This talk will draw on my recent research projects on smart cities, citizen apps and urban technologies, big data management, and challenges and competitions for crowdsourcing innovation.

Speaking at NASCIO Leadership Summit – Oct 2012

I will be speaking at the National Association of State Chief Information Officers (NASCIO) Leadership Summit on Oct 25, 2012 in San Diego, California. My talk will draw on my book, Intrapreneurship: Managing Ideas within Your Organization, and focus on how to lead through collaboration.

For more details on the event, please click here: -- NASCIO 2012 Leadership Summit

To have me speak at your event, please send me an email

James Pol, US Department of Transportation Intelligent Transportation Systems (ITS) Joint Program Office.

As part of our Citizen App project, we interviewed James Pol at the US Department of Transportation on his experience with the Connected Vehicle Technology Challenge.

Current Position: Team Leader, Program Management and Evaluation at the US Department of Transportation Intelligent Transportation Systems (ITS) Joint Program Office.

Bio: James is a classically trained civil engineer, receiving his Bachelors from Rensselaer Polytechnic Institute and a MS in Computer Systems Management from the University of Maryland. He has experience in both the public and private sector. Over the last decade, James has worked in the Federal Highway Administration as a program manager, with a particular focus of real-time information. In 2008, he began work with the US DOT and currently supervises the US DOT’s Intelligent Transportation System (ITS) research projects.

Federal App Challenge in Focus: The Connected Vehicle Technology Challenge is centered on collecting innovative ideas and uses for dedicated short range communications (DSRC); wireless technology enables vehicles to communicate. The challenge did not require technical submissions, and winners were awarded with a free trip to the Intelligent Transportation Systems World Congress. The challenge received a total of 76 submissions.

Motivation for initiating the challenge:

  • In general: The Intelligent Transportation Systems Joint Program Office has an eye towards advancing technologies, and helping to generate forward progress in the transportation practice. The office focuses on getting newly developed technologies out of the research space and into deployment. The challenges help them to carry out their mission, which includes engaging a larger audience of stakeholders and connecting beyond the usual RFPs. James notes that often when one works in the contracting arena, one is limited to certain audiences. This lessens one’s ability to reach some of the newer and more innovative technologies.
  • Challenge specific: ITS sought to broaden awareness of connected vehicles technology (establishing a network of communication between vehicles), and its role within the research field. Moreover, they targeted students, seeking to engage the next generation of engineers, economists and others interested in this arena. The ITS program realizes that these individuals are the ones coming into the workforce, therefore being in touch with this technology is essential.

Process for organizing the challenge:

They defined a challenge, targeted a specific group of participants and determined a prize. The prize was a paid trip to attend the World Congress on Intelligent Transportation Systems and provided exclusive access to technology demonstrations. A total of 3-4 Individuals were involved in defining the challenge, while an additional 6 provided further input. There were 10 panelists to review the challenge submissions upon their completion.
In terms of managing the platform, once a challenge is created and coordinated with GSA (the manager of, a moderator is established for the account. Both the moderator and James reviewed content and submissions on the site. In this challenge’s profile on the platform, some example submissions were provided. In addition, they also provided access to background information on connected vehicles to aid participants in understanding the underlying technologies.

They did borrow from existing programs, for things such as their judging criteria. However, the challenge was written by DOT’s ITS. The planning and design of the challenge was done prior to the establishment of the America Competes Law.


ITS developed a publicity plan to build awareness on the work the government was already doing with connected vehicles. The challenge was announced at a PR event at the annual Transportation Board Meeting. From there it immediately took off, and subsequently received 200 press impressions of all sorts; from blogs to webpages, to Wired magazine. One result of the immediate and extensive press coverage was the ability of ITS to refocus on other efforts and research. However, they did maintain support for the challenge, re-engaging the public at other events held by the agency.


Internal judges were volunteers from around the DOT and ITS. The department varies significantly, and ITS worked with 6 other agencies (Federal Highway Administration, National Highway Traffic and Safety Administration, Federal Transit Administration, Federal Motor Carrier Safety Administration, Research and Innovative Technology Administration, and Federal Railroad Administration) within the department. These internal judges then chose the five best submissions, while also eliminating irrelevant or unrelated proposals. In total 90 submissions were gathered, 75 submissions were made available for public voting. A submission from a team out of Clemson University won the people’s choice award. After the ITS World Congress, the agency has had minimal contact with the winners.

Meet expectations/goals:

Overall, the response out paid their expectations. They would’ve been happy with 40 submissions, however they received close to 90 submissions. allows people to ‘like’ and comment on the challenges, this is one way to gauge interest. The challenge was well within the top 10 of the challenges listed at the time for activity and submissions. The challenge out performed even those with cash-prizes.

In terms of content, the challenge was devised to gather new concepts. Overall, the submissions helped validate some of their research endeavors, such as electric vehicle fleets and autonomous vehicles. However, nothing specific from the challenge has been directly translated into their research activities.

Critical Lessons:

  • Spend time early-on developing and crafting easy to convey messages. Connected vehicles is an inherently complex research field; the solver community out there, even for smart people, requires a bit of effort to express what it takes to achieve this and submission expectations.
  • Low barrier to entry for this challenge. As such, must be very specific as to what is expected for submissions.
  • Require that it be an original thought. One concept was very closely aligned with another research project from around the country. Thus, better safeguards are needed against this type of submission.

Do you see challenges being important to other agencies:

In short, the answer is yes, but it is difficult to quantify. The Department of Transportation is rather committed to applying this type of initiative. In terms of, conducting research he sees it as evolving into a major component. Moreover, it is a useful method of achieving a knowledge/technology transfer, and a way to boost participation by individuals. Other agencies have contacted him on advice and he has been invited to judge other challenges.

Currently, there are three other agencies in the department defining challenges. He has been approached by agencies within DOT and colleagues in the ITS program with ideas for new challenges. James is additionally a part of a department-wide work group to provide guidance to agencies in conducting challenges.

Major risk/concern with running these challenges:

Major risk is making sure to abide by America COMPETES Act, whether or not the challenge is being conducted within what the law establishes agencies to do. A lot of discussion and negotiation with legal and procurement must exist during the design and implementation of a challenge. Challenges are new for agencies, and it will take time to gain awareness and a level of comfort. However, enthusiasm is building for it.

How clear is the America COMPETES Act:

Interpreting the law is still a work in progress and questions do remain. America COMPETES includes requirements such as, requiring judges from both agencies and the industry. As a result, vetting conflicts of interest becomes necessary. This adds a significant amount of time to just defining and selecting a judge. Efforts are still underway to figure out how early in the process to include the judging panel. As well as, how to compensate non-federal employees.

Community Intelligence Platforms: The Case of Open Government

Akshay Bhagwatwar (Kelley School of BusinessIndiana University) and I have a paper accepted at the Eighteenth Americas Conference on Information Systems (AMCIS), to be held in Seattle, Washington, August 9-12, 2012. This paper builds on our ongoing work in policy informatics, citizen apps, and design of participatory platforms.

Community Intelligence Platforms: The Case of Open Government

The focus on collaborative and participatory governance has led to interest in studying how ‘intelligence’ in citizen communities can be leveraged towards creating robust solutions for complex social and policy problems. In this paper, we present four models that uncover the process of leveraging community intelligence. We analyze multiple case studies that capture the varying roles of citizens and public agencies in the problem-solving process. Employing Arnstein’s (1969) ladder of citizen participation as an analytical tool, we outline the strengths and weaknesses of each model, and suggest design recommendations for the development of participatory platforms for open government. 


Brad Larson, Creator of Molecules App, Honorable Mention @ NIH/NLM “Show off your Apps” Competition

As promised, in my previous post, my research team and I will be featuring various app developers who have contributed to challenges run by federal agencies on

Developer: Brad Larson
Bio: Brad Larson is technically gifted with a track-record of programming. He has been working in the mobile app realm since the days of Palm. He is an engineer by training, earning a BS in chemical Engineering with a minor in Computer Science from Rose-Hulman Institute of Technology, in addition to an MS and PhD in Materials Science from the University of Wisconsin. His current interests in the mobile development space focus on advanced 3-D graphics and high performance image and video processing, particularly when used in scientific applications such as machine vision.
Current Position: Co-founder and Chief Technology Officer at SonoPlot, Inc., Consults and develops through Sunset Lake Software

App in Focus: Molecules
Federal Citizen App Program: NIH/NLM “Show off your Apps” Competition
Recognition: Honorable Mention

Description of the App: Molecules is an app that displays 3D renderings of molecules. It allows users to access and download various molecules from the RCSB Protein Data Bank or NCBI’s PubChem. Both are public repositories for molecules and compounds. Once a specified molecule or compound is downloaded, the app displays a 3D rendering and permits the user to manipulate their view; to zoom, pan or even to change from ball-and-stick to spacefilling visualization models. The molecules once downloaded are stored on the mobile device for later review.

Who is the App Intended to Serve: The original target audiences were researchers and high-end knowledge workers in the sciences. Now the primary users are in the education field, with usage even at the high school level.

Current Statistics: The application runs on iPhone, iPod touch and iPad. As of March 22, 2012, Molecules has been downloaded by 2,040,480 people.

Why was the app developed: Initial work on the application began on June 15, 2008 and the first version was submitted on July 6, 2008. It launched with the iPhone App Store on July 10, 2008. Molecules was created to solve a challenge identified by his brother, and one that if solved would benefit many researchers. It began and continues to be developed as a hobby project that focuses on 3D renderings of molecules and compounds. In addition to generating 3D renderings of specific molecules, the application was intended to provide mobile access to users anywhere.

How was the app developed: The development of Molecules began with the normal design process (defining the scope, etc.). With its development contingent upon Brad’s learning and mastery of particular skills, such as 3D rendering. The Apple Developer conferences played a critical role in not only teaching him some of these skills, but also in connecting him with others in the development community. Once able to 3D render, he partnered with RCSB Protein Bank and the NCBI’s PubChem to leverage their databases of molecules and compounds. Molecules makes its source code publicly available, allowing other developers to research and fix bugs/other issues. Complications and challenges in the development process were addressed via two avenues; his network of developers built through conferences and the online resource,

How was the availability of the app communicated to potential users: As one of the first 500 applications on the iPhone App Store, it was fairly visible from the launch of that service. It was the eighth most downloaded free utility in 2008. Since then, it has been featured by Apple multiple times on the App Store, and has appeared in passing on Apple's television commercials and in a couple of their keynote presentations. Other websites have listed it in collections of scientific and educational iOS applications. Beyond that, Brad has told friends and associates about the application, but have not actively advertised it beyond occasional mentions on various developer websites like Stack Overflow.

What were the key lessons learned during the development of the app: 1) Build small, throwaway test applications to explore specific areas that I didn't understand and then to use the lessons learned from these experiments in the larger finished application, and 2) Attend conferences for networking to build partnerships and working relationship with other developers.

What recommendations do you have for government agencies that are trying to incentivize the creation of citizen apps and the leveraging of open data programs:

  • Increase the visibility of the challenges and competitions. Many app developers do not know of them.
  • Improve communication with app developers and external parties. Leverage the app developer networks to increase the reach of the message.
  • Cash prizes while good need to be in touch with realities of developers. Seldom do developers have extensive resources to support travel to D.C for events (even when they win prizes or receive recognition).
  • Support the development of local events for app developers to meet, share ideas, learn from each other, and work on problems.

What do you plan on doing next with the Molecules app, and your interest in app development for tackling social and technical problems: Brad plans to continue the development and refinement of Molecules based upon the reviews he receives online. To-date 389 reviews have been written for all-versions of the app, referencing features such as color coding, interface layout and inclusion of more molecule information. His most recent efforts have been focused on adapting and improving the application for the iPad 3. He also, as noted above has begun work in ‘assisted visioning’ applications.

Citizen Apps to Solve Complex Urban Problems – Journal of Urban Technology

I have a paper accepted for publication in the Journal of Urban Technology. Co-authored with Akshay Bhagwatwar (Kelley School of Business, Indiana University) this paper looks at how citizen apps are employed to solve complex urban problems.


Tackling complex urban problems requires us to examine and leverage diverse sources of information. Today, cities of all kinds and sizes capture a large amount of information in real-time. Data is captured on transportation patterns, electricity and water consumption, citizen use of government services (e.g. parking meters), and even on weather events. Through open data initiatives, government agencies are making information available to citizens. In turn, citizens are building applications that exploit this information to solve local urban problems. Citizens are also building platforms where they can share information regarding government services. Information that was previously unavailable is now being used to gauge quality of services, choose services, and report illegal and unethical behaviors (e.g. requesting bribes). To the best of our knowledge, this is the first paper to examine the range of citizen applications (‘citizen apps’) targeted to solve urban issues and their ensuing impacts on planning, decision-making, problem solving, and urban governance. We examine citizen apps that address a wide range of urban issues from those that solve public transportation challenges to those advance management public utilities and services and even public safety.

Citation: Desouza, K.C., and Bhagwatwar, A. “Opening up Information for Tackling Complex Urban Problems:  A Study of Citizen Apps,” Journal of Urban Technology, Forthcoming.

IBM Center for the Business of Government Grant: Citizen Apps as a Democratizing Technology

I have received a grant from the IBM Center for the Business of Government for my research project, Citizen Apps as a Democratizing Technology:  Challenges and Opportunities for Federal Agencies. This project will be conducted as part of the policy informatics portfolio at the Metropolitan Institute.

Most US federal agencies have embraced President Obama's vision for 1) greater transparency, 2) increased citizen participation, and 3) greater collaboration. A critical outcome of these initiatives is the willingness of federal agencies to engage with citizens around open-data initiatives and the creation of technology for solving public policy problems - 'citizen apps.' We are witnessing an increasing proliferation of 'citizen apps', i.e. applications designed by citizens and developers to solve public policy challenges. Federal agencies are not only opening up data reservoirs, but are also incentivizing the development of citizen apps through competitions. In this research project, we propose to study citizen apps and the federal programs that fostered (incentivized) their creation.

There are many reasons why it is beneficial to involve citizens in the governance process. One, it opens up problem solving opportunities where citizens can participate. Second, it serves as a forum to increase the diversity of thought and knowledge brought to a problem. This increases the potential for innovation by engaging many minds to solve complex problems. Citizen participation leads to greater collective intelligence and hopefully more robust solutions for social issues. Third, it allows citizens to solve problems that a government agency might be challenged to address. Finally, it empowers the vision set forth by former President John F. Kennedy, "Ask not what your country can do for you - ask what you can do for your country." Citizen app programs normally come in two broad flavors. One set of citizen app programs are fueled by government open-data initiatives. In these cases, a government agency makes data available to the public and the public in turn responds by using this data creatively to generate technologies (the apps) that better the lives of citizens. The second set of citizen app programs is where a government agency issues a challenge or problem to the public. The public then responds by building solutions to the challenge. In this case, the government may incentivize the development of the apps through issue of recognition prizes and funding. This success of both types of citizen app programs depends on the dynamic collaboration of government agencies, app developers, and the citizenry. To date, our knowledge on what makes for successful collaboration among these three players is limited.

There are a number of design considerations that need to be addressed when building citizen app programs from the nature of incentives provided to goals of the apps, the motivations that drive citizens to create the apps, and how (and where) to deploy the apps, the involvement by the agency (e.g. staff time to interact with app developers), level and amount of data availability, and creation of problem-solving communities and forums, among others. In this research project, we will uncover design considerations that government executives need to bear in mind as they initiate citizen app programs. We will also compare and contrast citizen app programs to arrive at a set of best practices by looking at critical success factors that led to citizen app programs that were highly successful.

Our research project will thoroughly inventory and study the range of citizen apps to understand the typology of the apps, the data they use, the problems they address, the motivation of the designers, the usage by citizens, and the impact on government and governance. We propose to discover and define the inter-relations between the government agencies, the app developers, and the citizens. While our focus will be on studying citizen apps generated out of programs commissioned by the federal government, we will also look at programs started by progressive states (e.g. New York, California, etc).

The results of the final report will benefit public sector government executives, public managers, and the public-at-large in several ways: 1) it will enable government executives to avoid common pitfalls when incentivizing citizen app programs (for e.g. placing emphasis on the frontend, i.e. the creation of apps, and ignoring the more challenging aspect of ensuring that the apps are diffused into the agency's work practices or to citizens); 2) it will enable public managers to understand the landscape of citizen apps, the motivations of citizens who create them, and the factors that drive their usage; and 3) it will enable federal agencies to better engage citizens into the policy setting process through supporting technology development thereby increasing the chances of more effective solution generation for policy problems.