I had a wonderful time exchanging ideas with policy makers, researchers, practitioners, and even students at the NSF Workshop on Participatory Challenge Platforms with a Public Intent put on by the Center for Policy Informatics at Arizona State University. My formal remarks during the workshop drew on research results from our study of Challenge.gov. Since the workshop, I have heard from over 30 managers across the public, non-profit, and even private sectors for copies of the draft report. The feedback on the findings has been overwhelmingly positive. I hope to have a revised draft out for circulation by the end of the month.
See for a press release on the events in D.C. - "ASU Concludes White House Initiative in Nation's Capitol," ASU News, June 12, 2012.
I have just completed the first draft of my report on the Challenge.gov platform. This paper has been a few months in the making and builds on my recent work in community intelligence platforms, citizen apps, and innovation in the public sector. To receive a copy of the report, please send me an email.
Challenge.Gov: Landscape Analysis and Implications from the Citizen and Agency Perspective
To solve complex social and policy challenges we need to broaden the conversations, involve more minds and talent, and collaborate effectively and efficiently. Traditionally, public agencies have felt the burden to tackle challenges by relying on their own internal intellectual capital or through structured contracting with external partners. Seldom could an individual citizen share his or her talent, expertise, and skills with a public agency directly. Today, public agencies are becoming more participatory, inclusive, and transparent in how they engage with citizens as well as with each other. Challenge.gov is the crowdsourcing platform for US federal agencies that seek to engage citizens, leverage collective intelligence, and tackle complex social and technical challenges. In this paper we report on an exploratory landscape analysis of the competitions run on Challege.gov. We interviewed citizens who took part in competitions on Challenge.gov as well as public managers and government executives to understand their motivations, experiences, lessons learned, and future plans. Drawing on these interviews, we arrive at a set of actionable guidelines presented through implications to improve the state of competitions hosted by Challenge.gov.
Acknowledgments: This project was made possible through funding received from the IBM Center for the Business of Government. Tim Moon and Akshay Bhagwatwar served as research associates for the project. I am grateful to the assistance provided by Eric Park and Lauren Bulka during the project. I also thank all solution contributors to challenges and public managers who designed challenges that participated in our interviews. All errors and omissions are solely my responsibility. I acknowledge the thoughtful discussion and comments from participants at the NSF Workshop on Participatory Challenge Platforms with a Public Intent. The views represented in this paper are our own, and do not represent official positions of IBM, any of its affiliates, or the NSF.
Developer: Dean Jenkins
Bio: Dean holds an Executive MBA degree from the University of Washington. Prior to establishing PapayaHead, Inc. in 2006, Dean spent 14 years working as an Enterprise Software Program Manager at Intel Corporation. In addition to his work at PapayaHead, he serves on the Pastoral Staff at Mountain View Church in Tumwater, Washington.
App in Focus: PapayaHead
Federal Citizen App Program: Department of Agriculture’s “Apps for Healthy Kids”
Recognition: 2nd Place
Description of the app: PapayaHead is a family meal planning website and app that allows family members to fill out a unique and individual profile of food preferences. Logging things such as likes and dislikes, as well as allergies and other nutritional requirements. From these profiles, a family can build meal plans for the day which display the nutritional value of their meals and its impact on their profiles. In addition, the plans, recipes and shopping lists may be printed.
Who is the app intended to serve: Families and individuals looking to coordinate and plan their meals.
Why was the app developed: The initial motivation was for a website application to meet his own meal planning needs for his family. He always had the desire to be an entrepreneur and start his own company. Dean saw this as a business opportunity to do something he cares about and start a successful business.
The application was not developed for the contest. It was already under development. A registered dietician on their team heard of the contest and recommended they participate since they met the challenge’s criteria.
Examination of other apps: The team looked at what was available like Jenny Craig, they found that nobody was doing what they were specifically proposing to do. Other programs were doing bits and pieces, but not exactly what they were trying to build.
How was the app developed: They spent 2 years planning (benchmarking and functionality), and came up with the functional requirements that answered what they would want to be able to do with it. The PapayaHead team worked with an offshore development firm to build the web app, but due to communication issues and other complications, the app turned out to be more of a prototype. However, through the process they learned about things they hadn’t considered before. In addition, to technical roadblocks, they also had to pause development a couple times to fund raise, primarily from friends and families.
He launched a beta version in 2009 to gathered user feedback and addressed necessary changes. The app did a full launch in 2010. This app was a web-based application. They do have a derivative product that uses the main engine on their web app which was recently launched.
Communication of app availability: They started out using word of mouth to share their product. They sent emails to ask others to look at PapayaHead, to provide feedback, and to share it with others. They did some small Facebook ads, but did not spend much on a major marketing push and relied on word of mouth.
Issues of privacy: The best way to ensure the privacy and security of users is to limit the data they collect. All large organizations have breaches of security, so there really is no system that’s 100% safe from a breach. They just limit the data they collect in order to better protect their users. They do have system protections in place like firewalls. Within the application, rather than ask for sensitive things directly such as, “Do you have any heart diseases?” they would ask, “Do you want a healthy heart?”.
Realization of original goals: They would like to see more users on their site. They were hoping the site would go viral on its own, but unfortunately this hasn’t been the case. They have more features they want to add, but it’s a slow and gradual process.
Overall Challenge experience: In sum, Dean noted it was enjoyable to go to DC, but it would have been better to see the First Lady Michelle Obama there since she sponsored it. They were hoping to be able to get a picture with her it would have enhanced their experience.
Advice for federal agencies:
- After looking at other challenges, these challenges aren’t something an entrepreneur is able to take too seriously because the prizes and amount of effort going into most of these isn’t significant. Right now it seems more directed towards hobbyists and enthusiasts. It’s hard for someone who is looking to turn this into a serious business to take these challenges seriously.
- Some winners may want to take things further and turn their thing into a company. It would be more beneficial for the challenges to have the goal of helping their winners build companies that continue to tackle these problems if they so choose. It would be better putting together a larger prize because the current prize offerings for most challenges aren’t something you can gain much from. Even better, would be to provide connections and mentoring to build a business. Something similar to GE’s challenge.
- The only feedback they received from the challenge was being informed that they had won 2nd place in the challenge. People like Steve Wozniak and Mark Pincus were among the judges for the challenge, but they did not interact with them at all. It would have been extremely beneficial to receive feedback from them or, even better, have a chance to talk with them and others, such as venture capitalists.
Current Position: Team Leader, Program Management and Evaluation at the US Department of Transportation Intelligent Transportation Systems (ITS) Joint Program Office.
Bio: James is a classically trained civil engineer, receiving his Bachelors from Rensselaer Polytechnic Institute and a MS in Computer Systems Management from the University of Maryland. He has experience in both the public and private sector. Over the last decade, James has worked in the Federal Highway Administration as a program manager, with a particular focus of real-time information. In 2008, he began work with the US DOT and currently supervises the US DOT’s Intelligent Transportation System (ITS) research projects.
Federal App Challenge in Focus: The Connected Vehicle Technology Challenge is centered on collecting innovative ideas and uses for dedicated short range communications (DSRC); wireless technology enables vehicles to communicate. The challenge did not require technical submissions, and winners were awarded with a free trip to the Intelligent Transportation Systems World Congress. The challenge received a total of 76 submissions.
Motivation for initiating the challenge:
- In general: The Intelligent Transportation Systems Joint Program Office has an eye towards advancing technologies, and helping to generate forward progress in the transportation practice. The office focuses on getting newly developed technologies out of the research space and into deployment. The challenges help them to carry out their mission, which includes engaging a larger audience of stakeholders and connecting beyond the usual RFPs. James notes that often when one works in the contracting arena, one is limited to certain audiences. This lessens one’s ability to reach some of the newer and more innovative technologies.
- Challenge specific: ITS sought to broaden awareness of connected vehicles technology (establishing a network of communication between vehicles), and its role within the research field. Moreover, they targeted students, seeking to engage the next generation of engineers, economists and others interested in this arena. The ITS program realizes that these individuals are the ones coming into the workforce, therefore being in touch with this technology is essential.
Process for organizing the challenge:
They defined a challenge, targeted a specific group of participants and determined a prize. The prize was a paid trip to attend the World Congress on Intelligent Transportation Systems and provided exclusive access to technology demonstrations. A total of 3-4 Individuals were involved in defining the challenge, while an additional 6 provided further input. There were 10 panelists to review the challenge submissions upon their completion.
In terms of managing the challenge.gov platform, once a challenge is created and coordinated with GSA (the manager of challenge.gov), a moderator is established for the account. Both the moderator and James reviewed content and submissions on the site. In this challenge’s profile on the challenge.gov platform, some example submissions were provided. In addition, they also provided access to background information on connected vehicles to aid participants in understanding the underlying technologies.
They did borrow from existing programs, for things such as their judging criteria. However, the challenge was written by DOT’s ITS. The planning and design of the challenge was done prior to the establishment of the America Competes Law.
ITS developed a publicity plan to build awareness on the work the government was already doing with connected vehicles. The challenge was announced at a PR event at the annual Transportation Board Meeting. From there it immediately took off, and subsequently received 200 press impressions of all sorts; from blogs to webpages, to Wired magazine. One result of the immediate and extensive press coverage was the ability of ITS to refocus on other efforts and research. However, they did maintain support for the challenge, re-engaging the public at other events held by the agency.
Internal judges were volunteers from around the DOT and ITS. The department varies significantly, and ITS worked with 6 other agencies (Federal Highway Administration, National Highway Traffic and Safety Administration, Federal Transit Administration, Federal Motor Carrier Safety Administration, Research and Innovative Technology Administration, and Federal Railroad Administration) within the department. These internal judges then chose the five best submissions, while also eliminating irrelevant or unrelated proposals. In total 90 submissions were gathered, 75 submissions were made available for public voting. A submission from a team out of Clemson University won the people’s choice award. After the ITS World Congress, the agency has had minimal contact with the winners.
Overall, the response out paid their expectations. They would’ve been happy with 40 submissions, however they received close to 90 submissions. Challenge.gov allows people to ‘like’ and comment on the challenges, this is one way to gauge interest. The challenge was well within the top 10 of the challenges listed at the time for activity and submissions. The challenge out performed even those with cash-prizes.
In terms of content, the challenge was devised to gather new concepts. Overall, the submissions helped validate some of their research endeavors, such as electric vehicle fleets and autonomous vehicles. However, nothing specific from the challenge has been directly translated into their research activities.
- Spend time early-on developing and crafting easy to convey messages. Connected vehicles is an inherently complex research field; the solver community out there, even for smart people, requires a bit of effort to express what it takes to achieve this and submission expectations.
- Low barrier to entry for this challenge. As such, must be very specific as to what is expected for submissions.
- Require that it be an original thought. One concept was very closely aligned with another research project from around the country. Thus, better safeguards are needed against this type of submission.
Do you see challenges being important to other agencies:
In short, the answer is yes, but it is difficult to quantify. The Department of Transportation is rather committed to applying this type of initiative. In terms of, conducting research he sees it as evolving into a major component. Moreover, it is a useful method of achieving a knowledge/technology transfer, and a way to boost participation by individuals. Other agencies have contacted him on advice and he has been invited to judge other challenges.
Currently, there are three other agencies in the department defining challenges. He has been approached by agencies within DOT and colleagues in the ITS program with ideas for new challenges. James is additionally a part of a department-wide work group to provide guidance to agencies in conducting challenges.
Major risk/concern with running these challenges:
Major risk is making sure to abide by America COMPETES Act, whether or not the challenge is being conducted within what the law establishes agencies to do. A lot of discussion and negotiation with legal and procurement must exist during the design and implementation of a challenge. Challenges are new for agencies, and it will take time to gain awareness and a level of comfort. However, enthusiasm is building for it.
How clear is the America COMPETES Act:
Interpreting the law is still a work in progress and questions do remain. America COMPETES includes requirements such as, requiring judges from both agencies and the industry. As a result, vetting conflicts of interest becomes necessary. This adds a significant amount of time to just defining and selecting a judge. Efforts are still underway to figure out how early in the process to include the judging panel. As well as, how to compensate non-federal employees.
Volodymyr Lysenko and I have a paper accepted in Convergence: The International Journal of Research into New Media Technologies. Volodymyr is a former PhD student of mine that graduated from the Information School at the University of Washington. This paper draws on work he did while completing his dissertation. The paper is titled, Charting the co-Evolution of Cyberprotest and Counteraction: The Case of Former Soviet Union States from 1997-2011.
In this paper, we investigate the evolution of the modern information and communication technologies (ICTs) and the associated changes in protest-related tactics employed by two main stakeholders in the contemporary contentious political processes—dissenters and incumbent political authorities. Through in-depth investigation of the cyberprotest cases in the former Soviet states of Belarus, Moldova, Russia and Ukraine that occurred during the last decade, a coherent outline is developed of the co-evolution of ICTs-enabled protest tactics of the main counterparts in the contemporary political struggle in these countries. Particularly, it was found that there are at least three highly distinguishable levels of development of modern ICTs and the associated types of protest-related tactics employed by the main stakeholders in these events. We find that as soon as the authorities were able to effectively counteract the previous ICTs-enabled tactics by the dissenters, new developments in modern ICTs always empowered the latter to devise new effective strategies to overcome previously successful counter-revolutionary measures of the political authorities.
Reference: Lysenko, V.V., and Desouza, K.C. “Charting the co-Evolution of Cyberprotest and Counteraction: The Case of Former Soviet Union States from 1997-2011,” Convergence, Forthcoming.
Akshay Bhagwatwar (Kelley School of Business, Indiana University) and I have a paper accepted at the Eighteenth Americas Conference on Information Systems (AMCIS), to be held in Seattle, Washington, August 9-12, 2012. This paper builds on our ongoing work in policy informatics, citizen apps, and design of participatory platforms.
Community Intelligence Platforms: The Case of Open Government
The focus on collaborative and participatory governance has led to interest in studying how ‘intelligence’ in citizen communities can be leveraged towards creating robust solutions for complex social and policy problems. In this paper, we present four models that uncover the process of leveraging community intelligence. We analyze multiple case studies that capture the varying roles of citizens and public agencies in the problem-solving process. Employing Arnstein’s (1969) ladder of citizen participation as an analytical tool, we outline the strengths and weaknesses of each model, and suggest design recommendations for the development of participatory platforms for open government.
Since I was busy in Lisbon and Guimarães, I could not attend the 2012 Ridenour Faculty Fellowship Conference. Given that I was supposed to be on a panel discussing forms of resilience, I was interviewed before I left for Portugal. The interview was conducted by Maggie Cowell, an assistant professor of Urban Affairs and Planning in the School of Public and International Affairs, Virginia Tech.
Developer: John Schimmel
Bio: John is currently an adjunct faculty member at the NYU Interactive Telecommunications Program. He focuses on teaching courses on assistive technology design for the disabled and web programming. His background is in tech hacking, with a focus on web development (code/design/engineer).
App in Focus: Access Together
Federal Citizen App Program: FCC and the Knight Foundation Apps for Communities
Recognition: Runner Up and Most Replicable
Description of the App: Users open Access Together via their mobile phone’s web browser to check-in to places, and are prompted to answer a series of accessibility questions. The information provided on the app is primarily crowd-sourced, and provides individuals with accessibility restrictions with invaluable information.
Who is the App Intended to Serve: Citizens seeking to provide and view accessibility information about their community.
Why was the app developed: Access Together started in Spring 2011. The app idea was sparked at the New York Hall of Science during a discussion about human abilities and accessibility information in the city. Simultaneously, Foursquare released their location API, allowing individuals to query their system for information (venues). The timing was ideal to integrate accessibility needs because of the ability of the new API. John learned of challenge.gov through Twitter, and was interested in it due to his development background. The combination of accessibility and needs information along with the API could contribute additional, useful data as well as pave the way for future challenges. John thought, “Why not just build it?” so he started app development with a friend (designer).
The app challenge did provide motivation, but it was seen as more of an opportunity to present their app to an audience. The challenge did not start the development of the app, but helped motivate them to get the app project completed for the challenge deadline.
How was the availability of the app communicated to potential users: John shares information about his app to networks accessible to him such as his Facebook network. He also tries to directly contact the organizations he thinks will benefit from his app, and those he would also be interested in working with.
Advice for other developers/Lessons Learned: Using the data available from the government is the motivation behind the challenge. However, developers should not just focus on the provided data, but look towards integrating the data with the community involved and making it engaging. Don’t just make a client to access the data, but make it so that people can engage and interact with it. That was the appeal of this challenge. The only real challenge they faced was their data, as they created it from scratch. They had to create some accessibility data initially to make the app presentable and usable.
Issues of Privacy: Only issue was people with an “older mindset.” Individuals log in using a Google account or Foursquare account, but from there the user may to choose to stay anonymous, use their first name, or use a display name. This way the user’s identification is protected. Some people like to gain recognition by associating a name, but some people don’t want to be known, particularly when giving a negative review. Few concerns were raised regarding privacy so it didn’t need to be addressed.
What recommendations do you have for government agencies that are trying to incentivize the creation of citizen apps and the leveraging of open data programs:
- Having a monetary prize is good, but not sufficient. Provide a way for developers to create closer relationships with the organizations and other pertinent stakeholders. The idea would be to help developers find more resources to continue the project, or to potentially find someone to sustain/hand the project off to if the developer needs to move on to something. In place of money, you could find government organizations that might have some need or interest in the applications. If they want to have government involved, then try to build it into a sustainable program with people involved who keep the ball rolling instead of making it a quick hack competition with a cash prize. If the people involved aren’t pushing it then it won’t go anywhere.
- Increased advertising. There were a lot of false endings when the competition was supposed to end in July, but was pushed back to November due to lack of submissions. They could’ve pushed a little harder in terms of marketing by going to their target people like posting on Hacker News.
After the competition: The app got picked up by a few news sites focused on disability and accessibility, but during that time it didn’t have all the features it does now. John tries to tweet the FCC from time to time wondering if they’ll engage with Access Together more, but no response so far. He has also pursued engaging the city government, but he needs to create a 501c3 (non-profit) for this. However, he would rather be a sustainable business corporation. For now, he uses his networks on Facebook and direct contact.
What do you plan on doing next with the Access Together app, and your interest in app development for tackling social and technical problems: Used the prize money to spend 2 months to further enhance the app. The challenge now is to build an audience, adding new features to the mobile and desktop version, and making it more useful. He didn’t want to walk away when the project was over since he believes this can become something more. John has been in contact with 2 different organizations with similar apps. One organization is in Berlin, Germany called Wheelmap.org, and the other is AccessMap. He plans to meet with them to figure out how to combine their data into one accessible system because they share a common goal and the data is bigger than the individual apps.
Our second profile of developers who have contributed apps to federal challenges. The first profile was on Brad Larson.
Bio: Curtis Chang is the CEO and founder of Consulting Within Reach (CWR), a firm that works predominately with nonprofits and government agencies. He maintains expertise in the realm of social entrepreneurship and innovation, writing for both the Skoll Foundation’s Social Edge and Stanford Social Innovation Review.
App in Focus: Homeless SCC
Federal Citizen App Program: FCC and Knight Foundation Apps for Communities Challenge
Recognition: 2nd Prize
Description of the App: Frontline agency staff members during the referral process enter the profile of a client which then matches them to a provider that best serves their needs. The application shares useful information on how to reach service providers, up to date services available, while also tracking all referrals made. Ultimately, it streamlines the process for homeless referrals, and captures useful information for service providers and government agencies to better collaborate.
Who is the App Intended to Serve: The target audience for the Homeless SCC app is frontline staff assisting the homeless, as well as the government and philanthropic agencies.
Why was the app developed: The motivation for the app stemmed from the widespread recognition that many of the most challenging social issues are complex, constantly changing, and often have duplication. The organizations serving the County’s homeless had little data compiled, and furthermore had little idea what others were doing. The app emerged as a response to the understanding that the public sectors and these organizations need a rational way to better collaborate; having data is essential to planning and collaborating effectively. CWR developed the app with funding from private sources with an interest in homelessness. CWR is a private entity, therefore although they conduct work seeking to improve and address challenges in society, they must also think about how to cover development costs.
How was the availability of the app communicated to potential users: CWR worked with the County, who endorsed the application but did not mandate its usage. There was no real incentive for usage. To these ends, the consulting firm produced and conducted several presentations, individual reach-outs/visits, and mobilized a ‘sales team’. This was gauged as partially successful.
What were the key lessons learned during the development of the app: 1) There is a need for someone (e.g. a foundation, influencer, etc) or public agency with a big stick or a big carrot, or even better both with the authority to mandate and build urgency to ultimately spur usage of the application. 2) The amount of work it took to publicize and market the app is not seen as the best method moving forward. The consulting firm itself ate the cost of these efforts. It is not a sustainable process to have the developers themselves hitting the pavement. 3) The technology was the easiest part, building a coalition and getting the app in the hands of users is challenging. Public agencies should provide support and access to their networks for diffusing apps into communities. 4) Public agencies should do more to stay involved with the app developers after competitions so as to learn from their efforts, support them, and work on future iterations of the app.
What recommendations do you have for government agencies that are trying to incentivize the creation of citizen apps and the leveraging of open data programs:
- Cash-prizes very important. Usually these apps have been developed out of some cause-motivation or low-budget gig. Nice to have development costs covered by prize winnings.
- Follow-up motivation and commitment on the part of federal agencies to publicize and market these apps.
- Federal programs need to market the apps submitted, as app developers are looking at this as a viable marketing platform.
- Don’t put property rights on the apps developed for the competition. Too much burden.
- Public agencies who collaborate foundations to develop challenges is seen as attractive.
- As a consulting firm, they target competitions where a suitable app is already developed. Developing an app for a particular competition is pretty risky.
What do you plan on doing next with the Homeless SCC app, and your interest in app development for tackling social and technical problems: The Homeless SCC app model has already been replicated in Northern California and its outdoor youth programs. In this case, organizations and programs are mandated to use the app. In addition, Santa Clara County has expressed interest in developing a similar application for networking local church services for recently released prisoners.