*This piece was reposted from the Stanford Social Innovation Review on October 22, 2024. You can access the original posting here. 

Asking questions in new and participatory ways can complement advancements in data science and AI while enabling more inclusive and more adaptive democracies.

Image by AlexSecret from iStock

If I were given one hour to save the planet, I would spend 55 minutes defining the question and 5 minutes resolving it.—Albert Einstein

Recent years have witnessed an undeniable and dispiriting trend of global democratic backsliding. An erosion of trust in democratic institutions, the acceleration of social fragmentation and inequity, and the collapse of public curiosity in public discussions all contribute to the sense that we are living through what journalist Anne Applebaum has labeled “the twilight of democracy.” Shifts in how we connect with each other, particularly within our digital ecosystems, are at least partly responsible for driving the crisis of democracy, including the rise of misinformation and the decline of social capital.

Yet a crisis, as the saying goes, always contains kernels of opportunity. Buried within our current dilemma—indeed, within one of the underlying causes of it—is a potential solution. Democracies are resilient and adaptive, not static. And importantly, data and artificial intelligence (AI), if implemented responsibly, can contribute to making them more resilient. Technologies such as AI-supported digital public squares and crowd-sourcing are examples of how generative AI and large language models can improve community connectivity, societal health, and public services. Communities can leverage these tools for democratic participation and democratizing information. Through this period of technological transition, policy makers and communities are imagining how digital technologies can better engage our collective intelligence.

Asking Better Questions

Creating diverse communities across disciplines and industries—including academia, technology, data science, and the arts—requires that we organize around and sustain shared curiosity. Achieving this requires new tools and approaches, specifically the collective process of asking better questions.

Formulated inclusively, questions help establish shared priorities and impart focus, efficiency, and equity to public policy. For instance, school systems can identify indicators and patterns of experiences, such as low attendance rates, that signal a student is at risk of not completing school. However, they rarely ask the positive outlier question of what enables some at-risk students to overcome challenges and finish school. Is it a good teacher relationship, an after-school program, the support of a family member, or a combination of these and other factors? Asking outlier (and orphan, or overlooked and neglected) questions can help refocus programs and guide policies toward areas with the highest potential for impact.

Not asking the right questions can also have adverse effects. For example, many city governments have not asked whether and how people of different genders, in different age groups, or with different physical mobility needs experience local public transportation systems. Creating the necessary infrastructure for people with a variety of needs to travel safely and efficiently increases health and well-being. Questions like whether sidewalks are big enough for strollers and whether there is sufficient public transport near schools can help spotlight areas for improvement, and show where age- or gender-disaggregated data is needed most.

How we ask questions is also important, and the disintegration of NASA’s Space Shuttle Challenger—one of the most well-known space exploration disasters in history—offers a powerful example. The evening before the shuttle was set to launch, managers and engineers from NASA and Morton Thiokol, a company that made a segment of the shuttle, held a meeting to assess whether the shuttle would be safe to launch the next day and whether a specific part would work properly in unusually cold weather. During this meeting, the managers asked the engineers to demonstrate that it was not safe to launch, while previously they asked them to show it was safe during their safety review. As a result, the engineers could not provide sufficient evidence that the part in question would not operate properly in colder weather and to postpone the launch. While many other factors contributed to this failure, it demonstrates the enormous impact that reframing questions can have.

Much of the recent work at The Governance Laboratory (The GovLab), an action research center, has focused on initiating “a new science of questions” that is inclusive, iterative, and participatory. Emphasizing the “quest” of questions, The GovLab, together with the Human Technology Institute (a human-centered technology and policy institute) and others, is exploring how to incorporate adaptive questions into continuous learning and evaluation systems for communities, policy makers, and philanthropy. At the heart of this work is our belief that today's public policy development is held back by the way we ask questions and set priorities. By improving the way we question, we can create more effective and positive policymaking for democratic institutions and beyond.

Indeed, the questions we ask matter as much—possibly more—than the answers we find. They determine the focus, scope, and type of information we seek and help ensure that the data we collect is relevant, precise, actionable, and inclusive. What we ask affects what we measure. By focusing on the quality and depth of our inquiries, we can build a more robust foundation for thoughtful and effective policymaking.

A New Science of Questions for Democratic Resilience

There is considerable evidence that a process of asking questions has the potential to strengthen democracies in several ways. Iterative and collective questions inform dynamic institutions and establish sound first principles, and help organizations avoid racing ahead based on superficial—and often biased—assumptions. Better questions can help fuel thoughtful conversations, shape public debate, and overcome polarization. They shape what we measure to hold elected and public officials accountable. They can improve collective decision-making, explore what we don’t know to expand our beliefs, mobilize communities to make change, and advance shared agendas across domains and geographies. Better questions can also reduce deeply embedded biases by reframing problems within public policymaking and ensure that solutions truly address community needs. Ultimately, they help guide greater collective intelligence.

Despite these benefits, researchers and others have paid relatively little attention to formulating and prioritizing questions. Too often, public policy makers are constrained by existing paradigms and limited access to diverse perspectives. They may rely on outdated data, face pressure from special interest groups, and lack the tools or frameworks to challenge long-standing assumptions. This often results in a narrow scope of inquiry, reinforcing status quo solutions rather than innovative or holistic approaches that could better serve the public interest. The paucity of question-making also discourages creativity in policymaking, and leads to misallocation and inefficient use of financial, human, and technical resources.

The untapped potential of questions is all the more striking—and unfortunate—given the possibilities technical innovation offers. These advances provide opportunities to ask (and answer) questions in previously unimaginable ways. AI can analyze vast datasets to uncover hidden patterns and insights, enabling more targeted decision-making. Sophisticated analytical tools can help policy makers simulate scenarios and predict outcomes, leading to more proactive and effective governance. Moreover, the ability to crowdsource questions and solutions from a diverse and global population can democratize the policymaking process, ensuring that it reflects a broader range of experiences and needs. Without a more deliberate and strategic focus on the science of questioning, society may never fully leverage these technological capabilities—indeed, as noted, they may contribute to public harm rather than benefit.

For all these reasons, The GovLab’s new science of questions aims to guide policymaking to complement and steer advances in data science and AI. This new science uses a range of participatory approaches to solicit, evaluate, and prioritize shared questions with different stakeholders. It harnesses crowdsourcing, co-creation, and citizen science to incorporate the unique perspectives of those closest to society’s most pressing problems, and it strives to do this in a way that is both inclusive and democratic. These structured and semi-structured approaches to questions (and problem-solving) allow policy makers and stakeholders to tap into society’s collective intelligence and are important to reinvigorating democracy.

The Five Stages of a New Science of Questions

Generative collective intelligence can model the adoption of these new, inquiry-based approaches. While inquiry methods have been around for centuries, we are still in the early stages of adapting methodologies for modern institutions and new forums of public engagement. While policymaking cycles are rarely neat and linear in sequence, our work so far suggests that “questions science” has five main stages:

1. Pre-questioning: A systematic and inclusive approach to questions involves first exploring and framing the area of interest, providing a baseline understanding of an issue area to guide the question formulation process. One tool for this is topic mapping, which assists in rapidly assessing the scope of a complex problem, summarizing the most important or relevant information for the context at hand, and creating a visual map similar to a mind map or flowchart that illustrates the hierarchy of information. Researchers can also use living evidence systems to summarize what is known and not yet known about topic areas. These and other tools help broaden the potential question areas, frame potential connections, and provide context for question generation.

In partnership with UNICEF and the University of Edinburgh’s Futures Institute’s Data for Children Collaborative, The GovLab developed a topic map that helped identify how family dynamics, community practices, broader systems, and other factors are impacting adolescent mental health. Government, research, and other organizations can now use this map to guide new data initiatives. For instance, the topic map showed that many adolescents do not receive the care they need. Decision makers therefore might do well to ask how lowering the cost of care could impact the use of services, or how technologies like machine learning and sensors could help better identify those in need of care. A similar effort helped a private foundation and several community organizations understand topics surrounding justice reinvestment. Mapping the individual, societal, and systemic factors affecting people who have experienced incarceration helped decision makers determine the most promising areas for further research and investment.

Elsewhere, the Democracy Fund has advocated for the use of systems thinking to open up areas for questioning, and the nonprofit research organization More in Common has explored the use of cluster analysis to understand the factors impacting individuals’ perspectives. Other examples include the development of learning programs that promote constructive conversations by the Institute of Socratic Dialogue (a foundation), and in Australia, The Conversation published a series of articles on breaking the cycle of disadvantage for children that informed policy-relevant questions.

This taxonomy outlines four types of questions that data-driven approaches can address. Image by GovLab.

2. Participatory questioning: The next stage involves sourcing questions equitably by engaging people who have both lived experience and subject expertise. An inclusive process uses participatory approaches to source, formulate, and cluster questions. It both strengthens collective intelligence and reinforces democratic principles.

Several initiatives offer good models for how to source a diverse range of questions. The Right Questions Institute, an education nonprofit, is implementing its own question design method across several disciplines and communities, and the Siegel Family Endowment has deployed a process called “inquiry-driven grantmaking” that focuses on formulating questions that underpin its funding decisions. Another model is The GovLab’s 100 Questions Initiative, which is working to generate new questions and reimagine the questioning process across 10 domains, including migration, gender, and air quality. This involves asking interdisciplinary panels of experts with both subject and data expertise to draft and prioritize questions they feel are the most important for their assigned domain.

Another consideration is the taxonomy of questions. So far, The GovLab has identified four types of data-actionable questions: descriptive questions that seek to understand the state of a situation and what happened in the past, diagnostic questions that unpack why those situations happened, predictive questions that inquire about the future and seek to identify what will happen, and prescriptive questions that focus on what should be done. Using a taxonomy helps focus the question formulation process and ensure that the questions developed are feasible to address using data. It can also help demonstrate the hierarchy of questions and the importance of understanding the baseline ahead of asking diagnostic, predictive, and prescriptive questions.

3. Post-questioning: In the post-questioning stage, we seek to determine which questions, if answered, could have the greatest impact. Not all questions are equal—some hold the potential to drive significant change, while others may be less actionable or relevant. Establishing clear criteria helps us home in on the questions that are most likely to lead to meaningful solutions. These criteria might include factors like potential for high impact, novelty, and how likely the question is to be solved using data or technology.

Our participatory process or prioritization, using techniques inspired by the Delphi method, allows us to spend resources—time, money, and effort—on the questions that could bring about the highest return in terms of societal benefit.

4. Answering: The science we propose is focused on questions, but it also includes approaches to answering the most important questions once they are identified. This is where policymaking usually begins. In particular, a new science of questions would focus on using innovative and non-traditional data types and sources, often in combination.

Illustrating how new data methods and data collaborations can strengthen governance, during the COVID-19 pandemic, policy makers and disease specialists around the world sought to use non-traditional data to help understand the spread of the virus. Data such as mobility patterns, economic transactions, and even wastewater analysis helped governments and organizations rapidly track the spread of the virus, monitor public adherence to restrictions, and target health interventions in real time. Mobility data from smartphones, for example, allowed authorities to assess the effectiveness of lockdowns, while wastewater surveillance provided an early warning system for outbreaks in some regions. These non-traditional data sources filled gaps where traditional health data was limited, offering a broader and more dynamic view of the pandemic's spread and the effectiveness of mitigation measures. This in turn influenced decisions about testing, resource allocation, and public health communications.​

One challenge is that non-traditional data sources are often privately held, and thus gaining access to it often requires new partnerships. Data collaboratives are an emergent form of public-private partnership that help break down data silos and repurpose data already collected for a secondary purpose in the public’s interest. They have been used productively in a number of cases to strengthen public policymaking. For example, in Seoul, a Korean telecom company gave the Data and Statistics Division access to mobile call detail records to help better align the city’s bus routes with the way women and low-income groups travel at night.

5. Feedback and adjustment: A science of questions does not end with answers; it is a quest for questioning, a constantly updated and refined process based on how decision makers answer questions, changing circumstances, and available research. Organizations can use a number of methods to facilitate this.

One example for where the process can be seen comes from the initiative Thrive: Finishing School Well in Australia. Thrive combines adaptive Bayesian reasoning (advanced statistical methods) with community insights. The program facilitates collective intelligence and adaptive decision-making through dynamic feedback loops, where schools, families, and government departments collaboratively pose questions, gather diverse data, test outcomes of programs, and assess what is or isn’t working to pose the next cycle of questions. This continuous process updates prior beliefs based on new evidence from analysis and programs. In this case, state education department officials and several schools partnered with analysts and philanthropy to ask: What factors help young people overcome barriers to complete school? After posting this question and running an analysis on school data, questions focused on attendance. Then, after testing youth and community beliefs against further analysis, the question changed to focus on individuals’ sense of belonging and relationships in school. From there, the questions evolved to explore which types of relationships—peer or teacher—are most meaningful, and exploration that may lead to a trial of targeted mentoring programs.

Thrive provides an example of how government and civil society can collectively accelerate the discovery, hypotheses testing, question adaptation, and program trials in iterative stages, rather than waiting for top-down evaluations at the end of a philanthropic or government program.

What’s Next: Toward a New Science of Questions

Once the most important questions—and perhaps some answers—have been established, policy makers and other decision makers must turn to the task of implementing their newfound insights.

Going forward, three main areas of question science merit further investment and research. First, we must refine and increase the robustness of the science of questions methodology by identifying what participatory theories or approaches work best, and how to integrate them into decision-making systems. This helps move away from static topic mapping toward the use of generative questions to reinforce collective decision intelligence. Tapping into data-driven methods and tools may prove useful in this regard, depending on different circumstances and domains.

At the same time, we might consider what happens if our questions do not evolve—if we are asking the same questions in one, 10, or 25 years. By seeing where we are not progressing, we can identify which aspects of questions science are most in need of innovation and which have the greatest potential to strengthen democratic processes and institutions. Putting this approach into practice also requires new, inquiry-driven narratives and models to crystalize the purpose, processes, and potential of question science.

Second, making questions a fundamental part of our policy process and public conversations contributes to moving from data to dialogue but doing so requires a cultural shift that emphasizes curiosity, a willingness to embrace complexity, and the inevitable reality of tradeoffs in public policymaking. We are mindful of the challenging nature of achieving this shift in today’s polarized atmosphere. Cultivating a culture of questioning is a multi-step process that may include learning programs and educational outreach about the importance of democracy, more research into the relationship between questions and democratic resilience, and cross-sectoral convenings that embed participatory approaches to questioning. Philanthropists are uniquely positioned to work between sectors to advance this cultural shift.

Finally, we must focus on building capacity and accountability for this new science through dedicated institutions and sustainable funding. For example, an institute for questions could bring people together to apply new tools to questions in the context of resilient democracies. This new institute could help scale up questions science initiatives while establishing processes, policies, and infrastructure that ensure accountability to addressing questions.

To conclude, the future of the science of questions hinges on our ability to embed curiosity and inquiry into the very fabric of decision-making. By fostering a culture that values evolving questions and establishing institutions that support this work, we can ensure that policymaking remains dynamic, inclusive, and resilient. The journey toward a more thoughtful, resilient, and engaged democracy starts with asking better questions and ensuring that those questions drive meaningful change.