Why is it important for citizens to participate in decision-making?

Abstract

Big Data promises benefits for society as well as business. Do policy makers know how best to use this scale of data driven decision-making in an effective way for citizens? Citizen participation is portrayed in literature as a key component in policy decision-making. Yet, this decision-making process to date is often driven by other stakeholders such as scientific experts, academic institutions, national and international governing bodies, to name but a few. Furthermore, there is scant literature on the best way to create policy for new technology, taking into consideration the voice of the citizen. The prevailing question, therefore, is what extent does citizen participation in decision-making make a difference to shaping policy for technology? Our paper explores an experimental method for citizens to make a difference to European policy decision-making on the future of technology and the impact on society. Employing a case study of Irish citizens as part of the CIMULACT (Citizen and Multi-Actor Consultation on Horizon 2020) project this paper reports a new methodology for gathering citizens’ perspectives on future decision-making policies on technology. The findings reveal key advantages and disadvantages to this methodology. This paper makes a number of contributions to both the academic and practitioner communities.

It’s the action, not the fruit of the action, that's important. You have to do the right thing. It may not be in your power, may not be in your time, that there'll be any fruit. But that doesn't mean you stop doing the right thing. You may never know what results come from your action. But if you do nothing, there will be no result. ― Mahatma Gandhi

When it comes to decision-making, there are many ways to characterise a decision. One such way is to reflect on the scale of the decision and the implications (March & Olsen, 1976). The future of society is one of the biggest problems imaginable when you consider the implications of any relatable decision. We explore the activity of star-gazing into the future and question the value of citizen’s articulating their hopes and dreams with the objective to shape the future science and technology research agenda of Europe. Can citizens’ visions uncover legitimate needs? What role do they play in isolation and in conjunction with experts and stakeholders? Can the visions of citizens be translated into a viable basis for decision-making? Locating crime spots, or the next outbreak of a contagious disease, Big Data promises benefits for society as well as business. Do policy makers know how to best use this scale of data driven decision-making in an effective way for citizens?

This research is grounded in the emerging theoretical underpinnings of responsible research and innovation (RRI) (Stilgoe, Owen, & Macnaghten, 2013). RRI purports the need for critical reflection on the relationship between science and society as we enter the era of ‘solutionism, where the tendency to design technology without thinking deeply about the problems is upon us (Morozov, 2013). As science and technology advance at unpatrolled and unparalleled rates, proponents of responsible research and innovation argue now more than ever, social awareness and responsibility need to be addressed coupled with scientific excellence (Owen, Macnaghten, & Stilgoe, 2012; Von Schouberg, 2011). Here citizens are recognised to have an increasing role to play. However with many players dominating the space namely scientific experts, academic institutions, national and European governing bodies, industry protagonists and fledging start-ups, the prevailing question is to what extent citizen participation in decision-making can make a difference? (Lessig, 2001; Tapscott & Williams, 2006). Our paper explores an experimental method for citizens to make a difference to European policy decision-making on the future of society. This is of particular pertinence as policy makers begin to grapple with the societal considerations of Big Data. For example, the advancements in Big Data have changed the context of privacy and governments are beginning to look into the challenge as exemplified by the UN project Global Pulse.1

The research presented in this paper is part of the CIMULACT (Citizen and Multi- Actor Consultation on Horizon 2020) project,2 a Horizon 2020 (H2020) funded project with the aim of improving the methodological basis for engaging citizens, stakeholders and researchers in the processes that are defining the nature and objects of European Research and Innovation. The outputs of this project are twofold. Firstly, the project aims to provide concrete, timely and directly useable input to the formation of the Horizon 2020 agendas for 2017 and 2018, and subsequent EU framework programmes going forward. The CIMULACT project builds on the conviction that the collective intelligence of society gives Europe a competitive advantage, which may be activated to strengthen the relevance of the European science and technology system. Secondly, it is envisioned that the project will generate debate about desirable, sustainable and responsible research and innovation futures, and provide recommendations and input for the H2020 work programmes and other relevant EU and national STI policy initiatives.

The paper proceeds as follows. Firstly we discuss the extant theoretical perspectives on the citizen’s role in decision making, specifically addressing responsible research and innovation. Then a new data collection methodology, replicated in 30 different European countries, for gathering citizens’ perspectives is described. The findings from an Irish perspective are subsequently presented. Concluding this paper is a discussion of the findings where by the implications for the decision making scholarly community (i.e. the policy makers tasked with making decision on the future of society and for the citizens themselves) are outlined.

Science and Technology Policy is a much studied area for scholars, as countries grapple with new innovative technologies and question their impact (Lengwiler, 2008; Warnke & Heimeriks, 2008). Governments are seeking new methods to scientifically assess societal, ethical, legal and economic aspects of science and technology. However, Delvenne, Fallon, and Brunet (2011) argues uncertainty is no longer contained within modern structures of policy-making. Specifically, they argue the current challenge decision-making at policy level is to best accommodate to the uncertainty and dynamics of patterns to offer the decision- making process “a context-determined and temporally limited orientation for action that makes learning through experience possible’ (p. 18). Furthermore, no discussion on decision-making is meaningful without a discussion on bounded rationality (March & Simon, 1958). The authors argue under conditions of bounded-rationality, decision makers seemingly ‘do what they can’ or in some cases, ‘make-do’. Ironically, we spent much of the last century perfecting decision support systems to get useful information to a centralised place to that experts could make decisions. Much of this effort was misplaced. We ought to have been finding a way to collect the dispersed knowledge of individuals so that the aggregate insights of large groups could be harnessed (Tapscott & Williams, 2006). To further complicate matters, when contemplating boundaries in science and technology policy decision-making, traditional boundaries are no longer imposed, but constructed, bargained, negotiated and appropriated by stakeholders (Delvenne et al., 2011). We speculate such boundary-less domains call into question who are the stakeholders and what role can they play in such fluid environments. More specifically, what extent a citizen perspective can influence the decision-making environment of science and technology policy.

This study is situated against the backdrop of ‘responsible research and innovation’, a growing scholarly appreciation that the advancements of science and technology should be situated within a societal context focused on the future consciousness of societal well- being. Responsible Research and Innovation is recognised to be a dynamic concept enacted at multiple levels and is forecasted to feature on the political agenda in the coming years (Stilgoe et al., 2013). The term Responsible Research and Innovation is defined as ‘taking care of the future through collective stewardship of science and innovation in the present’ (Stilgoe et al., 2013, p. 3). The mechanism of feedback loops between different societal groups is articulated by Owen et al. (2012) as one approach towards addressing the challenge of making the research and innovation process responsive to society in terms of agenda setting. Furthermore, Warnke and Heimeriks (2008, p. 74) call the support of a ‘continuous policy learning process’ that is not predetermined but open to foster the development of a system which may cope with future uncertainties. Failing to consider this plurality of legitimate perspectives in the process of shaping future science and technology policies may exclude expectations of citizens in these areas.

Furthermore, there is the growing pressure on the scientific community to be accountable to society (Jasanoff, 2012). The inclusion of citizens demonstrates an appreciation of the value of multiple perspectives, interests and types of knowledge.

Legitimacy is especially important if decision-makers are to be supported in issues, which have implications for a wide variety of actors (Abels, 2007). The question of what makes one stakeholder more salient than others is a central concern of stakeholder theory research (Mitchell, Agle, & Wood, 1997). Stakeholders are actors- individuals, groups, agencies or organisations- who may lose or gain from the activities of an organisation. As such, they have an interest or ‘stake’ in the organisation’s performance. As a stakeholder, Irvin and Stansbury (Irvin & Stansbury, 2004) argue the increased participation from community in government decision-making produces important benefits in terms of visioning and foresight.

Therefore from our review of the extant literature, it can be argued that when considering the future of technology and its impact on society it is important for policy makers and scientists to consider the citizen to increase the legitimacy of their efforts. CIMULACT methodology aims to bring the citizen to the table to discuss their visions for the future of society and to take their voice into consideration for the future societal impact of science and technology. The next section outlines the method used in this study.

The research methodology is an innovative approach developed by the CIMULACT project to engage citizens in the decision-making process of the European Commission. The method was designed with the principles of responsible innovation (Stilgoe et al., 2013). An overview of the CIMULAT method is presented in Figure 1.

This paper presents the case study of the Irish National Citizen Vision Workshop, held on Friday December 4th 2015. The workshop was hosted by University College Cork. Similar workshops are being held throughout 2015 and 2016 in 30 different European countries with the same method replicated on each site. At the time of writing, results from a handful of countries had yet to be finalised, thus why an Irish case study is solely discussed herein.

All recipients were asked to register their interest in attending the citizen’s consultation through the event management website www.eventbrite.com. The site was live from 8th October to the day of the event. This channel was used to help centrally manage all received applications. In addition, the registration form included a number of questions which were used to help ensure diversity among confirmed attendants at the workshop i.e. based on age, gender, education, employment, and their personal motivation for attending.

Meanwhile, email was chosen as the primary communication medium to reach the target demographics, with four mailing lists utilised to cover the different age groups, and professional / educational backgrounds obtained from various departments within UCC. Social media sites such as Twitter and Facebook were used to reach younger demographics, while LinkedIn was used to target professionals working in the public and private sector. The response from citizens was further heightened by engagement with the national press. In total 164 citizens registered their interest in attending the workshop. From this number 45 citizens were invited to attend (based on the inclusion criteria for the workshop), with a waiting list of 25 additional people. Yet, despite the very keen interest expressed by citizens, 25 participants participated in person on the day. Outside of this, one person was also involved in the workshop vis-à-vis Twitter, providing his views throughout the day.

As part of this workshop, citizens were invited to share their hopes and dreams for the future of Europe. To help citizens establish such hopes and dreams and to stimulate group conversations visual imagery was utilised. A wide array of images were made available to each group. Figure 2 presents an example of the imagery used.

Noteworthy, images were used as ‘aids’ for conversation rather than a ‘guide’ from organisers as to how they wanted to ‘shape’ conversation. Five facilitators were present on the day for individual groups and every citizen had an equal opportunity to express their opinions. Facilitators were not permitted to express their own opinion to the group nor capture any research notes throughout the process. However the facilitators were asked to reflect on the process as part of the data collection for the paper. As a number of deliverables (i.e. storyboards and visions), from the citizens involved were required as part of the workshop power point presentations describing the purpose and requirements of each deliverable were utilised at various stages of the day. In CIMULACT the term vision is employed according to the following definition:

‘Vision [...] is a picture or an imagination of a desirable future. A vision can be based upon hopes and dreams – but also upon concerns and fears in relation to problems or imagined threats, which we do not want to become future reality. The time span of the vision is 30–40 years from now’

(Rask & Damianova, 2009, p. 9). The method allows citizens to include their expectations of the future, their thoughts and concerns about it into the visions (experiential and normative knowledge) though a vision template. To ensure this was achieved the project designed a methodological framework combining horizon scanning, visioning and joint exploration of what-if questions through scenario development and finally multi-criteria analysis (Van der Helm, 2014). Throughout the process extensive use of visual elements to bridge domains and elicit tacit knowledge and out of the box thinking such as the vision template were utilised.

The following section presents highlights of the visions created at the workshop, followed by reflections on the method, challenges on the day and lessons learnt.

The final visions produced by citizens from each table at the consultation are presented in the vignettes in the appendix of the paper to showcase the richness of ideas discussed on the day. The visions are outlined in this way as we did not want to dilute the depth of ideas by corroborating the visions. This process of corroboration will take place in a co- creation workshop to be held in Milan in April 2016, whereby all 30 European countries participating in CIMULACT will share their visions and the process of comparative analysis will take place.

The CIMULACT method offers a structured approach for harvesting diverse citizen views on desirable sustainable futures through structured dialogue with goal- oriented team work. The end goal is to produce artefacts that externalises the group’s vision of a desirable sustainable future in order to guide Europe’s future research agenda.

The method of a ‘facilitated workshop’ is considered to be very rich for acquiring various citizens’ input, but there is a challenge to massage that input towards consensus without imposing a facilitator view and respecting all the voices around the table, not just the most heard. The role of the facilitator was to encourage conversation without giving their own opinions, and allow citizens to offer their own unbiased real life opinions on the topics raised by each member of their group/table. Another important task for group facilitators was to ensure that each member of the group had their opinions heard, no matter how outspoken or shy a citizen was. The atmosphere in the room while professional was informal providing a safe place for citizens to express their true opinions without fear of judgement.

The selection criteria outlined in the method also ensures that a diverse group of citizens were represented at the consultation workshop. Consequently, a range of different attitudes, motivations, and desires were represented at each table which promotes interesting dialogue. As part of the recruitment process each citizen had to write a brief personal motivation statement why they were interested in attending this event, which in turn meant that the citizens who attended were those who had an interest in the topic and wished to voice their opinions. Essentially this meant that an ‘engaged’ group of citizens attended the event which resulted in high levels of citizen participation. This method worked very well and was very successful in collecting unbiased citizen opinions. However, some citizens in the room on the day had ‘an axe to grind’ and would not have been newcomers to this sort of event. Therefore, it would be interesting to think about how to recruit from the unheard voices in society in relation to future consultations. It would be interesting to consider a crowdfunding initiative to support this. Particularly this matters for getting engagement across the socioeconomic divide. Incentivising engagement in a way that would bring a representative sample of a population is important.

It is worth noting the atmosphere in the room was electric from the beginning. The citizens were very engaged in the process and participants were eager, enthusiastic to play their role in shaping the future. The method was very accessible to citizens and did not alienate laypeople with an overly academic approach i.e. citizens could pitch discussion topics in an informal way. As stated by one facilitator ‘The informality of the process was great. I think people within the groups settled quickly as the process did not require participants ‘jumping straight into the process’’. Getting people to contribute in small group size (4/5 to a table) also meant everyone had their say which resulted in tremendous thematic convergence across the groups (especially environment and social inclusion).

Furthermore, the logistics of the event were excellent. As it was a full day event this made it very easy for citizens to focus the mind at each stage of the event in terms of the required output. It proved very useful to have a key team member acting as an overall facilitator, keeping an eye on all groups and facilitators and providing assistance when and where needed i.e. if someone needed further clarification on something. From the general facilitators perspective it was perceived that an overall facilitator chairing the workshop ensured that time management was adhered to ensuring groups were conducting the same tasks in parallel and delivering on all the required outputs of the day.

Visual creativity was required by participants to depict their ideas, and the use of images was encouraged to stimulate their thinking process. For instance, the inspiring picture set was central to all group work during the day. The pictures stimulated creativity, and helped citizens to open up and share their thoughts about the future. The three templates provided citizens with an accessible medium for externalising their vision for the future, and offered a balance between individual and group work. It is worth noting some bias may have occurred as some people are visual leaners and some are now. Figure 3 presents an example of the templates.

In addition, the knowledge that the output of the consultation workshop would be delivered to the EU commission helped keep citizens very motivated throughout the day, and people were very open to sharing and co-operating with others.

However, there were also a number of challenges to this method. Some people turned up with a pet passion (for example, food sustainability) and were keen to exclusively create a vision on such topic. As a result, the facilitator was required to play a key role at a particular table to encourage a broader appreciation of other visions at the beginning of the process. Nevertheless, by the end of the day, some discussion points were deemed to be repetitive. In particular, one facilitator observed that ‘if a concrete vision is formed early (in the form of a story) then there is a lot of unnecessary repetition. When the group has a concrete idea early in the process they tend to keep refining it’. The final vision tended to be more creative if the group was still undecided as to what their stories/visions would be earlier in the process.

Balancing the views of each participant was essential as some individuals were very enthusiastic to participate and tended to dictate the process. As observed by one facilitator:

keeping a balanced contribution across the group was challenging as one member continually wanted to drag the discussion back to those members of society who are extremely disenfranchised. This was tricky because it worked against isolating specific challenges, because that section of society had multiple issues to address. (i.e. economic, legal, education)

Time keeping was a challenge also given the tight schedule of the event. Citizens often became very passionate about a topic/section and it was difficult at times to get the focus of the table to continue with the task at hand. The facilitator also had to maintain a good course of dialogue between participants and ensure that the conversation did not stray off topic. As the time allotted to each step in the method was limited, it was essential that productivity was maximised.

Finally, the facilitator also had to try and create a good dynamic between the participants at each table and balance different personality types. It was important that everyone was provided with a chance to contribute and that they felt confident in expressing their opinions openly. This required the facilitator to ensure that all voices at the table were heard; as some citizens were more opinionated than others, the facilitator had to take a proactive approach to ensure that each participant had a chance to speak. It was also important that participants felt that their input was reflected in the final visions. This was very challenging as compromises were necessary to speed up the process and deliver more specific and attainable visions. Furthermore, the facilitator had to ensure that the overall opinions from the table were desirable and had an element of realism to them i.e. avoid utopian visions.

Overall, the Irish National Citizen’s Vision consultation workshop proved to be a great success. The dynamic among the citizens at each table provided a rich catalyst for discussion, and the rules to good dialogue were well observed by all participants. The outputs from each step of the methodology were also of a very high standard. In doing so, the research team are confident that the visions produced by the citizens will offer valuable input to CIMULACT’s primary objective - ‘to add to the relevance and accountability of European research and innovation... by engaging citizens and stakeholders in co-creation of research agendas based on real and validated societal visions, needs and demands’.

The enthusiasm shown by the citizens throughout the workshop indicates a strong desire among Irish citizens to take part in similar consultation events in the future. In particular, the interest generated among the public during the recruitment process came as a welcome surprise to all involved. Many citizens contacted us to express their sincere interest in attending the consultation workshop and the high calibre of personal motivation statements from registrants made it challenging to choose only 45 people to attend.

The eagerness of Irish citizens to engage in the policy-making process signals a real need for more citizen consultations to be carried out, and we are hopeful that the outputs from this consultation will demonstrate the creative and informed contribution that citizens can make to the policy-making agenda once they are offered a platform to be heard.

We learned a number of insights on the process of citizens engaging in decision-making from our findings from cognitive psychology, complexity science, and social psychology.

Firstly, the future of technology is a complex nexus and many technology issues will continue to be live issues as the technology progresses. When considering technology at an individual level, there can be a bipolar reaction from citizens. Citizens are keen to see the development and are hopeful that a development in technology will lead to an improvement of some sort, be in economical, societal, political, physical, or psychological context (Goldstein & Gigerenzer, 2009). However, often the costs are not appreciated or even anticipated. Individual considerations of technology lend itself to citizens seeking a panacea of some sorts whilst not appreciating there is no perfect solution.

However, engaging the collective citizen in the decision-making process offers an opportunity to reflect on the blurring lines between social, political, and technology and economic context as technology advances at an unparalleled and unpoliced rate (Jasanoff, 2012). Indeed the process of engaging citizens, rather than seeking a panacea, offers an opportunity to articulate the critical citizen concerns in order for policy makers to design an anti-fragile system, as highlighted by Taleb (2010) when he discusses the need for inherent resistance. This takes into consideration the difficulties of star-gazing into the future to forecast typically random and unexpected events.

Furthermore, the anonymity of the group, together with a common interest of evangelism and civic duty, supported good discussion and an avoidance of group think (Surowiecki, 2004). Citizens embraced the uncertainty and engaged in unstructured blue sky thinking, supported by the method. Diemand- Yauman, Oppenheimer, and Vaughan (2011) argue that individuals are more creative when they have to apply themselves in more challenging environments. In their study of fonts, they argued the more difficult the font of the text is, the greater the performance from the student. Our study reflects this as the group appeared to work well in spite of the aspirational nature of the challenge to articulate a vision of the future of Europe. Furthermore the method supported creative thinking.

Another challenge that emerged is how to manage vested interests gaming the process? Many citizens turned up on the day with a pet subject and tried to dominate the conversation. It took the skills of experienced facilitator to try and encourage a balanced conversation among all the citizens participating at the table. Furthermore, education is an important exercise so that citizens can appreciate the nuances of the technological challenges (Glicken, 2000).

Our paper presented the findings of an innovative method to engage citizens in decision-making. We advocate that as science and technology advances, traditional stakeholders3 in the policy-making process should work together with citizens to make an impact by exploiting individual and collective intelligence. The case study showed the need to appreciate unexpected advantages of having to cope with messy challenges. The H2020 landscape is characterised by seven grand challenges, with each encompassing a large degree of complexity. A method such as the case presented here, lends itself to creative problem solving and sheds light on a new way to solve a really complicated problem. Rather than approaching it in piecemeal, iterative, step by step approach, the use of images and the process of articulating visions, embraced the complexity early on in the process. Whilst the purpose of our study was not to justify the CIMULACT method, we deemed the process of sharing present needs and desires together with long term dreams and aspirations to be a fruitful one and think the case will be of interest to the decision-making scholarly community, the policy makers and to citizens as a whole.

No potential conflict of interest was reported by the authors.

This work was supported by the European Union’s Horizon 2020 Framework Programme for Research and Innovation under [grant number 665948].

Notes

1. Global Pulse is a innovation initiative of the United Nations Secretary-General on big data. Its vision is a future in which big data is harnessed safely and responsibly as a public good. http://www.unglobalpulse.org/about-new

2. http://www.cimulact.eu

3. Stakeholders in the policy-making process include researchers and innovators, research organisations, policy and decision makers at national, regional and local levels, primary, secondary and higher education establishments, science museums, libraries, media, artists, creative industries, associations and citizens groupings, etc.

• Abels, G. (2007). Citizen involvement in public policy-making: Does it improve democratic legitimacy and accountability? The case of pTA. Interdiscip Inf Sci, 13, 103–116. [Crossref], [Google Scholar]
• Delvenne, P., Fallon, C., & Brunet, S. (2011). Parliamentary technology assessment institutions as indications of reflexive modernization. Technology in Society, 33, 36–43. [Crossref], [Google Scholar]
• Diemand- Yauman, C., Oppenheimer, D. M., & Vaughan, E. B. (2011). Fortune favors the bold (and the italicized): Effects of disfluency on retention. Cognition, 118, 111–115. [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Glicken, J. (2000). Getting stakeholder participation ‘right’: A discussion of participatory processes and possible pitfalls. Environmental Science and Policy, 3(6), 305–310. [Crossref], [Google Scholar]
• Goldstein, D., & Gigerenzer, G. (2009). Fast and frugal forecasting. Int J Forecast, 25, 760–772. [Crossref], [Web of Science ®], [Google Scholar]
• Irvin, R., & Stansbury, J. (2004). Citizen participation in decision making: Is it worth the effort? Public Administration Review., 64, 55–65. [Crossref], [Web of Science ®], [Google Scholar]
• Jasanoff, S. (2012). Science and public reason. New York, NY: Routledge. [Crossref], [Google Scholar]
• Lengwiler, M. (2008). Participatory approaches in science and technology: Historical origins and current practices in critical perspective. Sci Technol Human Values, 33:186–200 [Crossref], [Web of Science ®], [Google Scholar]
• Lessig, L. (2001). The future of ideas: The fate of the commons in a networked world. New York, NY: Vintage. [Google Scholar]
• March, J. G., & Olsen, J. P. (1976). Ambiguity and choice in organisations (p. 37). Bergen: Universitetsforlaget. [Google Scholar]
• March, J. G., & Simon, H. A. (1958). Organizations. Oxford, England: Wiley-Blackwell. [Google Scholar]
• Mitchell, R. K., Agle, B. R., & Wood, D. J. (1997). Towards a theory of stakeholder identification and salience: Defining the principles of who and what really counts. Academy of Management Review, 22, 853–886. [Crossref], [Web of Science ®], [Google Scholar]
• Morozov, E. (2013). To save everything, click here: Technology, solutionism, and the urge to fix problems that don’t exist. UK: Penguin. [Google Scholar]
• Owen, R., Macnaghten, P., & Stilgoe, J. (2012). Responsible research and innovation: from science in society to science for society, with society. Science and Public Policy, 39, 751–760. [Crossref], [Web of Science ®], [Google Scholar]
• Rask and Damianova. (2009). Citizen visions – preliminary content report, 9. [Google Scholar]
• Stilgoe, J., Owen, R., & Macnaghten, P. (2013). Developing a framework for responsible innovation. Research Policy, 42, 1568–1580. [Crossref], [Web of Science ®], [Google Scholar]
• Surowiecki, J. (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. New York, NY: Doubleday. [Google Scholar]
• Taleb, N. M. (2010). The black swan: The impact of the highly improbable fragility, Vol. 2. New York, NY: Random House. [Google Scholar]
• Tapscott, D., & Williams, A. (2006). Wikinomics: How mass communication changes everything. London: Penguin Group. [Google Scholar]
• Van der Helm, R. (2014). The vision phenomenon: Towards a theoretical underpinning of visions of the future and the process of envisioning. Futures, 41, 96–104. [Crossref], [Web of Science ®], [Google Scholar]
• Von Schomberg, R. (Ed.). (2011). Towards responsible research and innovation in the information and communication technologies and security technologies fields. Luxembourg: Publications Office of the European Union & lt. [Crossref], [Google Scholar]
• Warnke, P., & Heimeriks, G. (2008). Technology foresight as innovation policy instrument: Learning from science and technology studies. In C. Cagnin, M. Cagnin, R. Johnston, F. Scapolo, & R. Barre (Eds.). Future-oriented technology analysis. Strategic intelligence for an innovative economy (pp. 71−87). London: Springer [Crossref], [Google Scholar]

Vignette 1

Download CSVDisplay Table

Vignette 2

Download CSVDisplay Table

Vignette 3

Download CSVDisplay Table

Vignette 4

Download CSVDisplay Table

Vignette 5

Download CSVDisplay Table