Pilot Project: Assessment for Responsible Artificial Intelligence together with Rijks ICT Gilde -Ministry of the Interior and Kingdom Relations (BZK)- and the province of Fryslân (The Netherlands)
Original Full text available from the Rijks ICT Gilde -Ministry of the Interior and Kingdom Relations (BZK)- web site in Dutch.
https://www.rijksorganisatieodi.nl/rijks-ict-gilde/mycelia/pilot-kunstmatige-intelligentie
Artificial Intelligence (AI) is in more and more aspects of our lives. The technology – based on algorithms and data – is in numerous devices and can be useful in solving social issues. For example, about energy, sustainability or poverty. As a government, we have an exemplary role. If we want to seize the opportunities of AI, then important questions about ethics, technology, transparency and the possible effects of AI applications on our society must be answered.
The pilot “Assessment for Responsible AI” is a step in this process. During a three-month pilot, the Rijks ICT Gilde (Ministry of the Interior and Kingdom Relations (BZK)) , in cooperation with the province of Fryslân and AI authority Prof. Dr. Zicari and his team, is investigating a deep learning algorithm in practice.
Reason for the pilot
During the conference “AI and the future of Europe” in Brussels on March 30, 2022, Secretary of State Alexandra van Huffelen told us that the digital transition and the use of AI should always be human-centered and based on our democratic values and rights. Governments should lead by example in this regard.
As a government, we want to seize the opportunities of AI, but the technology still raises many important questions. How reliable are algorithms? Can an algorithm discriminate? What are the ethical and social effects of AI and how transparent is its use? In addition, the use of AI must always be human-centered and based on our democratic values and rights.
With that comes an impressive amount of rules, frameworks and frameworks in the field of AI. How do you apply them in practice? What do you need to pay attention to? And how do you integrate them into the development and use of AI?
With the pilot “Assessment for responsible AI” we hope to get answers to these questions and more. First, to stimulate awareness and dialogue about AI within government. And then to be able to confidently deploy the technology for the questions of tomorrow.
Background for the pilot
During this six-month pilot, the practical application of a deep learning algorithm from the province of Fryslân will be investigated and assessed. The algorithm maps heathland grassland by means of satellite images for monitoring nature reserves. The testing of this algorithm is done in collaboration with an international interdisciplinary team, based on the ‘Z-Inspection® method’ – a process to assess Trustworthy AI in Practice.
This involves testing the algorithm for social, ethical, technical and legal implications. This is done in interdisciplinary teams according to a holistic methodology: it looks at the coherence, arrangement and interaction of the features to a system. The holistic nature of the method leaves room for different dimensions and views, and focuses in particular on the interpretation and discussion of ethical issues and tensions.
Pilot objectives
The pilot has multiple objectives, so each party can get its own win from the pilot:
A science-based assessment in practice based on a concrete AI application;
To understand how to carefully and responsibly organize your processes for developing and using AI;
Learning which frameworks, laws, and regulations are important and must be tested in the different phases of development and use;
Increasing insight and overview through recommendations;
Stimulating dialogue and increasing awareness about applying reliable AI.
Z-Inspection® : A method for responsible AI
One of the main characteristics of the Z-Inspection® method is its interdisciplinary nature. The complexity of an AI application is reflected in the composition of the team. The diversity of participants provides a more inclusive assessment of the reliability of an AI application.
Another strong feature of this method is its dynamic application. A standard checklist does not adapt to the case. In contrast, the holistic approach determines which issues are central to the use case (real-world application) at different stages of the process. And assesses which aspects of the case are most important, moving back and forth between intra- and interdisciplinary discussions.
With the application of this method, awareness grows within the own organization, a frame of mind (mindset) emerges for responsible data use and the reliable application of AI. Both Province of Fryslan and the Rijks ICT Gilde are testing the method in practice to learn together how to look at an AI application from different dimensions.
Pilot outcome
Using the Z-Inspection® method, the advantages and disadvantages of the AI application under investigation are described. This is done using an ongoing and iterative (repeating and increasingly refining) research process. Participants are hereby given space to openly reflect and document what is known (and unknown) about the capabilities of an AI application as a basis for later evaluations.
Participating parties
Several parties are working together in the pilot. The Province of Fryslân, AI authority Prof. Dr. Zicari and his team and the UBR| Rijks ICT Gilde are jointly investigating the reliability of AI applications and their responsible use. Leeuwarden municipality, the University of Groningen/Campus Fryslân and policy advisors of the Ministry of the Interior and Kingdom Relations are participating as observers.
Read more about the participating parties below.
Province of Friesland
During the pilot project, a deep learning algorithm of the Province of Fryslân will be evaluated. This algorithm visualizes heather grassing for monitoring nature reserves. The Province of Fryslân is investing in the coming years in the smart and effective use of data. The province sees that almost all provincial developments and social tasks contain a data component. This creates urgency in the subject. To be able to responsibly respond to technological developments as a province, a sharp vision on data and AI is needed. Participation in the pilot helps design the future digital infrastructure and outline ethical frameworks.
Read more > https://www.fryslan.frl/
UBR | Rijks ICT Gilde
The Rijks ICT Gilde (RIG) (Ministry of the Interior and Kingdom Relations (BZK) ) is an ambitious tech organization that implements projects across the central government. The organization uses its knowledge and network to create smart partnerships and solutions. It employs specialists with a drive to help the Netherlands move forward in the fields of data, software and security.
Mycelia is a young, dynamic and energetic programme of the Rijks ICT Gilde that uses the responsible use of data and AI to achieve impact. We do this by answering and filling in relevant questions without compromising the public values and fundamental rights that the government stands for.
With enthusiasm and specific knowledge, Mycelia’s data and AI experts work on impactful and ethically responsible projects for the public sector and collaborate with organizations and experts in order to learn, grow and develop from each other.
With the growth of data and the rise of AI, we see a government that will never be the same again. Responsible use of AI and trust is and will therefore become increasingly important. We believe in helping and supporting each other and an honest and transparent government. We want to leave a better world for the next generations.
In the pilot, the RIG provides expertise on AI, ethics and responsible data use within government.
Read more > https://www.ubrijk.nl/service/rijks-ict-gilde
Update (January 25, 2023)
Timeline
The Assessment for Responsible AI pilot took place from May 2022 through January 2023.
Project Members:
Sara M. Beery,
Marjolein Boonstra,
Frédérick Bruneault,
Subrata Chakraborty,
Tjitske Faber,
Alessio Gallucci,
Eleanore Hickman,
Gerard Kema,
Heejin Kim,
Jaap Kooiker,
Ruth Koole,
Elisabeth Hildt,
Annegret Lamadé,
Emilie Wiinblad Mathez
Florian Möslein,
Genien Pathuis,
Rosa Maria Roman-Cuesta,
Marijke Steege,
Alice Stocco,
Willy Tadema,
Jarno Tuimala,
Isabel van Vledder,
Dennis Vetter,
Jana Vetter,
Elise Wendt,
Magnus Westerlund,
Roberto V. Zicari
Update (July 15, 2023)
Rijks ICT Gilde has recently published (in Dutch):
– the technical results of the assessment.
Update (August 3d, 2023)
Rijks ICT Gilde has published (in English):