Announcing a new Pilot project with the Province of Friesland and UBR Rijks ICT Gilde (part of the Ministry of the Interior and Kingdom) the Netherlands.

Excited to announce that on May 16, 2022 we had our kick-off of the pilot project “Assessment for responsible AI” together with the Province of Friesland (Fryslân), a team of the  Z-inspection® initiative, and the UBR  Rijks ICT Gilde (part of the Ministry of the Interior and Kingdom Relations, the Netherlands).

Together we will investigate the reliability of AI applications for the Province of Friesland and their responsible use, using the  Z-inspection® process and the EU Framework for Trustworthy AI.

The Leeuwarden Municipality, University of Groningen/Campus Fryslân and policy advisors from the Ministry of the Interior were also invited as observers.

Announcement in Dutch :

Kick-off krachtig samenwerkingsverband voor verantwoorde AI

Nieuwsbericht | 17-05-2022 | 13:42- Rijks ICT Gilde. 

English Translation:

Kick-off powerful partnership for responsible AI

News release | 17-05-2022 | 13:42
Yesterday was the kick-off of the pilot ‘Assessment for responsible AI’. The Province of Fryslân, AI authority Prof. Dr. Zicari and the UBR Rijks ICT Gilde are jointly investigating the reliability of AI applications and their responsible use. Leeuwarden municipality, Groningen University/Campus Fryslân and policy advisors from the Ministry of the Interior are invited as observers.

Artificial Intelligence (AI) is appearing in more and more aspects of our lives. It is in all kinds of devices we use in our work and private lives. The technology, based on data and algorithms, can be useful in solving social issues about energy, sustainability or even poverty, for example.

As a government, we want to exploit the opportunities of AI, but the technology still raises many important questions.

How reliable are algorithms? Can an algorithm discriminate? And how transparent is the use of AI?

In the three-month pilot ‘Assessment for responsible AI’ we are looking for answers to the questions:

How do you as a government steer the development and use of responsible AI?
What frameworks, laws and regulations are important, and how do we test them in the development and use of AI?
How do you analyze, assess and improve AI applications? And are the applications in line with public values and human rights?

In the pilot we assess an algorithm of the province of Fryslân. We will analyze it using the Z-inspection® method of Roberto Zicari; a self-assessment in which participants discuss critical issues such as: the purpose of the algorithm, the development process, ethical dilemmas and conflicts of interest. The Z-inspection® method is a working method to analyze, assess and improve AI applications in a sustainable, demonstrable and transparent way. This enables organizations to develop and use responsible AI applications in a structured way.

Furthermore, it is very important that the knowledge and experiences from the pilot are shared. First of all, to stimulate digital awareness and dialogue about AI within the government. And then to be able to confidently deploy the technology for the questions of tomorrow.

Z-inspection®: A process to assess trustworthy AI in Practice has won a ISSIP Distinguished Recognition Award for Service Innovation!

The results of the ISSIP Excellence In Service Innovation Awards are in, and it is our great pleasure to inform you that Z-Inspection: A process to assess trustworthy AI in Practice has won a Distinguished Recognition Award for Service Innovation. 

The Distinguished Recognition Award is given each year to submissions that, in the judgment of the ISSIP Award Committee, represent innovative and impactful service designs and implementations from which service innovators around the world can derive inspiration.

The judging criteria are based on the uniqueness, creativity, technical merit, value generation and impact of the innovative solution. 

We plan to recognize your achievement at the next ISSIP Board of Directors & Progress call, July 27, 2022, 3-4 pm EDT. 

Again Congratulations! 

cc Ralph Badinelli, Chair of the ISSIP Awards Committee

cc Jim Spohrer, ISSIP Board of Directors 

Michele Carroll, Executive Director

List of Awards here.

Z-Inspection® process module is incorporated into a new Interdisciplinary Master’s Program on eXplainable Artificial Intelligence in Healthcare to equip a new generation of interdisciplinary students in assessment of Trustworthy AI.

Pavia, April 9, 2022

Z-Inspection® process module is incorporated into a new Interdisciplinary Master’s Program on eXplainable Artificial Intelligence in Healthcare, to equip a new generation of interdisciplinary students in assessment of Trustworthy AI.

With the Z-Inspection® process students will learn how to assess trustworthiness of AI systems for healthcare using socio-technical scenarios.

The new Master program is offered by a consortium composed of the University of Pavia, Goethe University (D), Keele University (UK), Leibniz University Hannover (D) and University of Ljubljana (SL), with funding from the European Commission program for “Connecting Europe Facility (CEF) Telecom” designed to offer co-financing to stimulate and support the spread of European Digital Service Infrastructures (DSI) in various sectors, including the design and implementation of specialized master’s programs in Artificial Intelligence (AI).

The overall grant is 1,664,557 euros.

The Master’s degree will be awarded by the University of Pavia

The duration of the  Master is 1 year and a half (2250 hours), and corresponds to 90 ECTS.

The Content of the Master is available here:’s%20Brochure.pdf

The application for the first intake (October 2022) opens in Spring 2022

For more information:

Professor Roberto V. Zicari gave a Guest Lecture at the Seoul National University Law School.

Professor Roberto V. Zicari was invited to give a guest lecture by Professor Heo, Seongwook, Professor of Law at the Seoul National University Law School.

Professor Zicari introduced to his law students:

i) the EU Framework for Trustworthy AI, 

ii) our research work on assessing Trustworthy AI, and 

ii) the EU AI Act.

The English presentation starts at minute 28 (before it is in Korean): Watch here.

The Center for the Study of Ethics in the Professions at Illinois Institute of Technology (Chicago, USA) launches The Ethical and Trustworthy AI Lab based on the Z-Inspection® Process.

Chicago, March 15, 2022

The Ethical and Trustworthy AI Lab at Illinois Institute of Technology’s Center for the Study of Ethics in the Professions is an interdisciplinary group of researchers interested in the social and ethical implications of Artificial Intelligence (AI).

The group investigates philosophical, ethical, and social aspects of AI including trustworthiness and the question of what it is that makes AI uses ethical, just, and trustworthy; the roles of ethics codes, ethical guidelines, and policy-making in the regulation of AI technology; as well as AI applications in agriculture and medical contexts. 

The mission of the Lab is to involve stakeholders from all fields, such as computer science, technology, engineering, philosophy, social sciences, practitioners, and students, in an interdisciplinary reflection on the ethical uses of AI.

The Lab closely collaborate with the AI@IllinoisTech initiative, in particular its AI Ethics Working Group (AIEWG) and the international Z-Inspection® network. The Z-Inspection® assessment method for Trustworthy AI is an approach based on the Ethics Guidelines for Trustworthy AI by the European Commission High-Level Expert Group on Artificial Intelligence.

The head of the new Lab is Prof. Elisabeth Hildt.

More information here.

Vision 2022

1. Trustworthy AI Labs are established, based on the Z-Inspection® process

Like this one in Helsinki;

2. Z-Inspection® process modules are incorporated into Master and PhD programs at selected universities, to equip a new generation of interdisciplinary students in assessment of ethical AI;

3. The Z-Inspection® process is leveraged in future European policy and regulation relating to AI.

Arcada University of Applied Sciences (Helsinki, Finland) launches The Laboratory for Trustworthy AI based on the Z-Inspection® Process.

Helsinki, November 12, 2021

The Laboratory for Trustworthy AI at Arcada University of Applied Sciences (Helsinki, Finland) is a transdisciplinary and international research community who trains organizations and actors to assess the use of artificial intelligence. The lab connects academia and civil society, including developers of AI solutions, students, end-users, researchers, and stakeholders.

The Lab promotes a human-centric approach to AI and towards closing the gap between ethically sound AI development and the technical and methodological practices. The Lab embraces technical innovativeness and assist organizations in mapping socio-technical scenarios that are used to assess risk. 

The Lab collaborates closely with international networks such as the Z-Inspection® assessment method for Trustworthy AI External link. The Z-Inspection® approach is a validated assessment method that helps organizations to deliver ethically sustainable, evidence based, trustworthy and user-friendly AI driven solutions. The method is published in IEEE Transactions on Technology and Society 

More information about the Lab here.

Lessons Learned: Co-design of Trustworthy AI. Best Practice. By Helga Brogger, President of the Norwegian Society of Radiology

Mission: …Aid the development of designs with reduced end-user vulnerability…

-“…Socio-technical scenarios can be used to broaden stakeholders’ understanding of one’s own role in the technology, as well as awareness of stakeholders’ interdependence…”

– “…Recurrent, open-minded, and interdisciplinary discussions involving different perspectives of the broad problem definition….”

– “…The early involvement of an interdisciplinary panel of experts broadened the horizon of AI designers which are usually focused on the problem definition from a data and application perspective…”

– “…Consider the aim of the future AI system as a claim that needs to be validated before the AI system is deployed..”

-“…Involve patients at every stage of the design process … it is particularly important to ensure that the views, needs, and preferences of vulnerable and disadvantaged patient groups are taken into account to avoid exacerbating existing inequalities…”

Thank you, Roberto V. Zicari and the rest of the team for these insights!

— Helga Brogger, President of the Norwegian Society of Radiology


Co-Design of a Trustworthy AI System in Healthcare: Deep Learning Based Skin Lesion Classifier.

Roberto V. Zicari, Sheraz Ahmed, Julia Amann, Stephan Alexander Braun, John Brodersen, Frédérick Bruneault, James Brusseau, Erik Campano, Megan Coffee, Andreas Dengel, Boris Düdder, Alessio Gallucci, Thomas Krendl Gilbert, Philippe Gottfrois, Emmanuel Goffi, Christoffer Bjerre Haase, Thilo Hagendorff, Eleanore Hickman, Elisabeth Hildt, Sune Holm, Pedro Kringen, Ulrich Kühne, Adriano Lucieri, Vince I. Madai, Pedro A. Moreno-Sánchez, Oriana Medlicott, Matiss Ozols, Eberhard Schnebel, Andy Spezzatti, Jesmin Jahan Tithi, Steven Umbrello, Dennis Vetter, Holger Volland, Magnus Westerlund and Renee Wurth.

Front. Hum. Dyn. |Human and Artificial Collaboration for Medical Best Practices, July 13, 2021


Learn more

Our paper ” Co-design of a Trustworthy AI System in Healthcare: Deep Learning based Skin Lesion Classifier.” has been accepted for publication in Frontiers in Human Dynamics

Co-design of a Trustworthy AI System in Healthcare: Deep Learning based Skin Lesion Classifier.

Roberto V. Zicari (1)(2)(3), Sheraz Ahmed (4), Julia Amann (5), Stephan Alexander Braun (6)(7), John Brodersen (8)(9), Frédérick Bruneault (10), James Brusseau (11), Erik Campano (12), Megan Coffee (13), Andreas Dengel (4)(14), Boris Düdder (15), Alessio Gallucci (16), Thomas Krendl Gilbert (17), Philippe Gottfrois (18), Emmanuel Goffi (19), Christoffer Bjerre Haase (20), Thilo Hagendorff (21), Eleanore Hickman (22), Elisabeth Hildt (23), Sune Holm (24), Pedro Kringen (1), Ulrich Kühne (25), Adriano Lucieri (4)(14), Vince I. Madai (26)(27)(28), Pedro A. Moreno-Sánchez (29), Oriana Medlicott (30), Matiss Ozols (31)(32), Eberhard Schnebel (1), Andy Spezzatti (33), Jesmin Jahan Tithi (34), Steven Umbrello (35), Dennis Vetter (1), Holger Volland (36), Magnus Westerlund (2), Renee Wurth (37).

(1) Frankfurt Big Data Lab, Goethe University Frankfurt, Germany
(2) Arcada University of Applied Sciences, Helsinki, Finland
(3) Data Science Graduate School, Seoul National University, South Korea
(4) German Research Center for Artificial Intelligence (DFKI) Kaiserslautern, Germany
(5) Health Ethics and Policy Lab,Swiss Federal Institute of Technology (ETH Zurich), Switzerland
(6) Department of Dermatology, University Clinic Münster, Germany
(7) Dept. of Dermatology, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany
(8) Section of General Practice and Research Unit for General Practice, Department of Public Health, Faculty of Health and Medical Sciences, University of Copenhagen, Danemark
(9) Primary Health Care Research Unit, Region Zealand, Denmark
(10) École des médias, Université du Québec à Montréal and Philosophie, Collège André-Laurendeau, Canada
(11) Philosophy Department, Pace University, New York, USA
(12) Department of Informatics, Umeå University, Sweden
(13) Department of Medicine and Division of Infectious Diseases and Immunology, NYU Grossman School of Medicine, New York, USA
(14) Department of Computer Science, TU Kaiserslautern, Germany
(15) Department of Computer Science (DIKU), University of Copenhagen (UCPH), Denmark
(16) Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands.
(17) Center for Human-Compatible AI, University of California, Berkeley, USA
(18) Department of Biomedical Engineering, Basel University, Switzerland
(19) The Global AI Ethics Institute, France
(20) Section for Health Service Research and Section for General Practice, Department of Public Health, University of Copenhagen, Denmark. Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, Australia (21) Ethics & Philosophy Lab, University of Tuebingen , Germany
(22) Faculty of Law, University of Cambridge, UK
(23) Center for the Study of Ethics in the Professions, Illinois Institute of Technology Chicago, USA
(24) Department of Food and Resource Economics, Faculty of Science, University of Copenhagen, DK
(25) “Hautmedizin Bad Soden”, Germany
(26) Charité Lab for AI in Medicine, Charité Universitätsmedizin Berlin, Germany
(27) QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Charité Universitätsmedizin Berlin, Germany
(28) School of Computing and Digital Technology, Faculty of Computing, Engineering and the Built Environment, Birmingham City University, United Kingdom
(29) School of Healthcare and Social Work, Seinäjoki University of Applied Sciences (SeAMK), Finland
(30) Freelance researcher, writer and consultant in AI Ethics, UK
(31) Division of Cell Matrix Biology and Regenerative Medicine, The University of Manchester, UK
(32) Human Genetics, Wellcome Sanger Institute, UK
(33) Industrial Engineering & Operation Research, UC Berkeley, USA
(34) Intel Labs, Santa Clara, CA, USA
(35) Institute for Ethics and Emerging Technologies, University of Turin, Italy

(36) Z-Inspection® Initiative
(37) T.H Chan School of Public Health, Harvard University, USA

* Correspondence:

Corresponding Author Roberto V. Zicari

Z-inspection® is a registered trademark

Accepted on 09 June 2021
Front. Hum. Dyn. doi: 10.3389/fhumd.2021.688152


This paper documents how an ethically aligned co-design methodology ensures trustworthiness in the early design phase of an artificial intelligence (AI) system component for healthcare. The system explains decisions made by deep learning networks analyzing images of skin lesions. The co-design of trustworthy AI developed here used a holistic approach rather than a static ethical checklist and required a multidisciplinary team of experts working with the AI designers and their managers. Ethical, legal, and technical issues potentially arising from the future use of the AI system were investigated. This paper is a first report on co-designing in the early design phase. Our results can also serve as guidance for other early-phase AI-similar tool developments.

Keywords: Artificial Intelligence, Healthcare, Explainable AI, Trust, Case-Studies, Trustworthy AI, Ethics, Malignant Melanoma, Z-inspection®, Ethical co-design.

Our paper ” On Assessing Trustworthy AI in Healthcare Best Practice for Machine Learning as a Supportive Tool to Recognize Cardiac Arrest in Emergency Calls.” has been accepted for publication in Frontiers in Human Dynamics