The European Approach to Artificial Intelligence

– Ethics as a service: a pragmatic operationalisation of AI Ethics. Jessica Morley et al. LINK to .PDF.

– Ethics Guidelines for Trustworthy AI. Independent High-Level Expert Group on Artificial Intelligence. European commission, 8 April, 2019. Link to .PDF

– WHITE PAPER. On Artificial Intelligence – A European approach to excellence and trust.  European Commission, Brussels, 19.2.2020 COM(2020) 65 final. Link to .PDF

– The EU’s approach to artificial intelligence. LINK

The Commission published its AI package in April 2021, proposing new rules and actions to turn Europe into the global hub for trustworthy AI. This package consisted of:

• a Communication on Fostering a European Approach to Artificial Intelligence;
• the Coordinated Plan with Member States: 2021 update;
• a proposal for an AI Regulation laying down harmonised rules for the EU (Artificial Intelligence Act).

–  Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL LAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE (ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE ACTS. LINK

UNESCO Recommendation on the ethics of artificial intelligence

–  Recommendation on the ethics of artificial intelligence. UNESCO, November 2021. LINK

WHO guidance. Ethics and governance of artificial intelligence for health

– Ethics and governance of artificial intelligence for health, WHO guidance, 28 June 2021. LINK

OECD Recommendation of the Council on Artificial Intelligence

–  Recommendation of the Council on Artificial Intelligence. OECD,22/05/2019 LINK

Meta-study on AI ethics guidelines

Hagendorff, T. The Ethics of AI Ethics: An Evaluation of Guidelines. Minds & Machines 30, 99–120 (2020). https://doi.org/10.1007/s11023-020-09517-8  LINK

Z-Inspection®: A process to assess Trustworthy AI. Best Practices.

How to Assess Trustworthy AI in Practice.
Roberto V. Zicari, Julia Amann, Frédérick Bruneault, Megan Coffee, Boris Düdder, Eleanore Hickman, Alessio Gallucci, Thomas Krendl Gilbert, Thilo Hagendorff, Irmhild van Halem, Elisabeth Hildt, Georgios Kararigas, Pedro Kringen, Vince I. Madai, Emilie Wiinblad Mathez, Jesmin Jahan Tithi, Dennis Vetter, Magnus Westerlund, Renee Wurth
On behalf of the Z-Inspection® initiative (2022)
Abstract
This report is a methodological reflection on Z-Inspection®.  Z-Inspection® is a holistic process used to evaluate the trustworthyness of AI-based technologies at different stages of the AI lifecycle. It focuses, in particular, on the identification and discussion of ethical issues and tensions through the elaboration of socio-technical scenarios. It uses the general European Union’s High-Level Expert Group’s (EU HLEG) guidelines for trustworthy AI. This report illustrates for both AI researchers and AI practitioners how the EU HLEG guidelines for trustworthy AI can be applied in practice. We share the lessons learned from conducting a series of independent assessments to evaluate the trustworthiness of AI systems in healthcare. We also share key recommendations and practical suggestions on how to ensure a rigorous trustworthy AI assessment throughout the life-cycle of an AI system.

Cite as: arXiv:2206.09887 [cs.CY] The full report is available on arXiv .

Link: https://arxiv.org/abs/2206.09887

Full report as .PDF: https://arxiv.org/pdf/2206.09887.pdf

– How to Assess Trustworthy AI  in practice, Roberto V. Zicari, Innovation, Governance and AI4Good, The Responsible AI Forum Munich, December 6, 2021. DOWNLOAD .PDF: Zicari.Munich.December6,2021

– Trustworthy AI, Roberto Zicari, October 25, 2021: Zicari.Zurich Insurance Group.725.10.2021

– Mindful Use of AI.  Z-Inspection: A holistic and analytic process to assess Ethical AI – Roberto V. Zicari, Frankfurt Big Data Lab, July 2, 2020, Youtube video.  Copy of the slides: Zicari.Lecture.October15.2020

– Z-Inspection®: A Process to Assess Trustworthy AI. Roberto V. Zicari, et al. IEEE Transactions on Technology and Society, VOL. 2, NO. 2, JUNE 2021, DOWNLOAD THE PAPER

– On Assessing Trustworthy AI in Healthcare. Machine Learning as a Supportive Tool to Recognize Cardiac Arrest in Emergency Calls-. Roberto V. Zicari, et al. Front. Hum. Dyn., Human and Artificial Collaboration for Medical Best Practices, 08 July 2021 |VIEW ORIGINAL RESEARCH article

– Co-Design of a Trustworthy AI System in Healthcare: Deep Learning Based Skin Lesion Classifier. Roberto V. Zicari, et. Front. Hum. Dyn. |Human and Artificial Collaboration for Medical Best Practices, July 13, 2021 VIEW ORIGINAL RESEARCH article

– Brusseau J. , What a Philosopher Learned at an AI Ethics Evaluation, AI Ethics Journal 2020, 1(1)-4,

Ethics, Moral Values, Humankind, Technology, AI Examples.

– Perspectives on Issues in AI Governance, Lynette Webb, Charina Chou, Google White Paper, 2019. Link to .PDF

–  AI on the Case: Legal and Ethical Issues. Richard Austin, Deeth Williams Wall LLP , May 17, 2019. Link to .PDF

– Report on Algorithmic Risk Assessment Tools in the U.S. Criminal Justice System, Partnership on AI, 2019. Link

– Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice (February 13, 2019). Richardson, Rashida and Schultz, Jason and Crawford, Kate, 94 N.Y.U. L. REV. ONLINE 192 (2019). Available at SSRN

Fairness, Bias and Discrimination in AI.  From Philosophy to Machine Learning.

– Improving Fairness in Machine Learning Systems: What Practitioners Need? K. Holstein et al. CHI 2019; May 4-0, 2019. Link to .PDF

– Ensuring, Fairness in Machine Learning to Advance Health, Alvin Rajkomar et al.  Equity, Annals of Internal Medicine (2018). DOI: 10.7326/M181990. Link to .PDF

– Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements. Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Allison Woodruff, Christine Luu, Pierre Kreitmann, Jonathan Bischof, Ed H. Chi (Submitted on 14 Jan 2019). Link to .PDF

– Dissecting racial bias in an algorithm used to manage the health of populations. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Science. 2019 Oct 25;366(6464):447-453. doi: 10.1126/science.aax2342. Link to .PDF

–AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias, Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, Yunfeng Zhang, 2018, Paper link, Open Source project link, Published as IBM Journal of Research and Development 63(4/5), 2019

AI: Explainability, Transparency.

– Experiences with Improving the Transparency of AI Models and Services. Michael Hind, Stephanie Houde, Jacquelyn Martino, Aleksandra Mojsilovic, David Piorkowski, John Richards, Kush R. Varshney (Submitted on 11 Nov 2019), arXiv:1911.08293v1. Link to .PDF

– Petkovic D, Kobzik L, Re C. “Machine learning and deep analytics for biocomputing: call for better explainability”. Pacific  Symposium on Biocomputing Hawaii, January 2018;23:623-7, Link to .PDF

– Petkovic D, Kobzik L, Ganaghan R,“AI Ethics and Values in Biomedicine – Technical Challenges  and Solutions”, Pacific Symposium on Biocomputing, Hawaii January 3-7, 2020, Link to .PDF

Gunning D, Aha D.:”DARPA’s Exianable Artificial Intelligence Program”, AI magazine, Association for the Advancement of Artificial  Intelligence, Summer 2019, slides

– Ribeiro M, Singh S, Guestrin C. “Why Should I Trust You? Explaining the Predictions of Any Classifier”, KDD ’16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningAugust, 2016,  Link to .PDF or ACM PDF

– Ribeiro M, Singh S,  Guestrin C.: ”Nothing Else Matters: Model-Agnostic Explanations by Identifying Prediction Invariance”, 30th Conf. of Neural Information Processing Systems (NIPS 2016), Barcelona, Spain 2016, Link to .PDF

– Petkovic D, Altman R, Wong M, Vigil A.: “Improving the explainability of Random Forest classifier – user centered approach”. Pacific Symposium on  Biocomputing. 2018;23:204-15, Link to .PDF

– D. Petkovic, A. Alavi, D. Cai, J. Yang, S. Barlaskar:  “RFEX – Simple random Forest Model and Sample Explainer for non-ML experts”, Link to .PDF

– Barlaskar S, Petkovic D: “Applying Improved Random Forest Explainability (RFEX 2.0) on synthetic data”, SFSU TR 18.01, 11/27/20181; with related toolkit at  https://www.youtube.com/watch?v=neSVxbxxiCE

– One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques, Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilovic, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, Yunfeng Zhang, 2019, Link to .PDF

– Explaining explainable AI, Michael Hind, XRDS: Crossroads, The ACM Magazine for Students 25(3), ACM, 2019, Link to .PDF

– Explaining decisions made with AI. – ICO and The Alan Turing Institute, May 2020, Link

Human in the loop, Security, and Accountability.

– Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).  Link to. PDF

– Concrete Problems in AI Safety, Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané. Link to PDF

– Anomalous Instance Detection in Deep Learning: A Survey – Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K. Varshney, Dawn Song. Link to PDF

– Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. Miles Brundage et al. Link to PDF

Assessing AI use cases. Socio-Technical Scenarios.

– Ethical Framework for Designing Autonomous Intelligent Systems. J Leikas et al. J. of Open Innovation, 2019, 5, 1. Link

Assessing AI use cases. Ethical tensions, Trade offs.

– Algorithmic Impact Assessment: A Practical Framework for Public Agency Accountability, Reisman D., Schultz J, Crawford K, Whittake M, AI Now, April 2018. Link to .PDF

– FactSheets: Increasing trust in AI services through suppliers declarations of conformity. Arnold, M.; Bellamy, R. K. E.; Hind, M.; Houde, S.; Mehta, S.; Mojsilovic ́, A.; Nair, R.; Natesan- Ramamurthy, K.; Olteanu, A.; Piorkowski, D.; Reimer, D.; Richards, J.; Tsay, J.; and Varshney, K. R. 2019. IBM Journal of Research & Development 63(4/5). Link to .PDF

–  Datasheets for datasets. Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J. W.; Wallach, H.; Daume ́, III, H.; and Craw- ford, K. 2018.. In Proceedings of the Fairness, Accountability, and Transparency in Machine Learning Workshop. Link to .PDF

–  IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, First Edition. Pp. 211 – 281.Link to .PDF

– COVID-19 Rapid Evidence Review: Exit through the App Store?, Nuffield Foundation. Link

EMPOWERING CITIZENS AGAINST COVID-19 WITH AN ML-BASED AND DECENTRALIZED RISK AWARENESS APPSpeaker : Professor Yoshua Bengio (Mila), Friday, May 8, 2020Recording: here

AI Ethics in Healthcare

–  Stealth research: Lack of peer‐reviewed evidence from healthcare unicorns, Ioana A. Cristea Eli M. Cahan John P. A. Ioannidis, European Journal of Clinical Investigation, 28 January 2019. Link

– Grote T, Berens P. On the ethics of algorithmic decision-making in healthcare, J Med Ethics 2019;0:1–7. doi:10.1136/medethics-2019-105586. Link to .PDF

– Schonberg D. Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications.  International Journal of Law and Information Technology, 2019, Link

– Dorian Peters, et. al, Responsible AI- Two Frameworks for Ethical Design Practice. IEEE Transactions on Technology and Society, Vol. 1, No. 1, March 2020. Link to .PDF

Legal Relevance of AI Ethics

– Trustworthy AI and Corporate Governance: The EU’s Ethics Guidelines for Trustworthy Artificial Intelligence from a Company Law Perspective, Eleanore Hickman1 · Martin Petrin, European Business Organization Law Review (2021) 22:593–625 https://doi.org/10.1007/s40804-021-00224-0.  Download .pdf: AIActLawView

– OECD, ‘Recommendation of the Council on Artificial Intelligence’ (22 May 2019), Link

– G20, ‘G20 Ministerial Statement on Trade and Digital Economy’ (9 June 2019), Link to .PDF

– Ethics Guidelines for Trustworthy AI. Independent High-Level Expert Group on Artificial Intelligence. European commission, 8 April, 2019, Link to .PDF

– F Möslein, ‘Robots in the boardroom: artificial intelligence and corporate law’ in  W Barfield and U Pagallo (eds), Research Handbook on the Law of Artificial Intelligence (Edward Elgar Publishing 2018), Link to .PDF

– F Möslein, ‘Regulating Robotic Conduct: On ESMA’s New Guidelines and Beyond’ in N Aggarwal and others (eds), Autonomous Systems and the Law (Beck, Nomos 2019) 45

– F Möslein ‘Leitlinien für den Einsatz künstlicher Intelligenz’ in D Linardatos (ed), Rechtshandbuch Robo-Advice (Beck, Vahlen 2020) 58

– Ethical Business Regulation:Understanding the Evidence, Christopher Hodges, Professor of Justice Systems, and Fellow of Wolfson College, University of Oxford February 2016. Link .PDF

– Ethical Theories, By Larry Chonko, Ph.D. The University of Texas at Arlington, Slides, Notes to slides

–  German Data Ethics Commission. The Data Ethics Commission presented its Opinion to the Federal Government on 23 October 2019 at a closing ceremony at the Federal Ministry of Justice and Consumer Protection.

• Opinion of the Data Ethics Commission (PDF, 2,673 KB, file is barrier-free)
• Opinion of the Data Ethics Commission – Executive Summary (PDF, 777 KB, file is barrier-free)

Big Data and its implication for AI regulation

–  Big Data and its implication for AI regulation. Roberto V. Zicari, GIZ Workshop on Regulatory Requirements in the context of Industry 4.0 Hanoi Province, November 12, 2021: Zicari.GIZ.PartII12.11.2021

AI Certifications

–  The IEEE CertifAIEd Mark. LINK

EU Funded AI Master Classes

– eXplainable Artificial Intelligence in healthcare Management (xAIM). LINK

– Master on AI for public services (AI4Gov). LINK

– Master in AI for Careers in EU (MAI4CAREU). LINK

– Human Centered AI Master. LINK