The Ethics of Artificial Intelligence (AI), AI and Trust.
 Ethical and societal implications of algorithms, data, and artificial intelligence: a roadmap for research. Whittlestone, J. Nyrup, R. Alexandrova, A. Dihal, K. Cave, S. (2019), London. Nuffield Foundation. Link to .PDF
 Ethics Guidelines for Trustworthy AI. Independent High-Level Expert Group on Artificial Intelligence. European commission, 8 April, 2019. Link to .PDF
 WHITE PAPER. On Artificial Intelligence – A European approach to excellence and trust. European Commission, Brussels, 19.2.2020 COM(2020) 65 final. Link to .PDF
Ethics, Moral Values, Humankind, Technology, AI Examples.
 Perspectives on Issues in AI Governance, Lynette Webb, Charina Chou, Google White Paper, 2019. Link to .PDF
 AI on the Case: Legal and Ethical Issues. Richard Austin, Deeth Williams Wall LLP , May 17, 2019. Link to .PDF
 Report on Algorithmic Risk Assessment Tools in the U.S. Criminal Justice System, Partnership on AI, 2019. Link
Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice (February 13, 2019). Richardson, Rashida and Schultz, Jason and Crawford, Kate, 94 N.Y.U. L. REV. ONLINE 192 (2019). Available at SSRN
Fairness, Bias and Discrimination in AI. From Philosophy to Machine Learning.
 Improving Fairness in Machine Learning Systems: What Practitioners Need? K. Holstein et al. CHI 2019; May 4-0, 2019. Link to .PDF
 Ensuring, Fairness in Machine Learning to Advance Health, Alvin Rajkomar et al. Equity, Annals of Internal Medicine (2018). DOI: 10.7326/M181990. Link to .PDF
 Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements. Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Allison Woodruff, Christine Luu, Pierre Kreitmann, Jonathan Bischof, Ed H. Chi (Submitted on 14 Jan 2019). Link to .PDF
 Dissecting racial bias in an algorithm used to manage the health of populations. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Science. 2019 Oct 25;366(6464):447-453. doi: 10.1126/science.aax2342. Link to .PDF
AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias, Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, Yunfeng Zhang, 2018, Paper link, Open Source project link, Published as IBM Journal of Research and Development 63(4/5), 2019
AI: Explainability, Transparency.
 Experiences with Improving the Transparency of AI Models and Services. Michael Hind, Stephanie Houde, Jacquelyn Martino, Aleksandra Mojsilovic, David Piorkowski, John Richards, Kush R. Varshney (Submitted on 11 Nov 2019), arXiv:1911.08293v1. Link to .PDF
Petkovic D, Kobzik L, Re C. “Machine learning and deep analytics for biocomputing: call for better explainability”. Pacific Symposium on Biocomputing Hawaii, January 2018;23:623-7, Link to .PDF
Petkovic D, Kobzik L, Ganaghan R,“AI Ethics and Values in Biomedicine – Technical Challenges and Solutions”, Pacific Symposium on Biocomputing, Hawaii January 3-7, 2020, Link to .PDF
Gunning D, Aha D.:”DARPA’s Exianable Artificial Intelligence Program”, AI magazine, Association for the Advancement of Artificial Intelligence, Summer 2019, slides
Ribeiro M, Singh S, Guestrin C. “Why Should I Trust You? Explaining the Predictions of Any Classifier”, KDD ’16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningAugust, 2016, Link to .PDF or ACM PDF
Ribeiro M, Singh S, Guestrin C.: ”Nothing Else Matters: Model-Agnostic Explanations by Identifying Prediction Invariance”, 30th Conf. of Neural Information Processing Systems (NIPS 2016), Barcelona, Spain 2016, Link to .PDF
Petkovic D, Altman R, Wong M, Vigil A.: “Improving the explainability of Random Forest classifier – user centered approach”. Pacific Symposium on Biocomputing. 2018;23:204-15, Link to .PDF
D. Petkovic, A. Alavi, D. Cai, J. Yang, S. Barlaskar: “RFEX – Simple random Forest Model and Sample Explainer for non-ML experts”, Link to .PDF
Barlaskar S, Petkovic D: “Applying Improved Random Forest Explainability (RFEX 2.0) on synthetic data”, SFSU TR 18.01, 11/27/20181; with related toolkit at https://www.youtube.com/watch?v=neSVxbxxiCE
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques, Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilovic, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, Yunfeng Zhang, 2019, Link to .PDF
Explaining explainable AI, Michael Hind, XRDS: Crossroads, The ACM Magazine for Students 25(3), ACM, 2019, Link to .PDF
Explaining decisions made with AI. – ICO and The Alan Turing Institute, May 2020, Link
Human in the loop, Security, and Accountability.
Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020). Link to. PDF
Concrete Problems in AI Safety, Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané. Link to PDF
Anomalous Instance Detection in Deep Learning: A Survey – Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K. Varshney, Dawn Song. Link to PDF
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims – Miles Brundage et al. Link to PDF
Introduction to Z-inspection. A framework to assess Ethical AI
“Z-inspection: Towards a process to assess Ethical AI” – Roberto V. Zicari – With contributions from: Irmhild van Halem, Matthew Eric Bassett, Karsten Tolle, Timo Eichhorn, Todor Ivanov, Jesmin Jahan Tithi. CSGI(Cognitive Systems Group) Talk, Oct.31, 2019, Youtube video, Link to .PDF
Assessing AI use cases. Socio-Technical Scenarios.
 Ethical Framework for Designing Autonomous Intelligent Systems. J Leikas et al. J. of Open Innovation, 2019, 5, 1. Link
Assessing AI use cases. Ethical tensions, Trade offs.
 Algorithmic Impact Assessment: A Practical Framework for Public Agency Accountability, Reisman D., Schultz J, Crawford K, Whittake M, AI Now, April 2018. Link to .PDF
 FactSheets: Increasing trust in AI services through suppliers declarations of conformity. Arnold, M.; Bellamy, R. K. E.; Hind, M.; Houde, S.; Mehta, S.; Mojsilovic ́, A.; Nair, R.; Natesan- Ramamurthy, K.; Olteanu, A.; Piorkowski, D.; Reimer, D.; Richards, J.; Tsay, J.; and Varshney, K. R. 2019. IBM Journal of Research & Development 63(4/5). Link to .PDF
 Datasheets for datasets. Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J. W.; Wallach, H.; Daume ́, III, H.; and Craw- ford, K. 2018.. In Proceedings of the Fairness, Accountability, and Transparency in Machine Learning Workshop. Link to .PDF
 IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, First Edition. Pp. 211 – 281.Link to .PDF
COVID-19 Rapid Evidence Review: Exit through the App Store?, Nuffield Foundation. Link
EMPOWERING CITIZENS AGAINST COVID-19 WITH AN ML-BASED AND DECENTRALIZED RISK AWARENESS APPSpeaker : Professor Yoshua Bengio (Mila), Friday, May 8, 2020Recording: here
AI Ethics in Healthcare
 Stealth research: Lack of peer‐reviewed evidence from healthcare unicorns, Ioana A. Cristea Eli M. Cahan John P. A. Ioannidis, European Journal of Clinical Investigation, 28 January 2019. Link
 Grote T, Berens P. On the ethics of algorithmic decision-making in healthcare, J Med Ethics 2019;0:1–7. doi:10.1136/medethics-2019-105586. Link to .PDF
 Schonberg D. Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications. International Journal of Law and Information Technology, 2019, Link Dorian Peters, et. al, Responsible AI- Two Frameworks for Ethical Design Practice. IEEE Transactions on Technology and Society, Vol. 1, No. 1, March 2020. Link to .PDF
Legal relevance of AI Ethics
1) OECD, ‘Recommendation of the Council on Artificial Intelligence’ (22 May 2019), Link
2) G20, ‘G20 Ministerial Statement on Trade and Digital Economy’ (9 June 2019), Link to .PDF
3) Ethics Guidelines for Trustworthy AI. Independent High-Level Expert Group on Artificial Intelligence. European commission, 8 April, 2019, Link to .PDF
4) F Möslein, ‘Robots in the boardroom: artificial intelligence and corporate law’ in W Barfield and U Pagallo (eds), Research Handbook on the Law of Artificial Intelligence (Edward Elgar Publishing 2018), Link to .PDF
5) F Möslein, ‘Regulating Robotic Conduct: On ESMA’s New Guidelines and Beyond’ in N Aggarwal and others (eds), Autonomous Systems and the Law (Beck, Nomos 2019) 45
6) F Möslein ‘Leitlinien für den Einsatz künstlicher Intelligenz’ in D Linardatos (ed), Rechtshandbuch Robo-Advice (Beck, Vahlen 2020) 58
1) Ethical Business Regulation:Understanding the Evidence, Christopher Hodges, Professor of Justice Systems, and Fellow of Wolfson College, University of Oxford February 2016. Link .PDF
2) Ethical Theories, By Larry Chonko, Ph.D. The University of Texas at Arlington, Slides, Notes to slides
German Data Ethics Commision The Data Ethics Commission presented its Opinion to the Federal Government on 23 October 2019 at a closing ceremony at the Federal Ministry of Justice and Consumer Protection.