Pilot Project: Assessing Trustworthiness of the use of Generative AI for higher Education.

This pilot project of the Z-inspection® initiative  (https://z-inspection.org) aims at assessing the use of Generative AI in higher level education considering specific use cases. 

For this pilot project, we will assess the ethical, technical, domain-specific (i.e. education) and legal implications of the use of Generative AI-product/service within the university context.

We follow the UNESCO guidance for policymakers on AI and education. In particular the policy recommendation 6. : Pilot testing, monitoring and evaluation, and building an evidence base.

Expected Output: a white paper and a peer reviewed journal article as best practice and a set of recommendations for each specific use case. Such recommendations could be also useful for helping clarify the Guidelines that each university is creating for this.

Participants (over 140 participants from around the world! See list below)

– Affiliated Labs: https://z-inspection.org/affiliated-labs/

– Members of the Z-inspection® initiative: https://z-inspection.org

– Ministries and Universities

– Specialized agencies

– Others

Distributed steering committees

The pilot project is lead by distributed steering committees by geographical areas, as follows:

– Australia, Malaysia and South Korea:

Members

Caslon Chua, Acting Department Chair,

Computing Technologies, School of Software and Electrical Engineering, Department of Computer Science and Software Engineering

Swinburne University of Technology, Melbourne, Australia

Expertise: Computer Science

Catarina Moreira, Associate Professor | UNESCO Co-Chair AI&XR, Lead Research Scientist, HTI

Human Technology Institute,  (HTI), University of Technology Sydney 

Expertise: explainable AI, more specifically explainable user interfaces and counterfactual explanations; causality and probabilistic graphical models; eye-tracking for deep learning

Sian Lun Lau, Professor and Head, Department of Engineering, Associate Dean (Internationalisation)
Cluster Lead, Sunway DataXSight Research Cluster, School of Engineering and Technology, Sunway University, Malasya

Expertise: Computer Science

BeeTheng Lau,  The Laboratory for Ethical and Trustworthy AI in Practice at the Swinburne University of Technology Sarawak Campus (Sarawak, Malaysia)

Expertise: Computer Science

– Canada

Coordinators:

Somaieh Nikpoor , The Trustworthy AI Lab @ TIM, Carleton University (Ottawa, Canada)

Expertise: ethics, certified Z-Inspection® instructor.

Andreane Sabourin Laflamme, Ethical Assessment for Trustworthy AI at LEN.IA – AI & Digital Ethics Lab (Montréal, Québec, Canada).

Expertise: ethics, 

Members

Frédérick Bruneault , Ethical Assessment for Trustworthy AI at LEN.IA – AI & Digital Ethics Lab (Montréal, Québec, Canada).

Expertise: ethics, certified Z-Inspection® instructor

Martin Petrin , Western University, Canada  

Expertise: Law.

–  Europe and UK:

Coordinators:

Elisabeth Hildt , The Trustworthy AI Lab at L3S Research Center Leibniz, University Hannover (Hannover, Germany)

Expertise: ethics, certified Z-Inspection® instructor

Magnus Westerlund ,   The Laboratory for Trustworthy AI at Arcada University of Applied Sciences (Helsinki, Finland). Finland

Expertise: AI, certified Z-Inspection® instructor

Henrika Franck , The Laboratory for Trustworthy AI at Arcada University of Applied Sciences (Helsinki, Finland). Finland

Expertise: ethics,

Emilie Wiinblad Mathez , Switzerland

Expertise: law, human rights, certified Z-Inspection® instructor

Members

Alen, Anette K , Faculty of Law University of Helsinki, Finland

Expertise: Law

Alessio Gallucci, The Netherlands

Expertise: AI , certified Z-Inspection® instructor

Frode Ramstad Johansen , Trustworthy AI Lab at the University College Østfold, (Fredrikstad, Østfold, Norway)

Expertise: Innovation

Pedro  Kringen, Trustworthy AI Lab at the University College Østfold, (Fredrikstad, Østfold, Norway)

Expertise: certified Z-Inspection® instructor

Pedro Moreno Sanchez, Trustworthy AI for healthcare Lab, Tampere University (Finland)

Expertise: certified Z-Inspection® instructor

– India:

Members:

Gawande, Gokul, The Trustworthy AI in Practice lab at the DY Patil College of Engineering, Akurdi Campus, (Pune, India)

Expertise: technical aspects and overall team oversight activities in India.

Srivastava, Adarsh,  The Trustworthy AI in Practice lab at the DY Patil College of Engineering, Akurdi Campus, (Pune, India)

Expertise: technical aspects and overall team oversight activities in India.

Himanshi Allahabadi , The Trustworthy AI in Practice lab at the DY Patil College of Engineering, Akurdi Campus, (Pune, India)

Expertise: Generative AI from a broad sociotechnical perspective. More specific NLP and LLMs.

Divya Dwivedi, Advocate, Supreme Court of India

Expertise: Law.

Sharmistha Chatterjee,  The Trustworthy AI in Practice lab at the DY Patil College of Engineering, Akurdi Campus, (Pune, India)

Expertise: AI/ML, Cloud Computing, MLOps,   building large-scale distributed ethical AI systems, Responsible AI, LLMs

Ishwar Chavan, The Trustworthy AI in Practice lab at the DY Patil College of Engineering, Akurdi Campus, (Pune, India)
Expertise:  AI technical, ML technical aspects.

Vinayak Kottawar , D Y Patil College of Engineering, India

Expertise: AI technical, SME (Education)

Alessio Gallucci, The Netherlands

Expertise: AI, certified Z-Inspection® instructor

–  USA:

Members

Megan Coffee, NYU Grossman School of Medicine, USA

Expertise: certified Z-Inspection® instructor

Elisabeth Hildt , The Ethical and Trustworthy AI Lab at Illinois Institute of Technology’s Center for the Study of Ethics in the Professions (Chicago, USA).

Expertise: ethics, certified Z-Inspection® instructor

 Andy Spezzatti, Taltrics and AI for Good Foundation, USA

Expertise: AI technical.

Approach

An interdisciplinary team of experts will assess the trustworthiness of Generative AI for selected use cases in High Education using the Z-Inspection® process: https://z-inspection.org 

Z-Inspection® is a holistic process based on the method of evaluating new technologies, where ethical issues need to be discussed through the elaboration of socio-technical scenarios. In particular, Z-Inspection® can be used to perform independent assessments and/or self-assessments together with the stakeholders owning the use case. 

For the context of this pilot project we define ethics in line with the essence of modern democracy i.e. “respect for others, expressed through support for fundamental human rights”. We take into consideration that “trust” in the development, deployment and use of AI systems concerns not only the technology’s inherent properties, but also the qualities of the socio-technical systems involving AI applications.

Specifically, we consider the ethics guidelines for trustworthy artificial intelligence defined by the EU High-Level Expert Group on AI, which defined trustworthy AI as:

(1) lawful – respecting all applicable laws and regulations 

(2) ethical – respecting ethical principles and values 

(3) robust – both from a technical perspective and taking into account its social environment

And we use the four ethical principles, rooted in fundamental rights defined in [13], acknowledging that tensions may arise between them:

(1) Respect for human autonomy 

(2) Prevention of harm 

(3) Fairness 

(4) Explicability 

Furthermore, we also consider the seven requirements of Trustworthy AI defined by the High Level experts group set by the EU. Each requirement has a number of sub-requirements as indicated in Table 1.

Table 1. Requirements and sub-requirements Trustworthy AI. 

1 Human agency and oversight Including fundamental rights, human agency and human oversight

2 Technical robustness and safety Including resilience to attack and security, fall back plan and general safety, accuracy, reliability and reproducibility

3 Privacy and data governance Including respect for privacy, quality and integrity of data, and access to data

4 Transparency Including traceability, explainability and communication 

5 Diversity, non-discrimination and fairness Including the avoidance of unfair bias, accessibility and universal design, and stakeholder participation

6 Societal and environmental wellbeing Including sustainability and environmental friendliness, social impact, society and democracy

7 Accountability Including auditability, minimization and reporting of negative impact, trade-offs and redress.

While we consider the seven requirements comprehensive, we believe additional ones can still bring value. Two of such additional requirements proposed by the Z-Inspection® initiative are “Assessing if the ecosystems respect values of Western Modern democracy” and “Avoiding concentration of power” .

………………………………………………………………………………………………………………………………………………………..

From the UNESCO guidance for policymakers on AI and education sets out policy recommendations in seven areas:

(https://unesdoc.unesco.org/ark:/48223/pf0000376709)

1. A system-wide vision and strategic priorities 

2. Overarching principle for AI and education policies 

3. Interdisciplinary planning and inter-sectoral governance 

4. Policies and regulations for equitable, inclusive, and ethical use of AI 

5. Master plans for using AI in education management, teaching, learning, and assessment

6. Pilot testing, monitoring and evaluation, and building an evidence base 

7. Fostering local AI innovations for education 

……………………………………………………………………………………………………………………………………………………………….

from the Guidelines for the use of AI in teaching at the University of Helsinki Academic Affairs Council 16.2.2023

(https://teaching.helsinki.fi/instructions/article/artificial-intelligence-teaching)

“Large artificial intelligence (AI)-based language models such as Chat GPT, Google Bard, and DeepL have evolved to the point where they can produce human-like text and conversations and correct and transform text at such a high level that it can be difficult to distinguish the result from human- generated text. It is foreseeable that more such models will emerge, and their functionalities will continue to evolve, so their existence should be taken into account in university teaching and research.

The existence of large language models should be seen as an opportunity. Degree programmes and teachers are encouraged to use AI in their teaching and to prepare students for a society of the future where AI methods will be widely used.

As AI brings new possibilities for producing text whose origin and reliability is unclear, they should be used in a controlled way. Use may be restricted in teaching in situations where the use would not promote student learning.

At EU level, an AI regulation is under preparation, which will also apply to AI systems in education. In addition, there is an ethical policy on AI and its use, as well as an ethical code for teachers1 . The University’s guidelines may be further specified in the light of future regulation and technological developments.” 

______________________________________________________

Resources

1. Policy paper. Generative artificial intelligence in education. UK: The Department for Education’s (DfE) position on the use of generative artificial intelligence (AI) in High Education.  Link to .PDF

2Do Foundation Model Providers Comply with the Draft EU AI Act? Stanford researchers evaluate foundation model providers like OpenAI and Google for their compliance with proposed EU law on AI.  Link

They identified a final list of 12 requirements and scored the 10 models using a 5-point rubric. The methodology for the study can be found here.

3. Leading universities in the UK (the Russell Group universities) have developed a set of principles on the use of generative AI tools in education. Here is the link.

4. Frontier AI Regulation: Managing Emerging Risks to Public Safety arXiv:2307.03718 [cs.CY] LINK

5. Zhaki Abdullah, Students, teachers will learn to properly use tools like ChatGPT: (Singapore Education Minister) Chan Chun Sing, Straits Times, 12 February 2023, LINK

6. U.S. Department of Education, Office of Educational Technology, Artificial Intelligence and Future of Teaching and Learning: Insights and RecommendationsWashington D.C., May 2023, LINK

7. Japanese schools to be allowed limited use of generative AI, Kyodo News, 22 June 2023, LINK

8. Higher Education Webinar: Implications of Artificial Intelligence in Higher Education , Tuesday, June 27, 2023, Council on Foreign RelationsLINK

9. Novelli, C., Casolari, F., Rotolo, A. et al. Taking AI risks seriously: a new assessment model for the AI ActAI & Soc (2023). LINK

10. Academic without Borders, Bimonthly Newsletter n°59, July 2023.

11. On the use of artificial intelligence and in particular of ChatGPT in higher education. (UNESCO). Link to .PDF

12. KU Leuven, Responsible use of Generative Artificial Intelligence (GenAI) in research.These guidelines will be updated with new information and insights to keep them in line with the rapidly evolving technology (last updated June 24, 2023). The further integration of teaching and research guidelines is still on the agenda. LINK

13.Stanford HAI, ChatGPT Out-scores Medical Students on Complex Clinical Care Exam Questions. Jul 17, 2023 |  Adam Hadhazy

14. The Norwegian Consumer Council published a detailed report “Ghost in the machine – Addressing the consumer harms of generative AI” outlining the harms, legal frameworks, and possible ways forward. In conjunction with this launch, the Norwegian Consumer Council and 14 consumer organizations from across the EU and the US demand that policymakers and regulators act.  https://storage02.forbrukerradet.no/media/2023/06/generative-ai-rapport-2023.pdf

15. University of Melbourne: Inquiry into the use of generative AI in the education system. Submission to the House Standing Committee on Employment, Education and Training 14 July 2023. https://about.unimelb.edu.au/__data/assets/pdf_file/0032/396446/UoM-Submission-Inquiry-into-Generative-AI-in-Education-FINAL.pdf

16. Cornell University: Generative Artificial Intelligence for Education and Pedagogy.July 18, 2023. https://teaching.cornell.edu/sites/default/files/2023-08/Cornell-GenerativeAIForEducation-Report_2.pdf

17. The University of North Carolina at Chapel Hill: Teaching Use Guidelines for Generative Artificial Intelligencehttps://provost.unc.edu/wp-content/uploads/2023/07/Teaching-Generative-AI-Use-Guidance_UNC-AI-Committee-June-15-202348.pdf

18. University of Sydney: 13 March, 2023 Students answer your questions about generative AI – part 2: Ethics, integrity, and the value of university. https://educational-innovation.sydney.edu.au/teaching@sydney/students-answer-your-questions-about-generative-ai-part-2-ethics-integrity-and-the-value-of-university/

19. The Berkman Klein Center for Internet & Society at Harvard University: Exploring the Impacts of Generative AI on the Future of Teaching and Learning  https://cyber.harvard.edu/story/2023-06/impacts-generative-ai-teaching-learning

20. Stanford University: Pedagogic strategies for adapting to generative AI chatbots. Eight strategic steps to help instructors adapt to generative AI tools and chatbots. June 19, 2023, Center for Teaching and Learning. https://docs.google.com/document/d/1la8jOJTWfhUdNna5AJYiKgNR2-54MBJswg0gyBcGB-c/edit

21. Council of Europe: ARTIFICIAL INTELLIGENCE AND EDUCATION A critical view through the lens of human rights, democracy and the rule of law.November 2022

https://rm.coe.int/artificial-intelligence-and-education-a-critical-view-through-the-lens/1680a886bd

22. ANU Centre for Learning and Teaching: Chat GPT and other generative AI tools: What ANU academics need to know February 2023. https://teaching.weblogs.anu.edu.au/files/2023/02/Chat_GPT_FAQ-1.pdf

23.Guidance for Generative AI in education and research | UNESCO, 7 September 2023

24. The Canadian federal government just released its guidelines for using generative AI. These are guidelines for the GoC’s internal use of generative AI.

………………………………………………………………………………………………………………………………………………………………………………………………………………………….

Participants

First NameLast NameAffiliationCountry
AasthaPantMonash UniversityAustralia
CatarinaMoreiraQueensland University of Technology (QUT)Australia
SubrataChakrabortyUniversity of New EnglandAustralia
VishalRanaGriffith UniversityAustralia
CaslonChuaSwinburne University of TechnologyAustralia
ChrisMcCarthy Swinburne University of TechnologyAustralia
EnzoPalomboSwinburne University of TechnologyAustralia
BitaZaferanlooSwinburne University of TechnologyAustralia
RashidianNiki Universitair Ziekenhuis GentBelgium
RobHEYMANVrije Universiteit BrusselBelgium
MartinPetrinWestern University, Canada Belgium
AnthonyNovaesUniversidade Presbiteriana Mackenzie Brazil
TodorIvanovDXC TechnologyBulgaria
AndréaneSabourin LaflammeÉcole des médias at the Université du Québec à Montréal (UQAM)Canada
EmmanuelleMarceauÉcole de santé publique de l’Université de Montréal (ESPUM)Canada
CameronPiersonLeibniz Universität HannoverCanada
ParisaOsivandUniversity of Toronto Canada
FrédérickBruneaultÉcole des médias at the Université du Québec à Montréal (UQAM)Canada
TonyBailettiCarleton UniversityCanada
SomaiehNikpoorCarleton UniversityCanada
StoyanTanevCarleton UniversityCanada
LianaRazmeritaCBSDenmark
HelleCollatz ChristensenUniversity of CopenhagenDenmark
ChristofferBjerre Haase  University of CopenhagenDenmark
JohnBrandt Brodersen University of CopenhagenDenmark
BorisDüdderUniversity of CopenhagenDenmark
DanielHershcovichUniversity of CopenhagenDenmark
Sonia Claudiada Costa SousaTallinn UniversityEstonia
DavidLamasTallinn UniversityEstonia
JoannaKarajanovFinland
LaineSamiFinland
MarkkuSuksiÅbo Academi UniversityFinland
CamillaWikström-GrotellArcada University of Applied SciencesFinland
Markvan GilsEYTampere University,
Faculty of Medicine and Health Technology,
Finland
PedroMoreno SánchezSeinäjoki University of Applied SciencesFinland
AnetteAlénUniversity of HelsinkiFinland
JarnoTuominenUniversity of TurkuFinland
LeonardoEspinosaArcadaFinland
MatteoStocchettiArcada University of Applied ScienceFinland
MagnusWesterlundArcada University of Applied SciencesFinland
HannaSormunenFinnish Tax AdministrationFinland
ValérieSaintotFrance
PaulGardinerDocumentum, Inc.SKEMA Business School France
BarbaraSeeligerUniversity Hospitals of StrasbourgFrance
EunikaMercier-Laurent EpitaUniversity of Reims France
EmmanuelGoffiHuman Technology FoundationFrance
BirgittaDrespCNRS, University of StrasbourgFrance
ValerieBürgerBerlin Institute of Health 
Charité (BIH)
Germany
VinceIstvan MadaiBerlin Institute of Health 
Charité (BIH)
Germany
DagmarMonett DíazBerlin School of Economics and Law (HWR Berlin)Germany
UlrichKühneHautmedizin Bad SodenGermany
EduardBuzilaOtto-von-Guericke- Universität MagdeburgGermany
Ludwig ChristianHinskeUniversitätsklinikum AugsburgGermany
StephanBraunUniversitätsklinikum MünsterGermany
MarieAlaviUniversity of Applied Sciences Ostwestfalen-Lippe Germany
ThiloHagendorffUniversity of StuttgartGermany
DanielAmthorAmelia LLC, NYGermany
GemmaRoigGoethe University FrankfurtGermany
DennisVetterGoethe University FrankfurtGermany
ElisabethHildtLeibniz, University Hannover Germany
MichaelLeyerPhilipps-Universität Germany
FlorianMösleinPhilipps-Universität Germany
HolgerVollandbrand eins Medien AGGermany
WolfgangNejdluni hannoverGermany
KarstenTolleGoethe University FrankfurtGermany
MathiasMentzingAnima_Alma NetworkGermany
AlessandroDonatiGermany
TanusreeDeAccentureIndia
HimanshiAllahabadiEYIndia
SharmisthaChatterjeeIndia
DivyaDwivedi Supreme Court of IndiaIndia
GokulGawandeRoche Information Solutions India Pvt. Ltd.India
IshwarChavanUniversity of BresciaIndia
Vinayak KottawarIndia
Usha RengarajuIndia
AdarshSrivastavaRoche Information SolutionsIndia
GeorgeKararigasUniversity of IslandIsland
JosephDavid Sapir Academic CollegeIsrael
DilettaHuyskesFondazione Bruno Kessler (FBK)Italy
FrancescoBalzanUniversità di BolognaItaly
AlbertoSignoroniuniversità di BresciaItaly
DavideFarinaUniversity of BresciaItaly
MattiaSavardiUniversity of BresciaItaly
SilviaZulloUniv. BolognaItaly
GayaneAghakhanyanUniversity of PisaItaly
MoniqueMunariniUniversity of PisaItaly
ClaudioFanniUniversità di PisaItaly
RobertoFrancischelloUniversità di PisaItaly
FrancescaCaputoUniversity of PisaItaly
DaniaCioniUniversity of PisaItaly
EmanueleNeriUniversity of PisaItaly
LorenziTumminelloUniversity of PisaItaly
AliceStoccoCa Foscari Univ. VeniceItaly
Yueh-HsuanWengFRIS, Tohoku UniversityJapan
YasamanYousefiUniversity of LuxemburgLuxembourg
SianLun LauSunway UniversityMALAYSIA
MarkTeeKit TsunSwinburne University of TechnologyMalaysia
BeeTheng LauSwinburne University of TechnologyMalaysia
FrodeRamstad JohansenØstfold University CollegeNorway
GunnarAnderssonZ-Inspection® InitiativeNorway
PedroKringenZ-Inspection® InitiativeNorway
KatarzynaKaczmarek-MajerPolish Academy of SciencesPoland
MickieDe Wet  Z-Inspection® Initiative Portugal
AssiatuJallohSierra Leone
RalfBeuthanMyongji UniversitySouth Korea
AlvaroFuentesJeonbuk National University South Korea
PatrickOseweAsian Development Bank South Korea
JasonJungChung-Ang UniversitySouth Korea
Se-youngJungDepartment of hospital information, Seoul National University Bundang Hospital.South Korea
Cornelius KalenziKorea Advanced Institute of Science and Technology (KAIST)South Korea
HyunjooJUNGSeoul National UniversitySouth Korea
GiEunKimSeoul National UniversitySouth Korea
HeejinKimSeoul National UniversitySouth Korea
JaeminLeeSeoul National UniversitySouth Korea
SeunggeunLeeSeoul National UniversitySouth Korea
HyunwooParkSeoul National UniversitySouth Korea
Nam-JongPaikSeoul National University – Bundang Hospital South Korea
Hyun-WooShinSeoul National University College of Medicine
Department of Biomedical Sciences, Seoul National 
South Korea
SeongwookHeoSeoul National University Law SchoolSouth Korea
JihaeSuhSeoul National University of Science and Technology,South Korea
JaydenYoonSouth Korea
Kyu SupHAHNSeoul National UniversitySouth Korea
DongjinLeeSeoul National University South Korea
NikoHildebrandtSouth Korea/ Canada
SiraMalipholThe State University of New York, Songdo, IncheonSouth Korea
GuillermoDe Haro RodriguezIE Business School, MadridSpain
KarinTafurZ-Inspection® Initiative Spain
WalterOsikaKarolinska InstitutetSweden
EmilieWiinblad MathezUNHCR, the UN Refugee Agency, Z-Inspection® Initiative Switzerland
AlesssioGallucciEindhoven University of Technology, Z-Inspection® InitiativeThe Netherlands
RogerBonsOpen UniversityThe Netherlands
NadineRoijakkersOpen UniversityThe Netherlands
CamillaTerziThe Netherlands
JudyWawira GichoyaEmory University  School of MedicineU.S.A
ReneeWurthHarvard School of Public Health, Z-Inspection® InitiativeU.S.A
AngelosChatzimparmpasNorthwestern UniversityU.S.A
MeganCoffeeNYU Grossman School of Medicine and Columbia Mailman School of Public HealthU.S.A
JesminJahann TithiZ-Inspection® Initiative , Intel U.S.A
PabloGarcia MolinaGeorgetown UniversityU.S.A
SonooThadaney IsraniStanford MedicineU.S.A
JamesBrusseauPhilosophy, Pace UniversityU.S.A
AndySpezzattiTaltrics, AI for Good FoundationU.S.A
PaulWallerThorney Isle ResearchUnited Kingdom
EleanoreHickmanUniversity of BristolUnited Kingdom
JonasBozenhardUniversity of Oxford United Kingdom