How could AI fool the humans responsible for critical infrastructure?

Technology Catalysts for Malevolent Actors

This project has concluded.

So What?

The rapid growth of AI and its widespread adoption has opened the potential for threat actors to use this technology to compromise critical infrastructure operations, such as water supply, agriculture, public health, internet, mobile networks, and more. As AI technologies increasingly become a primary tool for humans working in critical infrastructure, researchers have raised concerns about human ability to detect hacked or compromised AI agents.

Project Summary

Given AI's integration into infrastructure operations, this project highlights the effect of human-machine teaming on trust in or acceptance of AI machine agents. Understanding this is crucial for identifying vulnerabilities to critical infrastructure and developing tactics to combat these vulnerabilities.

Purpose/Objectives

This research aims to build knowledge on how VR may moderate the relationship between human and machine agents in complex problem-solving environments. The study does this by characterizing ways in which emerging technologies (VR and AI agents) may be used to manipulate human operators within critical infrastructure applications. The project will highlight how the widespread use of such technology relates to current and emerging forms of terrorism.

Method

The NCITE project team will conduct the study using three research methodologies:

• A systematic review of literature to gather key findings on the use of human-AI teaming, applications, vulnerabilities, and existing tactics for threat mitigation
• A mixed-methods study characterizing human-AI team performance in complex collaborative problem-solving tasks for critical infrastructure
• Empirical investigations to assess the impact of malign actors in human-AI teams, exploring the role of these actors in virtual environments

Research Team

• Penn State University
• Associate Professor of Engineering Design and Mechanical Engineering
• College of Engineering
• Director of the Institute for Computational and Data Sciences Center for Immersive Experiences (CIE)
• Expertise: Engineering design and mechanical engineering

• Penn State University
• Professor of Engineering Design and Industrial Engineering
• College of Engineering
• Director of the Center for Research in Design and Innovation
• Expertise: Ergonomic product design, design cognition, and human-computer interaction