Abstract
The historical model of user-centred design (UCD) is based on the creation of deterministic graphical and physical interfaces optimised for human perception. The integration of autonomous agents based on Large Language Models (LLMs) and equipped with action capabilities (tool use) is revolutionising this paradigm: the interface no longer addresses humans directly, but becomes an intermediate semantic layer orchestrated by and for the machine. This article analyses this epistemological shift through a comprehensive industrial case study: the design of a next-generation internet box combining a hardware unit, an embedded screen, a companion mobile app and home network/home automation services. We demonstrate how the designer’s workflow is transforming, shifting from the design of static screens to the engineering of ontologies, the definition of semantic action capabilities (tools) and the configuration of agency sliders. Finally, we propose a framework of mixed evaluation metrics (AIX) that enables the computational performance of agents to be assessed whilst preserving the intelligibility and decision-making autonomy of human users.
Keywords: Human-Computer Interaction (HCI); Agent-Centred Design (ACD); Artificial Intelligence Experience (AIX); IoT & Smart Home; Semantic Affordance; Ephemeral Interfaces; Autonomy Governance; Governance of Autonomy
References
- Amershi S., et al. “Guidelines for human-AI interaction”. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019): 1-13.
- Bainbridge L. “Ironies of automation”. Automatica 19.6 (1983): 775-779.
- Chiou L-Y., et al. “Designing with AI: An exploration of co-ideation with image generators”. In: Proceedings of the 2023 ACM Designing Interactive Systems Conference (2023): 1410-1424.
- Dove G., et al. “UX design innovation: Challenges for working with machine learning as a design material”. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (2017): 278-288.
- Gibson JJ. “The ecological approach to visual perception”. Houghton Mifflin, Boston (1979).
- Heer J. “Agency plus automation: Designing artificial intelligence into interactive systems”. Proceedings of the National Academy of Sciences 116.6 (2019): 1844-1850.
- Inie N, Falk J and Tanimoto S. “Designing participatory AI: Creative professionals' worries and expectations about generative AI”. In: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (2023): 1-8.
- Nielsen J. “Usability engineering”. Academic Press, San Diego (1993).
- Norman DA. “The psychology of everyday things”. Basic Books, New York (1988).
- Shneiderman B. “Human-centered artificial intelligence: Reliable, safe & trustworthy”. International Journal of Human-Computer Interaction 36.6 (2020): 495-504.
- Shneiderman B. “Human-centered AI”. Oxford University Press, Oxford (2022).
- Subramonyam H., et al. “Solving separation-of-concerns problems in collaborative design of human-AI systems through leaky abstractions”. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (2022): 1-16.
- Vosgiens M, Olry A and Vivian R. “When AI becomes the user: Redefining the boundaries of human-machine interaction in the age of AIX”. Manuscript submitted for publication (2026).
- Yang Q., et al. “Re-examining whether, why, and how human-AI interaction is uniquely difficult to design”. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020): 1-13.
- Zamfirescu-Pereira JD., et al. “Why Johnny can't prompt: How non-AI experts try (and fail) to design LLM prompts”. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (2023): 1-21.