What is it like to experience sound while playing educational games? : an interpretive phenomenological investigation (2024)

Encouraging the use of immersive sonic design for learning through the interactive media content, 2021

The educational system aspires to use interactive tools more often as it proved to benefit a learning process. In media content, the design direction of the audio is a key element to the overall experience of the audience. Specifically, the good experience mostly comes from a sense of " presence experienced immersion. The present study experiment with " , a term used in psychology to describe an sonic design technics that can be used to combine an immersive experience to learning modules through the developmen t of a video game adventure, designed to experiment with the discussed theories in the literature review. A post mortem critical analysis of the project showed that it is possible to lead a player progression through the audio immersion, based on a learnin g process associating sounds and images, while gradually obscure from the latter. Then, a survey investigating the effectiveness of this practiceled research project. It identifies that audio immersion can be used ...

TechTrends, 2010

In this article, we describe the instructional potential of digital music to enhance postsecondary students’ experience in online courses by involving them in music-driven instructional activities. We describe how music-driven instructional activities, when used appropriately, can (a) humanize, personalize, and energize online courses by enhancing social presence through student-to-student interaction; (b) tap into students’ interests, and elicit positive feelings and associations; and (c) involve students in relevant and meaningful student-to-content interaction by engaging them in active knowledge construction. Finally, we share several music-driven instructional activities that rely on digital music resources to engage students in generative, multisensory student-content interactions that leverage their interest in music, as well as a set of guidelines to support the design and use of music-driven instructional activities in online courses.

The International Journal of Arts Education, 2021

Emotional capacity in music creates an innate human resource for well-being, learning, and interaction, inspired by sound environments, and integrated into life experiences through holistic, multisensory experiences. Sounds connect with natural instincts and cultural history. Today, the weak position of the arts strengthens the negative perception of them, shown in hardening societal values. In musical-artistic experiences, individuals and societies breathe better, and cultural phenomena and societies are thus recreated. This article proposes Multisensory Musical Design (MMD) as a musical cornerstone for human-oriented education. MMD is motivated by an understanding of human sensitive, social, and emotional qualities as they relate to tacit knowledge supported by music. I contemplate human artistic resources as a support for societal wellbeing, grounded in education. At the core of the MMD model, sounds and music create a natural power for human behavior via three basic concepts: faces, spaces, and timelines.Fetal multisensory processes linked with musical sounds for multidisciplinary dialogues are defined in music. The emotional dimension with flow (Csikszentmihalyi 2008), and the comprehension of multiple ways of knowledge with experience connect with music and tacit knowledge (Polanyi 1983), highlighted here as a foundation for human nature.

Proceedings of the 17th European Conference on e-Learning (ECEL), ACPI, 2018

Electronic computer games used for educational purposes currently constitute an essential component of the learning process. By combining entertaining elements with interactive technologies to deliver the respective curricula they can enhance the development of various skills, as well as serve as platforms for the application of modern educational theories. Audio Games (AG) are a particular genre of electronic games, in which all information is conveyed mainly or exclusively through sound. Thus, players need to employ their sense of hearing to understand and accomplish the necessary tasks, a process that promotes their concentration, memory, fantasy, emotion, perception, data management and cooperation. Even though research findings have shown that both game and audio interaction have positive effects on the user, there has been no systematic approach in designing educational AG and implementing them in the learning process. In this paper the authors attempt to establish the theoretical frame for the design of educational AG by arguing that their features comply with the goals of music education, as these are formulated in the official Program of Music Studies by the Greek Ministry of Education, and in the Primary Years Program on learning Arts by the International Baccalaureate Institution, on local and global level respectively. In that context, three different audio gamification approaches developed by the authors are discussed as suggestions for incorporation into formal education. The first one aims at raising users’ awareness about the harmful impact of noize on the acoustic environment, the second one introduces players to concepts and techniques of electronic music composition, whereas the third one informs them about the layers of Earth’s atmosphere. All three approaches employ similar mechanics, but each one addresses a different topic: acoustics, music, and even non- music respectively, suggesting that the application of educational AG can extend to a variety of subject matters.

Asia-Pacific Journal for Arts Education, 2023

Technological progress over the course of the past few decades has transformed how children interact with sound and music, offering new and extended ways of expression, creation, and learning. This digital environment can provide opportunities for constructing a framework of sound perception, musical praxis, and creativity enhancement, the emergence of cross-platforms and a growing variety of hardware and software serving to support these developments through an emerging context of ubiquitous acoustic ecologies. The aim of this research was to involve educational scenarios in music lessons following such ecological perspectives through a pilot study for children aged 7 to 9 in a conservatoire setting in Greece. Actions for the current practical intervention have been designed following a STEAM project-based learning approach, which offers students cooperative activities,transdisciplinarity, game-based, augmented reality, playful learning, and authentic problem-solving experiences. Analysis revealed four distinct, emerging thematic categories that drew on the development of auditory perceptual ability, creativity development, computational thinking cultivation, and the shaping of digital and physical musical worlds. The results of the educational intervention underlined the fundamental role of ubiquitousmusic ecologies in planned actions, which served to widen students’ musical horizons.

Computers & Education, 2014

Technology’s place in music education is largely related to how it is socially constituted. Despite how technology enables intersections of and blurred boundaries between ways of being musical, it is often situated in terms of hard boundaries and compartmentalized notions of musical engagement. Furthermore, music education often situates technology as tools without necessarily considering related social, cultural or musical contexts. This chapter addresses how philosophical, pedagogical, and curricular perspectives play a key role in the types and degree of change that occur in relation to technology and music education. To forward related praxis, I propose that music educators re-conceptualize curriculum and re-situate technology to address social and cultural issues explicitly. I invite music educators to consider the potential of digitally mediated musical engagement within the contexts of curriculum as experience and as social reconstruction. The chapter considers how such change might occur and conceptual frameworks that might help in forwarding such work.

Oxford Handbook of Interactive Audio, 2014

The question of how interactive music should function in games is perhaps a misleading one, as there are many different types of games and many different types of players. One of the most compelling explanations for the huge popularity of video games is that they meet people's intrinsic psychological needs quickly, with consistency, and with great frequency (Rigby, 2010). The apparent drivers of the development of games and their marketing-such as the fidelity of graphics and audio, or as the popular press would have us imagine, the degree of violence-are far less significant factors than the drive to increase our sense of well-being through meeting the basic needs of competence (or mastery), autonomy (or volition) and relatedness (social connection) (Przblinkski, 2009) or the desire to become immersed in narrative worlds (Cairns, 2006). Since it is clear that player satisfaction is a product of " needs met " over " needs " , it is important that we recognize that music should operate in different ways in different circ*mstances.

This anthology presents research projects that examine the intersection between music, technology and education from a variety of perspectives. The contributors are from a range of educational programs within traditional pre-, primary and lower secondary school education, as well as music performance and technology educational programs. Data for the studies stems from primary and lower secondary school, as well as informal learning environments, in addition to the contributors’ respective education programs. The research projects examine a wide range of topics such as gamification of ukulele and song teaching, composition with iPads in the classroom, live looping as an approach to ensemble conducting, authentic music technology learning spaces, music-making in the “laptop-era”, sound, the notion of net-based presence, and challenges in higher electronic music education. As this anthology is the first publication in the MusPed:Research series, it also contains an introductory chapter about the series and the research network Musikkpedagogikk i utdanning (MiU). This anthology makes a distinct contribution to the research field of music technology in education and questions educational practices in the school and higher educational levels, the goals and content of music education, and our understanding of music and music creation in itself.

Interaction design research often models philosophical and theoretical principles in concrete form and the observation of how these ideas interact with users is a commonplace research activity. This approach is strongly influenced by ideas about reflection in action. The musician’s actions are also creative and reflection in and on action is also commonplace but because musical practice is ephemeral and does not always leave behind an object to reflect upon this presents a problem. Sound and video recording technologies have been used in music education to provide objects for reflection. The use of digital capture of musical events is an important basis for ways in which computers can assist music education.

PhD thesis, 2017

This research project deals with the user-experience related to embodied music mediation technologies. More specifically, adoption and policy problems surrounding new media (art) are considered, which arise from the usability issues that to date pervade new interfaces for musical expression. Since the emergence of new wireless mediators and control devices for musical expression, there is an explicit aspiration of the creative industries and various research centers to embed such technologies into different areas of the cultural industries. The number of applications and their uses have exponentially increased over the last decade. Conversely, many of the applications to date still suffer from severe usability problems, which not only hinder the adoption by the cultural sector, but also make culture participants take a rather cautious, hesitant, or even downright negative stance towards these technologies. Therefore, this thesis takes a vantage point that is in part sociological in nature, yet has a link to cultural studies as well. It combines this with a musicological frame of reference to which it introduces empirical user-oriented approaches, predominantly taken from the field of human-computer-interaction studies. This interdisciplinary strategy is adopted to cope with the complex nature of digital embodied music controlling technologies.Within the Flanders cultural (and creative) industries, opportunities of systems affiliated with embodied interaction are created and examined. This constitutes an epistemological jigsaw that looks into 1) “which stakeholders require what various levels of involvement, what interactive means and what artistic possibilities?”, 2) “the way in which artistic aspirations, cultural prerequisites and operational necessities of (prospective) users can be defined?”, 3) “how functional, artistic and aesthetic requirements can be accommodated?”, and 4) “how quality of use and quality of experience can be achieved, quantified, evaluated and, eventually, improved?”. Within this multi-facetted problem, the eventual aim is to assess the applicability of the foresaid technology, both from a theoretically and empirically sound basis, and to facilitate widening and enhancing the adoption of said technologies.Methodologically, this is achieved by 1) applied experimentation, 2) interview techniques, 3) self-reporting and survey research, 4) usability evaluation of existingxiiidevices, and 5) human-computer interaction methods applied – and attuned – to the specific case of embodied music mediation technologies. Within that scope, concepts related to usability, flow, presence, goal assessment and game enjoyment are scrutinized and applied, and both task- and experience-oriented heuristics and metrics are developed and tested.In the first part, covering three chapters, the general context of the thesis is given. In the first chapter, an introduction to the topic is offered and the current problems are enumerated. In the second chapter, a broader theoretical background is presented of the concepts that underpin the project, namely 1) the paradigm of embodiment and its connection to musicology, 2) a state of the arts concerning new interfaces for musical expression, 3) an introduction into HCI-usability and its application domain in systematic musicology, 4) an insight into user-centered digital design procedures, and 5) the challenges brought about by e-culture and digitization for the cultural-creative industries. In the third chapter, the state of the arts concerning the available methodologies related to the thesis’ endeavor is discussed, a set of literature-based design guidelines are enumerated and from this a conceptual model is deduced which is gradually presented throughout the thesis, and fully deployed in the “SoundField”-project (as described in Chapter 9).The following chapters, contained in the second part of the thesis, give a quasi- chronological overview of how methodological concepts have been applied throughout the empirical case studies, aimed specifically at the exploration of the various aspects of the complex status quaestionis. In the fourth chapter, a series of application-based tests, predominantly revolving around interface evaluation, illustrate the complex relation between gestural interfaces and meaningful musical expression, advocating a more user- centered development approach to be adopted. In the fifth chapter, a multi-purpose questionnaire dubbed “What Moves You” is discussed, which aimed at creating a survey of the (prospective) end-users of embodied music mediation technologies. Therefore, it primarily focused on cultural background, musical profile and preferences, views on embodied interaction, literacy of and attitudes towards new technology and participation in digital culture. In the sixth chapter, the ethnographical studies that accompanied the exhibition of two interactive art pieces, entitled "Heart as an Ocean" & "Lament", are discussed. In these studies, the use of interview and questionnaire methodologies together with the presentation and reception of interactive art pieces, are probed. In the seventh chapter, the development of the collaboratively controlled music-game “Sync-In-Team” is presented, in which interface evaluation, presence, game enjoyment and goal assessment are the pivotal topics. In the eighth chapter, two usability studies are considered, that were conducted on prototype systems/interfaces, namely a heuristic evaluation of the “Virtual String” and a usability metrics evaluation on the “Multi-Level Sonification Tool”. The findings of these two studies in conjunction with the exploratory studies performed in association with the interactive art pieces, finally gave rise to the “SoundField”-project, which is recounted in full throughout the ninth chapter. The integrated participatory design and evaluation method, presented in the conceptual model is fully applied over the course of the “SoundField”-project, in which technological opportunities and ecological validity and applicability are investigated through user- informed development of numerous use cases.The third and last part of the thesis renders the final conclusions of this research project. The tenth chapter sets out with an epilogue in which a brief overview is given on how the state of the arts has evolved since the end of the project (as the research ended in 2012, but the research field has obviously moved on), and attempts to consolidate the implications of the research studies with some of the realities of the Flemish cultural- creative industries. Chapter eleven continues by discussing the strengths and weaknesses of the conceptual model throughout the various stages of the project. Also, it comprises the evaluation of the hypotheses, how the assumptions that were made held up, and how the research questions eventually could be assessed. Finally, the twelfth and last chapter concludes with the most important findings of the project. Also, it discusses some of the implications on cultural production, artistic research policy and offers an outlook on future research beyond the scope of the “SoundField” project.

This paper presents MINUET, a framework for musical interface design grounded in the experience of the player. MINUET aims to provide new perspectives on the design of musical interfaces, referred to as a general term that comprises digital musical instruments and interactive installations. The ultimate purpose is to reduce the complexity of the design space emphasizing the experience of the player. MINUET is structured as a design process consisting of two stages: goal and specifications. The reliability of MINUET is tested through a systematic comparison with the related work and through a case study. To this end, we present the design and prototyping of Hexagon, a new musical interface with learning purposes.