Music, Mind and Technology Resources
Items having a green asterisk (*) are my work, they are free to use only for non-commercial purposes and giving credit of authorship.
They are here mainly for backup and despite they all work quite well, many could still be further revised.
Improvement on them is strongly encouraged as well as telling me about those improvements :)
Click on the "yellow link" to download a file or to open a page.
Feedback is most welcome. Contact me (click here).
They are here mainly for backup and despite they all work quite well, many could still be further revised.
Improvement on them is strongly encouraged as well as telling me about those improvements :)
Click on the "yellow link" to download a file or to open a page.
Feedback is most welcome. Contact me (click here).
* Master's Thesis
Self-Report Measurement of Segmentation, Mimesis and Perceived Emotions in Acousmatic Electroacoustic Music
Download from the University of Jyväskylä
Mendoza Garay, Juan Ignacio (2014)
Abstract
Acousmatic Electroacoustic Music is a form of art in which any sound is regarded to have the potential to be used as a basic unit to build a musical discourse. This study addresses the measurement of three important aspects of perception of Acousmatic Electroacoustic Music: Segmentation, which is the separation of the audio stream into units that could have meaning by themselves or that could be part of larger meaningful structures; Mimesis, which in this context is defined as the perceptual ability of identifying a possible and meaningful source that produced a sound and the action that produced it; Perception of Emotions, which shapes the meaning of perceived events. This document reports the development and adaptation of tools and techniques to perform self-report measurements of these three aspects and the use of them in perceptual experiments with human participants. The results of these experiments suggest that participants’ strategies to segment sounds can correspond both to their acoustic features and to their meaning inferred by source and action. Participants’ identification of sound source and action can greatly vary and narrative responses suggest that sounds trigger meaning rather than having meaning by themselves. Post-hoc and continuous measurements of perceived emotions were remarkably similar, suggesting that perceived emotions in discrete sounds are established constructs that might not change considerably when acting in context. Finally, it is shown that continuous measurement of Segmentation and Perceived Emotions is reliable only for sounds with a duration over 4 seconds, which leaves an open window for further investigation on the measurement of perception of shorter sonic events.
Download from the University of Jyväskylä
Mendoza Garay, Juan Ignacio (2014)
Abstract
Acousmatic Electroacoustic Music is a form of art in which any sound is regarded to have the potential to be used as a basic unit to build a musical discourse. This study addresses the measurement of three important aspects of perception of Acousmatic Electroacoustic Music: Segmentation, which is the separation of the audio stream into units that could have meaning by themselves or that could be part of larger meaningful structures; Mimesis, which in this context is defined as the perceptual ability of identifying a possible and meaningful source that produced a sound and the action that produced it; Perception of Emotions, which shapes the meaning of perceived events. This document reports the development and adaptation of tools and techniques to perform self-report measurements of these three aspects and the use of them in perceptual experiments with human participants. The results of these experiments suggest that participants’ strategies to segment sounds can correspond both to their acoustic features and to their meaning inferred by source and action. Participants’ identification of sound source and action can greatly vary and narrative responses suggest that sounds trigger meaning rather than having meaning by themselves. Post-hoc and continuous measurements of perceived emotions were remarkably similar, suggesting that perceived emotions in discrete sounds are established constructs that might not change considerably when acting in context. Finally, it is shown that continuous measurement of Segmentation and Perceived Emotions is reliable only for sounds with a duration over 4 seconds, which leaves an open window for further investigation on the measurement of perception of shorter sonic events.
Software
The following software is open-sourced and free:
- DarwiinRemoteOSC for Mac OSX 10.5 and 10.6,
- DarwiinRemoteOSC for Mac OSX 10.8.
- DarwiinRemoteOSC gets messages from Wiimote via Bluetooth and sends them as OSC.
- DarwiinRemoteOSC for Mac OSX 10.5 and 10.6,
- DarwiinRemoteOSC for Mac OSX 10.8.
- Music Controller for Mac OSX 10.5 . Gets messages from Wiimote via Bluetooth and sends them as MIDI.
- Pure Data (also called Pd) is a visual environment aimed to develop audio and visual applications. Available for Windows, Mac OSX and Linux. Pd-extended is recommended and often required to open the patches listed hereinbelow.
Pure Data patches
- * VisuAural is a powerful, flexible and easy-to-use tool for qualitative and quantitative music cognition research. Very similar to its predecessor Aural (see below), it allows more use of keyboard and mouse, record wiimote pitch in real time (ideal for measurement of parametric real-time self-responses), perform Stroop test, display text and questionnaires including likert scales, nominal responses and text entry. As its predecessor Aural it can play back files in predetermined sequential order, record real-time self-response data while audio is playing, record audio and record questionnaires answers. It does everything in an automatized sequential way programmed by the user and outputs data files that can be processed for analysis, for example in Matlab. The responses can be input by means of the computer alphanumeric keyboard or a Wiimote (set by default to receive OSC data from DarwiinRemoteOSC but easily hackable). Currently Alpha release has had little testing and might need bug fixes. If you are interested feel free to contact me for I could give you an updated version. Basic documentation here.
- * Dynamic_Canvas, Dynamic_Hradio, Dynamic_Hradio+Canvas, Dynamic_Likert, Dynamic_Nominal and Dynamic_Toggle dynamically create and format different objects. Most of them are included in Text_Questionnaire which is a module of VisuAural.
- * JÄMME-Kokko is a very first prototype of a musical instrument featuring algorithmic generation of musical structure, basics of artificial intelligence, partially controlled by infrared light and capable of multi-channel diffusion (currently a 8-channel circle of equally spaced speakers). A brief description here. Needs a software that captures a Wiimote infrared camera data (set by default to receive OSC data from DarwiinRemoteOSC but easily hackable). Watch a demonstration video here.
- * Aural. Its name says it all. Aural (which sounds "Oral") is a powerful, flexible and easy-to-use tool for qualitative and quantitative music cognition research. It can play back files in predetermined sequential order, record real-time self-response data while audio is playing, record audio and record questionnaires answers. It does everything in an automatized sequential way programmed by the user and outputs data files that can be processed for analysis, for example in Matlab. The responses can be input by means of the computer alphanumeric keyboard or a Wiimote (set by default to receive OSC data from DarwiinRemoteOSC but easily hackable). It can be used for example as a sequential playback, as a "neutral" interviewer for qualitative research, to record self-report real-time data (segmentation, emotions, etc.), currently only 1 or 0, record only answers for any kind of stimuli with up to 5 buttons, and etcetera. As everything happens aurally and orally there is no need for visual or written questionnaires (ideal for blind participants) although stimuli and questionnaires can be reinforced with an external visual or written component. No documentation available. If you need information send me an e-mail message.
- * Granspat Wiichuk. A musical instrument consisting of 3 synthesizers whose sound can be controlled and moved around space using Wiimote and Nunchuk. Needs DarwiinRemoteOSC running. Currently working on 4 channels but easily upgradable to 8 or even more. This is a work in progress, it is still very crude but it works.
- * Control_activity_meter and control_amplitude_meter are abstractions that measure activity of a control signal. The first just measures raw amount of activity and the second measures amplitude of the signal. They can be used for example to detect the amount of agitation of an HID controller, like a Wiimote. This information can then be mapped to a musical parameter, or anything else.
- * DarwiinRemoteOSC+Nunchunpacker unpacks Wiimote and Nunchuk OSC messages from DarwiinRemoteOSC: Wiimote buttons, roll, pitch and acceleration in 3 axis; Nunchuk buttons, acceleration in 3 axis and joystick 2 axis.
- * Middlesizedaudioplayer plays an audio file from disk, shows its waveform, has a counter with deciseconds accuracy and allows scrolling back and forward among other features. Handles WAV and most of AIFF formats. For puredaters: the trick is to use supermiposed tables to display the waveform while the actual playback is done from disk. Loading a file might take a short or long while depending on the speed and memory of the computer being used, because the machine draws the waveform image every time a file is opened.
- * Tinyaudiofileplayer is a cute little machine that plays an audio file from disk. Handles WAV and some AIFF.
- * One Wiimote Real Time Capture allows to visualize and capture a Wiimote's roll and pitch data from DarwiinRemoteOSC, then save it to a text file. It can be used for example to record real time self-report data. Connects to port 5600 to receive OSC messages (default for DarwiinRemoteOSC) but it can be hacked to connect to other port to use with Osculator or other software. Open the main subpatch and modify the subpatches over the red "OSC" square to change the port and the calibration of the Wiimote. For further use (such as analysis) the output text file can be imported by other software like for example Matlab.
- * Visualize OSC activity from DarwiinRemoteOSC.
- * CPU load meter with a linear VU display. Looks nicer that the one that comes in the Pure Data package.
Matlab scripts:
- * Binary Sequences Similarity: A measure to compare sequences of music segmentation (or any binary data). Documentation here.
- * Permutations Processor
* Courses assignments:
| Sound Processing 1: MIDI Assignment (ver. 2) - Short electroacoustic music exercise - Runs best in Protools LE 8 |
Audio
* Courses assignments:
| Sound Processing 1: MIDI Assignment - Short electroacoustic music exercise featuring synthesized sounds |
| Sound Processing 1: Audio Assignment - Short electroacoustic music exercise featuring vocal gestures |
Research Methods
Music Psychology
Music Perception I course literature:
* Music Psychology II course work:
- Video "Emotion entrainment: Anger" (motion capture): http://www.flickr.com/photos/86358941@N04/8215156772/in/photostream
- Video "Motion capture test": http://www.flickr.com/photos/86358941@N04/8128633166/in/photostream
Music Psychology I course literature:
| Music Psychology 1: list of texts |
- Reading material for Music Psychology 1 course full texts: https://rapidshare.com/files/1661051126/ReadingMaterial_MusicPsychology1.zip
