Research Papers

 Below are three articles which relate to the ideas I have for this class.


1. Evaluating the WiiMote as a Musical Controller

http://hdl.handle.net/2027/spo.bbp2372.2008.092

This paper contains the results of a small investigative study into the applications and intuitiveness of the Wii Remote's motion controls when applied to a music-creation context. They were drawn to this controller because of its unique motion control system, which is not as direct to map to a digital instrument in the same way keyboard or controller buttons are. They used a mixture of neural networks to allow the system to identify specific gestures from the controller's acceleration data, which is inherantly noisey and imprecise. The users had a wide degree of experience both on a musicianship scale as well as game-experience. They found that for discrete tasks such as playing a drum to a beat or indicating a scale, the Wii Remote was imprecise and lacked feedback to the user necessary to allow for those precise inputs; however, when used just to create sounds users reported not only more intuition and ease of use but more enjoyment with the activity. Overall, it was indicated that this type of control system would not be good for the precise work of composing or arranging, however it would be very interesting for more freeform content or live performances.


2. claVision: Visual Automatic Piano Music Transcription

https://nime.org/proc/nime2015_makbari/index.html

I thought it was really interesting how this team approached the live transcription. While not something I would be looking to do, the alternative input-in this case, the video/recorded view of a standard piano-provides a lot of inspiration for the type of out of the box inputs that would be available for a customized arrangment software. Additionally, this type of project reminds me of the laser/light keyboards which allow an individual to type simply by projecting a keyboard onto a surface and having a computer analyze the footage to determine which keys were pressed. In a similar way, this type of program would allow for one to transcribe piano notes while only requiring a flat surface and light projection.


3. Arrangements: Flexibly Adapting Music Data for Live Performance

https://nime.org/proc/nime2015_rdannenbergb/index.html

This article discusses a prgram developed to allow preset, recorded, or generated computer signals to be synced up with a live performance. While appearing straight forward, the inherant variation of live performance-new or changing tempos, restructuring of a song, a soloist entering a beat late-all create variations that a computer system needs to be able to adapt to live in order to remain in sync with the performance. These researchers used a series of mapping functions, assigning discrete indexes to measures or other appropriate points within the computerized, "unrolled" score, so even repeated sections are uniquely identified by index. These values are them mapped to places in the performance score, and those various mapping functions will change as the performance itself progresses, altering the computer score to remain in sync with the live performance. While not directly in line with the arrangement program that I am hoping to create, it was helpful in reading about the difficulties with syncing up music and what parts of the process can be automated versus which parts must be hard-coded by the user prior to the performance.


Comments

Post a Comment

Popular posts from this blog

Project - Brawl Time

Image
After looking around at both in-browser games and on steam, I ended up deciding to use Brawlhalla as my game engine for this project. It's one of many Super Smash Brothers clones available, but this one is free to download and had good reviews regarding its ability to work with a variety of controllers, as I was unsure how plugging in my roommate's off-brand X-Box controller would work.  It went incredibly smoothly, and after turning off the in-game music I opened up the project file in ChucK, started the virtual machine, and was able to get the sounds running simultaneously with the gameplay. My familiarity and skill with Super Smash Bros. is unfortunately quite limited (I never completed the tutorial levels) but I was able to jump into the game with ease and begin getting a feel for how the controls worked. However, I have some small concerns about my more experienced friends as they are very used to Super Smash Bros. controls and, at least for the button inputs, they are qui...
Read more

Project - Controller Outputs

Image
This past week I did some initial testing of ChucK using the example files provided by the program, and verified the outputs of the off-brand X-Box controller could be read by ChucK. I first tested the device's ability to generate midi sounds from the keyboard inputs, to ensure that the audio on the device was functioning. I was unable to get sound to play through headphones during the sessions, so tested subsequently without additional audio devices. I first used the command line to "--probe" ChucK to see the list of devices, and verified that the off-brand X-Box controller I'd plugged in via USB cable was registering in the system. I used the basic joystick controller example to begin reading output from the controller. Unexpected, though in hind sight very predictable, was that while the controller buttons had a single "on" or "off" state, the joysticks and triggers had a very rapidly changing range of values. These were initially read from the ...
Read more