The term "algorithmic composition" is loaded with baggage. Although the phrase pops up in our conversation fairly often, because of the loaded nature of the words, Alexis Kirke prefers "Computer Aided Composition" to describe the process he uses to compose music.
Although perfectly capable of coding, and having done so in the past, Alexis now prefers to work collaboratively with a professional programmer so that he can focus on the overall composition. "Programming is a creative act in itself" and it is all too easy to get sucked into it, he explains. Listening to Alexis describe how he is developing Insight, I couldn't help but think of a Vulcan Mind Meld. Alexis desires to "share the unsharable" - his consciousness, internal feelings, his Palinopsia hallucinations. (See the previous post for more on Alexis and how Palinopsia inspired this particular body of work)
According to the current plan, if you are in the audience at the February 10th performance you will see Alexis standing on stage behind a music stand holding an iPad. Perhaps he will be holding a pen. A Macintosh laptop is within arm's reach. The programmer sits in the front row of seats, and there is a flute player somewhere on stage with his own computer monitor. Alexis sees you; he also sees lighting designed to trigger his hallucinations.
It is difficult to formulate words to describe what this application program will probably look like, but I'm going to give it a whirl. The iPad will show Alexis a camera image of what he is seeing at any moment, an "augmented reality". The software on the iPad will incorporate parameters that correspond to common elements of Alexis' hallucinations. As Alexis experiences hallucinations, he will activate afterimage functionality and trigger an afterimage on the iPad. (At this point, a poor video connection on our Skype call produced a well timed "trail" as Alexis moved his hand across my line of sight). When this happens he is presented with visual options, filters and parameters. These are produced in partial response to an iterative feedback loop between himself and the programmer: he selects, via multi-touch, tapping, double tapping, 3 fingered presses, which parameters are appropriate and what filters to apply. For example, he can adjust the iPad screen brightness to correspond to his perception of brightness. He can likewise adjust screen size, the specific pattern of an afterimage, the rate of visual decay, single or multiple images, random patterns, and many other aspects of the visual echo. Alexis is simultaneously saying "yes, that" "no, not that" to presented options, and creating from scratch on the fly what he sees. He has to make the iPad "see" what he sees.
All this is happening extremely rapidly as his hallucinations develop, exist, and are replaced (sometimes in milliseconds) with the next hallucination. And we must not forget that this is not a completely free form activity - there will be at least 3 sections to the composition. Each section of the piece has different parameters. Alexis will reach over to the laptop to change sections. All of the visuals are handled by the iPad. When a hallucination is accurate, it is packaged up and sent wirelessly to the laptop, where the music resides. One reason for this division of labor is that the visual software is processor intensive. Memory optimization has been critical, and Alexis brought in someone who focused exclusively on iPad memory optimization.
Somehow, it all gets put together and coordinated: visual hallucinations from the iPad, musical scores from the laptop, contribution from a live flute player watching the iPad output.
You, sitting in the audience would see Alexis' hallucinations projected on a large screen, and hear his musical score to go along with it. All in 12 minutes.
The performance of Insight, at the Peninsula Arts Contemporary Music Festival, is going to be filmed. Let's hope it becomes available online!
Showing posts with label music. Show all posts
Showing posts with label music. Show all posts
Wednesday, January 25, 2012
Sunday, January 22, 2012
Computing...Music...Palinopsia
What would you do if, under the right circumstances, you saw hallucinations (no drugs involved)? If your name was Alexis Kirke you would take your two PhDs (one in arts, one in technology) and create a professional demonstration to share with the public.
I had no idea what to expect when, prior to our conversation, I read about Alexis and learned we were going to talk about his upcoming performance that would share live visual hallucinations. Who is this guy, I wondered?
Alexis Kirke is a member of Plymouth University's (UK) Interdisciplinary Centre for Computer Music Research, and Composer in Residence at the Plymouth Marine Institute. As part of his latter role, he has composed a score for a performance of "artificially intelligent whales" interacting with a saxophone. Impossible to describe; you just have to go here and expose yourself to it. Then there are other performances with titles such as: "Phrased and Confused" and "Drum Abuse". What next?
Afterimages. Ever look straight at a lightbulb and then been half blinded with the image of the bulb well after you stop looking at it? That is an afterimage. Afterimages come in two types, positive and negative. Positive afterimages are the same color as the original image and thought (according to my brief research) to have a neurological origin. An often cited example of positive afterimages are the "trails" associated with use of LSD. Conversely, negative afterimages invert the colors and are thought to have an optical/retinal origin.
Alexis has Palinopsia, a neurological condition that produces (most often positive) afterimages. What this means to Alexis is that when he is extremely tired, stressed, or in just the right lighting conditions, he will start to see afterimages. No drugs involved. He describes it as unnerving sometimes, but, he stresses, unlike schizophrenia there is never any question about what is real and what is a hallucination.
Most people with Palinopsia presumably find ways to work with and around the condition and there are resources out there to help such as The Palinopsia Foundation. Alexis, on the other hand, is going beyond co-existence. His first PhD was in the area of theoretical multi-robot systems and he spent time interacting with neuroscience research groups. After earning his degree, he worked in the fields of finance, and speech and sound recognition. Following a calling to move deeper into music, he then completed a second PhD in Computing Music and became a composer.
Composers work is performed; artists often take their inspiration from their own experiences. As I spoke to Alexis about the lead up to the creation of Insight, which will premier February 10 of this year, it no longer seemed bizarre to create a composition that uses computers to enable the sharing of visual hallucinations.
In the next post, I'll explain more about how it all works. Don't stray too far...
I had no idea what to expect when, prior to our conversation, I read about Alexis and learned we were going to talk about his upcoming performance that would share live visual hallucinations. Who is this guy, I wondered?
Alexis Kirke is a member of Plymouth University's (UK) Interdisciplinary Centre for Computer Music Research, and Composer in Residence at the Plymouth Marine Institute. As part of his latter role, he has composed a score for a performance of "artificially intelligent whales" interacting with a saxophone. Impossible to describe; you just have to go here and expose yourself to it. Then there are other performances with titles such as: "Phrased and Confused" and "Drum Abuse". What next?
Afterimages. Ever look straight at a lightbulb and then been half blinded with the image of the bulb well after you stop looking at it? That is an afterimage. Afterimages come in two types, positive and negative. Positive afterimages are the same color as the original image and thought (according to my brief research) to have a neurological origin. An often cited example of positive afterimages are the "trails" associated with use of LSD. Conversely, negative afterimages invert the colors and are thought to have an optical/retinal origin.
Alexis has Palinopsia, a neurological condition that produces (most often positive) afterimages. What this means to Alexis is that when he is extremely tired, stressed, or in just the right lighting conditions, he will start to see afterimages. No drugs involved. He describes it as unnerving sometimes, but, he stresses, unlike schizophrenia there is never any question about what is real and what is a hallucination.
Most people with Palinopsia presumably find ways to work with and around the condition and there are resources out there to help such as The Palinopsia Foundation. Alexis, on the other hand, is going beyond co-existence. His first PhD was in the area of theoretical multi-robot systems and he spent time interacting with neuroscience research groups. After earning his degree, he worked in the fields of finance, and speech and sound recognition. Following a calling to move deeper into music, he then completed a second PhD in Computing Music and became a composer.
Composers work is performed; artists often take their inspiration from their own experiences. As I spoke to Alexis about the lead up to the creation of Insight, which will premier February 10 of this year, it no longer seemed bizarre to create a composition that uses computers to enable the sharing of visual hallucinations.
In the next post, I'll explain more about how it all works. Don't stray too far...
Thursday, August 18, 2011
Science is Rock and Roll
I almost missed this, by not keeping up with NPR Science Friday. Last week there was a short spot on ... get this... rock and roll musicians plugging science. And not just on an NPR blurb but they are doing it on TV in order to hit a huge audience. The web page (below) says in big letters on it: "Who is Your Favorite Geek"? and I don't know how many times I heard, to the sound of background music "My Robot is Better Than Your Robot". And yes they say "programming" is a cool thing - science includes computing for sure and it gets you hopping...you can't help it.
This is so cool, I shall say no more, but send you to this web site to check it out yourself. Woo hoo!
Science is Rock and Roll
This is so cool, I shall say no more, but send you to this web site to check it out yourself. Woo hoo!
Science is Rock and Roll
Saturday, January 8, 2011
What CS Gains From Interdisciplinary Computing
Following up on last night's post about the reasons why many people engage in interdisciplinary computing work, I'd like to briefly list off some examples that came out in our meeting discussion today. At one point we decided to get specific and share examples of how the computer science discipline has directly benefited from interdisciplinary collaborations.
Here are some of them, written as close to verbatim as I could take notes on the fly. I'm certainly not expert in many of them, so anything I say that is missing an important or interesting piece hopefully someone will chime in and amplify for me. However! this is a classic facet of interdisciplinary collaboration - no one individual can know multiple fields at the same depth and accuracy. That is part of why it is such productive work!
The Folding at Home project involves experts in bio-medicine, distributed computing, bio-technology, high performance computing. As the site implies, computer science has been stimulated in HPC (high performance computing), algorithm development, networking, and simulations. We have expanded boundaries of computational understanding in all these areas.
Distributed computation not only in that project but in other very large scale projects has pushed the boundaries of computational efficiency to new levels as we develop the necessary algorithms to tackle ever more seemingly intractable problems, that are not necessarily so intractable.
Online auctions. I missed the intro to this conversation, but when I picked up, the discussion was about the movement from manual to electronic auctions requiring a change in how economists worked on what turn out to be NP-hard problems. Computer science theory is making advances so that these auctions can function properly. I could use someone helping to fill me in on what I missed in this conversation because it sounds very interesting!
Working with the film industry has driven both the development of 3D graphics and User Interface development. Many of the "old" rules of thumb (meaning circa early 90s) no longer apply and computer science has stepped up to revamp our understanding of what we can do with graphics at very fundamental levels. User Interface theory and application has evolved right along with it. For example, imagine how far we have come from Star Wars (1977) to Toy Story (1995) to Avatar (2009).
These advances in computer science from interdisciplinary work with the film industry in turn spurred development on the side of GPUs (Graphic Processor Unit), which were then deployed in so many areas of computing from games to simulations and beyond that they cannot be easily itemized.
Music downloads - the virtually ubiquitous desire to stream music in one form or another, has led to advances in basic streaming technologies and support algorithms that now are used in far flung applications such as digital image processing in medicine and transfer of image data (MRIs for example) to medical service providers on short notice across long distances. There was mention specifically of recognition algorithms as a subset of algorithms that have advanced, - I could use some supplementary information on this one!
Just a few ideas to whet your appetite. Speaking of appetite, I may not have one for a week. Two days of intense cognitive load and equally intense gastric load have left me with wonderful memories and a bit of a tummy ache. Brings a new meaning to "Brain Food".
Here are some of them, written as close to verbatim as I could take notes on the fly. I'm certainly not expert in many of them, so anything I say that is missing an important or interesting piece hopefully someone will chime in and amplify for me. However! this is a classic facet of interdisciplinary collaboration - no one individual can know multiple fields at the same depth and accuracy. That is part of why it is such productive work!
The Folding at Home project involves experts in bio-medicine, distributed computing, bio-technology, high performance computing. As the site implies, computer science has been stimulated in HPC (high performance computing), algorithm development, networking, and simulations. We have expanded boundaries of computational understanding in all these areas.
Distributed computation not only in that project but in other very large scale projects has pushed the boundaries of computational efficiency to new levels as we develop the necessary algorithms to tackle ever more seemingly intractable problems, that are not necessarily so intractable.
Online auctions. I missed the intro to this conversation, but when I picked up, the discussion was about the movement from manual to electronic auctions requiring a change in how economists worked on what turn out to be NP-hard problems. Computer science theory is making advances so that these auctions can function properly. I could use someone helping to fill me in on what I missed in this conversation because it sounds very interesting!
Working with the film industry has driven both the development of 3D graphics and User Interface development. Many of the "old" rules of thumb (meaning circa early 90s) no longer apply and computer science has stepped up to revamp our understanding of what we can do with graphics at very fundamental levels. User Interface theory and application has evolved right along with it. For example, imagine how far we have come from Star Wars (1977) to Toy Story (1995) to Avatar (2009).
These advances in computer science from interdisciplinary work with the film industry in turn spurred development on the side of GPUs (Graphic Processor Unit), which were then deployed in so many areas of computing from games to simulations and beyond that they cannot be easily itemized.
Music downloads - the virtually ubiquitous desire to stream music in one form or another, has led to advances in basic streaming technologies and support algorithms that now are used in far flung applications such as digital image processing in medicine and transfer of image data (MRIs for example) to medical service providers on short notice across long distances. There was mention specifically of recognition algorithms as a subset of algorithms that have advanced, - I could use some supplementary information on this one!
Just a few ideas to whet your appetite. Speaking of appetite, I may not have one for a week. Two days of intense cognitive load and equally intense gastric load have left me with wonderful memories and a bit of a tummy ache. Brings a new meaning to "Brain Food".
Subscribe to:
Posts (Atom)