The term "algorithmic composition" is loaded with baggage. Although the phrase pops up in our conversation fairly often, because of the loaded nature of the words, Alexis Kirke prefers "Computer Aided Composition" to describe the process he uses to compose music.
Although perfectly capable of coding, and having done so in the past, Alexis now prefers to work collaboratively with a professional programmer so that he can focus on the overall composition. "Programming is a creative act in itself" and it is all too easy to get sucked into it, he explains. Listening to Alexis describe how he is developing Insight, I couldn't help but think of a Vulcan Mind Meld. Alexis desires to "share the unsharable" - his consciousness, internal feelings, his Palinopsia hallucinations. (See the previous post for more on Alexis and how Palinopsia inspired this particular body of work)
According to the current plan, if you are in the audience at the February 10th performance you will see Alexis standing on stage behind a music stand holding an iPad. Perhaps he will be holding a pen. A Macintosh laptop is within arm's reach. The programmer sits in the front row of seats, and there is a flute player somewhere on stage with his own computer monitor. Alexis sees you; he also sees lighting designed to trigger his hallucinations.
It is difficult to formulate words to describe what this application program will probably look like, but I'm going to give it a whirl. The iPad will show Alexis a camera image of what he is seeing at any moment, an "augmented reality". The software on the iPad will incorporate parameters that correspond to common elements of Alexis' hallucinations. As Alexis experiences hallucinations, he will activate afterimage functionality and trigger an afterimage on the iPad. (At this point, a poor video connection on our Skype call produced a well timed "trail" as Alexis moved his hand across my line of sight). When this happens he is presented with visual options, filters and parameters. These are produced in partial response to an iterative feedback loop between himself and the programmer: he selects, via multi-touch, tapping, double tapping, 3 fingered presses, which parameters are appropriate and what filters to apply. For example, he can adjust the iPad screen brightness to correspond to his perception of brightness. He can likewise adjust screen size, the specific pattern of an afterimage, the rate of visual decay, single or multiple images, random patterns, and many other aspects of the visual echo. Alexis is simultaneously saying "yes, that" "no, not that" to presented options, and creating from scratch on the fly what he sees. He has to make the iPad "see" what he sees.
All this is happening extremely rapidly as his hallucinations develop, exist, and are replaced (sometimes in milliseconds) with the next hallucination. And we must not forget that this is not a completely free form activity - there will be at least 3 sections to the composition. Each section of the piece has different parameters. Alexis will reach over to the laptop to change sections. All of the visuals are handled by the iPad. When a hallucination is accurate, it is packaged up and sent wirelessly to the laptop, where the music resides. One reason for this division of labor is that the visual software is processor intensive. Memory optimization has been critical, and Alexis brought in someone who focused exclusively on iPad memory optimization.
Somehow, it all gets put together and coordinated: visual hallucinations from the iPad, musical scores from the laptop, contribution from a live flute player watching the iPad output.
You, sitting in the audience would see Alexis' hallucinations projected on a large screen, and hear his musical score to go along with it. All in 12 minutes.
The performance of Insight, at the Peninsula Arts Contemporary Music Festival, is going to be filmed. Let's hope it becomes available online!
Wednesday, January 25, 2012
Sunday, January 22, 2012
Computing...Music...Palinopsia
What would you do if, under the right circumstances, you saw hallucinations (no drugs involved)? If your name was Alexis Kirke you would take your two PhDs (one in arts, one in technology) and create a professional demonstration to share with the public.
I had no idea what to expect when, prior to our conversation, I read about Alexis and learned we were going to talk about his upcoming performance that would share live visual hallucinations. Who is this guy, I wondered?
Alexis Kirke is a member of Plymouth University's (UK) Interdisciplinary Centre for Computer Music Research, and Composer in Residence at the Plymouth Marine Institute. As part of his latter role, he has composed a score for a performance of "artificially intelligent whales" interacting with a saxophone. Impossible to describe; you just have to go here and expose yourself to it. Then there are other performances with titles such as: "Phrased and Confused" and "Drum Abuse". What next?
Afterimages. Ever look straight at a lightbulb and then been half blinded with the image of the bulb well after you stop looking at it? That is an afterimage. Afterimages come in two types, positive and negative. Positive afterimages are the same color as the original image and thought (according to my brief research) to have a neurological origin. An often cited example of positive afterimages are the "trails" associated with use of LSD. Conversely, negative afterimages invert the colors and are thought to have an optical/retinal origin.
Alexis has Palinopsia, a neurological condition that produces (most often positive) afterimages. What this means to Alexis is that when he is extremely tired, stressed, or in just the right lighting conditions, he will start to see afterimages. No drugs involved. He describes it as unnerving sometimes, but, he stresses, unlike schizophrenia there is never any question about what is real and what is a hallucination.
Most people with Palinopsia presumably find ways to work with and around the condition and there are resources out there to help such as The Palinopsia Foundation. Alexis, on the other hand, is going beyond co-existence. His first PhD was in the area of theoretical multi-robot systems and he spent time interacting with neuroscience research groups. After earning his degree, he worked in the fields of finance, and speech and sound recognition. Following a calling to move deeper into music, he then completed a second PhD in Computing Music and became a composer.
Composers work is performed; artists often take their inspiration from their own experiences. As I spoke to Alexis about the lead up to the creation of Insight, which will premier February 10 of this year, it no longer seemed bizarre to create a composition that uses computers to enable the sharing of visual hallucinations.
In the next post, I'll explain more about how it all works. Don't stray too far...
I had no idea what to expect when, prior to our conversation, I read about Alexis and learned we were going to talk about his upcoming performance that would share live visual hallucinations. Who is this guy, I wondered?
Alexis Kirke is a member of Plymouth University's (UK) Interdisciplinary Centre for Computer Music Research, and Composer in Residence at the Plymouth Marine Institute. As part of his latter role, he has composed a score for a performance of "artificially intelligent whales" interacting with a saxophone. Impossible to describe; you just have to go here and expose yourself to it. Then there are other performances with titles such as: "Phrased and Confused" and "Drum Abuse". What next?
Afterimages. Ever look straight at a lightbulb and then been half blinded with the image of the bulb well after you stop looking at it? That is an afterimage. Afterimages come in two types, positive and negative. Positive afterimages are the same color as the original image and thought (according to my brief research) to have a neurological origin. An often cited example of positive afterimages are the "trails" associated with use of LSD. Conversely, negative afterimages invert the colors and are thought to have an optical/retinal origin.
Alexis has Palinopsia, a neurological condition that produces (most often positive) afterimages. What this means to Alexis is that when he is extremely tired, stressed, or in just the right lighting conditions, he will start to see afterimages. No drugs involved. He describes it as unnerving sometimes, but, he stresses, unlike schizophrenia there is never any question about what is real and what is a hallucination.
Most people with Palinopsia presumably find ways to work with and around the condition and there are resources out there to help such as The Palinopsia Foundation. Alexis, on the other hand, is going beyond co-existence. His first PhD was in the area of theoretical multi-robot systems and he spent time interacting with neuroscience research groups. After earning his degree, he worked in the fields of finance, and speech and sound recognition. Following a calling to move deeper into music, he then completed a second PhD in Computing Music and became a composer.
Composers work is performed; artists often take their inspiration from their own experiences. As I spoke to Alexis about the lead up to the creation of Insight, which will premier February 10 of this year, it no longer seemed bizarre to create a composition that uses computers to enable the sharing of visual hallucinations.
In the next post, I'll explain more about how it all works. Don't stray too far...
Tuesday, January 17, 2012
What Would You Like?
A short note this afternoon.
I have some very interesting material to share with you over the next few weeks if all goes according to schedule. I'm investigating several intriguing people and projects.
Meanwhile, as things are percolating here, I think this is a good time to throw a question out there. You can respond to me privately, as many of you already do, or post your ideas here as a comment if you feel brave.
Are there any any particular topics in interdisciplinary computing and/or socially beneficial computing that you would like to see me write about?
My ears are always open to your ideas.
Friday, January 6, 2012
Processing: Data Visualization and More
Today's meeting of the Processing workshop was as stimulating as yesterday's (see the post from yesterday). As before, the day was a blend of interactive instruction about Processing and hand's on time to code with the system. However, we went deeper - one of today's topics was data visualization using Processing. Check out these examples of linguistic poetry visualization by Ira Greenberg.
Here is an absolutely fascinating data visualization of Eurozone debt. Although I don't know if this is written in Processing or not, I have learned enough these past two days to tell you it would be fairly straightforward to do so.
Data visualization brings together the computer science and the generative art solidly. It isn't just about making pretty pictures. In order to create meaning and relevance in your visualization you need to manipulate, perhaps transform, your data. Depending upon the application, this could mean pulling in calculus, trigonometry, geometry, physics, and chemistry. It could mean creating well thought out OOP code, efficient threading and algorithmic selections, array manipulation, random number generation. Another one of the workshop leaders, Deepak Kumar, showed us a class assignment where students must use an array of Objects to create a row of streetlamps (any design they want) that randomly turn on and off.
We experimented with creating code that behaves like those art boards where you paint with water: as you draw lines/shapes with your mouse they fade behind you at whatever rate you dictate.
Web interfaces, network communication, digital audio generation. One of our workshop participants expressed an interest yesterday in working with sound in Processing - today we were listening to her computer broadcasting music from across the conference room.
If you are interested in using Processing in your classroom, the workshop leaders (Deepak Kumar, Dianna Xu, Ira Greenberg) would be happy to talk with you.
Here is an absolutely fascinating data visualization of Eurozone debt. Although I don't know if this is written in Processing or not, I have learned enough these past two days to tell you it would be fairly straightforward to do so.
Data visualization brings together the computer science and the generative art solidly. It isn't just about making pretty pictures. In order to create meaning and relevance in your visualization you need to manipulate, perhaps transform, your data. Depending upon the application, this could mean pulling in calculus, trigonometry, geometry, physics, and chemistry. It could mean creating well thought out OOP code, efficient threading and algorithmic selections, array manipulation, random number generation. Another one of the workshop leaders, Deepak Kumar, showed us a class assignment where students must use an array of Objects to create a row of streetlamps (any design they want) that randomly turn on and off.
We experimented with creating code that behaves like those art boards where you paint with water: as you draw lines/shapes with your mouse they fade behind you at whatever rate you dictate.
Web interfaces, network communication, digital audio generation. One of our workshop participants expressed an interest yesterday in working with sound in Processing - today we were listening to her computer broadcasting music from across the conference room.
If you are interested in using Processing in your classroom, the workshop leaders (Deepak Kumar, Dianna Xu, Ira Greenberg) would be happy to talk with you.
This little guy is expressing his excitement:
(click on him and then be patient for a moment or two)
Thursday, January 5, 2012
Computer Science via Generative Art and Vice Versa
Reporting this evening from Dallas (Texas) where I am attending a workshop about the Processing system, a unique way to learn programming concepts using generative art. Within 10 minutes of starting to use Processing, I was sucked completely into coding, and was producing animated abstractions. This was my first program (a lot more interesting than any Hello World):
About 12 of us were sitting around the conference table and we were all heads down, heads up (to look at the overhead display where one of the presenters was explaining various features), heads down, heads up, type type type. College and University faculty, high school teachers, computer scientists, digital designers. Type, type type.
Before I knew it we were not only coding in Java (a language I generally dislike with a passion) but trying to figure out how to make loops, swoops, and geometric patterns bounce around the screen. And bounce they did - or grow, shrink, grow-shrink-grow-shrink. Change colors, size, shapes, speed up, slow down, fade in, out, turn corners, move around smooth rollercoaster-like curves...the only boundary was our imagination. When was the last time your first day of class was that productive and absorbing?
And this is no toy system. Processing has some unique attributes. Not only does the system use a real language (Java), it is accessible (such an easy interface), it is powerful and it is fun. It is also open source, contains excellent online help, and there is an exhibition space for uploading your creations along with their code. As open source, you can download other people's creations and code as well.
Designed originally by people affiliated with the MIT Media Lab as a language for artists to explore coding as a medium, it is now becoming well known in the computer science education community. One of our workshop leaders, Ira Greenberg, is a painter by training; he downloaded an early version of Processing around 2001 and not long after wrote a book (the first ever I believe) "Processing - Creative Coding and Computational Art". Fast forward: Now Ira has a dual appointment in Computer Science and Engineering and in the Meadows School of the Arts at Southern Methodist University.
What you can't see from these pictures I created (before lunch) is that they are dynamic. Both pictures, especially the one on the right, were moving under the direction of my code. Now yes, I have a background in programming and teaching Java, but I have no particular expertise in graphics. And I really do despise Java. Such a pain ...Usually. Not today.
More to the point, the two creations of mine included here are far less sophisticated than what others in our group were creating; one participant, a public high school teacher, created a brilliant beautiful demo on recursion this afternoon. Someone else created what looked like an ever expanding universe of pulsating multi-shaped colored particles.
There were a lot of fried brains by the end of the day because people barely looked up - most were glued to their laptops straight through lunch. By the end of the day it clear that we were well on our way into sophisticated concepts in science, math and computing. It just seemed to ... happen. I can't wait for tomorrow.
Before I knew it we were not only coding in Java (a language I generally dislike with a passion) but trying to figure out how to make loops, swoops, and geometric patterns bounce around the screen. And bounce they did - or grow, shrink, grow-shrink-grow-shrink. Change colors, size, shapes, speed up, slow down, fade in, out, turn corners, move around smooth rollercoaster-like curves...the only boundary was our imagination. When was the last time your first day of class was that productive and absorbing?
Designed originally by people affiliated with the MIT Media Lab as a language for artists to explore coding as a medium, it is now becoming well known in the computer science education community. One of our workshop leaders, Ira Greenberg, is a painter by training; he downloaded an early version of Processing around 2001 and not long after wrote a book (the first ever I believe) "Processing - Creative Coding and Computational Art". Fast forward: Now Ira has a dual appointment in Computer Science and Engineering and in the Meadows School of the Arts at Southern Methodist University.
What you can't see from these pictures I created (before lunch) is that they are dynamic. Both pictures, especially the one on the right, were moving under the direction of my code. Now yes, I have a background in programming and teaching Java, but I have no particular expertise in graphics. And I really do despise Java. Such a pain ...Usually. Not today.
More to the point, the two creations of mine included here are far less sophisticated than what others in our group were creating; one participant, a public high school teacher, created a brilliant beautiful demo on recursion this afternoon. Someone else created what looked like an ever expanding universe of pulsating multi-shaped colored particles.
There were a lot of fried brains by the end of the day because people barely looked up - most were glued to their laptops straight through lunch. By the end of the day it clear that we were well on our way into sophisticated concepts in science, math and computing. It just seemed to ... happen. I can't wait for tomorrow.
Subscribe to:
Posts (Atom)