Part I: in dialogue with Mark Coniglio

Part II: comments from Jean-Baptiste Barrière, Jem Finer, Armando Menicacci, Giorgio Olivero, Steina Vasulka

[short biographies and reference URLs are at the end]

Edited by Scott deLahunta

15 September 2002

Part II:

Scott: The type of artist/ toolmaker relationship Mort Subotnick and Mark had is an interesting one to trace back historically. Steina, you have been working as an artist and researcher in electronic media arts for over thirty years. Would you have something to say to this?

Steina Vasulka: It is a big topic, but it would be interesting to investigate ALL artist/ toolmaker relations in history; French Horn, Stradivarius, the well tempered clavier, the invention of photography, film, acrylics, video, etc. We (the Vasulkas) have almost always worked with toolmakers; analog first, then digital and now software. On our website we have the "Eigenwelt der Apparatewelt" exhibition catalog (put together for the Ars Electronica Festival 1992) devoted to early audio and video toolmakers. There is a going model here: sometimes creators themselves, sometimes collaborators with artists, these toolmakers have invariably been gifted visionary individuals far removed from the industries who adapt to as well as inspire and invent within the technology environments of their time. In the early days of video, the buzzword was "modification"; when one would go into these tools created for consumers and retrofit them for the artist. We also see developments and knowledge passed along from one tool generation to another; as Mark mentioned, the software packages of now are the synthesizers of recent past.

Jean-Baptiste Barrière: I would just like to add here that there have been many graphical software tools created for music. One that could be mentioned, if only because of its complementary nature to more performance oriented languages, is Patchwork, which became more recently Open Music (both developed at IRCAM, Paris). It is a language based on Lisp, for computer-assisted composition, with a strong emphasis on musical notation representation. Incidentally, unlike most people think, the patch chord metaphor (first used in software by Max Mathews at Bell Labs), is not derived from analog synthesizers. Quite the contrary: Robert Moog came to know the work of Mathews and then was inspired by it to design modular synthesizers. The patch chord paradigm was applied to software from the experience of telephone switchboards.

Scott: Just for those who may not know, Max Mathews has been referred to as the "father of computer music" who, in 1957, was the first to synthesize music on a computer; performing a seventeen second piece on an IBM 704. In 1970, when fast digital chips and new algorithms made 'real time' possible Mathews developed Groove, the "first computer system for live performance" (http://www.csounds.com/mathews/). So, here we have the early research into systems out of which eventually software and hardware like Mark's Interactor and the MidiDancer would emerge.

Steina: A very interesting part of the dialogue was when Mark talks about the implications of this dancer interface, the MidiDancer; the attempt to have a dancer be a musical instrument player. When I was a kid, it was not presumed that dancers could act, that actors could sing, that singers could dance. Now they have to know it all. Interestingly, in talking about developing Isadora, Mark refers to wanting to make an interface that a choreographer or dancer can learn to use easily. The interface for Image/ine seems impossible for dancers and musicians to learn; while it is a piece of cake for video makers who are perplexed by Max, etc. Another solution might be to think of multiple interfaces to a single program, like Final Cut Pro (otherwise not my favorite software), that makes the same features available in an "effects" interface for film/video people and "photo shop type" interface for computer artists.

Armando Menicacci: Getting back to the points that Mark was making regarding the limitations of the MidiDancer for dance; what he says is very true, in my opinion. In fact, for a 'dancer to be a dancer' means not to have to show the control over media. I don't want to re-open the thread about "should we see the dancer's control over the media"? I think that Mark's comments are sufficient on this particular subject, to which I can add a brief comment. In my opinion, if what we want to do is 'pedagogy of interactivity' then we have to show the result of the interaction. But if what we want to do is art then we shouldn't care about the visibility of interaction, unless it is vital for a particular aesthetic project.

As regards my own use of Isadora; I am not sure, but maybe, if we don't consider Mark and Dawn's own work, I've been the first person to use Isadora in the context of a full-length dance performance on stage. We bought a copy of Isadora in the beginning of May; in my experience even the beta was very stable and we managed to use it in a way that was artistically successful. I was working with the choreographer Rachid Ouramdame in Reims where we have a residency. We'll use it now in Dijon for the same work, titled "+ ou - la", and from October 30 to November 4th we show the same piece in the Grande Salle of the Centre Pompidou in the Festival d'Automne à Paris. What was interesting for Rachid, was to find particular inspiration for the choreography in the way in which Isadora transforms digital video media. This is important to note ways in which emerging forms of composition / transformations in the digital medium can be translated to and used in mediums of the body; specifically dance. It broadens the idea of these softwares beyond just their specific functionality.

Scott: Giorgio and Jem, neither of you as far as I am aware come out of the contemporary dance field. Can you say something about your backgrounds and how you arrived at this point of working with Isadora?

Giorgio Olivero: We (myself and Andrea Clemente) come from a background as web and graphic designers. In 2000, we started doing visuals for the club scene when we were students of the 'Scienze della Comunicazione' faculty in Torino. We were very excited by the possibility offered by this evolution of clubbing, and we began experimenting with aesthetics and technology. We were ignorant of what was going on in other places like London, Berlin or Amsterdam, so we built up our knowledge from scratch, finding only later that similar stuff was happening elsewhere. At the beginning, we used mostly found video footage with short inserts of wild motion graphics. Now we produce (shoot, edit, post produce) nearly all the basic material that we use.

I had been looking at Max and Nato (a graphic programming environment similar to Max designed to handle Quicktime media) a lot in the last few months but the learning curve was so steep. I discovered Isadora by chance by following a link to the Troika Ranch website and downloaded the beta version. I learnt everything quickly (note that my previous experience with programming was self taught action script - at a basic-medium level) and in three hours I had built a very complex patch that performed many different tasks. It's so different from the other application that we've been using. First of all you get a candy factory and not a candy packet. There are all the basic functions one could perform on digital video. And you have them in real time, which allows for much more improvisation. I can build and modify a patch as we play... it's the Lego for video junkies, definitely. Our VJing practice will shift with this tool, because now we do much less postproduction and can use video that is more 'raw'. For example, in Isadora you can easily bring in live video input of clubbers dancing into the overall mix.

And we're producing a live audiovisual show that is the most interesting stuff we've been working on so far, and a shift from the work in the club environment. It's called 'Città Invisibili', and it's a multiprojection event that will be premiered at the beginning of October based on city shots taken in Tokyo, LA, London, Detroit and Barcelona. The concept is of the contemporary flaneur fused with the poetry of the city jungle. We're preparing many Isadora patches to play visual and audio in a very tight relation, and it will be the first time that we'll use the application without the backup of other programs. Perhaps if one is a very skilled programmer fluent with the Quicktime API (application program interface) Isadora would not be so useful, but for the rest of us it is. I was a big player with Lego years ago, and I can't see the difference. I feel like I am playing while trying out things. And after sessions of wild patch cord dragging, I have run into consequences of a chained 'effect' that were not planned but are very interesting.

Jem Finer: Ever since I first got hold of a computer, probably the Sinclair Spectrum in the early 80's, I have been interested in programming them to make music. This interest continued and gained momentum to the point of my composing a 1000 year long piece of music, Longplayer, in 1995. I spent a few years exploring numerous possibilities for composing such a piece; artificial life, AI, chaos theory, neural nets, interaction with the environment, etc. At first the problem was that I had no language to program in, but eventually I discovered SuperCollider which I've used pretty much exclusively ever since (SuperCollider is an environment and programming language for real time audio synthesis in which you can write programs to generate or process sound).

For the last two or three years I have wanted to create a visual counterpart to the music I make. My live performances are based on a symbiotic relationship between my computer, running a library of SuperCollider patches I've written, and me. The patches do various things to grabbed or streamed input and can run in parallel (up to the limit of the cpu). I want to be able to do the kind of things I do to audio to visuals, or at least to experiment with that approach. It may well be not so interesting as new things I discover. The problem has been finding an environment in which to do this. I tried Image/ine, but had difficulty with the interface, it wasn't intuitive enough for me (something Steina mentioned earlier). I tried Arkaos too (a VJ authoring tool); never tried Nato because I couldn't get a demo, and it was too expensive to buy on spec.

So when Isadora turned up it seemed an answer to a prayer. Intuitive, flexible, not too expensive and it produced good quality images. Up to now I've had little time to explore it, but enough to know that it's worth an investment of time to get to grips with. I've just been writing simple patches to explore the objects and interaction between them. There are a few things I would like to be able to do that seem out of its range at the minute; one is to have a far greater interaction with sound. I want to be able to link things to precise frequencies for example which involves FFT (fast fourier transforms) stuff. The only way at the moment is to write this in SuperCollider and convert it to MIDI info to send to Isadora, which may be messy. I prefer to run just one thing if I can, but I have a feeling that I might need to use Jitter for some stuff. In that scenario I imagine using Isadora for any tasks not dependant on anything more complex than listening to sound in. It certainly requires less programming and the interface is simple and informative, really great. I love the simplicity of the stage set up and the rendering possibilities.

Scott: One of my interests with writing up this discussion is to provide some insight for those who may be relatively new to software like Isadora; so I think what is important is to establish the range of different types of environments and contexts the software might be used in; from VJing to installation and stage performances. However, I also don't want to misrepresent this field of possibilities by implying that Isadora is the only or even the best programme available for everything; in fact, there are several cross media synthesis programs that take advantage of the possibility to manipulate digital video material in real time. So, it's important that both Giorgio and Jem have mentioned other programs including the newly available Jitter, which has been created for the Max graphical programming environment. Another is Keystroke, which is a media mixing multiuser environment and should be mentioned as it is another development of Image/ine and Tom Demeyer has been working with that team. There are links to these programs provided at the end of this article. It is beyond the scope of this conversation to delve too deeply into each one or to make any real comparisons. However, I think Jean-Baptiste Barrière may be able to offer some interesting thoughts on Isadora in relation to some of these other softwares.

Jean-Baptiste: Well, I pretty much have them all and at least tried each extensively at some point. I consider them for their respective and different qualities and do not want to be restricted to using only one, whatever its capabilities. Until I discovered Isadora, I used Image/ine controlled by Max, starting from September 1997, for a still ongoing series of installations and performances called Reality Checks. I first used Isadora for an installation last year called "Chasing Wind: the Well of Vanities", presented in the Abbaye of Maubuisson, an ancient convent 30 km north west of Paris. In this piece, people enter through one side of a large empty room. At the center of this room is a sort of well where people can see their own image floating, mixed with other images. When they turn around they modify these images, as well as the sound in the room. Another image, much larger, is projected at the other end of the room on the wall. By moving about the space, people are triggering specific sequences of images and sounds, elements of texts (mostly extracted from the Ecclesiast in the Bible). Generally speaking, this is a very meditative installation piece, calling on the viewer to reflect on identity and death.

Technically speaking, first one computer is running Max with Cyclops, software than can detect people moving around the space by analysing the video image of the space. Then this information is transmitted by MIDI to another computer running Max to do the mapping between what is detected and what I call the 'interactive scenario': what is to happen for every 'situation' and succession of situations which I call 'trajectories'. This second computer controls with MIDI two others running Isadora; each one receiving the video input of a different camera: one for the well, the other for the wall. It also produces/ synthesizes all the sound, and Isadora is used similarly to play previously prepared video materials (with Final Cut and After Effects), and process it together with live video capture, mostly doing keying and displace.

What I appreciate with Isadora is its clarity and straightforwardness. It is very easy to use, and all sorts of interactive ideas can be implemented with it, both very quickly and efficiently. It has been growing rapidly and intelligently from the original modularization of the Image/ine model, to become an original and unique tool. It obviously grows out of Mark's extensive dual complementary experience with music and dance composition needs. It has reached a kind of ideal balance between an application and a graphical language; that is a very clear and easy one to master. Which makes a clear difference with Max for instance, and should provide it a distinct audience.

It has been a pleasure to watch it evolve and see Mark's responsiveness to users; at the moment he quite literally is collaborating with each user. I hope he will continue to be able to develop it in the same way that he has been doing until now; without losing track of his artistic projects since clearly one activity nourishes the other reciprocally. That may be difficult, while he stays the only developer or source of novelty. This is why I want to encourage him to open the possibility for users to make their own modules and/or link external codes such as Photoshop's Plug-ins or Director's Xtras. Why not also have for instance Jitter's Pluggins inside of Isadora? This would be making the most of the two paradigms, would be equally profitable for Isadora and Jitter. This is the way in which I would like to see Isadora evolve; to continue to offer new ideas related to transformations, control and interface, and at the same time allow other artist programmers to insert/ import specific code. This would be the most satisfying for users, and therefore help to provide a sustained development for Isadora.

Scott: I think this last comment will be challenging for our readers who are not so familiar with computer programming. However, it is essential to help develop the understanding amongst a broader public how a unique software like Isadora, the manifestation of Mark's creative vision and hard work, contributes to an open arena for artists to exchange ideas and materials that relate both to the tools they use as well as the artworks they will make with them. Thank you everyone, thank you Mark for taking part in this conversation.

END/ END/ END/ END/ END/ END

Contact:

Mark Coniglio troika@panix.com
Troika Ranch
Brooklyn, NY
w. http://www.troikaranch.org/

Scott deLahunta sdela@ahk.nl
Writing Research Associates, NL
Amsterdam, Netherlands
w. http://huizen.dds.nl/~sdela/

Biographies:

Jean-Baptiste Barrière has made studies in the fields of music, philosophy and mathematical logic. From 1981 to 1998, he worked at Ircam/Centre Georges Pompidou in Paris, France. Besides making his own compositions and media installation works, he has worked with other artists such as Maurice Benayoun (for whom he composed the music of several virtual reality pieces including Worldskin which won the Prix Ars Electronica 1998 for Interactive Art) and Peter Greenaway. http://www.barriere.org/

Scott deLahunta does research, writing, speaking and consultation work related to the impact of new media and information technologies on live performance arts practice with a particular focus on dance. http://huizen.dds.nl/~sdela/

Jem Finer began playing music in the 1970s, having left university with a degree in Computer Science. In 1981, he became a founder member of the Pogues, writing, recording and touring for the following 15 years. Recent work includes Longplayer, TILT, music for Copenhagen Town Square, various recordings for installations, films and television and Autodestruct I & II, a live music performance. http://www.longplayer.org (Longplayer)

Armando Menicacci has a background in music and dance studies. He is the Director of Mediadanse, a research laboratory of Paris 8 University Dance Department, a member of ANOMOS and of the Rachid Ouramdame's dance company "Fin Novembre" and serves as a consultant for various art institutions and artists. http://www.anomos.org

Giorgio Olivero and Andrea Clemente (SOFTLY.KICKING) won the VJ competition of the ArezzoWave festival (the biggest free rock festival in Italy) in 2001. Their current engagements include a residency with a crew of famous Italian DJs; several one-off clubnights in different locations around; and playing at Fiesta des Sud in Marseille in October. http://www.softlykicking.com/

Steina Vasulka is a key figure in the field of 'video art' since its beginnings. With her husband Woody Vasulka, she has won numerous awards, and their collaborative works have been widely exhibited internationally. In 1971, they co-founded The Kitchen in New York City, the celebrated media arts theatre. Exhibitions of her individual work have been seen at festivals and institutions including Centre Georges Pompidou, Paris; The Kitchen, New York; Museum of Art, Carnegie Institute, Pittsburgh; and the Whitney Museum of American Art Biennial, among many others. Steina lives in Santa Fe, New Mexico. http://www.artscilab.org

Links:

You can download and read information on the following sites about software mentioned in this article and related programs.

Isadora (TroikaTronix):
http://www.troikatronix.com/

Max and Jitter (Cycling 74):
http://www.cycling74.com/

"Putting Max in Perspective" (published in Computer Music Journal, 1993):
http://mitpress2.mit.edu/e-journals/Computer-Music-Journal/EdNotes/Max

Auto-Illustrator (Adrian Ward):
http://www.auto-illustrator.com/

Nato.0+55 (Netochka Nezvanova):
http://www.eusocial.com/nato.0+55+3d/242.0000.html

"A Discussion of NATO.0+55+3d Modular" (Jeremy Bernstein):
http://www.bootsquad.com/nato/index.html

Keystroke (a real time cross media synthesis multiuser environment):
http://www.keyworx.org/

Image/ine (Tom Demeyer):
http://www.image-ine.org/

SoftVNS (David Rokeby):
http://www.interlog.com/~drokeby/softVNS.html

Supercollider
http://www.audiosynth.com/

Arkaos
http://www.arkaos.net/

Go to Part I, back to main page or to the top of this page