Notation. Prozess. Musik.

Patrick Borgeat
*1985 in Öhringen (DE), lives and works in Karlsruhe (DE)

»Notation. Prozess. Musik.«
2017, Video presentation and interactive station

»Normally notes are points and lines on the two-dimensional surface of a piece of paper. The notation of the tone sequences is made up of signs on a surface. Nonetheless, these notes are interpreted as a temporal sequence, a chronological order. This is why music is considered the mother of all the time-based arts.«[1]   

In an algorithmic composition, the composers do not directly write a musical score, but rather describe a process with an outcome that can be heard directly. A musical notation can also emerge as an intermediary step, which is in turn interpreted by musicians. The tonal result of an algorithmic composition can be identical every time or can be completely different each time if it is influenced by the artists’ chance decisions or interventions in the process it is running or by changing external factors. The music can also be stretched out temporally to become infinite through repetitions or jumps in the process description. The computer is the suitable instrument for implementing the algorithms. Understood as step-by-step instructions, however, the command sequences can also be handled by people much as cooking recipes are. Deft arrangement of the algorithms can enable highly complex tonal structures – which in many cases were never anticipated when the algorithmic compositions were conceived – to emerge through just a few instructions

This idea is systematically taken further in live coding, with the composition process appearing onstage in this case. The algorithms are written and performed live and with frequent improvisation in the form of programming source texts. Those who are programming live enter into a dialogue with the audience and also with the process that is underway. Live coding should be understood not as a musical genre, but rather as a musical performance practice. The programmers determine the musical result, which can touch on every possible genre, from abstract noise music to jazz to electronic dance music, which is currently enjoying great popularity through the label Algorave. The idea that underlies live coding, however, is not limited to the purely auditory – visuals are also a popular medium for live coding.

Through a multi-channel video presentation of »Notation. Prozess. Musik.« on two walls facing each other at the »Open Codes« exhibition, an arc is traced from traditional notation to algorithmic composition to processual live coding by four renowned artists. Exhibition visitors can also make their own first attempts at live coding at an interactive station.

[1] Peter Weibel, »Zellulare und molekulare Musik – Zur Kluft zwischen zwei Tönen,« in: Peter Weibel, »Enzyklopädie der Medien, Band 2, Musik und Medien«, University of Applied Arts Vienna, ZKM | Karlsruhe, Hatje Cantz, Berlin, 2016, p. 383.


Bernd Lintermann
*1967 in Düsseldorf (DE), lives and works in Karlsruhe (DE)

Julia Gerlach
*1967 in Hannover (DE), lives and works in Frankfurt a. M. (DE) and Berlin (DE)

Peter Weibel
*1944 in Odessa (UA), lives and works in Karlsruhe (DE)

2012, interactive augmented reality installation, AR audio database browser for iPad

The IDEAMA (International digital electro-acoustic music archive) was created in 1990 with the aim of globally protecting the most important early works of electro-acoustic music against deterioration, and make them accessible to the public. The IDEAMA basic collection consists of over a hundred hours of music, which is part of the holdings of the ZKM | Media Library.

»SoundARt IDEAMA« presents selected works from the music archive. At four stations, works from the areas of musique concrète, electronic music, computer music, and music for loudspeakers are staged in special ways and made available to visitors. On a beam showing computer-readable codes, four topics from the IDEAMA database are marked out as examples: for each work a square code – similar to the familiar QR-code – is applicated on the floor as well as an oscillograph of the work. With the aid of an on-site rentable iPad, the work can be experienced, heard, and seen in a 3-D simulation as augmented reality experience as soon as the iPad’s camera registers the code. Here, the iPad functions as a pickup which scans the sound carrier.

Concept: Bernd Lintermann, Julia Gerlach, Peter Weibel
Curator: Hartmut Jörg
Software: Bernd Lintermann
Technical coordination: Manfred Hauffen
Production: ZKM | Institute for Visual Media


Bernd Lintermann
*1967 in Düsseldorf (DE), lives and works in Karlsruhe (DE)

2018, Interactive installation with multi-channel projection

YOU:R:CODE opens the exhibition Open Codes. The title can be read in two different ways: the interpretation “your code” indicates that in the installation visitors experience different kinds of digital transformations of themselves. Whereas on entering, a visitor still sees their familiar reflection in a mirror – the most real virtual depiction that we can imagine – the mirror image gradually transforms into a digital data-body until finally, the visitor is reduced to an industrially readable code. In the end he/she breaks free from the virtual depiction, and is materialized in a flip-dot display. The second way of reading the piece’s title, “you are code,” emphasises that we ourselves consist of code, which amongst other things is manifested in the genetic code. The genetic code constitutes the algorithm of life and from birth it determines what we do. In current research projects synthetic DNA strands even serve as long-term storage for digital data. And for the data analysts and artificial intelligences operating in cloud computing, too, which via smartphones give us our daily instructions for acting, we are only perceived in a mediated way in the form of sensor data and via our electronic traces and expressions – to them we are codes.

Concept, realization: Bernd Lintermann
Audiodesign: Ludger Brümmer, Yannick Hofmann
Technological support: Manfred Hauffen, Jan Gerigk
Setup, planning: Martin Häberle, Nikolaus Völzow
Production: ZKM_Hertz-Lab
Inspired by: Peter Weibel

The Man with the Personal Computer

Robert Luxemburg
a.k.a. Sebastian Lütgert, * 1972 in Bielefeld (DE), lives in Berlin (DE) and Bombay (IN)

The Man with the Personal Computer
2010, Digital Video, 4 min, Public Domain
Remastered Version, 2017, for Joulia Strauss

The Man with the Personal Computer is a mosaic of the last four minutes of Dziga Vertov’s The Man with the Movie Camera, rendered from matching stills taken from the last four minutes of Dziga Vertov’s The Man with the Movie Camera. Unlike most other films, this one is best seen from afar, or on a small screen.

Visitors in front of rain from CAMP rain
*2007-present, open-source media art platform, housed at CAMP in Mumbai (IN) and 0x2620 in Berlin (DE) rain
2018, Automated Scroll of Thumbnail view of ID’s in, sorted by date created is an online archive of text-annotated video material, primarily footage and not finished films contributed by filmmakers, artists, cultural organisations, predominantly from, but not limited to India. went online on 29th February 2008 from Bombay; a collaboration between a group of friends; artists, filmmakers, lawyers, coders, cinephiles and free software enthusiasts living between Bangalore, Bombay and Berlin. The entire collection is searchable and viewable online, and is free to download for non-commercial use.

We saw not only as an open-source, collective, non-state video repository of a recent pasts and continuing presents, but importantly as a way of opening up a set of images, intentions and effects present in video footage; resources that conventions of video-making, editing, political exigencies and spectatorship have tended to suppress, or leave behind. This expanded treatment leads us into lesser-known territory for video itself, a slower reading of reserve material, beyond the finite documentary film or the online video clip.

The design of the archive makes possible various types of “viewing” and contextualisation: from an overview of themes and timelines to much closer readings of transcribed dialogue and geographical locations, to layers of “writing” on top of the image material. Views for each ID include an info page, map view, documents view, edit and player view. Timeline options include waveform, keyframe and slit-scan. Descriptions, keywords and other interpretative annotations have been placed on timelines by both archive contributors and users. allows for publishable video essays and online edits traversing through material in the archive.

Daniel Temkin
*1973 in Boston (US), lives and works in New York City (US)
2018, Video with two interactive touchpads

Conventionally codes, programming languages and software are seen as language dictionaries helping humans to encode processes and logic into sequences of operations to be executed by machines. Depending on what we would like to achieve, we chose a set of tools.

Yet code can be more than a fixed tool whereby creativity does not only reside in its output but also in its handling. Each language has different approaches that lead to the same result: this variety allows for artistic creation by disrupting common workflows through different visual coding styles to abstract constructs of logic.

The next steps consist of the creation of new languages where the process and uses are not the main motivation any more.  It is about the breaking boundaries of existing languages, creating a frame set by the code or the input, and output method, and less by the information processed.

The installation invites visitors to explore the world of codes in all its creativity and whimsy, its playfulness, and potential for disruption. The video gives an insight into this collection of codes, while visitors can explore the many facets of the archive on the tablets.

Installation view »Open Codes«

Symbolism in Circuit Diagrams

Where Dogs Run
Founded in 2000 in Yekaterinburg (RU)

Alexey Korzukhin
*1973 in Sverdlovsk (RU)
Olga Inozemtseva
*1977 in Jalutorovsk (RU)
Natalia Grekhova
*1976 in Kamensk-Uralsky (RU)
Vladislav Bulatov
*1974 in Sverdlovsk (RU)
All live and work in Yekaterinburg (RU)

»Symbolism in Circuit Diagrams«
Since 2006 ongoing, mixed-media installation

The installation consists of six electrical circuits, which depict the encoding of six classical poems into electrical circuitry symbols. Behind the podiums are six films animating this process of transforming the poems into electrical diagrams.

In electronics, a diagram contains standardized symbols to schematize the work of an electrical network. These symbols are succinct metaphors, which transmit a vast amount of technical information. The artist collective investigates the »linguistics« of these pictograms, and interprets their possible translation into a poetic language. After converting the six poems into the »vocabulary« of electrical circuits, the artists create DIY assemblages based on the developed diagrams. The resulting circuits have a metaphorical connection to the content of the poems. For instance, Pushkin’s »Prophet« took the form of a radio set.

Following the linguistic experiments in sound symbolism and language creation of the Russian Futurists known as zaum, »Symbolism in Circuit Diagrams« can be seen as an attempt to create the foundations for a universal language, for the laws of physics and electricity, in particular, are universal for all living beings and machines.