Chapter 22
Synthetic Video
Video Synthesis is the visual equivalent of electronic music and one of the first truly electronic of the electronic visual arts. A sort of abstract expressionism gone mobile and fluid, its patterns and colours shift over time; waves of colour slewing across the screen and shapes that refer to nothing in the world but geometry, echoes of shapes and objects, edges and details of things highlighted and coloured in ways previously unseen. All these are aspects of the synthesised image.
Paik’s initial manipulations (in his March 1963 exhibition, Exposition of Music. Electronic Television, at the Galerie Parnass, see Chapter 3) involved both altering the TV raster by adding audio waveforms to the deflection circuits thus distorting the size and shape of broadcast images, or microphone signals creating fireworks of points of light on the TV screen , [1] and altering the signal pathway in the video display electronics to mess with the synchronisation or contrast and brightness elements of the TV picture. [2] He subsequently began using degaussing coils [3] and then permanent magnets (Magnet TV, 1965) [4] and later using signals imposed on the deflection coils augmented with the addition of a third deflection coil to add an extra complexity to the image (Dancing Patterns, 1966), and extended the possible interactions with later versions of Participation TV (1963-66). [5] Paik hinted at synthetic video in 1963 and manifested it with the Paik-Abe video synthesiser built between 1969 and 1971.
In late 1963 Paik had visited Japan. There he met the engineer Shuya Abe in the Akihabara electronics district of Tokyo and they began a long and productive collaboration. [6] Their first project together was the robot K456 [7] but more importantly they also began the deeper exploration of what electronic manipulations could be made to the video signal to effect the pictures. [8] Paik said in 1965 that he envisaged:
the development of an adapter with dozens of possibilities which anyone could use in his own home, using his increased leisure to transform his TV set from a passive pastime to active creation. [9]
As Paik began to show, it is possible to manipulate the actual video signal, filtering it and changing its colour, and thereby producing entirely different results compared to deflection circuit manipulation. Here begins the story of the video synthesiser.
Eric Siegel is credited with the first video image colouriser, his Processing Chrominance Synthesizer (PCS) of 1968. [10] He followed up the colouriser with his Electronic Video Synthesizer (EVS) in 1970, of which he says
On the EVS you just have to put in sync, and everything is composed for you right inside the synthesizer. But you can put in live cameras too, and do things that involve pictures and synthesizer images. [11]
The Siegel devices were rapidly followed by Stephen Beck with his Direct Video Synthesiser and his Video Weaver [12] and numerous other electronic manipulations of the video image were developed and refined over the decade from 1969 with the further development of video synthesisers.
In 1969 Paik invited Abe to Boston to build a video synthesiser. The first version was completed in 1970 and used at WGBH-TV for a four hour live production, Video Commune. [13] The Paik-Abe synthesiser was a combination of video-camera mixer in which the cameras were set up to supply an image in one of each of the seven colours of the rainbow and scan-manipulation device which produced colour video images that range from realistic images to pure abstract patterns and every shade in between . [14]
Fig.1: the CoxBox
Prior to this, in Britain, experiments regarding colour television on the 405 and 625 line systems led to the development of colour encoders and, in 1967, to the Michael Cox colourizer [Fig.5] which divided up the black & white television signal into black-to-dark grey, mid-grey, and light-grey-to-white bands each of which could be coloured by the use of R, G, and B controls. The device was intended to allow the production of coloured titles and captions from images shot on a black and white rostrum camera for super-imposing into colour television programmes. The ABC in Australia bought several and they were also available to hire from a local equipment distributor. Bush Video hired one on a regular basis for colourizing a number of their pieces. [See Bush Video in Chapter 6.]
In Britain the notion of the synthetic sound or image was first given a platform in Jasia Reichardt's Cybernetic Serendipity at the Institute of Contemporary Art (ICA) in London 1968.[15] There was a considerable amount of music composed and played by computers, computer choreography for dance, electronic visuals of various kinds and several large installation structures using audience feedback (interaction) to modulate the process of various analogue and digital machines. A significant one was Gordon Pask's large scale installation Colloquy of Mobiles, a very early example of machine learning. In an extract from his description of the work, it is
“a group of objects, the individual mobiles, that engage in discourse, that compete, co-operate and learn about one another. Their discourse evolves at several levels in a hierarchy of control and a hierarchy of abstraction. if you find them interesting then you can join in the discourse as well and bring your influence to bear by participating in what goes on. Each individual mobile has a set of programmes that determine its motions and its visible state. Each individual learns how to deploy its programmes in order to achieve a goal.” [16]
Visual display works included one of many variations of Nam June Paik's Dancing Patterns series of magnetically manipulable TV sets. These were described as “responsive environments [in which] the observer can actually touch things and make it the way he [sic] likes it.” [17] Tango Electronique, the version shown at Cybernetic Serendipity, sits passively showing “shimmering coloured lines. You turn a knob, and the screen explodes into patterns.” [18] Thus, to make it into an artwork the viewer actually has to physically engage with the work. As Norman Bauman, who wrote the catalogue entry for Paik's works in the exhibition, comments:
“Passive art is a real threat to our culture. If the viewer does not enter art, he [sic] cannot possibly hope to understand it. A major part of the thrill of Mr. Paik's exhibit is pushing the button yourself, and knowing that you made that little blip there.“ [19]
And, as Paik, quoted in the same entry, asks:
“... why not paint with magnetic fields, the working stuff of our technology to get a completely new and completely different feel? The beautiful patterns of
'loga Cage 3.5√McLuhan = sorry'
Norbert Wiener
the black and white pattern which is generated from a single vertical line,... [20]
The equation more or less sums up Paik's influences after he arrived in the US.[21]
Paik also showed Robot K-456 described as a female robot known for her disturbing and idiosyncratic behaviour. [22
Other television works at Cybernetic Serendipity included Roger Dainton's piece called Simulated Synasthesia [sic], of which he noted:
I wanted to explore effects as a means of transforming from medium to another, from sounds in time to light in space and time.
From the frequencies of the music, all the colours of the spectrum are reproduced. Some patterns appear to be stationary, or float very slowly. This is because the music at this moment has a close relationship in frequency to the display, thus the structure and texture of the coloured patterns is formed. [23]
From the look of the object in the picture in the Cybernetic Serendipity catalogue it was a TV set with an added package of of electronics below it. What those electronics did I have not been able to discover, nor have I been able to find any images of the actual visualisation produced, i.e. whether it was manipulated raster (producing Lissajous figures) or waveform generator (oscillator) circuits fed into the RGB driver transistors of the TV (producing colour fields) that created the images. Though I do suspect the latter.
There were also computer generated films from two American computer artists, John Whitney's Pemutations (1966), and Ken Knowlton's film on computer animation using a mosaic technique, A Computer Technique for the Production of Animated Movies, (n.d., probably 1966 or 67).
Later, in 1973, Richard Monkhouse was working for Peter Zinovieff's Electronic Music Studios (EMS) who had already been manufacturing audio synthesisers such as the VCS 3 (the Putney , 1969) and the Synthi A (1971) and AKS (1972) in London [24]. He was developing audio monitoring tools for Zinovieff's studio and went on to develop the hybrid analogue and digital Spectron video synthesiser for EMS. In an interview with Chris Meigh Andrews, Monkhouse talked about the project started:
I was given the job of designing some video sync circuitry. So I got a colour video monitor and a sync circuit and I started to plug direct RGB video signals from the digital timing circuit into the colour monitor. I suddenly realised how amazing pure colour video imagery actually is. In fact I got so excited by the pure colours that I was getting that I damaged the Trinitron monitor feeding in stronger and stronger colours, I heated up and bent the masks!!
These images and colours intrigued and excited me and led me to design some circuits which did more than generate stripes and squares.
I was interested to explore what it could do within its limitations, and to explore what I thought its powers were given the limitations of what I had available to me. I wasn't in an art college, and I didn't have access to a lot of colour cameras. I only had the resources that I got from EMS. (When I left, I did, in fact replicate those facilities.) [25]
EMS Spectron
Fig. 6: the EMS Spectron video synthesiser
Spectron (aka. Spectre) was a hybrid digital/analogue video synthesiser. It could generate its own patterns or a video camera could be input to it. Patterns and camera input could be mixed and colourised and the output then be sent to a monitor and a video recorder.
It had two pin-matrix patch panels, one for combinations of the digital signals and one for analogue control of the two top-level shape generators, the hue and luminance of the output picture and the grey scale slice levels of the video input, thus controlling the brightness (luminance) bands over which they operated.
Four shape outputs were produced from 2 basic shapes, a circle and a saw-tooth pattern. These were combined into 16 preset shapes made up of abstract circles, arcs, segments and saw-tooth geometric patterns made up from logic gates that combined the basic circular and saw-tooth shapes. Patterns could be digitally delayed so that, say, edges only appeared at an output. The patterns had several outputs and could be independently deployed.
The shapes could be modulated by analog control voltages from the analog control matrix or from an external audio synthesiser. Two sine or square wave oscillators produced horizontal and vertical bands locked to the line and frame syncs. It also had patchable sliders, a random voltage generator, bass and treble audio filter outputs.
Input video was broken into 7 grey levels and these were sent to colour busses with their own Luminace, R-Y and B-Y level controls giving a wide range of colours. This was not natural colouring but digital colour inserted into the flat bands of the grey-scale image.
All the digital signals could be gated with video-key outputs, the shape generators and the horizontal and vertical bands, thus allowing patterns to be inserted into video camera images. Outputs could be given 7-bit luminance and chroma values. All theses signals could be sent to colour allocation inputs which then produce Red, Green and Blue or encoded video outputs.
It was a voltage controlled synthesiser so audio waveforms and music could be used to modulate things like pattern position, size and colour. [26]
In 1975 Warren Burt, [27] an American electronic music composer teaching at the Latrobe University Music Department (where Keith Humble was the head of department) bought a Spectron for the department's electronic music studio. He and his students, particularly David Chesworth, proceeded to use it to great effect. See Burt and David Chesworth below.
The Fairlight Electric Paintbox
In Sydney Fairliht Instruments, famous for the Fairlight CMI, got interested in video as well as music. They developed a device that would insert colour bands of colour into each of 6 levels of brightness (grey levels).
The Fairlight Electric Paintbox, otherwise known as the colouriser, was developed by Peter Vogel of Fairlight Instruments in 1975. It was originally designed for colourising the output of video caption cameras for inserting titles into video productions. Peter also built a prototype of an image synthesiser, but it was never developed as a product. Fairlight lent me one for the Open Processes show at Watters Gallery in February 1977 in which it featured heavily.
The Fairlight Electric Paint Box (or Colouriser)
The Electric Paintbox installation manual describes it as:
a six-level colour synthesiser incorporating many features which make it more versatile than a caption colouriser. It accepts a black-and-white video input (e.g. from a camera) and assigns a different colour to each of six shades of grey in the picture, generating a colourised image as its output."
This colouriser worked by taking in the monochrome image from a camera or video-tape and slicing the image area up on brightness levels as selected by the five slice (or "turnover") level sliders. This created areas of the image having similar gray values. In detail: The input video signal is fed to a set of six comparators in the device. Each comparator has its comparison voltage set by the slice-level sliders. These act as six keyers. When the voltage level (brightness) of the input signal is less than the first level of comparator all outputs are off and the lowest set of colour sliders determine the colour of the image output. When the voltage level of the input signal is greater than the comparison level of the first comparator it turns on, shutting off the first level switching signal, and the second set of colour sliders determine the colour of the output image. When the voltage level of the input rises to be greater than the comparison level of the second comparator it in turn switches on. This switching signal turns off the previous outputs and allows through the colour determined by the third set of colour slider. And so on for all six levels.
The colour sliders are Red, Green, Blue triples. For each sliced grey area the amounts of Red, Green and Blue making up the desired colour are switched into an RGB output signal. These are then sent to a colour video encoder and turned into a standard composite video signal.
|
|
|
|
Te grey-scale from black (L) to white (R)
|
|
|
|
| Finally, here is an example of what the colouriser could do. A frame from Stephen Jones' Tai Chi Transforms |
Video Synthesis
Slightly before this, Melbourne-based technician/artist, John Hansen, was mucking around with Lissajous figure generators. He subsequently built a video synthesiser (in 1974) and in the early 1980s developed one of the first computer graphics paint systems.[28]
I shall come to Hansen's, Burt's and Chesworth's work duly.
There are four classes of video manipulation operating in video synthesisers,
raster (or scan) manipulation which changes the shape of the picture using magnets or signals superimposed on the deflection coils,
image processing which variously colours and filters the image.
direct synthesis of the image using pattern generators and waveform synthesis.
video feedback, which produces very slightly delayed versions of a frame that are then fed back into the video image on the next frame causing an echo effect.
The equipment used to produce this “synthetic” video rarely predicted the image (or sound) that was going to come out of it. Unless you had a thorough understanding of the circuit and what such circuits could do you had little means of knowing (other than your own experience) of what the equipment might be capable. Unlike a paint brush or a cello which itself entails its own productions, electronics is just electronics, circuits and wiring, the use of which could be applied to video, sound, computers and so on, all having the same basic components.
At the time that the technology of the video portapak was appearing, other kinds of electronic technologies which appeared (i.e., the first commercially available integrated circuits) also allowed for a major shift in electronic music through the use of analogue computer technology [29] and the first sound synthesisers, which, through the research of people like Robert Moog [30] in the US, and Peter Zinovief at EMS [31] in Britain, grew out of the audio equipment testing laboratory. [32] The people experimenting with the new video capability were often also working in electronic music, so it was natural that some of the designers of audio synthesisers also started building video synthesisers. [33] The video synthesiser was really the same piece of equipment but with its output stages set up to drive video monitors and oscilloscopes in various ways.
One of the primary ideas behind a lot of video experimentation was the production of visual music , a kind of pictorial equivalent to the experimental music of the day, having its antecedents in the synsthesia of German poet E.T.A. Hoffman [find a reference that mentions Hoffmann's synaesthesia] and the colour organ of Scriabin. [34] In its new technological form, this non-referential synsthesia is the dance of the electron across the screen as driven by the music, almost independently of any kind of known visual source or natural image.
I argue that the artificial (electronically synthesised) image has as much legitimacy as the electronically generated sound and that the two go well together, being derived from the same source waveforms, even if handled through quite different means. It is as though the visual image could be played in much the way one played an analogue sound synthesiser, with the patching of connections and the twisting and turning of knobs, giving the feel of real-time control. The electronics becomes an instrument as much as any acoustical musical instrument and much the same could be experienced in live video synthesis. [35]
The synthesised image comes out of thin air , so to speak, and bears no necessary relation to the real world. There is nothing already known to hang it on, from which to extract meaning and thus dilemma starts, how do we inject meaning into this stuff? How do we use it? To what does it point, refer? What does it signify when there is nothing being pointed at, just the electronic image pointing... It is almost a one hand clapping problem, signifier without signified, pointing only to itself. Think about Paik's TV Buddha.
But then, given that the video mage is being inscribed onto the phosphor screen, the question arises, what actually is being written about and to what extent can it be understood? What are the semiotics of this clearly abstract form of writing? What were the authors of these inscriptions trying to say to their viewers/readers, regardless of whether the viewers knew how to read the texts? I shall have more to say about this when looking at some of Bush Video s work (below) and the answer is extended through Doug Richardson s description of what it was that motivated him with his computer art developments, for which see Computer Art and the PDP-8.
Over the period from 1978 to 1986 I built three analog video synths and started, but never completed, a digital video synthesiser.
| The first synth was a test of a number of ideas and primarily taught me a great deal about electronics.
It consisted in a number of video oscillators which were not synchronised to the video syncs. A vertical interval switcher which allowed selection between four inputs. These could be cameras or the video oscillators. A keying circuit which could take a camera input or an external key input, eg. one of the video oscillators. A function generator which could voltage control the video oscillators and the video signal mixer circuit. The signals generated by the various modules could be selected into a colour video encoder which gave a limited range of colour. I used this synth with a couple of, by then antique, Marconi monochrome cameras and a Marconi Sync Pulse Generator that I'd bought in Brisbane from the Uni of Queensland Audio-Visual Unit, and the old Bush Video vision mixer. The only tape I made with it was the SPK video. |
|
 |
|
4 frames from the SPK video.
| The next synth (built in 1979) was a more sophisticated version of the first one.
It consisted in: two non-synchronous voltage controlled oscillators that could be set for horizontal or vertical video frequencies, a voltage controlled function generator that produced sine, triangle or square waves and two pulse width modulation oscillators that could be made synchronous or not, as desired. These oscillators could be patched to four inputs of an 8 x 8 switching matrix. Four video inputs: camera, video tape or other effects processor could be sources for the other four inputs to the switching matrix. The matrix then fed a key generator and three video mixers which could be controlled by any of the video, key or oscillator/function generator sources. These three mixers supplied three outputs, which could then be considered as red, green and blue sources and fed into a colour encoder. Alternatively one output could be fed to, say, the Fairlight Electric Paintbox. The output of the Electric Paintbox could be fed back into the switching matrix and mixed with an off-tape source. This is what I did for most of the Severed Heads concerts where I played the video synths live on stage. I made a considerable number of videos for Severed Heads. Jetlag is but one of them. |
 |
 |
 |
 |
 |
| Images produced with Video Synthesiser no.2 | ||
| The Pattern Generator, which I called my "Mondrian Generator".
It consisted in: a collection of synchronous oscillators running at vertical and horizontal frequencies. The oscillators all produced square waves which are notionally digital signals. These signals were selected by manual or automatic switching or by random selection using a pseudo-random number generator. The selected signals were sent to the inputs of three three-bit digital-to-analogue converters which made up the red, green and blue colour values which were sent to the colour encoder for output to a monitor or video-tape recorder. I used this machine for an installation in the 1982 Biennale of Sydney and also live with the Melbourne based experimental music group Laughing Hands in some shows we put on at the Departure Lounge in Sydney in 1981. |
 |
 |
 |
 |
|
|
||
I used the main synth (number 2) for years at Heuristic Video. It was one of many devices that went into production of the video clips for Severed Heads and other bands we post-produced there. It was also used live on stage when I was performing as a member of Severed Heads from 1983 to 1992.
Endnotes
1 Decker-Phillips, 1998, op cit, p.38.
2 See Electronic TV & Color TV Experiment by Nam June Paik , in Paik, 1974, Videa 'n' Videology, Syracuse, N.Y.: Everson Museum of Art, p.8. The exhibition is described in Decker-Phillips, 1998, op cit, pp.36-40.
3 See pictures in Paik, Nam June (1974), op cit, Syracuse, N.Y.: Everson Museum of Art, p.14.
4 See discussion of Magnet TV in Decker-Phillips, 1998, op cit, p.63.
5 See discussion of Participation TV in Decker-Phillips, 1998, op cit, pp.66-8.
6 Bu mann and Matzner, op cit, 1993, p.146.
7 Pictures of which are in Herzogenrath, 1983, op cit, p.45; Decker-Phillips, 1998, op cit, p.75; and Stoos, Toni and Kellein, Thomas (eds) (1993) Nam June Paik: Video Time Video Space, New York: Harry N. Abrams., pp.28 and 78.
8 Their manipulations are described in Paik, 1974, op cit,.p.8.
9 Paik, 1974, op cit, p.11.
10 Dunn, David (ed.); Weibel, Peter; Vasulka, Woody and Vasulka, Steina (curators) (1992) Eigenwelt der Apparatenwelt,- Pioneers of Electronic Art, Linz, Austria: Ars Electronica, pp.116-121. See p.120 for the electronic detail.
Shamberg et al, 1971, p.44.
11 Dunn, et al, 1992, op cit, pp.116. See p.120-1 for the electronic detail.
12 Dunn, et al, 1992, op cit, pp.122-125.
13 Decker-Phillips, 1998, op cit, pp.152-155.
14 Connor, Russell (1971) electronic art III - Paik-Abe video synthesiser with Charlotte Moorman, catalogue for an exhibition at Galeria Bonino, New York, 1971. See also Dunn, et al, 1992, op cit, p.129.
15 Reichardt, Jasia, Cybernetic Serendipity: the computer and the arts, catalogue for the exhibition at the ICA, London, August 2 October 20, 1968. Published as a special issue of Studio International, London, 1968.
16 Gordon Pask in Reichardt, Jasia, op cit, p.34-35.
17 Norman Bauman in Reichardt, 1968, op cit., p.42.
18 Ibid, p.42.
19 Ibid. p.42.
20 Ibid, p.43
21 Paik also showed Robot K-456 described in the caption to a picture of it as a female robot known for her disturbing and idiosyncratic behaviour.” From Bauman, in Reichardt, 1968, op cit., p.42. Why the female is known for her disturbing and idiosyncratic behaviour is beyond me.
22 Roger Dainton in Reichardt, 1968 op cit., p.43.
23 Graham Hinton, 1998, EMS: The Inside Story
25 Richard Monkhouse in an interview with Chris Meigh-Andrews.
26 Kirk, David, EMS Spectron Colour Video Synthesiser , Focus: Video & Audio Visual Review, June 1975, London, Link House, Dingwall Avenue, Croydon.
28 John Hansen web-page at Synthetics.video
29 During the 1950s, composers in France and Germany were using the facilities of the radio studio maintenance workshop to make sounds for tape. Northwest German Radio in K ln (Cologne) set up an electronic music studio using oscillators and other test instruments specifically to make strictly electronic sounds. The original approach of the Cologne Radio studio was a form of additive synthesis. Karlheinz Stockhausen is the best known of the composers who used the studio, but the emphasis on the pure electronic sound was soon hybridised with recorded concrete sounds in his Gesang der J nglinge. [Dunn, et al, 1992, op cit, p.32]
It is just a short step from assembling a collection of test oscillators on a bench to putting them in a box with filters and other modifiers and producing a sound synthesiser. Once you have an electronic sound studio in a room, it only takes a little redesign and miniaturisation to put it into a box and make it cheap and portable so that musicians could buy a synthesiser and take it home or to the concert hall. [Dunn, et
al, 1992, op cit, p.38] As demonstrated by Max Matthews at Bell Labs in 1957. [Dunn, et al, 1992, op cit, p.45]
30 Pinch, Trevor and Trocco, Frank (2002) analog days The invention of the Moog synthesizer. Cambridge, Mass.: Harvard University Press.
31 As well as the Spectron (above) EMS produced the VCS3 and the Synthi A and Synthi AKS which were the main synthesisers used by people like Philippa Cullen and Stan Ostoja-Kotkowski. See Pinch and Trocco, 2002, op cit, ch.14.
32 During the 1950s, composers in France and Germany were using the facilities of the radio studio maintenance workshop to make sounds for tape. Northwest German Radio in K ln (Cologne) set up an electronic music studio using oscillators and other test instruments specifically to make strictly electronic sounds. The original approach at the Cologne Radio studio was a form of additive synthesis. Karlheinz Stockhausen is the best known of the composers who used the studio, but the emphasis on the pure electronic sound was soon hybridised with recorded concrete sounds in his Gesang der J nglinge. Dunn, et al, 1992, op cit, p.32]
It is just a short step from assembling a collection of test oscillators on a bench to putting them in a box with filters and other modulators and producing a sound synthesiser. Once you have an electronic sound studio in a room, it only takes a little redesign and miniaturisation to put it into a box and make it cheap and portable so that musicians could buy a synthesiser and take it home or to the concert hall. [Dunn, et al, 1992, op cit, p.38]
33 This development occurs, for example, with the production of the EMS Spectron, a device that was brought to Australia for the Electronic Music Studio at Latrobe University Music Department by Warren Burt in 1976.
34 Cytowic, Richard E. (1995) Synesthesia: Phenomenology And Neuropsychology: A Review of Current Knowledge Psyche, vol.2, no.10, originally available at <"http://psyche.cs.monash.edu.au/v2/psyche-2-10-cytowic.html">, now on the Internet Archive.
https://web.archive.org/web/20050203005711/http://psyche.cs.monash.edu.au/v2/psyche-2-10-cytowic.html,
Further commentary on this article is provided by van Campen, Cr tien (1997) Synesthesia and Artistic Experimentation Psyche, vol.3, no.6, originally available at <http://psyche.cs.monash.edu.au/v2/psyche-3-06-vancampen.html>.
See also Ramachandran, Vilayanur S. and Hubbard, Edward M. (2003) Hearing Colors, Tasting Shapes. Scientific American, vol.288, no.5, May 2003, p.53.
35 My own work in the 1980s with the electronic music band Severed Heads was, for me, a strong demonstration of this involvement with the machine.