A creative person is always interested in bold experiments that open up new horizons and possibilities. The fantastic idea of painting music, creating unique paintings that embody graphics and sound, goes back to the beginning of the 20th century. In this article I will talk about the history of the issue, as well as two of my developments that allow you to do something amazing - record and play sounds in graphical form.
Background
In 1904, French inventor Eugene Augustin Last presented a prototype system for optical recording of sound on film, and in 1911 staged what may have been the first ever screening of a film using the new technique. The era of the decline of silent cinema and revolutionary discoveries in the field of synthetic sound began - for the first time it was possible to obtain a simple, convenient and very visual way to manage audio information.
At the end of the 1920s, when working on one of the first Soviet sound films, the advantages of such technology were noted by composer Arseny Avraamov, designer Evgeny Sholpo and director-animator Mikhail Tsekhanovsky. The logical chain was built as follows: if we clearly see a track with a recorded sound wave, it means that we can create the same wave artificially, simply by drawing it by hand. What if you put an ornament there, a complex combination of patterns or primitives of Euclidean geometry? How fantastic will the result be? After all, in this way you can draw a completely unique sound that does not exist in nature, and music can be written without real instruments, microphones and performers.
Several laboratories soon began studying these questions. And as a result, optical phonogram synthesizers appeared: “Variofon” by Evgeniy Sholpo, “Vibroexponent” by Boris Yankovsky, Nikolai Voinov’s machine for marking “combs” of paper - the basic fragments of synthesized sound. To the ear, all this was very reminiscent of modern 8-bit music, but with a greater degree of freedom: any form of vibration, unlimited polyphony, the most unimaginable rhythmic patterns. Just think about it - an optical synthesizer, a music computer in the thirties of the last century! But these are just flowers. The thinking of Soviet engineers went further.
Unlike his colleagues, acoustician Boris Yankovsky was one of the first to realize that to create complex, life-like sounds, it is not enough to describe the vibration shape alone. The most important part of acoustic information is the spectrum, which clearly defines the frequency composition of the sound, its color, by which we give such subjective definitions as bright, warm, metallic, similar to a human voice, and so on.
Yankovsky began to structure the basic spectrum graphs into a kind of “periodic table” of sound elements, while simultaneously developing algorithms for their processing and hybridization to obtain new sounds based on “spectrostandards”. Unfortunately, changes in the country and the war did not allow Yankovsky to bring his work to its logical conclusion.
The topic was continued by his acquaintance, the young inventor Evgeny Murzin, impressed by the developments in the field of “graphic sound” and conceived a grandiose project - a universal photoelectronic machine capable of synthesizing any sound, any musical structure by drawing a spectrogram (dependence of the spectrum on time) on a special canvas without distracting operations like developing and drying film. This would simplify the painstaking work of the composer, providing unprecedented freedom for creativity.
Literally on his knees, working in the evenings in the room of a two-story barracks, Murzin completed a working model of the apparatus in 1958. The device weighed more than a ton and outwardly had little in common with a musical instrument in the classical sense. The invention was named “ANS” in honor of the composer Alexander Nikolaevich Scriabin. Despite appearance, ANS became a global sensation, decades ahead of its time and very successfully fitting into the period of cosmic euphoria with its unique atmospheric sound.
The ANS is somewhat reminiscent of a modern scanner, only it is not the scanning strip that moves in it, but the surface with the image itself - a large glass plate (score) covered with opaque paint. The paint is removed in the right places with a thin cutter, forming a drawing of a spectrogram of a musical work. The score moves smoothly, passing over the hole from which comes an intermittent “modulated” beam of light from an optical-mechanical generator of pure sound tones based on five special optical phonogram discs. Some of the light passes through the transparent areas of the score, after which it is focused on a set of photocells, from which the sound is released, ready to be played, in the form of oscillations of electric current.
The heart of the ANS is the aforementioned disk with a pattern of 144 tracks (like on a gramophone record), the transparency of which varies along a sinusoid with a certain frequency. The frequency difference between adjacent tracks is 1/72 octave. Thus, one disc contains two octaves, and the octave is divided into 72 pure tones - Murzin considered the classical 12-tone temperament to be a significant limitation. Essentially, each disc is an optical implementation of the Fourier transform algorithm that underlies many modern software synthesizers and effects. This is par for the course now, in the days of gigahertz and gigabytes, but 50 years ago it was simply incredible - a spectral synthesizer capable of playing 720 pure tones simultaneously! It’s not for nothing that ANS is considered the world’s first multi-voice musical synthesizer.
If you think you've never heard ANS sounds before, you're probably wrong. Just remember Andrei Tarkovsky’s films “Solaris”, “Mirror”, “Stalker”, mesmerizing with the magical music of Eduard Artemyev. Or a nightmare scene from Leonid Gaidai’s comedy “The Diamond Arm.” It is worth noting that Artemyev’s career as an electronic composer began precisely with his acquaintance with ANS and its creator in 1960. In addition to Artemyev, Alfred Schnittke, Edison Denisov, Sofia Gubaidulina, Stanislav Krejci managed to work with the instrument, and the sounds of ANS were used at different times in their music by such groups as Coil and Bad Sector.
Unfortunately, only one copy of the ANS synthesizer, manufactured industrially at the end of 1963, has survived to this day. It is located in Moscow in the Glinka State Museum of Musical Culture. Despite the difficult fate, the device is still in working order and from time to time plays for museum visitors under the careful supervision of Stanislav Krejci. For those who are far from Moscow or would simply like to experiment with the sound of the ANS at home, there is a software simulator called Virtual ANS.
Virtual ANS: graphic editor
The development of Virtual ANS has been carried out by the author of this article since 2007. The goal of the program is to recreate as much as possible the key features and atmosphere of the iron ANS, while expanding the original idea taking into account the rich capabilities of modern computers. Among the main differences:
- The program is cross-platform (Windows, Linux, OS X, iOS, Android), which allows you to enjoy working with the instrument anywhere and on anything: from a cheap phone to a powerful studio computer;
- the number of basic pure tone generators is now limited only by the user’s imagination and the speed of the central processor;
- it became possible to convert back from sound to spectrum.
Virtual ANS is a graphic editor with a classic set of tools: primitives, brushes, layers, effects, loading/saving PNG, GIF, JPEG. But the picture that you see on the screen is actually the score of a musical work (also known as a sonogram or spectrogram), which you can listen to at any time or listen and draw at the same time. The score breaks down the composition into “sound atoms” - indivisible pieces of pure tones (sinusoidal oscillations). Horizontal - X time axis (from left to right). Vertical - pitch Y (from bottom to top from bass to high frequencies). The brightness of an individual pixel is the volume of a pure tone with frequency Y at time X. The image of the spectrum is divided vertically into octaves, an octave into 12 semitones, a semitone into even smaller, barely perceptible microtones, for an accurate description of any musical system, any the most unimaginable timbre. If we draw a horizontal line one pixel thick on the ANS score, we will hear a single sinusoid with a constant frequency. The thicker the line, the more pure tones it will contain, the more complex the sound will be, and the more closely the sound will approach white noise, saturated with overtones of all frequencies in the audible range. The combination of such lines with other figures of different brightness gives unexpected and interesting sound variations.
While working on Virtual ANS, an interesting idea came up. A fragment of an audio file or, say, a voice recording from a microphone can be converted into an ANS score, that is, into a spectrogram - a picture with sound encoded in it. And this sound can be easily reproduced back using the same program. There is a natural desire to print out a picture of the spectrum on a printer and get a paper copy of your voice or music.
It was for these purposes that PhonoPaper was conceived - another project that inherits the ideas of the sound revolutionaries of the last century. What is PhonoPaper?
- The image format in which the audio is encoded. This code differs from the ANS spectrogram only in that special markers have appeared at the top and bottom, by which the reading device accurately determines the boundaries of the block with the spectrum.
- A scanner application for reading PhonoPaper codes in real time using a camera.
- Recorder application for converting 10 seconds of audio into PhonoPaper code. Although for more precise control of the conversion it is best to use the Virtual ANS described above.
The PhonoPaper code can be called analog, since it does not contain digital information, and it itself can be written on any available surface (paper, plastic, wood). For this reason, various kinds of distortions are not critical for him: in poor lighting and crumpled paper, you will at least hear the “outlines” of the original message. To listen to the code, you do not need access to the network - all the necessary information is stored directly on the picture, and playback begins instantly after entering the camera’s field of view. At the same time, as in Murzin’s ANS synthesizer, the user himself controls the speed and direction of the game by scanning the sound code manually (although there is also an automatic mode).
Does it make any practical sense? Imagine: sound cues in children's books or textbooks; a piece of a new song on a disc or promotional poster for the group; audio tags on goods; secret messages on the walls of buildings; sound cards and various kinds of art experiments. This would make sense if there were very simple way reading such images. After all, you need to photograph it, load it into the program and accurately indicate the boundaries of the spectrum, the base frequency and the number of octaves.
Instructions for use
- Install the PhonoPaper application on your iPhone or Android smartphone.
- Launch the application.
- Point at each soundtrack.
Instead of a conclusion
As we see, the next turn of the spiral takes us back to the origins. And this is natural, because the world today is oversaturated with information processing processes hidden from humans and is increasingly immersed in virtual space, digitized, encoded and packaged. Musical instruments They hide their nature, you cannot touch them or look under the lid to touch the magic of the birth of a new sound, to feel its energy. Drawing music at the “atomic” level and transferring this process into the real world is undoubtedly a big step towards reducing the distance between the composer and the embodiment of his creative ideas. At the same time, music creation becomes accessible to amateurs and representatives of related arts; we are no longer limited by rigid boundaries and rules, and musical notation is now just an addition. Take a pen and paper and start creating a new masterpiece.
In general, I recently decided to look at Wikipedia (a truly inexhaustible reservoir of knowledge), and there I came across the definition of a spectrogram. As it turned out, there is something to borrow on the topic of drawing with sound. Firstly, this is a list of programs that allow you to synthesize sound from pictures. The list includes the Coagula program, which is very well known to us (by the way, it is in our software subsection), but there are others, namely:
- MetaSynth for Macintosh;
- Coagula for Windows;
- FL Studio "s "BeepMap" additive synthesizer.
The open-source, multi-platform project showcases fun but highly educational experiences. The program allows you to turn sound into a spectral image (with the specified resolution), and vice versa, to synthesize sound from the image (with the specified parameters).
Another joker is the group Plaid. In the song "3recurring" it contains a logo in its spectrogram.
And Nine Inch Nails also use the technique of hiding images in the spectrum of tracks from the album "Year Zero".
In general, this method clearly appealed to some musicians. In principle, the same method can easily be used as a steganography tool.
This topic is very interesting and, I think, many more discoveries will be discovered related to drawing in the spectrum and sounding pictures.
A person is confronted with speech from birth. Initial acquaintance occurs with sounds. Speech sounds are what we make when we speak. We hear them when other people speak.
Acquaintance with letters begins later. We write letters and see them when we read written text.
Sound cannot be written or seen. But the letter cannot be pronounced. But each letter has its own name: “A”, “Be”, “Er”, “Sha”. And they are needed to indicate sounds in writing.
If we try to pronounce a sound that is indicated in writing by the sign “b”, then we will not succeed. At best, the name of the letter “Soft Sign” will sound. But the soft sign does not indicate any sound. In Russian it has a completely different role.
Why do you need a soft sign?
Despite the fact that this letter does not indicate a sound, it has several functions in the Russian language.
An indicator of the softness of a consonant sound. If in a written word a soft sign appears after the letter denoting a consonant, then this sound is pronounced softly when read. An example showing the difference in the pronunciation of sounds denoted by the same letter, with and without a soft sign, can be the words “dal” and “dal”.
Separating function. In writing, a soft sign separates the letter denoting a consonant sound and the vowels I, E, Yo, Yu, I. In this case, the consonant sound is read softly, and the indicated vowels indicate two sounds: I - [Y, A]; E - [Y, E]; Yo - [Y, O]; Yu - [Y, U]; I - [Y, I].
Designation of grammatical forms of words. At the end of feminine nouns singular(3 declensions) a soft sign is written.
It is also written in indefinite form verbs, incl. in front of TSYA. A soft sign is used in all verb forms after sibilants and in verbs in the imperative mood, as well as in verbs of the present and future tenses in the second person singular.
If the stem of an adverb ends in hissing, this letter is also written.
And although the letter “Soft Sign” itself does not denote any sound, it has big influence on the pronunciation of consonant sounds.
