The basis for the mathematical roots of non-linear editing begins with its roots in film.
At the end of day of shooting, rolls of film for images, and reels of magnetic tape for sound, were gathered for developing and processing. Once the film was developed, the audio was registered with the image frames, by transfer to a so called “mag track”. If the source film was 35mm, with four perforations per frame, the audio was transferred to a a 35mm mag track, recorded at the appropriate speed such that each four perforations on the mag track corresponded to 1/24 of a second of sound.
A clapper was recorded for each take recorded to film and tape; and a frame was manually marked on the film where the boards of the clapper first visually meet. The audio was then matched to the film so that the peak in the audio recording of the clapper matches the frame where the clapper's “sticks” met. The perforations in the film and the audio track provide the increments forward and back that the mag-track can be synchronized by. As a consequence, the visual correspondence of the sound to the picture, on 4-perf 24 frames per second material, is accurate to within 1/96 of a second.
The film frames had numbers embossed or otherwise imprinted on the edge of the film. These “edge codes” were the numeric indices of frames on a strip. When the synchronization of the film and mag-track was complete, the film strip’s edge codes were imprinted upon the mag-track. This affordance allowed editors to easily verify that sound and image were in sync during the editing process. These two reels were set aside and labeled according to an editor’s preference into a collection of raw material, called “takes”, to be used later when the film was assembled.
Every subsequent print of a take would also photographically transfer the edge codes to the new print, so the correspondence between originating materials was in principal never lost.
Mechanical film editing systems, such as the Movieola, allowed an editor to physically cut printed takes, and join them together in a new order with gum, into a “cut” of a film. When two takes are meant to overlap via an effect such as a cross dissolve, a long diagonal cut, known as a “straight cut”, was made in the two segments and then the two pieces of the film would be joined along the diagonal cut. The process proceeds, incorporating many different elements, such as visual effects. The elements were prepared, possibly independently, and slowly worked into the “final cut”. Throughout this process, the take names and edge codes ensured that the source materials could be found and utilized by the optical laboratory responsible for combining all of the materials into an “answer print” which hopefully represents a final form of the film. Once satisfactory, a “release print” for distribution to theaters was prepared from the “answer print.”
Throughout this meticulous and laborious process, the source materials are maintained in synchronization by virtue of a fixed sampling rate for all sources and products, and edge codes that dictate an exact correspondence between media. An "edit decision list" is a recipe to produce an answer print given a library of sources, edge code ranges, and effect types. Edit decision lists were originally written by hand on paper.
The introduction of video tape and the non-linear editing process was a revolution in this process. Tapes were prepared from source materials, with time codes in the video serving the same role as film’s edge codes. The edit decision list no longer directed a series of photographic composites to produce a new strip of film, but rather queued portions of video tape to play from possibly several machines to another recording machine. EditDroid introduced the next level of sophistication in 1984, replacing video tapes with source material on laserdisc. Instead of waiting for tape to rewind, or play forward in real time, sources could be randomly accessed, and represented in rough form through computer interfaces. Throughout the 1990s the sophistication of playback and recording systems improved until the editing process converged on what might be considered an emulation of the earlier film editing process.
This organic but meticulous evolution gave rise to the form of the modern NLE, which at its very heart, represents a bin of film strips, layered in rows that represent the order of an optical composite, indexed by fixed sample rate frame codes, and registered and synchronized to those frame codes by modern day sprocket holes.
If this was the end of the story, our work here would be done. The classic format of an EDL would be sufficient to describe the composition of a film from source materials in a reliable way, irrespective of the non-linear editing system in use. As you might expect, life is in fact not so easy. The relentless primacy of 24 frames per second and 96 perforations per second is an uneasy fit over modern technology. Film can be shot at speeds over 100 frames per second; video at rates that are a multiple of 30 or 25. North American broadcast video sources may be at NTSC rates, which are rational rates with a divisor of 1001; the well known rate of 29.97 is 30 * 1000 / 1001. Audio might be record at 44.1kHz, 48kHz or other rates, and the modern NLE allows the elements to be composed in time with an arbitrary degree of precision.
The next step in this evolution is to shed the emulation and concomitant limitations of historic analog materials, and to push forward into a modern realm based on sampling and reconstruction theory. This will make frame rate and other technical aspects of film making creative choices, and will also eliminate the quality losses incurred by frame rate conform steps, and other intermediate processes once required to fully emulate a classical film and magtape based workflow.