This was the first project tackled by the newly formed team - an experimental technique to restore colour to broadcast quality monochrome 16mm film recordings by utilising the colour signal from domestic colour recordings made in American in the mid-seventies. Steve Roberts takes up the story, from his articles published in 'Doctor Who Magazine' in 1992 and 1993.
[First Article, 1992]
This month sees the BBC Home Video release of `The
Dæmons', barely a fortnight after finishing its run on BBC2. Although it may appear to be
rather a waste of time releasing the story so soon, Home Video are clearly hoping that
sales will be higher because of its freshness in peoples minds. This was clearly the case
three years ago, when excellent sales of the `Blackadder' tapes were achieved even as the
stories were being rebroadcast.
This article is concerned with the techniques used
to restore this classic third-Doctor story (reportedly Pertwee's personal favourite), and
will also try to explain why a full restoration to its original state would have been
impossible.
At the beginning of this year, four out of five
episodes of `The Dæmons' existed in two basic forms - as 16mm black & white broadcast
quality film recordings, and as non-broadcast quality colour tapes, recorded on U-matic
video cassettes in the U.S. 525-line NTSC format. Only episode four existed as a UK
standard 625-line PAL colour transmission tape.
The black & white film recordings had been made
in the early seventies by BBC Enterprises, for sale to foreign TV stations that had yet to
make the move to colour transmission. 16mm film recordings had distinct advantages over
videotape:- they were cheap to produce, durable, and could be used irrespective of the
transmission standard used in any particular country. They were also very high quality -
forget what you might have read about them being made by pointing a film camera at a TV
monitor! The BBC's film recorders were extremely sophisticated machines, easily capable of
recording the full detail of a television signal. Luckily, BBC Enterprises had retained
the original film recording negatives, which eventually made their way to the BBC Archive
when it was discovered that the colour videotapes had been destroyed.
The American U-matic tapes only existed due to the
efforts of one British `Doctor Who' fan to enlarge his own video collection. Back in 1978,
Ian Levine heard that KCET TV in Los Angeles was about to show `The Dæmons' as a two-hour
compilation. He wired an American friend the money to go out and hire one of the then
brand-new Betamax VCR's, and to buy two one-hour tapes, at that time the longest tape
available. The machine was obtained, and the broadcast recorded in its entirety, except
for a gap of about twenty seconds during which the tapes were changed over. The Betamax
tapes were brought over to England and, as TV standards conversion equipment was not
generally available to the public, the tapes were copied onto 525-line U-matic cassettes,
retained by Levine ever since.
At this point, Levine suspected that the BBC had
wiped most of their colour tapes of the story, but assumed that Time-Life, the American
distributors, still retained theirs. He was horrified when he later learnt that Time-Life
had also junked their masters. He visited KCET TV in the hope that they still had their
tapes, but unfortunately they had been destroyed just weeks previously. This meant that
the Betamax and U-matic cassettes were the only colour copies known to exist.
For many years there has been speculation in fan
circles as to whether it was possible to produce a transmittable colour version of `The
Dæmons' by overlaying the colour from the U-matic tapes over the high resolution
monochrome picture on the film recording. The simple answer is Yes, it's been possible for
years. BBC employee Simon Anthony and fellowfan KeithHunter experimented with
restoring colour to 'Terror of the Autons' in 1986, but they were unable to drum up
sufficient interest to generate funding for the project. However, it is only fairly
recently that broadcast equipment has developed to the stage where it could produce high
quality results easily, and more importantly, cheaply. Also, the renewed public interest
in both the repeat broadcasts, and the Home Videos made 1992 the ideal time to approach
`the powers that be' with the idea of a properly funded restoration attempt.
This approach was made by BBC Graphic Designer Ralph
Montagu, and broadcast equipment designer James Russell, both long time fans of the
series. They took their idea to John Whiston, the `Late Show' producer responsible for
`Resistance is Useless' and the repeat seasons, and he agree to fund the test restoration
of episode one. Within a week of permission being given, Montagu and Russell presented
Whiston with a result - not perfect, but more than good enough to demonstrate the
potential of their technique. The money to allow a serious restoration of the entire story
was quickly forthcoming, this time co-funded by `The Late Show' and the BBC Archive.
My own involvement in the restoration began at this
point. Ralph Montagu called in to see me at Television Centre, as he had been told by Adam
Lee that I was engaged in experiments towards the same ends, although at a much less
advanced stage. After showing me the very impressive test results, we began discussing the
improvements that could be made, and it soon became clear that the project would benefit
from a pooling of our resources.
To understand the techniques used in the
restoration, it is important to understand a little about the television system. A colour
TV picture is really a trick:- it consists of a highly detailed black & white image,
known as the luminance image, over which is laid a much less detailed colour image.
Because the brain responds much more strongly to luminance changes than to colour changes,
the detailed luminance image fools you into believing that you are looking at a highly
detailed colour picture.
TV signals have traditionally been recorded,
processed and edited as composite signals, in which the luminance and colour signals are
combined into a single electronic signal. However, the modern trend in video
post-production is towards a component way of working, in which the luminance and colour
signals are kept separate, and can thus be processed separately as well. It was the
existence of GE3, a component Graphics Edit suite at the BBC's White City site, that was
to simplify what would otherwise have been a nearly impossible job. It will come as no
surprise to long suffering `Doctor Who' fans that, in true BBC fashion, this extremely
useful facility is soon to be closed down!
If it was just a matter of taking the luminance
signal from a videotape of the film recording and adding to it a colour signal from the
U-matic tape to form a complete colour picture, we would have probably managed the job in
a couple of hours. In the event, it took nearer fifty....
The first, and probably most important, problem was
that frame-for-frame the images from the two sources were both a different size, and a
different shape. The result of combining the two images would be a picture in which the
colour did not appear in the correct place. The image on the film recording was distorted,
due to both inherent non-linearities of the process, and misalignments of the film
recording machinery. Viewed in isolation, no problem is apparent, but when compared with
the U-matic sourced image, it becomes clear that the black & white picture contains
subtle twists, tilts and stretches over the entire image. The edges of the picture are
also cropped by about 3%, making the image appear to be slightly zoomed in when compared
with the U-matic. To compound the problem, each episode's film recordings had been made on
different days, and on different machines, meaning that each was uniquely distorted.
The solution to this problem was to make use of one
of the clever little units that `Top of the Pops' use to twist the picture and fly it
around the screen. GE3 came fully equipped with a Questech Charisma video effects
processor, which was to prove almost ideal for the job. It would have been nice if we
could have used Charisma to un-distort the film recording, but unfortunately, due to the
image cropping mentioned earlier, this would have resulted in a finished colour picture
with a black border around its edges. It was therefore necessary to distort the colour
signal from the U-matic source to match the distortions on the film recording. This
alignment was carried out under James Russell's critical eye, and would take anywhere
between one and two hours per episode. This was the most crucial alignment, as any error
would result in a visible coloured fringe around objects.
Once the Charisma was aligned, we could actually
begin work. Two component video recorders played together in sync into the system - one
machine contained a tape of the film recording, recorded in an earlier session in
telecine, and the other contained a standards converted copy of Ian Levine's colour
U-matic tape. The combined colour signal was recorded on a third component recorder.
Recording would continue until one player went out
of sync with the other. This was due to edits which had been made to the story as
transmitted in the States. Usually, these were very short trims to the end of scenes, and
we never really found out the reason for them. It is unlikely that they were trims for
timing purposes as they were so short - the favourite theory is that they were to remove
duff edits on the original BBC tapes. Whatever the reason, they were a real pain, as
whenever one occurred, the colour would go wildly out of sync and we were forced to stop
and consider our options.
It was decided right from the beginning that as we
were undertaking a restoration, we would only drop programme content as a last resort.
Luckily, as many of the trims were of short duration, and at the beginning or end of
scenes with very little movement in them, one option we had was to freeze the nearest
available frame of colour whilst the black & white continued running. This technique
saved our proverbial bacon quite a few times - an example of this can be seen in episode
three (16'01"), the final shot of Bok.
There were some occasions where this was impossible
however - either there was too much movement in the frame, or the colour was entirely
missing from a scene. This situation commonly occurred around episode junctions due to the
way the compilation had been edited. However, we still had two powerful tools at our
disposal.
The first was to use the vision mixer to provide colour
washes,and false-colour highlights. We used this technique in episode five (08'57"),
in a scene where Azal is clutching his head and screaming. Luckily, the action takes place
in a fairly dark cave, which made the job a little easier. The vision mixer was used to
provide a dark brown/yellow wash over the entire picture, then medium brightness areas of
Azal were picked out in red. As a finishing touch, the high brightness parts of the
picture (his horns) were given a slight blue tint. Similarly, the final shot of Bok in
episode two (22'59") is simply the black & white image with a slight orange wash
over it.
Our other major tool was the ubiquitous Quantel
Paintbox. As a last resort, black & white sequences could be pulled into Paintbox,
individually hand painted frame-by-frame, then sent back out to the recorder. As you can
imagine, this was a very time consuming job! To be honest, we didn't use Paintbox very
much during the restoration of `The Dæmons' - it was mostly used to replace missing
sections of colour in individual frames, caused by oxide dropout on the U-matic tapes. It
was used extensively during the restoration of `Doctor Who and the Silurians' a few weeks
later however, particularly around the junction of episodes three and four, some scenes of
which were missing entirely from the colour tapes.
When we began the restoration of `The Dæmons', we
decided to leave episode three until last. It was this episode which had twenty seconds
missing from it due to the Betamax tapes being changed over halfway through the recording,
and we assumed that this would be a massive headache for us. In the event, it caused very
few problems! It transpired that Ian Levine's friend in the States was not the only person
recording the 1978 transmission after all - another recording had been made on VHS, and
contained the missing sequence (14'40"). A copy was kindly loaned to us by Keith
Hunter, and although the quality of the recording was lower, it was good enough to bridge
the gap.
There was one other major loss of colour near the end of episode three (22'04"). The colour tape off-locked, causing a wide band of missing colour to run up and down the screen over a period of about two seconds. Thinking that we would be able to use Keith Hunter's VHS, we looked at that, only to find that his tape was even worse, losing colour completely. Both tapes had obviously been recording the same transmission, which must have suffered from a technical fault at this point.
We went back to our original colour tape and played
the damaged section into a solid-state video recorder - like a videotape recorder, but
using computer memory instead of tape. One of the main advantages offered by an SSVR is
the ability to re-record pictures many times without any visible degradation. As the scene
was fairly static we were able to lift an undamaged strip of colour from four frames
earlier, and use it to replace our missing strip. This was done for each of over forty
frames, and took the best part of two hours to complete. The result was thought to be well
worth the effort. If you examine the scene closely, you can just about make out a
disturbance in Roger Delgado's cloak - I bet you didn't see it first time around though!
Some shots from the end of episode three and the
beginning of episode five appear in episode four, and as this episode existed in perfect
quality we took the required shots from there.
New beginning and end title sequences were required for the story, as the compilation obviously only had episode one beginning and episode five end titles. Luckily, the BBC Archive still retains the original clean background films. Both sequences underwent colour correction during the telecine transfer, as it was felt that we should attempt to produce colours consistent with those on the surviving transmission tape for part four. However, in the opening titles of episode four, Pertwee's face looks distinctly golden, and so we took the opportunity to dynamically colour correct his face to produce a more natural flesh tone - a distinct improvement! In fact, the title sequences are a great improvement on the originals in general. Telecine technology has advanced somewhat in the last twenty years, and I don't think you'll ever have seen the Pertwee titles looking so sharp!
To produce the captions, we simply used the ones on
the film recording, electronically coloured them yellow, and keyed them into the clean
backgrounds. The closing credits for episode five are slightly different to the other
episodes, being a straight combination of the film recording and colour signal. As the two
stayed in sync after the last scene of the episode, it seemed pointless to go to all the
trouble of keying credits into clean backgrounds as we had done for the other episodes.
A different technique was used for `Doctor Who and the Silurians' when we came to restore that story a couple of months later. We tried to key the captions in from the film recording, but were troubled by fuzzy edges on the letters. Instead, Ralph Montagu produced credits on paper from his Apple Mac, using the same font as the originals. Each caption was then placed under a rostrum camera, grabbed into Paintbox, and resized to match the original. A credit roll tape was then made up from these images, with each credit being on screen for exactly the same amount of time (to the frame!) as in the original sequence. This tape was then used as a key over the clean backgrounds.
You might ask why we resorted to such a long winded
method, when we could have made up sequences on a modern electronic character generator.
Well, the answer is that the character generator looked too good! We felt it had lost some
of the `Letraset caption card in front of camera' look of the originals. Ralph's method
produced a more `traditional' look to the credits.
As far as programme sound is concerned, the great majority of it was taken from the film recording separate magnetic soundtrack film, as it was of slightly higher quality than on the U-matic tapes. The exception to this is part of the Doctor's slideshow in episode three, during which a portion of Pertwee's speech was missing from the magnetic soundtrack film. Fortunately, it was intact on the videotape, and so the sound was taken from that.
Beginning and end title music was lifted from the
transmission tape for episode four and mixed in and out of the programme sound as
required.
Once the four episodes had been restored to colour,
the component video master cassettes, and the 2" tape of episode four, were taken
over to Television Centre to be copied onto transmission tapes. This process involved the
addition of VT countdown clocks at the beginning of each episode, a certain amount of
colour correction to make the pictures look more natural, and finally, transfer to D-3
composite digital transmission tapes.
There are a number of faults that can still be seen
in the final version of the restored stories which, unfortunately, proved either
impossible or unfeasibly difficult to remove.
The first of these is a problem known as `picture
phasing', which can be seen on many film recordings, including most `Doctor Who's. It
manifests itself as double-imaging on programme sequences which originally came off film
(the first five minutes of episode one of `The Dæmons', for example), causing motion
judder, rather like a poor quality standards conversion. The reasons for the problem are
rather complex, and beyond the scope of this article. Suffice it to say that whenever a
telecined film sequence was recorded by a film recorder, there was a 50% chance that it
would suffer from picture phasing problems. As a telecine engineer, I find it particularly
frustrating that twenty years ago the addition of a simple circuit to the telecine system
would have completely eradicated this problem. Once recorded, however, there is absolutely
no way the error can be removed. A variation of the picture phasing problem, this time due
to the design of the film recorder, is responsible for some of the vision cuts on the
studio material appearing as single frame mixes on the film recording.
Episode five suffers from some vertical instability
of the picture, due to misalignment of the film recorders claw box, the device which pulls
the film down frame by frame prior to exposure. Again, this fault is recorded onto the
film and is impossible to remove.
Another difficult problem is dirt and scratches on
the film recording. Although brand-new prints for all of the restoration work were struck
from the original negatives via a wet-gate printing process, they still exhibit dirt
problems. Most apparent is white `sparkle' caused by damage on the negatives, and which
therefore always appears in each new print. There is also an enormous amount of sparkle on
original film sequences which was there when it was first transmitted, and would certainly
not be acceptable in a modern-day programme. The BBC does possess a device for
electronically removing sparkle from images, but it has to be used very sparingly as it
tends to produce nasty side-effects on areas of fast movement. We used it only once during
`The Dæmons', during the final seconds before the credit roll in episode one. If your VHS
possesses the ability to step through recordings frame-by-frame, try looking closely at
this sequence, and you will be able to see some of the side-effects produced by the
device.
After finishing work on `The Dæmons', the same team
successfully applied for funding to enable similar restoration to be applied to three
other stories, namely `Terror of the Autons', `Doctor Who and the Silurians', and `The
Ambassadors of Death'. This time, the funding was provided from three sources - BBC
Archives, Home Video, and Enterprises Programme Sales. The first two of these stories have
been completely restored, with `Terror of the Autons' receiving a great deal of praise for
its outstanding picture quality. The third poses more of a problem, as the colour copy
exhibits terrible patterning. One episode may be recoverable, but unfortunately the others
may not.
However, if any reader (particularly in America) has a long forgotten off-air copy of `The Ambassadors of Death', `The Mind of Evil', `Planet of the Daleks':3 or `Invasion of the Dinosaurs':1 lying around gathering dust, let us know!
Hopefully, whilst we are waiting for one to turn up,
we'll be able to look forward to the Home Video release of at least two more restorations
in the near future.
[Second article, 1993]
After completing work on the restoration of `The Dæmons',
project supervisor Ralph Montagu applied for further funding to enable an attempt to be
made to restore three more Pertwee stories. The application was successful, with funding
this time coming from three sources - the Film & Videotape Library, Home Video, and
BBC Enterprises Programme Sales.
It was hoped that `Terror of the Autons', `Doctor Who and the Silurians' and `The Ambassadors of Death' would all be able to be restored using the same techniques developed for `The Dæmons'.
To recap briefly, the restoration process is based
on combining the colour signal from an American off-air Betamax videotape, with the BBC's
high resolution black & white film recordings.
The first stage of the work was to have the 16mm
film recordings transferred to videotape. The team had experienced problems with
scratches, dirt, and sparkle on the film prints of `The Dæmons', and so the BBC's Archive
Selector, Adam Lee, agreed to have brand-new prints struck especially for the project. A
process called `wet gate printing' was used, in which the original negative film is
immersed in a liquid which fills in the scratches on the film, rendering them invisible on
the copy print.
The film recordings were transferred to tape in the
BBC's Telecine department, mostly by Dave Hawley, who has worked on all the ongoing Doctor
Who projects since `Resistance is Useless' at the beginning of last year. Initially,
`Terror of the Autons' and `Doctor Who and the Silurians' were transferred, with `The
Ambassadors of Death' being done a few weeks later, by Duncan Bragg. Wherever possible,
the restoration team try to use fans of the series to do this sort of work, as their
empathy with the material helps to produce the best results. At the same time the telecine
work was being done, the American off-air recordings were being converted to our PAL video
standard, and copied onto a new set of tapes.
After all the problems that the team had experienced
during work on `The Dæmons', it was decided to start with the easiest story first - in
this case `Terror of the Autons'. We were fortunate to have very good colour copies, and
excellent film copies of the story, and so hopes were high that the results of the
restoration would be very good indeed. It was a bonus to us that, unlike the other
stories, the American tapes were in complete episodic form, rather than as a compilation.
Restoration of `Terror of the Autons' took place in
a single day - about a quarter of the time that it had taken to restore `The Dæmons'. At
the same time that the Sunday morning audience at the DWAS' PanoptiCon XII convention were
enjoying a specially produced BBC trailer for `The Dæmons', the rest of the team were
hard at work back at White City, setting up the critical alignment between the colour and
black & white images. Once this was done, the two images stayed locked together
practically all the way through each episode, the only breaks in recording being made to
sort out minor picture faults. As the colour tapes were of complete episodes, there was no
need to produce new end credit rolls, as had to be done for the other stories, and so this
was a major saving of time. However, the beginnings of the episodes were not particularly
clean, so new credits were made up on paper, loaded into Paintbox, and keyed in over a
clean background film supplied by the BBC Library. As with `The Dæmons', an attempt was
made to make Pertwee's face appear to have a natural flesh tone - however the colours of
the rest of the sequence are much brighter and more unusual than those used for that
story.
There was one major picture fault which was to cause the team a few problems. It occurred during the Doctor's dissection of the plastic doll. The black & white film recording had a major fault, lasting about half a second, in which the picture completely broke up due to the film recorder losing lock when the film was made back in the early Seventies. The first attempt to overcome this problem was to replace that particular part of the scene with `raw' picture from the American tapes. Unfortunately, the effect of this was that the picture suddenly appeared to go out of focus for half a second, which was almost as disconcerting as the original fault!
The team had heard of Gazelle, an experimental device built by the BBC Research Department, which was designed to make slow-motion pictures appear smoother by `inventing' new frames of video between the real frames. It was reasoned that if we fed the machine with video from before and after the disturbance, it might be able to make up the missing eleven frames. Unfortunately, it was all too much for the poor machine, the result being a mix-through from the first to the last frame, with only the movement of Benton's elbow being guessed at! However, Gazelle is still under development, and it may be possible to use it in the future. The team have talked about the possibility of using it to clean-up part three of `The Faceless Ones', which has numerous picture jumps all the way through it.
The final solution was to use the raw colour
pictures, but put through a device called an `aperture corrector', which gives the
appearance of high resolution by artificially enhancing the edges of any object in the
picture. The viewer is still aware of something happening, but the amount of distraction
caused is much less than it was.
As predicted, the quality of the finished
restoration was nothing short of excellent, much better than `The Dæmons', which was now
looking slightly poor by comparison!
Work began the following week on `Doctor Who and the Silurians', the longest story which the team had attempted to restore, with seven full episodes to be done.
As with `The Dæmons', the colour tapes were a
recording of a compilation transmission - this meant that there were no beginning and end
titles, and there were bits of scenes missing, particularly around the episode overlap
points. Luckily, the team's experience on `The Dæmons' had led them to develop a number
of ways to deal with missing colour footage, and all of them (and more!) would be employed
over the following few weeks.
Some of the worst problems occurred during the scene
in which the Doctor meets the Silurian in the cottage. Some scenes were entirely missing
from the colour copy, and a great deal of time was spent trying not to have to lose any
footage because of this. Some sequences had to be hand painted frame by frame on Paintbox,
an exceptionally tedious and time-consuming process.
There is also some interference recorded on the
colour tape around the junction of episodes five and six, which causes rainbows of colour
to be superimposed on the picture. At the time of writing, James Russell is developing a
hardware/software package for the Apple Macintosh computer, which it is hoped will be able
to reduce, if not remove, this patterning. Fortunately, it is fairly mild, and only lasts
for about five minutes.
One thing which the team had to constantly be wary
of was the temptation to not just restore, but to also improve on the original. For
instance, during the scene in which the Silurian is about to attack Liz Shaw in the barn,
there are annoying white `edit-flashes' at each picture cut. This was due to the way the
original film sequences had been spliced together, and it would have been very easy to
take out a frame at the beginning of each new shot to remove them. However, this would not
be in the spirit of restoration, and the viewer would not be seeing a fair representation
of the story as it originally was.
Unfortunately, work on the restoration of `The Ambassadors of
Death' has been halted after only two episodes (five and six) had been completed. Much of
the colour tape exhibits the same rainbow patterning as was seen on episode five of
`Doctor Who and the Silurians', but to a much greater degree. Unless a technique can be
developed to remove this patterning, it would appear that the only way to proceed with
this restoration would be to employ a `colourisation computer', as used to colour Laurel
& Hardy films, for instance.
A sample of `The Ambassadors of Death' has been colourised for us using this technique, by `American Film Technologie', the largest company of its kind in the States. The results are very impressive, but at a cost of over $2000 per minute of finished programme, it will probably prove financially unviable to restore four episodes of one story using this method.
However, $50,000 may not be too much to pay if
colourisation of one episode effectively releases a whole story, which would be the case
for `Planet of the Daleks' and `Invasion of the Dinosaurs'. Of course, this will only
happen if Home Video perceive a strong enough market for the tapes, so if you want them,
speak up!
[After this article was
written we actually used American Film Technologie's process to fix a ninety second colour
fault at the junction of episodes five and six of 'Doctor Who and the Silurians'. The
scene starts with the Doctor and the Brigadier arriving at the Cottage Hospital in Bessie
and ends with the Doctor and Liz in the laboratory. All the colour in this section was
produced artificially by computer.]
As noted above, there are still several episodes that could be restored to colour, if only we had the material to work from - Can you help us?
Copyright Steve Roberts 1992,1996