Pertwee Colour Restorations

[BBC Video Covers for colour restorations]

This was the first project tackled by the newly formed team - an experimental technique to restore colour to broadcast quality monochrome 16mm film recordings by utilising the colour signal from domestic colour recordings made in American in the mid-seventies. Steve Roberts takes up the story, from his articles published in 'Doctor Who Magazine' in 1992 and 1993.




[First Article, 1992]

This month sees the BBC Home Video release of `The Dæmons', barely a fortnight after finishing its run on BBC2. Although it may appear to be rather a waste of time releasing the story so soon, Home Video are clearly hoping that sales will be higher because of its freshness in peoples minds. This was clearly the case three years ago, when excellent sales of the `Blackadder' tapes were achieved even as the stories were being rebroadcast.

This article is concerned with the techniques used to restore this classic third-Doctor story (reportedly Pertwee's personal favourite), and will also try to explain why a full restoration to its original state would have been impossible.

At the beginning of this year, four out of five episodes of `The Dæmons' existed in two basic forms - as 16mm black & white broadcast quality film recordings, and as non-broadcast quality colour tapes, recorded on U-matic video cassettes in the U.S. 525-line NTSC format. Only episode four existed as a UK standard 625-line PAL colour transmission tape.

The black & white film recordings had been made in the early seventies by BBC Enterprises, for sale to foreign TV stations that had yet to make the move to colour transmission. 16mm film recordings had distinct advantages over videotape:- they were cheap to produce, durable, and could be used irrespective of the transmission standard used in any particular country. They were also very high quality - forget what you might have read about them being made by pointing a film camera at a TV monitor! The BBC's film recorders were extremely sophisticated machines, easily capable of recording the full detail of a television signal. Luckily, BBC Enterprises had retained the original film recording negatives, which eventually made their way to the BBC Archive when it was discovered that the colour videotapes had been destroyed.

The American U-matic tapes only existed due to the efforts of one British `Doctor Who' fan to enlarge his own video collection. Back in 1978, Ian Levine heard that KCET TV in Los Angeles was about to show `The Dæmons' as a two-hour compilation. He wired an American friend the money to go out and hire one of the then brand-new Betamax VCR's, and to buy two one-hour tapes, at that time the longest tape available. The machine was obtained, and the broadcast recorded in its entirety, except for a gap of about twenty seconds during which the tapes were changed over. The Betamax tapes were brought over to England and, as TV standards conversion equipment was not generally available to the public, the tapes were copied onto 525-line U-matic cassettes, retained by Levine ever since.

At this point, Levine suspected that the BBC had wiped most of their colour tapes of the story, but assumed that Time-Life, the American distributors, still retained theirs. He was horrified when he later learnt that Time-Life had also junked their masters. He visited KCET TV in the hope that they still had their tapes, but unfortunately they had been destroyed just weeks previously. This meant that the Betamax and U-matic cassettes were the only colour copies known to exist.

For many years there has been speculation in fan circles as to whether it was possible to produce a transmittable colour version of `The Dæmons' by overlaying the colour from the U-matic tapes over the high resolution monochrome picture on the film recording. The simple answer is Yes, it's been possible for years. BBC employee Simon Anthony and fellowfan KeithHunter experimented with
restoring colour to 'Terror of the Autons' in 1986, but they were unable to drum up sufficient interest to generate funding for the project. However, it is only fairly recently that broadcast equipment has developed to the stage where it could produce high quality results easily, and more importantly, cheaply. Also, the renewed public interest in both the repeat broadcasts, and the Home Videos made 1992 the ideal time to approach `the powers that be' with the idea of a properly funded restoration attempt.

This approach was made by BBC Graphic Designer Ralph Montagu, and broadcast equipment designer James Russell, both long time fans of the series. They took their idea to John Whiston, the `Late Show' producer responsible for `Resistance is Useless' and the repeat seasons, and he agree to fund the test restoration of episode one. Within a week of permission being given, Montagu and Russell presented Whiston with a result - not perfect, but more than good enough to demonstrate the potential of their technique. The money to allow a serious restoration of the entire story was quickly forthcoming, this time co-funded by `The Late Show' and the BBC Archive.

My own involvement in the restoration began at this point. Ralph Montagu called in to see me at Television Centre, as he had been told by Adam Lee that I was engaged in experiments towards the same ends, although at a much less advanced stage. After showing me the very impressive test results, we began discussing the improvements that could be made, and it soon became clear that the project would benefit from a pooling of our resources.

To understand the techniques used in the restoration, it is important to understand a little about the television system. A colour TV picture is really a trick:- it consists of a highly detailed black & white image, known as the luminance image, over which is laid a much less detailed colour image. Because the brain responds much more strongly to luminance changes than to colour changes, the detailed luminance image fools you into believing that you are looking at a highly detailed colour picture.

TV signals have traditionally been recorded, processed and edited as composite signals, in which the luminance and colour signals are combined into a single electronic signal. However, the modern trend in video post-production is towards a component way of working, in which the luminance and colour signals are kept separate, and can thus be processed separately as well. It was the existence of GE3, a component Graphics Edit suite at the BBC's White City site, that was to simplify what would otherwise have been a nearly impossible job. It will come as no surprise to long suffering `Doctor Who' fans that, in true BBC fashion, this extremely useful facility is soon to be closed down!

If it was just a matter of taking the luminance signal from a videotape of the film recording and adding to it a colour signal from the U-matic tape to form a complete colour picture, we would have probably managed the job in a couple of hours. In the event, it took nearer fifty....

The first, and probably most important, problem was that frame-for-frame the images from the two sources were both a different size, and a different shape. The result of combining the two images would be a picture in which the colour did not appear in the correct place. The image on the film recording was distorted, due to both inherent non-linearities of the process, and misalignments of the film recording machinery. Viewed in isolation, no problem is apparent, but when compared with the U-matic sourced image, it becomes clear that the black & white picture contains subtle twists, tilts and stretches over the entire image. The edges of the picture are also cropped by about 3%, making the image appear to be slightly zoomed in when compared with the U-matic. To compound the problem, each episode's film recordings had been made on different days, and on different machines, meaning that each was uniquely distorted.

The solution to this problem was to make use of one of the clever little units that `Top of the Pops' use to twist the picture and fly it around the screen. GE3 came fully equipped with a Questech Charisma video effects processor, which was to prove almost ideal for the job. It would have been nice if we could have used Charisma to un-distort the film recording, but unfortunately, due to the image cropping mentioned earlier, this would have resulted in a finished colour picture with a black border around its edges. It was therefore necessary to distort the colour signal from the U-matic source to match the distortions on the film recording. This alignment was carried out under James Russell's critical eye, and would take anywhere between one and two hours per episode. This was the most crucial alignment, as any error would result in a visible coloured fringe around objects.

Once the Charisma was aligned, we could actually begin work. Two component video recorders played together in sync into the system - one machine contained a tape of the film recording, recorded in an earlier session in telecine, and the other contained a standards converted copy of Ian Levine's colour U-matic tape. The combined colour signal was recorded on a third component recorder.

Recording would continue until one player went out of sync with the other. This was due to edits which had been made to the story as transmitted in the States. Usually, these were very short trims to the end of scenes, and we never really found out the reason for them. It is unlikely that they were trims for timing purposes as they were so short - the favourite theory is that they were to remove duff edits on the original BBC tapes. Whatever the reason, they were a real pain, as whenever one occurred, the colour would go wildly out of sync and we were forced to stop and consider our options.

It was decided right from the beginning that as we were undertaking a restoration, we would only drop programme content as a last resort. Luckily, as many of the trims were of short duration, and at the beginning or end of scenes with very little movement in them, one option we had was to freeze the nearest available frame of colour whilst the black & white continued running. This technique saved our proverbial bacon quite a few times - an example of this can be seen in episode three (16'01"), the final shot of Bok.

There were some occasions where this was impossible however - either there was too much movement in the frame, or the colour was entirely missing from a scene. This situation commonly occurred around episode junctions due to the way the compilation had been edited. However, we still had two powerful tools at our disposal.

[Azal - false coloured using vision mixer]The first was to use the vision mixer to provide colour washes,and false-colour highlights. We used this technique in episode five (08'57"), in a scene where Azal is clutching his head and screaming. Luckily, the action takes place in a fairly dark cave, which made the job a little easier. The vision mixer was used to provide a dark brown/yellow wash over the entire picture, then medium brightness areas of Azal were picked out in red. As a finishing touch, the high brightness parts of the picture (his horns) were given a slight blue tint. Similarly, the final shot of Bok in episode two (22'59") is simply the black & white image with a slight orange wash over it.

Our other major tool was the ubiquitous Quantel Paintbox. As a last resort, black & white sequences could be pulled into Paintbox, individually hand painted frame-by-frame, then sent back out to the recorder. As you can imagine, this was a very time consuming job! To be honest, we didn't use Paintbox very much during the restoration of `The Dæmons' - it was mostly used to replace missing sections of colour in individual frames, caused by oxide dropout on the U-matic tapes. It was used extensively during the restoration of `Doctor Who and the Silurians' a few weeks later however, particularly around the junction of episodes three and four, some scenes of which were missing entirely from the colour tapes.

When we began the restoration of `The Dæmons', we decided to leave episode three until last. It was this episode which had twenty seconds missing from it due to the Betamax tapes being changed over halfway through the recording, and we assumed that this would be a massive headache for us. In the event, it caused very few problems! It transpired that Ian Levine's friend in the States was not the only person recording the 1978 transmission after all - another recording had been made on VHS, and contained the missing sequence (14'40"). A copy was kindly loaned to us by Keith Hunter, and although the quality of the recording was lower, it was good enough to bridge the gap.

There was one other major loss of colour near the end of episode three (22'04"). The colour tape off-locked, causing a wide band of missing colour to run up and down the screen over a period of about two seconds. Thinking that we would be able to use Keith Hunter's VHS, we looked at that, only to find that his tape was even worse, losing colour completely. Both tapes had obviously been recording the same transmission, which must have suffered from a technical fault at this point.

We went back to our original colour tape and played the damaged section into a solid-state video recorder - like a videotape recorder, but using computer memory instead of tape. One of the main advantages offered by an SSVR is the ability to re-record pictures many times without any visible degradation. As the scene was fairly static we were able to lift an undamaged strip of colour from four frames earlier, and use it to replace our missing strip. This was done for each of over forty frames, and took the best part of two hours to complete. The result was thought to be well worth the effort. If you examine the scene closely, you can just about make out a disturbance in Roger Delgado's cloak - I bet you didn't see it first time around though!

Some shots from the end of episode three and the beginning of episode five appear in episode four, and as this episode existed in perfect quality we took the required shots from there.

New beginning and end title sequences were required for the story, as the compilation obviously only had episode one beginning and episode five end titles. Luckily, the BBC Archive still retains the original clean background films. Both sequences underwent colour correction during the telecine transfer, as it was felt that we should attempt to produce colours consistent with those on the surviving transmission tape for part four. However, in the opening titles of episode four, Pertwee's face looks distinctly golden, and so we took the opportunity to dynamically colour correct his face to produce a more natural flesh tone - a distinct improvement! In fact, the title sequences are a great improvement on the originals in general. Telecine technology has advanced somewhat in the last twenty years, and I don't think you'll ever have seen the Pertwee titles looking so sharp!

To produce the captions, we simply used the ones on the film recording, electronically coloured them yellow, and keyed them into the clean backgrounds. The closing credits for episode five are slightly different to the other episodes, being a straight combination of the film recording and colour signal. As the two stayed in sync after the last scene of the episode, it seemed pointless to go to all the trouble of keying credits into clean backgrounds as we had done for the other episodes.

A different technique was used for `Doctor Who and the Silurians' when we came to restore that story a couple of months later. We tried to key the captions in from the film recording, but were troubled by fuzzy edges on the letters. Instead, Ralph Montagu produced credits on paper from his Apple Mac, using the same font as the originals. Each caption was then placed under a rostrum camera, grabbed into Paintbox, and resized to match the original. A credit roll tape was then made up from these images, with each credit being on screen for exactly the same amount of time (to the frame!) as in the original sequence. This tape was then used as a key over the clean backgrounds.

You might ask why we resorted to such a long winded method, when we could have made up sequences on a modern electronic character generator. Well, the answer is that the character generator looked too good! We felt it had lost some of the `Letraset caption card in front of camera' look of the originals. Ralph's method produced a more `traditional' look to the credits.

As far as programme sound is concerned, the great majority of it was taken from the film recording separate magnetic soundtrack film, as it was of slightly higher quality than on the U-matic tapes. The exception to this is part of the Doctor's slideshow in episode three, during which a portion of Pertwee's speech was missing from the magnetic soundtrack film. Fortunately, it was intact on the videotape, and so the sound was taken from that.

Beginning and end title music was lifted from the transmission tape for episode four and mixed in and out of the programme sound as required.

Once the four episodes had been restored to colour, the component video master cassettes, and the 2" tape of episode four, were taken over to Television Centre to be copied onto transmission tapes. This process involved the addition of VT countdown clocks at the beginning of each episode, a certain amount of colour correction to make the pictures look more natural, and finally, transfer to D-3 composite digital transmission tapes.

There are a number of faults that can still be seen in the final version of the restored stories which, unfortunately, proved either impossible or unfeasibly difficult to remove.

The first of these is a problem known as `picture phasing', which can be seen on many film recordings, including most `Doctor Who's. It manifests itself as double-imaging on programme sequences which originally came off film (the first five minutes of episode one of `The Dæmons', for example), causing motion judder, rather like a poor quality standards conversion. The reasons for the problem are rather complex, and beyond the scope of this article. Suffice it to say that whenever a telecined film sequence was recorded by a film recorder, there was a 50% chance that it would suffer from picture phasing problems. As a telecine engineer, I find it particularly frustrating that twenty years ago the addition of a simple circuit to the telecine system would have completely eradicated this problem. Once recorded, however, there is absolutely no way the error can be removed. A variation of the picture phasing problem, this time due to the design of the film recorder, is responsible for some of the vision cuts on the studio material appearing as single frame mixes on the film recording.

Episode five suffers from some vertical instability of the picture, due to misalignment of the film recorders claw box, the device which pulls the film down frame by frame prior to exposure. Again, this fault is recorded onto the film and is impossible to remove.

Another difficult problem is dirt and scratches on the film recording. Although brand-new prints for all of the restoration work were struck from the original negatives via a wet-gate printing process, they still exhibit dirt problems. Most apparent is white `sparkle' caused by damage on the negatives, and which therefore always appears in each new print. There is also an enormous amount of sparkle on original film sequences which was there when it was first transmitted, and would certainly not be acceptable in a modern-day programme. The BBC does possess a device for electronically removing sparkle from images, but it has to be used very sparingly as it tends to produce nasty side-effects on areas of fast movement. We used it only once during `The Dæmons', during the final seconds before the credit roll in episode one. If your VHS possesses the ability to step through recordings frame-by-frame, try looking closely at this sequence, and you will be able to see some of the side-effects produced by the device.

After finishing work on `The Dæmons', the same team successfully applied for funding to enable similar restoration to be applied to three other stories, namely `Terror of the Autons', `Doctor Who and the Silurians', and `The Ambassadors of Death'. This time, the funding was provided from three sources - BBC Archives, Home Video, and Enterprises Programme Sales. The first two of these stories have been completely restored, with `Terror of the Autons' receiving a great deal of praise for its outstanding picture quality. The third poses more of a problem, as the colour copy exhibits terrible patterning. One episode may be recoverable, but unfortunately the others may not.

However, if any reader (particularly in America) has a long forgotten off-air copy of `The Ambassadors of Death', `The Mind of Evil', `Planet of the Daleks':3 or `Invasion of the Dinosaurs':1 lying around gathering dust, let us know!

Hopefully, whilst we are waiting for one to turn up, we'll be able to look forward to the Home Video release of at least two more restorations in the near future.

[Second article, 1993]

[Restored scene from 'The Ambassadors of Death']After completing work on the restoration of `The Dæmons', project supervisor Ralph Montagu applied for further funding to enable an attempt to be made to restore three more Pertwee stories. The application was successful, with funding this time coming from three sources - the Film & Videotape Library, Home Video, and BBC Enterprises Programme Sales.

It was hoped that `Terror of the Autons', `Doctor Who and the Silurians' and `The Ambassadors of Death' would all be able to be restored using the same techniques developed for `The Dæmons'.

To recap briefly, the restoration process is based on combining the colour signal from an American off-air Betamax videotape, with the BBC's high resolution black & white film recordings.

The first stage of the work was to have the 16mm film recordings transferred to videotape. The team had experienced problems with scratches, dirt, and sparkle on the film prints of `The Dæmons', and so the BBC's Archive Selector, Adam Lee, agreed to have brand-new prints struck especially for the project. A process called `wet gate printing' was used, in which the original negative film is immersed in a liquid which fills in the scratches on the film, rendering them invisible on the copy print.

The film recordings were transferred to tape in the BBC's Telecine department, mostly by Dave Hawley, who has worked on all the ongoing Doctor Who projects since `Resistance is Useless' at the beginning of last year. Initially, `Terror of the Autons' and `Doctor Who and the Silurians' were transferred, with `The Ambassadors of Death' being done a few weeks later, by Duncan Bragg. Wherever possible, the restoration team try to use fans of the series to do this sort of work, as their empathy with the material helps to produce the best results. At the same time the telecine work was being done, the American off-air recordings were being converted to our PAL video standard, and copied onto a new set of tapes.

After all the problems that the team had experienced during work on `The Dæmons', it was decided to start with the easiest story first - in this case `Terror of the Autons'. We were fortunate to have very good colour copies, and excellent film copies of the story, and so hopes were high that the results of the restoration would be very good indeed. It was a bonus to us that, unlike the other stories, the American tapes were in complete episodic form, rather than as a compilation.

Restoration of `Terror of the Autons' took place in a single day - about a quarter of the time that it had taken to restore `The Dæmons'. At the same time that the Sunday morning audience at the DWAS' PanoptiCon XII convention were enjoying a specially produced BBC trailer for `The Dæmons', the rest of the team were hard at work back at White City, setting up the critical alignment between the colour and black & white images. Once this was done, the two images stayed locked together practically all the way through each episode, the only breaks in recording being made to sort out minor picture faults. As the colour tapes were of complete episodes, there was no need to produce new end credit rolls, as had to be done for the other stories, and so this was a major saving of time. However, the beginnings of the episodes were not particularly clean, so new credits were made up on paper, loaded into Paintbox, and keyed in over a clean background film supplied by the BBC Library. As with `The Dæmons', an attempt was made to make Pertwee's face appear to have a natural flesh tone - however the colours of the rest of the sequence are much brighter and more unusual than those used for that story.

There was one major picture fault which was to cause the team a few problems. It occurred during the Doctor's dissection of the plastic doll. The black & white film recording had a major fault, lasting about half a second, in which the picture completely broke up due to the film recorder losing lock when the film was made back in the early Seventies. The first attempt to overcome this problem was to replace that particular part of the scene with `raw' picture from the American tapes. Unfortunately, the effect of this was that the picture suddenly appeared to go out of focus for half a second, which was almost as disconcerting as the original fault!

The team had heard of Gazelle, an experimental device built by the BBC Research Department, which was designed to make slow-motion pictures appear smoother by `inventing' new frames of video between the real frames. It was reasoned that if we fed the machine with video from before and after the disturbance, it might be able to make up the missing eleven frames. Unfortunately, it was all too much for the poor machine, the result being a mix-through from the first to the last frame, with only the movement of Benton's elbow being guessed at! However, Gazelle is still under development, and it may be possible to use it in the future. The team have talked about the possibility of using it to clean-up part three of `The Faceless Ones', which has numerous picture jumps all the way through it.

The final solution was to use the raw colour pictures, but put through a device called an `aperture corrector', which gives the appearance of high resolution by artificially enhancing the edges of any object in the picture. The viewer is still aware of something happening, but the amount of distraction caused is much less than it was.

As predicted, the quality of the finished restoration was nothing short of excellent, much better than `The Dæmons', which was now looking slightly poor by comparison!

Work began the following week on `Doctor Who and the Silurians', the longest story which the team had attempted to restore, with seven full episodes to be done.

As with `The Dæmons', the colour tapes were a recording of a compilation transmission - this meant that there were no beginning and end titles, and there were bits of scenes missing, particularly around the episode overlap points. Luckily, the team's experience on `The Dæmons' had led them to develop a number of ways to deal with missing colour footage, and all of them (and more!) would be employed over the following few weeks.

Some of the worst problems occurred during the scene in which the Doctor meets the Silurian in the cottage. Some scenes were entirely missing from the colour copy, and a great deal of time was spent trying not to have to lose any footage because of this. Some sequences had to be hand painted frame by frame on Paintbox, an exceptionally tedious and time-consuming process.

There is also some interference recorded on the colour tape around the junction of episodes five and six, which causes rainbows of colour to be superimposed on the picture. At the time of writing, James Russell is developing a hardware/software package for the Apple Macintosh computer, which it is hoped will be able to reduce, if not remove, this patterning. Fortunately, it is fairly mild, and only lasts for about five minutes.

One thing which the team had to constantly be wary of was the temptation to not just restore, but to also improve on the original. For instance, during the scene in which the Silurian is about to attack Liz Shaw in the barn, there are annoying white `edit-flashes' at each picture cut. This was due to the way the original film sequences had been spliced together, and it would have been very easy to take out a frame at the beginning of each new shot to remove them. However, this would not be in the spirit of restoration, and the viewer would not be seeing a fair representation of the story as it originally was.

[Rainbow patterning on 'The Ambassadors of Death']Unfortunately, work on the restoration of `The Ambassadors of Death' has been halted after only two episodes (five and six) had been completed. Much of the colour tape exhibits the same rainbow patterning as was seen on episode five of `Doctor Who and the Silurians', but to a much greater degree. Unless a technique can be developed to remove this patterning, it would appear that the only way to proceed with this restoration would be to employ a `colourisation computer', as used to colour Laurel & Hardy films, for instance.

[Computer colourisation by AFT]A sample of `The Ambassadors of Death' has been colourised for us using this technique, by `American Film Technologie', the largest company of its kind in the States. The results are very impressive, but at a cost of over $2000 per minute of finished programme, it will probably prove financially unviable to restore four episodes of one story using this method.

However, $50,000 may not be too much to pay if colourisation of one episode effectively releases a whole story, which would be the case for `Planet of the Daleks' and `Invasion of the Dinosaurs'. Of course, this will only happen if Home Video perceive a strong enough market for the tapes, so if you want them, speak up!

[Computer colourisation by AFT][After this article was written we actually used American Film Technologie's process to fix a ninety second colour fault at the junction of episodes five and six of 'Doctor Who and the Silurians'. The scene starts with the Doctor and the Brigadier arriving at the Cottage Hospital in Bessie and ends with the Doctor and Liz in the laboratory. All the colour in this section was produced artificially by computer.]

As noted above, there are still several episodes that could be restored to colour, if only we had the material to work from - Can you help us?

Copyright Steve Roberts 1992,1996