Gravity won the 2014 Oscar for Best Visual Effects, completing the trio of top Visual Effects (VFX) honours alongside the Bafta and VES. All three awards were testament to remarkable engineering innovations and achievements delivered by Framestore – the London based agency which produces creative images for film and advertising.
Here, in an exclusive interview, Framestore’s Head of Engineering Andy Howard, and Chief Technology Officer, Steve MacPherson give a sense of the extraordinary engineering and technology challenges they faced in the journey bringing Gravity from script to screen, and how the film could help to inspire the next generation of engineers.
Q: Andy, as Head of Engineering at Framestore, can you talk us through the scale of the engineering challenge that your team faced, when they first started to work on the Gravity project? As an engineer, how did you approach a project like this?
AH: “From the engineering side we had to design build and implement electronic circuits to allow us to configuring the special cameras required to capture the motion data on set. This would normally be a challenge as there are multiple equipment vendors that are involved but our challenge was to get this working in a matter of weeks rather than months. In addition we also had to accommodate modifications as a result of feedback from the set which would have to be turned around in a day or even over night.”
“To approach this type of project we divide it into four stages. First we have to get a brief from the creative technical supervisor to determine exactly what they are trying to achieve. The next stage is to draw up the circuit design using a CAD system and then source and cost the components required. There is a parallel stage that involves discussions with the manufactures of the equipment we have to connect into which includes bespoke control interfaces.”
“The second stage is to go through what is possible in the time scale and if there are any major issues that will make it impossible to go with the initial requirements from the special effects supervisors. In the case of Gravity it looked like it was possible, although it was going to be a challenge given the time scale and current technology.”
“The third stage is to build the prototype and carry out testing with the camera manufactures. This often involved trips to the manufacturer’s UK offices or film equipment hire companies with the unit and test equipment, scopes, multimeters, and so on. Once we have a working unit we then had to run tests on set before the main shoot started. This is always stressful as we were not the only people trying to run tests on a very expensive set. Here timing is everything and when you get your narrow window of testing time you have to make sure you have everything ready to go plus spares that can be swapped out if the first units, cables cameras or custom PCs don't work.”
“The final stage is to refine the design and support the system during shooting. Feedback from the technical operators on set is normally quick and we had an engineer on set throughout the whole of the shoot. Changes were then relayed back to Framestore's engineering department, implemented and shipped back to set. The key thing here is documentation, every change and its effect needs to be logged and available to all concerned, including third party manufacturers. Any lapse in this would have caused major delays, for example: if the firmware in the A-camera was updated by the manufacturer to assist in a timing issue and we had modified our circuit to match, it is vital that the camera engineers know that if they switch to the B-camera this must have the same revision of firmware in order to work accurately.”
Q: The practical solutions to the problems that the team came up with were revolutionary. Can you give an insight into the processes involved in delivering the solutions?
AH: “A lot of departments came together on this film, software developers/programmers, pipeline or workflow developers, hardware electronics engineers, installation engineers (broadcast engineers), colour scientists as well as a large system admin and technical support team. The big challenge is getting all the teams to work as one and to make sure all departments have the latest information and milestone dates are hit. The process starts off with pre-visualisation of the visual effects, in the case of Gravity this was nearly every shot. Once the director and all creatives are happy then we have to start testing/rehearsing the complete process.”
“From an engineering prospective we installed our first generation hardware and software on set and then started test shoots. The image data was transferred over a dedicated network link back to Framestore where animators and 3D graphics artists could then determine how useful the material was. This is then communicated back to set and the engineering hardware and software departments to make the required changes. As this was a new way of working the number of modifications both in software and hardware were initially very high.”
Q: As an engineer, was this as close to working on an actual space programme?
AH: “From the point of view of accuracy then yes as we were trying to get as much detail as possible for people in the space technology field. The main studio production has a research team that deals with this so our input is small but it did include visits to universities and satellite manufacturing facilities.”
Q: Steve, what was it like working on a virtual space project having actually worked on real-world projects while with NASA and Lockheed Martin?
SM: “That's an interesting parallel. When I worked at NASA Ames on their Cray Research supercomputer systems they were the largest and most powerful systems available. The scale they operated bore almost no relationship to commodity equipment (early PCs and Macs) and this was before workstations became widely available.”
“At Framestore, we knew we would have to retool large parts of our infrastructure to support Gravity and if it's not stretching the analogy too much we also knew that if we didn't get the design right by the time Gravity ramped up we would not be able to make the changes. At NASA, changes to the infrastructure required extensive planning due to the demands and expectations of the various departments using the Cray. And one thing that is the same - if the systems do not work you end up with a room full of doctors asking when it will be back up!”
Q: What were the giant leaps forward for the UK-based CGI industry that Gravity has delivered, and what are the unique blend of skills, expertise and technical disciplines which are required to deliver world-class products and results of this kind?
SM: “I'd say the big step for us was in proving to the studio that a UK post-house could single-handedly manage and deliver a project of this scale and complexity. There is no question around the capabilities of the UK houses in terms of delivering big VFX sequences but where Framestore stepped outside the boundaries has to do with the level of production involvement – the level of pre-visualisation, the way we housed the actual studio production team, how closely we worked with the director in realising his vision of what the movie would be."
“For the technology team at Framestore, we did as much engineering and systems design as our creative counterparts were doing. In a nutshell, we completely gutted the technology framework and rebuilt it while at the same time continuing to provide a level of service where the artists and developers were never interrupted by our activities.”
“In a ‘Google’ world where so many error codes can be quickly defined I am looking for people who can navigate through tricky and unpredictable situations with composure and skill. This is not to say that a strong foundation in the sciences is not required – it absolutely is as I suspect it is this bedrock of information that allows people to put together the big picture and reason out solutions that might not be obvious at first pass. In what we do there are fundamentals to storage and networking that evolve rather than change radically so having someone who has done their homework on these fundamentals and is willing to keep an open mind to new approaches.”
Q: Among the many ground breaking innovations of the project, what was the engineering innovation that gave the most pride?
AH: “From the engineering side it was the ability to lock our high detail reference camera data with the main film camera. On Gravity all cameras are digital and as such there is very little mechanical sensing data coming back. We had to synchronise the electronic shutter of our cameras with that of the main shoot cameras as all devices have different characteristics and timings (in ms) – the achievement I am most proud of.”
Q: How did you both first get into engineering?
AH: “From a teenager I was always playing around with audio equipment and any electronic device I could find to take apart. It was not just electronics but electromechanical systems as well. An enjoyment of physics and the innovations in technology we studied in those lessons at school made up my mind that a career in some form of engineering was where I wanted to go.”
“Originally, I was an electronics engineer but quite early on in my career I went in the direction of broadcast electronics. This is a specialised area dealing with images and sound and originally was all bespoke electronic hardware, though now it utilises a lot of standard computing technology. As most films are no longer shot on conventional celluloid but on electronic cameras, the similarity to TV broadcast is very close. As the film VFX industry is always striving for the next new thing then there is a lot of scope for engineering students from pure electronics, electro mechanical through to disciplines such as fluid simulation.”
SM: “I started out self-taught, learning basic DC and AC electronics from books in my spare time. This led to a stint at a technical college where I discovered I had both an aptitude and a true sense of joy for the logic and symmetry that defined the early days of computing (early 80s).”
“I ended up at Cray Research and this was a huge step for me – at Cray you went to Wisconsin for over a year's worth of training. During that time we learned at a Boolean level the ability to trace circuits at any point in the Cray architecture. Memory control was particularly brutal. It was a brilliant time and the chance to work on the Big Iron at NASA Ames and Lawrence Livermore National Labs was super exciting.”
Q: What have been your career highlights to date?
SM: “I suppose top on the list will now be the delivery of Gravity. Like any big project, there are numerous opportunities at the beginning to figure out why what you are embarking upon is absolutely impossible! Our CEO, William Sargent, was resolute that this would be a great thing for Framestore and that gave us the push and momentum to kick off. Once this got going there were so many challenges, big and small, and each one represents some reward for effort in its own way.”
“Another thing I will always be proud of is my involvement in the early days of setting up Double Negative ‘from a blank sheet of paper.’ Working closely with the artists and developers and building a facility at the same time as delivering that facility’s first film was fun and rewarding. I consider myself really fortunate – the people I work with are consistently the kind of people I would choose as friends even if we weren't working together. And the chance to sit in a dark movie theatre and see that effort materialise on the screen is a great feeling.”
AH: “Gravity is by far the leader in the amount of technology used, but not long before Gravity some of the camera processes were being tested and refined on films such as War Horse. It was early days but we could see were this technology could take us. Setting up stereoscopic viewing systems for the work we did on Avatar was very interesting – it was physics and electronics with a bit of psychology thrown in.”
“When I first joined the broadcast industry it was nearly all analogue electronics with the first digital kit just appearing. I worked for several companies that pioneered the move to digital video and audio, despite many people at the time saying it was either too early or would not be as good. Nowadays we take it for granted in a digital world with phones that have more audio and video capability than racks worth of highly expensive equipment, but there was a time when this could not be imagined.”
Q: Having worked on a project like Gravity, what would be the next ultimate ‘engineering for film’ challenge?
AH: “This would be further work on the live compositing of CG. We will capture the scene with motion capture data and use this data to do a real time / real motion composition of the CG character. There are plenty of people working on this with varying degrees of success but the challenge is to get it to a stage that the director can put on a set of LCD glasses and direct the live action in the 3D graphics environment.”
“The analogy is that of actually being in a computer game environment. It may sound far-fetched, but we can feed motion capture data into software platforms and see its effect on the CG model in real-time. There are some fundamental technical issues to resolve and these are as simple as: how you can get the images to the director in real-time without cables, with enough wireless bandwidth to produce acceptable images and sound, allowing him to move freely around the physical as well as virtual set.”
Q: Many IMechE members – the current Institution President included – became engineers because they were inspired by seeing the Moon landings and NASA’s space programme in the 1960s. Could Gravity inspire youngsters to consider careers in engineering?
AH: “From my point of view, Gravity is a feature film compared with the monumental achievement of space exploration especially given the technology of the day. However, I watched many films and documentaries when I was younger and all fuelled my interest in all things engineering.”