Three*4

WRITTEN COMPONENT ENTRY  Low Cost Mocap notes.

-       motion capture, or mocap, is a technique of digitally recording the movements of real beings, usually humans of animals.

-       mocap is considered to be a better technique for accurately generating movements for computer animation.

-       3 types;

o   optical motion capture

o   magnetic motion capture

o   electro-mechanical motion capture

-       in this paper, we describe the design and implementation of a low cost motion capture system that requires two low cost calibrated webcams.

-       as all mocap systems involve a tracking phase, we adopt the mean-shift algorithm as the basis of object tracking.

-       system has an inability to handle occlusion.

-       provides input to animation applications, such as Poser.

-       the mean-shift algorithm is one of the tracking techniques commonly used in computer vision research when the motion of the object to be tracked cannot be described by a motion model.

-       the key notion in the mean-shift algorithm is the definition of a multivariate density function with a kernel function ‘K’ over a region in the image.

-       the commonly used kernel functions are the normal, uniform and epanechnikov kernel functions.

-       amongst the commonly used kernel function above, the epanechikov kernel, which has its kernel profile as a uniform distribution, is preferable than the other two.

-       the Bhattacharyya coefficient is used to calculate the similarity measure between the two distributions.

-       the necessary pieces of equipment required are two low-cost webcams, two tripods and a calibration frame.

-       each webcam mounted on a tripod must be calibrated prior to any experiments. The current version of the system focuses on the capture of movements of the lower part of white circular markers to be put on the following joints; hip, two upper legs, knees, ankles and feet.

-       to simplify the tracking process we darken the background.

-       subject wears a dark non-glossy tight suit so that the white circular markers can be easily detected.

-       the two webcams are directly connected to a computer via two USB ports.

-       we currently use functions under the Matlab Image Acquisition Toolbox for image acquisition.

-       camera calibration is a step for determining the 3x4 matrix that maps coordinates in the 3D world into the 2D image.

-       in our system, we use a calibration target with two orthogonal faces, each of which has 6 reference points. The calibration target also implicitly defines in the scene a global coordinate system that can be referenced to in some other applications, such as Poser, for graphics rendering.

-       markers were detected via a thresholding process.

-       involves choosing a threshold value ‘t’ from the pixel intensity range of 0 to 255.

-       the threshold value can then be estimated from the flat region in the intensity value histogram.

-       the 9 markers are automatically labelled using a heuristic method.

-       after all 9 markers have been detected, the system labels the top middle marker as marker #1.

-       the assignment of marker number on the ‘y’ component of the marker coordinates.

-       the mean-shift algorithm is employed to track the 9 white markers independently.

-       there are 3 free parameters that can be set to fine tune the performance of the mean shift algorithm;

o   radius

o   threshold value

o   number of histogram bins

-       for the computation of the 3D coordinates of each marker, the two 3x4 matrices obtained above are combined to give 4 linear equations for the detected image coordinates of the marker in the two images. The 3D coordinates of each marker, relative to the implicit global coordinate system defined by the calibration frame, can be estimated using least-squares.

-       in every experiment, we tested our system to track the movement of markers over 200 frames.

-       we found that there is a 0.016 seconds delay between the acquisition of an image by the first webcam and the second.

-       we found that the radius of the kernel windows is a crucial parameter to the performance of the mean-shift algorithm.

-       issue has also been reported in that a window size that is too large can cause the tracker to become more easily distracted by background clutter and a window size that is too small can cause the kernel to roam around on a likelihood plateau around the mode, leading to poor object localisation.

-       system captures movement of the lower body, it can be further extended to include the upper body.

-       the notion of low cost mocap is important for demonstrating the fundamental idea of motion capture and for providing inputs for various advanced animation applications.

R. Budiman, M.Bennamoun, D.Q.Huynh. “Low Cost Motion Capture”. University of Western Australia.

 

Three*3

WRITTEN COMPONENT ENTRY  5D notes.

5D concerns the coming together of designing in virtual space in film, TV, animation, interactive and architecture. Rick Carter (Production Designer, Avatar) wants 5D to ‘sprout’, “Story is design and design is the story”.

A few definitions were unveiled at the ‘5D: The Future of Immersive Design Conference”;

- Previs - a collaborative process of cinematic discovery based in a 3D virtual environment.

- Pitchvis – intended for the purpose of pitching projects.

- Technical Previs – real-world camera and shot layout to streamline production.

- On-Set Previs – on location, on the fly, to respond to the needs of the director and crew, utilising realtime techniques.

- Postvis – pre-emptive VFX in which previs assets are integrated into live action plates and a director’s tool for editing.

John Underkoffler (Scientist and Designer of human-machine interfaces) discussed design and world building in terms of using your hands for touch and intimate contact. “The language of cinema itself can be used to describe space, time and narrative”. He envisioned on interactive, multimedia, realtime experience. There is the question of how technology change storytelling (realtime in particular).

http://www.awn.com/articles/people/getting-immersed-5d

 

Three*2

WRITTEN COMPONENT ENTRY  Impact of 300 – Stylised VFX notes.

The filmmakers of 300 voided the convention of being very conservative with their camera work when they are unsure of their environments and compositions. They used every camera move that they could think of in order to recreate the graphic novel in a faithful way with the knowledge that they would compose their shots later on.

The style of the film became a hybrid; not photoreal, not cartoon and not completely CG. The style may be appropriate for some mediums and subject matter, but it will most definitely not work for all films.

Chris Watts (VFX Supervisor) believes that a successful stylised movie has to have flaws and variations in style or it can get novel but boring. For instance, he purposefully allowed each battle to have its own unique characteristics.

Other professionals agree that a film shot the same way seeking a more realistic look would be more expensive.

It’s not easy to create a stylised film, the challenge lies in creating the work in an attractive style, and depending on the style – it may ignore real world effects such as lighting.

http://www.awn.com/articles/production/impact-i300i-more-stylized-vfx/page/1%2C1

Three*1

WRITTEN COMPONENT ENTRY Surrogates vfx notes.

Focus on the dead Surrogates sequence at the end of the film;

-       began with still photography.

-       added CG cars, smoke, CG satellites on rooftops and dead surrogates.

-       full 3D recreation of the 350% camera push in of the CG trees, matte paintings and CG buildings.

-       new matte painting for anything that wasn’t moving originally (and need to be moving).

-       trees were created through Paint FX in Maya.

  

http://www.vfxblog.com/vfxblog/2010/03/the-visual-effects-of-surrogates.html#more

Three*0

WRITTEN COMPONENT ENTRY Terminator Salvation vfx notes.

 The style of the film affected the visual effects; McG (Director) wanted a grittier and dirtier look for the film, which involved colour correction on elements such as matte paintings. To achieve the bleak, oppressive look of the film the matte’s were colour corrected to a desaturated mono-chromatic look.

For Sam Worthington’s character (Terminator, Marcus Wright) the vfx involved a complex mixture of live action, CG and prosthetics. The CG elements had to mesh with the live action seamlessly in a way that did not take did not take away from the performance of the actor. Worthington was applied with his makeup (prosthetic wound) and blue screen material where the 3D elements would be exposed. Precise data was taken when filming each scene, and the vfx crew would always film additional reference footage with 3 additional cameras, from 3 different angles.

The major challenge with the character was imitating the movements of the live action. This also involved 3D rotoscoping, followed by 2D tracking to refine and improve upon the 3D match-move animation. For each shot a matte of the wounds, keying and rotoscoping the digital blue portions of the live action makeup appliance. The matte is converted to a geometry outline, the outline was projected onto the 3D rotoscoped deforming mesh of the actor’s performance. The projected outline was used as the input for a procedural process in Houdini that modelled the wound edges ‘tearing open’ the deforming 3D mesh and revealing the CG endoskeleton.

Rising Sun Pictures approached the robot hand shots by using a provided physical hand model as reference. The live action hand was ‘painted’ out and replaced with the robot hand, which had been move matched.

http://www.fxguide.com/modules.php?name=News&file=article&sid=549

Two*9

WRITTEN COMPONENT ENTRY District 9 fxguide notes

The handheld digital camera work resulted in considerable match-moving work for the visual effects people. Image Engine Producer Stefanie Boose noted, “There was a huge amount of camera tracking involved. Actually, there almost wasn’t a single locked off shot for us, which really added to the grittiness of the filkm”. The studio utilised PFTrack, Equalizer, Boujou (3x tracking) and Silohette (paint – roto work).

Grey mocap suit provided good lighting reference. Director, Neil Blomkamp knew the importance of capturing HDRI imagery (some laser scans were taken of the environments in Africa).

At times Blomkamp was content to duplicate the mocap suited actor’s performance exactly, and other times the performance was altered to tone down the actions of alter the emotional weight i.e. aggravation. The animator’s added some very ‘sharp’ movements and ticks that human actor could not achieve; they could simply not move that fast. Motion Capture Supervisor (Animatrik Film Design) Brett Ineson commented that, “It was kind of a guerrilla way of working – we’d shoot a lot of mocap but then throw out half of it and decide to keyframe it if that worked better for the shot. “They felt it was important that the aliens movements were not perfect; some would limp, others would appear to have physical impediments.

To integrate the aliens into the scenes a HDRI pipeline and lighting rig that could be applied to both the exterior and interior environments was developed. Nuke was used for the compositing for the reason that it had good 3D functionality as well as other tools that matched the look of the other media in regards to grain and noise.

http://www.fxguide.com/modules.php?name=News&file=article&sid=561

 

Two*8

WRITTEN COMPONENT ENTRY Avatar VFX notes.

According to James Cameron (& colleagues), Avatar is a game-changer for the way that VFX movies are made, watched, discussed and written about.

The virtual cinematography workflow which allowed Cameron to directly observe how an actor’s CG performance would interact with the CG Pandora environments in realtime on an LCD monitor (on-set), furthered the process infinitely. Rob Legato developed the workflow, and it is revolutionary due to its allowance for the firector to interact with CG elements as if he/she were shooting live action. The arts of digital and live action movie making, as well as the ideas of pre and post-production are becoming more and more merged. The paradigm of “5D” apparently a reality.

The ability to make key creative decisions within the digital environment is something that will alter the industry and its roles for the next few years. The access to ‘explore’ this digital realm, which he was responsible for, allowed Cameron to use his filmmaking practically in a virtual set, which is something, which has not been possible before Avatar. Cameron scouted these virtual sets, as he would have a physical environment.

Rob Powers was responsible for the development of the virtual environment; allowances for Cameron to alter foliage layouts, day/night shifts in the moment. Another technique allowed for the creation of an environment ‘sphere’ which contained a never-ending environment which was set to a certain radius so that the rendering demands on the software were not excessive – so the realtime render engine could keep up with matte paintings, and the organisation of all these elements was completely configurable.

Creating such a reality was only possible through the precise manipulations allowed by a computer. Such creatures and world were simply not possible any other way because they could not exist in reality, or be expressed as efficiently though the use of prosthetics or the like.

A new compositing system was devised for 3D; placing depth information on everything that was being rendered, which will become the standard for compositing because of the flexibility it allowed for, according to Joe Letteri “Even if you’re doing a non-stereo movie, it’s just easier to composite in 3D than in 2D”.

One breakthrough that is highlighted in the article (aside from those already mentioned) was the explosions. The need for the explosions to interact with CG objects required that the explosions themselves be CG. Chris Horvath (who was instrumental on the fire fx in Harry Potter and the Half-Blood Prince) was responsible for the shading. The explosions used the same fluid simulation engine used for ‘The Maelstrom’ in Pirates of the Caribbean: At Worlds End. There were modifications to the engine so that it behaved appropriately.

Letteri states that there will be a lot learned from the experience(s) of Avatar, “I think what everyone discovered as you went along is that if you’re going to put a virtual stage together like a live-action shoot, then this becomes the front end to a visual effects piece. Because you not only start thinking in terms of takes and selects, but as shot design. You have to be able to switch from one to the other. And it requires a level of infrastructure for the whole thing that I think is going to benefit everyone if we can come up with some system across the board to make that easier”.

http://www.awn.com/articles/visual_effects/avatar-game-changer/page/6%2C1