1. VFX Sup Guide to Camera Tracking
Introduction
Processes in tracking
Software
Camera tracking can sometimes be an afterthought when the fur is flying onset. This quick guide is to
help a vfx supervisors ensure important information is recorded, laying out tracking markers correctly
and identify problematic shots early on.
As tracking is at such an early stage of the pipeline it is an often an overlooked foundation for the rest of
the facility. Getting this right, and with economy, helps keep projects on time and budget, saving dollars
and headaches.
There are three programs that seem to have become established in vfx facilities that are most note-
worthy to my eyes being 3dequalizer, Boujou and Syntheyes. I’ve used all plus pftrack and a couple of
others but for the purposes of this guide I’ll be talking in Equalizer.
The reason 3de is my preference is that its one of those rare pieces of software that has been nurtured
over its lifetime by the same group of people that are looking for excellence rather than flogging a
shallow upgrade. Other software seems to push its auto tracking heavily, which they admittedly do
well but where 3de shines is in its manual tracking and lens distortion models. This is important when
tracks get difficult making it worth the extra dollars.
It may be helpful in identifying the processes in tracking to better understand why the information
your recording will help the camera department. The processes are as follows.
Loading footage - a straight forward process but.... You want to provide virgin footage. Cropping,
stabilizing or trying to give reconciled stereo footage will only hurt as the programs used are made to
work with the unedited image structures. You can safely play with colour and contrasts as long as the
data has good visibility. I’ve once seen a facility try to get an editor to apply lens distortion to footage
before it was tracked (based on lens grids) - doing so will almost always end in tears.
2d tracking - this can be manual (tracking individual, manually placed markers), or auto (where the
program throws a point cloud of tracking points at footage in order to get the result). Both methods
have pros and cons. An experienced tracker should be able to decide the best method on a given
piece of footage. This is pretty much a physical task and you can gauge roughly how long it will take
based on length of footage, amount of data and spread of needed data. Often a combination of both
manual and auto will give you the best result.
Manual 2d Tracking
pros
The tracker is able to gauge the validity of points in
order to give the best spread of information.
The tracker can focus on extending vague points
to get a better length of information.
Its a clean process helping to minimise bad data
that would hurt the result
cons
It takes time to build for a result
unless you spend a lot of time you don’t get a
point cloud with a vast amount of points
Auto 2d Tracking
pros
- It can be fast on simple tracks
- Because there are large numbers of points
involved it will‘average out’and give a good
smooth result when it works.
- You end up with a large point cloud which can
help with modelling and lineup
cons
- If you need to create a mask for actors and
moving objects the time can blow out as you need
to clean up the data.
- Difficult footage like motion blur, grain, blur, and
poor contrast can leave you bad data you need to
clean up

2. Lens and camera calculation
This is where things get tricky. A tracker needs three things to get a result; camera filmback, focal length
and lens distortion.
All of these items can be recorded on set and go a great way to getting an accurate result quickly.
It sounds easy as I write it here but the logistics of getting it together its not so straight forward.
Camera - if you can name the camera, say a red epic, we can find out the filmback, usually without too
much problem. Its important to get this right as it sets up consistency for the project and will make focal
length figures work correctly. It will also provide valuable information such as if the camera is using
rolling shutter.
Focal Length - this is usually taken care of by the assistant filling in the camera sheets. It really needs to be
better than a scrawl.
fig 1 is an example of a camera report that is actually fairly clear compared to some. Because the onset
information is used by the editor and several other hands the linking of what roll, scene and take makes it
way to what vfx shot number is rarely made and so it becomes detective work to get it right. By the time
the footage gets to vfx we have lost the slate. Notes can sometimes be a clue and you can ballpark lens
information. What is needed is the link to vfx number, a serial number on the lens (productions often
carry more than one kit) and what camera it is filmed on (often more than one brand of camera can be
used).
fig 1. - camera report
Tracking Markers
The shape of tracking markers can be in a variety of different flavours. I find small
squares best as you can track in two modes - pattern or marker.
Marker mode will track a shape - such as a circle so long as the shape does not
change too much where as pattern will look for a pattern within the search area -
such as a corner of a square. Squares are good because if your far away you can
use them as a marker and if you get close you can still use them as a pattern.

3. When laying out markers on a green/blue screen its desirable to give patterns. Looking at fig2 if we were
to go into a portrait shot with a long lens we will often lose our orientation of where on the blue screen
we are looking. If the purpose of the vfx is to do a set extension this can be extremely important to make
sure the orientation stays true between cuts. Patterns in the markers can help this. A grouping, different
colours or a variety of shapes can be good indicators. On the below set we have too few markers, they
used the same shape (a ‘+’ sign) and colour barring a few yellow ones. The markers aren’t laid out evenly
but too sparse to easily identify a particular marker. Also the markers are loaded to where a person can
conveniently reach them rather than giving good coverage.
fig 2. - bluescreen markers
Lidar and Survey
Not every production can afford or find time for lidar and survey, and this guide is for those that cannot
or choose not to provide lidar. I felt it needed a mention however as it is incredibly useful and will push
the tracking into a new level.
Good survey can help pinpoint camera position and greatly improve track accuracy. Not only this but it
will provide consistency in scale and position in the tracks for lighting and animation artists. Without
those benefits it should be created manually and when working with several people across a sequence it
can sometimes get missed.
Lidar and survey however is not without its issues. You need to treat the set as hot once the survey is
done.
Often a long lens cuts out distinctive pieces of the survey and so you end up spending time looking for
what points in the footage match the survey.
If a set changes, such as a screen gets moved for lighting, then the survey needs to be done again if that
screen is in shot. You then need to record what survey matches which shots. And remember it only needs
to move a smidge before its considered moved.
Once you have the survey another benefit is that it can go through the modelling department and you
get some great consistent geometry to use for placement and shadow catching. This is work that often
needs to be done anyway and lidar can help with that process.
On the whole it is well worth the investment if you have a set that will require significant vfx across a
great number of shots. Especially if points of contact for digital elements are important or direction of
the action requires a special consistency. If however your talking one shot or the requirements allow for
digital elements to be more ambiguous then survey can be overkill and you can save the dollars.

4. fig 3. - good data
fig 4. - lack of depth
Natural Markers
Just because you don’t have markers
doesn’t mean you cannot track. Fig 3
shows a shot that has an abundance of
data. The best data here is on flat geometry
such as the boxes or the corner of the door.
‘Flat’means that there would be no back-
ground parallax to interfere with the track
like you would get on the corners of
objects. Other points to make are that
there is good information in the depth,
good contrast and spread.
Although this has all those benefits this
isn’t a great candidate for auto tracking.
Auto tracking will pick up all the intersec-
tions of foreground and background
patterns and so you would spend a bunch
of time cleaning up all the bad points
Lack of Depth
A lack of depth can be a major problem
with getting good solves on translating
cameras. You often get this with walls or
screens.
Because there is a lack of depth the
translates become ambiguous even
when working with survey. Sometimes,
to help, you can get away with putting in
‘c’stands to introduce a foreground
elements. If the translates are not big you
might be able to put the camera on a
tripod making it a simple nodal solve.
Sometimes however, like with this shot,
you cannot really do either an just have
to work a solution.
Data Swipe
Here our two heros are taking tour of the
warehouse. To get a feel for the ware-
house the camera is on the far side of the
shelving and tracks past the rows. This
leads to a situation where the data is
constantly changing or jumping in and
out of the solve. At best this can lead to a
time consuming management of the data
but occasionally it can also lead to broken
tracks where we get a solve for the start
and a solve for the end. Imagine if the
swipe was an actor walking past and
blotting out all the data. Survey can help
this but often it takes some manual
cutting and pasting of animation curves.
fig 5. - data swipe

5. Parallax
Parallax is the difference in position a
foreground element will make with a
background element between frames
due to the movement of camera or
object.
A common example of parallax is seen
when watching a landscape from a
moving car. The foreground points whip
by while the background moves slowly.
Parallax is essential for tracking a
moving camera. The shot to the right is
awkward as much of the parallax used to
track the camera is obscured by the car.
Motion Blur
Motion Blur is just a necessary part of
tracking that needs to be dealt with.
Importantly most tracking software and
renderers track to the centre of motion
blur so they are in sync. However you can
track and render at the beginning and
end. If tracks are sliding at the blur you
may be out of sync.
When thinking about settings matching
3d cameras are able to increase and
decrease the shutter angle so you should
think about how that is calibrated at
lighting too.
Sub frame motion blur can also be an
issue. An animation curve hits a precisefig 6. - motion blur
point in time, but peaks in the motion on footage don’t necessarily hit the centre of the motion blur so
you may get a subtle motion blur mismatch. It can also occur when motion blur is slowing down or
speeding up - what fun.
Another interesting effect people often don’t identify occurs when a blur runs over a background blur.
Because the background points exposure to camera is interfered with, that motion blur is clipped throw-
ing out the data.
See the still - the girl is running across the background and each of those background points inside of her
motion blur is clipped for that frame.
The car is a separate object track but even then it lacks good depth and so the parallax information is
shallow. Tracking objects is a house of cards as often the accuracy of the camera track is important to
calculating the object track accurately. If getting an accurate track on the car in this shot is important I’d
suggest fixing an upright post with a marker on it on the far side of the car. This would improve parallax
and give a more accurate result.
An interesting error can sometimes occur with parallax. As with a car, commonly the foreground moves
faster than the background, however consider a hand held camera tracking around an object on a table.
Here the object is holding its position in frame while the background swings past! This can lead us to
seeing inverted results where foreground points register as background and vise versa. Survey will
certainly help here - even artificial survey, but often it falls to tweaking and weighing tracking points.
It is here that infinitely distant points are gold (such as the farthest buildings here). These points are far
enough away that little or no parallax is registering. These points represent pure rotational values and
tracking software is able to identify and use them.
fig 7. - parallax

6. fig 8. - refraction
Refraction
Refraction should be avoided at all costs. You would
be better off removing this window an putting it
back in digitally than trying to track and render it.
Old glass is especially nasty but so too water, car
windows or heat distortion.
Also any glass that comes between the filmback
and the environment should be recorded in a
distortion grid included anything used in stereo-
scopic work such as mirrors etc.
Producing artificial refraction, even distorted such
as this has been around a long time so you
shouldn’t feel shy about asking for it. Just remem-
ber if you were to put a digital gremlin in this shot
your lining up for a headache.
fig 9. - caustic lighting
Caustic Lighting
Probably not the best example but
caustic lighting can be caused by
firelight, gas light (like here), reflected
pools, rain on a window. Its a really cool
practical effect but tracking which relies
on consistent patterns can struggle with
it.
The best way to get around this is to use
LED lights for the tracking markers as
they generate their own light.
You will also want to appreciate that it
may take longer to track and could be
difficult to match at lighting.
Unseen Points of Contact
This is often a blessing rather than a
curse, but it can be. In this shot we don’t
really know where the floor is.
If we need to put a digital character in
here we would be relying on the track
we get from the background as we
wouldn’t be able to track the characters.
Further to that if we needed to put the
character in the foreground we would be
pushing what we don't know and so the
track starts to become how it feels rather
than what we can see. Putting a marker
on a c stand would help here.

7. Focus Pull
For those that don’t know, pulling focus
is a mechanical process. When we do so
we are turning the lens on a thread to
make an adjustment to the focal length
bringing an element into focus. This
effect is often called Barrelling but it
effectively gives a subtle zoom.
Zoom lenses are another degree of
difficulty to trackers and consideration
should be given as we are often dealing
with blurred footage which has a
changing‘circle of confusion’as well as
the zoom.
fig 10. - focus pull
fig 11. - facial tracking
Facial Tracking
Facial tracking often comes up in produc-
tions either due to a piece of digital pros-
thetics or blood work.
I will often look to the most stable parts of
the face such as corner of the eyes, around
nostrils and the ear.
There are two things that can help this
work.
1. Getting some reference photos or scan
so we can have a good base model to use
2. Having makeup place some markers on
the face to aid the process (disguised or
not).
Btw I never want to see a cats head again.
You wouldn’t believe how they can move.
That's it for a first edition guide - nice and short and hopefully will save some people some grief. I think I’ll
be elaborating on some topics and adding in some new ones, such as zoom lenses and distortion, so feel
free to drop some suggestions.
If you ever need some help feel free to get in touch for a quote or some free advise.
Mark Evans
mark@southpacificanimators.com.au