What is going on to create the illusion that an image is popping out of the screen?
This week, I will share with you how we do this.
Although the technology may seem complex, I have structured this piece in such a way that I hope it is easy to understand without using too much jargon.
First, I will share more about the biology of the human eyes and how they are designed to make this technique so accessible.
I will then explore how the filmmaker harnesses these natural laws, and then share a bit about how the compositing artist brings it together once the images are captured.
And, finally, I will show you the magical way science and engineering come together in the cinema so that you may experience this phenomenon.
First, let’s start with some basic terminology se we may kick off from the same page.
- “3D”, which stands for 3 dimensional, is always going to be a bit of a misnomer for any image, because nothing you see on a flat screen is actually in three dimensions.
But in the world of filmmaking, and especially visual effects, the term “3d” typically refers to something modeled in “3d space.” This means there is a vertical axis, a horizontal axis, and a depth axis. (The best example is the popular computer animated films, which are created in a 3d space inside the computer.)
- Stereoscopy is a more accurate term to describe the movies one watches with the special glasses that give the effect of images floating in front of the screen. Just as we may hear two sounds in stereo, with this kind of filmmaking, we are also seeing in stereo.
So, for our purposes today, we will be discussing this kind of image making as stereoscopic filmmaking, rather than “3d”.
In order to understand more how the filmmakers, artists, and engineers create this optical illusion, it is helpful to understand how the brain receives and interprets light through “binoculars” (two eyes).
If you are to alternate shutting one eye and leaving one open, you may notice the illusion that whichever object you are looking at has moved.
This is, of course, not the object moving, but a result of seeing the same object from two eyes, which are naturally in different positions (I hope).
But the eyes are never looking “straight” forward, they actually each have the peculiar and independent ability to shift their respective angle, based on the object they are both looking at.
If the eyes were, in fact, looking parrallel, you would, of course, see two images, and probably go slightly crazy.
This makes sense, because the angles are the determining factor in preventing double images.
This phenomenon is called “convergence.”
If the object is far away, there’s less of an angle needed for convergence to take place. As your eyes will never be parallel, they will also never invert the triangle, i.e., shift to view the extreme outer sides independently.
On the other hand, the angles sharpen the closer the object is, so the more “cross eyed” one appears when looking at it as it is closer.
This is all due to the angle of convergence.
A good example for understanding parallax is to look out the window of a moving car and see objects close to you moving quicker than those in the distance. This is because of variation in distance, not any variation in the speed of you or the objects. No matter how fast you go, the relative distance between foreground, middle ground, and background is what is affecting the parallax.
The evolutionary benefit of this system includes the ability to perceive depth at a close range, thereby protecting the organism from a world that is not flat, but a world full of the teeth of predators in three dimensions, making them harder to escape from. If you are constantly bumping into a flat and confusing Picasso world of 2 dimensions, you probably will be eaten!
Two eyes help you with this.
So, let’s think about these same principles, but look at it through the lens of filmmaking. (Puns are always intended.)
In filmmaking, the camera simulates the mechanism of the eyeball by receiving light through the iris and the lens, where it is then recorded for playback.
Playback is the one thing cinematography gives us better than the brain itself, and why it’s so powerful for storytelling. Cinema allows us to experience, and re-experience a collective lucid dream (Whereas the playback of my own dreams is usually dysfunctional before I have my first coffee).
Instead, with stereoscopic filmmaking, we are simulating not only the projection of light onto a sensor, through a lens and iris like an eye, but also by creating a system of convergence that happens with a binocular system.
So, stereoscopic filmmaking actually simulates the binocular, by recording images from two different positions.
So, the stereoscopic filmmaker mimics the interocular separation by also creating an “interaxial” separation between the two cameras.
(This distance is forgiving and should be calculated based on your focal length and distance to subject, and your final output projection. This calculation will be best arrived at through the results you want, artisanship, practice, and experience. But a typical safe calculation of interaxial distance is 1/30th of the focal distance.)
- For these purposes, we should just always remember the premise of stereoscopy: Simulating the binocular. The distances are based on relationships between the eyes and the subject and how you want the viewer to experience it.
So, at the stage of recording the image, the stereoscopic filmmaker is essentially triangulating on an object and creating parallax through a two camera system designed to simulate the binocular system of the eyes, which allows for a trick of depth the way the eyes do.
The process of stereopsis is simply tricking people into thinking they perceive depth floating off, on, and behind a flat screen.
Objects in a scene will have different horizontal offsets, due to the binocular system, so the wider the range of foreground, middle ground, and background, the more opportunity for parallax.
However, the point where the angled lenses converge is also the point of “no parallax.” In simple terms, this means that the triangulation on an object has created a point where there are no “double images.” This occurs where "the eyes meet," in a manner of speaking.
With the image projected, this point of convergence will be flat on a viewing screen, because there is no parallax at the point of convergence. Moreover, anything outside of that point will produce the illusion of either being in front of the screen, or behind it, hence the so-called "3d" effect.
Just as we have convergence, we can understand the phenomenon of the object floating between us and the screen, as well as an object appearing behind the screen, because of another, opposite principle:
A simple way of understanding divergence is to invert your understanding of convergence!
So, the eyes are not only converging on an object, there still exists a kind of double image based on how close that point is to the eyes. This is due to each eye, again, being in a different position. In other words, we still see what is in front of the object of convergence, as well as behind it - another great evolutionary benefit that also helps filmmakers!
This causes the furthest object to appear “behind” the screen, creating positive parallax, so to speak, as a result.
Because of the triangulation of the eye combined with the separation of the two images, we are able to see “under the hood” of the parallax that’s created by looking at either the right camera or the left camera.
From this triangulation, we also see the foreground with a corresponding image between our eye and the point of convergence, creating the illusion that it is floating between us an the screen. This is negative parallax.
That which is behind the point of convergence is positive parallax because our eyes have crossed the point where no parallax exists (because our eyes have converged on that one spot.)
Once the images are recorded, a compositing artist takes the work done in camera and brings it together to create the final output for projection. The post production process can be very challenging for a stereoscopic compositor, because not only does a lot of work often need to be done on a single shot, the artist is also working on two versions of a film, plus any decisions made when shooting it, and let's not forget that rarely is a film composed of only one shot! (Not to mention lots of sushi breaks. Never mention the sushi breaks).
Shadows are often dealt with for each of the two cameras for every shot, and any changes are made to improve the effect, color corrections applied, and various visual effects work done to both images.
So, once all the final artisan work is done, the final motion picture is output and projected for viewing.
However, there still exists one problem that must be solved:
How does the viewer watch these double images as one, without seeing double?
To eliminate the problem of doubled images, many ways have been explored, but we will focus on the most popular, and most effective, which is through circular polarization.
Polarized waves are an interesting phenomenon that humans artists and engineers have harnessed, in effect, by switching light waves on and off for each eye.
You can imagine horizontal vents on one filter, and vertical vents on another. When the nonpolarized light passes through that filter, only a horizontal or vertical light wave makes it to the other side. The other planes of light are “switched off” so, in effect, one vent is receiving only horizontal light waves, and another is only receiving vertical light waves.
To use these “vents” for viewing stereoscopic images, we may construct the so-called “3d glasses” with one eye using a vertical filter, and the other using a horizontal filter. Now, only a vertical wave will arrive at one eye, and a horizontal wave will arrive at the other.
To complete the interaction, the projection must correspond to those vertical and horizontal receiving filters.
So, the film reel is played from two projectors, but each one with it’s own polarization filtering only one type of wave:
This allows the viewer to see two images while his and her brain is tricked into thinking he or she is seeing one. The trick here is that each lens of the glasses is turning off one while leaving on the other.
To sum it up, when you watch a film with the special glasses, it is not, in fact “3D.”
What you are seeing are stereoscopic images, and it is a result of the artists and engineers simulating the way your eyes already see, with two lenses that converge on one point, and then a post production artist manipulates the resulting double image, probably giggling in an evil manner as he does.
Through a sleight of hand, filmmakers use the science of turning off both your eyes to different wave vibrations without you knowing they have done so.
When you hold your your finger up close to your eye, the sense of depth you experience is your binocular system seeing, not only the point of convergence, but the parallax created by such a system.
But when filmmakers do it, it is because they have mimicked the way your eyes already function to create depth in a three dimensional world.