cs291 »

Contents

Unit 1

Interactive 3D Rendering

Photographs throughout this course are from Wikimedia Commons, unless otherwise noted. Building, XYZ, interactive.

You can try the brain program yourself. Chrome is the best browser for it. If it doesn’t work for you, don’t worry - the next lesson will help guide you through setting up WebGL on your machine.

Answer for Interactivity

Games such as Angry Birds use a form of 2.5D rendering.

WegGL Setup

To see if your machine is set up right, try this site. If your machine doesn’t show a spinning cube, read our help page or go to this page and look under “Implementations”. For Safari on the Mac, follow these instructions. An excellent summary of what supports WebGL is here. Some graphics cards are blacklisted because their drivers are old, never to be updated, and won’t work with WebGL. See this page for more information. It’s possible to override the blacklisting in Firefox, see this article - this might be an acceptable “if all else fails” solution, since this course will be using fairly vanilla WebGL features. Google Chrome has blacklisted XP, so there’s a similar workaround. If all else fails, try different browsers, as they have different limitations.

Interactivity and FPS

The Wikipedia page on motion blur gives a start on the topic. You can see a little bit of some motion blur correction in this demo, Go to view 4 (hit the 4 key) and toggle motion blur on an off with the “b” key. The ground will be blurrier as you move when motion blur is on.

FPS and Refresh Rate

The Wikipedia page on motion blur gives a start on the topic.

Some applications will aim to avoid a rate between 30 and 60 FPS, since then the frame rate doesn’t align with the refresh rate. This video explains in detail how this mismatch can cause a sense that the interaction with the application is not smooth. That said, many games simply strive for as fast a rate as possible.

Question for Math Refresher

This question is simplifying the situation. The 50 Hz rate is actually the interlaced field update rate for European TV, the frame rate is then 25 Hz. Here we're interested in the time between field refreshes at 50 Hz. If you want to learn more, see this page.

Answer for Math Refresher

Some applications will aim to avoid a rate between 30 and 60 FPS, since then the frame rate doesn’t align with the refresh rate. This video explains in detail how this mismatch can cause a sense that the interaction with the application is not smooth. That said, many games simply strive for as fast a rate as possible.

The Eye

See the Wikipedia article on the eye for some truly amazing facts about different types of eyes.

Incognito is a great book on the brain. The first part is all about how the brain interprets what the eye sees.

Eye versus Camera

For this question, compare the human visual system (eye and brain) to the lens mechanism of a camera.

Screen Door

Photo from Wikimedia Commons.

See my blog entry for way too much information on spelling the word “frustum”.

3D Scene

You can look at the demo shown in the video by going to this link, letting it load and then clicking “Start”.

For an in-depth overview of how three.js labels elements in a scene, see this page.

Light and Rendering

Here’s a rough back of the envelope computation of photons per second.

Answer for How Many Computers?

Here’s a rough back of the envelope computation of photons per second for a light.

The Teapot

The demos shown can be run in your browser: teapot, teaspoon, and teacup. If WebGL is not working for you, read this.

There are good articles about the history of the teapot by Frank Crow, S.J. Baker, and on Wikipedia. There are a number of iconic models and images in computer graphics. Some famous models can be found here and here; my own teapot code is still available. Teapots still rule over all, with their own fan club and teapot sightings page.

The teapot sketch is courtesy of Martin Newell, who is working to put it onto Wikimedia Commons. The teapotahedron image is courtesy of Erin Shaw. The teapot photos are from here and here on Wikimedia Commons.

Question for Simple Materials

Photos from Wikimedia Commons: shiny ball, glass ball and light bulb.

A Jog Down the Pipeline

A book I helped write covers the pipeline in depth, and the chapter is free online here.

For an even more thorough walk down the pipeline, read this set of articles.

Pipeline Parallelism

I should note that what I’m describing is a typical desktop or laptop computer’s GPU. Portable devices such as smart phones and tablets will usually use tile-based rendering instead.

There’s a brief explanation of this algorithm here.

The good news is that even this type of architecture can still be controlled by WebGL.

Answer for Advanced Box Factory

I should note that what I’m describing is a typical desktop or laptop computer’s GPU. Portable devices such as smart phones and tablets will usually use tile-based rendering instead. There’s a brief explanation of this algorithm here. The good news is that even this type of architecture can still be controlled by WebGL.

WebGL and Three.js

The three.js homepage is well worth exploring.

I interviewed the creator of three.js - I found his background inspiring.

WebGL’s homepage is here.

To track the latest WebGL eye candy and other news, see this and this site. Three.js also has a subreddit.

On Windows the Chrome browser actually converts WebGL calls to DirectX; read about it in this free chapter from the book OpenGL Insights.

Problem Set 1

Question: What is (not) Paintable?

Image credits at Wikimedia Commons: Isometric, spheres, rings, and star.

Question: Firefly

Firefly image from Wikimedia Commons.

Question: Light Field Dimensions

Another way to say it: how many dimensions are needed if you want to capture the radiance from any location and direction in the universe?

USMC and lighthouse from Wikimedia Commons.

Answer: Light Field Dimensions

The Light Field is described here. For more on computational photography, Ramesh Raskar and Jack Tumblin’s site is a good place to start.

Drawing from Wikimedia Commons.

Interview Jesse Harrington Au

First part (headshot)

The teapot model is viewable (in WebGL and other forms) at the Sculpteo site.

Some of the service firms for making 3D prints for consumers include: Shapeways, Sculpteo, i.materialise, and Ponoko.

Second part (at the Gallery)

This video gives a virtual tour of the The Gallery at Autodesk. You can also visit for free if you’re ever in San Francisco.

Third part (how 3D printing works)

Wikipedia has a good overview of 3D printing.

Seventh part (design process)

There are at least two worthwhile free design programs with 3D printing in mind: 123D Design and Tinkercad. If you'd rather make models by specifying things in a text file: OpenSCAD is good.

A good, free program for turning a set of photos into a 3D model: 123D Catch.

These products and much else is covered in depth in the Make Ultimate 3D printing guide. Highly recommended if you’re thinking of getting a home 3D printer.

Eight part (the future)

For a general background about 3D printing, the new book Fabricated is inspiring and full of ideas.

Unit 2

Cartesian Coordinates

You can read more about the Cartesian coordinate system on Wikipedia. For a fuller mathematical presentation of the material in this unit, see Gortler’s “Foundations of 3D Computer Graphics”.

Coordinate system image if from Wikimedia Commons.

Left-handed vs. Right-handed

This is a long-standing question, whether to use left-handed or right-handed coordinates and when. Here is some of the ancient debate.

Question: The Sun Also Rises

My little hobby project is 3D printing Minecraft models.

Points and Lines

The Battlefield 3 screenshot is courtesy of Duncan Harris at Dead End Thrills.

Triangulation and Tessellation

If you can spell “frustum” and “tessellation” correctly, you’ve avoided about 95% of the spelling errors you can make in the field of computer graphics; see this article.

The mosaic photo and Biosphere photo are from Wikimedia Commons.

Return of the Square

Culling is on by default, in other words "side" is set to THREE.FrontSide. The other settings are THREE.DoubleSide and THREE.BackSide.

Model Creation

See the Loader class and the various model loaders available for three.js. Webgl-loader convert models for fast loading into WebGL web pages.

Various ways to load geometry into WebGL are discussed in Chapter 2 of the “WebGL Beginner’s Guide” and Chapter 7 in “WebGL: Up and Running”. I recommend the Kindle version of each; see my review.

As a Udacity student, you can register for free use of Autodesk software. When you register, note that Udacity is considered to be in Sunnyvale, CA for purposes of registration.

Conclusion

You can run the demo shown.

Problem Set 2

Exercise Question: Build a Stairway

My sphereflake generator and other procedural model makers are available for download. L-Systems are a form of procedural modeling that is particularly suited for making organic forms. For more resources, see the Wikipedia page on procedural modeling.

Exercise Question: Drinking Bird

See this web page for three.js documentation. The cube, sphere, and cylinder methods are listed towards the bottom left.

There are many other geometric objects supported by three.js. See Lee Stemkoski’s demos for shapes, extrusion, and 3D text. There are also demos for subdivision surfaces, a way to smooth an initial mesh, and CSG, a way to subtract one object from another.

Learn more about the drinking bird here and here.

Exercise Code: Drinking Bird

Formal plan drawings are available on the Course Notes page, along with photos of the original drinking bird.

If you toggle on "Show axes", the axes shown are X == red, Y == green, Z == blue.

See this web page for three.js documentation. The cube, sphere, and cylinder methods are listed towards the bottom left.

There are many other geometric objects supported by three.js. See Lee Stemkoski’s demos for shapes, extrusion, and 3D text. There are also demos for subdivision surfaces, a way to smooth an initial mesh, and CSG, a way to subtract one object from another.

Learn more about the drinking bird here and here.

Exercise Answer: Drinking Bird

As a Udacity student, you can register for free use of Autodesk software. When you register, note that Udacity is considered to be in Sunnyvale, CA for purposes of registration.

Unit 3

Overview of the Lesson

You can try out the demo in the video.

Color Definition

For more about wavelengths of light, the visible spectrum, and color in general, see Wikipedia’s articles here and here.

Articles on Wikipedia also cover additive color and subtractive color, as well as CMYK.

Setting Colors

Look at the Color class documentation for three.js for further help. If you don’t know about hexadecimal, the Wikipedia page is a good place to start. There are plenty of decimal to hexadecimal converters online. Here’s one, for example.

RGB Color Demo

This demo may not run for you. It is a known problem that certain GPUs will run out of shader instructions. This color mixing applet is a reasonable alternative. It lets you vary the intensity of each color contribution and see the effect, and shows both additive and subtractive color mixing.

You can also play with additive colored lights in a WebGL application here from the book “WebGL Beginner’s Guide”.

Gouraud Shading

Mach Bands are described further on Wikipedia.

Introduction to Transparency

Try the jellyfish demo by Aleksandr Rodic yourself. If it runs too slowly, Aleksandr has a more optimized version here.

Another good use of transparency is the Zygote Body demo. There is a vertical slider on the left side of the screen - zoom in and move it up and down slowly.

Wikimedia Commons credits: refracted pencil, crystal ball, ashtray, and 3D glasses.

The Over Operator

A blending demo that shows how the various different blend modes interact is available, from the book “WebGL Beginner’s Guide. Three.js comes with a demo that examines many of the possible combinations - it’s a bit cryptic, but you can modify things with the controls on the left.

Transparency and three.js

See Lee Stemkoski’s tutorial code for more examples of how to set transparent materials.

Advanced Transparency Methods

You can read more about depth peeling here. This stochastic transparency research article discusses this technique, and also provides an overview of previous solutions. Ordering triangles in a mesh to render in proper draw order regardless of view is discussed in this paper.

There is a depth-peeling demo that runs in WebGL, along with technical details.

Future of Transparency

An excellent overview of transparency techniques can be found here.

Some notes from Aleksandr Rodic on the jellyfish:

The transparency was one of the biggest challenges I had with the demo. Actually, that was the first time I got introduced with transparency sorting problem. What I did was a dirty hack. I pre-sorted all triangles along Z axis, then I twisted the skeleton so each joint looks at the camera facing Z.

Here is a quick video showing the modeling process. The purpose of the whole demo was a video installation in Hong Kong. The challenge there was to sync the simulation across 4 computers and make it look seamless. I did that by running first stage of the sim on node.js and streaming that to browsers. Here is a demo of that.

Problem Set 3

Question for RGB Representation

Look at the Color class documentation for three.js for further help. There are plenty of decimal to hexadecimal converters online. Here’s one, for example.

Question for Shiny Bird

You can find the three.js documentation here. You will see the MeshPhongMaterial in the left column.

Question for Paper Lantern Shading

The paper lantern image is from Wikimedia Commons.

Answer for Glass Cube

You can read more about this transparency problem and many others in this article, “It’s Really Not a Rendering Bug, You See...”, which is where I got the idea for this problem.

Unit 4

Lesson: Overview

The demo running in this lesson is here.

Vector Operation

You can find the Vector3 documentation here.

Translation

Oops, I wrote "160 + 390 = 540", it should equal "550" in most universes.

Demo: Euler Angles

There’s a lot more about Euler angles on Wikipedia. If you want to play with Euler angles more and see how they affect various objects, you might try the three.js interactive editor. The Euler quote is from here.

Rotate, then Scale

The three.js interactive scene editor is great for experimentation. First define a directional light, then add an object to the scene, then try changing the translation, rotation, and scale factors.

Lesson: Object3D

The TRS-80 is described on Wikipedia, and here.

Lesson: Hierarchy of Objects

"The Hunting of the Snark" is here.

Question: Instancing

See this demo for 5000 instanced spheres on the screen. More in-depth coverage of instancing can be found in this technical article.

Robot Arm

Yes, the upper arm is below the forearm as shown here. Think of it that my T-850 robot’s arm is in construction.

Unit 5: Matrices

Lesson: Matrix Math

To use the debugger you'll have to run the githubbed code on your own. The Chrome debugger is described here, Firefox's Firebug debugger here, Safari here
and then here.

Lesson: Translation Matrix

For more on general matrix multiplication and the basics of linear algebra, see this guide.

An ancient (but interesting) discussion of row-major vs. column major form in computer graphics is here. It also discusses left-handed vs. right-handed, another source of conflict.

Lesson: Cross Product

To understand the computations for the cross product, see Wikipedia. The best paper I know on solving the 0-length vector problem is by Tomas Möller and John F. Hughes, "Efficiently Building a Matrix to Rotate One Vector to Another".

Exercise (Question): Make an Ornament/Caltrop

It is worth noting that you can set an object's Euler rotation angles from a given arbitrary rotation matrix using three.js's Object3D.setEulerFromRotationMatrix method. Doing so has the advantage of letting you then use the position and scale parameters as usual.

The axis/angle transform and its derivation is discussed further here.

Lesson: Frames

You may want to try the Euler angle demonstration program again to see how the X rotation controls the other two rotations, since it is applied last. By changing X, you change the frame of reference for how Y and Z rotation are applied.

Lesson: Correct Normal Transformation

Writing about this problem was about my first article for the internet, back in 1987.

Inverses and transposes are part of the field of linear algebra. If you have a interest in the subject, you might consider taking the Khan Academy's course. For a book specific to computer graphics, I like Farin and Hansford's book "Practical Linear Algebra", as it builds up intuition through numerous figures throughout. For a derivation of why the inverse transpose is used, along with much other useful math, see Lengyel's "Mathematics for 3D Game Programming and Computer Graphics".

Lesson: Congratulations

If you want to continue on your own, I highly recommend downloading three.js from github and playing with the hundred or so demos. Or play with the demos directly and look at them in the debugger. Also worthwhile are the three.js HTML editor and the three.js scene editor - in beta, but still quite usable for experimentation and learning.

For demo-ey goodness, I recommend the WebGL and three.js subreddits, and webgl.com. Also worthwhile is Acko.net’s page.

Problem Set for Unit 5

Answer to Exercise: Cylinder Positioning

If you feel like converting my little tree growing code over, it's here. A popular free CAD program for procedural modeling is http://www.openscad.org/. Many people use it for creating models for 3D printing.

Answer to Problem: Helix

To submit to three.js's Github codebase, see the FAQ. This page answers many other questions about three.js, too.

Question for Problem: Where to Rotate

Put the transforms in the usual right to left order.

There’s a little drawing error, the X and Z axes’ labels should be swapped. This is a right-handed coordinate system (just so there’s no confusion about direction of rotation).

Unit 6: Lights

Lights

The demo run in this lesson is here. It uses a technique called “deferred shading”, which we’ll talk about later.

As a reminder, course code, lesson scripts, and much else is available from links on the wiki. All Instructor Notes are also available in one place. Finally, don’t forget the forums.

Set a Directional Light

Correction: the direction to the light should be -200,200,-400, not 400.

Photons as Particles

I mentioned it before, but a wonderful short video of Richard Feynman talking about electromagnetic waves is worth watching. Even most mirrors are not fully reflective.

Spot Light in three.js

Instructions for contributing to three.js are here.

The three.js scene editor is here.

Deferred Rendering

The three.js distribution has three deferred rendering examples, here, here, and here.

AlteredQualia’s work in this area is great, see his site for more examples.

Shadow Mapping

Here’s an interesting article on the history of the shadow, when children first understand shadows, and more.

Lance William’s original article on this algorithm is, “Casting Curved Shadows on Curved Surfaces”.

Exercise: Shadows in three.js

In this answer video, "light.castShadow" should be "spotlight.castShadow".

Try out the “shadow bias” slider once you have this exercise working. I’ll discuss this value briefly in the next lesson, but it’s a good one to get a feel for by trying out yourself.

Ray Tracing

There are interactive WebGL ray tracers out there, such as this one and this.

Currently my favorite book for learning about ray tracing is “Ray Tracing from the Ground Up”.I gave a talk comparing the strengths and weaknesses of ray tracing and rasterization here; related links are at my own site. I should mention I edited an informal journal about ray tracing for 20-odd years.

Ray Tracing History and Future

The 256 byte BAZE demo mentioned can be downloaded from here and seen in action here. This demo is a part of the Demoscene - as it turns out, the author of three.js was also involved with and inspired by the demoscene.

Path Tracing

Try out the WebGL path tracer by Evan Wallace.

Wikipedia has articles on path tracing and photon mapping, thouse these tend to be a little too technical for my tastes.

A link to the Chunky renderer and some sample renderings can be found here. If you play Minecraft, try it yourself on your own world!

Fog

Try the demo at the start of the lesson yourself.

Three.js has two types of fog built in, Fog, which falls off linearly, and FogExp2, which falls off exponentially.

For the godrays demo, see the comments at the top of the shader code.

Problem Set for Unit 6

Exercise: Swivel and Tilt Light Control

The altitude-azimuth diagram is from Wikipedia.

Solution: Swivel and Tilt Light Control

I goofed here: the solution should have a square root to compute the length left. My thanks go to Mikel Ortega for noting this problem. The solution should read:

var length = Math.sqrt(1 - light.position.y*light.position.y);

See this thread for more information.

Unit 7: Cameras

Introduction to Cameras

The fisheye lens, http://commons.wikimedia.org/wiki/File:Swampy_fisheye_lens_(crop).jpg, perspective, and the elevation drawing are from Wikimedia Commons.

Orthographic Camera

See the Wikipedia article on graphical projections for more on this subject, including a wide range of other architectural projections.

Images from Wikimedia Commons: The Umbria Plantation and hexayurt shelter images are from Wikimedia Commons.

Perspective Camera

One clever form of illusion is based on what’s called forced perspective. See this video, for example. People have played tricks with perspective for hundreds of years, such as this drawing from the 18th century.

Exercise: FOV Slider

Check previous demo code or the dat.gui documentation for how to create a slider.

Exercise: Perspective Coordinate Transform

Did you know that you can type mathematical expressions into Google’s search box and it will compute the answer for you? Very handy.

Homogeneous Coordinates

For more on homogeneous coordinates see Wikipedia.

Demo: Dolly and Zoom

Here’s more about the dolly-zoom technique and its use in films.

Dolly, Pan, Orbit

Panning means something a bit different for cinematography: http://en.wikipedia.org/wiki/Panning_(camera)

Circle strafing is explained here: http://en.wikipedia.org/wiki/Strafing_(gaming)#Circle_strafing). Wikipedia has just about everything.

Near and Far Clipping

At 1:45 I say "9.0" when I mean "0.9".

You can see the effect on precision with this z-depth calculator. There’s more about optimizing your z-depth range in this article post.

For more on the stencil buffer, this page is a start. Really, a book such as “the Red Book” or McReynolds & Blythe’s are good for getting a serious education about its uses.

Depth of Field

The three.js depth of field demo is here. AlteredQualia’s tilt-shift demo is here.

Window Coordinates

Lee Stemkoski has commented code for how to put two viewports and four viewports on the screen.

Three.js has an interesting demo showing an artistic viewport effect.

A fairly good overview of the OpenGL transformation pipeline can be found on this page. Although dated, this page gives a more friendly walk down the camera pipeline.

The ancient article, "What are the Coordinates of a Pixel?" by Paul Heckbert, in the book "Graphics Gems", is worth tracking down. It’s my strongest argument for why the center of a pixel or texel has the coordinates 0.5,0.5, not 0.0,0.0.

Antialiasing

See the FXAA algorithm in use in this three.js demo, though you’ll have to disable it yourself to see the difference.

My view of FXAA is here. There was a whole SIGGRAPH course about filtering techniques.

Antialiasing does not work on Macs due to an obscure OpenGL driver bug on NVIDIA chipsets. By the time you read this, it may be fixed.

Conclusion

"Man Drawing a Lute" from Wikimedia Commons.

Problem Set for Unit 7

Graphics Pipeline Coordinates

The idea for this question is from Ronen Barzel, with permission. The slidesets for his graphics course are worth working through.

Unit 8: Textures and Reflections

How Texturing Works

For more about WebGL security issues with textures and how to handle them, see this and this article. To run texture demos locally on your browser, you’ll have to set some permissions or set up a local servers.

Here’s the list of videogames with crates. Another collection of crate screenshots is here.

There are a number of places where you can download free textures, such as Humus’ site, Open Game Arts’ site, TurboSquid, and Autodesk’s AREA site.

Texture UVs

For a commented example of how to texture a box, see Lee Stemkoski’s code.

UVs in three.js

Try the demo shown to see the latest mappings - bugs do get fixed, so it may well be different. If you want to submit fixes to three.js, you can learn about the process here.

To look at the implicit mappings and other texture methods discussed in this unit, definitely try the online editor. Load a directional light, load an object, then apply textures to your heart’s content.

Texture Mapping

The character demo is located here.

It’s worth noting that using mosaicing and putting a large number of objects into a single texture can help avoid changing textures, which can increase performance.

Answer: What Causes Texture Discontinuities?

You can read more about this problem (and many others) in this article, “It’s Really Not a Rendering Bug, You See...”. This article also shows how meshing can help solve it.

There is one case I know where triangles are not sufficient to properly render an object, namely, the cone. There are workarounds, but they’re a bit unnatural.

Wrap Modes

If you want to manipulate the wrap mode separately for each axis, try out the demo here from the book “WebGL Beginner’s Guide”. Another wrap mode example of using repeat is in Lee Stemkoski’s code.

Note that wrap modes in WebGL will only work for textures that are a power of two.

Demo: Wrap Mode

If you change the wrap mode, you need to set “texture.needsUpdate = true;”. See the demo code for an example.

Texture Transform

I give a poor way of setting the repeat and offset variables on the Texture. It's better to use the ".set" method. My way will work, but it's inefficient and could lead to bugs. See this discussion for more on this subtlety.

If you want to know more about advanced color tiling, read about Wang Tiles.

Minecraft now allows all textures to be animated, leading to madness.

Demo: Texture Minification

Try different texture resolutions to see where minification takes place. Change the minification to “nearest” to see a small improvement, but more will need to be done... in the next lesson.

Mipmapping

You can see the effects of different filtering and also which mipmap level is used (by its color) in this demo. Also try this demo to see the effect of various sampling and filtering settings.

Demo: Mipmapping

Change the minification to “mipmap” to see the improvement.

Demo: Anisotropy

You can and should try out this additional anisotropy demo. Depending on your machine’s GPU, the anisotropy demo may not work for you.

Premultiplied Alpha

There are a number of benefits to premultiplied alpha, some of which are discussed in this blog post by Tom Forsyth.

A number of free packages let you work with texture alpha, including GIMP and paint.net. Careful where you click on the paint.net page, it’s a minefield.

Particles and Billboards

Almost all of the demos shown are a part of the three.js distribution. Here is the list:

Simple squares
Circle cutouts
Kinect
Point cloud from Markus Schütz on his potree.org site
Spheres
Snowflakes
Clouds
Lensflare

For more examples of using particles to represent real scenes, try the potree.org demos - the longer you wait, the more particles get loaded.

For commented code that shows how to create and use billboards, see Lee Stemkoski’s code.

Lens flare is described here, and some ways to simulate it on the GPU are here.

Displacement and Normal Mapping

My original lesson plan spent more time on this area. See the Lesson Scripts if you want to see the initial, considerably longer lessons.

The original version of the heads program can be found here. Note that this program may not run on older computers, as it requires Shader Model 3.0 level of GPU support.

Evan Wallace’s lovely water demo in WebGL shows displacement mapping in action.

Ambient Occlusion

The demo that created these images is here. Ambient occlusion has some basic coverage on Wikipedia, as does screen-space ambient occlusion. To see ambient occlusion and screen-space ambient occlusion in use, along with many other effects, see the Agni’s Philosophy page - I recommend clicking the “translate” button.

Time to Explore

The demo shown can be found here - warning, music will play.

Skybox

There is a commented tutorial by Lee Stemkoski on skyboxes. You can also find skybox code with the three.js distribution.

See Humus’ site for a large variety of freely-reusable reflection cube maps and other textures.

For a nice implementation of a surrounding environment, see this demo.

Reflection Mapping

For commented code on how to set up reflection mapping, see Lee Stemkoski’s demo.

See Humus’ site for a large variety of freely-reusable reflection cube maps and other textures.

Demos shown in the video were done with three.js, here and here. Here’s yet another car demo. It’s fun in that you can drive the car around with the arrow keys. One more nicely-done car is here.

Refraction Mapping

Three.js provides a number of refraction mapping demos: spheres, angels, and heads.

Glossy Reflection

A free utility to blur cube maps is provided by AMD. The site has a good runthrough of the effects achievable.

The car videos were made using Showcase, which is a pretty fun program to toy with for styles and materials. Beyond the free trial, as a Udacity student you can register for free use of Autodesk software. When you register, note that Udacity is considered to be in Sunnyvale, CA for purposes of registration.

Diffuse Mapping Example

There’s more about diffuse maps, aka irradiance maps, at this site, along with a demo.

There are ways to extend to irradiance map idea to vary with location, see this seminal paper from Gene Greger et al., and this recent work in “Far Cry 3” for just one application of it. Better yet, there’s a demo in three.js along with a full explanation.

On the Fly Cube Maps

The demo shown is here.

To see how to make a recursive video screen, see Lee Stemkoski’s commented demo code.

Conclusion

The video display demo shown is distributed with three.js. A demo by Lee Stemkoski has commented code to display a video on a surface. You can also see how to capture webcam data in this and this demo of his, and how to warp camera views here.

Problem Set for Unit 8

Exercise: Grassy Plain

The grass texture is a lower-resolution version from Humus’ site, which has many freely-reusable reflection cube maps and other textures.

Exercise: Specular Mapping

To play with this feature and other texture mapping methods discussed in this unit, try the three.js online editor. Load a directional light, load an object, then apply textures to your heart’s content.

As far as searching through the code base goes, I found the free program Agent Ransack invaluable on Windows for looking for various classes and their use. If you know of an even better program on Windows, or a similar program for the Mac or Linux, please post it on the forums.

Problem: ReflectionMapping

By default the reflection map is multiplied by the computed surface. There are other ways to mix the two results using the “combine” parameter, such as mixing and addition. See this demo’s code, for example.

Unit 9: Shader Programming

Programmable Shaders

Here is the OpenGL pipeline in great detail; from the book “OpenGL Insights’.

Question: Fragment Shader Bottleneck

Interestingly, you can visit this web page, which will report on the capabilities of your machine. Within WebGL there are also extensions, proposed new capabilities not supported by all (or sometimes, even any) machines.

Shader Inputs

Look here for a list of uniform types.

In three.js you can use custom attributes with ShaderMaterial. It's a bit awkward but it works, see examples here and here.

GLSL ES

Here are some GLSL (the OpenGL Shading Language) resources: reference site, shading language description, manual pages, data types, and reference guide. The official specification is here.

Vertex Shader Example

See this posting for what matrices are built-in for shaders in three.js. You can also look at code and see the list.

Fragment Shader Example

This nice in-browser GLSL tutorial lets you change lines and immediately see the result. Larger editor view here.

If you use the Sublime Text 2 editor, you may want to get the GLSL syntax add-on. There are a few, this was the best one I found.

There is also a GLSL shader validator I just noticed; I haven’t tried it yet.

Procedural Textures

Here’s another beautiful example program, done by overlaying textures.

Exercise: 3D Procedural Texturing

The classic book “Texturing and Modeling: A Procedural Approach” is still quite relevant, despite its age. If you delve into procedural textures at all, this is the first book to get.

Debugging Shaders

There’s a bit on using Chrome’s Canvas inspection for debugging WebGL itself.

For image examination and capture I recommend the (not free, but worth it) FastStone Capture.

Fresnel Reflectance

See the Wikipedia article on the Fresnel equations for more background. A more in-depth article by Sébastien Lagarde is worth the effort. Our book also covers this topic.

The Schlick approximation to the Fresnel term is often used in interactive rendering, since it’s quick to compute.

One problem you’ll see in the Fresnel demo that follows is that at some angles the specular highlight simply disappears. Read more about this problem in this article, which I’ve pointed at many times so far.

Energy-Balanced Materials

This slide set gives a great runthough of why physically-based materials are worth using.

BRDF

Naty Hoffman has taught extensively on this subject at SIGGRAPH. Here is an early introduction. This slideset and background notes bring things up to date.

Exercise: Anisotropic Material

While the code here is something of a hack, there are real implementations of anisotropy, where you define a direction on the surface for how the material is oriented. See this article, http://content.gpwiki.org/index.php/D3DBook:(Lighting)_Ward, for one classic model.

BSDF and BSSRDF

Now you can handle the truth about diffuse vs. specular.

The Hideo Kojima team has a great video about their latest physically-based-rendering game engine for Metal Gear Solid. Also, compare night and day.

Skin demos can be seen here, which uses a simple wrap-around lighting model and has a more shiny and sharp look, and here, with a softer, more realistic feel, and that uses a more complex multipass texturing approach. There are a few related articles online from the GPU Gems books about skin in particular: this and this from “GPU Gems”, and this newer article in “GPU Gems 3”.

Monitor Response and Perception

The gamma correction article on Wikipedia is pretty good. To see what your device's gamma is, see Cornell's gamma page - it’s usually 2.2. If you try this on a laptop, make sure to look at the screen face on. Many laptop displays vary considerably with view angle.

For an excellent comparison of gamma-corrected edges, see this properly corrected image vs. this uncorrected one.

Here’s a compare of lighting of a single light and how the falloff is poor without gamma correction. Note the sharp edge for the lighting falloff in the upper image.

Gamma Correction

Three.js does not properly correct texture colors as they’re read in, but rather on sampling the uncorrected mipmap levels. This leads to mipmaps still getting darker as higher mip levels are used.

A quick visual rundown of the effect of gamma correction, and much else having to do with physically-based materials, is in this worthwhile slide set. My favorite article on gamma correction is from the free and wonderful book GPU Gems 3. See our online books page for more free books. Other good articles are here and here, among many others, such as Wikipedia’s coverage.

There's a bit more to monitor response and gamma correction, see this article on sRGB for what the industry aims for. See our blog for way more on sRGB and related monitor and color issues. WebGL has a proposed (but unimplemented) sRGB extension, more here.

Gamma Correction Demo

This demo shows the effect of gamma correction on output. Light colors are not adjusted on input (renderer.gammaInput = false;). Note how the overlapping area is comparatively brighter to the individual lights when gamma correction is turned off.

Texturing and Post-Processing

A much more in-depth tutorial is here, along with the demo we used. Lee Stemkoski has related commented tutorial programs here, here, and render to texture here.

Another good image manipulation demo in WebGL is here. Image processing can do great things, like this.

For a peek at what graphics hardware can do to perform image processing, try out this demo, http://voxelent.com/html/beginners-guide/1727_10/ch10_PostProcessing.html, from the book The “ShaderX^2: Tips & Tricks” book is quite old, but free to download. It has some excellent, comprehensive articles on image processing on the GPU.

The “WebGL Beginner’s Guide” also has a post-processing demo.

The difference between luma and luminance is discussed here.

Conclusion

Extremely inspirational shaders can be found at shadertoy, GLSL sandbox, and Iñigo Quílez site.

For some idea where the state of the art is today, see these talks from the “Advances in Real-Time Rendering” course from SIGGRAPH. Better yet, attend SIGGRAPH! Volunteers are welcome.

An excellent source of information about the internals and optimizing for the GPU is the Beyond Programmable Shading course at SIGGRAPH. Despite the name, it has much material directly relevant to shader programming.

Problem Set for Unit 9

Exercise: Make a Moving Flashlight

Here are some GLSL (the OpenGL Shading Language) resources: reference site, shading language description, manual pages, data types, and reference guide.

Exercise: Vertex Normal Creation

Lee Stemkoski has a nice tutorial program allowing you to put in a function and have the mesh generated for it. This is not vertex-shader-based, but is worth a look.

Worth a second look, Evan Wallace’s water ripples demo in WebGL shows normals generated on the fly in a great way.

Exercise: Sharp Specular

Thresholding is described on Wikipedia: http://en.wikipedia.org/wiki/Thresholding_(image_processing)

See the Beyond Programmable Shading course notes for more about optimizing code for GPUs.

Exercise: Wrap Lighting

The article that talks about wrap lighting is Real-Time Approximations to Subsurface Scattering.

Unit 10: Animation

Introduction

The demos shown are by Max Ogden and James Halliday, (also see the related blog), and Daniel Hede.

In case you haven’t looked at the documentation, the dat.GUI library is how we’ve been adding sliders to programs.

This tutorial by Lee Stemkoski shows how to set a camera to follow behind a moving object; use WASDQE to move. Switching between cameras is shown in this demo, hit the 1 and 2 key.

Events

See this page for a good run through of events.

For examples of the various events and how to use them, I recommend looking through the three.js code examples themselves, direct download. Search the HTML code examples for “addEventListener”. For mass file searching on Windows, I like the free Agent Ransack utility.

Demo: Picking

This demo is based on a sample from three.js. Note that the sample works without using WebGL at all! Three.js has limited support for non-WebGL 3D rendering.

Picking

Note: in this lesson I set only the X and Y values when creating the mouseVector Vector3. This is valid in JavaScript, the third value defaults to 0. In retrospect it would have been clearer to simply put the 0.

For a more thorough tutorial on picking in three.js see this page.

Look at the TrackballControllers.js code in three.js if you want to see how touch events are handled.

For a picking demo in WebGL, see this code from the book “WebGL Beginner’s Guide”.

One method I don’t discuss in this lesson is the idea of first rendering each object with a unique color. This “ID image” can then be accessed and the color used to find what object is visible at a given pixel.

The Rendering Loop

See this article for a bit more about requestAnimationFrame.

Incremental Animation

Adding new parameters to an object can also hurt the performance of accessing properties of that object, see this presentation.

Motion Capture

Wikipedia’s article on motion capture is a reasonable summary. Photo used in lesson is from Wikimedia Commons.

The wonderful head-tracker demo in three.js is at Jerome Etienne’s site. Try it!

There are motion capture databases out there for experimentation, such as this and this.

Keyframing

This website has a good explanation of tween.js and a worthwhile demo of easing. More about Blender export here. Here’s a tutorial of another tween library, Greensock.

Wikipedia’s overview of keyframing is a reasonable start, though is mostly concerned with 2D content. The article on interpolation for computer graphics is bare-bones, but talks about some basics. Much more can be found in the Essential Math for Games Programmers course notes.

ThreeFab lets you animate scenes, it was used for the lamp animation. It’s an alpha, and looks inactive, but has some interesting features. Read more about how to export from Blender and use ThreeFab. Try it here.

This demo shows an assembly animation done in three.js.

Demo: Three.js Tween Library

I encourage you to examine my example to see how to use the tweening library. The three.js tweening library is described in this tutorial. Try this demo to see the effects of the many easing curves supported.

Texture Animation

The animated texture tutorial program is by Lee Stemkoski. He has many other commented tutorials available.

Just in case you don’t know what a flipbook is, read this and then make your own.

Solution: Flowing River

Here’s the code snippet to copy and paste to see my more artistic solution:

texture[effectController.mtlName].offset.set( 0.2Math.sin(2time), time );
texture[effectController.mtlName].repeat.set( effectController.repeat, effectController.repeat/3 );

Quaterion Interpolation

Wikipedia discusses slerping a bit if you want to know a little more.

This presentation on rotations gives a thorough run through of rotation interpolation and quaternions.

Skinning

The Team Fortress 2 demo is here. Another skinning demo is here.

Morphing

The Ginger demo shown in this lesson gives a great demo of the power of morph targets. Further technical information is here.

Lee Stemkoski has a morph animation code tutorial - use the arrow keys. Three.js has a few relevant demos: morphing between animations and skinning and morphing.

A bit more about morph targets can be found on Wikipedia.

See the Loader class and the various model loaders available for three.js. Webgl-loader convert models for fast loading into WebGL web pages. There’s more information on the three.js FAQ page.

Various ways to load geometry into WebGL are discussed in Chapter 2 of the “WebGL Beginner’s Guide” and Chapter 7 in “WebGL: Up and Running”. I recommend the Kindle version of each; see my review.

Register as a student or teacher for free use of Autodesk software. When you register, note that Udacity is considered to be in Sunnyvale, CA for purposes of registration.

Previsualization

The full interview clip is here. There is an extensive interview with Chris Edwards about previsualization here, here, and here.

Particle Systems

There is a tutorial here about creating and animating particles.

For a code sample showing how to create and animate a simple particle system, see Lee Stemkoski’s tutorial.

Demos used in this lesson:
Varying sizes
Particle people
Firebolt heart
150K particles

Jerome Etienne has a quick page of code showing an animated puff of smoke.

This seminal paper on particles is quite readable and goes over the Genesis sequence from Star Trek 2.

See the video linked from this page for some modern, involved particle effects and much more.

Collision Detection and Response

Stefan Hedman has a fun little collision detection and constraint solver library that works with three.js, and that were used in this lesson. There’s also another nice physics library for three.js, Physijs, with great demos.

Lee Stemkoski has code for detecting the intersection of a block with other blocks http://stemkoski.github.com/Three.js/Collision-Detection.html. Use the arrow keys and WASD to move the block around.

There’s a short tutorial on using ray casting for basic character collision detection.

I maintain an object/object intersection table with pointers to algorithms and books.

Simulation

The physical simulations shown were done in GlowScript. The car was made with Schteppe's physics library

The Physics-Based Animation site tracks the latest research in the field.

A wide range of presentations about physics and collision detection at interactive rates was part of the Game Developers Conference in 2012.

Simulation Examples

The videos in this lesson were made by Jos Stam and others working on the Nucleus dynamics solver. You can read about the main idea behind Nucleus here.

Guided Simulation

The final render of the cloth simulation was by the 3DS MAX team under Neil Hazzard.

For cloth simulation, these course notes by Witkin and Baraff are worth your time.

Level of Detail

The demo used is here.

A performance tip: you can turn on an FPS meter in Chrome to any app, from the debugger.

Texture levels and level of detail are discussed in detail in the books “Level of Detail for 3D Graphics” and “3D Engine Design for Virtual Globes”, among others. A free related resource is "Virtual Texturing in Software and Hardware" SIGGRAPH course from 2012.

Quadric Error Metrics is the most common algorithm today for mesh simplification. MeshLab is a free package that implements mesh simplification algorithms and much else.

Terrain LOD System

This is the terrain demo site, wonderfully instructive. The demo itself is here.

This bit of video talks about the advantages of the efficient BufferGeometry class in three.js.

There is another form of culling called occlusion culling. Algorithms based on occlusion culling perform tests to rapidly determine if an object is fully hidden by other objects in the scene. Two articles which discuss GPU-based occlusion culling are available in the free GPU Gems series of books. The articles are here and here. There are many other occlusion techniques, such as this one for globes.

This is not the End

The short demo of Sponza by Florian Boesch shows irradiance volumes, the long demo is from the wonderful ro.me interactive film.

Now what? Plenty. I track and host various resources at our book’s site - dig around. I’ve certainly been blogging about WebGL and three.js.

It’s worth visiting this wonderful “Why WebGL?” page for its clever header and the demos. This math visualization page at the same site is lovely.

New demos? Visit webgl.com and learningwebgl.com. This second site also has 16 WebGL lessons.

Mr. Doob & AlteredQualia give a worthwhile talk about three.js, past and future.

Here’s three.js’s development over a year, in just two minutes. If you want to see your name shoot lasers at three.js, contribute!

Finally, I’m the guy in front of the camera and above the table, but there are many people who worked as hard or harder. First and foremost Gundega Dekena, as well as the video editing team and all Udacity in general, for all their hard work and friendly attitude throughout this arduous and exciting process. Mauricio Vives, Patrick Cozzi, and near the end Branislav Ulicny, aka Altered Qualia, for the truly massive amounts of help in technical reviewing all this material - it honestly wouldn’t be half as good without them. Particular thanks to Ricardo Cabello, aka mr.doob, the creator and manager of three.js, for quick and helpful replies to my questions. Beyond those interviewed, I received huge amounts of help from Neil Hazzard, Qilin Ren, Yong Guo, Jos Stam, and dozens more inside Autodesk. Many thanks to all those who let me use their fantastic demos and images in these videos; every person I asked was more than happy to give me permission and answer questions, which is one thing that makes the graphics community great. Finally, thanks to Carl Bass for getting me involved in the first place, Brian Mathews for all the support, and my manager Jean-Luc Corenthin, the rest of my team, and especially my wife Cathy for their patience and understanding.

Final

Question: Dot and Cross Products

The idea for this question is from Ronen Barzel, with permission. The slidesets for his graphics course are worth working through http://graphics.ucsd.edu/courses/cse167_w06/

Question: Volume Lighting

The image is from the paper "The irradiance volume", by Greger et al., Program of Computer Graphics, Cornell University. This technique has been used in a number of game engines, see here, here, and here.

Answer: Perspective vs. Orthographic

There is also reverse perspective, which has the peculiar effect of making more distant objects larger.

Answer: Brightest Illumination

The idea for this question is from Ronen Barzel, with permission. The slidesets for his graphics course are worth working through if you’d like to learn more than the basics I’ve covered.