Archive for June, 2010

Graphics Trick: Nonlinear transformations

Thursday, June 24th, 2010

Short one today, just a note on nonlinear transformations. The usual translations, rotations and scaling transformations can all be described as a linear transformation (= matrix multiply). Even linear blend skinning, which blends between per-bone linear transformations is effectively piecewise linear. That basically means that each of x,y,z and w after transformation depend only on linear terms of x,y,z and w before transformation:

p1.x = a p0.x + b p0.y + c p0.z + d p0.w

Anything that includes higher powers of x,y,z or w, or some non-linear function is a nonlinear transformation:

p1.x = ap0.x p0.x + ap0.x + b p0.y/p0.w + c sin(p0.z)

I haven’t seen nonlinear transformations used too much in real-time graphics. There have been a few non-linear extensions to the linear blend skinning and some approaches for generating non-linear environment maps (e.g. paraboloid maps). They were popular for a time for free form deformation for production animation.

None the less, understanding what happens to points, tangents and normals helps explain where some of the well-known rules of graphics originate.

First, assume we’re transforming p0 to p1 by some function f:

\begin{align*}
p_1 & = f(p_0) \\
& = \begin{pmatrix}
f^x(p_0) \\
f^y(p_0)\\
f^z(p_0)
\end{pmatrix}
\end{align*}

According to the rules of differential geometry, surface tangent vectors should transform by the Jacobian of the transform. The Jacobian is a matrix of partial derivatives (which I’ll indicate by subscripts, making my choice to label the components with superscripts above make a little more sense)

J_f=\begin{pmatrix}
{f^x}_x(p_0) _y(p_0) _z(p_0) \\
{f^y}_x(p_0) _y(p_0) _z(p_0) \\
{f^z}_x(p_0) _y(p_0) _z(p_0) \\
\end{pmatrix}

It depends on where p is, but it’s still just a matrix, so this is just a matrix multiply. I like to write tangent vectors as columns, so the multiply would look like this

t_1 = J_f \cdot t_0

Also, according to the rules of differential geometry, surface normal vectors should transform by the inverse Jacobian. I like to write normal vectors as rows, in which case the multiply would look like this:

n_1 = n_0 \cdot J_f^{-1}

One property of this is that the dot product of a normal and tangent are not affected by the transformation from one space to another (which is good):

dot(n_1,t_1) = n_1 \cdot t_1 = n_0 \cdot J_f^{-1} \cdot J_f \cdot t_0 = n_0 \cdot t_0 = dot(n_0,t_0)

What does this have to do with ordinary linear transforms? Assuming no perspective, a linear transform looks something like this:

p_1 = \begin{pmatrix}
a & b & c & T.x \\
d & e & f & T.y \\
g & h & i & T.z \\
0 & 0 & 0 & 1
\end{pmatrix} \cdot \begin{pmatrix}
p_0.x \\
p_0.y \\
p_0.z \\
1
\end{pmatrix} = \begin{pmatrix}
a\ p_0.x + b\ p_0.y + c\ p_0.z + T.x\\
d\ p_0.x + e\ p_0.y + f\ p_0.z + T.y\\
g\ p_0.x + h\ p_0.y + i\ p_0.z + T.z\\
1
\end{pmatrix}

Dropping the constant row, the Jacobian of that guy is:

J_f = \begin{pmatrix}
a & b & c \\
d & e & f \\
g & h & i \\
\end{pmatrix}

So that’s where “transform vectors by the upper left 3×3 corner of the transformation matrix” comes from. Similarly, we can understand transforming normals by the inverse transpose: inverse because it’s the inverse Jacobian, and transpose so you can multiply as a column on the right rather than a row on the left.

But… now we know how to do other nonlinear transforms, including what to do if this matrix actually has some perspective in it. Build a Jacobian matrix.

[Originally, this post attempted to use html for the math, which sucked. Updated to use LaTeX + MathURL]

Graphics Trick: Distributing stuff

Thursday, June 17th, 2010

Sometimes you want to uniformly distribute some objects over an area. Maybe they’re trees, maybe just blades of grass, maybe an army of guys. Whatever they are, there are a bunch of ways you could try. Here are a few.

These differ in the distribution of points and ease of generation. For distribution, the important characteristics are whether it’s clumpy or evenly spaced, as well whether it’s regular or more random. Ease of generation depends a lot on how you’re going to use it. If you can precompute positions, the complexity doesn’t matter too much. If computed at load time, the complexity might matter a little. However, sometimes you want to be able to quickly answer questions about the distribution without necessarily knowing the location of all the points. That could be “where is point n” (useful for shader-based instancing), or “are there any points near here” (useful for generating partial layouts, or proximity queries).

Easiest (and ugliest) is the uniform grid. If these were tree positions, they might do for an orchard, but would look really strange for a natural forest. You can easily find point n without placing all of the other points, and just as easily find nearby points by searching the neighboring grid cells.

We could do a little better on the layout with a hexagonal grid, since that’s the natural tightest packing on the plane. The points lie in little equilateral triangles. To keep the spacing even, we need to squish the vertical spacing by the height of an equilateral triangle, sqrt(3/4) ≈ .866. To keep about the same density, it comes out to s=sqrt(sqrt(3/4)) fewer rows in y and 1/s more columns in x. The closest integer layout near 16×16 is 15×17, so we have one fewer point. Since this is basically a skewed grid, it is still easy to find point n independently of the other points, and also not too hard to find nearby points. However, it’s still pretty regular, with all the points in nice neat rows.

The last of the deterministic layouts is a Hammersley sequence, one example of a quasi-random sequence. For Hammersley points, the x is just the point number, and y is the point number with the bits reversed. There are definitely still patterns, but no long rows of points. Once again, it is easy to find the location for point n, but much harder to find nearby points. They’ll be near in sequence to n, which guarantees their x is close, but some of those points will have vastly different y. You can play some bit tricks to reduce the search set, or just sweep through the points nearby in x.

The rest of the examples are all different flavors of random distributions. The first chooses each x and y as independent uniformly distributed random numbers (= calls to random() or rand() or drand48() or whatever your random number generator you happen to be using). Since the points are each independent, in theory point n shouldn’t depend on any of the other points. In practice, random number generators keep some state and return numbers in a sequence. That means for most random number generators, you’ll have to run through points 0 to n-1 before you can get the location of point n. There are stateless random number generators (see our High Performance Graphics paper for example), which let you break that artifical dependence and get the points in any order. Nearby points are hard to find without some auxillary data structure though, since any point in the list may be near point n. Also, the independence introduces some heavy clumping, which is usually not good.

The jittered one avoids some of the clumping by going back to the grid. Each point is placed in a random location within its grid cell, but every grid cell has a point and only one point appears in any grid cell. With a stateless random number generator, you can place point n without dependence on the other points. However, now proximity tests are once again possible by looking at the positions in nearby cells. Since the points may appear anywhere in their cell, you need to look at all cells that intersect the search radius. Jittered grids do have some clumping though, since “anywhere in the grid cell” could include four points all landing near the same corner.

The last example is a Poisson disk distribution. This is a random distribution where each point is no closer than a given radius to any other point. There are gobs of algorithms for generating these, though all that I know of require pre-generating the entire distribution. This particular plot was created with a dart-throwing algorithm, where you choose a random location, toss it out if it is too close to any previously placed points, then try again. Dart throwing is about the least efficient way to generate a Poisson distribution, but it’s easy to find other more efficient algorithms. You usually need to create some spatial data structure to help find nearby points, so if you keep that structure around, proximity queries are not too bad. Actually, jittered grids were first introduced to graphics as an approximation to the Poisson disk distribution, though the real thing is clearly much more evenly distributed.

If you’re curious, all of these were created with a point-placement vertex shader. Random numbers were two iterations of TEA (the Tiny Encryption Algorithm, see the above paper) on the point index. The blue component of the color is the point index, while red and green are randomly chosen based on the index.

Akisakio is on the App Store!

Wednesday, June 16th, 2010

Way to go, TeamSuperCool! Another UMBC student game, and they’ve got 60 reviews!

I’ve got it– of course. Akisakio is usually the first thing my kids run when I leave my phone within reach.

Graphics Trick: Homogeneous fun

Thursday, June 10th, 2010

Anyone who’s done any 3D graphics knows how to use homogeneous coordinates for translation and perspective, but they’re good for other places where you’d rather put off a division or avoid it entirely.

Basic homogeneous coordinates use four coordinates (I’ll use lower case to keep things straight) to represent a 3D point (here in upper case)

p = [x y z w]
P = [X  Y  Z] = [x/w  y/w  z/w]

Or more compactly, in shader notation

P = p.xyz / p.w

For regular points, we use w=1. If w=1/2, we get a point twice as far out in the p.xyz direction. If it’s 1/3, we get a point three times as far out in the p.xyz direction. In the limit, as w goes to 0, we get a point infinitely far away in the p.xyz direction.There’s no need to worry about a pesky division by 0, as long as we don’t ever actually have to do the division for the evil w’s.

That’s the basic idea behind my homogeneous rasterization paper (sometimes called clipless rasterization), used in quite a few GPUs. You normally compute the intersection between each triangle edge and a clipping plane to throw away the parts behind you where w might be 0 or negative. If you put off the division and keep everything in homogeneous coordinates long enough, eventually you can get all the way to visible pixels on the screen, with w’s guaranteed to be positive, before you actually have to divide.

But that’s not what this graphics trick is about. What else can we do using homogeneous coordinates to avoid division? Let’s say we want to subtract two points

V = Q – P = q.xyz/q.w – p.xyz/p.w

I can avoid dividing by putting that over a common denominator

V = (q.xyz * p.w  –  p.xyz * q.w) / (q.w*p.w)
v.xyz = q.xyz*p.w – p.xyz*q.w;  v.w = q.w*p.w

That lets either p or q be infinitely far away. So if P is a point on a surface, and Q is the location of a light, then V is a vector from the surface to the light (I’d use the traditional L, but lower-case l and number 1 tend to look a bit too similar). This will handle both point and directional lights with the same code. Of course, if you know a particular w is always 1 or always 0, you’d factor that constant into the equation in your code. Assuming p.w=1 to make things a little easier, for point lights (q.w=1), that reduces to a vector subtraction between the surface and light location

v.xyz = q.xyz – p.xyz;  v.w = 1

For directional lights, which are infinitely far away in a given direction (often used to approximate the sun), q.w=0, and we see the exact surface position doesn’t matter

v.xyz = q.xyz;  v.w=0

Also, though less common, you can use the same methods do parallel and perspective projection with the same code.

So that puts off the division and keeps everything in homogeneous coordinates, but in many lighting computations, you want a unit length vector pointing in the given direction

normalize(V) = V/length(V)

When you normalize, any scale factor that affects all three components has no effect on the final direction. If V is twice as long, or three times as long, or infinitely long, it still points the same way. In particular, as long as we avoid negative w’s, the division by v.w doesn’t affect the final direction, even if v.w=0.

normalize(v.xyz/v.w) = normalize(v.xyz) = normalize(q.xyz * p.w  –  p.xyz * q.w)

It’s amazing how far you can get with two simple rules: if the division might blow up, put it off. If a constant scaling won’t affect the result, don’t do it.

Marc

Graphics Trick: Polygon Area

Thursday, June 3rd, 2010

This time, how about polygon area (plus circular loops).

The area of a 2D triangle is easy, it’s just half the magnitude of the cross product of two edges where you assume the z component is 0:

|cross((v1−v0),(v2−v0))|

The sign of the cross product tells you if the triangle is front or back facing (counter clockwise or clockwise). If you multiply out the cross product, it has an interesting repeating pattern if you collect it in terms with pairs of indices i and (i+1)%3

(v1.x−v0.x)*(v2.y−v0.y) − (v1.y−v0.y)*(v2.x−v0.x)
= v0.x*v1.y − v1.x*v0.y  +  v1.x*v2.y − v2.x*v1.y  +  v2.x*v0.y − v0.x*v2.y

As a bonus, for 3D triangles, the cross product points in the direction normal to the triangle, and its length is twice the area of the triangle. What’s more, the cyclical pattern appears again for each component (the order and signs here make more sense if you consider x,y,z to have their own mod cycle as well: x is followed by y is followed by z is followed by x …)

n.x = v0.y*v1.z − v1.y*v0.z  +  v1.y*v2.z − v2.y*v1.z  +  v2.y*v0.z − v0.y*v2.z
n.y = v0.z*v1.x − v1.z*v0.x  +  v1.z*v2.x − v2.z*v1.x  +  v2.z*v0.x − v0.z*v2.x
n.z = v0.x*v1.y − v1.x*v0.y  +  v1.x*v2.y − v2.x*v1.y  +  v2.x*v0.y − v0.x*v2.y

As a loop

n = float3(0,0,0);
for(int i0=0; i0<3; ++i0) {

int i1 = (i0+1)%3;

n.x += v[i0].y*v[i1].z − v[i1].y*v[i0].z;
n.y += v[i0].z*v[i1].x − v[i1].z*v[i0].x;
n.z += v[i0].x*v[i1].y − v[i1].x*v[i0].y;

}

Maybe overkill for a triangle, but the cool thing is that the same basic pattern works to find the area of any planar polygon with non-intersecting edges. It doesn’t even need to be convex. Just change the loop limit and mod from 3 to N. Just like the cross product, the magnitude will be twice the polygon area, and the sign will tell you if it is front or back facing.

Why does it work? It’s a simple application of Green’s theorum (or Stoke’s theorum in 3D). Bet you never thought you’d see that again, huh?

Mods add ugly unnecessary conditionals inside the loop, but if you shift the loop order a little, you can actually get rid of the mod:

for(int i0=N-1, i1=0;  i1<N;  i0=i1, ++i1) {

// loop order
// i0=N-1, i1=0
// i0=0,   i1=1
// i0=1,   i1=2
// …
// i0=N-2, i1=N-1

}

Extra bonus facts: in 3D, the resulting vector points in the direction of the polygon normal with length equal to twice the polygon area (just like the triangle cross product). Also, if you do it for the 1-ring of vertices around a given vertex (the vertices one edge away from a given vertex), you get exactly the triangle-area weighted average normal. Yes, that does mean that the area-weighted vertex normal doesn’t actually depend on the vertex position at all. Wierd, huh?

Marc