Explaining how a modern GPU works in completeness
would take a book. Or two. Per class of chip. Per vendor. They're
extraordinarily complex pieces of engineering and production, and
the end result contains more transistors than multiple modern x86
processors. The cost for research and development of just one modern
graphics product generation, before production even begins, is closing
in on the half billion dollar mark.
So the task of explaining how such a thing works, in the confines
of a TekSpek article, should be an impossibility, right? Awesomely
for us, wrong! While to cover absolutely everything would take a
fat tome, covering the basics is easily done if you're willing to
learn, and it's something the layman should have no problem understanding.
Allow us to have a go.
Shader Programs
Before we begin, we need to explain the concept of shader programs
and texturing. Shader programs are what define a modern graphics
processor as it's used by developers. A shader program is a set
of instructions, as in any other programming language, that operate
on vertices or pixels, that modify or change the attributes of the
vertex or pixel they're working on, to change it's position or appearance.
That task of 'shading', using a set of math or texture instructions,
is common to both vertex and pixel (although the actual instructions
may differ), and it's the reason the modern GPU is designed the
way it is. The programmability is the key to allowing developers
to use the GPU for ever more advanced, complex and realistic effects,
more easily.
The advancement of the GPU is designed to add in more programmable
functionality, while simultaneously endeavouring to make it easier
to exploit the new and existing abilities. So keep the concept of
shading an object in mind, where it's changed by a set of program
instructions that are run, defined by the developer.
Texturing
Texturing in older 3D hardware meant the process of texture mapping,
or applying an image to sections of geometry according to perspective.
Texturing in a modern 3D GPU means the process of sampling a texture,
whether it contains an actual picture image or other data, and using
that as input into a shader program for further processing. The sampling
and use inside the shader program might well be to perform perspective
correct texture mapping, but more often than not an actual image to
be sampled is in the minority.
Instead, the majority of textures sampled will contain data other than a coloured image. Textures
are bound to samplers in the shader program and the shader program
can arbitrarily sample from anywhere in the texture, with the GPU
filtering the data if needed.
Most GPUs have a single-cycle bilinear filter available for texture
sampling, where four points surrounding where you want to sample
from in the texture are sampled instead, and the samples are then
averaged and returned as the result. Trilinear filtering is the
combination of two bilinear filter operations (two cycles in modern
hardware), and anisotropic filtering starts at a minimum of sixteen
samples and a similar number of cycles to perform and complete.
It all starts with geometry
To explain how a modern GPU works, we start with geometry. A 3D
application uses the CPU in your system to generate geometry to
sent to the GPU for processing, as a collection of vertices. Geometry
can be pre-generated and read from disk, or generated on the fly
by the program code. A vertex consists of attributes that define
its position in 3D space (relatively, usually), along with anything
else the developer wants to define such as a colour for the vertex
or some other relevant piece of information.
The CPU, interfacing with the driver for the GPU, sends the collection
of vertices to the GPU to start the rendering process, using the
vertex shader units. When the vertex lists are present on the hardware
inside the GPU's accessible memory, the GPU can either process them
as-is without changing them in any way, or vertex shading can happen
using the processes of shading and texturing outlined earlier. The
vertex shader program will process and alter the attributes of the
vertex, on a vertex-by-vertex basis, before they're passed to the
next step in the rendering process, by the vertex processing hardware.
Rasterisation into pixels
The process of rasterisation takes the geometry processed
by the vertex hardware and converts it into screen pixels to be processed
by the pixel shader (or more accurately pixel fragment) hardware.
The GPU basically walks the big list of geometry, per frame, analysis
it per vertex, then outputs a pixel fragment for the pixel units to
work on. The fragment designation comes from the fact that depending
on how the geometry is to appear on screen, parts of the triangle
primitives displayed can lie inside a pixel on your screen, but not
totally cover it. Two triangles (or more) can be rendered inside of
one pixel, so since the actual output from rasterisation is part of
a pixel, the data is actually a pixel fragment.
So rasterisation
is simply the conversion of geometry into screen pixel fragments,
generated by walking over the geometry lists and analysing them
to see where they lie on the screen. It's a mostly fixed-function,
high speed process, and it's very rare to be bound by the performance
of that rasteriser hardware.
Pixel processing
Pixel processing is almost identical to the steps of vertex processing,
just the processing hardware works on pixel fragments instead. Pixel
shader programs are run, fragment-by-fragment, to alter the fragment
attributes before they're displayed on the screen. The pixel shader
program exists to alter the colour of the pixel fragment in some
way, based on the instructions in the shader program which may or
may not be texturing, to have it combine in the end with the colour
of all the fragments on screen to generate your image.
Pixel shading is usually the most compute-intensive part of the
graphics rendering process on a modern GPU and so usually takes
the most time, and is the place in rendering where you're most likely
to be bottlenecked.
Rendering pixels to the screen
Processed pixel fragments are stored in card memory
ready to be resolved into completed screen pixels, for output onto
your display. This task is handled by a GPU unit called the ROP. A
modern GPU implements a number of ROPs, based on how likely the GPU
is to be bottlenecked by pixel output, to perform the final tasks
of rendering. As well as simply resolving and drawing pixels on your
screen, the ROP hardware also performs a number of optimisations to
save memory bandwith when reading and writing pixels to and from a
framebuffer, such as colour compression (even saving 1 byte of colour
data per pixel is a heady saving in bandwidth terms).
The ROP units
also deal with depth compression and compare, the compare - where
you test pixels against each other to see which is on top of the
other - being the main facilitator for multisample antialiasing.
Multisample antialiasing uses depth information (Z) to alter the
colour of pixels so that geometry rasterised earlier in the render
process is antialiased and looks better.
Antialiasing
Antialiasing works by effectively filtering a high frequency signal.
Got a stepped black line of geometry against a white background
(the black and white colour data being the high frequency signal)?
Filtering the signal will result in greys along the stepped edge,
providing a better representation of the data. That's really all
there is to multisample antialiasing. Use the depth of the pixel
to filter the colour data where geometry is.
Final output by the GPU
After pixel resolve, antialiasing and optimisation of memory bandwidth
using Z and colour compression, the completely rendered output is
pushed to your monitor via the GPU hardware. If it's a digital display
being rendered to, the framebuffer data is converted into a binary
respresentation and squirted at high speed to the digital monitor.
If it's an analogue display, the colour data of the pixels is converted
to an analogue signal across the scanlines by a DAC. Repeat for
as many frames as you want to draw.
And that set of steps, from geometry generation and shading, through
rasterisation and pixel processing, and finally drawing the fully
rendered output, is the render process of a modern GPU. Broken down
into those four component steps it's easy to understand how a modern
immediate mode 3D processor works, without going into the details
of caches, buffers, memory access, shader models, texture filtering
(although we touched on that) and other implementation details that
are specific to chip variations.
Just remember this
Processed vertices get turned into pixels, then they're processed
and drawn on your screen. Repeat. The rest will come back to you
pretty quickly if you grasp those steps.
