1.01.0

Texture space notation

Texture 2D u

v

Texture Space (or “parametric space" or "u-v space")

Texture Space = [0,1] x [0,1]

eg: 512 texels

e.g.: 1024 texels

1.0

1.0

Two notations

s-t (es OpenGL)

s t

1.0

1.0 1.0 u

1.0

u-v (es DirectX) (0,0)

(0,0)

Commonest

(in game industry)

Commonest

(in game industry)

Note: Texture coords do not depend on texture res

Texture 2D

s t

1.0 1.0

1024x512

1.0 s 1.0

128x64

Convenient!

We can reduce texture sheets res (balancing quality / memory) without affecting the UV mapof the mesh.

Eg:

loading RAM only top MIP-map levels in GPU

(a quality setting) t

Note: Texture coords do not depend on texture res

Texture 2D

s t

1.0 1.0

1024x512

s 1.0

128x64

Convenient!

We can reduce texture sheets res (balancing quality / memory) without affecting the UV mapof the mesh.

Eg:

loading RAM only top MIP-map levels in GPU

(a quality setting) t

Two types of UV-maps:

 NOT injective UV-maps

 Different zones of the mesh refer to the same texture region

 e.g.: with overlapping charts

 Optimization of texture space

 Exploiting of symmetries / repetitions

 InjectiveUV-maps

 Each (non empty) point on the texture:

1 only point on the mesh

 e.g.: non-overlapping charts

 Generality / Flexibility

 Used for several scopes (e.g. light backing)

 Different objectives

 often, both are present: 2 distinct UV mapping

 (two UV attributes for each vertex)

Q: Types of the UV maps shown so far?

aka: “Unwrapping”

or: “Unwrapped UVs”

or:“1:1UV-mapping”

or:“LightmapUV-mapping”

aka: “UV-map”

(standard)

Construction of the UV-map

 One typical task of modelling

 (semi-)automatic algorithm (very studied problem!)

 Professional modeler

 using specific tools

 In practice, we need to find a place in the texture space to hold for each mesh triangle

 Analog to:

 Peel an apple (cutting)

 Flatten the produced peels (unfolding)

 Assemble the peels on a rectangular plane (packing)

 Note: the (almost invariably) required “cuts”

(aka texture seams) are discontinuities of u,v values

 How to store them? vertex-seams

think about tetris!

Modeling task:

“u-v mapping”



Practical strategies:

 1. select the cutting edge

…or…

1. assign faces to “charts”

 either way, decide where “texture seams” are

 2. unfolding

 minimizing “distortion”

 3. charts packing

 Minimizing empty spaces

 Assign areas according necessity

(e.g., important parts  bigger texture space) (texel sampling becomes partially adaptive!)

DEMO!

Tileable Textures

A A

B

B

Tileable textures

Tileable textures



Typical use

Very efficient in space

Signals typically stored in textures (in games)

 Each texel a base-color (components: R-G-B)

 Texture is a “diffuse-map” / “color-map” / “RGB-map”

 Each texel a transparency factor (components: α)

 Texture is an “alpha-map” o “cutout-texture”

 Each texel a normal (components: X-Y-Z)

 Texture is a “normal-map” or “bump-map” (more later)

 Each texel contains a value di specularity

 Texture is a “specular-map”

 Each texel contains a glossiness value

 Texture is a “glossiness-map”

 Each texel contains lighting baked value...

 Texture is a (baked) “light-map”

 Each texel contains a dist. From surface value

 Texture is a “displacemnt map” o “height texture”

RGB maps:

How are they obtained?



Image first, then UV-mapping

 e.g. Images from photos

 e.g. tileable images



UV-mapping

paint 2D

 paint with 2D app(e.g. photoshop)



UV-mapping

paint 3D

 paint within 3D modelling software,

 or: 1. export 2D rendering,

2. paint over with e.g. photoshop, 3. reimport images

UV-mapper

UV-mapper 2D painter

UV-mapper 3D painter

RGB maps:

How are they obtained?

…or:

 first Paint 3D

 on hi-res model,

 “paint” on vertex attributes

 e.g. with Z brush…

 then coarsen

 build / autobuild final low-poly version

 then

UV-map

 the low-poly model

 must be a 1:1 mapping!

 then

auto-texture

 auto build texture

more about

this later…

Cutout textures

texels = alpha (transp. lvl)

Alpha map

RGB map

M a r c o T a r i n i ‧ C o m p u t e r G r a p h i c s ‧ 2 0 1 1 / 1 2 ‧ U n i v e r s i t à d e l l ’ I n s u b r i a

Cutout textures

texels = alpha (transp. lvl)

 e.g.: drapes, beard...

by Micheal Filipowski 2004

texture

Cutout textures

texels = alpha (transp. lvl)

 e.g.: trees, foliage

M a r c o T a r i n i ‧ C o m p u t e r G r a p h i c s ‧ 2 0 1 1 / 1 2 ‧ U n i v e r s i t à d e l l ’ I n s u b r i a

Texture mapping and Alpha Test



Eg: fur, fur coats

(repeated) texture

“Procedural Textures”



Idea: textures as function of (u,v)



E.g.: color texture for the Japanese flag:

u

v 1.0 0.5

0.3 0.5 1.0











 





else if v

f u

) 1 , 1 , 1 (

) 0 , 0 , 1

( ²

2

3 . 5 0

. 0

5 . 0  



 







 



 v u

Bump-Map (*)



texture in charge of providing an illusion of a hi- freq geometrical detail

 (not modeled by the mesh geometry)

 recall: meshes should be low-res (low-poly)



aka “Texture-for-Geometry”

 much cheaper to render/store than real geometry!

 details may extrude outwardor be engraved inward the mesh surface

 usually: this affects lighting only

 sufficient to trick the eye!

 especially under dynamic lighting (*) Terminology not universally adopted…

Often, «bump-map» is understood to be a «displacement map», or other types

Bump-Map

From the modeler/artist perspective:

 macro-structureof the object  low-poly mesh

 e.g.: the general shape of the horse

 e.g.: the general shape of the face

 e.g.: the general shape of the dragon

 meso-structureof the object  bump-map

 e.g.: the musculature of the horse

 e.g.: the wrinkles of the face

 e.g.: the flakes of the dragon

 micro-structureof the object material parameters

 e.g.: the fur of the horse

 e.g.: the structure of the dermis / del sebum

 e.g.: the variable roughness / smoothnes of the flakes

Bump maps:

Categories

Bump maps

Displacement maps

Normal maps

Scalar Vectorial Object

Space Tangent

Space most common

Bump-Mapping

see demo!

+ =

Low-poly mesh

(uv-mapped!)

Bump-map

(here: a tangent space normal map)

lots of cheap geometric detail

(apparently)

Low-poly mesh Bump-map

Bump maps:

Categories

 Bump map:

 Any texture that encodes hi-freq details

 Displacement Map :

 Details encoded by storing the differences between low-res and hi-freq

 Either as scalars(distance along the normal) or as vectors

 Used for re-tessellation, or for parallax mapping

 Normal Map:

 Details encoded by storing the normals of the hi-freq surface

 Modifying the lighting

 Normal vectors expressed in either

 Object Space, or

 Tangent Space (TBN space)

Displacement map (scalar):

concept

Store the distancesof the detailed surfaces from the geometric mesh

 example -- a bump-map for a screw-head :

Detailed surfaces head of the screw

low-poly mesh

displacement map (scalar)

0 0 0 0 .1 .5 .6 .6 .7 .5 .4 .2 0 0 0 0 0 0 0 0 0 0

0.2 0.6 0.4

Displacement map (scalar):

notes

 Each texel:

distanceof the described surface

 Along the normaldirection

 (of low-poly mesh)

 1 scalarper texel – 1 channel texture

 Directions:

 outwards (extrusion)

 inwards (excavation)

 both

 Positive values: extrusion

 Negative values: excavation inwards

 Storage:

 gray-scale image

 (1 scalar per pixel)

 Values are remapped in the interval [0..1]

 Uses:

 Global illumination computation (ambient occlusion)

 Intermediate data for the construction of a normal map

 Older use: rough, approximated lighting

 “embossing” effect

white = upwards black = downwards

(see later)

Easy to paint and Manipulate!

Practically, an height field defined on the

surface of the mesh

½ · + ½·( 1- ) = lighting

Displacement map (scalar):

Rendering – embossing effect

Displ.-map Displ.-map

(approximated)

shifted: !

Approx. too rough:

non used anymore (in games)

Image processing method

for approximating the lighting onto a (scalar) displacement map

 concept:

finite differences : approximate 2D gradient

approximate (X,Y) normal surface  approximate lighting

(scalar) Displacement map:

Rendering – parallax mapping



Technique used to simulate the parallax effect

 (onto a scalar Displacement Map)



We will see it in the lecture on rendering

Vectorial displacement map : concept

Store Vectorsfrom the low-poly mesh to the detailed surfaces

Detailed surface

Low-poly mesh

displacement map (vectorial)

Not an height field

More expressive variant, but more expensive and less usable Almost never used (in games).

Normal Map:

concept

Store the Normalsof the detailed surfaces

 example -- a normal-map for a screw-head :

Detailed surface head of the screw

low-poly mesh

normal map

(one normal per texel)

Normal Map:

notes



It affects lighting only

 noeffect on the silhouette

 noparallaxeffect

 still resulting illusion of shape details is convincing

 as long as macro-structure is not attempted



The lighting follows the hi-freq details

 variable lighting conditions (dynamic lighting) possible



Rendering algorithm:

 Just use the normal from the texture for lighting purposes

 (in place of the interpolated per-vertex normal)



Normals are stored as cartesian coord

 Although alternatives exist (to express unit vectors)

 Question: in which space?

Normal Maps: in which space are normals expressed?



Object space: Object-Space Normal-Maps (The same in which vertex pos are expressed)

 per-vertex normals: no longer needed

 (use per-texel normals instead)

 Trivial rendering algorithm

 normal map bounded to a specific object

 a normal map cannot be shared by different objects

 Each region of the normal map is bounded to responsible region of the object!

 Only injective (1:1) UV-maps!

 e.g. no tiling, no exploitation of simmetries

Tangent space (aka TBN space)



Vector space defined ∀ point of the surface:

 Z axis: Normal

 (to the surface)

 X and Y axis: tangent vector

 (of the surface)

 X = Tangent

 Y = “Bi-Tangent”

(sometimes, but less appropriately: “Bi-Normal”)

 stored: per-vertex, in the mesh structure

 i.e., as an attribute, interpolated inside triangles

 optimzations possible! Not necesessarly 3 vectors

Tangent space (aka TBN space)



How do I compute them?

 Normal

 as usual (see lec. on mesh)

 Tangent &Bi-Tangent

 Computable from UV mapping!

 They are the gradientof U coord and V coord respectively.

 (Implementation detail:

compute per face, avarage at vertex)

 Notes:

 The three vectors are not necessarely exactly orthogonal

 The system can be left-handed or right-handed, even in the same mesh

 The tangent directions riquires discontinuities  seams

 (The same required by UV mapping, but possibly more! why? see point above)

Normal Maps: in which space do I express the normal?



Tangent space: Tangent Space Normal-Maps (the default «bump-maps», in games)

 extra attributes are needed per vertex:

 Normal direction

 Tangent direction

 Bitangent direction

 the normal map can be shared by different objects

 non-injective UV-maps are ok

 e.g. tileable ones

 e.g. exploitation of simmetries

 the NM can be authored independently from the object

 e.g. starting from a displacement map The

tangent space

pratically, the normal map specifies how to modifythe normal sotred per vertex, instead of overwritingit

Mesh GPU Object

LOAD

Tangent Directions (B+T) as per vertex attributes

DISK CENTRAL RAM GPU RAM

PREPROCESS:

COMPUTE TANGENT DIRS

Mesh Object

IMPORT

Mesh File

Stored into asset mesh, or computed after import:

WITH TANGENT DIRS

(per vertex)

 Idea:

as RGB texture

 RX

 GY

 BZ

 but X,Y,Z∈ [-1,+1] while R,G,B∈ [0,+1]

thus remapping is necessary:

 Advantage: same compression of RGB textures/images

Normal-maps:

Storage

+1

-1 0

1.0

0.0

X∈ ^∈ R Y = 2^{X = 2R – 1}G – 1

Z = 2B – 1 R = ½ (X + 1 )

G = ½ (Y + 1 ) B = ½ (Z + 1 ) (normals = unitvectors)

Normal-maps:

Storage



Examples:

 tangent space normal-map case

prevailing normal: X=~0 , Y=~0 , Z=~1



prevailing color: R=~0.5 , G=~0.5, B=~1 ( ~light blue)

Example: Tiled

(tangent-space) Normal Map

+ =

with UV-mapping (using tiling!) + Tangent dirs.

Tileable!

Low-poly mesh Normal-map

It wouldn’t be possible with object space NM

Bump-maps assets at a glance

(can you tell which is which?)

Object Space Normal map Tangent Space

Normal map

Displacement Map (scalar)

the default kind

Note

Object Space Normal map UV-mapping 1:1

right leg !=

left leg

(Tangent Space) Normal map UV-mapping NOT injective

Exploited symmetries!

Normal maps:

how are they obtained (1/4)



From displacement maps!

Displacement map as a grayscale:

= extruded outwards

= engraved inward

Filter e.g.

photoshop filter e.g. unity texture importer 2D texture

painter / etc

Normal map (tangent space)

see demo!

Normal maps:

how are they obtained (1/4)



Conversion: displacement map  normal map



Algorithm:

 ∀ texel t of displacement map

 extract 3D points of a local neighborhood

 3D point  (x , y , z = displacement[x,y] )

 compute best fitting plane of 3x3 neighborhood

 plane that minimizes the distance from the points

 simple quadratic minimizatin problem

 The normal in this plane is the normal for t



Note: it’s computed in texture space

 hence it is a tangent space normal map

or 5x5, or 7x7…

Normal maps:

how are they obtained (2/4)



Photometric Stereo

 (a form of “inverse lighting”)

 from: N real images (N>=4)

 Same viewpoint

 Different illumination

 (possibly, controlled and known)

 a Normal Map

 expressed in screen space!

 convert in object space  then TBN

Normal maps:

how are they obtained? (3/4)



Direct “painting” of normals on the model

 (e.g. with Z-brush, Sculptris Alpha…)

 Similar to a painting of diffuse color maps

 brush-strokes are recorded in texture image

model can be low-res and needs to be UV-mapped

 but: the system stores the normals or displacements, not the color

the artist “paints” shape details: bump, extrusions…

 Similar to 3D sculpting

 same “brushes” (e.g. extrude, engrave, smooth…)

 but: the system stores the normals – or displacement, not the position of vertices

only small scale details can be scuplted

Normal maps



How are they obtained (bonus):

Procedurally

time varying, procedural

“ocean waves”

normal map

All the usual considerations

aboutproceduralityapply.

Procedural textures:

Can be computed on the fly per-pixel (fragment shader task)

Normal maps:

how are they obtained (4/4)



“Detail texture” synthesis aka “texture for geometry”

aka “detail texture” synthesis aka baking of normal-map

 from:

 Hi-Res mesh A

 Low-res mesh B

+ with UV-map (of the 1:1 kind)

 to:

 a Normal map for B

that mimics the detail originally present in A

representing the same object

Detail texture synthesis



Concept:

 the attribute / geometry data originally present in the vertices of High-level res model A is transferred to a texture for low-res model B



Not limited to normal maps!

Any attribute can be transferred:

 normal (result: an object space normal map)

 position (result: a displacement map)

 diffuse color (result: a diffuse texture) or any other material parameter

 global lighting pre-computed on B (result: a AO-map)

Modelling + Texturing:

Pipeline production example 1

 Concept drawings

 by 2D artists

 Low-poly model A

 by 3D modelers, low-poly editing tools

 UV-map (1:1) for A

 by UV-mapper, aided by automatic tools

 Hi-Res model B from A (e.g. subdivision + digital sculpting)

 by 3D modelers, digital sculptors

 Painting over B

 per vertex RGB color

 Texture synthesis:

 Automatic construction of Textures for A with attributes from B:

 Normals from B, (normal map)

 Colors from B (diffuse map)

 Baked lighting from B (light-map)

Discard B, keep A (+textures)

Modelling + Texturing:

Pipeline production example 2

 Concept drawings

 by 2D artists

 Physical sculpting of a colored statuette

 by (non-digital) 3D modeler

 Hi-Res model B as a 3D Scanning of the statuette

 by a 3D scanner (note: B has per-vertex color attributes)

 Low-Res model A built from B

 manually (low-poly construction of A guided by B)

or automatically (A is the automatic polygonal decimation of B)

 note: A has no colors

 UV-mapping built for A

 Texture synthesis:

 Automatic construction of Textures for A with attributes from B:

 Normals, (normal map)

 Colors from B (diffuse map)

 Baked lighting from B (AO-map)

 Discard B, keep A (+textures)

M a r c o T a r i n i ‧ [ G A M E - D E V ] ‧ V e r o n a ‧

hi-res mesh

low res mesh

automatic simplification

still low-res, but textured!

rendering

TEXTURE Made up (e.g.. BumpMap

normals or RGB map x colors) detail

recover

M a r c o T a r i n i ‧ [ G A M E - D E V ] ‧ V e r o n a ‧ 2 0 1 3

simplification

2K triangles

original

500K triangles

Simplified but with texture

2K triangles

Detail Recovery: algorithm

Hi-res model Low-res

model

Texture map

u v

find a suitable spot

Encode & Store

repeat ∀ texel in tri, Some

attribute

e.g.: color, precomputed lighting, normal…

Example

Example

esempi da cg-talks – CG society

sempi da cg-talks–CG society

esempi da cg-talks–CG society

“Procedural Textures”



Functions from (u,v) to texel values

 To be dynamically computed, during rendering

 (for each fragment – task of the fragment shader)

 Instead of accessing a 2D array of texels



Advantages:

 texture RAM costs: none

 costsALUtime instead

 resolution independent

 aliasing of screen pixel only:

“infinite resolution”



Example:

a procedural diffuse texture:



Easy for simple images only

“Procedural Textures”



Example (exercise!):

 how would you code a

procedural diffuse texture for … the flag of Japan?

u

v

1

1

0.5

0.375

𝑓 𝑢 𝑣 =

𝑟 𝑔 𝑏

vec3 procedural_texture( vec2 p ) {

…;

}

Solid Textures

Solid Textures

 Structure: 3D array of texels

 1 texel == 1 voxel

 e.g. each voxel one color RGB solid RGB textures

 Just as any other texture:

 is kept in video RAM

 fast HW access during rendering

 filtering (tri-linear) in access, hardwired MIP mapping …

 can be tillable (and often is)

 It models color inside volume

 how objects are colored internally

 useful, e.g., for breakable objects, wood, marble

 Note: no UV-maprequired!

 The texture indexed by mesh geometry (appropriatedly rescaled)

 Problem: storage size

 Cubic w.r.t. the resolution!

 One solution: procedural 3D texture?

Procedural Solid Textures