Master of Science in Engineering: Game and Software Engineering June 2020

## Real-Time Ray Tracing With Polarization

## Parameters

### Viktor Enfeldt

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Engineering: Game and Software Engineering. The thesis is equivalent to 20 weeks of full time studies.

The author declares that they are the sole author of this thesis and that they have not used any sources other than those listed in the bibliography and identified as references. They further declare that they have not submitted this thesis at any other institution to obtain a degree.

Contact Information: Author: Viktor Enfeldt E-mail: vien15@student.bth.se viktor.enfeldt@gmail.com University advisor: Dr. Prashant Goswami

Department of Computer Science

Faculty of Computing Internet : www.bth.se

Blekinge Institute of Technology Phone : +46 455 38 50 00

### Abstract

Background.The real-time renderers used in video games and similar graphics applica-tions do not model the polarization aspect of light. Polarization parameters have previously been incorporated in some offline ray-traced renderers to simulate polarizing filters and various optical effects. As ray tracing is becoming more and more prevalent in real-time renderers, these polarization techniques could potentially be used to simulate polarization and its optical effects in real-time applications as well.

Objectives.This thesis aims to determine if an existing polarization technique from offline renderers is, from a performance standpoint, viable to use in real-time ray-traced applica-tions to simulate polarizing filters, or if further optimizaapplica-tions and simplificaapplica-tions would be needed.

Methods. Three ray-traced renderers were implemented using the DirectX RayTracing API: one polarization-less Baseline version; one Polarization version using an existing polarization technique; and one optimized Hybrid version, which is a combination of the other two. Their performance was measured and compared in terms of frametimes and VRAM usage in three different scenes and with five different ray counts.

Results.The Polarization renderer is ca. 30% slower than the Baseline in the two more complex scenes, and the Hybrid version is around 5–15% slower than the Baseline in all tested scenes. The VRAM usage of the Polarization version was higher than the Baseline one in the tests with higher ray counts, but only by negligible amounts.

Conclusions. The Hybrid version has the potential to be used in real-time applications where high frame rates are important, but not paramount (such as the commonly featured photo modes in video games). The performance impact of the Polarization renderer’s im-plementation is greater, but it could potentially be used as well. Due to limitations in the measurement process and the scale of the test application, no conclusions could be made about the implementations’ impact on VRAM usage.

Keywords:ray tracing, polarizing filters, Mueller calculus, real-time, rendering

### Sammanfattning

Bakgrund.Realtidsrenderarna som används i videospel och liknande grafikapplikationer simulerar inte ljusets polarisering. Polariseringsinformation har tidigare implementerats i vissa stålföljningsbaserade (ray-traced) offline-renderare för att simulera polariseringsfilter och diverse optiska effekter. Eftersom strålföljning har blivit allt vanligare i realtidsrenderare så kan dessa polariseringstekniker potentiellt också användas för att simulera polarisering och dess optiska effekter i sådana program.

Syfte. Syftet med denna rapport är att avgöra om en befintlig polariseringsteknik från offline-renderare, från en prestandasynpunkt, är lämplig att använda för att simulera polaris-eringsfilter i stålföljningsbaserade realtidsapplikationer, eller om ytterligare optimeringar och förenklingar behövs.

Metod.DirectX RayTracing API:et har använts för att implementera tre stålföljningsbaser-ade realtidsrenderare: en polarisationsfri Baseline-version; en Polarization-version med en befintlig polariseringsteknik; och en optimerad Hybrid-version, som är en kombination av de andra två. Deras prestanda mättes och jämfördes med avseende på frametime och VRAM-användning i tre olika scener och med fem olika antal strålar per pixel.

Resultat. Polarization-versionen är ca 30% långsammare än Baseline-versionen i de två mest komplexa scenerna, och Hybrid-versionen är ca 5–15% långsammare än Baseline-versionen i alla testade scener. Polarization-Baseline-versionens VRAM-användningen var högre än Baseline-versions i testerna med högre strålantal, men endast med försumbara mängder. Slutsatser. Hybrid-versionen har potential att användas i realtidsapplikationer där höga bild-hastigheter är viktiga, men inte absolut nödvändiga (exempelvis de vanligt förekommande fotolägena i videospel). Polarization-versionens implementation hade sämre prestanda, men även den skulle potentiellt kunna användas i sådana applikationer. På grund av mät-processens begränsningar och testapplikationens omfattning så kunde inga slutsatser dras gällande implementeringarnas påverkan på VRAM-användning.

Nyckelord:strålspårning, polariseringsfilter, Mueller-kalkyl, realtid, rendering

### Acknowledgments

I would like to thank the following people for their contributions to this thesis: my supervisor Dr. Prashant Goswami, for his guidance and advice on how to improve the project, the report, and the presentation; everyone who read and gave feedback on the report, for their helpful comments and suggestions; Stefan Petersson, for the initial discussions about the project idea; and lastly, my partner Jessica, for her love and support throughout the course of this project and the past few years of my studies.

### Contents

Abstract i

Sammanfattning iii

Acknowledgments v

Lists of Figures, Tables, Algorithms, and Listings ix

Nomenclature xi

1 Introduction 1

1.1 Aim, Research Question, and Objectives . . . 3

1.2 Delimitations . . . 3

1.3 Contribution . . . 3

1.4 Societal, Ethical, and Sustainability Aspects . . . 4

1.5 Thesis Outline . . . 4

2 Basic Concepts 5 2.1 Ray Tracing . . . 5

2.1.1 Comparison With Rasterization . . . 6

2.1.2 Hardware Acceleration . . . 7

2.2 Light . . . 7

2.2.1 Polarization . . . 8

2.2.2 Light Rays and Their Polarization States . . . 8

2.3 Fresnel Reflectance . . . 9

2.3.1 Implementation in Non-Polarizing Real-Time Renderers . . . 10

2.4 Polarizing Filters . . . 11

2.5 Polarization Calculus . . . 11

2.5.1 Stokes Parameters . . . 12

2.5.2 Mueller Matrices . . . 13

2.5.3 Operations on Stokes Vectors and Mueller Matrices . . . 14

3 Related Work 17 4 Implementation 19 4.1 Shading Model . . . 19

4.1.1 Diffuse Reflection Term . . . 20

4.1.3 Mirror Reflection Term . . . 21

4.2 Baseline Renderer . . . 22

4.2.1 Data Structures and Rendering Algorithm . . . 22

4.3 Polarization Renderer . . . 22

4.3.1 Data Structures and Rendering Algorithm . . . 23

4.3.2 Reference Frame Tracking . . . 24

4.4 Hybrid Renderer . . . 25

4.5 Implementation Summary . . . 25

4.6 Changes Made to the Falcor API . . . 25

4.7 Material Texture Limitations . . . 26

4.8 Polarization Demonstration . . . 26

5 Experiment Details 29 5.1 Measurements . . . 30

5.2 Validity . . . 30

5.3 System Under Test . . . 30

6 Results and Analysis 33 6.1 Average Frametimes and Frame Rates . . . 33

6.2 VRAM Usage . . . 34

6.3 Raw Frametime Measurements . . . 36

7 Discussion 39 7.1 Performance Impact . . . 39

7.1.1 Frametimes . . . 39

7.1.2 VRAM Usage . . . 40

7.1.3 Limitations of the Tests . . . 41

7.2 Implementation Considerations . . . 41

8 Conclusions and Future Work 43 8.1 Future Work . . . 43

8.1.1 Stokes-Mueller Calculus in Denoised Monte Carlo Ray Tracing . . 43

8.1.2 Performance Study in a Real-World Application . . . 43

8.1.3 Simplifying the Polarizing Fresnel Function . . . 44

8.1.4 Simplifying the Polarization Data . . . 44

8.1.5 Selective Polarization . . . 44

8.1.6 Polarizing Filters Without Polarization Parameters . . . 44

References 45

### List of Figures

1.1 Image rendered with ray tracing . . . 1

1.2 Photographs taken with and without a polarizing filter . . . 2

2.1 Ray tracing from the view-point . . . 5

2.2 A plane electromagnetic wave visualized as a transverse wave . . . 7

2.3 Three polarized waves with identical wavelengths . . . 8

2.4 Interaction between a ray of unpolarized light and a dielectric surface at Brewster’s angle . . . 10

2.5 Polarization ellipse . . . 12

4.1 Visualization of the diffuse, specular, and mirror reflection terms . . . 20

4.2 Visualization of a reference frame rotation for a sequence of reflections . . 25

4.3 Polarization and the visual impact of a polarizing filter in the Polarization and Hybrid renderers . . . 28

5.1 The three test scenes used in the performance tests . . . 29

6.1 Average frametimes and frametime differences from the Baseline . . . 33

6.2 Average frame rates and frame rate differences from the Baseline . . . 34

6.3 Raw and per–test configuration average frametime data for the Arcade test scene . . . 36

6.4 Raw and per–test configuration average frametime data for the Temple test scene . . . 37

6.5 Raw and per–test configuration average frametime data for the Bistro test scene . . . 37

### List of Tables

4.1 Summary of the three renderers . . . 255.1 Test scene triangle and light counts . . . 29

5.2 System under test . . . 31

6.1 Increase in VRAM usage of the Polarization renderer over the Baseline renderer . . . 35

6.2 Increase in VRAM usage of the Hybrid renderer over the Baseline renderer 35 6.3 Raw VRAM usage measurements from all the test configurations . . . 35

### List of Algorithms

2.1 Ray tracing rendering loop . . . 6

2.2 Rasterization rendering loop . . . 6

4.1 The Baseline implementation’s rendering algorithm . . . 22

4.2 The Polarization implementation’s rendering algorithm . . . 24

5.1 Performance test order . . . 30

### List of Listings

4.1 Ray payload used in the Baseline implementation . . . 22 4.2 Ray payload and data structures used in the Polarization implementation . 23### Nomenclature

### Terms

dielectric Nonconductor of electricity; not a metal. 2, 9–11, 26–28, 41, 44 microfacet Very tiny mirror-like facet of the surface of an object. 19, 20, 24

Monte Carlo Computational method using repeated random sampling. 6, 17, 19, 41, 43 partially polarized Mixture of unpolarized and polarized light. 8, 9, 11

polarization state Direction of the oscillations in a transverse wave or group of waves. 2, 8–12, 14, 15, 17, 23

polarizing filter Optical filter that only lets waves with a specific polarization state pass through. Also called polarizer. 2–4, 10, 11, 13, 14, 17, 23, 24, 26–28, 41–44

primary ray Ray that is cast from the camera into the scene. 5, 6, 19, 22, 24, 25, 40 quasi-monochromatic Consisting of a very limited wavelength spectrum. 9, 11–13, 23 reference frame Plane that is perpendicular to a ray or wave’s direction. 15, 24, 25, 41 unpolarized Consisting of randomly polarized waves. 8–11, 13, 15, 22–25, 27, 41

### Abbreviations

API Application Programming Interface. 7, 19, 25

BRDF Bidirectional Reflectance Distribution Function. 19–21, 44 CPU Central Processing Unit. 19, 30

DOP Degree of Polarization. 12, 28 DXR DirectX Raytracing. 7, 22, 30 FPS Frames Per Second. 30

GPU Graphics Processing Unit. 3, 6, 19, 26, 29, 30 IOR Index of Refraction. 9, 10, 17, 26, 41

VRAM Video Random Access Memory. 3, 4, 30, 34, 35, 39–41

### Symbols

ˆ

a Normalized vector.

α (alpha) The square of the material’s roughness value. 20, 21 δ (delta) Phase retardation. 9, 10, 14, 17, 41, 44

η (eta) Complex refractive index. Defined as η= n−κi. 9, 26

θ (theta) Angle of incidence; the angle between ˆv and ˆh. 9, 10, 19, 20, 22, 24 κ (kappa) Extinction coefficient of the refractive index. 9, 10, 26

λ (lambda) Wavelength. 7, 8
c_{diff} Diffuse material color. 20

cspec Specular material color. 10, 11, 26

E_{x} The x component of the electric field. 7, 8, 12, 24
Ey The y component of the electric field. 7, 8, 12

f Bidirectional reflectance function (BRDF). 19–21

F Fresnel reflectance function. 9, 10, 14, 17, 20–22, 25, 26, 44 ˆh Half vector between ˆl and ˆv. 20, 21, 24, 25

I Radiant intensity. 11–13, 23

Kd Diffuse reflection coefficient. 19, 20, 22, 24

K_{r} Mirror reflection coefficient. 19–22, 24
Ks Specular reflection coefficient. 19–22, 24

ˆl Vector from the intersection point to the light source. 19–21, 25 M Mueller matrix. 13–15, 25

n Simple refractive index. 9, 10, 26

ˆn Surface normal at the intersection point. 10, 20, 21 p Degree of polarization (DOP). 12, 13, 26

⃗S Stokes vector. ⃗S = (s0, s1, s2, s3). 13–15, 23

ˆv Vector from the intersection point to the viewer. 19–21, 24, 25

ˆx Local x-axis unit vector for a ray’s or a Mueller matrix’s reference frame. 24, 25

### Chapter 1

### Introduction

Shading and rendering techniques have developed over the years to simulate, or at least approximate, the visual appearance of real-world materials and how light interacts with them. A powerful rendering technique that is often used to create photorealistic visual effects in film and television productions is ray tracing. It uses virtual rays to simulate how light moves and interacts with objects in a scene, and can be used to achieve a multitude of visual effects (such as reflections and realistic soft shadows) [16]. A ray-traced image with some of those effects is shown in Figure 1.1. Notice how the square light source is visible in not just direct reflections, but also in secondary ones (e.g., the bright spot in the lower-left part of the white sphere is due to a prior reflection in the floor).

Figure 1.1:Image rendered with ray tracing and featuring effects such as global illumination, soft shadows, depth of field, and Fresnel reflections. (Image from Wikipedia user Mimigu [24]).

Ray tracing comes at a considerable computational cost, so it has historically not been used in complex real-time applications (such as video games) where cheaper, but more limiting, triangle rasterization techniques have been used instead [1, p. 415]. However, thanks to recent hardware and software advancements, ray tracing is now becoming increas-ingly common in higher budget computer games such as 4A Games’ Metro: Exodus [3], Eidos-Montréal’s Shadow of the Tomb Raider [15], andDICE’s Battlefield V [28].

2 Chapter 1. Introduction With these recent technological developments comes an opportunity to implement ray tracing features from offline renderers in real-time ones. One such feature is the simulation of light’s polarization state: a property that describes the orientation of the electromagnetic field, and which can be impacted by interactions with various materials. The light that is reflected in dielectric (i.e., non-metallic) surfaces, for example, becomes partially or fully linearly polarized depending on the angle of incidence [7, p. 43]. Several calculi have been developed over the years to analyze polarization [5, p. 22.8], and a few of them have been implemented in offline ray-traced renderers to simulate numerous visual effects [26, 33, 35, 40]. However, they have (to the best of the author’s knowledge) yet to be incorporated and tested in a real-time ray-traced renderer.

Figure 1.2:Photographs taken with (left) and without (right) a polarizing filter. In the version taken with a polarizing filter, the surface of the water appears to be significantly less reflective since the filter has prevented much of the reflected light from reaching the camera sensor. Note also how the color of the vegetation differs between the two photographs. (Image from Dmitry Makeev [23]).

The polarization of light generally has no directly noticeable impact on how our eyes perceive the light; however, it does have a significant impact on what the world looks like when viewed through a polarizing filter (which blocks out light that is not linearly polarized in the orientation of the filter). Photographers frequently use polarizing filters for various purposes: they can make glass and water surfaces appear more transparent, make foliage appear more colorful, and reduce the intensity of bright reflections (such as glare from sunlight). These effects are all difficult to replicate in software once a photograph is taken, yet they are easily obtainable by placing a polarizing filter in front of the camera lens and adjusting its rotation. An example of the visual impact this can have on a photograph is shown in Figure 1.2. In the left photograph (which was taken with a polarizing filter), the reflections in the water have been reduced significantly and made the stream bed more visible than it is in the unfiltered photograph on the right.

Many modern video games include a photo mode that allows the user to move around and take pictures within the game’s virtual environments [4]. Often with several camera-like settings such as aperture, exposure, depth of field, focal length, and various filters. However, since the renderers that are used in games do not model the polarization state of light in their shading equations, polarizing filters have not yet been an option in these modes.

1.1. Aim, Research Question, and Objectives 3

### 1.1

### Aim, Research Question, and Objectives

This research builds upon previous work about polarization ray tracing in offline renderers by, among others, Wolff and Kurlander [40], and which has been well summarized by Wilkie and Weidlich [38, 39]. The aim is to evaluate how feasible it is to simulate polarizing filters in a real-time ray-traced application (such as a video game) by using an existing polarization technique. The research question is:

RQ. “What is the performance impact of simulating polarizing filters and polarized light in a real-time ray-traced application?”

In order to answer it, the following objectives will be completed:

1. Create a Baseline real-time ray-traced renderer using conventional shading methods. 2. Create a Polarization renderer by duplicating the Baseline renderer and adding

polarization parameters to it.

3. Create a Hybrid renderer that uses the same primary rays as the Polarization version, and the same reflection rays as the Baseline version.

4. Compare the performance of the Polarization and Hybrid renderers with the Baseline one in terms of frame times and video memory (VRAM) usage in scenes of varying

geometrical complexity.

### 1.2

### Delimitations

The study is limited to a single application that is tested on a single computer, and only using one kind of polarization ray tracing implementation. It does not investigate the performance impact a polarization implementation would have on different applications, nor does it investigate how different graphics processing units (GPUs) are affected by the

implementation. The study only investigates the performance impact the implementations have in different scenes within the same application when running on the same hardware.

### 1.3

### Contribution

When Wolff and Kurlander [40] first added polarization parameters to an offline ray-traced renderer in 1990, the rendering time of the polarized version approached that of the unmodified one in complex scenes, but was approximately doubled in simpler scenes. Rendering hardware and software have changed and improved significantly since then; with real-timeray tracing now being in its infancy, this thesis shows what performance impact polarization parameters have on a modernGPUwith hardware-accelerated ray tracing.

4 Chapter 1. Introduction

### 1.4

### Societal, Ethical, and Sustainability Aspects

This thesis is not expected to have an impact on any societal or ethical aspects since it is limited to researching graphical improvements to computational models of light. It does not involve any test subjects and is not expected to have any consequences outside of the field of computer graphics.

The author acknowledges potential sustainability-related issues surrounding the pro-duction and use of computer hardware; however, such systematic issues are deemed to be beyond the responsibility of the author and the scope of this project. The environmental impact of this research is limited to the power consumption associated with the desktop computer that was used for the development and testing, and the production of the graphics card that was purchased to run the tests.

### 1.5

### Thesis Outline

Chapter 2explains some basic information and theory about ray tracing, the polariza-tion of light, Fresnel reflectance, polarizing filters, and the Stokes-Mueller calculus. This information is needed to understand the polarization-capable renderers.

Chapter 3touches upon some related work from the area of polarization rendering. Chapter 4describes the three renderers that have been implemented in the test application. It details their shading models, their rendering algorithms, and how the polarization theory from chapter 2 has been implemented in the polarization-capable versions. Section 4.8 closes out this chapter by demonstrating the visual impact polarizing filters have in the two polarization-capable implementations.

Chapter 5details the test scenes and settings that were used to evaluate the performance of the three renderers, how the tests were carried out, and the computer that was used to run the tests.

Chapter 6presents and analyses the resulting frametime andVRAMusage measurements

from the performance tests.

Chapter 7discusses the implications of the results, some limitations of the tests, and some considerations for future implementations.

### Chapter 2

### Basic Concepts

Some information about the underlying theory behind the rendering implementations is needed to understand their context and how they function. Those basic concepts will be presented in this chapter, and later referred to from chapter 4’s implementation details.

### 2.1

### Ray Tracing

Ray tracing is the process of tracing virtual rays of light as they scatter around a scene and then using the information gathered by them to render an image of that scene. The most true-to-nature approach to it would be to spawn rays at each light source in the scene and have them reflect and refract in the objects they hit until they eventually make their way to the viewer. This approach would be highly inefficient since the vast majority of light rays in a scene will not hit the viewer and, therefore, do not contribute to the final image.

### r

### r

′### r

### r

′### r

### r

′### r

′′Figure 2.1:Ray tracing from the view-point. Primary rays r are cast from the view-point to the center of each pixel. Recursive reflection rays (r′, r′′, etc.) are then cast from the primary rays’ and

the subsequent reflections rays’ intersection points. Shadow and refraction rays have been left out for simplicity, and they will not be used in this project.

The approach that is used instead is to trace rays from the viewer and into the scene (see
Figure 2.1 for a visualization). This was first introduced in 1969 by Appel [2] and has since
been developed into a few different variations. Appel’s technique casts one ray per pixel
(called a primary ray) into the scene and finds the first object that’s blocking the ray (which
is a process called ray casting*_{). Shadow rays are then cast from the pixel’s intersection}

point to the lights in the scene. If a shadow ray is obstructed, then its target light does not

*_{a term that is sometimes also used to refer to Appel’s rendering algorithm as a whole}

6 Chapter 2. Basic Concepts contribute to the shading of the pixel. Algorithm 2.1 describes the general algorithm that is used when ray tracing from the view-point.

Algorithm 2.1:Ray tracing rendering loop. for eachpixel (ray)∈ output image do

for eachtriangle∈ scene do

iftriangle is ray’s closest hit then SHADEPIXEL()

Whitted-style ray tracing*_{was developed by Whitted in 1979 and built upon Appel’s}

technique by adding recursive rays to render reflections and refractions [36]. Each intersec-tion point spawns a reflecintersec-tion and a refracintersec-tion ray, as well as rays to all light sources. This process is repeated recursively, creating a tree of rays for each pixel, until none of the new rays intersect any object or some depth- or ray-limit is reached. The color of each pixel is determined by the primary ray, which gathers data from all of its recursive rays.

Almost all modern ray-tracers are based on some variation of recursive Monte Carlo
ray tracing [16, p. 7], such as Kajiya’s path tracing [20] which was partially based on
Cook et al.’s distribution ray tracing†_{[10]. Recursive Monte Carlo ray tracing uses random}

sampling of possible reflection and light directions. This enables the rendering of soft shadows, depth of field, and fuzzy reflections. It also makes global illumination possible by taking the light contribution from all objects in the scene into account, instead of from just the light sources. The naive implementations of Monte Carlo ray tracing require many rays per pixel to get an image that does not have a considerable amount of noise; therefore, real-time applications need to use denoising techniques in order to achieve similar quality results with much fewer rays [16, p. 287].

### 2.1.1

### Comparison With Rasterization

Real-time applications have historically used rasterization techniques instead of ray tracing due to the latter’s computational complexity [1, p. 415]. As described in Algorithm 2.1, ray tracing iterates over pixels (rays) first and triangles second. Rasterization, on the other hand, instead iterates over triangles first and pixels second (as is described in Algorithm 2.2). Algorithm 2.2:Rasterization rendering loop.

for eachtriangle∈ scene do

for eachpixel∈ output image do iftriangle is closest then

SHADEPIXEL()

Both rasterization and ray tracing are so-called “embarrassingly parallel” algorithms (i.e., they can easily be split into many independent tasks), so in practice their outermost loops are computed in parallel across multiple threads and cores on the GPU, instead of

being iterated over sequentially as they would in a single-threaded process. The theoretical complexity of the two algorithms do not differ much when optimizations are taken into

*_{also called classical ray tracing}

2.2. Light 7 account; however, the computations involved in the ray tracing process are much more time-consuming than the ones in the rasterization process, thereby making ray tracing significantly slower than rasterization.

### 2.1.2

### Hardware Acceleration

Specialized hardware has been developed to speed up the expensive ray tracing–related calculations. In 2018, Nvidia debuted the first consumer graphics cards (theRTX 20 series)

that have hardware acceleration for ray tracing. Together with the simultaneous release of Microsoft’s DirectX RayTracing (DXR) feature to the Direct3DAPI, this has made ray

tracing a viable (albeit still considerably slower) alternative to rasterization for certain effects in real-time applications. At the time of writingDXRhas been used in several higher

budget games: it is used to create global illumination in 4A Games’ Metro: Exodus [3], shadows in Eidos-Montréal’s Shadow of the Tomb Raider [15], and reflections inDICE’s

Battlefield V [28].

### 2.2

### Light

Visible light is a subsection of the electromagnetic spectrum that represents electromagnetic radiation at wavelengths in the range of 400 to 700 nm. It can be generalized as a collection of plane electromagnetic waves, each consisting of synchronized oscillations in the electric and the magnetic field. Those plane waves are transverse waves, since their wave compo-nents’ oscillations are perpendicular to each other and to the direction of propagation. A visualization of an electromagnetic plane wave is shown in Figure 2.2.

z E

B

λ

Figure 2.2:A plane electromagnetic wave visualized as a transverse wave consisting of an electric (red) and a magnetic (blue) wave. E is the electric field component, B is the magnetic field component, zis the direction of propagation, and λ is the wavelength.

For modeling purposes, a photon can be seen as a combination of two parallel plane electromagnetic waves that have the same wavelength—but not necessarily the same phase or rotation. This combined electromagnetic wave can, in turn, be represented by a transverse wave of the oscillations in the perpendicular electric field components Exand

Ey, both of which are perpendicular to the wave’s direction of propagation z. Since the

two electromagnetic waves have the same constant wavelength, the oscillations in the field components Ex and Eywill also have the same constant wavelength as each other, and the

8 Chapter 2. Basic Concepts

### 2.2.1

### Polarization

E_{x}Ey z (a) Linear E

_{x}Ey z (b) Left-hand elliptical E

_{x}Ey z (c) Right-hand circular

Figure 2.3:Three polarized waves with identical wavelengths. The wave is linearly polarized if the wave components’ phase angles are equal (i.e., if their peaks coincide) and elliptically polarized if there is a phase difference and both components have non-zero amplitudes. Circular polarization is a special case of elliptical polarization, which occurs when the amplitudes are equal and the phase difference is exactly ±λ /4.

When looking at a plane that is orthogonal to a transverse wave’s direction, the polariza-tion state of the wave specifies the pattern created by tracing the tip of the wave vector in that plane as the wave moves in its direction of propagation. Since photons are monochro-matic, the waves in the electric field components Exand Eyare inherently synchronized

and the pattern (i.e., the polarization state) will not change as the photon moves through a perfect vacuum. Visualizations of three different polarization states of photons are shown in Figure 2.3.

The angle of the linear polarization depends on the difference between the two electric field components Exand Ey’s oscillation magnitudes (see Figure 2.3a). Likewise, elliptical

polarization is perfectly circular if the magnitudes are equal and the phase difference is exactly±90° (i.e., ±λ/4). Elliptical polarization is considered to be either right- or left-handed depending on the field’s rotation when viewed against the wave’s direction of propagation.

### 2.2.2

### Light Rays and Their Polarization States

For modeling purposes, a light ray is a representation of the photons that flow along a beam of light’s direction. These photons have the same direction and speed (since the speed of light is only dependent on the medium), but they do not necessarily have the same wavelength or polarization state.

2.3. Fresnel Reflectance 9 In non-spectral renderers (such as the ones used in video games), all light values are approximated by combinations of three color components (red, green, and blue). To model the color components as light waves, they can be seen as independent quasi-monochromatic light waves. A quasi-monochromatic light wave contains photons of more than one unique wavelength, but only of wavelengths within a minimal spectrum that is much smaller than the wave’s mean wavelength. Unlike monochromatic light, quasi-monochromatic light can be unpolarized; its polarization state can be modeled as the sum of one completely polarized and one completely unpolarized light wave [7, p. 551].

### 2.3

### Fresnel Reflectance

When unpolarized light is reflected in a dielectric (i.e., non-metallic) material, it becomes
fully or partially polarized unless the surface is perfectly diffuse or the angle of incidence is
zero. If the surface is perfectly smooth, then this behavior can be precisely described by
the Fresnel*_{terms [40, p. 47]. The Fresnel terms are a function of the material’s complex}

index of refraction (IOR) η and the angle of incidence θ. The value of η is dependent on

the wavelength and is defined as η= n+κi, with n > 0 being the simpleIORand κ≥ 0 being

the extinction coefficient.†_{By definition, κ is zero for dielectric materials [29, p. 13].}

For the reflection of a monochromatic or quasi-monochromatic wave at the angle of incidence θ in an optically isotropic (i.e., same properties regardless of rotation) material with the IORη, the Fresnel terms F⊥, F∥, δ⊥, and δ∥‡are defined as

F⊥(θ,η) =

a2+b2−2acosθ +cos2θ
a2+b2+2acosθ +cos2θ ,
F_{∥}(θ,η) =a

2_{+b}2_{−2asinθ tanθ +sin}2

θtan2θ a2+b2+2asinθ tanθ +sin2θtan2θ

F⊥(θ,η),
δ⊥(θ,η) = arctan
⎛
⎝
2bcosθ
cos2_{θ}_{−a}2_{−b}2
⎞
⎠,
δ∥(θ,η) = arctan
⎛
⎝
2cosθ((n2_{−κ}2_{)b−(2nκ)a)}
(n2_{+κ}2_{)}2_{cos}2_{θ}_{−a}2_{−b}2
⎞
⎠,
(2.1)

where a and b are given by the equations

2a2=√(n2_{−κ}2_{−sin}2_{θ}_{)}2_{+4n}2_{κ}2_{+n}2_{−κ}2_{−sin}2

θ ,
2b2_{=}√_{(n}2_{−κ}2_{−sin}2

θ)2+4n2κ2−n2+κ2+sin2θ .

(2.2)

The first pair of equations F⊥ and F∥ determine the amount of the incoming light that is

reflected in the electromagnetic fields that lie perpendicular to and parallel with the plane of incidence. Figure 2.4 shows how, when light is reflected in a dielectric surface at the angle known as Brewster’s angle, F∥becomes zero and the reflected light is consequently

*_{pronounced Freh-nel}

†_{The notation for complex and simple refraction indexes does not appear to be standardized; some texts}

might refer to n as the complexIORand η as the simple one. The same symbol, n or η, is sometimes used for

both terms with a marker (such as an overline) denoting the complex one. This paper will use the notation η = n + κ ithat is used by Wilkie and Weidlich [38, 39].

10 Chapter 2. Basic Concepts fully linearly polarized in the direction perpendicular to the plane of incidence P [7, p. 43]. This is of significance because it means that a polarizing filter will be able to filter out this reflected light (more on this in section 2.4).

P

ˆn

### r

′### r

θB

Figure 2.4:Interaction between a ray of unpolarized light and a dielectric surface at Brewster’s angle. At the angle of incidence θB, the reflection ray r′becomes fully linearly polarized in the

direction perpendicular to the plane of incidence P (which contains both the surface normal ˆn and the incident ray r). The refraction ray has been left out for simplicity.

The second pair of equations δ⊥and δ∥describe the phase retardation the perpendicular

and parallel wave components undergo in the plane of incidence [40, p. 49]. This matters in a polarization renderer because (as mentioned in subsection 2.2.1) the angle, handedness, and shape of a wave’s polarization are all affected by the phase offset. For reflections in metallic surfaces, δ⊥ and δ∥ are not always equal, and this can turn linearly polarized

incident light into elliptically polarized reflected light.

### 2.3.1

### Implementation in Non-Polarizing Real-Time Renderers

Fresnel reflectance is important in non-polarizing renderers as well, since it is what makes surfaces more reflective at grazing angles (i.e., when the angle of incidence θ is near 90°). If information about the reflection’s polarization state is not of interest, then it is sufficient to calculate the average value F =F⊥+F∥

2 to get this effect. The δ Fresnel terms are not

relevant in non-polarizing renderers, since they only affect the shape of the polarization and not the intensity of the light.

In real-time renderers, F is typically computed with Schlick’s approximation [27]. It is defined using the reflection coefficient R0of incoming light that is parallel to the surface

normal (i.e., θ = 0°):

F_{Schlick}= R_{0}+(1−R_{0})(1−cosθ)5. (2.3)

For reflections off of materials in air, R0can be calculated from the material’sIORvalues as

R_{0}= (n−1)

2_{+κ}2

(n+1)2_{+κ}2. (2.4)

In rendering engines, R0is usually referred to as the specular color cspecand is extracted

from a material’s texture files. The Falcor rendering framework that is used in this project calculates cspecfrom a base color cbaseand a metalness value m in the textures as

2.4. Polarizing Filters 11 It is always in the range[0.04,1] since the reflection coefficient of dielectric materials is usually between 0.02 and 0.08. The metalness value m is usually binary (0 for dielectrics, 1 for metals), so cspecwill be equal to 0.04 for dielectrics and be set by the cbasetexture for

metals.

### 2.4

### Polarizing Filters

A linear polarizing filter (also called polarizer) is an optical filter that produces linearly polarized light from unpolarized light. It will also block some of the light, which means that the incoming light’s irradiance will be greater than the outgoing light’s irradiance. This loss of brightness is described by Malus’s law [14, p. 213], which states that the intensity I that passes through a perfect linear polarizing filter is

I= I_{0}cos2φ , (2.6)

where I0is the intensity of the incoming linearly polarized light, and φ (phi) is the angle

between the filter’s polarizing angle and the incoming light’s angle of polarization. If the
incoming light is unpolarized, and thus consists of light polarized in all angles, then the
irradiance will be cut in half (since the average value of cos2_{φ} _{is 1/2). Therefore, the}

exposure settings on a camera with a polarizing filter need to be changed to get an image that is as bright as it would have been without the filter; in a renderer, this is easily achieved by doubling the light’s intensity after it has passed through the filter.

If the incoming light is completely linearly polarized at an angle of exactly ±90° in
relation to the filter, then all of the light will be blocked (since cos2_{(±90°) = 0). This effect}

can be seen by placing a polarizing filter in front of another one and rotating it 90°, or by rotating a single filter in front of anLCD screen (which produces linearly polarized light).

The polarizing filters used by photographers typically consist of a linear polarizer followed by a quarter-wave plate that transforms the linearly polarized light into circularly polarized light. This combination is called a circular polarizing filter (usually abbreviated as CPL on the filter) and is used to prevent images from potentially getting over- or

underexposed as they might with a linear polarizing filter. This is due to metering systems relying on partially reflecting mirrors that, as described by the Fresnel terms, do not reflect the same amount of linearly polarized light regardless of orientation. If the intensity is already known (as it is in a renderer), then there is no need for the quarter-wave plate, and a simpler linear polarizing filter can be used.

### 2.5

### Polarization Calculus

12 Chapter 2. Basic Concepts
In the opinion of Wilkie and Weidlich [39, p. 19], the Stokes vector–based Mueller
calculus*_{is the most convenient model of polarization to use in computer graphics. They}

argue that this is the case because the polarization-describing Stokes parameters do not involve any complex numbers and are more intuitively obvious than the coherency matrices. The Mueller calculus can not represent absolute phase, which the Jones calculus can and which is needed for certain calculations involving coherent light (i.e., light in which all photons have the same direction, wavelength, phase, and polarization state); however, this quantity does not appear to play any notable role in conventional graphics applications.

The Mueller calculus was developed by Hans Müller in the early 1940s. It involves the multiplication of Stokes vectors with Mueller matrices, and for those multiplications to work correctly: the rotation of the light rays needs to be tracked as well (which is detailed in section 2.5.3). For a more comprehensive introduction, see Shurcliff [30, Ch. 8].

### 2.5.1

### Stokes Parameters

The Stokes parameters can be used to describe the polarization state of any monochromatic or quasi-monochromatic light ray. They were first described in 1852 by George Gabriel Stokes [31], and are defined as the four real values s0, s1, s2, and s3:

s_{0}= I ,

s_{1}= Ipcos(2ψ)cos(2χ),
s_{2}= Ipsin(2ψ)cos(2χ),
s_{3}= Ipsin(2χ),

(2.7)

where p

### ∈

[0,1] is the degree of polarization (DOP), ψ (psi) and χ (chi) are the orientationand ellipticity angles in the polarization ellipse (as shown in Figure 2.5), and I is the intensity of the light.

Ex

E_{y}

ψ

χ

Figure 2.5:Polarization ellipse showing the relation between the orientation angle ψ, the ellipticity angle χ, and the shape of the polarization. The direction of propagation z is pointing towards the viewer.

2.5. Polarization Calculus 13
The Stokes parameters are typically grouped together in a Stokes vector ⃗S, which is
defined as
⃗S =⎡⎢⎢⎢_{⎢⎢}
⎢⎢
⎣
s_{0}
s_{1}
s_{2}
s_{3}
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
=
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
I
Q
U
V
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
=
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
↔ + ↕
↔ − ↕
⤡ − ⤢
⟳ − ⟲
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
, (2.8)

where I is the aforementioned intensity of the light, Q and U describe the angle of linear
polarization, and V is the handedness of the elliptical polarization (when viewed against
the light’s direction of propagation).*_{Right-handed polarization is indicated by V} > 0, and

left-handed by V< 0.

The Stokes vector that describes completely unpolarized quasi-monochromatic light is ⃗S= (Ic,0,0,0), where Ic is the same intensity value that would be used for that color

channel in a non-polarizing renderer. Properties of the Stokes Parameters

The degree of polarization p can be calculated from the Stokes parameters as

p= √

s2_{1}+s2_{2}+s2_{3}

s_{0} , (2.9)

with p= 1 describing fully polarized light and p = 0 describing completely unpolarized light. Since p

### ∈

[0,1], the first Stokes parameter s0is bound by the inequalitys_{0}≥
√

s2_{1}+s2_{2}+s2_{3}. (2.10)

Likewise, s1, s2, and s3are all constrained to the range[−s0, s0].

### 2.5.2

### Mueller Matrices

Mueller matrices are used to model how light rays’ Stokes vectors are altered by interactions with various samples (e.g., when the light is reflected off of a surface or passes through a polarizing filter). They are defined as real-valued 4×4 matrices:

M=
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
m_{11} m_{12} m_{13} m_{14}
m_{21} m_{22} m_{23} m_{24}
m_{31} m_{32} m_{33} m_{34}
m_{41} m_{42} m_{43} m_{44}
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
. (2.11)

Mueller matrices can be used to model various polarization elements [5, Sec. 22.17]. For the purposes of this thesis, the matrices of interest are the ones that describe Fresnel reflectance and linear polarizing filters.

*_{In Stokes’ original notation, the modern form of the Stokes vector ⃗S = (s}_{0}_{, s}_{1}_{, s}_{2}_{, s}_{3}_{)}_{would be written as}

⃗

14 Chapter 2. Basic Concepts

Mueller Matrix for Fresnel Reflectance

The Mueller matrix for the Fresnel reflectance is defined as

M_{Fresnel}=
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
A B 0 0
B A 0 0
0 0 C S
0 0 −S C
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
, (2.12)
where
A=F⊥+F∥
2 ,
B=F⊥−F∥
2 ,
C= cos(δ⊥−δ∥)
√
F⊥F∥,
S= sin(δ⊥−δ∥)
√
F⊥F∥,
(2.13)

and F⊥, F∥, δ⊥, and δ∥ are the Fresnel terms from Equation 2.1 [39, p. 23; 14, p. 178].

Mueller Matrix for Polarization Filters

The Mueller matrices for perfect horizontal and perfect vertical linear polarizing filters (called Mlphand Mlpv respectively) are defined as

M_{lph}=1
2
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
1 1 0 0
1 1 0 0
0 0 0 0
0 0 0 0
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
, M_{lpv}=1
2
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
1 −1 0 0
−1 1 0 0
0 0 0 0
0 0 0 0
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
. (2.14)

Other angles for linear polarizing filters can be achieved by rotating the matrices with two rotation matrices. The Mueller matrix for a horizontal linear polarizing filter that has been rotated clockwise by the angle φ is given by the equation

M_{lpφ} =
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
1 0 0 0
0 cos(2φ) −sin(2φ) 0
0 sin(2φ) cos(2φ) 0
0 0 0 1
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
M_{lph}
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
1 0 0 0
0 cos(2φ) sin(2φ) 0
0 −sin(2φ) cos(2φ) 0
0 0 0 1
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
, (2.15)

which can be rewritten as the single matrix

M_{lpφ} =1
2
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
1 cos(2φ) sin(2φ) 0

cos(2φ) cos2_{(2φ)} _{cos(2φ)sin(2φ) 0}

sin(2φ) cos(2φ)sin(2φ) sin2(2φ) 0

0 0 0 0 ⎤⎥ ⎥⎥ ⎥⎥ ⎥⎥ ⎦ . (2.16)

### 2.5.3

### Operations on Stokes Vectors and Mueller Matrices

2.5. Polarization Calculus 15 described by the Mueller matrix M, the new polarization state ⃗S′ is calculated as

⃗S′= M⃗S =
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
m_{11} m_{12} m_{13} m_{14}
m_{21} m_{22} m_{23} m_{24}
m_{31} m_{32} m_{33} m_{34}
m_{41} m_{42} m_{43} m_{44}
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
s_{0}
s_{1}
s_{2}
s_{3}
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
. (2.17)

Two rays of light which are moving along the same ray in space can be combined by adding their Stokes vectors together. The new combined Stokes vector ⃗S is calculated as the sum of the two light rays a and b’s Stokes parameters:

⃗S = ⃗Sa+ ⃗Sb=
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
sa_{0}
s_{a}_{1}
s_{a}_{2}
sa_{3}
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
+
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
s_{b}_{0}
s_{b}_{1}
s_{b}_{2}
s_{b}_{3}
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
=
⎡⎢
⎢⎢
⎢⎢
⎢⎢
⎣
sa_{0}+sb0
s_{a}_{1}+s_{b}_{1}
s_{a}_{2}+s_{b}_{2}
sa_{3}+sb3
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
. (2.18)

Since Stokes vectors can be rotated at some arbitrary angle around their direction of propagation, the above mathematical operations are only permissible if the orientations of the two terms’ local coordinate systems are aligned. However, if one of the Stokes vectors is either fully circularly polarized or completely unpolarized, then the alignment is irrelevant since those states are unaffected by rotations.

Reference Frame Tracking

The rotation of a Stokes vector is represented by reference frames, which can be visualized as the orientation of a plane that is perpendicular to the light wave’s direction; Mueller matrices have both an entry and an exit frame. For the multiplication of a Stokes vector with a Mueller matrix to be permissible, the Stokes vector’s reference frame has to match the Mueller matrix’s entry frame.

In order to make reference frames match, a Stokes vector can simply be rotated. The components of rotated Stokes vector ⃗S′are calculated as

s′ 0= s0, s′ 1= cos(2φ)s1+sin(2φ)s2, s′ 2= −sin(2φ)s1+cos(2φ)s2, s′ 3= s3, (2.19)

where φ is the angle between the old and the new reference frame. Note how the rotation does not change the intensity of the light (s′

0= s0) or the handedness of the polarization

(s′

3= s3), it only changes the orientation of the linear polarization (s′1and s′2) so that the

### Chapter 3

### Related Work

Several mathematical models have been created over the years to simulate light’s polariza-tion state and its interacpolariza-tions with various media. A few new models have been developed in the past five years to increase their physical accuracy in order to develop more accurate microscopes and polarimetric cameras [21, 25, 41]. Polarimetric cameras are used to mea-sure target polarization, and it is therefore important that the cameras themselves do not alter the polarization of the measured light.

Wolff and Kurlander [40] were the first to implement polarization parameters in a ray-traced renderer and used it to simulate polarizing filters and more realistic reflections. They did so using the same formulation of the Fresnel functions used in this thesis, but with the coherency matrix formalism to represent polarization instead of the Stokes-Mueller one. In order to reduce the amount of shading computations they used pre-computed lookup tables for the values of F⊥, F∥, and δ for each combination of material (i.e., complexIOR)

and wavelength (i.e., red, green, and blue) in each scene. If texture mapping is used, as it is in modern renderers, then each object could potentially consist of thousands of unique materials, resulting in thousands of lookup tables for just a single object if this approach were to be used.

Wilkie and Weidlich [37] have proposed a few standardized visualizations of the polarization state to assist in the development of polarizing renderers and to help assure their correctness. The visualizations are based on the Stokes parameters but can also be adapted for coherency matrices since the two formalisms are interchangeable (as was shown by Sánchez Almeida [32]). These visualizations were used during the development of this thesis and helped identify several bugs along the way; the author recommends that they are implemented in any rendering project involving polarization parameters.

Polarization rendering can be used to simulate many visual effects apart from polarizing filters. Weidlich and Wilkie [35] used the Mueller calculus to render uniaxial crystals more realistically. Such crystals produce doubly refracted images that are offset from each other and of different polarization states (one horizontal, the other one vertical).

Völker and Hamann [33] used the coherency matrix formalism to render cut diamonds in real-time. They used beam tracing for their rendering algorithm, which, performance-wise, is more suitable for high-quality rendering of large smooth flat surfaces such as the facets on a diamond than ray tracing is. Beam tracing is similar to ray tracing, but instead of using infinitesimally thin rays, thicker “pyramid-shaped” beams are used instead.

Although no such techniques are used in this thesis, recent developments in denoising and filtering algorithms have been a key factor in making the use of Monte Carlo ray tracing possible in real-time. This is an active area of research, and an in-depth overview of some fairly recent denoising algorithms can be found in the survey by Zwicker et al. [42].

### Chapter 4

### Implementation

Three renderers have been implemented in an application that was created with Nvidia’s research-focused Falcor rendering framework [6].

The Falcor rendering framework was chosen because of its built-in support for DirectX RayTracing and its ease of use. It abstracts some of theCPU-side DirectX 12 code, which

allowed for more of the development time to be spent on theGPU-side shader code. The

use of this thin abstraction layer is not expected to make a notable impact on the results, since the difference between the three renderers is limited to differences in their shader code (and noCPU-intensive techniques are used in the application). Falcor version 3.2.2

was used as the base for all renderers, and only two minor changes (which are detailed in section 4.6) were made to itsAPI. TheGPU-side shaders were written in the High-Level

Shading Language for DirectX (HLSL), and the code for those shaders (along with the

source code for the rest of the application) is publicly available on GitHub.*

All three renderers use Whitted-style ray tracing [36] with one primary ray per pixel, one reflection ray per intersection point, and no refraction or shadow rays. The implementations would have been more immediately applicable in modern real-time applications if Monte Carlo ray tracing had been used; however, that would have been more time-consuming to develop as it requires temporal accumulation and denoising of Stokes vector data. Therefore, the simpler Whitted-style approach was chosen to ensure that the project would be completed on time.

### 4.1

### Shading Model

Shading computations are implemented with the Cook-Torrance microfacet model [9], which was chosen since it is a commonly used shading model that is compatible with Whitted-style ray tracing and polarizing Fresnel functions [39, p. 36]. It uses tiny, perfectly flat, mirror-like surfaces called microfacets to model both smooth and rough materials. The shading of these materials is achieved with the use of one or more bidirectional reflectance distribution functions (BRDFs). ABRDFis a function f(ˆl, ˆv) that defines how much light

is reflected towards a viewer from a point X on a surface, depending on the normalized vectors ˆl and ˆv that point from X to the light source and to the viewer respectively. For more information aboutBRDFs, see Akenine-Möller et al. [1, Ch. 7].

The shading model used by the three renderers can be expressed as theBRDF

f(ˆl, ˆv) = (K_{r}E_{r})cosθ_{r}+∑

k ∈lights(Kd

E_{d}_{k}+K_{s}E_{s}_{k})cosθ_{k}, (4.1)

*_{https://github.com/viktor4006094/DegreeProject}

20 Chapter 4. Implementation
where Kr, Kd, and Ks are the mirror, diffuse, and specular reflection coefficients; Edkand
E_{s}_{k} is the diffuse and specular radiance at the point X from the light with index k; E_{r} is the
radiance of the point that is being reflected in X; θr is the angle between the surface normal

and the reflection vector; θk is the angle between the surface normal and a vector to the

light with index k; and cos is the cosine function clamped to non-negative values.

The approximate shape of the diffuse, specular, and mirror reflection coefficients are shown in Figure 4.1. The coefficients are defined as the amount of incoming light from the direction ˆl that is reflected in the view direction ˆv.

ˆl

K_{d}

ˆv ˆn

(a) Diffuse reflection

ˆl
K_{s}
ˆv ˆn
(b) Specular reflection
ˆl
K_{r}
ˆv ˆn
(c) Mirror reflection

Figure 4.1:Visualization of the diffuse, specular, and mirror reflection terms. The relative length of the vector K represents how much of the light is reflected from that direction to the view direction ˆv. If it is the same length as ˆl, then all of the incoming light from that direction is reflected in ˆv.

### 4.1.1

### Diffuse Reflection Term

For the diffuse component—which in this implementation is assumed to be completely unpolarized—the simple LambertianBRDFfunction

K_{d}= f (ˆl, ˆv) =cdiff

π (4.2)

is used [1, p. 240], with the term cdiffbeing the diffuse color of the material at the intersection

point X.

### 4.1.2

### Specular Reflection Term

The specular component uses the Cook-Torrance microfacet specularBRDF [9]. In its basic

form it is defined as

K_{s}= f (ˆl, ˆv) =D(ˆh,α)G(ˆl, ˆv,α)F(ˆh, ˆv)

4(ˆl⋅ ˆn)(ˆv⋅ ˆn) , (4.3)

where D

### ∈

[0,∞) is a normal distribution function, G### ∈

[0,1] is a geometric shadowing function, F### ∈

[0,1] is a Fresnel function, α is the square value of the material’s roughness, ˆn is the surface normal, ˆh is the half vector between ˆv and ˆl, and the operator⋅ is the dot product.The renderers all use the GGXnormal distribution function from Walter et al. [34]:

D_{GGX}(ˆh,α) = α

2

π((ˆn⋅ ˆh)2(α2−1)+1)2

4.1. Shading Model 21 The correct and exact geometric term G to use is the Smith one [17]. It is defined as

G_{Smith}(ˆl, ˆv,α) = G_{1}(ˆl,α)G_{1}(ˆv,α) (4.5)
where G1can be one of several alternatives. Due to its compatibility with the DGGX normal

distribution term, theGGXvariant of G_{1}is used. It is defined as

G_{1}(ˆv,α) = G_{GGX}(ˆv,α) = 2(ˆn⋅ ˆv)

(ˆn⋅ ˆv)+√α2+(1−α2)(ˆn⋅ ˆv)2

, (4.6)

and the G1(ˆl,α) term is identical to G1(ˆv,α), but with ˆv replaced by ˆl.

The denominator from Equation 4.3 can be used to simplify the geometric shadowing function into a visibility function V :

V(ˆl, ˆv,α) = G(ˆl, ˆv,α)
4(ˆn⋅ˆl)(ˆn⋅ ˆv)= V1(ˆl,α)V1(ˆv,α), (4.7)
where
V_{1}(ˆv,α) = 1
(ˆn⋅ ˆv)+√α2+(1−α2)(ˆn⋅ ˆv)2
. (4.8)

Just as with the G1terms, V1(ˆl,α) is identical to V1(ˆv,α), but with ˆv replaced by ˆl. With

this visibility function, the specularBRDF from Equation 4.3 can be simplified as

K_{s}= f (ˆl, ˆv) = D(ˆh,α)V(ˆl, ˆv,α)F(ˆh, ˆv), (4.9)
and it is this specularBRDFthat is used by all implemented renderers.

What sets the three renderers’ shading apart is the Fresnel function F. The Baseline renderer uses Schlick’s approximation from Equation 2.3; the Polarization renderer uses the Mueller matrix for Fresnel reflectance from Equation 2.12; and the Hybrid renderer uses the latter for primary rays and the former for all reflection rays.

### 4.1.3

### Mirror Reflection Term

Mirror-like reflections are implemented using a single non-stochastic recursive ray that is sent from the viewer to the center of each pixel. Unless the maximum recursion depth has been reached, a reflection ray is cast every time a ray hits a surface. The reflected radiance term Er in Equation 4.1 represents the light that is returned by these recursive rays.

The mirror reflection coefficient uses the same functions as the specularBRDF:

K_{r}= f (ˆl, ˆv) = D(ˆn,α)V(ˆr, ˆv,α)F(ˆn, ˆv), (4.10)
with the visibility term V ’s incoming light vector ˆl replaced by the reflection vector ˆr
(i.e., the mirror reflection of ˆv in the surface normal ˆn), and the half vector ˆh replaced by
the surface normal ˆn (since the two are equivalent for mirror reflections).

The result of the multiplication D(ˆn,α)V(ˆr, ˆv,α)cosθr is clamped to the range[0,1]

22 Chapter 4. Implementation

### 4.2

### Baseline Renderer

The Baseline implementation uses the shading model described in the previous section with Schlick’s approximation from Equation 2.3 as its Fresnel function F, and with all light data represented byfloat3RGB-color vectors.

### 4.2.1

### Data Structures and Rendering Algorithm

The information carried by rays in DXRis defined in programmable data structures called

payloads. A ray’s payload is used to return information from the ray’s operations and is initialized before the ray is cast.

Listing 4.1 shows the payload that is used in the Baseline renderer:coloris the light

data from the ray’s intersection point, andrecursionDepthis used to keep track of how deep

a ray is so that the number of recursive reflections can be limited. Listing 4.1:Ray payload used in the Baseline implementation.

1 struct Payload

2 {

3 float3 color;

4 uint recursionDepth; // How many reflections deep the payload is

5 };

Algorithm 4.1 describes the Baseline implementation’s rendering algorithm, using the data types from Listing 4.1 and the shading terms from Equation 4.1.

Algorithm 4.1:The Baseline implementation’s rendering algorithm.

1: procedure TRACERAY(ray)

2: if rayintersects with geometry then 3: for eachlight source k ∈ scene do 4: ray.color += (KdEdk+KsEsk)cosθk

5: if ray.recursionDepth < max recursion depth then 6: TRACERAY(r′)

▷Cast a reflection ray 7: ray.color += (Krr′.color)cosθ

r

8: procedure MAIN

9: for eachprimary ray r ∈ output image do

10: TRACERAY(r) ▷Cast a primary ray 11: output color = r.color

### 4.3

### Polarization Renderer

The Polarization implementation has the same structure as the Baseline version, but it differs in two key ways: the Fresnel function F uses the Mueller matrix for Fresnel reflectance from Equation 2.12 instead of Schlick’s approximation, and light data is represented by three Stokes vectors and a reference frame instead of afloat3vector.

4.3. Polarization Renderer 23
need to take its rotation into account; therefore, each color channel k in the diffuse term’s
resultingfloat3vector c= (I_{R}, I_{G}, I_{B}) can be added to the specular term’s Stokes vector ⃗S_{k}

for that color channel with the simple equation

⃗S_{k}_{+I}_{k}_{=}⎡⎢⎢⎢_{⎢⎢}
⎢⎢
⎣
s_{k}_{0}+I_{k}
s_{k}_{1}
s_{k}_{2}
s_{k}_{3}
⎤⎥
⎥⎥
⎥⎥
⎥⎥
⎦
. (4.11)

It is worth noting that subsurface scattering (i.e., diffuse lighting) in thin materials can show a significant degree of polarization in some scenarios [8]; however, the influence that might have on the visual effects of a polarizing filter is assumed to be mostly insignificant when compared to the influence of specular reflections.

### 4.3.1

### Data Structures and Rendering Algorithm

The Polarization implementation needs to keep track of more information about the light than the Baseline one does. As such, its ray payload is significantly larger (64 bytes instead of 16 bytes). The three data structures used in the Polarization renderer are shown in Listing 4.2.

Listing 4.2:Ray payload and data structures used in the Polarization implementation.

1 struct MuellerData

2 {

3 // red, green, and blue Mueller matrices

4 float4x4 mmR; 5 float4x4 mmG; 6 float4x4 mmB; 7 }; 8 9 struct StokesLight 10 {

11 // red, green, and blue Stokes vectors

12 float4 svR;

13 float4 svG;

14 float4 svB;

15

16 // local coordinate system's x-axis unit vector

17 float3 referenceX; 18 }; 19 20 struct Payload 21 { 22 StokesLight lightData;

23 uint recursionDepth; // How many reflections deep the payload is

24 };

Just like in the Baseline renderer,recursionDepthis used to limit the number of recursive

reflections, and light is approximated as the sum of three quasi-monochromatic color components (red, green, and blue); however, instead of using afloat3to represent them,

three Stokes vectors are used (one for each color component). Since the polarization state is per-color, it is possible (albeit unlikely) for a ray’s red wavelengths to be completely linearly polarized while its blue wavelengths are completely unpolarized.

24 Chapter 4. Implementation

Algorithm 4.2:The Polarization implementation’s rendering algorithm.

1: procedure TRACERAY(ray)

2: if rayintersects with geometry then 3: for eachlight source k ∈ scene do

4: ALIGNREFERENCEFRAME(KsEsk, ray.lightData)

5: ray.lightData += (KdEdk+KsEsk)cosθk

6: if ray.recursionDepth < max recursion depth then 7: TRACERAY(r′)

▷Cast a reflection ray

8: ALIGNREFERENCEFRAME(r′.lightData, ray.lightData)

9: ray.lightData += (Krr′.lightData)cosθ r

10: procedure MAIN

11: for eachprimary ray r ∈ output image do

12: TRACERAY(r) ▷Cast a primary ray 13: ifpolarizing filter enabled then

14: ALIGNREFERENCEFRAME(r.lightData, screen)

15: APPLYPOLARIZINGFILTER(r.lightData)

16: output color = r.lightData

### 4.3.2

### Reference Frame Tracking

As was discussed in section 2.5.3, information about the rotation of the Stokes vectors has to be maintained in order for operations with them to function correctly. This is done using thereferenceXvector in theStokesLightdata structure shown in Listing 4.2. It represents

the ray’s reference frame’s local coordinate system’s x-axis unit vector ˆx and is equivalent to the Exvector from Figure 2.3.

For the Mueller matrix representing reflection in a point on a surface, ˆx is calculated as the cross product ˆh× ˆv (i.e., it is a normalized vector that is perpendicular to both the microfacet normal and the view direction). The same ˆx can represent both the Mueller matrix’s entry and exit reference frames since their x-axis unit vectors are identical.

Before a color channel’s Stokes vector is multiplied with the Mueller matrix that represents a surface’s reflection component Krfor that color channel, the Stokes vector’s

ˆx vector is rotated to match that Mueller matrix’s entry frame (represented by ˆx′). This is

accomplished using Equation 2.19, where φ

### ∈

[−π,π] is the signed rotation angle between the two vectors. The angle φ is calculated with the help of the direction of the light ˆz*_{as}

φ = atan2(ˆz⋅(ˆx× ˆx′), ˆx⋅ ˆx′) , (4.12) where atan2(a, b) is the two-argument arctangent function that computes an angle in radians between the positive x-axis and the vector pointing from(0,0) to the point (a,b). Figure 4.2 shows a visualization of two reflections’ reference frames and the angle φ that is used to rotate the reflected Stokes vectors between the two reflections.

This reference frame rotation is also performed when a Stokes vector is added to another one (with ˆx′ then representing the reference x-axis vector of the target Stokes vector). No

rotation is needed before multiplying the specular radiance Eswith Kssince all light sources

in the application are assumed to be unpolarized. The resulting Stokes vectors, however, do need to be aligned correctly before they can be added to the ray’slightData.

4.4. Hybrid Renderer 25 ˆh M ˆl ˆz M′ ˆh′ ˆv ˆx′ M′ entry M′ exit ˆx Mentry ˆx Mexit ˆx φ

Figure 4.2:Visualization of a reference frame rotation for a sequence of reflections. The Stokes vectors in the ray that is reflected off of the surface represented by the Mueller matrices M are aligned with the reference frame Mexit, and they need to be rotated by the angle φ to match the entry

frame M′

entrybefore they can be multiplied by the second surface’s Mueller matrices M′.

### 4.4

### Hybrid Renderer

The Hybrid implementation is a combination of the Baseline and Polarization implemen-tations, and makes use of both of their payloads and Fresnel functions. Shading of the primary ray is accomplished with the data structures and functions from the Polarization implementation, and shading of all recursive reflection rays is done with the simpler payload and calculations from the Baseline implementation. Equation 4.11 is used to incorporate the reflection rays’ unpolarized result into the primary ray’s shading computations.

### 4.5

### Implementation Summary

A summary of the differences between the Baseline, Polarization, and Hybrid renderers (which have been described in the preceding sections) is shown in Table 4.1.

Table 4.1: Summary of which Fresnel function F and which light representation is used by the primary and reflection rays in the three renderers.

Primary rays Reflection rays

Renderer F Light representation F Light representation

Baseline F_{Schlick} float4 F_{Schlick} float4

Polarization MFresnel StokesLight M_{Fresnel} StokesLight
Hybrid M_{Fresnel} StokesLight F_{Schlick} float4

### 4.6

### Changes Made to the Falcor API

Two changes were made to the Falcor API to accommodate for the Polarization and

26 Chapter 4. Implementation

16*sizeof(float)to64*sizeof(float), to allow for larger ray payloads (which were

de-scribed in subsection 4.2.1 and 4.3.1); and a metalness value of typefloatwas added to

theShadingDatastruct, to be used to set theIORvalues (which will be described in the next

section).

### 4.7

### Material Texture Limitations

All the test scenes used in this project have textures defining their specular color cspecthat

is used by the Baseline implementation’s Fresnel function, but none of them have textures defining their complexIORη that is used in the Polarization and Hybrid implementations’ polarizing Fresnel function (as discussed in section 2.3).

Because of this, the values of η are approximated in the renderer from the available material data. For dielectrics, the values are set to n= 1.5 and κ = 0.0, which are near the

IOR values of most glass and many other common dielectric materials. For metals, they

are set from a list of pre-defined materials in the application’s interface, with gold being the default option. To prevent the shader compiler from optimizing away theIORvalues,

they are sent to theGPUvia a constant buffer. In the shader code, n and κ are then set to a

blend of these pre-defined values depending on the metalness value in the textures (which is typically either 1.0 or 0.0).

As a consequence of this, all metallic surfaces rendered with the Polarization and Hybrid implementations are given the sameIORvalues, making them look like the same

material (regardless of which type of metal they were supposed to be according to their textures). Since the Hybrid renderer uses the Polarization renderer’s Fresnel function for the first shading point and the Baseline one’s for all subsequent reflection points, this could also cause inconsistencies in how surfaces (metallic ones in particular) appear when viewed directly and when viewed in reflections. In order to prevent this, the R0values used by

the Hybrid renderer’s FSchlick function are calculated from the application-controlledIOR

values with Equation 2.4, instead of being set from the scenes’ material textures as they are in the Baseline renderer.

In a proper renderer, theIORvalues would be saved in material textures just as c_{spec}is

in the current implementation. Metallic materials would need onefloat3texture for their

nvalues and one float3 texture for their κ values. For dielectric materials, it would be

enough with just afloat3texture for n (since κ is zero for dielectrics). A singlefloatvalue

might even suffice if the value of n is deemed to be similar enough in the red, green, and blue wavelengths of light.

### 4.8

### Polarization Demonstration

Figure 4.3 demonstrates how a polarizing filter implemented with Equation 2.16 changes the appearance of a scene in the two polarization-capable renderers. Apart from functioning as a polarizing filter, the implemented filter also doubles the light’s intensity to compensate for the filter’s reduction of brightness (which was discussed in section 2.4).