Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
•Transferir como PPTX, PDF•
16 gostaram•30,922 visualizações
Talk by Sebastien Hillaire at SIGGRAPH 2016 in the Physically Based Shading in Theory and Practice course.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Semelhante a Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
Semelhante a Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite (20)
Mais de Electronic Arts / DICE
Mais de Electronic Arts / DICE (20)
Último
Último (20)
Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
- 1. empowers game creators to shape the future of gaming
- 2. Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite SEBASTIEN HILLAIRE - ELECTRONIC ARTS / FROSTBITE
- 3. 3SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice An EA wide collaboration within Frostbite Sky and atmosphere systems Master’s thesis: Gustav Bodare & Edvard Sandberg (Ghost) Bioware (Mass Effect Andromeda), DICE (Mirror’s Edge Catalyst), Ghost (Need for Speed) Volumetric cloud systems Master’s thesis: Rurik Högfeldt Bioware: Marc-Andre Loyer (Programmer), Don Arceta (Lead Environment Art) and Soren Hesse (Tech Environment Art) The rendering research community
- 4. 4SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Context Physically based rendering in Frostbite Transition done in 2014 [Lagarde&deRousiers14] Huge increase in visual quality
- 5. 5SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Context Previous techniques in Frostbite Sky HDRI captures, color gradients Atmosphere Depth, height fog, screen space light shafts Clouds Panoramic texture + Visual flow [Guerrette14] Translated planar cloud layers Cons Lots of manual work to get components to match Not dynamic, difficult to achieve time of day
- 6. 6SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Real-life sky, atmosphere and clouds With dynamic time of day and evolving weather Photo by DAVID ILIFF. License: CC-BY-SA 3.0
- 7. 7SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Scope and motivation Dynamic time of day and evolving weather at 60 FPS Physically based rendering Meaningful material parameters Decouple material from lighting Unified interactions Sky, atmosphere and cloud interact together Interaction with opaque and transparent objects, global illumination, etc.
- 8. 8SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Mirror’s Edge Catalyst - DICE [Christin16]
- 9. 9SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice
- 10. 10SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline Sky and atmosphere rendering Sun rendering Volumetric cloud rendering Photo by DAVID ILIFF. License: CC-BY-SA 3.0
- 11. 11SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline Sky and atmosphere rendering Photo by DAVID ILIFF. License: CC-BY-SA 3.0
- 12. 12SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky and atmosphere Atmosphere Rayleigh scattering Mie scattering Components
- 13. 13SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Previous work Analytical models [CIESky][Hosek12][Preetham99] Simpler but limited to ground view Spectral rendering [Elek09][Preetham99] Sky atmosphere simulation [Bruneton08] Pre integrated multi-scattering/absorption in 4D LUTs Scattering occlusion from terrain Ignore earth shadow 3D LUTs [Elek09] Improve LUT parameterization [Yusov13] [Bruneton08] [Hosek12]
- 14. 14SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky – algorithm overview Transmittance LUT (h, θv) Single scattering LUT (h, θv, θl) L θl V θv H h Final multi ScatteringLUT (Rayleigh, Mie) Accumulate scattering order Scattered light integration N-order scattering [Bruneton08] [Bruneton08] [Elek09] [Yusov13]
- 15. 15SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Atmosphere simulation coefficients [Nishita93][Riley04][Bruneton08][Adams74][Kutz12] Type Scattering (𝒎−𝟏 ) Extinction (𝒎−𝟏 ) distribution Rayleigh (standard air) σ 𝒔 𝒓𝒂𝒚 = 𝟓. 𝟖𝑒−6, 𝟏. 𝟑𝟓𝑒−5, 𝟑. 𝟑𝟏𝑒−5 σ 𝒔 𝒓𝒂𝒚 𝒆 −𝒉 𝟖.𝟎 𝒌𝒎 Mie (pollution, water, dust, etc.) σ 𝒔 𝒎𝒊𝒆 >= 3𝑒−6 a × σ 𝒔 𝒎𝒊𝒆 𝒆 −𝒉 𝟏.𝟐 𝒌𝒎 Ozone --- σ 𝒂 𝒐𝒛𝒐 𝒆 −𝒉 𝟖.𝟎 𝒌𝒎 a = 1.11 σ 𝒂 𝒐𝒛𝒐 = 𝟑. 𝟒𝟐𝟔, 𝟖. 𝟐𝟗𝟖, 𝟎. 𝟑𝟓𝟔 𝒎−𝟏 × 𝟔𝒆−𝟓% [Kutz12] [Bruneton08]
- 18. 18SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Ozone absorption “Essential […] to reproduce the blue of the zenith sky” [Adams74][Kutz12] Ozone absorption: “Essential […] to reproduce the blue of the zenith sky”[Adams74]
- 19. 19SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky and aerial perspective rendering Render the sky using ScatteringLUTs(h,θv,θl) Camera aerial perspective volumes 32x32 x 16 depth slices Integrate scattering/transmittance for each slices Easy to apply on transparents (per vertex evaluation) Scattered luminance Transmittance
- 20. 20SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Time of day - Moving sun LUTs + moving sun automatically updates: Sky Aerial perspective Triggers updates of Sky environment map Local reflection volumes Global illumination Light probes Sky environment lighting debug
- 21. 21SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky rendering performance LUT updates Needed when changing atmospheric properties Distribute evaluation over several frames Each scattering order, sub table parts While updating, interpolate last results Complete update frame count Mean frame GPU cost 1 3.50 ms 19 0.22 ms Component GPU cost AP camera volume 32x32x16 0.05 ms Sky/AP rendering 0.42 ms LUT updates with 4th order scattering measured on XB1 Sky/Atmosphere rendering cost on XB1 720p
- 22. 22SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline Sun rendering Photo by DAVID ILIFF. License: CC-BY-SA 3.0
- 23. 23SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sun disk luminance and ground illuminance In Frostbite, artists specify illuminance E for sun at zenith for sun solid angle 𝜴 [Lagarde&deRousiers14] E = 100000 Lux (default sun) L outer space luminance L outer space luminance E current sun position
- 24. 24SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Atmosphere transmittance on sun luminance = apply transmittance LUT per pixel to outer space luminance
- 25. 25SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline Volumetric cloud rendering Photo by DAVID ILIFF. License: CC-BY-SA 3.0
- 26. 26SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Previous work Mesh + HyperTexture [Bouthors08] High quality but overall expensive Splatting based rendering [Yusov14] Particle based, depth aware blending Specific look did not match our target Cloud layer [Schneider15] Completely dynamic solution Lighting, shadowing, weather [Yusov14] [Bouthors08] [Schneider15]
- 27. 27SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Volumetric cloud rendering Similar to [Schneider15] Cloud layer with weather animation Dynamic lighting and shadowing Perlin-Worley noise for realistic cloud details Beer-Powder Improvements Scattering energy conservation 2-lobe phase functions Time of day and weather interaction with sky and aerial perspective [Schneider15]
- 28. 28SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Volumetric cloud lighting components Transmittance Ambient Sun scattering Cloud self shadow Phase function Reality
- 29. 29SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Scattering integration improvement Ray marching: improved integration from [Hillaire15] float3 scattering = float3(0.0, 0.0, 0.0); float transmittance = 1.0; for (s= 0; s< StepCount; ++s) { 𝑆 = sample scattered luminance sampleTransmittance = 𝑒−𝝈𝒕 𝐷 = sample transmittance scattering += 𝑆 * transmittance transmittance *= sampleTransmittance } Wrong? Analytically integrate lighting w.r.t. transmittance over depth D (ShaderToy demo) 0 𝐷 𝑒−𝝈𝒕 𝑥 × 𝑆 𝑑𝑥 = 𝑆−𝑆×𝑒−𝝈𝒕 𝐷 𝝈𝒕 𝝈𝒕 : extinction over range 𝑆 : scattered luminance 𝐷 : integration distance
- 30. 30SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice scattering += 𝑆 * transmittance transmittance *= sampleTransmittance Non energy conservative
- 31. 31SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice transmittance *= sampleTransmittance scattering += 𝑆 * transmittance Over attenuation dark media
- 32. 32SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Analytic integration 𝑆−𝑆×𝑒−𝝈𝒕 𝐷 𝝈𝒕 Energy conservative
- 33. 33SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Default scattering integration, 21 samples Default scattering integration, 512 samples Improved scattering integration, 21 samples Improved scattering integration, 512 samples
- 34. 34SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function Two-lobes HG phase function 𝒑𝒉𝒂𝒔𝒆 𝑔0, 𝑔1, α, 𝜇 = 𝑙𝑒𝑟𝑝(𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔0, 𝜇 , 𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔1, 𝜇 , α) Tried best fit Back-scattering still not very visible Would require multi-scattering… Two-lobes HG phase + artist control More freedom to control cloud style Default: 𝑔0 = 0.8, 𝑔1 = −0.5, α = 0.5 [Bouthors08] Cloud Henyey/Greenstein Rayleigh 0.0 0.5 1.0 1.5 2.0 2.5 3.0 10 4 0.01 1 100 Two-lobe HG fitted to cloud phase Angle Phase
- 35. 35SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.3 0.2 0.1 0.1 0.2 0.3 HG phase with g=0.8
- 36. 36SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 0.5 1.0 1.5 2.0 0.3 0.2 0.1 0.1 0.2 0.3 2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5
- 37. 37SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.3 0.2 0.1 0.1 0.2 0.3 HG phase with g=0.8
- 38. 38SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 0.5 1.0 1.5 2.0 0.3 0.2 0.1 0.1 0.2 0.3 2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5
- 39. 39SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Aerial perspective and clouds Must look consistent under all time of day and weather conditions Visual mismatch (missing fog on clouds) Clouds without aerial perspective
- 40. 40SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Aerial perspective and clouds Per sample: expensive to evaluate Instead sampled once on cloud front interface Compute mean depth on cloud interface weighted by transmittance
- 41. 41SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Aerial perspective on cloud Clouds with aerial perspective
- 42. 42SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Cloud coverage and scattering influence Cloud hemisphere sampling around camera • Integrate cloud luminance • Mean cloud transmittance New AP scattered luminance = oldAP x transmittance + luminance
- 43. 43SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Cloud coverage and scattering influence RealityClouds with aerial perspective interactions
- 44. 44SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Volumetric clouds performance 720p 900p Hemisphere sampling: scattering & coverage 0.090 ms 0.090 ms Main view (with temporal re-projection) 0.720 ms 1.090 ms Planar reflection (optional) 0.035 ms 0.048 ms Total 0.835 ms 1.228 ms Measured on XB1 Cloud main view resolution = 640x360 (720p/2) Planar reflection resolution scale = 128x72 (720p/10) Default quality seen in this presentation Looking toward horizon
- 45. 45SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Final result Sun rise with animated clouds
- 46. 46SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Final result Mars blue sunset (eye-balled atmosphere) [NASA]
- 47. 47SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Conclusion Physically based sky, atmosphere and cloud rendering solution Participating media material definition Physically based and energy conservative scattering formulation Supports dynamic time of day and evolving weather Simulation of interactions between sky, sun, aerial perspective and clouds Default: earth/sun/moon physical values Simulation parameters can all be changed to simulate extra-terrestrial worlds Practical use in Mirror’s Edge Catalyst: [Christin16]
- 48. 48SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Used widely in production by EA Games Need for Speed, Ghost Mass Effect Andromeda, BiowareFIFA 2017
- 49. 49SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Future work Aerial perspective scattering shadowing by opaque and translucent clouds Can’t use epipolar sampling min/max optimisation due to translucent clouds Directional sky ambient by using cloud local gradient as main direction? Convolution of sky envmap with phase function for cloud ambient lighting Dynamic weather texture for cloud animation GI and reflection volumes update Still a problem when it comes to delay and performance A complex problem with different priorities for each games
- 50. 50SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice References [Adams74] The influence of Ozone and Aerosol on the Brightness and Color of the twilight Sky, JAS 1974. [Bouthors08] Interactive Multiple Anisotropic Scattering in Clouds, I3D08. [Bruneton08] Precomputed Atmospheric scattering, EGSR 2008. [Christin16] Lighting the City of Glass, GDC 2016. [CIESky] Spatial distribution of Daylight [Elek09] Rendering Parametrizable Planetary Atmospheres with Multiple Scattering in Real-time, CESCG 2009. [Guerrette14] Moving the heavens, GDC 2014. [Harris02] Real-Time Cloud Rendering for Games, GDC 2002. [Hestroffer98] Wavelength dependency of the Solar limb darkening, Astronomy and Physics 1998. [Hillaire15] Physically Based and Unifier Volumetric Rendering in Frostbite, SIGGRAPH 2015. [Hosek12] An Analytic Model for Full Spectral Sky-Dome Radiance, SIGGRAPH 2012 [Kutz12] Blog post, 2012. [Lagarde&deRousiers14] Moving Frostbite to PBR, SIGGRAPH 2014. [NASA] http://www.nasa.gov/multimedia/imagegallery/image_feature_347.html [Nishita93] Display of the Earth Taking into Account Atmospheric Scattering, SIGGRAPH 1993. [PBR] Physically Based Rendering book, www.pbrt.org. [Preetham99] A Practical Analytic Model for Daylight, SIGGRAPH 99. [Riley04] Efficient Rendering of Atmospheric Phenomena, EGSR 2004. [Schneider15] The Real-time Volumetric Cloudscapes of Horizon: Zero Dawn, SIGGRAPH 2015. [Yusov13] Outdoor Light Scattering, GDC 2013. [Yusov14] High-Performance Rendering of realistic Cumulus Clouds using Pre-Computed Lighting, HPG14. [Wenzel07] Real time atmospheric effects in game revisited, GDC 2007. [Wrenninge10] Oz: The Great and Volumetric, SIGGRAPH 2010.
- 51. 51SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Battlefield, Mass Effect, Need for Speed, FIFA, Plant vs Zombies, and many more. We are hiring!
- 52. 52SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Questions? Thanks! The Frostbite rendering Team and game teams Stephen Hill and Stephen McAuley for the course organisation All the reviewers Eric Bruneton and Fabrice Neyret For further discussions, get in touch sebastien.hillaire@frostbite.com https://twitter.com/SebHillaire Additional notes and Mathematica notebook to follow soon…
- 53. 53SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Bonus slides
- 54. 54SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky – how many scattering orders? Single Scattering 2nd scattering order 4th scattering order
- 55. 55SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Scattering texture packing ScatteringLUTs(h, θv, θl) is not storing (RayScat 𝒓𝒈𝒃, MieScat 𝒓𝒈𝒃) But (RayScat 𝒓𝒈𝒃, MieScat) MieScat 𝒓𝒈𝒃 = RayScat 𝒓𝒈𝒃 × MieScat RayScat 𝒓 × σ 𝒔 𝒓𝒂𝒚 𝒓 σ 𝒔 𝒓𝒂𝒚 (RayScat 𝒓𝒈𝒃, MieScat 𝒓𝒈𝒃) (RayScat 𝒓𝒈𝒃, MieScat)
- 56. 56SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Height fog Uses scattered luminance at horizon Seamless sky/fog transition Per pixel coverage Per pixel phase function Approximations Assumes low altitude fog Ignores opaque shadows and self shadowing Horizon P ScatteredLightP ≈ ScatteredLightHorizon
- 57. 57SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sun disk limb darkening Attenuate luminance near edge of sun disk (Default = sun model [Hestroffer98]) (Very low exposure) Nolimbdarkening Sunmodel Stronger
- 58. 58SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Night time Moon and stars rendered as sprite using celestial system Moon Angular radius ≈ 0,24 to 0,28 degree Luminance ≈ 2500 cd/m2 Secondary sky/atmosphere scattering source Stars Luminance? Angular radius?
- 59. 59SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Night time Sun set
- 60. 60SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Ambient contribution Ambient component Capture the sky environment without sun disk Integrate luminance to scatter Approximations Phase function as uniform Single environment color ignoring cloud self occlusion
- 61. 61SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Multi-scattering approximation Use [Wrenninge10]: multi-octaves single scattering / extinction / phase Without multi-scattering
- 62. 62SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Multi-scattering approximation Use [Wrenninge10]: multi-octaves single scattering / extinction / phase With multi-scattering
- 63. 63SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Cloud coverage and scattering influence Thick cloud + aerial perspective + hemisphere coverage/scattering Thick cloud only Clouds do not fit Thick cloud + aerial perspective Aerial perspective too bright
- 64. 64SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sunny 16 validation Sun illuminance at ground level: 100000 Camera: ISO 100, aperture f/16 and shutter speed 1/125
Notas do Editor
- Go full screen please: slides have lots of animations Do not hesitate to contact me: sebastien.hillaire@frostbite.com https://twitter.com/SebHillaire
- Hi and welcome to this course. I am Sebastien Hillaire from Frostbite and I will present to you our work on physically based sky, atmosphere and cloud rendering.
- But before we start, I would like to mention that this work has been conducted with several engineers, artists and game teams. So thank you very much! I also would like to thanks the rendering research community for the useful results and also the source code which really saves our implementation/investigation times. If you want your tech to have more chance of being used, you know what to do.
- So first a bit of context. Two years ago, we moved Frostbite to be a physically based rendering engine. This resulted in a huge raise in quality as you can see in this NFS picture. It is hard to tell which one is from Frostbite and which one is real.
- Despite all that, environment lighting was still achieved using mostly static procedural composition: Sky could only be an HDRI capture or a procedural composition of panoramic color gradients. Atmosphere was achieved using depth and height fog with screen space light shafts as an option. Clouds could also be achieved with high resolution planar or panoramic textures with motion flow as an option. All these techniques are great. But, for instance, it is hard to achieve dynamic time of day. Lots of work needs to be put into them to make sure all the different static elements match and interpolate well.
- These technologies are all good but both we and game teams wanted to be able to achieve dynamic time of day and evolving weather. This picture shows what we wanted to be able to achieve: a realistic physically based sky simulation that would support dynamic time of day, dynamic lighting according to sun position on our meshes, and clouds with matching atmospheric lighting. The picture is from https://en.wikipedia.org/wiki/Aerial_perspective.
- So we wanted all that, but of course this should be working for 60 frame per second titles and (as always) run as fast a possible. We wanted it to be physically based: this means that participating media materials are decoupled from the light sources and parameters are meaningful and represent real physical properties. Also, we wanted all these visual features to interact together coherently, and be as realistic and unified as possible by default. For instance, the cloud becoming larger and thicker should affect the light scattered through the atmosphere and in turn also affect global illumination.
- This video shows the result that can be achieved with the physically based sky as used in Mirror’s Edge Catalyst. You can see the sun sets and everything is affected by the change of in lighting: sky, sun light, global illumination, lights shaft, particles, etc.
- This video shows the early results that can be achieved using the physically based sky and dynamic clouds within an unannounced Bioware title.
- This is the outline of my presentation. I will first describe how we render physically based sky and atmosphere. I will then quickly describe how we render our sun disk Finally I will present how we render volumetric clouds and how they interact with other components.
- 3:00
- To be able to render sky and atmosphere we need to take into account several components. We must first consider the atmosphere as a slab around the earth with a exponential distribution of air molecules in it. Light interacting with air particles that are much smaller than the light’s wavelength results in the highly wavelength dependent Rayleigh scattering. For the earth’s atmosphere, wavelengths of blue light are scattered more and that is why the sky appears blue during the day. However when the sun is at the horizon, light will have to travel a longer distance in the atmosphere and most blue light will be scattered away. Blue light will not travel as far as the green and red light in the atmosphere. That is why sunset and sunrise appear reddish. Another important component of the atmosphere is the large particles concentrated near the ground. The concentration of these particles depends a lot on weather conditions, or pollution for instance. These particles cause the mostly wavelength independent Mie scattering. So the phase function describing how light will scatter is usually not uniform but biased toward the direction of the light. This phenomenon will cause the bright halo we usually see around the sun.
- Several sky rendering models exist. A few analytical models have been proposed. These models are however limited to ground view or atmosphere parameters that can’t be changed freely to simulate extra terrestrial planets. The spectral rendering of the sky has also been implemented for an improved accuracy but we cannot afford that for games on today’s platforms. Bruneton proposed a model to render sky and atmosphere by pre-integrating transmittance and scattering in look up tables. Using these tables, it is also possible to cheaply approximate multi-scattering. Some nice extensions to that work have been proposed: Elek suggests ignoring earth shadow and thus reducing the LUT dimensionality to 3 instead of 4. Yusov proposed an improved parameterisation that increases the scattering accuracy. In these models, the participating media material parameters are nicely decoupled from lighting. In our case, we have chosen the following options from the framework proposed by Bruneton with the highlighted improvements: RGB rendering, not spectral 3D LUT simplification from Elek Improved parameterization from Yusov
- I will only quickly expose how the LUTs are computed. Please refer to [Bruneton08] and [Elek09] for implementation details. If you consider a perfectly spherical earth and atmosphere as well as a viewer position. The first parameter is the viewer height in the atmosphere. The second parameter is based on the sun / zenith angle. The third and last parameter is the view direction / zenith angle. From these inputs, transmittance and single scattering tables can be built. Then as proposed by [Bruneton08], multi-scattering can be integrated by iterating over all these inputs to create a final multi=scattering table.
- What coefficients should we use? We have gathered them all here from these reference. The scattering coefficient depends on temperature and pressure but these are often ignored and given by their mean value. You can see here the wavelength dependent Rayleigh scattering as well as the wavelength independent Mie scattering coefficients describing how light bounces off particles in the atmosphere. We also need extinction describing how much light will pass through after it travels through a certain path according to absorption and scattering. You can also see the distribution we use. You can also notice that we use ozone which is not used traditionally. It helps in maintaining the blueness of the sky at sunset for instance. The coefficients have been recovered from measured data. After discussing with Bruneton, we have learned that these coefficients do not represent scattering gathered from all wavelengths. Instead the scattering coefficients for R, G and B were only taken for the corresponding wavelengths 680, 550 and 440nm.
- Light Rayleigh scattering Default sky Rayleigh scattering Heavy Rayleigh scattering
- No Mie scattering Default sky Mie scattering Very dusty atmosphere
- Ozone is particularly important to keep the zenith sky blue during sunset and sun rise. You can see the difference in these pictures. Ozone should be concentrated 32km up in the sky. But this was giving us unexpected results (probably due to using RGB instead of a more complete spectrum). Instead, in our simulation, ozone follows the same atmosphere distribution as the particle distribution for Rayleigh scattering. For more details please refer to the course notes.
- So how do we render the final frame? First we render opaque objects. Then we render the sky using the scattering look up table. We also prepare an aerial perspective volume fitted on the camera frustum. Aerial perspective is the effect of the atmosphere on objects (== atmosphere fog). Two textures contain the scattered light as well as the transmittance. Please note that this does not support scattering occlusion so we will not get any volumetric light shafts. Getting these sort of light shafts to very a good level of quality would have added a cost which our 60fps titles were not willing to pay. The aerial perspective is applied during the sky pass on opaque objects but also on all the transparent objects per vertex. This is because it is important for everything to fit into the scene. This is a low resolution texture but since we do not bake the phase function into it (applied per pixel later), we do not suffer from the low froxel resolution. The advantage of this approach is that this texture can cheaply be sampled per vertex on transparent objects to make sure these type of entities also fit into the world.
- Since we have the look up table ready, the sun can be freely moved around and as a result the sky and aerial perspective lighting will automatically be updated and match. We do have to update the sky environment map, local reflection volumes as well as our dynamic GI solution, as you can see in this video. We also take into account atmosphere transmittance on the sun illuminance and light shafts. I will tell you more about that later. A good example if this was also visible in the introductory videos at the beginning of this presentation.
- About performance: Once the LUTs have been computed, the sun can be moved freely. But changing an atmosphere parameter will trigger a LUT update that can be costly. To reduce this cost, we evaluate the integration over several frames. We even update only parts of LUTs during a frame. While we’re doing this, we can interpolate the latest result and catch up this way until done. Doing this, the GPU cost of evaluating multi-scattering is greatly reduced to an affordable cost. You can see here the cost for different distributions and the final sky and aerial perspective rendering cost. The distributed rendering is very valuable in order to reduce the cost.
- 11:15
- As mentioned previously, artists specify sun illuminance at ground level (easier for artists to predict the final look). From that, transmittance at zenith and sun solid angle, we can recover the sun outer space luminance. Once we know the sun outer space luminance, we can recover the final illuminance at ground level that is used to light the scene for the current sun direction. --- In course notes: We suppose the sun is usually very small in the sky so we can approximate the integral over the solid angle by a simple product with the solid angle. We apply the following process to recover sun luminance and illuminance: Evaluate atmosphere transmittance at zenith Use equation 1 to approximate outer space luminance according to this transmittance and sun solid angle Evaluate transmittance for the sun centre direction Use equation 2 to approximate sun illuminance at ground level. Code: solidAngle = FB_TWO_PI * (1 -cosf(angularRadius)) OuterSpaceSunLuminance = sunIlluminance / (sunSolidAngleInRad * zenithTransmittance)
- Now that we have the sun outer space luminance we can use it to render the sun disk. But without taking into account the atmospheric transmittance, the sun disk would look to bright. To solve this we simply apply the atmospheric transmittance per pixel to get the final luminance to store in our light buffer. As you can see in this picture, it is mandatory to get the correct look for the sun.
- 12:45
- Many cloud rendering techniques have been proposed. Harris proposed using impostors composed of particles. This is very performant thanks to the angle based update of impostors but visual fidelity is not where we would expect nowadays. Bouthors render clouds with a mix of meshes, hyper textures and ray marching. The final result is of very high visual quality. But overall it remains expensive and the data is not straightforward to author or edit. Yusov proposed a splat based approach using pre integrated scattering on particles elements. However the final look is very specific and we did not want to follow that visual target. Finally last year Schneider proposed a layer based ray-marched approach that is giving detailed results. It was also completely dynamic with spatial control and weather and applicable to real time games. We have decided to base our approach on this technique.
- We basically use the same approach to the one described by Schneider. I am simply going to describe extra interesting details about lighting, scattering energy conservation as well as dynamic weather and time of day interactions.
- The clouds are authored using regular physically based scattering, absorption and phase function material parameters. The resulting visual components are the following: - Transmittance represents the occlusion of what is behind the clouds - Ambient lighting (for example, from the sky) will be scattered by the cloud. - Light scattering from the sun also needs to be taken into account. However, by itself, it does not look good: one needs to evaluate the clouds self-shadowing according to extinction [PBR] [Hillaire15]. - Finally, as for the atmosphere, light can bounce non uniformly and usually, clouds have a very strong forward scattering resulting in the so-called silver lining. Compared to reality, what is missing here is high resolution and multi-scattering simulation.
- To be able to render clouds, you need to ray march them to evaluate transmittance and the scattered light toward the camera. We use the same approach I have described last year to improve the scattering integration. When we ray march, we have to integrate scattering and also transmittance and both are done in two operations. It is not correct to do one or the other first. So I proposed integrating scattering according to the extinction coefficients over the current range D. There is a small ShaderToy demo you can use as reference (https://www.shadertoy.com/view/XlBSRz).
- The issue is very visible on dense cloud, especially with ambient lighting. This is what clouds look like when updating scattering before transmittance. You can see that it is definitely not energy conserving especially when it come to the scattering of the ambient term and the end result is too bright.
- This is what clouds look like when updating transmittance before scattering. It is overall too dark and clouds look a bit like dirt. Instead, when very dense, they should converge to what an opaque diffuse surface would look like.
- And now the more correct result when using the improved integration. Dense participating media can now look more consistent and be energy conserving.
- It is also important to get correct results when using less samples. At the top left is the default incorrect scattering integration with 21 samples. If you use more samples, it will converge to the appropriate solution. With the improved scattering integration, it looks correct even with a low number of samples.
- The phase function is a very important component of the cloud lighting. It will give you all the complex lighting effects that clouds have such as the silver lining, pseudo-specularity and glory haloes. You can see a typical cloud phase function in blue here. We wanted to approximate it with a 2-lobe Henyey-Grenstein phase function, but in this case the backward scattering was still visually very low (as you can see on this figure). We think that this is because we are missing a proper multi-scattering simulation. In the end, we gave artists the possibility to tweak the 2-lobe phase functions, that are then combined and normalised for the purpose of energy conservation. This way artists can control the amount of forward and backward scattering they want. This is also coherent with real clouds since depending on the radius distribution of water particles, the shape of the phase function can change a lot.
- This is how clouds look like with a single forward phase function.
- This is how they look with a 2-lobe phase function. They look a bit less intense since some of the light will be sent backward and that is expected in order to be energy conservative.
- This is how clouds look when the sun is behind the camera and only forward scattering is used. There is only the ambient contribution which makes them look flat.
- When using a bit of backward scattering, you can now see that details are more visible and clouds look closer to what one would expect in reality.
- Now that we can render and animate clouds, we need to make sure they look consistent under all conditions. Especially when dynamic weather is used by making the cloud layer thicker. You can see here where we started: without aerial perspective applied on clouds during heavy weather condition changes, it looks like they do not fit in the environment.
- However, evaluating the AP for each ray marched samples would raise the rendering cost. Instead we propose to compute a weighted average of the cloud front interface evaluated as a function of the sampled density and depth. It is weighted by the transmittance to make sure the samples stay at the front of the cloud interface as we are raymarching deeper into it. This final depth can be used to sample the AP once.
- On this video you can now see the aerial perspective being applied on the clouds. Much better! This could be usable but something is still off. When cloud coverage increases, the aerial perspective should be affected and no longer scatter the sun/sky color but only the light scattering through the clouds.
- So we need to take into account the cloud coverage when evaluating the AP. But the clouds are also scattering light from the sun and the sky above. So we also need to take into account cloud scattered luminance as another light source lighting the aerial perspective. To this aim we sample the hemisphere around the camera position on the ground and integrate scattered luminance and cloud coverage. Assuming a uniform phase function, we will occlude previous AP scattering resulting from the sun by the cloud coverage and add the cloud scattered luminance back. The AP transmittance does not change however since this only depends on the particle density in the air which has not changed. This is considered as a global effect and a very cheap way to get it. A more involved way to achieve this would be to evaluate cloud shadow and simulate volumetric occlusion in the atmosphere. That would be more consistent for very spatially varying cloud coverage.
- And this is the result: you can notice the bright yellow Mie scattering getting progressively occluded and replaced with a dark blue color coming from the integrated cloud scattered luminance. It has a string blue tint due to the sky’s blue ambient color. You can see a real example in this picture. You could also get a more white atmosphere depending on the density of your clouds and how the sun light penetrates through them. Cloud pictures: http://cloud-maven.com/wp-content/uploads/2013/09/DSC05456.jpg http://www.wolken-online.de/images/wolken/nimbostratus/nimbostratus_pannus_2.jpg https://upload.wikimedia.org/wikipedia/commons/4/4f/Ns1.jpg
- This is the final performance we get on Xbox One for clouds rendered at half resolution in the main view while looking at the horizon with the quality seen in this presentation.
- 21:15 This is a video showing the final sky, atmosphere and clouds all integrated during a sun rise.
- Just for fun, I tried to simulate a Martian sunset using our sky model. I have been able to reproduce the red day color and the blue sunset due to Mars’ atmosphere scattering the red wavelengths more. See the following reference for Mars’ blue sunset/rise: http://www.nasa.gov/multimedia/imagegallery/image_feature_347.html http://www.cbc.ca/news/technology/blue-mars-sunset-captured-in-nasa-video-1.2970932 http://www.npr.org/sections/krulwich/2012/04/25/151376553/a-most-peculiar-sunset Curiosity: http://www.nasa.gov/sites/default/files/thumbnails/image/pia19142_malhi-mojave.jpg
- I have presented to you the current solution we have in Frostbite to render sky, atmospheric effect and clouds. We follow a physically based approach by being energy conservative and using physically based participating media material definitions. We also simulate many interactions between all these components under dynamic time of day. And by default we provide our artists with earth-like setup.
- The physically based sky and atmosphere simulation has already used in production by several game teams. The volumetric clouds technique is currently being used in production by unannounced titles.
- There is lots of room to improve this technology. Some of the games benefiting from it were under a 60fps constraint. These are examples of improvement we would like to investigate to improve the visual fidelity for our 30fps titles.
- Some references used in that presentation and this work.
- We are hiring so do not hesitate to get in touch if you are interested
- Thank you for your attention! Now it is time for questions. And if we do not have time do not hesitate to contact me online or get in touch after the course.
- So how many scattering orders do you need? It will depend on the performance and update reactivity you need (I’ll tell you more on that later). You can see the different results here for single scattering, 2nd order and 4th order multi-scattering. This can be selected as a setting by each game. Frostbite games usually go with the default 4th order scattering.
- Rayleigh and Mie scattering could be straightforwardly stored into two RGB result textures. However, this increases the memory bandwidth requirements when rendering the sky pass, integrating multiple scattering or applying the aerial perspective. Mie scattering is non wavelength dependent. But the scattered light will still be colored due to Rayleigh scattering and extinction, etc. To solve this issue we pack Mie scattering as single value representing its R coordinate. And recover its color right at the end when evaluating the final scattered luminance using a bunch of ALUs. This approximation works well when scattering coefficients are close to extinction (no absorption). but it is still a rough approximation for the following cases: scattering << extinction coefficients must not be == 0
- We can also apply height fog. To achieve this aim we sample the scattered luminance once at the horizon as our artists did not want seams at the horizon by default. We apply it as height fog scattered luminance taking into account the height fog coverage and phase function per pixel. From this diagram, we are considering the scattering light for both paths to be the same - only the coverage and phase function vary (with the coverage influencing how much luminance is sent back towards the camera).
- If the sun is simply rendered using transmittance, it will look like a perfectly solid disk when. One interesting visual feature of the sun is limb darkening. It makes sun edges appear less bright because the luminance coming from its centre has first to travel through a layer of media absorbing some of it. So we apply a wavelength dependent luminance reduction based on an astronomical model. We will provide the function in the course notes. The default sun limb darkening is not very strong but artists can change the coefficients to make it stronger if they need. Note that we ignore the luminance change when we evaluate luminance/illuminance mentioned in the previous slide. NOTE: The picture looks different from the previous slide because the view angle, exposure and sun elevation are different. Models: http://www.physics.hmc.edu/faculty/esin/a101/limbdarkening.pdf http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_query?1994SoPh..153...91N&defaultprint=YES&filetype=.pdf
- During the night, we can render the moon as a sprite using our celestial system. We use the following moon luminance. The moon is used as a secondary light source for the sky and atmosphere. And of course we can sprinkle stars in the sky (but we have never found stars mean angular radius and luminance).
- You can see here a sunset in progress progressively revealing the moon and then the stars. The moon and stars are rendered using our celestial system and the moon also acts as a very dim second scattering light source
- Cloud ambient lighting is very important. We compute cloud ambient lighting from the sky contribution only (without the sun disk). We render it in a small 64x64 buffer with uniform sphere sampling. This is used to compute cloud ambient lighting component using an average of all the incoming lighting direction(assuming a uniform phase function of 1/4PI ). This approximates ambient lighting using a uniform environment. Under investigation: we can approximate cloud self occlusion using the Beer-Powder function if necessary.
- We do have a multi-scattering approximation implemented as proposed by Wrenninge for film production. Several octaves of scattering are evaluated and each octave features a more diffuse phase function and less dense media. This picture shows clouds without multi-scattering
- This picture shows the result when the optional multi-scattering model is enabled. You can see that the light interaction is more spread out and can travel deeper in the cloud while still maintaining most of the sharp silver lining effect.
- This is a summary of the three conditions. No aerial perspective on the left. With AP applied on cloud in the middle. And finally with the AP being affected by cloud coverage and especially the cloud scattered color which is often overlooked.
- You can see here a shot following the Sunny 16 rule with the sun at the zenith showing the result on a variety of opaque materials and translucent particles with volumetric shadowing.