RENDERING, SHADERS, GPU compute, & optimization

Raven software - call of duty (2021 - Present)

I currently work at Raven Software as a Software Engineer on the rendering team. Due to the nature of working on internal projects/code-base, I am unfortunately limited in the technical details I can write about here.

However, as a general overview: Call of Duty is built upon an in-house game engine written in C++ featuring a multi-threaded render pipeline based on a “render graph” (referred to as Task Graph) architecture. Call of Duty builds for Playstation, Xbox, PC, and mobile targets. My daily work includes writing, testing, debugging, and optimizing engine (C++) and shader (HLSL) code for all target platforms. I also occasionally work on internal tooling and build pipeline/map compile (C++ & Python).

A few of the features I have worked on include:

Wrote new shaders for bullet trail VFX to replicate the cylindrical “volume” and preservation of energy of a bullet tracer while minimizing rendering cost (Call of Duty: Modern Warfare II - 2022).
Expanded the engine’s large-scale terrain system & associated tooling to give artists more freedom when painting/applying surface materials and material combinations. See this SIGGRAPH 2023 presentation on Call of Duty’s terrain system for more context.
- These developments significantly improved artist workflow efficiency by reducing/replacing the need to use external tools to make hand-authored material transitions/combinations as done previously.
Implemented low-level engine support for new categories of purchasable gameplay content with careful consideration for data, streaming, and rendering constraints.
Made improvements to the runtime image/mesh streaming system for Warzone’s Caldera map. Caldera introduced pilotable airplanes featuring the fastest player-controlled speeds seen in Call of Duty. The improvements involved heuristically prioritizing streaming to assets along the flight path in order to reduce streamer churn.

Manta C++ game engine:

Voxel Renderer Prototype (renderer stress test)

Wrote a quick voxel rendering prototype within my C++ game engine (Manta) as a stress test for the engine’s rendering abstraction layer/API. the demo features chunk LOD swapping, frustum culling, dynamic per-vertex shadow mapping, fog, and subtle world curvature. The highest quality LOD scheme attempts to swap when voxels are 0.5 pixels or smaller. All in all, it performs pretty well with a scene of ~2 billion blocks at 8 bytes/vertex on my system (~1000fps). There is still plenty of room for optimizations as I'm not sorting draw calls or batching distant LODs (octree).

Manta deferred renderer (2019, project):

This video showcases a custom 3D deferred rendering pipeline that I developed for Game Maker: Studio 1.4 while studying at University. It was created with the intention of expanding the scope of that engine’s limited built-in 3D capabilities.

Using a deferred rendering approach, this video showcases several combined 3D effects such as normal & specular mapping (PBR materials), parallax occlusion mapping (displacement), point lighting, directional lighting, and real-time shadows (toggleable per light emitter).

The shaders for this engine were primarily written in GLSL ES.

This video demonstrates an implementation of cascaded shadow mapping (CSM) to the renderer mentioned above. The transitions between the cascade levels are blended with linear interpolation to avoid noticeable artifacts between the slices.

The implementation also supports a customizable number and resolution of shadow maps. These can be fine tuned for performance and visual appearance depending on project needs.

You can read about cascaded/parallel-split shadow mapping here.

This video demonstrates an extension of the cascaded shadow mapping implementation above combined with self-shadowing parallax occlusion mapping. Note the self-shadowing of the parallax displacement materials in the scene.

You can read more about self-shadowing displacement mapping here.

This video features yet another effect layered into the rendering engine: screen space ambient occlusion (SSAO). The general goal of ambient occlusion is to simulate the effect of indirect, ambient bounce lighting within a scene (particularly in shadowed areas/crevasses).

SSAO is an optimization of conventional AO, using only the cached depth and normal buffers to approximate the “shading” of a given pixel relative to its surrounding pixels in screen space. More about this technique can be found here.

Demo Download:

You can download a demo executable of this engine here: Windows

Note: This engine is currently work in progress, and this demo currently only supports Windows.

Controls:
Movement: W, A, S, D Elevate/Descend: E, Q
Mouselook Toggle: M
Place Point Lights: P
Place Spotlights: V
Place Spotlights (Shadows): C
Clear Lights: Backspace

Vitality (~2016-2020, Manta games):

In an attempt to improve the performance and capabilities of our 2D lighting engine in Vitality, we decided to completely rewrite its previous slow, CPU-driven approach to a significantly more efficient GPU driven system.

This problem was particularly tricky to solve because it required the active transfer of dynamic CPU-based world & water data to the GPU for processing. Since we want the light to accurately propagate the scene—not clipping through blocks to reveal “hidden” areas of the world—managing how the data is dynamically transferred as the player interacts with or navigates the scene was essential.

The logic behind the lighting itself is actually quite simple. Since each lighting “cell” only has to be aware of it and its neighbors, a fluid-like “propagation” system was implemented. Essentially, each cell (accurate to the block level) in the lighting grid consists of an intensity and RGB color value. Each cell then transfers some of its lighting data to its neighbors according to a few rules. The final result of each lighting pass is then upscaled, layered, and smoothly interpolated onto the final composite scene render.

DemO Download:

You can download a demo executable of this engine here: Windows

Note: This demo is a standalone version of the lighting engine prior to its integration into the main Vitality source. As such, this version is less feature complete than the final implementation.

compute / gpu acceleration:

A tricky problem in computer science, particularly game development, is how to simulate the physics of large fluid bodies in real time. Our project, Vitality, was no exception to this.

Since Vitality is a 2D, side-scrolling sandbox RPG with a large, procedurally generated and modifiable world, this problem proved very difficult to solve. While we contemplated performing these fluid calculations on the CPU (where world data is more readily accessible), it became clear that performing this simulation on the GPU was much more advantages, as we could utilize the exceptional parallel computing capabilities of GPU architecture to significantly simplify and accelerate the simulation.

We designed the fluid simulation on a grid-based particle-like approach (see Euler Fluid Simulations), where essentially each fluid “cell” contributes its information to its direct neighbors according to specific sets of rules. Since each cell essentially behaves independently in the fluid matrix, this implementation lent itself perfectly for parallel computing on the GPU.

The video on the left is another example of this sort of GPU-accelerated, compute shader being used in Vitality—but this time for the game’s thermodynamic / heat propagation mechanic.

Since a large portion of Vitality is focused on survival (in prehistoric times), we thought it would be interesting to include a robust (maybe over the top) heat system that would affect various aspects of gameplay (player temperature, etc.).

The system is very similar to the water simulation mentioned above. As such, we were actually able to piggy-back on the water simulation for much of the heat implementation, cutting down on redundant overhead (the heat and water process in the same GPU call and access the same transferred world data memory).

Ultimately, the heat simulation is another example of utilizing strengths of the GPU to free up processing on the CPU, opening up resources on the CPU to do other useful work.

VIRTUAL REALITY (2021 & 2014, university)

The videos above are showcase a couple VR applications that I have worked on. Both were created using the Unity game engine with the Oculus Rift SDK.