Benefits of Image Based Lighting on Mobile

Image Based Lighting used in movies and video games for ages. In most engines, it’s available by default in form of Light & Reflection Probes. IBL gives us the ability to imitate complex lighting setup and reflection in the cost of couple texture samples. In fact, using Light Maps is not always possible. This could be the case when huge locations used or location generated procedurally. Usually, we have lighting information in form of an omni-directional image(cube map).

By sampling form different mip levels we can adjust glossiness of textured material

In this article, I want to focus on slightly different kind of IBL technique — MatCap.

MatCap texture

This shading model often seen in 3D modelling software, but we will try to use it to implement shading system. The 2D texture on the left has lighting information for each possible camera space normal. Top of the ball contains color information for normal — (0,1,0), bottom -(0,-1,0), left — (-1, 0, 0), right — (1,0,0). And we can interpolate everything in between. This technique could be used to simulate complex lighting rigs, for example, night city street or space game where multiple suns easily possible. The downside of the method — you need to recreate MatCap texture each time camera is rotated, so it’s a perfect candidate for scenes with static camera angles — some platformers, character showcases & top-down view games. Also, an object on the borders of the screen can be shaded incorrectly(depends on camera FoV), so centrally positioned objects look best.

I’ve created simple demo project IBLDemo which shows a couple of IBL rigs. Here essential parts to implement MatCap rendering:

• Renderer — each time camera angle changed or we’re booting up our system we need to generate MatCap texture (in case we not use prerendered data). In my demo, I did simple CPU raymarcher, but it’s possible to create compute shader version or simple GPU Bliting variant. CPU version also could utilize new unity data-driven approach with ECS and Burst.
• Factory — next we need some manager which controls texture baking process and is an entry point for system configuration.
• Listener — in case if we are not doing texture baking each frame we need this component on all objects with MatCap material. For the continuous texture baking approach, it’s not worth to use reactive systems, better go with ScriptableObject/RenderTexture and simply update it in the background without notifying all listeners.
• Shader — last but not least. Diffuse, Specular and Ambient parts could all be implemented here, it all depends on what is the output of your Renderer.

This is by no means best solution for all situation, but in some cases, it helps achieve high fidelity on low-end devices.