In the first post, we discussed what mixed mode is and how Subtractive mode works. If you are unfamiliar with mixed lighting, then please take your time to understand the functionality behind it.
The distinct feature of new mixed lighting modes is Shadowmask support. Besides, intensity and directionality maps, Unity can now also generate shadowmasks for all light types. Unlike subtractive mode, shadowmasks can seamlessly mix realtime and baked shadows regardless of the light type. This means we can achieve distant shadows. For performance, we can have nearfield realtime shadows and farfield baked shadows, thus keep lower draw call counts. In contrast to subtractive mode, shadowmask feature can also generate high quality realtime specular. In addition to these rewarding features, UI is improved, refactored and new debug modes are added to facilitate the way you can author your lighting in your scenes.
And here is a video that demonstrate Shadowmask and Distance Shadowmask in use. Precomputed realtime GI is completely disabled for this scene setup.
Now, we already mentioned that subtractive mode is the old mixed mode in Unity. But besides that you will find out there are two new options in the lighting mode menu: Baked indirect and shadowmask. It is important to note that baked indirect and shadowmask are not quality settings. All the modes offer different features and they can all serve useful based on your use case.
Essentially, the key feature of Shadowmask is very simple to understand. It resolves what Subtractive can’t do, which is, in addition to directional light you can use point light and spotlight to mix realtime and baked shadows without double shadowing/lighting therefore it allows a correct composition of overlapping shadows.
Before moving on, we need to define the concept of shadow distance which is tightly related to light modes. Many beginner users don’t know this concept but it is very crucial to grasp it in order to author your scene correctly and efficiently. Otherwise you can easily end up with unnecessary draw calls and drops in frame rates.
Concept of Shadow Distance and Shadow Casters
In Unity, the behaviour of shadows change based on shadow distance and this behaviour also depends on the light mode. A quick way to see your shadow distance is through Shadow Cascades mode. In the scene view, you can select Shadow Cascades mode to visualize the shadow distance. This parameter can be changed in Quality settings. Another way to change shadow distance is through API call but the use cases for changing it are very limited. It can sometimes be useful for edges cases though (e.g. when you need to reduce the shadow distance for a lower draw call in some light modes).
So the behaviour of the shadows can change whether your geometries are within the shadow distance or beyond the shadow distance, and it directly affects the performance.
In fact, if you go to Quality settings (Edit > Project Settings > Quality), you will realize that there are many parameters you can use to adjust your shadow settings. Unfortunately, this is a hidden part in UI and not many users are aware about it unless they specifically search for these parameters when they need it.
In particular, you will see that you can toggle the option for shadowmask between Distance Shadowmask and Shadowmask. By default, Distance Shadowmask option is selected and we will discuss the difference between these modes very soon. Let’s first start with Shadowmask mode itself.
Shadowmask
In Shadowmask mode, static objects receive shadows from other static objects via the shadowmask, regardless of the shadow distance. Shadows from dynamic objects are received via shadow maps only within the shadow distance. Dynamic objects receive shadows from other dynamic objects via shadow maps within the shadow distance. Shadows from static objects can only be received through light probes.
Now, when you are using Shadowmask mode with dynamic objects and lightmap static objects, the first thing you may realize is the inconsistency between dynamic objects’ shadows (shadow maps) and baked shadows from lightmap static objects. Currently (as of 5.6), Unity only supports a simple version of PCF shadows at runtime, which is a simple shadow that can sometimes render artifacts if you get too close. Thus, it is not possible to achieve soft shadows with dynamic objects. On the other hand, it is relatively easy to achieve soft shadows with static objects. In fact, you usually have to increase the resolution to render more pronounced shadows with static objects because you need more texels to define your shadows on a lightmap. Only this way, you can render shadows closer to shadow maps from dynamic objects.
With a high resolution lightmap, you can see that baked shadows from static objects are close to shadow maps from the dynamic object.
With a low resolution lightmap, you can see that baked shadows from static objects are inconsistent with shadow maps from the dynamic object.
A friendly reminder about shadowmask mode: Dynamic objects cast shadow maps only within the shadow distance and none beyond. On the other hand, static objects only cast baked shadows regardless of shadow distance. And both Shadowmask and Distance Shadowmask offer realtime specularity.
This brings us to our next question. What is Distance Shadowmask and how is it different from Shadowmask?
Distance Shadowmask
If you watched the video I shared at the beginning of the post, you have probably realized that regular shadowmask mode offer less draw calls than distance shadowmask mode. Now why is that?
It’s because of the behaviour of Distance Shadowmask based on the shadow distance. Within the shadow distance, both dynamic and static objects are rendered into the shadow map allowing static objects to cast sharp shadows onto dynamic objects. Beyond the shadow distance, static objects receive high quality shadows from other static objects via the precomputed shadowmask, and dynamic objects receive low resolution shadows from static objects via Light Probes and LPPVs.
I think this clarifies how dynamic and static objects cast shadow in Distance Shadowmask mode and why Shadowmask mode is cheapier than Distance Shadowmask. Basically, shadow maps cost more than baked shadows (because they are rendered every frame, whereas baked shadows are stored in a texture/lightmap).
And in Shadowmask mode, dynamic objects cast shadow maps only within shadow distance. Distance Shadowmask is more expensive because static objects can also cast realtime shadow maps within the shadow distance, hence increasing the draw call. That’s why Distance Shadowmask mode is more suitable for high-end PCs, consoles. And as a cheapier solution, shadowmask is recommended for low-end/mid-end devices.
Occlusion in Shadowmask
One more thing I would like to elaborate about Shadowmask feature is how occlusion works for different object types. In the previous post, we talked about why the placement of light probes plays an important role for dynamic objects to receive shadows from static objects. Essentially, dynamic objects need light probes to sample occlusion data from occluders. Therefore, probe occlusion concept from Subtractive mode can be directly applied to Shadowmask and Distance Shadowmask as well.
For a quick reference of how objects receive shadows based on their type and light mode, you can check this Lighting Modes Reference Card. (You can even print and paste it to one of your walls for later reference 
Shadowmask as Quality Setting
In Unity 2017.1, Shadowmask and Distance Shadowmask options are moved to Quality Settings, there are two reasons for that.
First, we wanted the users to artistically have more control over the behaviour of shadows so it is possible to change the mode at run-time. For instance, you can use Shadowmask for in-door environment (e.g. to achieve soft shadows in a hangar) and switch to Distance Shadowmask for out-door environment within the same scene. In fact, we will examine how to do this through API in the next post.
Secondly, you can set various requirements for different hardware setups. For example, in a game, it can be exposed to Menu UI in which user can adjust the setting or if you publish your game on Steam, you can rely on the regular Shadowmask for low-end hardwares and use Distance Shadowmask for high-end PCs.
We can do this because technically, it is very easy to switch between Shadowmask and Distance Shadowmask, the only drawback is the amount of draw calls but after all, it is up to user to handle it.
Technical Limitation and Debug Tools
Finally, let’s talk about technical limitations for Shadowmask feature (Shadowmask and Distance Shadowmask) and the debug tools that can help you to diagnose your problems while doing a setup.
Now that you know how and why to use Shadowmask, let’s talk about its limits. So, due to technical constraints, Unity only supports up to four overlapping mixed lights in a scene. If there are more than four mixed lights on the same spot, then one of them falls back to baked. You can usually observe this if one your lights suddenly become overbright. You can instantly sense that there’s something wrong.
Good news is we have a debug visualizer to help you solve your problems. If you switch your scene visualization mode from “Shaded” to “Light Overlap”, you will see that light in question is highlighted as red.
It is worth mentioning that unlike point or spotlight, directional light doesn’t have defined boundries, it’s omni-directional. So you always need to take that into account when debugging your overlapping mixed lights. On the screen, you may just see four overlapping lights but in fact, directional light might be intersecting from elsewhere. This can really be cumbersome if you have more than one directional light to lit your scene (e.g. as fill light) so use it wisely.
Baked Indirect
So far we have covered Subtractive mode, Distance Shadowmask and Shadowmask features. The only remaining mode is now Baked Indirect.
Given the complexity of options in Shadowmask, Baked Indirect mode is fairly easy to understand and that’s why I intentionally left it as the last mode to explain.
Baked Indirect mode doesn’t use any shadowmasks. That’s why there are no distant shadows in this mode. Within the shadow distance, both static and dynamic objects cast realtime shadow maps. But beyond the shadow distance, there are no shadows. In baked indirect, everything is also in realtime except indirect illumination. This means, you have realtime lights, realtime shadows and realtime specularity but bounce light information is stored in lightmaps and it is static. The things you can do in this mode are limited but you can get creative. Below, you will see how swaying mixed spotlights in baked indirect mode can fake realtime global illumination:
You can see that lights are animated and the intensity of lights flicker. However, if you specifically check the indirect illumination on the streetlight, you can see that it doesn’t change. So technically, you can move your lights but not too much, otherwise you can end up with artifacts. In the video below, you will see how indirect illumination can cause this type of artifacts if you move your objects across the scene:
As you can see, understanding baked indirect mode is much more easy. In the next post, we will see how to use Shadowmask feature in a simple but realistic use case.