How do you stop grass from clipping through the floor in your open-world van life game? In this guest post, Square Glade Games programmers Tony Fial and Michiel Procé offer a detailed look at how they approached and ultimately solved this problem in Outbound with a custom shader clip solution.
We’re Tony Fial and Michiel Procé, part of the Square Glade Games team, and we’re currently working on the studio’s latest title, Outbound, which is an open-world exploration game set in a utopian near future. The player starts with an empty camper van and can turn it into the mobile home of their dreams, building it out exactly like they want to.
The vehicle is a large focal point for the game, as is driving it through nature. The world in Outbound is hand crafted and includes a lot of foliage and grass, which is luscious, tall, and plentiful. Although we could create a beautiful world with these assets, combining them with a vehicle that drives through such environments caused some visual issues.
The player is able to drive their camper van through basically any open area. Bushes and grass are no blockers for this. With the van being quite close to the ground, this often resulted in grass from the terrain clipping through the bottom or sides of the vehicle.
There are also places where the van can reach the taller foliage like flowers and bushes. To show the problem at hand, the screenshot below shows a case where both grass and bushes are heavily clipping in the vehicle. This is not only visually unappealing, but also causes various gameplay problems like visually blocking interactions or important information.
To summarize our core problem, there are different types of foliage and grass that clip through the camper van, which is unwanted from a visual and gameplay perspective.
Now, on to solving it, shall we?
At Square Glade Games, before we actively start to work on a solution, we personally find it handy to compile a list of optimal requirements.
In this particular case, we needed our solution to:
• Be performant. There is a lot of grass in Outbound, so an unoptimized solution might be very expensive in the areas with a lot more grass and plants.
• Keep the original style intact. Currently we are in a state of development where we cannot alter the look of major elements in Outbound, so ideally the solution makes use of as much of the original foliage as possible.
• Be cross-platform compatible. Since the title is planned to come out on multiple platforms, the solution needs to work on Windows, Nintendo Switch™, Xbox and PlayStation®.
• Be intuitive to use. The solution should ideally be intuitive for both the designers and the programmers in the team.
• Be applied to multiple shapes. Ideally we would clip away foliage in an exact shape of the vehicle, possibly using multiple shapes.
Now to think about solutions that could satisfy this list of requirements. Our first thoughts went to an element which all blades of grass share... the shader.
Almost all of the flora in Outbound is placed on the Unity terrain using the terrain tools. A sizeable portion of this is the grass, which uses the default Grass shader. This shader uses the GPU to place and billboard the grass planes in a very performant manner. Other elements, like the bigger bushes shown in the screenshot above, are placed as detail meshes, using their own assigned material and shader.
This presented another important detail, namely that the proposed solution should be able to work on multiple entirely different shaders, in the same manner, at the same time.
All of the proposed solutions below share one major 'input' in common as well: The camper van’s position, or to be more precise, the area where the foliage should be clipped away.
Looking at the stated requirements, we wanted our solution to be intuitive for the rest of the Square Glade team to use. In our experience, Editor tools will only be used by team members when they are intuitive and easy to pick up. With this in mind, we decided to build a visual 3D cube that could be scaled, rotated, and manipulated to clip just enough of the vehicle's body and tweak it so it's just right. Any foliage within the cube would be clipped, while everything outside of it would look the same.
Stencil shader
The very first thing we tried was using a shader element called the 'stencil buffer.'
This part of shader programming is very fascinating, yet also a bit difficult to wrap your head around. What it boils down to for our purpose is that we tell the 'clipping element,' in this case, our cube, to write some information to the stencil buffer of a rendered frame. That means anywhere on the screen where the cube is, it will write a value of 1. The 'clipped' object (in our case, the grass) can read from that buffer and discard any pixels that have a value set to exactly 1.
In shader code, that would look something like this:
Clipping object 'Cube'
Stencil
{
Ref 1
Comp always
Pass replace
}
Clipped object 'Grass'
Stencil
{
Ref 1
Comp equal
}The clipping object writes a value of 1 to the buffer, stated by the line Ref 1, and will do this Always. If a later rendered stencil value matches, or Passes the stencil comparison, it will replace it with this shader's information. The grass has a similar implementation: It will also look for the value of Ref 1 and will only pass the check if the Comparison is Equal to that reference value.
This implementation did work for clipping away the grass, and it was very efficient, as it works on the pixels of the rendered frame and is not affected by the amount of grass in a given scene. However there was a fatal flaw in this solution. Because this implementation has no sense of depth, it will clip away anything behind the cube as well. Practically this meant that when the player was sitting inside of the vehicle, while looking from a first-person view, the entire screen would be marked as 'clipped,' so the player wouldn't see any grass anywhere. Because of this, we had to try some other methods which would also work when the player camera was inside of the 'clipper' object.
Manual clipping
A solution we briefly discussed was to manually remove the grass at our vehicle's position, taking it away from the terrain itself. We had already done so for other parts in the game, using the 'TerrainData.SetDetailLayer' function Unity provides on the terrain. This would set the grayscale color of the detail layer to 0 on the pixels just below the van, instructing the terrain to remove any detail meshes or grass at that set of locations.
Because Outbound’s maps are rather large, it means the resolution of the detail layer is on the lower side, making it a bit 'jagged.' This is perfectly fine for normal detail placement of grass and other meshes, but when manually clipping parts away, the lower resolution will result in a shape that would not be close enough to the size of the van, either being too small or too large.
This solution would also result in flickering in/out details when the vehicle was just on the border of two terrain detail pixels. For these reasons, we did not go forward with implementing this solution. Our journey continues!
Clip shader
With the stencil buffer shader, we thought we were almost there, as we rendered the pixels invisible where needed with the precision of the van's exterior body. If only there was another way to do so, while actually using the depth of the cube, knowing the solution should basically only clip the pixels inside of its bounding box.
As it turns out, there is a method that does just that! HLSL shaders provide the humble clip() function, which simply discards the pixel if the specified value is less than 0. You might've seen this before in some random shader where it’s often used for alpha clipping.
To provide an example, Outbound's grass looks like actual tufts of grass and not as square quads with an image of grass on it, because we 'clip' away wherever the alpha channel of our grass texture is black.
When we did a quick first prototype/check for this solution, we had high hopes that this implementation would be able to work, as we were able to render pixels invisible above a certain world position. In pseudocode, the function looked like the following:
// Return -1 when the Y position is above 0, and return 1 when it is not.
clip( worldPos.y > 0 ? -1 : 1 );By this point, we had a simple example that was showing a promising solution, namely using a clip shader. The next step was to create a function to supply the shader with the info that is needed to clip exactly where we wanted it to.This involved two parts:
• The part where we calculate in essence the 'shape,' including its dimensions and transformations, and supply this data to the shader.
• The part where the shader uses this data, checks if a given point is within the shape, and discards its pixels where needed.
For the first step of our solution, we created a 'GrassClipperShape' script, a MonoBehaviour that we could attach to an object in the scene, which would dictate where a clipping area would be. An example of this is shown below, where the area of the shape using OnDrawGizmos in the Editor view is displayed.
As we ideally would like to use multiple of these clippers, we need an overarching script (i.e. a "manager") to handle all the available clippers. Each clipper would supply the following properties to this overarching script, named the 'GrassClipperManager':
• Shape: the type of shape, We wanted this version to work with both cubes and spheres, so this is a simple enum set to either ‘cube’ or ‘sphere’
• Vector3: the size of the object in the scene
• Matrix4x4: the calculated rotated object in world space
The GrassClipperManager, of which there is only ever one in the scene, will fetch this information from the clippers each frame, and send it to the shader like so:
Shader.SetGlobalInteger("_ShapeCount", count);
Shader.SetGlobalMatrixArray("_ShapeInvMatrix", inv);
Shader.SetGlobalVectorArray("_ShapeParams", size);
Shader.SetGlobalFloatArray("_ShapeType", type);The lines above will set global shader values. To explain in short, this means that you can use shader values with these exact names and types, and use them in any shader.
Because we want our clipping to happen on multiple different shaders, we created a separate HLSL script to be included in whichever shader needed to be affected by our clipper. This script exposes a custom function named 'ApplyClipVolumeSDF'. It uses the information from the now filled global shader values, and will calculate if a pixel is within any of the bounds.
inline void ApplyClipVolumeSDF(float3 worldPos)
{
float clipVal = GetClipFade(worldPos);
if (clipVal <= 0.0)
clip(-1);
}As you can see above, if the pixel is supposed to be discarded, it will call the 'clip(-1)' function, returning a discarded pixel. Otherwise, it will just progress as normal through the rest of the shader.
With the clipping function now created and supplied with the necessary data, it was time to implement it into our shaders.
Let's first discuss how to do this for the detail meshes, where we could create a copy of the original and edit it. At the very top of the shader, we must reference the custom script like so:
#include "Assets/Shaders/ClipVolume.hlsl"And then when we want to actually use the function, we simply call it inside of the fragment part of the shader like so:
float3 worldPos = mul(unity_ObjectToWorld, float4(input.positionOS, 1.0)).xyz;
ApplyClipVolumeSDF(worldPos);In our case, only two shaders needed to include this, namely the default shader the Unity grass uses, and a custom shader used for all of the other foliage rendered as detail meshes. Now that we have this, it can be implemented easily in any other shader if we need to.
But our journey wasn’t over – a final hurdle presented itself. How could we now edit and actually retain changes made to the default grass shader? Unity uses some specific built-in shaders for rendering grass, in our case the 'WavingGrassBillboard.shader'. This shader is applied automatically to all of the grass, with no option to supply custom variants. This was crucial for making our solution work, as it needed to hook into that shader to be able to call the custom 'ApplyClip' function and discard the unwanted pixels.
After trying some solutions, fellow team member Michiel Procé figured out a way to edit and actually retain the changes to the default grass shader reliably. By running the following code during builds and in editor, our custom shader replaces the default URP shader:
string replacementShaderName = "Hidden/TerrainEngine/Details/UniversalPipeline/BillboardWavingDoublePass_Clipped";
if (GraphicsSettings.TryGetRenderPipelineSettings<UniversalRenderPipelineRuntimeShaders>(out var shadersResources))
{
if (shadersResources.terrainDetailGrassBillboardShader.name != replacementShaderName)
{
Shader replacementShader = Shader.Find(replacementShaderName);
shadersResources.terrainDetailGrassBillboardShader = replacementShader;
}
}Note that this only replaces the WavingGrassBillboard shader, but implementing this for other shaders would be similar.
Our end solution of using a clip shader works well for our purposes and we are very happy with the results it provides. See the screenshot below for a visualization of the solution, where a rectangular cube clips away the grass within. Note that the box is seen from above and is placed through the terrain for an optimal view of what is clipped.
Looking back at our list of requirements for our grass clipping solution, we were glad to see that it adheres to all of them!
• The solution is performant, as the functions used to calculate the clipping are very cheap. And because it outright discards the pixel, our implementation won't do further unnecessary processing.
• It keeps Outbound's original style intact because it is built on top of the shaders we were already using.
• The implementation is platform-agnostic, because the clip() function itself is.
• The solution is intuitive to use for the rest of the team. Designers can create and use multiple shapes and even have them intersect each other.
We believe that features like the above are extremely important, not only for sake of creativity, but also to prevent strange bugs from emerging later on.
To share this solution with the community, we created a sample project using this techniques detailed above, so you can try it yourself – check it out here on GitHub.
Thanks for reading our guest post. Hopefully this helps a lot of other developers that are facing the same problem as we did!
Outbound is currently in closed beta testing; follow the game on Steam for updates. Explore more Made with Unity games on our Steam Curator page, and check out more stories from Unity developers on our Resources hub.
Nintendo Switch™ is a trademark of Nintendo.