Creating Open World Environments in Unreal Engine 5: Part 1 - Landscape Setup

Creating Open World Environments in Unreal Engine 5: Part 1 - Landscape Setup

Recent Escape Studios graduate Rodolfo Silva explores environment art creation, emphasising pre-production and landscape setup in this initial article of a three-part series. Get ready for a journey into the complexities of 3D Environment Technical Art – buckle up and enjoy!

Recent Escape Studios graduate and Maxon ZBrush expert, Rodolfo Silva, delves deep into the intricacies of environment art creation in this article. In this first instalment, Rodolfo shares a more technical approach, exploring elements of pre-production and landscape setup. Brace yourselves for a three-part series where Rodolfo covers the complexities of 3D Environment Technical Art. Buckle up and enjoy the ride!


This project entitled "Forgotten Altar", was undertaken with the goal to advance my own personal research into new procedural workflows and to consolidate existing knowledge in the field of Game Art creation. The project was designed to serve as comprehensive documentation of the skills and techniques I had learned during my time at Escape Studios, as well as aiming to challenge myself  in creating a large, open-ended natural environment that would require extensive technical art setup, planning, and the creation of a diverse range of organic assets, including foliage, terrain, and some man-made objects for storytelling purposes.

0:00
/

This series of articles will reflect on that entire process, from the creation of the landscape to the modeling and texturing, tillable texture creation, hero props, VFX, material creation and performance troubleshooting of a large scale environment.

The creation of "Forgotten Altar" was inspired by several sources, including the outdoor environments in the Elder Scrolls series and a presentation by Alex Alvarez at the VFX and Games event Trojan Horse was a Unicorn. The talk centred on photorealistic environments and scenes made using Unreal Engine 5 for use as background during the band Lustmord's concerts. I also took further inspiration was drawn from games such as The Elder Scrolls 3, 4, and 5, the Fable series, and Assassin's Creed: Odyssey and Valhalla.

In this environment, the scenery takes centre stage. So it was paramount to focus on the terrain and natural assets within the scene, with a particular emphasis on the creation of a wide range of organic assets, such as procedural terrain systems, landscape cliff meshes, small foliage and trees.

Art direction, Concept and Research

For the research phase I wrote a broad synopsis for the environment, including the setting, date, and context of the level. This was followed by the creation of an Art Bible and a mood board, imagining an art style inspired by The Elder Scrolls: Skyrim and The Elder Scrolls: Online.

I divided the concept research phase into three stages: high concept, secondary inspiration, and individual asset research. These stages helped refine the overall vision and ensure the project kept consistent and coherent throughout production.

Figure 1 - The colour palette was one of the first things researched, along with the lighting in order to establish the mood.

The high concept phase started by choosing a colour palette (Figure 1) and a few concept art pieces to drive the overall mood of the scene. For this purpose, I chose a couple of images from Concept Art Director Conar Cross, who is currently working on the reboot of the Fable series. These pieces served as the cornerstone for the creation of the environment, informing the choice of colour, tone, and overall aesthetic.

Figure 2 & 3 - Concept mood paintings made by artist Conar Cross.

Mood board and References

Building upon the high concept and the chosen concept art pieces, the secondary inspiration phase required me to establish a visual narrative for the final piece which meant creating a mood board.

In the individual asset research phase, I gathered references to aid in the creation of the assets themselves. All of this information was compiled into a pre-production Art bible, serving as the basis for the mood board.

During the mood board creation, I used Pinterest and PureRef extensively and then organised the images based on each distinct theme or area within the environment.

Figure 4 - Overall look of the mood board.

The forest area, castle, and the ruin area required a diverse range of references, from the rocks to the modular components of the castle, vegetation such as trees, grass, and bushes, as well as a hero prop and smaller props. This is reflected in the organisation of the mood board, which I presented in the form of separate sections for each of these areas.

Figure 5 - An example of a smaller section of the moodboard highlighting the rock asset references.

In addition, I devoted a section to the art direction and cinematography, which was the central concept behind the project: a compact forest area through which the camera would traverse, leading to a grand unveiling of the sweeping landscape with a towering castle on the horizon.

Figure 6 - An example of a smaller section of the moodboard highlighting the cinematic structure, the high concept and some other key structures.

Tech Art setup and Landscape

For this project, I placed an emphasis on using non-destructive techniques and investigating procedural workflows to generate larger environments efficiently. Conducting research on using software such as Gaea and WorldMachine to generate terrain data, such as height maps and other ‘Data maps’, which could be incorporated into Unreal Engine. These ‘Data maps’ would drive the creation of the landscape material layers.

In addition, to build the final piece, I created an extensive list of modular assets, which included objects like trees, rocks, cliffs, and bushes. These assets were made in a way that they could be used multiple times in different combinations to create the vast and diverse environment.

The focus, early on, was on developing the tools and systems to work efficiently and make the creation process streamlined and easily repeatable, with minimal manual intervention which allowed for a more efficient workflow and a bigger focus on the actual look and feel of the environment, rather than worrying about the technical aspects of putting it together.

Procedural terrain workflow & landscape material

I chose Gaea for terrain generation due to its features smooth learning curve.

Gaea provides a vast array of nodes which are stored in categories such as: primitive nodes, erosion nodes, data nodes, colour nodes, water nodes, et

Using different ‘Primitive nodes’ blended together and then adding a few erosion systems on top of that allowed me to quickly iterate on the base shape of the terrain.

Figure 7 - Height map generation in Gaea.

The terrain was then textured inside of Gaea using ‘Colour nodes’ with erosion mixing nodes to output a mask that would tell Unreal where to assign particular ‘Material layers’.

In the image below you can see the node system has multiple outputs (purple nodes) which are exporting certain colour information as masks (non sRGB image files) to be used later in Unreal.

Figure 8 - Colour data maps generation in Gaea.

Layers, distance blend and procedural data maps

For the actual implementation of this workflow, the first step was to bring in the ‘’Height map’ to Unreal to generate the landscape. The resolution and component size was set to fit the 1023px to create a 2x2km landscape as stated in the landscape technical guide from Unreal documentation in order to avoid stretching and other issues.

Figure 10 - Landscape actor after importing the height map with the correct resolution.

The next step was to build the master material for the terrain, which would serve as the foundation for assigning the ‘Data maps’ to the various layers of the landscape.

The creation of the landscape material involved establishing all the necessary systems, including:

  • Designing layers to represent the various materials present in the landscape, such as rock, dirt, forest soil, grass, and snow.
  • Implementing a feature set that would hide repetition when viewed from a distance.
  • Developing a method to regulate the snowline based on a height value, which would determine the presence of snow on upward-facing surfaces.
  • Integrating a system to blend the terrain assets with the Landscape material, to ensure a seamless appearance throughout the environment.
Figure 13 & 14 - The ‘’Master material’ for the landscape with all the systems in place(top) & a close up of the layering system in the ‘Master material’ output (bottom).

To address the first challenge of hiding the repetition of textures, the initial strategy was to implement a ‘Distance blend’ system to manage the tiling. However, this approach ran into a problem when integrating the ‘Runtime Virtual Texture (RVT)’, as it resulted in a blurred transition between the 3D models and the landscape materials. This is due to the fact that ‘RVT’ basically captures all of the terrain layers from the top and from a distance, effectively meaning that the ‘RVT’ captured texture is tiled as if it was far aways, even when you're looking at the blended section up close, resulting in a "blurry" transition.

To resolve this issue, a workaround was found based on a technique described in an article by 80 Level .

Figure 15 - The system for hiding repetition through the 'distance blend' method.
Figure 16 - The system for hiding repetition using the new 'average blend' method.

The development team at Wargaming discovered a solution to conceal repetition without altering the tiling. Instead, they utilised an average bias and a variety of scales for the same texture, which were overlaid on top of each other. This technique was referred to as ‘Average blend’ within the ‘Material function’.

Figure 17 - The logic of the "average blend" method in theory.
Figure 18 - The same logic put into practice as a material function in Unreal.

To maintain a clean and organised ‘Master material’, each landscape layer was constructed within a ‘Material function’. This ‘Material function’ was responsible for managing all texture sampling, blending variations, and procedural snow, while also exposing relevant inputs via linked ‘Instance parameters’ or ‘Material parameter collections’ in the ‘Master material’. Similarly to how you would use graphs in Substance 3D Designer.

Figure 19 - The material function for the rock layer.
The material features a vast set of systems that enable procedural snow to settle on top facing verts or pixels controlled by a ‘snow height’ parameter, ‘RVT’ blending with props and an ‘Average blending’ system to hide repetition at a distance (instead of using the typical ‘Distance blend’ method).

Procedural Snow

Two levels of procedural snow were created to address the challenge of adding snow to the environment. This required precise control over both the landscape assets and terrain, where the height value determined the snowline and snow coverage.

For this system to work it meant creating the logic for the landscape and then duplicating it to the props ‘Master material’ and using a ‘Parameter collection’ to control the snowline for both materials at the same time.

Figure 20 - The snow blending system in the landscape prop ‘Master material’ (The “+snow” refers to the material with the vertex normal blending and the “base material” is then interpolated based on height), using a ‘Parameter collection’ to control both the landscape material and the prop material at the same time.

The system consists of an ‘Absolute world position’ centred logic to interpolate between the material without snow and the material with snow (figure 20). A set of different nodes within the material were used to generate snow just where the vertex normal and the pixel normal were facing up (figure 21).

Figure 21 - The normal blending system applied to one of the landscape layers in the landscape material. The ‘Pixel normal World Space’ node is used to interpolate all texture maps except the normal map which uses a vertex normal input node.

To ensure that snow would generate procedurally on top of a model even when it was rotated, requires a mix of ‘Vertex normal’ blending and ‘Pixel normal’ blending. Interpolating between two values on the pixel normals requires the normal map to be calculated first because it's looking at its data on the Z axis and the green channel. So for the normal map snow blending a ‘Vertex normal World Space’ node was used instead, in order to transition between the regular normal map and the snow normal map. The rest of the maps used a ‘Pixel normal WS’ input node (figure 21).

In the clip below we can see an example of this working on the same model being rotated around replacing the model's textures with the snow material textures on the pixels that are facing up in world orientation.

0:00
/
An example of the snow being projected onto a rock model based on the model's position in the Z axis.
0:00
/
The same model using the normal blending system rotated at different angles.

Once the blending system was in place, another system was created to manage the height at which any material would transition to display the snow layers on top of its upward-facing normals. This was achieved using a ‘Parameter collection’, allowing a single parameter to control the snow blending height values for all materials in the scene.(figure 22 & 23).

Runtime Virtual Texturing

To seamlessly blend some of the cliff assets into the landscape, I integrated a ‘Runtime Virtual Texture (RVT)’ system into the ‘Master materials’. The process of setting up the ‘RVT’ also involved creating a system of nodes within the material to mask the steep slopes of the models based on the vertex normal direction. This prevented any stretching issues that occurred as the ‘RVT’ is projected from the top down.

Figure 26 - The slope masking system in the ‘Material editor’.
Using ‘RVT’ allowed to blend the props with the landscape. But since the ‘RVT’ is projected from the top then texture blurring occurs, which means I had to create a way to mask off the steep slopes of the mesh.

Procedural foliage placement

For the procedural foliage scattering I used a ‘Landscape grass output’ node to spawn specific meshes of foliage on top of specific material layers. For example, the small grass meshes would only spawn on top of the grass landscape layer and the trees would spawn on both the dirt layer and grass layer.

Figure 27 & 28 - The procedural foliage scattering system (top) & a test using the first version of the grass static meshes (bottom).
Figure 29 - A first pass of the procedural placement system, scattering the first iteration of the tree model.

In the next part of this tutorial series I'll run you through my process of creating the game ready assets for the scene using ZBrush, Substance, Mixer, Speedtree and Unreal Engine which will include and in depth breakdown on foliage creation for games.

In the meantime, be sure to join my Discord or DM me on Instagram if you have any questions. Find me via this link.

I really hope you enjoyed reading about my process and stay tuned for the next article :)