UE 项目优化:PSO Caching

UE 中具有 PSO Caching 机制,全称 Pipeline State Object Caching,用于预先记录和构建出运行时所使用的材质依赖的 Shader 信息,当项目首次使用这些 Shader 时,该列表可以加速 Shader 的加载 / 编译过程。PSO Caching 会把渲染状态、顶点声明、Primitive 类型、RenderTarget 像素格式等数据保存到文件中,提升 Shader 的加载效率。
本篇文章主要介绍 PSO Caching 的启用及构建流程,并会分析 PSO Cache 在引擎中的加载流程以及实现热更 PSO 方式、错误处理等,PSO Caching 的原理有时间再进行详细分析。

PSO Caching 的官方文档:PSO Caching

PSO Cache 构建流程概览:

PSO Caching 的部署和使用大致分为以下几个步骤:

  1. 为项目开启 PSO Cahche 和 ShaderStableKeys,打包后可以从 Metadata/PipelineCaches 目录下获得ShaderStableInfo*.scl.csv
  2. 添加 logPSO 参数启动游戏,用于在运行时记录 PSO 数据(*.scl.upipelinecache
  3. 通过 ShaderStableInfo*.scl.csv*.scl.upipelinecache生成*.stablepc.csv
  4. 再次执行 Cook,通过 *.stablepc.csv 生成 upipelinecache 文件,打至包内;
  5. 启动游戏,引擎自动加载*.stable.upipelinecache,编译 Shader 时使用 PSO Caching

本篇文章的内容顺序也遵循着几个步骤。

启用 ShaderStableKeys

首先需要为项目开启ShaderStableKeys,在执行 Cook 时生成稳定的 ShaderKey,作为记录 Shader 的凭据。

DefaultEngine.ini(或平台相关如 AndroidEngine.ini)中添加以下值:

1
2
[DevOptions.Shaders]
NeedsShaderStableKeys=true

添加之后再执行打包(Cook),会创建以下目录:

1
Saved/Cooked/PLATFORM_NAME/PROJECT_NAME/Metadata/PipelineCaches

并且会在该目录下生成两个文件(分别对应项目、引擎):

1
2
ShaderStableInfo-PROJECT_NAME-GLSL_ES3_1_ANDROID.scl.csv
ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv

Cook 过程中会有以下 log,表明生成了这两个文件:

1
2
LogCook: Display: Saved scl.csv D:/PSOExample/Saved/Cooked/Android_ASTC/PSOExample/Metadata/PipelineCaches/ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv for platform Android_ASTC
LogCook: Display: Saved scl.csv D:/PSOExample/Saved/Cooked/Android_ASTC/PSOExample/Metadata/PipelineCaches/ShaderStableInfo-PSOExample-GLSL_ES3_1_ANDROID.scl.csv for platform Android_ASTC

可以使用它们通过 -run=ShaderPipelineCacheTools 这个 Commandlet 来生成*.stablepc.csv

*.scl.csv文件的内容:

运行时捕获 PSO 数据

启动游戏时加入 -logPSO 参数或者在 DefaultEngine.ini 中加入以下配置:

Config/DefaultEngine.ini
1
2
3
4
[ConsoleVariables]
r.ShaderPipelineCache.Enabled=1
r.ShaderPipelineCache.LogPSO=1
r.ShaderPipelineCache.SaveBoundPSOLog=1

也可以在 Devices Profile 中设置:

这两个参数在以下代码中使用:

Runtime/RHI/Private/PipelineFileCache.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
bool FPipelineFileCache::IsPipelineFileCacheEnabled()
{
static bool bOnce = false;
static bool bCmdLineForce = false;
if (!bOnce)
{
bOnce = true;
bCmdLineForce = FParse::Param(FCommandLine::Get(), TEXT("psocache"));
UE_CLOG(bCmdLineForce, LogRHI, Warning, TEXT("****************************** Forcing PSO cache from command line"));
}
return FileCacheEnabled && (bCmdLineForce || CVarPSOFileCacheEnabled.GetValueOnAnyThread() == 1);
}

bool FPipelineFileCache::LogPSOtoFileCache()
{
static bool bOnce = false;
static bool bCmdLineForce = false;
if (!bOnce)
{
bOnce = true;
bCmdLineForce = FParse::Param(FCommandLine::Get(), TEXT("logpso"));
UE_CLOG(bCmdLineForce, LogRHI, Warning, TEXT("****************************** Forcing logging of PSOs from command line"));
}
return (bCmdLineForce || CVarPSOFileCacheLogPSO.GetValueOnAnyThread() == 1);
}

运行游戏时会有 log:

1
2
3
4
5
6
7
8
9
10
11
12
LogConfig: Applying CVar settings from Section [ConsoleVariables] File [../../../FGame/Saved/Config/Android/Engine.ini]
LogConfig: Setting CVar [[r.ShaderPipelineCache.Enabled:1]]
LogConfig: Setting CVar [[r.ShaderPipelineCache.LogPSO:1]]
LogConfig: Setting CVar [[r.ShaderPipelineCache.SaveBoundPSOLog:1]]
...
LogRHI: Base name for record PSOs is ../../../FGame/Saved/CollectedPSOs/++UE4+Release-4.25-CL-0-FGame_GLSL_ES3_1_ANDROID_00084A4308D90436AC0F652223AA8D4F.rec.upipelinecache
LogRHI: Could not open FPipelineCacheFile: ../../../FGame/Content/PipelineCaches/Android/FGame_GLSL_ES3_1_ANDROID.upipelinecache
...
LogRHI: Display: Encountered a new graphics PSO: 3478445130
LogRHI: Display: New Graphics PSO (3478445130) Description: 416F798F22743F626513A16205A15ECB135C3791,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,<0 1 0 0 1 0 0 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0>,<0.000000 0.000000 2 1 0 0>,<0 3 0 7 0 0 0 0 7 0 0 0 255 255>,1,11,1051148,2,2,0,0,0,4,10,1114633,0,0,18,2569,0,0,37,1051145,0,0,37,2585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15360,15391,1055,1027,1025,0,0,0,0,1,<0 0 3 0 12 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>
LogRHI: Display: Encountered a new graphics PSO: 898510125
LogRHI: Display: New Graphics PSO (898510125) Description: 432A3E5557E9ED8328AC7E6CDA5AA6FC2F0B2439,0FED48EC019CFDD251488DE33D563DCFFD7E69DA,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,<0 1 0 0 1 0 7 0 1 0 0 1 0 0 0 1 0 0 1 0 0 0 1 0 0 1 0 0 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0>,<0.000000 0.000000 2 0 0 0>,<0 7 0 7 0 0 0 0 7 0 0 0 255 255>,1,11,1051148,2,2,0,0,0,4,10,1114633,0,0,18,2569,0,0,37,1051145,0,0,37,2585,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15360,15391,1055,1027,1025,0,0,0,2,2,<0 0 4 0 32 0>,<0 16 2 1 32 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>

并会在 Saved/CoolectedPSOs 中创建以下文件:

生成 *.stablepc.csv

使用以下 commandlet:

1
2
3
4
5
6
Engine/Binaries/Win64/UE4Editor-Cmd.exe
D:/Client/Client.uproject
-run=ShaderPipelineCacheTools expand
D:/PSOCache/*.rec.upipelinecache
D:/PSOCache/*.scl.csv
D:/PSOCache/Client_GLSL_ES3_1_ANDROID.stablepc.csv

以上命令会在引擎的 Binaries/Win64 下生成 Client_GLSL_ES3_1_ANDROID.stablepc.csv 文件,注意一定要匹配 {PROJECTNAME}_{SHADER_FORMART_NAME}.stablepc.csv 这个命名规则。

Android 的命名为:Client_GLSL_ES3_1_ANDROID.stablepc.csv
IOS 的命名为:Client_SF_METAL.stablepc.csv

生成时具有以下 Log:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
D:\PSOCache>"C:\Program Files\Epic Games\UE_4.25\Engine\Binaries\Win64\UE4Editor-Cmd.exe" "D:\PSOExample\PSOExample.uproject" -run=ShaderPipelineCacheTools expand D:/PSOCache/*.rec.upipelinecache D:/PSOCache/*.scl.csv D:/PSOCache/PSOExample_GLSL_ES3_1_ANDROID.stablepc.csv
[2021.04.22-08.57.39:623][0]LogTargetPlatformManager: Display: Building Assets For Windows
[2021.04.22-08.57.39:648][0]LogAudioDebug: Display: Lib vorbis DLL was dynamically loaded.
[2021.04.22-08.57.39:841][0]LogShaderCompilers: Display: Using Local Shader Compiler.
[2021.04.22-08.57.40:492][0]LogDerivedDataCache: Display: Max Cache Size: 512 MB
[2021.04.22-08.57.40:523][0]LogDerivedDataCache: Display: Loaded Boot cache: C:/Users/lipengzha/AppData/Local/UnrealEngine/4.25/DerivedDataCache/Boot.ddc
[2021.04.22-08.57.40:533][0]LogDerivedDataCache: Display: Pak cache opened for reading ../../../Engine/DerivedDataCache/Compressed.ddp.
[2021.04.22-08.57.44:616][0]LogAudioCaptureCore: Display: No Audio Capture implementations found. Audio input will be silent.
[2021.04.22-08.57.44:616][0]LogAudioCaptureCore: Display: No Audio Capture implementations found. Audio input will be silent.
[2021.04.22-08.57.45:274][0]LogShaderPipelineCacheTools: Display: Loading D:/PSOCache/ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv...
[2021.04.22-08.57.45:275][0]LogShaderPipelineCacheTools: Display: Loading D:/PSOCache/ShaderStableInfo-PSOExample-GLSL_ES3_1_ANDROID.scl.csv...
[2021.04.22-08.57.45:280][0]LogShaderPipelineCacheTools: Display: Loaded 548 shader info lines from D:/PSOCache/ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv.
[2021.04.22-08.57.45:287][0]LogShaderPipelineCacheTools: Display: Loaded 5707 shader info lines from D:/PSOCache/ShaderStableInfo-PSOExample-GLSL_ES3_1_ANDROID.scl.csv.
[2021.04.22-08.57.45:287][0]LogShaderPipelineCacheTools: Display: Loaded 6255 unique shader info lines total.
[2021.04.22-08.57.45:289][0]LogShaderPipelineCacheTools: Display: Loading D:/PSOCache/++UE4+Release-4.25-CL-13942748-PSOExample_GLSL_ES3_1_ANDROID_000843BC08D905AF09E24614C6A6086F.rec.upipelinecache....
[2021.04.22-08.57.45:293][0]LogShaderPipelineCacheTools: Display: Loaded 105 PSOs
[2021.04.22-08.57.45:297][0]LogShaderPipelineCacheTools: Display: Loaded 105 PSOs total [Usage Mask Merged = 0].
[2021.04.22-08.57.45:325][0]LogShaderPipelineCacheTools: Display: Generated 478 stable PSOs total
[2021.04.22-08.57.45:344][0]LogShaderPipelineCacheTools: Display: Wrote stable PSOs, 479 lines (541.8 KB) to D:/PSOCache/PSOExample_GLSL_ES3_1_ANDROID.stablepc.csv
[2021.04.22-08.57.45:346][0]LogInit: Display:
[2021.04.22-08.57.45:349][0]LogInit: Display: Success - 0 error(s), 0 warning(s)
[2021.04.22-08.57.45:353][0]LogInit: Display:
Execution of commandlet took: 0.08 seconds
[2021.04.22-08.57.45:408][0]LogShaderCompilers: Display: Shaders left to compile 0
[2021.04.22-08.57.45:621][0]LogContentStreaming: Display: There are 1 unreleased StreamingManagers

最终 PSO 所需要的所有文件:

1
2
3
4
5
6
7
8
D:\PSOCache>tree /a /f
卷 Data 的文件夹 PATH 列表
卷序列号为 004B-E876
D:.
++UE4+Release-4.25-CL-13942748-PSOExample_GLSL_ES3_1_ANDROID_000843BC08D905AF09E24614C6A6086F.rec.upipelinecache
PSOExample_GLSL_ES3_1_ANDROID.stablepc.csv
ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv
ShaderStableInfo-PSOExample-GLSL_ES3_1_ANDROID.scl.csv

我把测试工程生成的文件备份了一份:PSOCache.7z,可以查看每个文件中的内容。

生成 *.stable.upipelinecache

把生成的 *stablepc.csv 放到 Build/Android/PipelineCaches 目录下,注意 Build/PLATFORM 这个 Platform 是编译平台,不是 Cook 的资源平台,Android 的包就是 Android 而不是 Android_ASTC 等。
之后重新打包即可。

引擎在 Cook 时通过 stavlepc.csv 创建 PipelineCache 的代码:

Editor\UnrealEd\Private\CookOnTheFlyServer.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
void UCookOnTheFlyServer::CreatePipelineCache(const ITargetPlatform* TargetPlatform, const FString& LibraryName)
{
// make sure we have a registry generated for all the platforms
const FString TargetPlatformName = TargetPlatform->PlatformName();
TArray<FString>* SCLCSVPaths = OutSCLCSVPaths.Find(FName(TargetPlatformName));
if (SCLCSVPaths && SCLCSVPaths->Num())
{
TArray<FName> ShaderFormats;
TargetPlatform->GetAllTargetedShaderFormats(ShaderFormats);
for (FName ShaderFormat : ShaderFormats)
{
// *stablepc.csv or *stablepc.csv.compressed
const FString Filename = FString::Printf(TEXT("*%s_%s.stablepc.csv"), *LibraryName, *ShaderFormat.ToString());
const FString StablePCPath = FPaths::ProjectDir() / TEXT("Build") / TargetPlatform->IniPlatformName() / TEXT("PipelineCaches") / Filename;
const FString StablePCPathCompressed = StablePCPath + TEXT(".compressed");

TArray<FString> ExpandedFiles;
IFileManager::Get().FindFilesRecursive(ExpandedFiles, *FPaths::GetPath(StablePCPath), *FPaths::GetCleanFilename(StablePCPath), true, false, false);
IFileManager::Get().FindFilesRecursive(ExpandedFiles, *FPaths::GetPath(StablePCPathCompressed), *FPaths::GetCleanFilename(StablePCPathCompressed), true, false, false);
if (!ExpandedFiles.Num())
{
UE_LOG(LogCook, Display, TEXT("---- NOT Running UShaderPipelineCacheToolsCommandlet for platform %s shader format %s, no files found at %s"), *TargetPlatformName, *ShaderFormat.ToString(), *StablePCPath);
}
else
{
UE_LOG(LogCook, Display, TEXT("---- Running UShaderPipelineCacheToolsCommandlet for platform %s shader format %s"), *TargetPlatformName, *ShaderFormat.ToString());

const FString OutFilename = FString::Printf(TEXT("%s_%s.stable.upipelinecache"), *LibraryName, *ShaderFormat.ToString());
const FString PCUncookedPath = FPaths::ProjectDir() / TEXT("Content") / TEXT("PipelineCaches") / TargetPlatform->IniPlatformName() / OutFilename;

if (IFileManager::Get().FileExists(*PCUncookedPath))
{
UE_LOG(LogCook, Warning, TEXT("Deleting %s, cooked data doesn't belong here."), *PCUncookedPath);
IFileManager::Get().Delete(*PCUncookedPath, false, true);
}

const FString PCCookedPath = ConvertToFullSandboxPath(*PCUncookedPath, true);
const FString PCPath = PCCookedPath.Replace(TEXT("[Platform]"), *TargetPlatformName);


FString Args(TEXT("build "));
Args += TEXT("\"");
Args += StablePCPath;
Args += TEXT("\"");

int32 NumMatched = 0;
for (int32 Index = 0; Index < SCLCSVPaths->Num(); Index++)
{
if (!(*SCLCSVPaths)[Index].Contains(ShaderFormat.ToString()))
{
continue;
}
NumMatched++;
Args += TEXT(" ");
Args += TEXT("\"");
Args += (*SCLCSVPaths)[Index];
Args += TEXT("\"");
}
if (!NumMatched)
{
UE_LOG(LogCook, Warning, TEXT("Shader format %s for platform %s had this file %s, but no .scl.csv files."), *ShaderFormat.ToString(), *TargetPlatformName, *StablePCPath);
for (int32 Index = 0; Index < SCLCSVPaths->Num(); Index++)
{
UE_LOG(LogCook, Warning, TEXT(" .scl.csv file: %s"), *((*SCLCSVPaths)[Index]));
}
continue;
}

Args += TEXT(" ");
Args += TEXT("\"");
Args += PCPath;
Args += TEXT("\"");
UE_LOG(LogCook, Display, TEXT(" With Args: %s"), *Args);

int32 Result = UShaderPipelineCacheToolsCommandlet::StaticMain(Args);

if (Result)
{
LogCookerMessage(FString::Printf(TEXT("UShaderPipelineCacheToolsCommandlet failed %d"), Result), EMessageSeverity::Error);
}
else
{
UE_LOG(LogCook, Display, TEXT("---- Done running UShaderPipelineCacheToolsCommandlet for platform %s"), *TargetPlatformName);
}
}
}
}
}

实际使用 stablepc.csv 的地方就是用它来执行 ShaderPipelineCacheTools 这个 commandlet 生成 upipelinecache 文件并打至包内。

ShaderPipelineCacheToolsCommandlet 的执行命令为:

1
2
3
4
5
6
7
Engine\Binaries\Win64\UE4Editor-Cmd.exe
D:\PSOExample\PSOExample.uproject
-run=ShaderPipelineCacheTools build
"D:\PSOExample/Build/Android/PipelineCaches/*PSOExample_GLSL_ES3_1_ANDROID.stablepc.csv"
"D:\PSOExample/Saved/Cooked/Android_ASTC/PSOExample/Metadata/PipelineCaches/ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv"
"D:\PSOExample/Saved/Cooked/Android_ASTC/PSOExample/Metadata/PipelineCaches/ShaderStableInfo-PSOExample-GLSL_ES3_1_ANDROID.scl.csv"
"D:\PSOExample/Saved/Cooked/Android_ASTC/PSOExample/Content/PipelineCaches/Android/PSOExample_GLSL_ES3_1_ANDROID.stable.upipelinecache"

生成 *.stable.upipelinecache 文件的包内路径为Content\PipelineCaches\Android

1
2
3
4
5
6
D:\PSOExample\Saved\Cooked\Android_ASTC\PSOExample\Content\PipelineCaches>tree /a /f
卷 Windows 的文件夹 PATH 列表
卷序列号为 0C49-9EA3
C:.
\---Android
PSOExample_GLSL_ES3_1_ANDROID.stable.upipelinecache

因为它是位于 Content 下并会打包进 pak 的文件,我们也可以对其进行热更。

当安装了包含 upipelinecache 的包,在运行时就会有以下 log:

1
2
3
4
5
6
7
8
9
10
11
LogShaderLibrary: Display: Using ../../../PSOExample/Content/ShaderArchive-PSOExample-GLSL_ES3_1_ANDROID.ushaderbytecode for material shader code. Total 3053 unique shaders.
LogShaderLibrary: Display: Cooked Context: Using Shared Shader Library PSOExample
LogRHI: Display: Opened pipeline cache after state change and enqueued 0 of 0 tasks for precompile.
LogRHI: Base name for record PSOs is ../../../PSOExample/Saved/CollectedPSOs/++UE4+Release-4.25-CL-13942748-PSOExample_GLSL_ES3_1_ANDROID_00087B4B08D905BBC5A827F40CA03A0C.rec.upipelinecache
LogRHI: FPipelineCacheFile Header Game ue_version: 13942748
LogRHI: FPipelineCacheFile Header Engine Data ue_version: 17
LogRHI: FPipelineCacheFile Header TOC Offset: 38155
LogRHI: FPipelineCacheFile File Size: 51011 Bytes
LogRHI: Opened FPipelineCacheFile: ../../../PSOExample/Content/PipelineCaches/Android/PSOExample_GLSL_ES3_1_ANDROID.stable.upipelinecache (GUID: 00000000000000000000000000000000) with 102 entries.
LogRHI: Scanning Binary program cache, using Shader Pipeline Cache version 6988202F47BA858F3F0DE483D7DB0606
LogRHI: AndroidEGL:SwapBuffers eglGetCompositorTimingANDROID EGL_COMPOSITE_DEADLINE_ANDROID=2718926192606265, EGL_COMPOSITE_INTERVAL_ANDROID=16559027, EGL_COMPOSITE_TO_PRESENT_LATENCY_ANDROID=14559027

PSO Cache 的加载与热更

ShaderCode类似,引擎在启动时也是会自动加载 PSO Cache 的,在 FEngineLoop 中通过调用 FPipelineCacheFile::OpenPipelineFileCache 读取 *.stable.upipelinecache 的。

PreInitPreStartupScreen 中加载 PSO 的代码:

Runtime/Launch/Private/LaunchEngineLoop.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
int32 FEngineLoop::PreInitPreStartupScreen(const TCHAR* CmdLine)
{
// ...
{
bool bUseCodeLibrary = FPlatformProperties::RequiresCookedData() || GAllowCookedDataInEditorBuilds;
if (bUseCodeLibrary)
{
{
SCOPED_BOOT_TIMING("FShaderCodeLibrary::InitForRuntime");
// Will open material shader code storage if project was packaged with it
// This only opens the Global shader library, which is always in the content dir.
FShaderCodeLibrary::InitForRuntime(GMaxRHIShaderPlatform);
}

#if !UE_EDITOR
// Cooked data only - but also requires the code library - game only
if (FPlatformProperties::RequiresCookedData())
{
SCOPED_BOOT_TIMING("FShaderPipelineCache::Initialize");
// Initialize the pipeline cache system. Opening is deferred until the manual call to
// OpenPipelineFileCache below, after content pak's ShaderCodeLibraries are loaded.
FShaderPipelineCache::Initialize(GMaxRHIShaderPlatform);
}
#endif //!UE_EDITOR
}
}
// ...
//Handle opening shader library after our EarlyLoadScreen
{
LLM_SCOPE(ELLMTag::Shaders);
SCOPED_BOOT_TIMING("FShaderCodeLibrary::OpenLibrary");

// Open the game library which contains the material shaders.
FShaderCodeLibrary::OpenLibrary(FApp::GetProjectName(), FPaths::ProjectContentDir());
for (const FString& RootDir : FPlatformMisc::GetAdditionalRootDirectories())
{
FShaderCodeLibrary::OpenLibrary(FApp::GetProjectName(), FPaths::Combine(RootDir, FApp::GetProjectName(), TEXT("Content")));
}

// Now our shader code main library is opened, kick off the precompile, if already initialized
FShaderPipelineCache::OpenPipelineFileCache(GMaxRHIShaderPlatform);
}
// ...
}

FShaderPipelineCache::OpenPipelineFileCache有两个重载版本:

Runtime\RenderCore\Private\ShaderPipelineCache.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
bool FShaderPipelineCache::OpenPipelineFileCache(EShaderPlatform Platform)
{
bool bFileOpen = false;
if (GConfig)
{
FString LastOpenedName;
if ((GConfig->GetString(FShaderPipelineCacheConstants::SectionHeading, FShaderPipelineCacheConstants::LastOpenedKey, LastOpenedName, *GGameUserSettingsIni) || GConfig->GetString(FShaderPipelineCacheConstants::SectionHeading, FShaderPipelineCacheConstants::LastOpenedKey, LastOpenedName, *GGameIni)) && LastOpenedName.Len())
{
bFileOpen = OpenPipelineFileCache(LastOpenedName, Platform);
}
}

if (!bFileOpen)
{
bFileOpen = OpenPipelineFileCache(FApp::GetProjectName(), Platform);
}

return bFileOpen;
}

bool FShaderPipelineCache::OpenPipelineFileCache(FString const& Name, EShaderPlatform Platform)
{
if (ShaderPipelineCache)
return ShaderPipelineCache->Open(Name, Platform);
else
return false;
}

引擎启动的时候默认读取的就是 OpenPipelineFileCache(FApp::GetProjectName(), Platform),也就是PSOExample_GLSL_ES3_1_ANDROID.stable.upipelinecache,Platform 参数可以通过传递全局对象GMaxRHIShaderPlatform 来获取当前运行的平台。

UE 也提供了一个 console 命令可以指定加载stable.upipelinecacher.ShaderPipelineCache.Open,还有几个其他控制 PSO 的 console 命令:

Runtime/RenderCore/Private/ShaderPipelineCache.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
static FAutoConsoleCommand LoadPipelineCacheCmd(
TEXT("r.ShaderPipelineCache.Open"),
TEXT("Takes the desired filename to open and then loads the pipeline file cache."),
FConsoleCommandWithArgsDelegate::CreateStatic(ConsoleCommandLoadPipelineFileCache)
);

static FAutoConsoleCommand SavePipelineCacheCmd(
TEXT("r.ShaderPipelineCache.Save"),
TEXT("Save the current pipeline file cache."),
FConsoleCommandDelegate::CreateStatic(ConsoleCommandSavePipelineFileCache)
);

static FAutoConsoleCommand ClosePipelineCacheCmd(
TEXT("r.ShaderPipelineCache.Close"),
TEXT("Close the current pipeline file cache."),
FConsoleCommandDelegate::CreateStatic(ConsoleCommandClosePipelineFileCache)
);

static FAutoConsoleCommand SwitchModePipelineCacheCmd(
TEXT("r.ShaderPipelineCache.SetBatchMode"),
TEXT("Sets the compilation batch mode, which should be one of:\n\tPause: Suspend precompilation.\n\tBackground: Low priority precompilation.\n\tFast: High priority precompilation."),
FConsoleCommandWithArgsDelegate::CreateStatic(ConsoleCommandSwitchModePipelineCacheCmd)
);

这样只需要在热更包中包含最新的 *.stable.upipelinecache,之后调用OpenPipelineFileCache 加载最新的 PSO Cache 即可,可以与 ShaderCode 的热更流程保持一致。

生成新的 PSO Cache 需要关键的两种数据:

  1. 运行时捕获的 PSO 数据(upipelinecache)
  2. ShaderStableInfo(位于 Metadata 目录下)

因为 ShaderCode 是可以热更的,而 ShaderStableInfo 可以通过 Cook 最新的工程获得,所以 PSO Cache 也是可以通过热更 Shader 并不断地捕获最新的 PSO 数据进行迭代更新的。
准备有时间给 HotPacther 中增加 PSO Caching 的热更功能,这样也可以把 PSO Caching 的部署和打包集成至自动化地热更流程,先挖个坑。

因为当开启了 r.ShaderPipelineCache.Enabled=1,在引擎启动时就会自动加载项目的 PSO Cache,而引擎中做了限制,只能够加载一次,后续调用OpenPipelineFileCache 的都不会被加载:

Runtime\RHI\Private\PipelineFileCache.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
bool FPipelineFileCache::OpenPipelineFileCache(FString const& Name, EShaderPlatform Platform, FGuid& OutGameFileGuid)
{
bool bOk = false;
OutGameFileGuid = FGuid();

if(IsPipelineFileCacheEnabled())
{
FRWScopeLock Lock(FileCacheLock, SLT_Write);

if(FileCache == nullptr)
{
FileCache = new FPipelineCacheFile();

bOk = FileCache->OpenPipelineFileCache(Name, Platform, OutGameFileGuid);

// File Cache now exists - these caches should be empty for this file otherwise will have false positives from any previous file caching - if not something has been caching when it should not be
check(NewPSOs.Num() == 0);
check(NewPSOHashes.Num() == 0);
check(RunTimeToPSOUsage.Num() == 0);
}
}

return bOk;
}

当引擎默认加载执行之后 FileCache 就不为 nullptr 了,后续所有的加载调用都会直接返回 false,解决办法就是,让引擎启动时不自动加载 PSO Cache,等到运行时热更之后由我们手动加载,翻了下代码,可以从这个IsPipelineFileCacheEnabled 检测中做:

Runtime\RHI\Private\PipelineFileCache.cpp
1
2
3
4
5
6
7
8
9
10
11
12
bool FPipelineFileCache::IsPipelineFileCacheEnabled()
{
static bool bOnce = false;
static bool bCmdLineForce = false;
if (!bOnce)
{
bOnce = true;
bCmdLineForce = FParse::Param(FCommandLine::Get(), TEXT("psocache"));
UE_CLOG(bCmdLineForce, LogRHI, Warning, TEXT("****************************** Forcing PSO cache from command line"));
}
return FileCacheEnabled && (bCmdLineForce || CVarPSOFileCacheEnabled.GetValueOnAnyThread() == 1);
}

它的返回值依赖了两个值:FileCacheEnabled以及CVarPSOFileCacheEnabled

FileCacheEnabledFPipelineFileCache::Initialize 中被赋值,IOS 之外的平台总是 true,IOS 则依赖于FPipelineFileCache::ShouldEnableFileCache 的结果。

CVarPSOFileCacheEnabled是一个控制台变量,用来控制 r.ShaderPipelineCache.Enabled 的值:

Runtime\RHI\Private\PipelineFileCache.cpp
1
2
3
4
5
6
static TAutoConsoleVariable<int32> CVarPSOFileCacheEnabled(
TEXT("r.ShaderPipelineCache.Enabled"),
PIPELINE_CACHE_DEFAULT_ENABLED,
TEXT("1 Enables the PipelineFileCache, 0 disables it."),
ECVF_Default | ECVF_RenderThreadSafe
);

我们需要做的有三步:

  1. 引擎默认启动时 CVarPSOFileCacheEnabled 的值为 false
  2. 运行时手动修改 CVarPSOFileCacheEnabled 值,开启 PSO Cache
  3. 加载 PSO Cache

具体实现流程:

  1. DefaultEngine.ini 中将 r.ShaderPipelineCache.Enabled=0 并打包
DefaultEngine.ini
1
2
[ConsoleVariables]
r.ShaderPipelineCache.Enabled=0

然后写两个函数在运行时开启和加载 PSO:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#include "ShaderPipelineCache.h"
#include "RHIShaderFormatDefinitions.inl"
#include "HAL/IConsoleManager.h"

bool UFlibShaderPipelineCacheHelper::EnableShaderPipelineCache(bool bEnable)
{
UE_LOG(LogHotPatcher,Display,TEXT("EnableShaderPipelineCache %s"),bEnable?TEXT("true"):TEXT("false"));
auto Var = IConsoleManager::Get().FindConsoleVariable(TEXT("r.ShaderPipelineCache.Enabled"));
if(Var)
{
Var->Set(bEnable ? 1 : 0);
}
return !!Var;
}

bool UFlibShaderPipelineCacheHelper::LoadShaderPipelineCache(const FString& Name)
{
UE_LOG(LogHotPatcher,Display,TEXT("Load Shader pipeline cache %s for platform %d"),*Name,*ShaderPlatformToShaderFormatName(GMaxRHIShaderPlatform).ToString());
return FShaderPipelineCache::OpenPipelineFileCache(Name,GMaxRHIShaderPlatform);
}

这样即可实现 PSO Cache 的延迟加载,手动加载时机在热更之后加载即可。

延迟采集和存储

因为采集和存储 PSO Cache 具有额外的性能消耗,所以可以把采集和存储 PSO 数据关闭,根据需求在运行时再开启。
DefaultEngine.ini 中关闭 LogPSOSaveBoundPSOLog,打基础包时就不会自动采集和自动存储了:

DefaultEngine.ini
1
2
3
[ConsoleVariables]
r.ShaderPipelineCache.LogPSO=0
r.ShaderPipelineCache.SaveBoundPSOLog=0

然后在运行时开启:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
UENUM(BlueprintType)
enum class EPSOSaveMode : uint8
{
Incremental = 0, // Fast(er) approach which saves new entries incrementally at the end of the file, replacing the table-of-contents, but leaves everything else alone.
BoundPSOsOnly = 1, // Slower approach which consolidates and saves all PSOs used in this run of the program, removing any entry that wasn't seen, and sorted by the desired sort-mode.
SortedBoundPSOs = 2 // Slow save consolidates all PSOs used on this device that were never part of a cache file delivered in game-content, sorts entries into the desired order and will thus read-back from disk.
};

bool UFlibShaderPipelineCacheHelper::SavePipelineFileCache(EPSOSaveMode Mode)
{
return FShaderPipelineCache::SavePipelineFileCache((FPipelineFileCache::SaveMode)Mode);
}

bool UFlibShaderPipelineCacheHelper::EnableLogPSO(bool bEnable)
{
UE_LOG(LogHotPatcher,Display,TEXT("EnableLogPSO %s"),bEnable?TEXT("true"):TEXT("false"));
auto Var = IConsoleManager::Get().FindConsoleVariable(TEXT("r.ShaderPipelineCache.LogPSO"));
if(Var)
{
Var->Set(bEnable ? 1 : 0);
}
return !!Var;
}

bool UFlibShaderPipelineCacheHelper::EnableSaveBoundPSOLog(bool bEnable)
{
UE_LOG(LogHotPatcher,Display,TEXT("EnableSaveBoundPSOLog %s"),bEnable?TEXT("true"):TEXT("false"));
auto Var = IConsoleManager::Get().FindConsoleVariable(TEXT("r.ShaderPipelineCache.SaveBoundPSOLog"));
if(Var)
{
Var->Set(bEnable ? 1 : 0);
}
return !!Var;
}

开启 SaveBoundPSOLog 后会自动存储采集的 PSO 数据,可以不开启自动存储,在运行时通过调用 FShaderPipelineCache::SavePipelineFileCache 手动存储。

错误处理

Cook 没有生成.scl.csv

注意,一定要为项目开启ShaderStableKeys,不然不会生成.scl.csv 文件。

运行时没有生成 upipelinecache 文件

请严格按照 运行时捕获 PSO 数据 中的步骤执行。

  1. 确认是否开启r.ShaderPipelineCache.Enabled(DefaultEngine.ini 或 DeviceProfile)。
  2. 在 ue4commandline.txt 中添加 -logPSO 参数。

Bad PSO

如果使用公版引擎,上述流程就是完整的流程,但是有时项目需要修改引擎支持一些渲染特性,如添加 Multi-subpasshint 支持:

Runtime\RHI\Public\PipelineFileCache.h
1
2
3
4
5
6
7
8
9
10
11
12
struct RHI_API FPipelineCacheFileFormatPSO
{
// ...
struct RHI_API GraphicsDescriptor
{
// uint8 SubpassHint; to SubpassHint[8];
uint8 SubpassHint[8];
uint8 SubpassIndex;
// ...
};
// ...
};

这处变动需要同时修改 GraphicsDescriptor::StateToString()GraphicsDescriptor::StateFromString()两个函数,加入 multi-subpasshint 的序列化支持。

但是,修改之后使用 -run=ShaderPipelineCacheTools 生成 *.stablepc.csv 时有以下报错的 Log:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
LogShaderPipelineCacheTools: Expanding matched    1 files: D:\PipelineCaches\*.rec.upipelinecache
LogShaderPipelineCacheTools: : D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache
LogShaderPipelineCacheTools: Expanding matched 2 files: D:\PipelineCaches\*.scl.csv
LogShaderPipelineCacheTools: : D:\PipelineCaches\ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv
LogShaderPipelineCacheTools: : D:\PipelineCaches\ShaderStableInfo-PSOCaching-GLSL_ES3_1_ANDROID.scl.csv
LogShaderPipelineCacheTools: Display: Loading D:\PipelineCaches\ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv...
LogShaderPipelineCacheTools: Display: Loading D:\PipelineCaches\ShaderStableInfo-PSOCaching-GLSL_ES3_1_ANDROID.scl.csv...
LogShaderPipelineCacheTools: Display: Loaded 926 shader info lines from D:\PipelineCaches\ShaderStableInfo-Global-GLSL_ES3_1_ANDROID.scl.csv.
LogShaderPipelineCacheTools: Display: Loaded 9266 shader info lines from D:\PipelineCaches\ShaderStableInfo-PSOCaching-GLSL_ES3_1_ANDROID.scl.csv.
LogShaderPipelineCacheTools: Display: Loaded 10192 unique shader info lines total.
LogShaderPipelineCacheTools: Display: Loading D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache....
LogShaderPipelineCacheTools: Display: Loaded 115 PSOs
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]
LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: D:\PipelineCaches\++UE4+Release-4.25-CL-0-PSOCaching_GLSL_ES3_1_ANDROID_0008BF4D08D9069169A7B6820601243D.rec.upipelinecache]

这是因为 PSO 数据从 String 的可逆性验证失败了:

Editor\UnrealEd\Private\Commandlets\ShaderPipelineCacheToolsCommandlet.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
bool CheckPSOStringInveribility(const FPipelineCacheFileFormatPSO& Item)
{
FPipelineCacheFileFormatPSO TempItem(Item);
TempItem.Hash = 0;

FString StringRep;
if (Item.Type == FPipelineCacheFileFormatPSO::DescriptorType::Compute)
{
StringRep = TempItem.ComputeDesc.ToString();
}
else
{
StringRep = TempItem.GraphicsDesc.ToString();
}
FPipelineCacheFileFormatPSO DupItem;
FMemory::Memzero(DupItem.GraphicsDesc);
DupItem.Type = Item.Type;
DupItem.UsageMask = Item.UsageMask;
if (Item.Type == FPipelineCacheFileFormatPSO::DescriptorType::Compute)
{
DupItem.ComputeDesc.FromString(StringRep);
}
else
{
DupItem.GraphicsDesc.FromString(StringRep);
}
UE_LOG(LogShaderPipelineCacheTools, Verbose, TEXT("CheckPSOStringInveribility: %s"), *StringRep);

return (DupItem == TempItem) && (GetTypeHash(DupItem) == GetTypeHash(TempItem));
}

关键部分在于 DupItem.GraphicsDesc.FromString(StringRep); 这行代码中 GraphicsDesc 的数据没有恢复成功。

经过调试发现,引擎中具有记录 PipelineCacheGraphicsDesc 的字符串中可被解析元素的数量,也就是生成的 *.stablepc.csv 中第二列中数据的数量,公版引擎中默认是 63 个,使用 FPipelineCacheGraphicsDescPartsNum 记录:

Runtime\RHI\Private\PipelineFileCache.cpp
1
const int32  FPipelineCacheGraphicsDescPartsNum = 63; // parser will expect this number of parts in a description string

生成的 *.stablepc.csv 中各项状态和数据如下:

刚好是 63 个。需要注意的是,在 GraphicsDescriptor::StateFromString 对数据的数量做了检测,FromString 的数据数量要与 FPipelineCacheGraphicsDescPartsNum 的值一致:

Runtime\RHI\Private\PipelineFileCache.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
bool FPipelineCacheFileFormatPSO::GraphicsDescriptor::StateFromString(const FStringView& Src)
{
constexpr int32 PartCount = FPipelineCacheGraphicsDescPartsNum;

TArray<FStringView, TInlineAllocator<PartCount>> Parts;
UE::String::ParseTokens(Src.TrimStartAndEnd(), TEXT(','), [&Parts](FStringView Part) { Parts.Add(Part); });

// check if we have expected number of parts
if (Parts.Num() != PartCount)
{
// instead of crashing let caller handle this case
return false;
}
// ...
}

因为我们增加了 multi-subpasshint 支持,把 SubpassHintuint8改成了 uint8[8],增加了 7 个数据,所以与之对应的FPipelineCacheGraphicsDescPartsNum 也要加 7,改为 70,上面 StateFromString 验证才能够通过。

修改之后再通过 -run=ShaderPipelineCacheTools 生成 *.stablepc.csv 就没有 Bas PSO 的错误了。

使用与配置

可以通过 FShaderPipelineCache 的函数在运行时控制构建 PSO 数据:

Runtime\RenderCore\Public\ShaderPipelineCache.h
1
2
3
4
5
6
7
8
9
10
/** Pauses precompilation. */
static void PauseBatching();
/** Resumes precompilation batching. */
static void ResumeBatching();
/** Returns the number of pipelines waiting for precompilation. */
static uint32 NumPrecompilesRemaining();
/** Returns the number of pipelines actively being precompiled this frame. */
static uint32 NumPrecompilesActive();
/** Sets the precompilation batching mode. */
static void SetBatchMode(BatchMode Mode);

官方建议的做法是在加载屏幕时等待 PSO 构建完毕,再把 LoadingScreen 隐藏:

1
2
3
4
5
6
7
8
if(FShaderPipelineCache::NumPrecompilesRemaining() > 0)
{
if (OutDebugReason != nullptr)
{
*OutDebugReason = FString(TEXT("PC: PSO cache still compiling"));
}
return true;
}

也可以在打开 UI、过场动画、暂停菜单时构建,通过以下三个函数组合处理:

1
2
3
4
5
6
7
8
9
10
11
12
// 暂停 PSO 缓存编译
FShaderPipelineCache::PauseBatching();
// 设置 PSO 的处理模式
// enum class BatchMode
// {
// Background, // The maximum batch size is defined by r.ShaderPipelineCache.BackgroundBatchSize
// Fast, // The maximum batch size is defined by r.ShaderPipelineCache.BatchSize
// Precompile // The maximum batch size is defined by r.ShaderPipelineCache.PrecompileBatchSize
// };
FShaderPipelineCache::SetBatchMode(FShaderPipelineCache::BatchMode::Background);
// 恢复编译 PSO
static void ResumeBatching();

也可以使用配置在游戏启动时自动构建,修改 DefaultEngine.ini[ConsoleVariables]的配置,它们否时定义在 Runtime\RenderCore\Private\ShaderPipelineCache.cpp 中的ConsoleVariable,可以根据自己的需要在运行时或配置文件中进行修改:

PSO 引擎默认配置:

DefaultEngine.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
[ConsoleVariables]
;Sets the startup mode for the PSO cache, determining what the cache does after initialisation:
;0: Precompilation is paused and nothing will compile until a call to ResumeBatching().
;1: Precompilation is enabled in the 'Fast' mode.
;2: Precompilation is enabled in the 'Background' mode.
;Default is 1.
r.ShaderPipelineCache.StartupMode=1

;Set the number of PipelineStateObjects to compile in a single batch operation when compiling takes priority. Defaults to a maximum of 50 per frame, due to async. file IO it is less in practice.
r.ShaderPipelineCache.BackgroundBatchSize=1

;Set the number of PipelineStateObjects to compile in a single batch operation when pre-optimizing the cache. Defaults to a maximum of 50 per frame, due to async. file IO it is less in practice.
r.ShaderPipelineCache.PrecompileBatchSize=50

;The target time (in ms) to spend precompiling each frame when in the background or 0.0 to disable. When precompiling is faster the batch size will grow and when slower will shrink to attempt to occupy the full amount. Defaults to 0.0 (off).
r.ShaderPipelineCache.BackgroundBatchTime=0.0

;The target time (in ms) to spend precompiling each frame when compiling takes priority or 0.0 to disable. When precompiling is faster the batch size will grow and when slower will shrink to attempt to occupy the full amount. Defaults to 16.0 (max. ms per-frame of precompilation).
r.ShaderPipelineCache.BatchTime=16.0

;The target time (in ms) to spend precompiling each frame when cpre-optimizing or 0.0 to disable. When precompiling is faster the batch size will grow and when slower will shrink to attempt to occupy the full amount. Defaults to 10.0 (off).
r.ShaderPipelineCache.PrecompileBatchTime=0.0

;Set the number of PipelineStateObjects to log before automatically saving. 0 will disable automatic saving. Shipping defaults to 0, otherwise default is 100.
r.ShaderPipelineCache.SaveAfterPSOsLogged=100

;Set the time where any logged PSO's will be saved if the number is < r.ShaderPipelineCache.SaveAfterPSOsLogged. Disabled when r.ShaderPipelineCache.SaveAfterPSOsLogged is 0

;Set the time where any logged PSO's will be saved if the number is < r.ShaderPipelineCache.SaveAfterPSOsLogged. Disabled when r.ShaderPipelineCache.SaveAfterPSOsLogged is 0
r.ShaderPipelineCache.AutoSaveTime=30

;Mask used to precompile the cache. Defaults to all PSOs (-1)
r.ShaderPipelineCache.PreCompileMask=-1

;Set the time where any logged PSO's will be saved when -logpso is on the command line.
r.ShaderPipelineCache.AutoSaveTimeBoundPSO=10

;If > 0 then a log of all bound PSOs for this run of the program will be saved to a writable user cache file. Defaults to 0 but is forced on with -logpso.
r.ShaderPipelineCache.SaveBoundPSOLog=0

;Set non zero to use GameFileMask during PSO precompile - recording should always save out the usage masks to make that data availble when needed.
r.ShaderPipelineCache.GameFileMaskEnabled=0

;Set non zero to PreOptimize PSOs - this allows some PSOs to be compiled in the foreground before going in to game
r.ShaderPipelineCache.PreOptimizeEnabled=0

;The minimum bind count to allow a PSO to be precompiled. Changes to this value will not affect PSOs that have already been removed from consideration.
r.ShaderPipelineCache.MinBindCount=0

;The maximum time to allow a PSO to be precompiled. if greather than 0, the amount of wall time we will allow pre-compile of PSOs and then switch to background processing.
r.ShaderPipelineCache.MaxPrecompileTime=0.0

我修改的配置:

DefaultEngine.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
[ConsoleVariables]
;Sets the startup mode for the PSO cache, determining what the cache does after initialisation:
;0: Precompilation is paused and nothing will compile until a call to ResumeBatching().
;1: Precompilation is enabled in the 'Fast' mode.
;2: Precompilation is enabled in the 'Background' mode.
;Default is 1.
r.ShaderPipelineCache.StartupMode=1

;Set the number of PipelineStateObjects to compile in a single batch operation when compiling takes priority. Defaults to a maximum of 50 per frame, due to async. file IO it is less in practice.
r.ShaderPipelineCache.BatchSize=50

;The target time (in ms) to spend precompiling each frame when compiling takes priority or 0.0 to disable. When precompiling is faster the batch size will grow and when slower will shrink to attempt to occupy the full amount. Defaults to 16.0 (max. ms per-frame of precompilation).
r.ShaderPipelineCache.BatchTime=16.0

;Set the number of PipelineStateObjects to compile in a single batch operation when compiling takes priority. Defaults to a maximum of 50 per frame, due to async. file IO it is less in practice.
r.ShaderPipelineCache.BackgroundBatchSize=1

;Set the number of PipelineStateObjects to compile in a single batch operation when pre-optimizing the cache. Defaults to a maximum of 50 per frame, due to async. file IO it is less in practice.
r.ShaderPipelineCache.PrecompileBatchSize=50

;The target time (in ms) to spend precompiling each frame when in the background or 0.0 to disable. When precompiling is faster the batch size will grow and when slower will shrink to attempt to occupy the full amount. Defaults to 0.0 (off).
r.ShaderPipelineCache.BackgroundBatchTime=11.0

;The target time (in ms) to spend precompiling each frame when cpre-optimizing or 0.0 to disable. When precompiling is faster the batch size will grow and when slower will shrink to attempt to occupy the full amount. Defaults to 10.0 (off).
r.ShaderPipelineCache.PrecompileBatchTime=10.0

;Set the number of PipelineStateObjects to log before automatically saving. 0 will disable automatic saving. Shipping defaults to 0, otherwise default is 100.
r.ShaderPipelineCache.SaveAfterPSOsLogged=100

;Set the time where any logged PSO's will be saved if the number is < r.ShaderPipelineCache.SaveAfterPSOsLogged. Disabled when r.ShaderPipelineCache.SaveAfterPSOsLogged is 0

;Set the time where any logged PSO's will be saved if the number is < r.ShaderPipelineCache.SaveAfterPSOsLogged. Disabled when r.ShaderPipelineCache.SaveAfterPSOsLogged is 0
r.ShaderPipelineCache.AutoSaveTime=30

;Mask used to precompile the cache. Defaults to all PSOs (-1)
r.ShaderPipelineCache.PreCompileMask=-1

;Set the time where any logged PSO's will be saved when -logpso is on the command line.
r.ShaderPipelineCache.AutoSaveTimeBoundPSO=10

;Set non zero to use GameFileMask during PSO precompile - recording should always save out the usage masks to make that data availble when needed.
r.ShaderPipelineCache.GameFileMaskEnabled=0

;Set non zero to PreOptimize PSOs - this allows some PSOs to be compiled in the foreground before going in to game
r.ShaderPipelineCache.PreOptimizeEnabled=1

;The minimum bind count to allow a PSO to be precompiled. Changes to this value will not affect PSOs that have already been removed from consideration.
r.ShaderPipelineCache.MinBindCount=30

;The maximum time to allow a PSO to be precompiled. if greather than 0, the amount of wall time we will allow pre-compile of PSOs and then switch to background processing.
r.ShaderPipelineCache.MaxPrecompileTime=33

开启了启动时自动构建 PSO 数据,会有以下 log:

1
2
3
4
5
6
7
LogRHI: Base name for record PSOs is ../../../FGame/Saved/CollectedPSOs/++UE4+Release-4.25-CL-0-FGame_SF_METAL_8F3222B7964FE2A89C849E90E0000736.rec.upipelinecache
LogRHI: FPipelineCacheFile Header Game ue_version: 0
LogRHI: FPipelineCacheFile Header Engine Data ue_version: 17
LogRHI: FPipelineCacheFile Header TOC Offset: 293853
LogRHI: FPipelineCacheFile File Size: 380497 Bytes
LogRHI: Opened FPipelineCacheFile: ../../../FGame/Content/PipelineCaches/IOS/FGame_SF_METAL.stable.upipelinecache (GUID: 00000000000000000000000000000000) with 690 entries.
LogRHI: Display: Opened pipeline cache and enqueued 441 of 441 tasks for precompile with BatchSize 50 and BatchTime 10.000000.

也可以在 DefaultGameUserSettings.ini 中设置 PSO 的SortOrder

DefaultGameUserSettings.ini
1
2
3
4
5
6
[ShaderPipelineCache.CacheFile]
;default is 0
;Default = 0, // Whatever order they are already in.
;FirstToLatestUsed = 1, // Start with the PSOs with the lowest first-frame used and work toward those with the highest.
;MostToLeastUsed = 2 // Start with the most often used PSOs working toward the least.
SortOrder=1

参考资料