Compare commits

...

151 Commits

Author SHA1 Message Date
2f444a9fd5 Android #53 2023-08-28 00:57:35 +00:00
ada4697300 Merge pull request #11389 from FernandoS27/discard-fix
Buffer Cache: fix discard writes.
2023-08-27 04:26:59 +02:00
acc99433c7 Buffer Cache: fix discard writes. 2023-08-27 03:45:43 +02:00
6c4abd23be Merge pull request #11356 from lat9nq/console-mode-pg
general,config-qt: Present Console Mode as an enum with separate options in game properties
2023-08-26 19:15:00 -04:00
84b384fbea Merge pull request #11359 from Kelebek1/check_suitable_backend
Pre-test for valid audio backends
2023-08-26 19:14:47 -04:00
3620533995 Merge pull request #11350 from BenjaminHalko/button-padding
ui: Added padding to the reset button
2023-08-26 19:14:30 -04:00
c5105b65d5 Merge pull request #11317 from Kelebek1/macro_dumps
Mark decompiled macros on dump, dump shaders after translation
2023-08-26 19:14:25 -04:00
1ac2615adb Merge pull request #11338 from comex/warning-fixes-august-2023
Warnings cleanup for GCC 13 and Clang 16
2023-08-26 19:14:17 -04:00
d7a0b8c373 Mark decompiled macros as decompiled on dump, dump shaders after translation 2023-08-25 21:47:47 -04:00
6bb02dcb8a Skip additional mbedcrypto warnings options on MSVC 2023-08-25 19:23:34 -04:00
32c453a5f1 Avoid $<CXX_COMPILER_ID:Clang> because it doesn't include AppleClang. 2023-08-25 19:22:31 -04:00
91eb5afd0b Warnings cleanup for GCC 13 and Clang 16
Note: For GCC there are still a huge number of `-Warray-bounds` warnings
coming from `externals/dynarmic`.  I could have added a workaround in
`externals/CMakeLists.txt` similar to what this PR does for other
externals, but given Dynarmic's close affiliation with Yuzu, it would be
better to fix it upstream.

Besides that, on my machine, this makes the build warning-free except
for some warnings from glslangValidator and AutoMoc.

Details:

- Disable some warnings in externals.

- Disable `-Wnullability-completeness`, which is a Clang warning triggered
  by the Vulkan SDK where if any pointers in the header are marked
  _Nullable, it wants all pointers to be marked _Nullable or _Nonnull.
  Most of them are, but some aren't.  Who knows why.

- `src/web_service/verify_user_jwt.cpp`: Disable another warning when
  including `jwt.hpp`.

- `src/input_common/input_poller.cpp`: Add missing `override` specifiers.

- src/common/swap.h: Remove redundant `operator&`.  In general, this
  file declares three overloads of each operator.  Using `+` as an
  example, the overloads are:

  - a member function for `swapped_t + integer`
  - a member function for `swapped_t + swapped_t`
  - a free function for `integer + swapped_t`

  But for `operator&`, there was an additional free function for
  `swapped_t + integer`, which was redundant with the member function.
  This caused a GCC warning saying "ISO C++ says that these are
  ambiguous".
2023-08-25 19:22:31 -04:00
bc4e58eb51 Merge pull request #11377 from BenjaminHalko/reverse-slider-input
ui: Fixed inverted controls on ReverseSlider widgets
2023-08-25 18:06:03 -04:00
8674724ef0 Merge pull request #11378 from t895/game-flag
android: Use appCategory to specify the app is a game
2023-08-25 18:05:58 -04:00
a8edbb7019 Merge pull request #11370 from FearlessTobi/fix-filesize
filesystem: Return correct error for RenameFile when dest_path already exists
2023-08-25 18:02:54 -04:00
d8c8fbe41f Merge pull request #11371 from FearlessTobi/fix-cli-updates
yuzu/main: Ensure NCAs are registered in content provider when launching from CLI
2023-08-25 18:02:47 -04:00
234cc45192 ssl: tolerate handshake without hostname set (#11328) 2023-08-26 00:02:32 +02:00
b923f5aa7e registered_cache: create fake CNMT entries for program updates of multiprogram applications (#11319) 2023-08-26 00:00:15 +02:00
18ad55be0b kernel: offset code entry point for 39-bit address space type (#11326) 2023-08-25 23:59:32 +02:00
4e71628097 android: Use appCategory to specify the app is a game 2023-08-25 17:17:48 -04:00
92e6ff30a1 Merge pull request #11357 from liamwhite/lime-vfs
android: jni: ensure NCAs from loaded filepath are registered in manual content provider
2023-08-25 13:04:22 -07:00
2e55459e03 Updated to only the reset button 2023-08-25 10:45:42 -07:00
8677d98a10 Updated padded style 2023-08-25 10:29:23 -07:00
49df2b9715 ui: Fixed inverted controls on ReverseSlider widgets
fixes: #11236
2023-08-25 10:06:34 -07:00
2f2de400e1 Merge pull request #11375 from liamwhite/nvhost-as-gpu
nvhost_as_gpu: ensure mappings are aligned to big page size when deallocated
2023-08-25 17:04:16 +02:00
9e134c3da2 nvhost_as_gpu: ensure mappings are aligned to big page size when deallocated 2023-08-25 09:39:18 -04:00
59b3c30f94 yuzu/main: Ensure NCAs are registered in content provider when launching from CLI
Fixes updates and DLC not being loaded when launching yuzu from the command line.

Similar to https://github.com/yuzu-emu/yuzu/pull/11357.
Fixes https://github.com/yuzu-emu/yuzu/issues/8352,
2023-08-24 18:48:02 +02:00
a669e37ddb filesystem: Return correct error for RenameFile when dest_path already exists
Allows Grid Autosport to boot.

Fixes https://github.com/yuzu-emu/yuzu/issues/8287.
2023-08-24 17:07:39 +02:00
7d89f2c146 Merge pull request #11327 from liamwhite/skyline-2
sockets: avoid locking around socket session calls
2023-08-24 10:33:53 -04:00
51ffc2c66c Merge pull request #11367 from FearlessTobi/fix-filesize
game_list_worker: Display correct size for NAX games
2023-08-24 10:33:42 -04:00
e41655960e game_list_worker: Display correct size for NAX games
This was a regression from https://github.com/yuzu-emu/yuzu/pull/1837.

Fixes https://github.com/yuzu-emu/yuzu/issues/1938.
2023-08-24 01:16:19 +02:00
1cdd11d9f5 main: Fix docked mode button, clang 14 error 2023-08-23 14:26:34 -04:00
ccd163ab2c Merge pull request #11352 from t895/recurse-subfolders
android: Search game directory recursively
2023-08-23 10:20:02 -04:00
182fb83556 android: Set default build variant to mainlineRelWithDebInfo (#11358) 2023-08-23 16:12:39 +02:00
39c8ddcda2 Pre-test opening a stream for audio backends, fall back to null if not suitable. 2023-08-23 08:33:26 +01:00
2c4ebeb51d android: jni: ensure NCAs from loaded filepath are registered in manual content provider 2023-08-22 22:47:25 -04:00
00af46c356 native: Use Docked Mode helper 2023-08-22 22:40:36 -04:00
ce0f1baf51 main: Access by reference
Old Clang is fussy about this.
2023-08-22 22:35:55 -04:00
75f5b3177d config-android: Translate console mode setting
Translates the previous boolean to the enum.
2023-08-22 22:00:28 -04:00
3c45452fae general: Use console mode helper across project 2023-08-22 21:58:23 -04:00
ab862207d7 settings: Add docked mode helper function 2023-08-22 21:58:09 -04:00
7f8335f4ae config(qt): Sanitize docked handheld controller 2023-08-22 16:07:53 -04:00
6ed5b581f0 shared_translation: Define use_docked_mode texts 2023-08-22 16:07:53 -04:00
387ede76d2 general: Convert use_docked_mode to an enumeration
Allows some special interactions with it in the Qt frontend.
2023-08-22 16:07:52 -04:00
8a4cb3f902 shared_widget: Implement radio groups 2023-08-22 16:07:52 -04:00
35b77b9599 android: Search game directory recursively 2023-08-22 15:16:20 -04:00
bc4ad5e62d Merge pull request #11302 from vonchenplus/vulkan_macos
Add macos moltenvk bundle, Add copy moltevk dylib script
2023-08-22 13:10:26 -04:00
0e443dcb05 fix: Added padding to buttons
Some buttons did not have enough padding, now they do!
2023-08-22 10:01:12 -07:00
ef61d129d3 Merge pull request #11303 from lat9nq/screenshots-configurable
yuzu-qt: Add configuration for screenshot resolution
2023-08-22 11:30:25 -04:00
b8bab551a4 Merge pull request #11316 from FernandoS27/stop-premature-christmas-decorating
Shader Recompiler: implement textureGrad 3D
2023-08-22 11:30:08 -04:00
a9f223cd9f Merge pull request #11346 from t895/ktlint-fix
android: lint: Delete generated ktlint folder between builds
2023-08-22 11:30:01 -04:00
87022a4833 Add macos moltenvk bundle, Add copy moltevk dylib script 2023-08-22 10:22:28 +08:00
1bc832c9b1 android: lint: Delete generated ktlint folder between builds
There's a bug in ktlint where it will run into an error if you build the project, delete a source file, and then build again. It will be unable to find the file you deleted and can't recover until these files are deleted. This just deletes those files before every run.
2023-08-21 17:31:13 -04:00
df00da1760 android: Show associated value in home settings (#11272) 2023-08-21 22:25:11 +02:00
9d6ac28999 Merge pull request #11309 from liamwhite/full-xci
file_sys/card_image: support dumps with prepended key area
2023-08-21 16:09:22 -04:00
a921851ba6 Merge pull request #11342 from liamwhite/skyline-4
patch_manager: apply manual HTML patches when present
2023-08-21 16:09:15 -04:00
18c08cee43 Merge pull request #11149 from ameerj/astc-perf-prod
host_shaders: ASTC compute shader optimizations
2023-08-21 16:08:51 -04:00
062113374d android: Use sensor landscape for landscape mode (#11337) 2023-08-21 21:56:12 +02:00
133ff3989b patch_manager: apply manual HTML patches when present 2023-08-21 10:58:23 -04:00
861597eb2e Merge pull request #11284 from liamwhite/nca-release
vfs: expand support for NCA reading
2023-08-21 16:29:04 +02:00
0cd9d51e06 sockets: avoid locking around socket session calls 2023-08-19 23:09:35 -04:00
6a5db5679b Merge pull request #11320 from Kelebek1/mask_depthstencil_clear
Support masked depthstencil clears
2023-08-19 17:28:28 +02:00
f2f99a8c31 Masked depthstencil clears 2023-08-19 03:29:46 +01:00
c03f0b3c89 Shader Recomnpiler: implement textuzreGrad 3D emulation constant propagation 2023-08-18 22:17:02 -04:00
ae1421265a Merge pull request #11278 from Kelebek1/dma_sync
Mark accelerated DMA destination buffers and images as GPU-modified
2023-08-18 09:12:27 -04:00
314d3858a1 Merge pull request #11288 from liamwhite/svc-tick
kernel: remove relative task registration
2023-08-18 09:12:19 -04:00
0383ae1dbf Merge pull request #11310 from vonchenplus/vulkan_format
video_core: Fix vulkan format assert error
2023-08-18 09:12:11 -04:00
1dcb0c2232 video_core: Fix vulkan assert error 2023-08-18 14:40:11 +08:00
8be3a041e0 file_sys/card_image: support dumps with prepended key area 2023-08-17 22:03:47 -04:00
ddedaa8875 Merge pull request #10989 from comex/epipe
sockets: Improve behavior when sending to closed connection
2023-08-17 11:59:47 -04:00
0e3a995bf4 cmake: mark warning disable for gcc 11 (#11301) 2023-08-17 16:03:34 +02:00
6af8cca2c1 uisettings: Add TODO for stretched aspect being ignored 2023-08-16 22:57:19 -04:00
775bf8e215 file_sys: tolerate empty NCA 2023-08-16 16:30:41 -04:00
e28b936950 configure_ui: Silence MSVC warning 2023-08-16 16:28:44 -04:00
6fe51b48e9 yuzu-qt: Screenshots depend more on the graphics settings 2023-08-16 16:12:42 -04:00
96c98d09cb yuzu-qt: Implement unspecified screenshot ratio 2023-08-16 00:18:47 -04:00
76a03e99b6 bootmanager: Remove old path
Causes issues with different selected aspect ratios in graphics.
2023-08-16 00:18:16 -04:00
95409c6859 configure_ui: Update the screenshots data 2023-08-15 23:08:02 -04:00
227950ac99 config: Read the entire screenshots category 2023-08-15 23:07:49 -04:00
bc5ec10498 bootmanager: Consider the default resolution 2023-08-15 22:57:38 -04:00
d9275b7757 yuzu-qt: Enable specifying screenshot resolution 2023-08-15 22:42:28 -04:00
3e28e85468 settings: Add AspectRatio enum, split res scale function 2023-08-15 22:41:50 -04:00
755bcc459b Improve behavior when sending to closed connection
- On Unix, this would previously kill the Yuzu process with SIGPIPE.
  Send MSG_NOSIGNAL to opt out of this.

- Add support for the proper error code in this situation, EPIPE.

- Windows has nonstandard behavior in this situation; translate it to
  the standard behavior.  Kind of pointless, but isn't it nice to be
  correct?
2023-08-15 20:59:57 -04:00
50eee9b218 fssystem: rework for yuzu style 2023-08-15 17:47:40 -04:00
0398b34370 fssystem: reduce overalignment of unbuffered storage operations 2023-08-15 17:47:25 -04:00
86f6b6b7b2 vfs: expand support for NCA reading 2023-08-15 17:47:25 -04:00
a8c4f01f6c Merge pull request #11287 from liamwhite/replaced-bytes
gdbstub: fixup replaced instruction bytes in memory reads
2023-08-15 15:36:14 +02:00
6d665a94ea Merge pull request #11256 from FearlessTobi/revert-10075
Partially Revert "Silence nifm spam"
2023-08-14 16:28:13 -07:00
bbc6b08fc7 Merge pull request #11273 from t895/setup-completion
android: Setup additions
2023-08-14 15:41:35 -07:00
0bd9a4456c kernel: remove relative task registration 2023-08-14 18:12:06 -04:00
fbda084acb gdbstub: fixup replaced instruction bytes in memory reads 2023-08-14 16:33:27 -04:00
2694f81462 Revert "Silence nifm spam"
This reverts commit 4da4ecb1ff.
2023-08-14 21:23:09 +02:00
d5adaeafdf Merge pull request #11271 from t895/settings-tweaks
android: Settings tweaks
2023-08-14 11:44:38 -07:00
58a4c86797 Merge pull request #11282 from ameerj/glasm-xfb
gl_graphics_pipeline: GLASM: Fix transform feedback with multiple buffers
2023-08-14 09:19:20 -04:00
35a77c3bb2 Merge pull request #11283 from ameerj/glasm-pipeline-detection
gl_graphics_pipeline: Fix GLASM storage buffer detection
2023-08-14 09:19:10 -04:00
c1016b68ae Merge pull request #11281 from liamwhite/vi-scale-mode
nvnflinger: add missing scale mode
2023-08-14 09:19:03 -04:00
b30df50076 Merge pull request #11259 from german77/hid
service: hid: Implement functions needed by QLaunch
2023-08-14 09:18:55 -04:00
5afe1367ba Merge pull request #11263 from liamwhite/my-feature-branch
vulkan_device: disable features associated with unloaded extensions
2023-08-14 09:18:47 -04:00
24700af3c2 Merge pull request #11264 from liamwhite/stray-code
ssl_backend_securetransport: remove stray .Code()
2023-08-14 09:18:32 -04:00
f9ef721ca6 gl_graphics_pipeline: Fix GLASM storage buffer detection 2023-08-13 17:06:45 -04:00
c34ed4bbd8 gl_graphics_pipeline: GLASM: Fix transform feedback with multiple buffers 2023-08-13 16:50:01 -04:00
7351884588 nvnflinger: add missing scale mode 2023-08-13 13:57:02 -04:00
5a37b8f2c1 Mark accelerted DMA destination buffers and images as GPU-modified 2023-08-13 02:22:39 +01:00
242ce2a0b3 android: Page forward on setup step completion 2023-08-12 20:21:47 -04:00
8ab3685a39 android: Adjust setup fragment layout
Fixes padding issues in small and large layouts and allows viewpager to reach into system insets.
2023-08-12 17:02:59 -04:00
8bd0521b58 android: Show complete indicator during setup 2023-08-12 16:53:14 -04:00
64ea5522d3 android: Remove redundant option from slider dialog
You can already reset any setting by long pressing the settings item.
2023-08-12 15:45:27 -04:00
798a439eb1 android: Reduce opacity of non-editable settings 2023-08-12 15:42:55 -04:00
786b609151 android: Use string resource for slider value/units 2023-08-12 15:42:54 -04:00
89a2d308c3 android: Display setting value in setting list items 2023-08-12 14:38:46 -04:00
0d4bf53ad9 android: Set switch listener before assigning new value
Previously the switch could have its old listener triggered when recycled.
2023-08-12 01:00:42 -04:00
8b98c4e5a0 ssl_backend_securetransport: remove stray .Code() 2023-08-11 23:32:46 -04:00
26ff214719 Merge pull request #11219 from zeltermann/title-id-search
Allow searching by a substring of the title ID
2023-08-11 16:53:27 -04:00
640f7cd945 Merge pull request #11253 from liamwhite/i-hate-this-toolchain
general: fix apple clang build
2023-08-11 16:53:20 -04:00
7d8f748696 vulkan_device: disable features associated with unloaded extensions 2023-08-11 14:54:12 -04:00
bdd96118d1 service: hid: Implement functions needed by QLaunch 2023-08-11 10:13:21 -06:00
1ed9e8812b Allow searching by a substring of the title ID 2023-08-11 00:07:12 +07:00
023b9b38cc general: fix apple clang build 2023-08-09 22:38:37 -04:00
5c25712af9 flatten color_values 2023-08-09 18:45:52 -04:00
0f7220c9c8 flatten encoding_values 2023-08-09 18:38:37 -04:00
71857e889e flatten result vector 2023-08-09 18:34:57 -04:00
70f8ffb787 GetUnquantizedWeightVector 2023-08-09 17:45:39 -04:00
9058486b9b Revert "HACK: Avoid swizzling and reuploading ASTC image every frame"
This reverts commit b18c1fb1bb.
2023-08-06 14:55:05 -04:00
b18c1fb1bb HACK: Avoid swizzling and reuploading ASTC image every frame 2023-08-06 14:54:58 -04:00
913803bf65 Compute Replicate 2023-08-06 14:54:58 -04:00
31a0cff036 minor 2023-08-06 14:54:58 -04:00
b36e645fee undo uint 2023-08-06 14:54:58 -04:00
8ce158bce6 Revert "vulkan dims specialization"
This reverts commit e6243058f2269bd79ac8479d58e55feec2611e9d.
2023-08-06 14:54:58 -04:00
5a78b35b1a vulkan dims specialization 2023-08-06 14:54:58 -04:00
7a0d7e7668 small_block opt 2023-08-06 14:54:58 -04:00
fd2051b401 remove TexelWeightParams 2023-08-06 14:54:57 -04:00
75ac7845ce error/void extent funcs 2023-08-06 14:54:57 -04:00
441b847107 more packing 2023-08-06 14:54:57 -04:00
f2cf81e0b6 Revert "uint result index"
This reverts commit 0e978786b5a8e7382005d8b1e16cfa12f3eeb775.
2023-08-06 14:54:57 -04:00
f41fb3ec0b Revert "bfe instead of mod"
This reverts commit 86006a3b09e8a8c17d2ade61be76736a79e3f58a.
2023-08-06 14:54:57 -04:00
553dd3e120 Revert "global endpoints"
This reverts commit d8f5bfd1df2b7469ef6abcee182aa110602d1751.
2023-08-06 14:54:57 -04:00
c077e467c4 global endpoints 2023-08-06 14:54:57 -04:00
5c16559694 bfe instead of mod 2023-08-06 14:54:57 -04:00
6b0b584eba uint result index 2023-08-06 14:54:57 -04:00
05ee37a1f0 amd opts 2023-08-06 14:54:57 -04:00
3494fce864 gl 2023-08-06 14:54:57 -04:00
5248fa926d const, pack result_vector and replicate tables,
undo amd opts
2023-08-06 14:54:57 -04:00
998246efc2 minor redundancy cleanup 2023-08-06 14:54:57 -04:00
d17a51bc59 extractbits robustness 2023-08-06 14:54:57 -04:00
0078e5a338 reuse vectors memory 2023-08-06 14:54:57 -04:00
b8ca47e094 EncodingData pack 2023-08-06 14:54:57 -04:00
27c8bb9615 flattening 2023-08-06 14:54:57 -04:00
ac09cc3504 weights refactor 2023-08-06 14:54:57 -04:00
6ff65abd62 params.max_weight 2023-08-06 14:54:57 -04:00
e0c59c7b0b skip bits 2023-08-06 14:54:57 -04:00
7ef879b296 restrict 2023-08-06 14:54:57 -04:00
226 changed files with 10847 additions and 2343 deletions

View File

@ -49,7 +49,7 @@ option(YUZU_TESTS "Compile tests" "${BUILD_TESTING}")
option(YUZU_USE_PRECOMPILED_HEADERS "Use precompiled headers" ON)
cmake_dependent_option(YUZU_ROOM "Compile LDN room server" ON "NOT ANDROID" OFF)
CMAKE_DEPENDENT_OPTION(YUZU_ROOM "Compile LDN room server" ON "NOT ANDROID" OFF)
CMAKE_DEPENDENT_OPTION(YUZU_CRASH_DUMPS "Compile Windows crash dump (Minidump) support" OFF "WIN32" OFF)
@ -63,6 +63,8 @@ option(YUZU_DOWNLOAD_TIME_ZONE_DATA "Always download time zone binaries" OFF)
CMAKE_DEPENDENT_OPTION(YUZU_USE_FASTER_LD "Check if a faster linker is available" ON "NOT WIN32" OFF)
CMAKE_DEPENDENT_OPTION(USE_SYSTEM_MOLTENVK "Use the system MoltenVK lib (instead of the bundled one)" OFF "APPLE" OFF)
set(DEFAULT_ENABLE_OPENSSL ON)
if (ANDROID OR WIN32 OR APPLE)
# - Windows defaults to the Schannel backend.

View File

@ -36,3 +36,21 @@ endif()
message(STATUS "Using bundled binaries at ${prefix}")
set(${prefix_var} "${prefix}" PARENT_SCOPE)
endfunction()
function(download_moltenvk_external platform version)
set(MOLTENVK_DIR "${CMAKE_BINARY_DIR}/externals/MoltenVK")
set(MOLTENVK_TAR "${CMAKE_BINARY_DIR}/externals/MoltenVK.tar")
if (NOT EXISTS ${MOLTENVK_DIR})
if (NOT EXISTS ${MOLTENVK_TAR})
file(DOWNLOAD https://github.com/KhronosGroup/MoltenVK/releases/download/${version}/MoltenVK-${platform}.tar
${MOLTENVK_TAR} SHOW_PROGRESS)
endif()
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xf "${MOLTENVK_TAR}"
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/externals")
endif()
# Add the MoltenVK library path to the prefix so find_library can locate it.
list(APPEND CMAKE_PREFIX_PATH "${MOLTENVK_DIR}/MoltenVK/dylib/${platform}")
set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH} PARENT_SCOPE)
endfunction()

View File

@ -1,3 +1,11 @@
| Pull Request | Commit | Title | Author | Merged? |
|----|----|----|----|----|
End of merge log. You can find the original README.md below the break.
-----
<!--
SPDX-FileCopyrightText: 2018 yuzu Emulator Project
SPDX-License-Identifier: GPL-2.0-or-later

View File

@ -78,6 +78,11 @@ QPushButton#buttonRefreshDevices {
max-height: 21px;
}
QPushButton#button_reset_defaults {
min-width: 57px;
padding: 4px 8px;
}
QWidget#bottomPerGameInput,
QWidget#topControllerApplet,
QWidget#bottomControllerApplet,

View File

@ -2228,6 +2228,10 @@ QPushButton#buttonRefreshDevices {
padding: 0px 0px;
}
QPushButton#button_reset_defaults {
padding: 3px 6px;
}
QSpinBox#spinboxLStickRange,
QSpinBox#spinboxRStickRange,
QSpinBox#vibrationSpinPlayer1,

View File

@ -42,6 +42,11 @@ endif()
# mbedtls
add_subdirectory(mbedtls)
target_include_directories(mbedtls PUBLIC ./mbedtls/include)
if (NOT MSVC)
target_compile_options(mbedcrypto PRIVATE
-Wno-unused-but-set-variable
-Wno-string-concatenation)
endif()
# MicroProfile
add_library(microprofile INTERFACE)
@ -94,6 +99,12 @@ if (ENABLE_CUBEB AND NOT TARGET cubeb::cubeb)
set(BUILD_TOOLS OFF)
add_subdirectory(cubeb)
add_library(cubeb::cubeb ALIAS cubeb)
if (NOT MSVC)
if (TARGET speex)
target_compile_options(speex PRIVATE -Wno-sign-compare)
endif()
target_compile_options(cubeb PRIVATE -Wno-implicit-const-int-float-conversion)
endif()
endif()
# DiscordRPC
@ -151,6 +162,9 @@ endif()
if (NOT TARGET LLVM::Demangle)
add_library(demangle demangle/ItaniumDemangle.cpp)
target_include_directories(demangle PUBLIC ./demangle)
if (NOT MSVC)
target_compile_options(demangle PRIVATE -Wno-deprecated-declarations) # std::is_pod
endif()
add_library(LLVM::Demangle ALIAS demangle)
endif()

View File

@ -114,16 +114,19 @@ else()
-Wno-attributes
-Wno-invalid-offsetof
-Wno-unused-parameter
$<$<CXX_COMPILER_ID:Clang>:-Wno-braced-scalar-init>
$<$<CXX_COMPILER_ID:Clang>:-Wno-unused-private-field>
$<$<CXX_COMPILER_ID:Clang>:-Werror=shadow-uncaptured-local>
$<$<CXX_COMPILER_ID:Clang>:-Werror=implicit-fallthrough>
$<$<CXX_COMPILER_ID:Clang>:-Werror=type-limits>
$<$<CXX_COMPILER_ID:AppleClang>:-Wno-braced-scalar-init>
$<$<CXX_COMPILER_ID:AppleClang>:-Wno-unused-private-field>
)
if (CMAKE_CXX_COMPILER_ID MATCHES Clang) # Clang or AppleClang
add_compile_options(
-Wno-braced-scalar-init
-Wno-unused-private-field
-Wno-nullability-completeness
-Werror=shadow-uncaptured-local
-Werror=implicit-fallthrough
-Werror=type-limits
)
endif()
if (ARCHITECTURE_x86_64)
add_compile_options("-mcx16")
add_compile_options("-fwrapv")
@ -134,7 +137,7 @@ else()
endif()
# GCC bugs
if (CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL "12" AND CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
if (CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL "11" AND CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
# These diagnostics would be great if they worked, but are just completely broken
# and produce bogus errors on external libraries like fmt.
add_compile_options(

View File

@ -95,6 +95,7 @@ android {
// builds a release build that doesn't need signing
// Attaches 'debug' suffix to version and package name, allowing installation alongside the release build.
register("relWithDebInfo") {
isDefault = true
resValue("string", "app_name_suffixed", "yuzu Debug Release")
signingConfig = signingConfigs.getByName("debug")
isMinifyEnabled = true
@ -122,6 +123,7 @@ android {
flavorDimensions.add("version")
productFlavors {
create("mainline") {
isDefault = true
dimension = "version"
buildConfigField("Boolean", "PREMIUM", "false")
}
@ -160,6 +162,11 @@ android {
}
}
tasks.create<Delete>("ktlintReset") {
delete(File(buildDir.path + File.separator + "intermediates/ktLint"))
}
tasks.getByPath("loadKtlintReporters").dependsOn("ktlintReset")
tasks.getByPath("preBuild").dependsOn("ktlintCheck")
ktlint {

View File

@ -25,6 +25,7 @@ SPDX-License-Identifier: GPL-3.0-or-later
android:hasFragileUserData="false"
android:supportsRtl="true"
android:isGame="true"
android:appCategory="game"
android:localeConfig="@xml/locales_config"
android:banner="@drawable/tv_banner"
android:extractNativeLibs="true"

View File

@ -3,19 +3,25 @@
package org.yuzu.yuzu_emu.adapters
import android.text.TextUtils
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import androidx.appcompat.app.AppCompatActivity
import androidx.core.content.ContextCompat
import androidx.core.content.res.ResourcesCompat
import androidx.lifecycle.LifecycleOwner
import androidx.recyclerview.widget.RecyclerView
import org.yuzu.yuzu_emu.R
import org.yuzu.yuzu_emu.databinding.CardHomeOptionBinding
import org.yuzu.yuzu_emu.fragments.MessageDialogFragment
import org.yuzu.yuzu_emu.model.HomeSetting
class HomeSettingAdapter(private val activity: AppCompatActivity, var options: List<HomeSetting>) :
class HomeSettingAdapter(
private val activity: AppCompatActivity,
private val viewLifecycle: LifecycleOwner,
var options: List<HomeSetting>
) :
RecyclerView.Adapter<HomeSettingAdapter.HomeOptionViewHolder>(),
View.OnClickListener {
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): HomeOptionViewHolder {
@ -79,6 +85,22 @@ class HomeSettingAdapter(private val activity: AppCompatActivity, var options: L
binding.optionDescription.alpha = 0.5f
binding.optionIcon.alpha = 0.5f
}
option.details.observe(viewLifecycle) { updateOptionDetails(it) }
binding.optionDetail.postDelayed(
{
binding.optionDetail.ellipsize = TextUtils.TruncateAt.MARQUEE
binding.optionDetail.isSelected = true
},
3000
)
}
private fun updateOptionDetails(detailString: String) {
if (detailString.isNotEmpty()) {
binding.optionDetail.text = detailString
binding.optionDetail.visibility = View.VISIBLE
}
}
}
}

View File

@ -49,6 +49,7 @@ class LicenseAdapter(private val activity: AppCompatActivity, var licenses: List
val context = YuzuApplication.appContext
binding.textSettingName.text = context.getString(license.titleId)
binding.textSettingDescription.text = context.getString(license.descriptionId)
binding.textSettingValue.visibility = View.GONE
}
}
}

View File

@ -5,13 +5,19 @@ package org.yuzu.yuzu_emu.adapters
import android.text.Html
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import androidx.appcompat.app.AppCompatActivity
import androidx.core.content.res.ResourcesCompat
import androidx.lifecycle.ViewModelProvider
import androidx.recyclerview.widget.RecyclerView
import com.google.android.material.button.MaterialButton
import org.yuzu.yuzu_emu.databinding.PageSetupBinding
import org.yuzu.yuzu_emu.model.HomeViewModel
import org.yuzu.yuzu_emu.model.SetupCallback
import org.yuzu.yuzu_emu.model.SetupPage
import org.yuzu.yuzu_emu.model.StepState
import org.yuzu.yuzu_emu.utils.ViewUtils
class SetupAdapter(val activity: AppCompatActivity, val pages: List<SetupPage>) :
RecyclerView.Adapter<SetupAdapter.SetupPageViewHolder>() {
@ -26,7 +32,7 @@ class SetupAdapter(val activity: AppCompatActivity, val pages: List<SetupPage>)
holder.bind(pages[position])
inner class SetupPageViewHolder(val binding: PageSetupBinding) :
RecyclerView.ViewHolder(binding.root) {
RecyclerView.ViewHolder(binding.root), SetupCallback {
lateinit var page: SetupPage
init {
@ -35,6 +41,12 @@ class SetupAdapter(val activity: AppCompatActivity, val pages: List<SetupPage>)
fun bind(page: SetupPage) {
this.page = page
if (page.stepCompleted.invoke() == StepState.COMPLETE) {
binding.buttonAction.visibility = View.INVISIBLE
binding.textConfirmation.visibility = View.VISIBLE
}
binding.icon.setImageDrawable(
ResourcesCompat.getDrawable(
activity.resources,
@ -62,9 +74,15 @@ class SetupAdapter(val activity: AppCompatActivity, val pages: List<SetupPage>)
MaterialButton.ICON_GRAVITY_END
}
setOnClickListener {
page.buttonAction.invoke()
page.buttonAction.invoke(this@SetupPageViewHolder)
}
}
}
override fun onStepCompleted() {
ViewUtils.hideView(binding.buttonAction, 200)
ViewUtils.showView(binding.textConfirmation, 200)
ViewModelProvider(activity)[HomeViewModel::class.java].setShouldPageForward(true)
}
}
}

View File

@ -207,8 +207,11 @@ class SettingsAdapter(
val sliderBinding = DialogSliderBinding.inflate(inflater)
textSliderValue = sliderBinding.textValue
textSliderValue!!.text = sliderProgress.toString()
sliderBinding.textUnits.text = item.units
textSliderValue!!.text = String.format(
context.getString(R.string.value_with_units),
sliderProgress.toString(),
item.units
)
sliderBinding.slider.apply {
valueFrom = item.min.toFloat()
@ -216,7 +219,11 @@ class SettingsAdapter(
value = sliderProgress.toFloat()
addOnChangeListener { _: Slider, value: Float, _: Boolean ->
sliderProgress = value.toInt()
textSliderValue!!.text = sliderProgress.toString()
textSliderValue!!.text = String.format(
context.getString(R.string.value_with_units),
sliderProgress.toString(),
item.units
)
}
}
@ -225,10 +232,6 @@ class SettingsAdapter(
.setView(sliderBinding.root)
.setPositiveButton(android.R.string.ok, this)
.setNegativeButton(android.R.string.cancel, defaultCancelListener)
.setNeutralButton(R.string.slider_default) { dialog: DialogInterface, which: Int ->
sliderBinding.slider.value = item.defaultValue!!.toFloat()
onClick(dialog, which)
}
.show()
}

View File

@ -25,12 +25,17 @@ class DateTimeViewHolder(val binding: ListItemSettingBinding, adapter: SettingsA
binding.textSettingDescription.setText(item.descriptionId)
binding.textSettingDescription.visibility = View.VISIBLE
} else {
binding.textSettingDescription.visibility = View.GONE
}
binding.textSettingValue.visibility = View.VISIBLE
val epochTime = setting.value.toLong()
val instant = Instant.ofEpochMilli(epochTime * 1000)
val zonedTime = ZonedDateTime.ofInstant(instant, ZoneId.of("UTC"))
val dateFormatter = DateTimeFormatter.ofLocalizedDateTime(FormatStyle.MEDIUM)
binding.textSettingDescription.text = dateFormatter.format(zonedTime)
}
binding.textSettingValue.text = dateFormatter.format(zonedTime)
setStyle(setting.isEditable, binding)
}
override fun onClick(clicked: View) {

View File

@ -23,6 +23,9 @@ class RunnableViewHolder(val binding: ListItemSettingBinding, adapter: SettingsA
} else {
binding.textSettingDescription.visibility = View.GONE
}
binding.textSettingValue.visibility = View.GONE
setStyle(setting.isEditable, binding)
}
override fun onClick(clicked: View) {

View File

@ -5,6 +5,8 @@ package org.yuzu.yuzu_emu.features.settings.ui.viewholder
import android.view.View
import androidx.recyclerview.widget.RecyclerView
import org.yuzu.yuzu_emu.databinding.ListItemSettingBinding
import org.yuzu.yuzu_emu.databinding.ListItemSettingSwitchBinding
import org.yuzu.yuzu_emu.features.settings.model.view.SettingsItem
import org.yuzu.yuzu_emu.features.settings.ui.SettingsAdapter
@ -33,4 +35,18 @@ abstract class SettingViewHolder(itemView: View, protected val adapter: Settings
abstract override fun onClick(clicked: View)
abstract override fun onLongClick(clicked: View): Boolean
fun setStyle(isEditable: Boolean, binding: ListItemSettingBinding) {
val opacity = if (isEditable) 1.0f else 0.5f
binding.textSettingName.alpha = opacity
binding.textSettingDescription.alpha = opacity
binding.textSettingValue.alpha = opacity
}
fun setStyle(isEditable: Boolean, binding: ListItemSettingSwitchBinding) {
binding.switchWidget.isEnabled = isEditable
val opacity = if (isEditable) 1.0f else 0.5f
binding.textSettingName.alpha = opacity
binding.textSettingDescription.alpha = opacity
}
}

View File

@ -17,28 +17,33 @@ class SingleChoiceViewHolder(val binding: ListItemSettingBinding, adapter: Setti
override fun bind(item: SettingsItem) {
setting = item
binding.textSettingName.setText(item.nameId)
binding.textSettingDescription.visibility = View.VISIBLE
if (item.descriptionId != 0) {
binding.textSettingDescription.setText(item.descriptionId)
} else if (item is SingleChoiceSetting) {
val resMgr = binding.textSettingDescription.context.resources
binding.textSettingDescription.visibility = View.VISIBLE
} else {
binding.textSettingDescription.visibility = View.GONE
}
binding.textSettingValue.visibility = View.VISIBLE
if (item is SingleChoiceSetting) {
val resMgr = binding.textSettingValue.context.resources
val values = resMgr.getIntArray(item.valuesId)
for (i in values.indices) {
if (values[i] == item.selectedValue) {
binding.textSettingDescription.text = resMgr.getStringArray(item.choicesId)[i]
return
binding.textSettingValue.text = resMgr.getStringArray(item.choicesId)[i]
break
}
}
} else if (item is StringSingleChoiceSetting) {
for (i in item.values!!.indices) {
if (item.values[i] == item.selectedValue) {
binding.textSettingDescription.text = item.choices[i]
return
binding.textSettingValue.text = item.choices[i]
break
}
}
} else {
binding.textSettingDescription.visibility = View.GONE
}
setStyle(setting.isEditable, binding)
}
override fun onClick(clicked: View) {

View File

@ -4,6 +4,7 @@
package org.yuzu.yuzu_emu.features.settings.ui.viewholder
import android.view.View
import org.yuzu.yuzu_emu.R
import org.yuzu.yuzu_emu.databinding.ListItemSettingBinding
import org.yuzu.yuzu_emu.features.settings.model.view.SettingsItem
import org.yuzu.yuzu_emu.features.settings.model.view.SliderSetting
@ -22,6 +23,14 @@ class SliderViewHolder(val binding: ListItemSettingBinding, adapter: SettingsAda
} else {
binding.textSettingDescription.visibility = View.GONE
}
binding.textSettingValue.visibility = View.VISIBLE
binding.textSettingValue.text = String.format(
binding.textSettingValue.context.getString(R.string.value_with_units),
setting.selectedValue,
setting.units
)
setStyle(setting.isEditable, binding)
}
override fun onClick(clicked: View) {

View File

@ -22,6 +22,7 @@ class SubmenuViewHolder(val binding: ListItemSettingBinding, adapter: SettingsAd
} else {
binding.textSettingDescription.visibility = View.GONE
}
binding.textSettingValue.visibility = View.GONE
}
override fun onClick(clicked: View) {

View File

@ -25,12 +25,12 @@ class SwitchSettingViewHolder(val binding: ListItemSettingSwitchBinding, adapter
binding.textSettingDescription.text = ""
binding.textSettingDescription.visibility = View.GONE
}
binding.switchWidget.isChecked = setting.isChecked
binding.switchWidget.setOnCheckedChangeListener { _: CompoundButton, _: Boolean ->
adapter.onBooleanClick(item, bindingAdapterPosition, binding.switchWidget.isChecked)
}
binding.switchWidget.isChecked = setting.isChecked
binding.switchWidget.isEnabled = setting.isEditable
setStyle(setting.isEditable, binding)
}
override fun onClick(clicked: View) {

View File

@ -297,11 +297,11 @@ class EmulationFragment : Fragment(), SurfaceHolder.Callback {
emulationActivity?.let {
it.requestedOrientation = when (IntSetting.RENDERER_SCREEN_LAYOUT.int) {
Settings.LayoutOption_MobileLandscape ->
ActivityInfo.SCREEN_ORIENTATION_USER_LANDSCAPE
ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
Settings.LayoutOption_MobilePortrait ->
ActivityInfo.SCREEN_ORIENTATION_USER_PORTRAIT
Settings.LayoutOption_Unspecified -> ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
else -> ActivityInfo.SCREEN_ORIENTATION_USER_LANDSCAPE
else -> ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
}
}
}

View File

@ -129,7 +129,11 @@ class HomeSettingsFragment : Fragment() {
mainActivity.getGamesDirectory.launch(
Intent(Intent.ACTION_OPEN_DOCUMENT_TREE).data
)
}
},
{ true },
0,
0,
homeViewModel.gamesDir
)
)
add(
@ -201,7 +205,11 @@ class HomeSettingsFragment : Fragment() {
binding.homeSettingsList.apply {
layoutManager = LinearLayoutManager(requireContext())
adapter = HomeSettingAdapter(requireActivity() as AppCompatActivity, optionsList)
adapter = HomeSettingAdapter(
requireActivity() as AppCompatActivity,
viewLifecycleOwner,
optionsList
)
}
setInsets()

View File

@ -19,6 +19,7 @@ import androidx.core.content.ContextCompat
import androidx.core.view.ViewCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.isVisible
import androidx.core.view.updatePadding
import androidx.fragment.app.Fragment
import androidx.fragment.app.activityViewModels
import androidx.navigation.findNavController
@ -32,10 +33,13 @@ import org.yuzu.yuzu_emu.adapters.SetupAdapter
import org.yuzu.yuzu_emu.databinding.FragmentSetupBinding
import org.yuzu.yuzu_emu.features.settings.model.Settings
import org.yuzu.yuzu_emu.model.HomeViewModel
import org.yuzu.yuzu_emu.model.SetupCallback
import org.yuzu.yuzu_emu.model.SetupPage
import org.yuzu.yuzu_emu.model.StepState
import org.yuzu.yuzu_emu.ui.main.MainActivity
import org.yuzu.yuzu_emu.utils.DirectoryInitialization
import org.yuzu.yuzu_emu.utils.GameHelper
import org.yuzu.yuzu_emu.utils.ViewUtils
class SetupFragment : Fragment() {
private var _binding: FragmentSetupBinding? = null
@ -112,14 +116,22 @@ class SetupFragment : Fragment() {
0,
false,
R.string.give_permission,
{ permissionLauncher.launch(Manifest.permission.POST_NOTIFICATIONS) },
{
notificationCallback = it
permissionLauncher.launch(Manifest.permission.POST_NOTIFICATIONS)
},
true,
R.string.notification_warning,
R.string.notification_warning_description,
0,
{
NotificationManagerCompat.from(requireContext())
if (NotificationManagerCompat.from(requireContext())
.areNotificationsEnabled()
) {
StepState.COMPLETE
} else {
StepState.INCOMPLETE
}
}
)
)
@ -133,12 +145,22 @@ class SetupFragment : Fragment() {
R.drawable.ic_add,
true,
R.string.select_keys,
{ mainActivity.getProdKey.launch(arrayOf("*/*")) },
{
keyCallback = it
getProdKey.launch(arrayOf("*/*"))
},
true,
R.string.install_prod_keys_warning,
R.string.install_prod_keys_warning_description,
R.string.install_prod_keys_warning_help,
{ File(DirectoryInitialization.userDirectory + "/keys/prod.keys").exists() }
{
val file = File(DirectoryInitialization.userDirectory + "/keys/prod.keys")
if (file.exists()) {
StepState.COMPLETE
} else {
StepState.INCOMPLETE
}
}
)
)
add(
@ -150,9 +172,8 @@ class SetupFragment : Fragment() {
true,
R.string.add_games,
{
mainActivity.getGamesDirectory.launch(
Intent(Intent.ACTION_OPEN_DOCUMENT_TREE).data
)
gamesDirCallback = it
getGamesDirectory.launch(Intent(Intent.ACTION_OPEN_DOCUMENT_TREE).data)
},
true,
R.string.add_games_warning,
@ -163,7 +184,11 @@ class SetupFragment : Fragment() {
PreferenceManager.getDefaultSharedPreferences(
YuzuApplication.appContext
)
preferences.getString(GameHelper.KEY_GAME_PATH, "")!!.isNotEmpty()
if (preferences.getString(GameHelper.KEY_GAME_PATH, "")!!.isNotEmpty()) {
StepState.COMPLETE
} else {
StepState.INCOMPLETE
}
}
)
)
@ -181,6 +206,13 @@ class SetupFragment : Fragment() {
)
}
homeViewModel.shouldPageForward.observe(viewLifecycleOwner) {
if (it) {
pageForward()
homeViewModel.setShouldPageForward(false)
}
}
binding.viewPager2.apply {
adapter = SetupAdapter(requireActivity() as AppCompatActivity, pages)
offscreenPageLimit = 2
@ -194,15 +226,15 @@ class SetupFragment : Fragment() {
super.onPageSelected(position)
if (position == 1 && previousPosition == 0) {
showView(binding.buttonNext)
showView(binding.buttonBack)
ViewUtils.showView(binding.buttonNext)
ViewUtils.showView(binding.buttonBack)
} else if (position == 0 && previousPosition == 1) {
hideView(binding.buttonBack)
hideView(binding.buttonNext)
ViewUtils.hideView(binding.buttonBack)
ViewUtils.hideView(binding.buttonNext)
} else if (position == pages.size - 1 && previousPosition == pages.size - 2) {
hideView(binding.buttonNext)
ViewUtils.hideView(binding.buttonNext)
} else if (position == pages.size - 2 && previousPosition == pages.size - 1) {
showView(binding.buttonNext)
ViewUtils.showView(binding.buttonNext)
}
previousPosition = position
@ -215,7 +247,8 @@ class SetupFragment : Fragment() {
// Checks if the user has completed the task on the current page
if (currentPage.hasWarning) {
if (currentPage.taskCompleted.invoke()) {
val stepState = currentPage.stepCompleted.invoke()
if (stepState != StepState.INCOMPLETE) {
pageForward()
return@setOnClickListener
}
@ -264,9 +297,15 @@ class SetupFragment : Fragment() {
_binding = null
}
private lateinit var notificationCallback: SetupCallback
@RequiresApi(Build.VERSION_CODES.TIRAMISU)
private val permissionLauncher =
registerForActivityResult(ActivityResultContracts.RequestPermission()) {
if (it) {
notificationCallback.onStepCompleted()
}
if (!it &&
!shouldShowRequestPermissionRationale(Manifest.permission.POST_NOTIFICATIONS)
) {
@ -277,6 +316,27 @@ class SetupFragment : Fragment() {
}
}
private lateinit var keyCallback: SetupCallback
val getProdKey =
registerForActivityResult(ActivityResultContracts.OpenDocument()) { result ->
if (result != null) {
if (mainActivity.processKey(result)) {
keyCallback.onStepCompleted()
}
}
}
private lateinit var gamesDirCallback: SetupCallback
val getGamesDirectory =
registerForActivityResult(ActivityResultContracts.OpenDocumentTree()) { result ->
if (result != null) {
mainActivity.processGamesDir(result)
gamesDirCallback.onStepCompleted()
}
}
private fun finishSetup() {
PreferenceManager.getDefaultSharedPreferences(YuzuApplication.appContext).edit()
.putBoolean(Settings.PREF_FIRST_APP_LAUNCH, false)
@ -284,33 +344,6 @@ class SetupFragment : Fragment() {
mainActivity.finishSetup(binding.root.findNavController())
}
private fun showView(view: View) {
view.apply {
alpha = 0f
visibility = View.VISIBLE
isClickable = true
}.animate().apply {
duration = 300
alpha(1f)
}.start()
}
private fun hideView(view: View) {
if (view.visibility == View.INVISIBLE) {
return
}
view.apply {
alpha = 1f
isClickable = false
}.animate().apply {
duration = 300
alpha(0f)
}.withEndAction {
view.visibility = View.INVISIBLE
}
}
fun pageForward() {
binding.viewPager2.currentItem = binding.viewPager2.currentItem + 1
}
@ -326,15 +359,29 @@ class SetupFragment : Fragment() {
private fun setInsets() =
ViewCompat.setOnApplyWindowInsetsListener(
binding.root
) { view: View, windowInsets: WindowInsetsCompat ->
) { _: View, windowInsets: WindowInsetsCompat ->
val barInsets = windowInsets.getInsets(WindowInsetsCompat.Type.systemBars())
val cutoutInsets = windowInsets.getInsets(WindowInsetsCompat.Type.displayCutout())
view.setPadding(
barInsets.left + cutoutInsets.left,
barInsets.top + cutoutInsets.top,
barInsets.right + cutoutInsets.right,
barInsets.bottom + cutoutInsets.bottom
val leftPadding = barInsets.left + cutoutInsets.left
val topPadding = barInsets.top + cutoutInsets.top
val rightPadding = barInsets.right + cutoutInsets.right
val bottomPadding = barInsets.bottom + cutoutInsets.bottom
if (resources.getBoolean(R.bool.small_layout)) {
binding.viewPager2
.updatePadding(left = leftPadding, top = topPadding, right = rightPadding)
binding.constraintButtons
.updatePadding(left = leftPadding, right = rightPadding, bottom = bottomPadding)
} else {
binding.viewPager2.updatePadding(top = topPadding, bottom = bottomPadding)
binding.constraintButtons
.updatePadding(
left = leftPadding,
right = rightPadding,
bottom = bottomPadding
)
}
windowInsets
}
}

View File

@ -3,6 +3,9 @@
package org.yuzu.yuzu_emu.model
import androidx.lifecycle.LiveData
import androidx.lifecycle.MutableLiveData
data class HomeSetting(
val titleId: Int,
val descriptionId: Int,
@ -10,5 +13,6 @@ data class HomeSetting(
val onClick: () -> Unit,
val isEnabled: () -> Boolean = { true },
val disabledTitleId: Int = 0,
val disabledMessageId: Int = 0
val disabledMessageId: Int = 0,
val details: LiveData<String> = MutableLiveData("")
)

View File

@ -3,9 +3,15 @@
package org.yuzu.yuzu_emu.model
import android.net.Uri
import androidx.fragment.app.FragmentActivity
import androidx.lifecycle.LiveData
import androidx.lifecycle.MutableLiveData
import androidx.lifecycle.ViewModel
import androidx.lifecycle.ViewModelProvider
import androidx.preference.PreferenceManager
import org.yuzu.yuzu_emu.YuzuApplication
import org.yuzu.yuzu_emu.utils.GameHelper
class HomeViewModel : ViewModel() {
private val _navigationVisible = MutableLiveData<Pair<Boolean, Boolean>>()
@ -14,6 +20,17 @@ class HomeViewModel : ViewModel() {
private val _statusBarShadeVisible = MutableLiveData(true)
val statusBarShadeVisible: LiveData<Boolean> get() = _statusBarShadeVisible
private val _shouldPageForward = MutableLiveData(false)
val shouldPageForward: LiveData<Boolean> get() = _shouldPageForward
private val _gamesDir = MutableLiveData(
Uri.parse(
PreferenceManager.getDefaultSharedPreferences(YuzuApplication.appContext)
.getString(GameHelper.KEY_GAME_PATH, "")
).path ?: ""
)
val gamesDir: LiveData<String> get() = _gamesDir
var navigatedToSetup = false
init {
@ -33,4 +50,13 @@ class HomeViewModel : ViewModel() {
}
_statusBarShadeVisible.value = visible
}
fun setShouldPageForward(pageForward: Boolean) {
_shouldPageForward.value = pageForward
}
fun setGamesDir(activity: FragmentActivity, dir: String) {
ViewModelProvider(activity)[GamesViewModel::class.java].reloadGames(true)
_gamesDir.value = dir
}
}

View File

@ -10,10 +10,20 @@ data class SetupPage(
val buttonIconId: Int,
val leftAlignedIcon: Boolean,
val buttonTextId: Int,
val buttonAction: () -> Unit,
val buttonAction: (callback: SetupCallback) -> Unit,
val hasWarning: Boolean,
val warningTitleId: Int = 0,
val warningDescriptionId: Int = 0,
val warningHelpLinkId: Int = 0,
val taskCompleted: () -> Boolean = { true }
val stepCompleted: () -> StepState = { StepState.UNDEFINED }
)
interface SetupCallback {
fun onStepCompleted()
}
enum class StepState {
COMPLETE,
INCOMPLETE,
UNDEFINED
}

View File

@ -266,10 +266,12 @@ class MainActivity : AppCompatActivity(), ThemeProvider {
val getGamesDirectory =
registerForActivityResult(ActivityResultContracts.OpenDocumentTree()) { result ->
if (result == null) {
return@registerForActivityResult
if (result != null) {
processGamesDir(result)
}
}
fun processGamesDir(result: Uri) {
contentResolver.takePersistableUriPermission(
result,
Intent.FLAG_GRANT_READ_URI_PERMISSION
@ -288,20 +290,23 @@ class MainActivity : AppCompatActivity(), ThemeProvider {
).show()
gamesViewModel.reloadGames(true)
homeViewModel.setGamesDir(this, result.path!!)
}
val getProdKey =
registerForActivityResult(ActivityResultContracts.OpenDocument()) { result ->
if (result == null) {
return@registerForActivityResult
if (result != null) {
processKey(result)
}
}
fun processKey(result: Uri): Boolean {
if (FileUtil.getExtension(result) != "keys") {
MessageDialogFragment.newInstance(
R.string.reading_keys_failure,
R.string.install_prod_keys_failure_extension_description
).show(supportFragmentManager, MessageDialogFragment.TAG)
return@registerForActivityResult
return false
}
contentResolver.takePersistableUriPermission(
@ -324,14 +329,17 @@ class MainActivity : AppCompatActivity(), ThemeProvider {
Toast.LENGTH_SHORT
).show()
gamesViewModel.reloadGames(true)
return true
} else {
MessageDialogFragment.newInstance(
R.string.invalid_keys_error,
R.string.install_keys_failure_description,
R.string.dumping_keys_quickstart_link
).show(supportFragmentManager, MessageDialogFragment.TAG)
return false
}
}
return false
}
val getFirmware =

View File

@ -11,6 +11,7 @@ import kotlinx.serialization.json.Json
import org.yuzu.yuzu_emu.NativeLibrary
import org.yuzu.yuzu_emu.YuzuApplication
import org.yuzu.yuzu_emu.model.Game
import org.yuzu.yuzu_emu.model.MinimalDocumentFile
object GameHelper {
const val KEY_GAME_PATH = "game_path"
@ -29,15 +30,7 @@ object GameHelper {
// Ensure keys are loaded so that ROM metadata can be decrypted.
NativeLibrary.reloadKeys()
val children = FileUtil.listFiles(context, gamesUri)
for (file in children) {
if (!file.isDirectory) {
// Check that the file has an extension we care about before trying to read out of it.
if (Game.extensions.contains(FileUtil.getExtension(file.uri))) {
games.add(getGame(file.uri))
}
}
}
addGamesRecursive(games, FileUtil.listFiles(context, gamesUri), 3)
// Cache list of games found on disk
val serializedGames = mutableSetOf<String>()
@ -52,6 +45,30 @@ object GameHelper {
return games.toList()
}
private fun addGamesRecursive(
games: MutableList<Game>,
files: Array<MinimalDocumentFile>,
depth: Int
) {
if (depth <= 0) {
return
}
files.forEach {
if (it.isDirectory) {
addGamesRecursive(
games,
FileUtil.listFiles(YuzuApplication.appContext, it.uri),
depth - 1
)
} else {
if (Game.extensions.contains(FileUtil.getExtension(it.uri))) {
games.add(getGame(it.uri))
}
}
}
}
private fun getGame(uri: Uri): Game {
val filePath = uri.toString()
var name = NativeLibrary.getTitle(filePath)

View File

@ -0,0 +1,35 @@
// SPDX-FileCopyrightText: 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
package org.yuzu.yuzu_emu.utils
import android.view.View
object ViewUtils {
fun showView(view: View, length: Long = 300) {
view.apply {
alpha = 0f
visibility = View.VISIBLE
isClickable = true
}.animate().apply {
duration = length
alpha(1f)
}.start()
}
fun hideView(view: View, length: Long = 300) {
if (view.visibility == View.INVISIBLE) {
return
}
view.apply {
alpha = 1f
isClickable = false
}.animate().apply {
duration = length
alpha(0f)
}.withEndAction {
view.visibility = View.INVISIBLE
}.start()
}
}

View File

@ -11,6 +11,7 @@
#include "common/fs/path_util.h"
#include "common/logging/log.h"
#include "common/settings.h"
#include "common/settings_enums.h"
#include "core/hle/service/acc/profile_manager.h"
#include "input_common/main.h"
#include "jni/config.h"
@ -144,7 +145,9 @@ void Config::ReadValues() {
Service::Account::MAX_USERS - 1);
// Disable docked mode by default on Android
Settings::values.use_docked_mode = config->GetBoolean("System", "use_docked_mode", false);
Settings::values.use_docked_mode.SetValue(config->GetBoolean("System", "use_docked_mode", false)
? Settings::ConsoleMode::Docked
: Settings::ConsoleMode::Handheld);
const auto rng_seed_enabled = config->GetBoolean("System", "rng_seed_enabled", false);
if (rng_seed_enabled) {

View File

@ -30,6 +30,7 @@
#include "core/cpu_manager.h"
#include "core/crypto/key_manager.h"
#include "core/file_sys/card_image.h"
#include "core/file_sys/content_archive.h"
#include "core/file_sys/registered_cache.h"
#include "core/file_sys/submission_package.h"
#include "core/file_sys/vfs.h"
@ -224,6 +225,42 @@ public:
m_system.Renderer().NotifySurfaceChanged();
}
void ConfigureFilesystemProvider(const std::string& filepath) {
const auto file = m_system.GetFilesystem()->OpenFile(filepath, FileSys::Mode::Read);
if (!file) {
return;
}
auto loader = Loader::GetLoader(m_system, file);
if (!loader) {
return;
}
const auto file_type = loader->GetFileType();
if (file_type == Loader::FileType::Unknown || file_type == Loader::FileType::Error) {
return;
}
u64 program_id = 0;
const auto res2 = loader->ReadProgramId(program_id);
if (res2 == Loader::ResultStatus::Success && file_type == Loader::FileType::NCA) {
m_manual_provider->AddEntry(FileSys::TitleType::Application,
FileSys::GetCRTypeFromNCAType(FileSys::NCA{file}.GetType()),
program_id, file);
} else if (res2 == Loader::ResultStatus::Success &&
(file_type == Loader::FileType::XCI || file_type == Loader::FileType::NSP)) {
const auto nsp = file_type == Loader::FileType::NSP
? std::make_shared<FileSys::NSP>(file)
: FileSys::XCI{file}.GetSecurePartitionNSP();
for (const auto& title : nsp->GetNCAs()) {
for (const auto& entry : title.second) {
m_manual_provider->AddEntry(entry.first.first, entry.first.second, title.first,
entry.second->GetBaseFile());
}
}
}
}
Core::SystemResultStatus InitializeEmulation(const std::string& filepath) {
std::scoped_lock lock(m_mutex);
@ -254,8 +291,14 @@ public:
std::move(android_keyboard), // Software Keyboard
nullptr, // Web Browser
});
// Initialize filesystem.
m_manual_provider = std::make_unique<FileSys::ManualContentProvider>();
m_system.SetContentProvider(std::make_unique<FileSys::ContentProviderUnion>());
m_system.RegisterContentProvider(FileSys::ContentProviderUnionSlot::FrontendManual,
m_manual_provider.get());
m_system.GetFileSystemController().CreateFactories(*m_vfs);
ConfigureFilesystemProvider(filepath);
// Initialize account manager
m_profile_manager = std::make_unique<Service::Account::ProfileManager>();
@ -377,7 +420,7 @@ public:
return false;
}
return !Settings::values.use_docked_mode.GetValue();
return !Settings::IsDockedMode();
}
void SetDeviceType([[maybe_unused]] int index, int type) {
@ -489,6 +532,7 @@ private:
bool m_is_paused{};
SoftwareKeyboard::AndroidKeyboard* m_software_keyboard{};
std::unique_ptr<Service::Account::ProfileManager> m_profile_manager;
std::unique_ptr<FileSys::ManualContentProvider> m_manual_provider;
// GPU driver parameters
std::shared_ptr<Common::DynamicLibrary> m_vulkan_library;

View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout
<RelativeLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:id="@+id/setup_root"
@ -8,19 +8,24 @@
<androidx.viewpager2.widget.ViewPager2
android:id="@+id/viewPager2"
android:layout_width="0dp"
android:layout_height="0dp"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_alignParentTop="true"
android:layout_alignParentBottom="true"
android:clipToPadding="false" />
<androidx.constraintlayout.widget.ConstraintLayout
android:id="@+id/constraint_buttons"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_alignParentBottom="true"
android:layout_margin="8dp">
<com.google.android.material.button.MaterialButton
style="@style/Widget.Material3.Button.TextButton"
android:id="@+id/button_next"
style="@style/Widget.Material3.Button.TextButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_margin="16dp"
android:text="@string/next"
android:visibility="invisible"
app:layout_constraintBottom_toBottomOf="parent"
@ -31,10 +36,11 @@
style="@style/Widget.Material3.Button.TextButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_margin="16dp"
android:text="@string/back"
android:visibility="invisible"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintStart_toStartOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>
</androidx.constraintlayout.widget.ConstraintLayout>
</RelativeLayout>

View File

@ -21,45 +21,76 @@
</LinearLayout>
<LinearLayout
<androidx.constraintlayout.widget.ConstraintLayout
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_weight="1"
android:orientation="vertical"
android:gravity="center">
android:layout_weight="1">
<com.google.android.material.textview.MaterialTextView
style="@style/TextAppearance.Material3.DisplaySmall"
android:id="@+id/text_title"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:textAlignment="center"
style="@style/TextAppearance.Material3.DisplaySmall"
android:layout_width="0dp"
android:layout_height="0dp"
android:gravity="center"
android:textColor="?attr/colorOnSurface"
android:textStyle="bold"
app:layout_constraintBottom_toTopOf="@+id/text_description"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent"
app:layout_constraintVertical_weight="2"
tools:text="@string/welcome" />
<com.google.android.material.textview.MaterialTextView
style="@style/TextAppearance.Material3.TitleLarge"
android:id="@+id/text_description"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginTop="16dp"
android:paddingHorizontal="32dp"
android:textAlignment="center"
android:textSize="26sp"
app:lineHeight="40sp"
style="@style/TextAppearance.Material3.TitleLarge"
android:layout_width="0dp"
android:layout_height="0dp"
android:gravity="center"
android:textSize="20sp"
android:paddingHorizontal="16dp"
app:layout_constraintBottom_toTopOf="@+id/button_action"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@+id/text_title"
app:layout_constraintVertical_weight="2"
app:lineHeight="30sp"
tools:text="@string/welcome_description" />
<com.google.android.material.textview.MaterialTextView
android:id="@+id/text_confirmation"
style="@style/TextAppearance.Material3.TitleLarge"
android:layout_width="0dp"
android:layout_height="0dp"
android:paddingHorizontal="16dp"
android:paddingBottom="20dp"
android:gravity="center"
android:textSize="30sp"
android:visibility="invisible"
android:text="@string/step_complete"
android:textStyle="bold"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@+id/text_description"
app:layout_constraintVertical_weight="1"
app:lineHeight="30sp" />
<com.google.android.material.button.MaterialButton
android:id="@+id/button_action"
android:layout_width="wrap_content"
android:layout_height="56dp"
android:layout_marginTop="32dp"
android:layout_marginTop="16dp"
android:layout_marginBottom="48dp"
android:textSize="20sp"
app:iconSize="24sp"
app:iconGravity="end"
app:iconSize="24sp"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@+id/text_description"
tools:text="Get started" />
</LinearLayout>
</androidx.constraintlayout.widget.ConstraintLayout>
</LinearLayout>

View File

@ -53,6 +53,23 @@
android:layout_marginTop="5dp"
tools:text="@string/install_prod_keys_description" />
<com.google.android.material.textview.MaterialTextView
style="@style/TextAppearance.Material3.LabelMedium"
android:id="@+id/option_detail"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:textAlignment="viewStart"
android:textSize="14sp"
android:textStyle="bold"
android:singleLine="true"
android:marqueeRepeatLimit="marquee_forever"
android:ellipsize="none"
android:requiresFadingEdge="horizontal"
android:layout_marginTop="5dp"
android:visibility="gone"
tools:visibility="visible"
tools:text="/tree/primary:Games" />
</LinearLayout>
</LinearLayout>

View File

@ -5,23 +5,16 @@
android:layout_height="wrap_content"
android:orientation="vertical">
<TextView
<com.google.android.material.textview.MaterialTextView
android:id="@+id/text_value"
style="@style/TextAppearance.Material3.LabelMedium"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignParentTop="true"
android:layout_centerHorizontal="true"
android:layout_marginBottom="@dimen/spacing_medlarge"
android:layout_marginTop="@dimen/spacing_medlarge"
tools:text="75" />
<TextView
android:id="@+id/text_units"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignTop="@+id/text_value"
android:layout_toEndOf="@+id/text_value"
tools:text="%" />
tools:text="75%" />
<com.google.android.material.slider.Slider
android:id="@+id/slider"

View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout
<RelativeLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:id="@+id/setup_root"
@ -8,35 +8,39 @@
<androidx.viewpager2.widget.ViewPager2
android:id="@+id/viewPager2"
android:layout_width="0dp"
android:layout_height="0dp"
android:clipToPadding="false"
android:layout_marginBottom="16dp"
app:layout_constraintBottom_toTopOf="@+id/button_next"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_above="@+id/constraint_buttons"
android:layout_alignParentTop="true"
android:clipToPadding="false" />
<androidx.constraintlayout.widget.ConstraintLayout
android:id="@+id/constraint_buttons"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_margin="8dp"
android:layout_alignParentBottom="true">
<com.google.android.material.button.MaterialButton
style="@style/Widget.Material3.Button.TextButton"
android:id="@+id/button_next"
style="@style/Widget.Material3.Button.TextButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_margin="12dp"
android:text="@string/next"
android:visibility="invisible"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent" />
<com.google.android.material.button.MaterialButton
style="@style/Widget.Material3.Button.TextButton"
android:id="@+id/button_back"
style="@style/Widget.Material3.Button.TextButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_margin="12dp"
android:text="@string/back"
android:visibility="invisible"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintStart_toStartOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>
</androidx.constraintlayout.widget.ConstraintLayout>
</RelativeLayout>

View File

@ -1,9 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
<RelativeLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="wrap_content"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:background="?android:attr/selectableItemBackground"
android:clickable="true"
android:focusable="true"
@ -11,31 +12,40 @@
android:minHeight="72dp"
android:padding="@dimen/spacing_large">
<com.google.android.material.textview.MaterialTextView
style="@style/TextAppearance.Material3.HeadlineMedium"
android:id="@+id/text_setting_name"
android:layout_width="0dp"
<LinearLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:orientation="vertical">
<com.google.android.material.textview.MaterialTextView
android:id="@+id/text_setting_name"
style="@style/TextAppearance.Material3.HeadlineMedium"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_alignParentEnd="true"
android:layout_alignParentStart="true"
android:layout_alignParentTop="true"
android:textSize="16sp"
android:textAlignment="viewStart"
app:lineHeight="28dp"
android:textSize="16sp"
app:lineHeight="22dp"
tools:text="Setting Name" />
<TextView
style="@style/TextAppearance.Material3.BodySmall"
<com.google.android.material.textview.MaterialTextView
android:id="@+id/text_setting_description"
android:layout_width="wrap_content"
style="@style/TextAppearance.Material3.BodySmall"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_alignParentEnd="true"
android:layout_alignParentStart="true"
android:layout_alignStart="@+id/text_setting_name"
android:layout_below="@+id/text_setting_name"
android:layout_marginTop="@dimen/spacing_small"
android:visibility="visible"
android:textAlignment="viewStart"
tools:text="@string/app_disclaimer" />
<com.google.android.material.textview.MaterialTextView
android:id="@+id/text_setting_value"
style="@style/TextAppearance.Material3.LabelMedium"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginTop="@dimen/spacing_small"
android:textAlignment="viewStart"
android:textStyle="bold"
tools:text="1x" />
</LinearLayout>
</RelativeLayout>

View File

@ -21,11 +21,12 @@
app:layout_constraintVertical_chainStyle="spread"
app:layout_constraintWidth_max="220dp"
app:layout_constraintWidth_min="110dp"
app:layout_constraintVertical_weight="3" />
app:layout_constraintVertical_weight="3"
tools:src="@drawable/ic_notification" />
<com.google.android.material.textview.MaterialTextView
android:id="@+id/text_title"
style="@style/TextAppearance.Material3.DisplayMedium"
style="@style/TextAppearance.Material3.DisplaySmall"
android:layout_width="0dp"
android:layout_height="0dp"
android:textAlignment="center"
@ -44,23 +45,42 @@
android:layout_width="0dp"
android:layout_height="0dp"
android:textAlignment="center"
android:textSize="26sp"
android:textSize="20sp"
android:paddingHorizontal="16dp"
app:layout_constraintBottom_toTopOf="@+id/button_action"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@+id/text_title"
app:layout_constraintVertical_weight="2"
app:lineHeight="40sp"
app:lineHeight="30sp"
tools:text="@string/welcome_description" />
<com.google.android.material.textview.MaterialTextView
android:id="@+id/text_confirmation"
style="@style/TextAppearance.Material3.TitleLarge"
android:layout_width="wrap_content"
android:layout_height="0dp"
android:paddingHorizontal="16dp"
android:paddingTop="24dp"
android:textAlignment="center"
android:textSize="30sp"
android:visibility="invisible"
android:text="@string/step_complete"
android:textStyle="bold"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@+id/text_description"
app:layout_constraintVertical_weight="1"
app:lineHeight="30sp" />
<com.google.android.material.button.MaterialButton
android:id="@+id/button_action"
android:layout_width="wrap_content"
android:layout_height="56dp"
android:textSize="20sp"
android:layout_marginTop="16dp"
android:layout_marginBottom="48dp"
android:textSize="20sp"
app:iconGravity="end"
app:iconSize="24sp"
app:layout_constraintBottom_toBottomOf="parent"

View File

@ -29,6 +29,7 @@
<string name="back">Back</string>
<string name="add_games">Add Games</string>
<string name="add_games_description">Select your games folder</string>
<string name="step_complete">Complete!</string>
<!-- Home strings -->
<string name="home_games">Games</string>
@ -149,6 +150,7 @@
<string name="frame_limit_slider">Limit speed percent</string>
<string name="frame_limit_slider_description">Specifies the percentage to limit emulation speed. 100% is the normal speed. Values higher or lower will increase or decrease the speed limit.</string>
<string name="cpu_accuracy">CPU accuracy</string>
<string name="value_with_units">%1$s%2$s</string>
<!-- System settings strings -->
<string name="use_docked_mode">Docked Mode</string>

View File

@ -8,6 +8,7 @@
#include "audio_core/sink/cubeb_sink.h"
#include "audio_core/sink/sink_stream.h"
#include "common/logging/log.h"
#include "common/scope_exit.h"
#include "core/core.h"
#ifdef _WIN32
@ -332,25 +333,38 @@ std::vector<std::string> ListCubebSinkDevices(bool capture) {
return device_list;
}
u32 GetCubebLatency() {
cubeb* ctx;
namespace {
static long TmpDataCallback(cubeb_stream*, void*, const void*, void*, long) {
return TargetSampleCount;
}
static void TmpStateCallback(cubeb_stream*, void*, cubeb_state) {}
} // namespace
bool IsCubebSuitable() {
#if !defined(HAVE_CUBEB)
return false;
#else
cubeb* ctx{nullptr};
#ifdef _WIN32
auto com_init_result = CoInitializeEx(nullptr, COINIT_MULTITHREADED);
#endif
// Init cubeb
if (cubeb_init(&ctx, "yuzu Latency Getter", nullptr) != CUBEB_OK) {
LOG_CRITICAL(Audio_Sink, "cubeb_init failed");
// Return a large latency so we choose SDL instead.
return 10000u;
LOG_ERROR(Audio_Sink, "Cubeb failed to init, it is not suitable.");
return false;
}
SCOPE_EXIT({ cubeb_destroy(ctx); });
#ifdef _WIN32
if (SUCCEEDED(com_init_result)) {
CoUninitialize();
}
#endif
// Test min latency
cubeb_stream_params params{};
params.rate = TargetSampleRate;
params.channels = 2;
@ -361,12 +375,32 @@ u32 GetCubebLatency() {
u32 latency{0};
const auto latency_error = cubeb_get_min_latency(ctx, &params, &latency);
if (latency_error != CUBEB_OK) {
LOG_CRITICAL(Audio_Sink, "Error getting minimum latency, error: {}", latency_error);
latency = TargetSampleCount * 2;
LOG_ERROR(Audio_Sink, "Cubeb could not get min latency, it is not suitable.");
return false;
}
latency = std::max(latency, TargetSampleCount * 2);
cubeb_destroy(ctx);
return latency;
if (latency > TargetSampleCount * 3) {
LOG_ERROR(Audio_Sink, "Cubeb latency is too high, it is not suitable.");
return false;
}
// Test opening a device with standard parameters
cubeb_devid output_device{0};
cubeb_devid input_device{0};
std::string name{"Yuzu test"};
cubeb_stream* stream{nullptr};
if (cubeb_stream_init(ctx, &stream, name.c_str(), input_device, nullptr, output_device, &params,
latency, &TmpDataCallback, &TmpStateCallback, nullptr) != CUBEB_OK) {
LOG_CRITICAL(Audio_Sink, "Cubeb could not open a device, it is not suitable.");
return false;
}
cubeb_stream_stop(stream);
cubeb_stream_destroy(stream);
return true;
#endif
}
} // namespace AudioCore::Sink

View File

@ -97,10 +97,11 @@ private:
std::vector<std::string> ListCubebSinkDevices(bool capture);
/**
* Get the reported latency for this sink.
* Check if this backend is suitable for use.
* Checks if enabled, its latency, whether it opens successfully, etc.
*
* @return Minimum latency for this sink.
* @return True is this backend is suitable, false otherwise.
*/
u32 GetCubebLatency();
bool IsCubebSuitable();
} // namespace AudioCore::Sink

View File

@ -9,6 +9,7 @@
#include "audio_core/sink/sdl2_sink.h"
#include "audio_core/sink/sink_stream.h"
#include "common/logging/log.h"
#include "common/scope_exit.h"
#include "core/core.h"
namespace AudioCore::Sink {
@ -84,6 +85,7 @@ public:
}
Stop();
SDL_ClearQueuedAudio(device);
SDL_CloseAudioDevice(device);
}
@ -227,8 +229,42 @@ std::vector<std::string> ListSDLSinkDevices(bool capture) {
return device_list;
}
u32 GetSDLLatency() {
return TargetSampleCount * 2;
bool IsSDLSuitable() {
#if !defined(HAVE_SDL2)
return false;
#else
// Check SDL can init
if (!SDL_WasInit(SDL_INIT_AUDIO)) {
if (SDL_InitSubSystem(SDL_INIT_AUDIO) < 0) {
LOG_ERROR(Audio_Sink, "SDL failed to init, it is not suitable. Error: {}",
SDL_GetError());
return false;
}
}
// We can set any latency frequency we want with SDL, so no need to check that.
// Check we can open a device with standard parameters
SDL_AudioSpec spec;
spec.freq = TargetSampleRate;
spec.channels = 2u;
spec.format = AUDIO_S16SYS;
spec.samples = TargetSampleCount * 2;
spec.callback = nullptr;
spec.userdata = nullptr;
SDL_AudioSpec obtained;
auto device = SDL_OpenAudioDevice(nullptr, false, &spec, &obtained, false);
if (device == 0) {
LOG_ERROR(Audio_Sink, "SDL failed to open a device, it is not suitable. Error: {}",
SDL_GetError());
return false;
}
SDL_CloseAudioDevice(device);
return true;
#endif
}
} // namespace AudioCore::Sink

View File

@ -88,10 +88,11 @@ private:
std::vector<std::string> ListSDLSinkDevices(bool capture);
/**
* Get the reported latency for this sink.
* Check if this backend is suitable for use.
* Checks if enabled, its latency, whether it opens successfully, etc.
*
* @return Minimum latency for this sink.
* @return True is this backend is suitable, false otherwise.
*/
u32 GetSDLLatency();
bool IsSDLSuitable();
} // namespace AudioCore::Sink

View File

@ -22,7 +22,7 @@ namespace {
struct SinkDetails {
using FactoryFn = std::unique_ptr<Sink> (*)(std::string_view);
using ListDevicesFn = std::vector<std::string> (*)(bool);
using LatencyFn = u32 (*)();
using SuitableFn = bool (*)();
/// Name for this sink.
Settings::AudioEngine id;
@ -30,8 +30,8 @@ struct SinkDetails {
FactoryFn factory;
/// A method to call to list available devices.
ListDevicesFn list_devices;
/// Method to get the latency of this backend.
LatencyFn latency;
/// Check whether this backend is suitable to be used.
SuitableFn is_suitable;
};
// sink_details is ordered in terms of desirability, with the best choice at the top.
@ -43,7 +43,7 @@ constexpr SinkDetails sink_details[] = {
return std::make_unique<CubebSink>(device_id);
},
&ListCubebSinkDevices,
&GetCubebLatency,
&IsCubebSuitable,
},
#endif
#ifdef HAVE_SDL2
@ -53,14 +53,17 @@ constexpr SinkDetails sink_details[] = {
return std::make_unique<SDLSink>(device_id);
},
&ListSDLSinkDevices,
&GetSDLLatency,
&IsSDLSuitable,
},
#endif
SinkDetails{Settings::AudioEngine::Null,
SinkDetails{
Settings::AudioEngine::Null,
[](std::string_view device_id) -> std::unique_ptr<Sink> {
return std::make_unique<NullSink>(device_id);
},
[](bool capture) { return std::vector<std::string>{"null"}; }, []() { return 0u; }},
[](bool capture) { return std::vector<std::string>{"null"}; },
[]() { return true; },
},
};
const SinkDetails& GetOutputSinkDetails(Settings::AudioEngine sink_id) {
@ -72,18 +75,22 @@ const SinkDetails& GetOutputSinkDetails(Settings::AudioEngine sink_id) {
auto iter = find_backend(sink_id);
if (sink_id == Settings::AudioEngine::Auto) {
// Auto-select a backend. Prefer CubeB, but it may report a large minimum latency which
// causes audio issues, in that case go with SDL.
#if defined(HAVE_CUBEB) && defined(HAVE_SDL2)
iter = find_backend(Settings::AudioEngine::Cubeb);
if (iter->latency() > TargetSampleCount * 3) {
iter = find_backend(Settings::AudioEngine::Sdl2);
// Auto-select a backend. Use the sink details ordering, preferring cubeb first, checking
// that the backend is available and suitable to use.
for (auto& details : sink_details) {
if (details.is_suitable()) {
iter = &details;
break;
}
#else
iter = std::begin(sink_details);
#endif
LOG_INFO(Service_Audio, "Auto-selecting the {} backend",
}
LOG_ERROR(Service_Audio, "Auto-selecting the {} backend",
Settings::CanonicalizeEnum(iter->id));
} else {
if (iter != std::end(sink_details) && !iter->is_suitable()) {
LOG_ERROR(Service_Audio, "Selected backend {} is not suitable, falling back to null",
Settings::CanonicalizeEnum(iter->id));
iter = find_backend(Settings::AudioEngine::Null);
}
}
if (iter == std::end(sink_details)) {

View File

@ -3,6 +3,7 @@
#pragma once
#include <bit>
#include <cstddef>
#include <new>
#include <type_traits>
@ -10,8 +11,10 @@
namespace Common {
template <typename T>
requires std::is_unsigned_v<T>
[[nodiscard]] constexpr T AlignUp(T value, size_t size) {
requires std::is_integral_v<T>
[[nodiscard]] constexpr T AlignUp(T value_, size_t size) {
using U = typename std::make_unsigned_t<T>;
auto value{static_cast<U>(value_)};
auto mod{static_cast<T>(value % size)};
value -= mod;
return static_cast<T>(mod == T{0} ? value : value + size);
@ -24,8 +27,10 @@ template <typename T>
}
template <typename T>
requires std::is_unsigned_v<T>
[[nodiscard]] constexpr T AlignDown(T value, size_t size) {
requires std::is_integral_v<T>
[[nodiscard]] constexpr T AlignDown(T value_, size_t size) {
using U = typename std::make_unsigned_t<T>;
const auto value{static_cast<U>(value_)};
return static_cast<T>(value - value % size);
}
@ -55,6 +60,30 @@ template <typename T, typename U>
return (x + (y - 1)) / y;
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T LeastSignificantOneBit(T x) {
return x & ~(x - 1);
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T ResetLeastSignificantOneBit(T x) {
return x & (x - 1);
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr bool IsPowerOfTwo(T x) {
return x > 0 && ResetLeastSignificantOneBit(x) == 0;
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T FloorPowerOfTwo(T x) {
return T{1} << (sizeof(T) * 8 - std::countl_zero(x) - 1);
}
template <typename T, size_t Align = 16>
class AlignmentAllocator {
public:

View File

@ -71,4 +71,10 @@ std::vector<u8> DecompressDataLZ4(std::span<const u8> compressed, std::size_t un
return uncompressed;
}
int DecompressDataLZ4(void* dst, size_t dst_size, const void* src, size_t src_size) {
// This is just a thin wrapper around LZ4.
return LZ4_decompress_safe(reinterpret_cast<const char*>(src), reinterpret_cast<char*>(dst),
static_cast<int>(src_size), static_cast<int>(dst_size));
}
} // namespace Common::Compression

View File

@ -56,4 +56,6 @@ namespace Common::Compression {
[[nodiscard]] std::vector<u8> DecompressDataLZ4(std::span<const u8> compressed,
std::size_t uncompressed_size);
[[nodiscard]] int DecompressDataLZ4(void* dst, size_t dst_size, const void* src, size_t src_size);
} // namespace Common::Compression

View File

@ -2,6 +2,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <version>
#include "common/settings_enums.h"
#if __cpp_lib_chrono >= 201907L
#include <chrono>
#include <exception>
@ -145,6 +146,10 @@ bool IsFastmemEnabled() {
return true;
}
bool IsDockedMode() {
return values.use_docked_mode.GetValue() == Settings::ConsoleMode::Docked;
}
float Volume() {
if (values.audio_muted) {
return 0.0f;
@ -207,9 +212,7 @@ const char* TranslateCategory(Category category) {
return "Miscellaneous";
}
void UpdateRescalingInfo() {
const auto setup = values.resolution_setup.GetValue();
auto& info = values.resolution_info;
void TranslateResolutionInfo(ResolutionSetup setup, ResolutionScalingInfo& info) {
info.downscale = false;
switch (setup) {
case ResolutionSetup::Res1_2X:
@ -269,6 +272,12 @@ void UpdateRescalingInfo() {
info.active = info.up_scale != 1 || info.down_shift != 0;
}
void UpdateRescalingInfo() {
const auto setup = values.resolution_setup.GetValue();
auto& info = values.resolution_info;
TranslateResolutionInfo(setup, info);
}
void RestoreGlobalState(bool is_powered_on) {
// If a game is running, DO NOT restore the global settings state
if (is_powered_on) {

View File

@ -379,7 +379,13 @@ struct Values {
Setting<s32> current_user{linkage, 0, "current_user", Category::System};
SwitchableSetting<bool> use_docked_mode{linkage, true, "use_docked_mode", Category::System};
SwitchableSetting<ConsoleMode> use_docked_mode{linkage,
ConsoleMode::Docked,
"use_docked_mode",
Category::System,
Specialization::Radio,
true,
true};
// Controls
InputSetting<std::array<PlayerInput, 10>> players;
@ -519,12 +525,15 @@ bool IsGPULevelHigh();
bool IsFastmemEnabled();
bool IsDockedMode();
float Volume();
std::string GetTimeZoneString(TimeZone time_zone);
void LogSettings();
void TranslateResolutionInfo(ResolutionSetup setup, ResolutionScalingInfo& info);
void UpdateRescalingInfo();
// Restore the global state of all applicable settings in the Values struct

View File

@ -1,7 +1,9 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <functional>
#include <string>
#include <vector>
#include "common/settings_common.h"
namespace Settings {

View File

@ -56,6 +56,7 @@ enum Specialization : u8 {
Scalar = 5, // Values are continuous
Countable = 6, // Can be stepped through
Paired = 7, // Another setting is associated with this setting
Radio = 8, // Setting should be presented in a radio group
Percentage = (1 << SpecializationAttributeOffset), // Should be represented as a percentage
};

View File

@ -12,8 +12,8 @@ namespace Settings {
template <typename T>
struct EnumMetadata {
static constexpr std::vector<std::pair<std::string, T>> Canonicalizations();
static constexpr u32 Index();
static std::vector<std::pair<std::string, T>> Canonicalizations();
static u32 Index();
};
#define PAIR_45(N, X, ...) {#X, N::X} __VA_OPT__(, PAIR_46(N, __VA_ARGS__))
@ -66,11 +66,11 @@ struct EnumMetadata {
#define ENUM(NAME, ...) \
enum class NAME : u32 { __VA_ARGS__ }; \
template <> \
constexpr std::vector<std::pair<std::string, NAME>> EnumMetadata<NAME>::Canonicalizations() { \
inline std::vector<std::pair<std::string, NAME>> EnumMetadata<NAME>::Canonicalizations() { \
return {PAIR(NAME, __VA_ARGS__)}; \
} \
template <> \
constexpr u32 EnumMetadata<NAME>::Index() { \
inline u32 EnumMetadata<NAME>::Index() { \
return __COUNTER__; \
}
@ -85,7 +85,7 @@ enum class AudioEngine : u32 {
};
template <>
constexpr std::vector<std::pair<std::string, AudioEngine>>
inline std::vector<std::pair<std::string, AudioEngine>>
EnumMetadata<AudioEngine>::Canonicalizations() {
return {
{"auto", AudioEngine::Auto},
@ -96,7 +96,7 @@ EnumMetadata<AudioEngine>::Canonicalizations() {
}
template <>
constexpr u32 EnumMetadata<AudioEngine>::Index() {
inline u32 EnumMetadata<AudioEngine>::Index() {
// This is just a sufficiently large number that is more than the number of other enums declared
// here
return 100;
@ -146,8 +146,10 @@ ENUM(AntiAliasing, None, Fxaa, Smaa, MaxEnum);
ENUM(AspectRatio, R16_9, R4_3, R21_9, R16_10, Stretch);
ENUM(ConsoleMode, Handheld, Docked);
template <typename Type>
constexpr std::string CanonicalizeEnum(Type id) {
inline std::string CanonicalizeEnum(Type id) {
const auto group = EnumMetadata<Type>::Canonicalizations();
for (auto& [name, value] : group) {
if (value == id) {
@ -158,7 +160,7 @@ constexpr std::string CanonicalizeEnum(Type id) {
}
template <typename Type>
constexpr Type ToEnum(const std::string& canonicalization) {
inline Type ToEnum(const std::string& canonicalization) {
const auto group = EnumMetadata<Type>::Canonicalizations();
for (auto& [name, value] : group) {
if (name == canonicalization) {

View File

@ -190,7 +190,7 @@ public:
}
}
[[nodiscard]] std::string constexpr Canonicalize() const override final {
[[nodiscard]] std::string Canonicalize() const override final {
if constexpr (std::is_enum_v<Type>) {
return CanonicalizeEnum(this->GetValue());
} else {
@ -256,11 +256,11 @@ public:
* @param runtime_modifiable_ Suggests whether this is modifiable while a guest is loaded
* @param other_setting_ A second Setting to associate to this one in metadata
*/
template <typename T = BasicSetting>
explicit SwitchableSetting(Linkage& linkage, const Type& default_val, const std::string& name,
Category category_, u32 specialization_ = Specialization::Default,
bool save_ = true, bool runtime_modifiable_ = false,
BasicSetting* other_setting_ = nullptr)
requires(!ranged)
typename std::enable_if<!ranged, T*>::type other_setting_ = nullptr)
: Setting<Type, false>{
linkage, default_val, name, category_, specialization_,
save_, runtime_modifiable_, other_setting_} {
@ -282,12 +282,12 @@ public:
* @param runtime_modifiable_ Suggests whether this is modifiable while a guest is loaded
* @param other_setting_ A second Setting to associate to this one in metadata
*/
template <typename T = BasicSetting>
explicit SwitchableSetting(Linkage& linkage, const Type& default_val, const Type& min_val,
const Type& max_val, const std::string& name, Category category_,
u32 specialization_ = Specialization::Default, bool save_ = true,
bool runtime_modifiable_ = false,
BasicSetting* other_setting_ = nullptr)
requires(ranged)
typename std::enable_if<ranged, T*>::type other_setting_ = nullptr)
: Setting<Type, true>{linkage, default_val, min_val,
max_val, name, category_,
specialization_, save_, runtime_modifiable_,

View File

@ -460,11 +460,6 @@ S operator&(const S& i, const swap_struct_t<T, F> v) {
return i & v.swap();
}
template <typename S, typename T, typename F>
S operator&(const swap_struct_t<T, F> v, const S& i) {
return static_cast<S>(v.swap() & i);
}
// Comparison
template <typename S, typename T, typename F>
bool operator<(const S& p, const swap_struct_t<T, F> v) {

View File

@ -37,6 +37,49 @@ add_library(core STATIC
debugger/gdbstub.h
device_memory.cpp
device_memory.h
file_sys/fssystem/fs_i_storage.h
file_sys/fssystem/fssystem_aes_ctr_counter_extended_storage.cpp
file_sys/fssystem/fssystem_aes_ctr_counter_extended_storage.h
file_sys/fssystem/fssystem_aes_ctr_storage.cpp
file_sys/fssystem/fssystem_aes_ctr_storage.h
file_sys/fssystem/fssystem_aes_xts_storage.cpp
file_sys/fssystem/fssystem_aes_xts_storage.h
file_sys/fssystem/fssystem_alignment_matching_storage.h
file_sys/fssystem/fssystem_alignment_matching_storage_impl.cpp
file_sys/fssystem/fssystem_alignment_matching_storage_impl.h
file_sys/fssystem/fssystem_bucket_tree.cpp
file_sys/fssystem/fssystem_bucket_tree.h
file_sys/fssystem/fssystem_bucket_tree_utils.h
file_sys/fssystem/fssystem_compressed_storage.h
file_sys/fssystem/fssystem_compression_common.h
file_sys/fssystem/fssystem_compression_configuration.cpp
file_sys/fssystem/fssystem_compression_configuration.h
file_sys/fssystem/fssystem_crypto_configuration.cpp
file_sys/fssystem/fssystem_crypto_configuration.h
file_sys/fssystem/fssystem_hierarchical_integrity_verification_storage.cpp
file_sys/fssystem/fssystem_hierarchical_integrity_verification_storage.h
file_sys/fssystem/fssystem_hierarchical_sha256_storage.cpp
file_sys/fssystem/fssystem_hierarchical_sha256_storage.h
file_sys/fssystem/fssystem_indirect_storage.cpp
file_sys/fssystem/fssystem_indirect_storage.h
file_sys/fssystem/fssystem_integrity_romfs_storage.cpp
file_sys/fssystem/fssystem_integrity_romfs_storage.h
file_sys/fssystem/fssystem_integrity_verification_storage.cpp
file_sys/fssystem/fssystem_integrity_verification_storage.h
file_sys/fssystem/fssystem_memory_resource_buffer_hold_storage.h
file_sys/fssystem/fssystem_nca_file_system_driver.cpp
file_sys/fssystem/fssystem_nca_file_system_driver.h
file_sys/fssystem/fssystem_nca_header.cpp
file_sys/fssystem/fssystem_nca_header.h
file_sys/fssystem/fssystem_nca_reader.cpp
file_sys/fssystem/fssystem_pooled_buffer.cpp
file_sys/fssystem/fssystem_pooled_buffer.h
file_sys/fssystem/fssystem_sparse_storage.cpp
file_sys/fssystem/fssystem_sparse_storage.h
file_sys/fssystem/fssystem_switch_storage.h
file_sys/fssystem/fssystem_utility.cpp
file_sys/fssystem/fssystem_utility.h
file_sys/fssystem/fs_types.h
file_sys/bis_factory.cpp
file_sys/bis_factory.h
file_sys/card_image.cpp
@ -57,8 +100,6 @@ add_library(core STATIC
file_sys/mode.h
file_sys/nca_metadata.cpp
file_sys/nca_metadata.h
file_sys/nca_patch.cpp
file_sys/nca_patch.h
file_sys/partition_filesystem.cpp
file_sys/partition_filesystem.h
file_sys/patch_manager.cpp

View File

@ -263,6 +263,23 @@ void GDBStub::ExecuteCommand(std::string_view packet, std::vector<DebuggerAction
std::vector<u8> mem(size);
if (system.ApplicationMemory().ReadBlock(addr, mem.data(), size)) {
// Restore any bytes belonging to replaced instructions.
auto it = replaced_instructions.lower_bound(addr);
for (; it != replaced_instructions.end() && it->first < addr + size; it++) {
// Get the bytes of the instruction we previously replaced.
const u32 original_bytes = it->second;
// Calculate where to start writing to the output buffer.
const size_t output_offset = it->first - addr;
// Calculate how many bytes to write.
// The loop condition ensures output_offset < size.
const size_t n = std::min<size_t>(size - output_offset, sizeof(u32));
// Write the bytes to the output buffer.
std::memcpy(mem.data() + output_offset, &original_bytes, n);
}
SendReply(Common::HexToString(mem));
} else {
SendReply(GDB_STUB_REPLY_ERR);

View File

@ -31,13 +31,9 @@ XCI::XCI(VirtualFile file_, u64 program_id, size_t program_index)
: file(std::move(file_)), program_nca_status{Loader::ResultStatus::ErrorXCIMissingProgramNCA},
partitions(partition_names.size()),
partitions_raw(partition_names.size()), keys{Core::Crypto::KeyManager::Instance()} {
if (file->ReadObject(&header) != sizeof(GamecardHeader)) {
status = Loader::ResultStatus::ErrorBadXCIHeader;
return;
}
if (header.magic != Common::MakeMagic('H', 'E', 'A', 'D')) {
status = Loader::ResultStatus::ErrorBadXCIHeader;
const auto header_status = TryReadHeader();
if (header_status != Loader::ResultStatus::Success) {
status = header_status;
return;
}
@ -183,9 +179,9 @@ u32 XCI::GetSystemUpdateVersion() {
}
for (const auto& update_file : update->GetFiles()) {
NCA nca{update_file, nullptr, 0};
NCA nca{update_file};
if (nca.GetStatus() != Loader::ResultStatus::Success) {
if (nca.GetStatus() != Loader::ResultStatus::Success || nca.GetSubdirectories().empty()) {
continue;
}
@ -296,7 +292,7 @@ Loader::ResultStatus XCI::AddNCAFromPartition(XCIPartition part) {
continue;
}
auto nca = std::make_shared<NCA>(partition_file, nullptr, 0);
auto nca = std::make_shared<NCA>(partition_file);
if (nca->IsUpdate()) {
continue;
}
@ -316,6 +312,44 @@ Loader::ResultStatus XCI::AddNCAFromPartition(XCIPartition part) {
return Loader::ResultStatus::Success;
}
Loader::ResultStatus XCI::TryReadHeader() {
constexpr size_t CardInitialDataRegionSize = 0x1000;
// Define the function we'll use to determine if we read a valid header.
const auto ReadCardHeader = [&]() {
// Ensure we can read the entire header. If we can't, we can't read the card image.
if (file->ReadObject(&header) != sizeof(GamecardHeader)) {
return Loader::ResultStatus::ErrorBadXCIHeader;
}
// Ensure the header magic matches. If it doesn't, this isn't a card image header.
if (header.magic != Common::MakeMagic('H', 'E', 'A', 'D')) {
return Loader::ResultStatus::ErrorBadXCIHeader;
}
// We read a card image header.
return Loader::ResultStatus::Success;
};
// Try to read the header directly.
if (ReadCardHeader() == Loader::ResultStatus::Success) {
return Loader::ResultStatus::Success;
}
// Get the size of the file.
const size_t card_image_size = file->GetSize();
// If we are large enough to have a key area, offset past the key area and retry.
if (card_image_size >= CardInitialDataRegionSize) {
file = std::make_shared<OffsetVfsFile>(file, card_image_size - CardInitialDataRegionSize,
CardInitialDataRegionSize);
return ReadCardHeader();
}
// We had no header and aren't large enough to have a key area, so this can't be parsed.
return Loader::ResultStatus::ErrorBadXCIHeader;
}
u8 XCI::GetFormatVersion() {
return GetLogoPartition() == nullptr ? 0x1 : 0x2;
}

View File

@ -128,6 +128,7 @@ public:
private:
Loader::ResultStatus AddNCAFromPartition(XCIPartition part);
Loader::ResultStatus TryReadHeader();
VirtualFile file;
GamecardHeader header{};

View File

@ -12,546 +12,110 @@
#include "core/crypto/ctr_encryption_layer.h"
#include "core/crypto/key_manager.h"
#include "core/file_sys/content_archive.h"
#include "core/file_sys/nca_patch.h"
#include "core/file_sys/partition_filesystem.h"
#include "core/file_sys/vfs_offset.h"
#include "core/loader/loader.h"
#include "core/file_sys/fssystem/fssystem_compression_configuration.h"
#include "core/file_sys/fssystem/fssystem_crypto_configuration.h"
#include "core/file_sys/fssystem/fssystem_nca_file_system_driver.h"
namespace FileSys {
// Media offsets in headers are stored divided by 512. Mult. by this to get real offset.
constexpr u64 MEDIA_OFFSET_MULTIPLIER = 0x200;
constexpr u64 SECTION_HEADER_SIZE = 0x200;
constexpr u64 SECTION_HEADER_OFFSET = 0x400;
constexpr u32 IVFC_MAX_LEVEL = 6;
enum class NCASectionFilesystemType : u8 {
PFS0 = 0x2,
ROMFS = 0x3,
};
struct IVFCLevel {
u64_le offset;
u64_le size;
u32_le block_size;
u32_le reserved;
};
static_assert(sizeof(IVFCLevel) == 0x18, "IVFCLevel has incorrect size.");
struct IVFCHeader {
u32_le magic;
u32_le magic_number;
INSERT_PADDING_BYTES_NOINIT(8);
std::array<IVFCLevel, 6> levels;
INSERT_PADDING_BYTES_NOINIT(64);
};
static_assert(sizeof(IVFCHeader) == 0xE0, "IVFCHeader has incorrect size.");
struct NCASectionHeaderBlock {
INSERT_PADDING_BYTES_NOINIT(3);
NCASectionFilesystemType filesystem_type;
NCASectionCryptoType crypto_type;
INSERT_PADDING_BYTES_NOINIT(3);
};
static_assert(sizeof(NCASectionHeaderBlock) == 0x8, "NCASectionHeaderBlock has incorrect size.");
struct NCABucketInfo {
u64 table_offset;
u64 table_size;
std::array<u8, 0x10> table_header;
};
static_assert(sizeof(NCABucketInfo) == 0x20, "NCABucketInfo has incorrect size.");
struct NCASparseInfo {
NCABucketInfo bucket;
u64 physical_offset;
u16 generation;
INSERT_PADDING_BYTES_NOINIT(0x6);
};
static_assert(sizeof(NCASparseInfo) == 0x30, "NCASparseInfo has incorrect size.");
struct NCACompressionInfo {
NCABucketInfo bucket;
INSERT_PADDING_BYTES_NOINIT(0x8);
};
static_assert(sizeof(NCACompressionInfo) == 0x28, "NCACompressionInfo has incorrect size.");
struct NCASectionRaw {
NCASectionHeaderBlock header;
std::array<u8, 0x138> block_data;
std::array<u8, 0x8> section_ctr;
NCASparseInfo sparse_info;
NCACompressionInfo compression_info;
INSERT_PADDING_BYTES_NOINIT(0x60);
};
static_assert(sizeof(NCASectionRaw) == 0x200, "NCASectionRaw has incorrect size.");
struct PFS0Superblock {
NCASectionHeaderBlock header_block;
std::array<u8, 0x20> hash;
u32_le size;
INSERT_PADDING_BYTES_NOINIT(4);
u64_le hash_table_offset;
u64_le hash_table_size;
u64_le pfs0_header_offset;
u64_le pfs0_size;
INSERT_PADDING_BYTES_NOINIT(0x1B0);
};
static_assert(sizeof(PFS0Superblock) == 0x200, "PFS0Superblock has incorrect size.");
struct RomFSSuperblock {
NCASectionHeaderBlock header_block;
IVFCHeader ivfc;
INSERT_PADDING_BYTES_NOINIT(0x118);
};
static_assert(sizeof(RomFSSuperblock) == 0x200, "RomFSSuperblock has incorrect size.");
struct BKTRHeader {
u64_le offset;
u64_le size;
u32_le magic;
INSERT_PADDING_BYTES_NOINIT(0x4);
u32_le number_entries;
INSERT_PADDING_BYTES_NOINIT(0x4);
};
static_assert(sizeof(BKTRHeader) == 0x20, "BKTRHeader has incorrect size.");
struct BKTRSuperblock {
NCASectionHeaderBlock header_block;
IVFCHeader ivfc;
INSERT_PADDING_BYTES_NOINIT(0x18);
BKTRHeader relocation;
BKTRHeader subsection;
INSERT_PADDING_BYTES_NOINIT(0xC0);
};
static_assert(sizeof(BKTRSuperblock) == 0x200, "BKTRSuperblock has incorrect size.");
union NCASectionHeader {
NCASectionRaw raw{};
PFS0Superblock pfs0;
RomFSSuperblock romfs;
BKTRSuperblock bktr;
};
static_assert(sizeof(NCASectionHeader) == 0x200, "NCASectionHeader has incorrect size.");
static bool IsValidNCA(const NCAHeader& header) {
// TODO(DarkLordZach): Add NCA2/NCA0 support.
return header.magic == Common::MakeMagic('N', 'C', 'A', '3');
}
NCA::NCA(VirtualFile file_, VirtualFile bktr_base_romfs_, u64 bktr_base_ivfc_offset)
: file(std::move(file_)),
bktr_base_romfs(std::move(bktr_base_romfs_)), keys{Core::Crypto::KeyManager::Instance()} {
NCA::NCA(VirtualFile file_, const NCA* base_nca)
: file(std::move(file_)), keys{Core::Crypto::KeyManager::Instance()} {
if (file == nullptr) {
status = Loader::ResultStatus::ErrorNullFile;
return;
}
if (sizeof(NCAHeader) != file->ReadObject(&header)) {
LOG_ERROR(Loader, "File reader errored out during header read.");
reader = std::make_shared<NcaReader>();
if (Result rc =
reader->Initialize(file, GetCryptoConfiguration(), GetNcaCompressionConfiguration());
R_FAILED(rc)) {
if (rc != ResultInvalidNcaSignature) {
LOG_ERROR(Loader, "File reader errored out during header read: {:#x}",
rc.GetInnerValue());
}
status = Loader::ResultStatus::ErrorBadNCAHeader;
return;
}
if (!HandlePotentialHeaderDecryption()) {
RightsId rights_id{};
reader->GetRightsId(rights_id.data(), rights_id.size());
if (rights_id != RightsId{}) {
// External decryption key required; provide it here.
const auto key_generation = std::max<s32>(reader->GetKeyGeneration(), 1) - 1;
u128 rights_id_u128;
std::memcpy(rights_id_u128.data(), rights_id.data(), sizeof(rights_id));
auto titlekey =
keys.GetKey(Core::Crypto::S128KeyType::Titlekey, rights_id_u128[1], rights_id_u128[0]);
if (titlekey == Core::Crypto::Key128{}) {
status = Loader::ResultStatus::ErrorMissingTitlekey;
return;
}
has_rights_id = std::ranges::any_of(header.rights_id, [](char c) { return c != '\0'; });
const std::vector<NCASectionHeader> sections = ReadSectionHeaders();
is_update = std::ranges::any_of(sections, [](const NCASectionHeader& nca_header) {
return nca_header.raw.header.crypto_type == NCASectionCryptoType::BKTR;
});
if (!ReadSections(sections, bktr_base_ivfc_offset)) {
if (!keys.HasKey(Core::Crypto::S128KeyType::Titlekek, key_generation)) {
status = Loader::ResultStatus::ErrorMissingTitlekek;
return;
}
auto titlekek = keys.GetKey(Core::Crypto::S128KeyType::Titlekek, key_generation);
Core::Crypto::AESCipher<Core::Crypto::Key128> cipher(titlekek, Core::Crypto::Mode::ECB);
cipher.Transcode(titlekey.data(), titlekey.size(), titlekey.data(),
Core::Crypto::Op::Decrypt);
reader->SetExternalDecryptionKey(titlekey.data(), titlekey.size());
}
const s32 fs_count = reader->GetFsCount();
NcaFileSystemDriver fs(base_nca ? base_nca->reader : nullptr, reader);
std::vector<VirtualFile> filesystems(fs_count);
for (s32 i = 0; i < fs_count; i++) {
NcaFsHeaderReader header_reader;
const Result rc = fs.OpenStorage(&filesystems[i], &header_reader, i);
if (R_FAILED(rc)) {
LOG_ERROR(Loader, "File reader errored out during read of section {}: {:#x}", i,
rc.GetInnerValue());
status = Loader::ResultStatus::ErrorBadNCAHeader;
return;
}
if (header_reader.GetFsType() == NcaFsHeader::FsType::RomFs) {
files.push_back(filesystems[i]);
romfs = files.back();
}
if (header_reader.GetFsType() == NcaFsHeader::FsType::PartitionFs) {
auto npfs = std::make_shared<PartitionFilesystem>(filesystems[i]);
if (npfs->GetStatus() == Loader::ResultStatus::Success) {
dirs.push_back(npfs);
if (IsDirectoryExeFS(npfs)) {
exefs = dirs.back();
} else if (IsDirectoryLogoPartition(npfs)) {
logo = dirs.back();
} else {
continue;
}
}
}
if (header_reader.GetEncryptionType() == NcaFsHeader::EncryptionType::AesCtrEx) {
is_update = true;
}
}
if (is_update && base_nca == nullptr) {
status = Loader::ResultStatus::ErrorMissingBKTRBaseRomFS;
} else {
status = Loader::ResultStatus::Success;
}
}
NCA::~NCA() = default;
bool NCA::CheckSupportedNCA(const NCAHeader& nca_header) {
if (nca_header.magic == Common::MakeMagic('N', 'C', 'A', '2')) {
status = Loader::ResultStatus::ErrorNCA2;
return false;
}
if (nca_header.magic == Common::MakeMagic('N', 'C', 'A', '0')) {
status = Loader::ResultStatus::ErrorNCA0;
return false;
}
return true;
}
bool NCA::HandlePotentialHeaderDecryption() {
if (IsValidNCA(header)) {
return true;
}
if (!CheckSupportedNCA(header)) {
return false;
}
NCAHeader dec_header{};
Core::Crypto::AESCipher<Core::Crypto::Key256> cipher(
keys.GetKey(Core::Crypto::S256KeyType::Header), Core::Crypto::Mode::XTS);
cipher.XTSTranscode(&header, sizeof(NCAHeader), &dec_header, 0, 0x200,
Core::Crypto::Op::Decrypt);
if (IsValidNCA(dec_header)) {
header = dec_header;
encrypted = true;
} else {
if (!CheckSupportedNCA(dec_header)) {
return false;
}
if (keys.HasKey(Core::Crypto::S256KeyType::Header)) {
status = Loader::ResultStatus::ErrorIncorrectHeaderKey;
} else {
status = Loader::ResultStatus::ErrorMissingHeaderKey;
}
return false;
}
return true;
}
std::vector<NCASectionHeader> NCA::ReadSectionHeaders() const {
const std::ptrdiff_t number_sections =
std::ranges::count_if(header.section_tables, [](const NCASectionTableEntry& entry) {
return entry.media_offset > 0;
});
std::vector<NCASectionHeader> sections(number_sections);
const auto length_sections = SECTION_HEADER_SIZE * number_sections;
if (encrypted) {
auto raw = file->ReadBytes(length_sections, SECTION_HEADER_OFFSET);
Core::Crypto::AESCipher<Core::Crypto::Key256> cipher(
keys.GetKey(Core::Crypto::S256KeyType::Header), Core::Crypto::Mode::XTS);
cipher.XTSTranscode(raw.data(), length_sections, sections.data(), 2, SECTION_HEADER_SIZE,
Core::Crypto::Op::Decrypt);
} else {
file->ReadBytes(sections.data(), length_sections, SECTION_HEADER_OFFSET);
}
return sections;
}
bool NCA::ReadSections(const std::vector<NCASectionHeader>& sections, u64 bktr_base_ivfc_offset) {
for (std::size_t i = 0; i < sections.size(); ++i) {
const auto& section = sections[i];
if (section.raw.sparse_info.bucket.table_offset != 0 &&
section.raw.sparse_info.bucket.table_size != 0) {
LOG_ERROR(Loader, "Sparse NCAs are not supported.");
status = Loader::ResultStatus::ErrorSparseNCA;
return false;
}
if (section.raw.compression_info.bucket.table_offset != 0 &&
section.raw.compression_info.bucket.table_size != 0) {
LOG_ERROR(Loader, "Compressed NCAs are not supported.");
status = Loader::ResultStatus::ErrorCompressedNCA;
return false;
}
if (section.raw.header.filesystem_type == NCASectionFilesystemType::ROMFS) {
if (!ReadRomFSSection(section, header.section_tables[i], bktr_base_ivfc_offset)) {
return false;
}
} else if (section.raw.header.filesystem_type == NCASectionFilesystemType::PFS0) {
if (!ReadPFS0Section(section, header.section_tables[i])) {
return false;
}
}
}
return true;
}
bool NCA::ReadRomFSSection(const NCASectionHeader& section, const NCASectionTableEntry& entry,
u64 bktr_base_ivfc_offset) {
const std::size_t base_offset = entry.media_offset * MEDIA_OFFSET_MULTIPLIER;
ivfc_offset = section.romfs.ivfc.levels[IVFC_MAX_LEVEL - 1].offset;
const std::size_t romfs_offset = base_offset + ivfc_offset;
const std::size_t romfs_size = section.romfs.ivfc.levels[IVFC_MAX_LEVEL - 1].size;
auto raw = std::make_shared<OffsetVfsFile>(file, romfs_size, romfs_offset);
auto dec = Decrypt(section, raw, romfs_offset);
if (dec == nullptr) {
if (status != Loader::ResultStatus::Success)
return false;
if (has_rights_id)
status = Loader::ResultStatus::ErrorIncorrectTitlekeyOrTitlekek;
else
status = Loader::ResultStatus::ErrorIncorrectKeyAreaKey;
return false;
}
if (section.raw.header.crypto_type == NCASectionCryptoType::BKTR) {
if (section.bktr.relocation.magic != Common::MakeMagic('B', 'K', 'T', 'R') ||
section.bktr.subsection.magic != Common::MakeMagic('B', 'K', 'T', 'R')) {
status = Loader::ResultStatus::ErrorBadBKTRHeader;
return false;
}
if (section.bktr.relocation.offset + section.bktr.relocation.size !=
section.bktr.subsection.offset) {
status = Loader::ResultStatus::ErrorBKTRSubsectionNotAfterRelocation;
return false;
}
const u64 size = MEDIA_OFFSET_MULTIPLIER * (entry.media_end_offset - entry.media_offset);
if (section.bktr.subsection.offset + section.bktr.subsection.size != size) {
status = Loader::ResultStatus::ErrorBKTRSubsectionNotAtEnd;
return false;
}
const u64 offset = section.romfs.ivfc.levels[IVFC_MAX_LEVEL - 1].offset;
RelocationBlock relocation_block{};
if (dec->ReadObject(&relocation_block, section.bktr.relocation.offset - offset) !=
sizeof(RelocationBlock)) {
status = Loader::ResultStatus::ErrorBadRelocationBlock;
return false;
}
SubsectionBlock subsection_block{};
if (dec->ReadObject(&subsection_block, section.bktr.subsection.offset - offset) !=
sizeof(RelocationBlock)) {
status = Loader::ResultStatus::ErrorBadSubsectionBlock;
return false;
}
std::vector<RelocationBucketRaw> relocation_buckets_raw(
(section.bktr.relocation.size - sizeof(RelocationBlock)) / sizeof(RelocationBucketRaw));
if (dec->ReadBytes(relocation_buckets_raw.data(),
section.bktr.relocation.size - sizeof(RelocationBlock),
section.bktr.relocation.offset + sizeof(RelocationBlock) - offset) !=
section.bktr.relocation.size - sizeof(RelocationBlock)) {
status = Loader::ResultStatus::ErrorBadRelocationBuckets;
return false;
}
std::vector<SubsectionBucketRaw> subsection_buckets_raw(
(section.bktr.subsection.size - sizeof(SubsectionBlock)) / sizeof(SubsectionBucketRaw));
if (dec->ReadBytes(subsection_buckets_raw.data(),
section.bktr.subsection.size - sizeof(SubsectionBlock),
section.bktr.subsection.offset + sizeof(SubsectionBlock) - offset) !=
section.bktr.subsection.size - sizeof(SubsectionBlock)) {
status = Loader::ResultStatus::ErrorBadSubsectionBuckets;
return false;
}
std::vector<RelocationBucket> relocation_buckets(relocation_buckets_raw.size());
std::ranges::transform(relocation_buckets_raw, relocation_buckets.begin(),
&ConvertRelocationBucketRaw);
std::vector<SubsectionBucket> subsection_buckets(subsection_buckets_raw.size());
std::ranges::transform(subsection_buckets_raw, subsection_buckets.begin(),
&ConvertSubsectionBucketRaw);
u32 ctr_low;
std::memcpy(&ctr_low, section.raw.section_ctr.data(), sizeof(ctr_low));
subsection_buckets.back().entries.push_back({section.bktr.relocation.offset, {0}, ctr_low});
subsection_buckets.back().entries.push_back({size, {0}, 0});
std::optional<Core::Crypto::Key128> key;
if (encrypted) {
if (has_rights_id) {
status = Loader::ResultStatus::Success;
key = GetTitlekey();
if (!key) {
status = Loader::ResultStatus::ErrorMissingTitlekey;
return false;
}
} else {
key = GetKeyAreaKey(NCASectionCryptoType::BKTR);
if (!key) {
status = Loader::ResultStatus::ErrorMissingKeyAreaKey;
return false;
}
}
}
if (bktr_base_romfs == nullptr) {
status = Loader::ResultStatus::ErrorMissingBKTRBaseRomFS;
return false;
}
auto bktr = std::make_shared<BKTR>(
bktr_base_romfs, std::make_shared<OffsetVfsFile>(file, romfs_size, base_offset),
relocation_block, relocation_buckets, subsection_block, subsection_buckets, encrypted,
encrypted ? *key : Core::Crypto::Key128{}, base_offset, bktr_base_ivfc_offset,
section.raw.section_ctr);
// BKTR applies to entire IVFC, so make an offset version to level 6
files.push_back(std::make_shared<OffsetVfsFile>(
bktr, romfs_size, section.romfs.ivfc.levels[IVFC_MAX_LEVEL - 1].offset));
} else {
files.push_back(std::move(dec));
}
romfs = files.back();
return true;
}
bool NCA::ReadPFS0Section(const NCASectionHeader& section, const NCASectionTableEntry& entry) {
const u64 offset = (static_cast<u64>(entry.media_offset) * MEDIA_OFFSET_MULTIPLIER) +
section.pfs0.pfs0_header_offset;
const u64 size = MEDIA_OFFSET_MULTIPLIER * (entry.media_end_offset - entry.media_offset);
auto dec = Decrypt(section, std::make_shared<OffsetVfsFile>(file, size, offset), offset);
if (dec != nullptr) {
auto npfs = std::make_shared<PartitionFilesystem>(std::move(dec));
if (npfs->GetStatus() == Loader::ResultStatus::Success) {
dirs.push_back(std::move(npfs));
if (IsDirectoryExeFS(dirs.back()))
exefs = dirs.back();
else if (IsDirectoryLogoPartition(dirs.back()))
logo = dirs.back();
} else {
if (has_rights_id)
status = Loader::ResultStatus::ErrorIncorrectTitlekeyOrTitlekek;
else
status = Loader::ResultStatus::ErrorIncorrectKeyAreaKey;
return false;
}
} else {
if (status != Loader::ResultStatus::Success)
return false;
if (has_rights_id)
status = Loader::ResultStatus::ErrorIncorrectTitlekeyOrTitlekek;
else
status = Loader::ResultStatus::ErrorIncorrectKeyAreaKey;
return false;
}
return true;
}
u8 NCA::GetCryptoRevision() const {
u8 master_key_id = header.crypto_type;
if (header.crypto_type_2 > master_key_id)
master_key_id = header.crypto_type_2;
if (master_key_id > 0)
--master_key_id;
return master_key_id;
}
std::optional<Core::Crypto::Key128> NCA::GetKeyAreaKey(NCASectionCryptoType type) const {
const auto master_key_id = GetCryptoRevision();
if (!keys.HasKey(Core::Crypto::S128KeyType::KeyArea, master_key_id, header.key_index)) {
return std::nullopt;
}
std::vector<u8> key_area(header.key_area.begin(), header.key_area.end());
Core::Crypto::AESCipher<Core::Crypto::Key128> cipher(
keys.GetKey(Core::Crypto::S128KeyType::KeyArea, master_key_id, header.key_index),
Core::Crypto::Mode::ECB);
cipher.Transcode(key_area.data(), key_area.size(), key_area.data(), Core::Crypto::Op::Decrypt);
Core::Crypto::Key128 out{};
if (type == NCASectionCryptoType::XTS) {
std::copy(key_area.begin(), key_area.begin() + 0x10, out.begin());
} else if (type == NCASectionCryptoType::CTR || type == NCASectionCryptoType::BKTR) {
std::copy(key_area.begin() + 0x20, key_area.begin() + 0x30, out.begin());
} else {
LOG_CRITICAL(Crypto, "Called GetKeyAreaKey on invalid NCASectionCryptoType type={:02X}",
type);
}
u128 out_128{};
std::memcpy(out_128.data(), out.data(), sizeof(u128));
LOG_TRACE(Crypto, "called with crypto_rev={:02X}, kak_index={:02X}, key={:016X}{:016X}",
master_key_id, header.key_index, out_128[1], out_128[0]);
return out;
}
std::optional<Core::Crypto::Key128> NCA::GetTitlekey() {
const auto master_key_id = GetCryptoRevision();
u128 rights_id{};
memcpy(rights_id.data(), header.rights_id.data(), 16);
if (rights_id == u128{}) {
status = Loader::ResultStatus::ErrorInvalidRightsID;
return std::nullopt;
}
auto titlekey = keys.GetKey(Core::Crypto::S128KeyType::Titlekey, rights_id[1], rights_id[0]);
if (titlekey == Core::Crypto::Key128{}) {
status = Loader::ResultStatus::ErrorMissingTitlekey;
return std::nullopt;
}
if (!keys.HasKey(Core::Crypto::S128KeyType::Titlekek, master_key_id)) {
status = Loader::ResultStatus::ErrorMissingTitlekek;
return std::nullopt;
}
Core::Crypto::AESCipher<Core::Crypto::Key128> cipher(
keys.GetKey(Core::Crypto::S128KeyType::Titlekek, master_key_id), Core::Crypto::Mode::ECB);
cipher.Transcode(titlekey.data(), titlekey.size(), titlekey.data(), Core::Crypto::Op::Decrypt);
return titlekey;
}
VirtualFile NCA::Decrypt(const NCASectionHeader& s_header, VirtualFile in, u64 starting_offset) {
if (!encrypted)
return in;
switch (s_header.raw.header.crypto_type) {
case NCASectionCryptoType::NONE:
LOG_TRACE(Crypto, "called with mode=NONE");
return in;
case NCASectionCryptoType::CTR:
// During normal BKTR decryption, this entire function is skipped. This is for the metadata,
// which uses the same CTR as usual.
case NCASectionCryptoType::BKTR:
LOG_TRACE(Crypto, "called with mode=CTR, starting_offset={:016X}", starting_offset);
{
std::optional<Core::Crypto::Key128> key;
if (has_rights_id) {
status = Loader::ResultStatus::Success;
key = GetTitlekey();
if (!key) {
if (status == Loader::ResultStatus::Success)
status = Loader::ResultStatus::ErrorMissingTitlekey;
return nullptr;
}
} else {
key = GetKeyAreaKey(NCASectionCryptoType::CTR);
if (!key) {
status = Loader::ResultStatus::ErrorMissingKeyAreaKey;
return nullptr;
}
}
auto out = std::make_shared<Core::Crypto::CTREncryptionLayer>(std::move(in), *key,
starting_offset);
Core::Crypto::CTREncryptionLayer::IVData iv{};
for (std::size_t i = 0; i < 8; ++i) {
iv[i] = s_header.raw.section_ctr[8 - i - 1];
}
out->SetIV(iv);
return std::static_pointer_cast<VfsFile>(out);
}
case NCASectionCryptoType::XTS:
// TODO(DarkLordZach): Find a test case for XTS-encrypted NCAs
default:
LOG_ERROR(Crypto, "called with unhandled crypto type={:02X}",
s_header.raw.header.crypto_type);
return nullptr;
}
}
Loader::ResultStatus NCA::GetStatus() const {
return status;
}
@ -579,21 +143,24 @@ VirtualDir NCA::GetParentDirectory() const {
}
NCAContentType NCA::GetType() const {
return header.content_type;
return static_cast<NCAContentType>(reader->GetContentType());
}
u64 NCA::GetTitleId() const {
if (is_update || status == Loader::ResultStatus::ErrorMissingBKTRBaseRomFS)
return header.title_id | 0x800;
return header.title_id;
if (is_update) {
return reader->GetProgramId() | 0x800;
}
return reader->GetProgramId();
}
std::array<u8, 16> NCA::GetRightsId() const {
return header.rights_id;
RightsId NCA::GetRightsId() const {
RightsId result;
reader->GetRightsId(result.data(), result.size());
return result;
}
u32 NCA::GetSDKVersion() const {
return header.sdk_version;
return reader->GetSdkAddonVersion();
}
bool NCA::IsUpdate() const {
@ -612,10 +179,6 @@ VirtualFile NCA::GetBaseFile() const {
return file;
}
u64 NCA::GetBaseIVFCOffset() const {
return ivfc_offset;
}
VirtualDir NCA::GetLogoPartition() const {
return logo;
}

View File

@ -21,7 +21,7 @@ enum class ResultStatus : u16;
namespace FileSys {
union NCASectionHeader;
class NcaReader;
/// Describes the type of content within an NCA archive.
enum class NCAContentType : u8 {
@ -45,41 +45,7 @@ enum class NCAContentType : u8 {
PublicData = 5,
};
enum class NCASectionCryptoType : u8 {
NONE = 1,
XTS = 2,
CTR = 3,
BKTR = 4,
};
struct NCASectionTableEntry {
u32_le media_offset;
u32_le media_end_offset;
INSERT_PADDING_BYTES(0x8);
};
static_assert(sizeof(NCASectionTableEntry) == 0x10, "NCASectionTableEntry has incorrect size.");
struct NCAHeader {
std::array<u8, 0x100> rsa_signature_1;
std::array<u8, 0x100> rsa_signature_2;
u32_le magic;
u8 is_system;
NCAContentType content_type;
u8 crypto_type;
u8 key_index;
u64_le size;
u64_le title_id;
INSERT_PADDING_BYTES(0x4);
u32_le sdk_version;
u8 crypto_type_2;
INSERT_PADDING_BYTES(15);
std::array<u8, 0x10> rights_id;
std::array<NCASectionTableEntry, 0x4> section_tables;
std::array<std::array<u8, 0x20>, 0x4> hash_tables;
std::array<u8, 0x40> key_area;
INSERT_PADDING_BYTES(0xC0);
};
static_assert(sizeof(NCAHeader) == 0x400, "NCAHeader has incorrect size.");
using RightsId = std::array<u8, 0x10>;
inline bool IsDirectoryExeFS(const VirtualDir& pfs) {
// According to switchbrew, an exefs must only contain these two files:
@ -97,8 +63,7 @@ inline bool IsDirectoryLogoPartition(const VirtualDir& pfs) {
// After construction, use GetStatus to determine if the file is valid and ready to be used.
class NCA : public ReadOnlyVfsDirectory {
public:
explicit NCA(VirtualFile file, VirtualFile bktr_base_romfs = nullptr,
u64 bktr_base_ivfc_offset = 0);
explicit NCA(VirtualFile file, const NCA* base_nca = nullptr);
~NCA() override;
Loader::ResultStatus GetStatus() const;
@ -110,7 +75,7 @@ public:
NCAContentType GetType() const;
u64 GetTitleId() const;
std::array<u8, 0x10> GetRightsId() const;
RightsId GetRightsId() const;
u32 GetSDKVersion() const;
bool IsUpdate() const;
@ -119,26 +84,9 @@ public:
VirtualFile GetBaseFile() const;
// Returns the base ivfc offset used in BKTR patching.
u64 GetBaseIVFCOffset() const;
VirtualDir GetLogoPartition() const;
private:
bool CheckSupportedNCA(const NCAHeader& header);
bool HandlePotentialHeaderDecryption();
std::vector<NCASectionHeader> ReadSectionHeaders() const;
bool ReadSections(const std::vector<NCASectionHeader>& sections, u64 bktr_base_ivfc_offset);
bool ReadRomFSSection(const NCASectionHeader& section, const NCASectionTableEntry& entry,
u64 bktr_base_ivfc_offset);
bool ReadPFS0Section(const NCASectionHeader& section, const NCASectionTableEntry& entry);
u8 GetCryptoRevision() const;
std::optional<Core::Crypto::Key128> GetKeyAreaKey(NCASectionCryptoType type) const;
std::optional<Core::Crypto::Key128> GetTitlekey();
VirtualFile Decrypt(const NCASectionHeader& header, VirtualFile in, u64 starting_offset);
std::vector<VirtualDir> dirs;
std::vector<VirtualFile> files;
@ -146,11 +94,6 @@ private:
VirtualDir exefs = nullptr;
VirtualDir logo = nullptr;
VirtualFile file;
VirtualFile bktr_base_romfs;
u64 ivfc_offset = 0;
NCAHeader header{};
bool has_rights_id{};
Loader::ResultStatus status{};
@ -158,6 +101,7 @@ private:
bool is_update = false;
Core::Crypto::KeyManager& keys;
std::shared_ptr<NcaReader> reader;
};
} // namespace FileSys

View File

@ -17,4 +17,74 @@ constexpr Result ERROR_INVALID_ARGUMENT{ErrorModule::FS, 6001};
constexpr Result ERROR_INVALID_OFFSET{ErrorModule::FS, 6061};
constexpr Result ERROR_INVALID_SIZE{ErrorModule::FS, 6062};
constexpr Result ResultUnsupportedSdkVersion{ErrorModule::FS, 50};
constexpr Result ResultPartitionNotFound{ErrorModule::FS, 1001};
constexpr Result ResultUnsupportedVersion{ErrorModule::FS, 3002};
constexpr Result ResultOutOfRange{ErrorModule::FS, 3005};
constexpr Result ResultAllocationMemoryFailedInFileSystemBuddyHeapA{ErrorModule::FS, 3294};
constexpr Result ResultAllocationMemoryFailedInNcaFileSystemDriverI{ErrorModule::FS, 3341};
constexpr Result ResultAllocationMemoryFailedInNcaReaderA{ErrorModule::FS, 3363};
constexpr Result ResultAllocationMemoryFailedInAesCtrCounterExtendedStorageA{ErrorModule::FS, 3399};
constexpr Result ResultAllocationMemoryFailedInIntegrityRomFsStorageA{ErrorModule::FS, 3412};
constexpr Result ResultAllocationMemoryFailedMakeUnique{ErrorModule::FS, 3422};
constexpr Result ResultAllocationMemoryFailedAllocateShared{ErrorModule::FS, 3423};
constexpr Result ResultInvalidAesCtrCounterExtendedEntryOffset{ErrorModule::FS, 4012};
constexpr Result ResultIndirectStorageCorrupted{ErrorModule::FS, 4021};
constexpr Result ResultInvalidIndirectEntryOffset{ErrorModule::FS, 4022};
constexpr Result ResultInvalidIndirectEntryStorageIndex{ErrorModule::FS, 4023};
constexpr Result ResultInvalidIndirectStorageSize{ErrorModule::FS, 4024};
constexpr Result ResultInvalidBucketTreeSignature{ErrorModule::FS, 4032};
constexpr Result ResultInvalidBucketTreeEntryCount{ErrorModule::FS, 4033};
constexpr Result ResultInvalidBucketTreeNodeEntryCount{ErrorModule::FS, 4034};
constexpr Result ResultInvalidBucketTreeNodeOffset{ErrorModule::FS, 4035};
constexpr Result ResultInvalidBucketTreeEntryOffset{ErrorModule::FS, 4036};
constexpr Result ResultInvalidBucketTreeEntrySetOffset{ErrorModule::FS, 4037};
constexpr Result ResultInvalidBucketTreeNodeIndex{ErrorModule::FS, 4038};
constexpr Result ResultInvalidBucketTreeVirtualOffset{ErrorModule::FS, 4039};
constexpr Result ResultRomNcaInvalidPatchMetaDataHashType{ErrorModule::FS, 4084};
constexpr Result ResultRomNcaInvalidIntegrityLayerInfoOffset{ErrorModule::FS, 4085};
constexpr Result ResultRomNcaInvalidPatchMetaDataHashDataSize{ErrorModule::FS, 4086};
constexpr Result ResultRomNcaInvalidPatchMetaDataHashDataOffset{ErrorModule::FS, 4087};
constexpr Result ResultRomNcaInvalidPatchMetaDataHashDataHash{ErrorModule::FS, 4088};
constexpr Result ResultRomNcaInvalidSparseMetaDataHashType{ErrorModule::FS, 4089};
constexpr Result ResultRomNcaInvalidSparseMetaDataHashDataSize{ErrorModule::FS, 4090};
constexpr Result ResultRomNcaInvalidSparseMetaDataHashDataOffset{ErrorModule::FS, 4091};
constexpr Result ResultRomNcaInvalidSparseMetaDataHashDataHash{ErrorModule::FS, 4091};
constexpr Result ResultNcaBaseStorageOutOfRangeB{ErrorModule::FS, 4509};
constexpr Result ResultNcaBaseStorageOutOfRangeC{ErrorModule::FS, 4510};
constexpr Result ResultNcaBaseStorageOutOfRangeD{ErrorModule::FS, 4511};
constexpr Result ResultInvalidNcaSignature{ErrorModule::FS, 4517};
constexpr Result ResultNcaFsHeaderHashVerificationFailed{ErrorModule::FS, 4520};
constexpr Result ResultInvalidNcaKeyIndex{ErrorModule::FS, 4521};
constexpr Result ResultInvalidNcaFsHeaderHashType{ErrorModule::FS, 4522};
constexpr Result ResultInvalidNcaFsHeaderEncryptionType{ErrorModule::FS, 4523};
constexpr Result ResultInvalidNcaPatchInfoIndirectSize{ErrorModule::FS, 4524};
constexpr Result ResultInvalidNcaPatchInfoAesCtrExSize{ErrorModule::FS, 4525};
constexpr Result ResultInvalidNcaPatchInfoAesCtrExOffset{ErrorModule::FS, 4526};
constexpr Result ResultInvalidNcaHeader{ErrorModule::FS, 4528};
constexpr Result ResultInvalidNcaFsHeader{ErrorModule::FS, 4529};
constexpr Result ResultNcaBaseStorageOutOfRangeE{ErrorModule::FS, 4530};
constexpr Result ResultInvalidHierarchicalSha256BlockSize{ErrorModule::FS, 4532};
constexpr Result ResultInvalidHierarchicalSha256LayerCount{ErrorModule::FS, 4533};
constexpr Result ResultHierarchicalSha256BaseStorageTooLarge{ErrorModule::FS, 4534};
constexpr Result ResultHierarchicalSha256HashVerificationFailed{ErrorModule::FS, 4535};
constexpr Result ResultInvalidNcaHierarchicalIntegrityVerificationLayerCount{ErrorModule::FS, 4541};
constexpr Result ResultInvalidNcaIndirectStorageOutOfRange{ErrorModule::FS, 4542};
constexpr Result ResultInvalidNcaHeader1SignatureKeyGeneration{ErrorModule::FS, 4543};
constexpr Result ResultInvalidCompressedStorageSize{ErrorModule::FS, 4547};
constexpr Result ResultInvalidNcaMetaDataHashDataSize{ErrorModule::FS, 4548};
constexpr Result ResultInvalidNcaMetaDataHashDataHash{ErrorModule::FS, 4549};
constexpr Result ResultUnexpectedInCompressedStorageA{ErrorModule::FS, 5324};
constexpr Result ResultUnexpectedInCompressedStorageB{ErrorModule::FS, 5325};
constexpr Result ResultUnexpectedInCompressedStorageC{ErrorModule::FS, 5326};
constexpr Result ResultUnexpectedInCompressedStorageD{ErrorModule::FS, 5327};
constexpr Result ResultInvalidArgument{ErrorModule::FS, 6001};
constexpr Result ResultInvalidOffset{ErrorModule::FS, 6061};
constexpr Result ResultInvalidSize{ErrorModule::FS, 6062};
constexpr Result ResultNullptrArgument{ErrorModule::FS, 6063};
constexpr Result ResultUnsupportedSetSizeForIndirectStorage{ErrorModule::FS, 6325};
constexpr Result ResultUnsupportedWriteForCompressedStorage{ErrorModule::FS, 6387};
constexpr Result ResultUnsupportedOperateRangeForCompressedStorage{ErrorModule::FS, 6388};
constexpr Result ResultBufferAllocationFailed{ErrorModule::FS, 6705};
} // namespace FileSys

View File

@ -0,0 +1,58 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/overflow.h"
#include "core/file_sys/errors.h"
#include "core/file_sys/vfs.h"
namespace FileSys {
class IStorage : public VfsFile {
public:
virtual std::string GetName() const override {
return {};
}
virtual VirtualDir GetContainingDirectory() const override {
return {};
}
virtual bool IsWritable() const override {
return true;
}
virtual bool IsReadable() const override {
return true;
}
virtual bool Resize(size_t size) override {
return false;
}
virtual bool Rename(std::string_view name) override {
return false;
}
static inline Result CheckAccessRange(s64 offset, s64 size, s64 total_size) {
R_UNLESS(offset >= 0, ResultInvalidOffset);
R_UNLESS(size >= 0, ResultInvalidSize);
R_UNLESS(Common::WrappingAdd(offset, size) >= offset, ResultOutOfRange);
R_UNLESS(offset + size <= total_size, ResultOutOfRange);
R_SUCCEED();
}
};
class IReadOnlyStorage : public IStorage {
public:
virtual bool IsWritable() const override {
return false;
}
virtual size_t Write(const u8* buffer, size_t size, size_t offset) override {
return 0;
}
};
} // namespace FileSys

View File

@ -0,0 +1,46 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/common_funcs.h"
namespace FileSys {
struct Int64 {
u32 low;
u32 high;
constexpr void Set(s64 v) {
this->low = static_cast<u32>((v & static_cast<u64>(0x00000000FFFFFFFFULL)) >> 0);
this->high = static_cast<u32>((v & static_cast<u64>(0xFFFFFFFF00000000ULL)) >> 32);
}
constexpr s64 Get() const {
return (static_cast<s64>(this->high) << 32) | (static_cast<s64>(this->low));
}
constexpr Int64& operator=(s64 v) {
this->Set(v);
return *this;
}
constexpr operator s64() const {
return this->Get();
}
};
struct HashSalt {
static constexpr size_t Size = 32;
std::array<u8, Size> value;
};
static_assert(std::is_trivial_v<HashSalt>);
static_assert(sizeof(HashSalt) == HashSalt::Size);
constexpr inline size_t IntegrityMinLayerCount = 2;
constexpr inline size_t IntegrityMaxLayerCount = 7;
constexpr inline size_t IntegrityLayerCountSave = 5;
constexpr inline size_t IntegrityLayerCountSaveDataMeta = 4;
} // namespace FileSys

View File

@ -0,0 +1,251 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/file_sys/fssystem/fssystem_aes_ctr_counter_extended_storage.h"
#include "core/file_sys/fssystem/fssystem_aes_ctr_storage.h"
#include "core/file_sys/fssystem/fssystem_nca_header.h"
#include "core/file_sys/vfs_offset.h"
namespace FileSys {
namespace {
class SoftwareDecryptor final : public AesCtrCounterExtendedStorage::IDecryptor {
public:
virtual void Decrypt(
u8* buf, size_t buf_size, const std::array<u8, AesCtrCounterExtendedStorage::KeySize>& key,
const std::array<u8, AesCtrCounterExtendedStorage::IvSize>& iv) override final;
};
} // namespace
Result AesCtrCounterExtendedStorage::CreateSoftwareDecryptor(std::unique_ptr<IDecryptor>* out) {
std::unique_ptr<IDecryptor> decryptor = std::make_unique<SoftwareDecryptor>();
R_UNLESS(decryptor != nullptr, ResultAllocationMemoryFailedInAesCtrCounterExtendedStorageA);
*out = std::move(decryptor);
R_SUCCEED();
}
Result AesCtrCounterExtendedStorage::Initialize(const void* key, size_t key_size, u32 secure_value,
VirtualFile data_storage,
VirtualFile table_storage) {
// Read and verify the bucket tree header.
BucketTree::Header header;
table_storage->ReadObject(std::addressof(header), 0);
R_TRY(header.Verify());
// Determine extents.
const auto node_storage_size = QueryNodeStorageSize(header.entry_count);
const auto entry_storage_size = QueryEntryStorageSize(header.entry_count);
const auto node_storage_offset = QueryHeaderStorageSize();
const auto entry_storage_offset = node_storage_offset + node_storage_size;
// Create a software decryptor.
std::unique_ptr<IDecryptor> sw_decryptor;
R_TRY(CreateSoftwareDecryptor(std::addressof(sw_decryptor)));
// Initialize.
R_RETURN(this->Initialize(
key, key_size, secure_value, 0, data_storage,
std::make_shared<OffsetVfsFile>(table_storage, node_storage_size, node_storage_offset),
std::make_shared<OffsetVfsFile>(table_storage, entry_storage_size, entry_storage_offset),
header.entry_count, std::move(sw_decryptor)));
}
Result AesCtrCounterExtendedStorage::Initialize(const void* key, size_t key_size, u32 secure_value,
s64 counter_offset, VirtualFile data_storage,
VirtualFile node_storage, VirtualFile entry_storage,
s32 entry_count,
std::unique_ptr<IDecryptor>&& decryptor) {
// Validate preconditions.
ASSERT(key != nullptr);
ASSERT(key_size == KeySize);
ASSERT(counter_offset >= 0);
ASSERT(decryptor != nullptr);
// Initialize the bucket tree table.
if (entry_count > 0) {
R_TRY(
m_table.Initialize(node_storage, entry_storage, NodeSize, sizeof(Entry), entry_count));
} else {
m_table.Initialize(NodeSize, 0);
}
// Set members.
m_data_storage = data_storage;
std::memcpy(m_key.data(), key, key_size);
m_secure_value = secure_value;
m_counter_offset = counter_offset;
m_decryptor = std::move(decryptor);
R_SUCCEED();
}
void AesCtrCounterExtendedStorage::Finalize() {
if (this->IsInitialized()) {
m_table.Finalize();
m_data_storage = VirtualFile();
}
}
Result AesCtrCounterExtendedStorage::GetEntryList(Entry* out_entries, s32* out_entry_count,
s32 entry_count, s64 offset, s64 size) {
// Validate pre-conditions.
ASSERT(offset >= 0);
ASSERT(size >= 0);
ASSERT(this->IsInitialized());
// Clear the out count.
R_UNLESS(out_entry_count != nullptr, ResultNullptrArgument);
*out_entry_count = 0;
// Succeed if there's no range.
R_SUCCEED_IF(size == 0);
// If we have an output array, we need it to be non-null.
R_UNLESS(out_entries != nullptr || entry_count == 0, ResultNullptrArgument);
// Check that our range is valid.
BucketTree::Offsets table_offsets;
R_TRY(m_table.GetOffsets(std::addressof(table_offsets)));
R_UNLESS(table_offsets.IsInclude(offset, size), ResultOutOfRange);
// Find the offset in our tree.
BucketTree::Visitor visitor;
R_TRY(m_table.Find(std::addressof(visitor), offset));
{
const auto entry_offset = visitor.Get<Entry>()->GetOffset();
R_UNLESS(0 <= entry_offset && table_offsets.IsInclude(entry_offset),
ResultInvalidAesCtrCounterExtendedEntryOffset);
}
// Prepare to loop over entries.
const auto end_offset = offset + static_cast<s64>(size);
s32 count = 0;
auto cur_entry = *visitor.Get<Entry>();
while (cur_entry.GetOffset() < end_offset) {
// Try to write the entry to the out list.
if (entry_count != 0) {
if (count >= entry_count) {
break;
}
std::memcpy(out_entries + count, std::addressof(cur_entry), sizeof(Entry));
}
count++;
// Advance.
if (visitor.CanMoveNext()) {
R_TRY(visitor.MoveNext());
cur_entry = *visitor.Get<Entry>();
} else {
break;
}
}
// Write the output count.
*out_entry_count = count;
R_SUCCEED();
}
size_t AesCtrCounterExtendedStorage::Read(u8* buffer, size_t size, size_t offset) const {
// Validate preconditions.
ASSERT(this->IsInitialized());
// Allow zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
ASSERT(Common::IsAligned(offset, BlockSize));
ASSERT(Common::IsAligned(size, BlockSize));
BucketTree::Offsets table_offsets;
ASSERT(R_SUCCEEDED(m_table.GetOffsets(std::addressof(table_offsets))));
ASSERT(table_offsets.IsInclude(offset, size));
// Read the data.
m_data_storage->Read(buffer, size, offset);
// Find the offset in our tree.
BucketTree::Visitor visitor;
ASSERT(R_SUCCEEDED(m_table.Find(std::addressof(visitor), offset)));
{
const auto entry_offset = visitor.Get<Entry>()->GetOffset();
ASSERT(Common::IsAligned(entry_offset, BlockSize));
ASSERT(0 <= entry_offset && table_offsets.IsInclude(entry_offset));
}
// Prepare to read in chunks.
u8* cur_data = static_cast<u8*>(buffer);
auto cur_offset = offset;
const auto end_offset = offset + static_cast<s64>(size);
while (cur_offset < end_offset) {
// Get the current entry.
const auto cur_entry = *visitor.Get<Entry>();
// Get and validate the entry's offset.
const auto cur_entry_offset = cur_entry.GetOffset();
ASSERT(static_cast<size_t>(cur_entry_offset) <= cur_offset);
// Get and validate the next entry offset.
s64 next_entry_offset;
if (visitor.CanMoveNext()) {
ASSERT(R_SUCCEEDED(visitor.MoveNext()));
next_entry_offset = visitor.Get<Entry>()->GetOffset();
ASSERT(table_offsets.IsInclude(next_entry_offset));
} else {
next_entry_offset = table_offsets.end_offset;
}
ASSERT(Common::IsAligned(next_entry_offset, BlockSize));
ASSERT(cur_offset < static_cast<size_t>(next_entry_offset));
// Get the offset of the entry in the data we read.
const auto data_offset = cur_offset - cur_entry_offset;
const auto data_size = (next_entry_offset - cur_entry_offset) - data_offset;
ASSERT(data_size > 0);
// Determine how much is left.
const auto remaining_size = end_offset - cur_offset;
const auto cur_size = static_cast<size_t>(std::min(remaining_size, data_size));
ASSERT(cur_size <= size);
// If necessary, perform decryption.
if (cur_entry.encryption_value == Entry::Encryption::Encrypted) {
// Make the CTR for the data we're decrypting.
const auto counter_offset = m_counter_offset + cur_entry_offset + data_offset;
NcaAesCtrUpperIv upper_iv = {
.part = {.generation = static_cast<u32>(cur_entry.generation),
.secure_value = m_secure_value}};
std::array<u8, IvSize> iv;
AesCtrStorage::MakeIv(iv.data(), IvSize, upper_iv.value, counter_offset);
// Decrypt.
m_decryptor->Decrypt(cur_data, cur_size, m_key, iv);
}
// Advance.
cur_data += cur_size;
cur_offset += cur_size;
}
return size;
}
void SoftwareDecryptor::Decrypt(u8* buf, size_t buf_size,
const std::array<u8, AesCtrCounterExtendedStorage::KeySize>& key,
const std::array<u8, AesCtrCounterExtendedStorage::IvSize>& iv) {
Core::Crypto::AESCipher<Core::Crypto::Key128, AesCtrCounterExtendedStorage::KeySize> cipher(
key, Core::Crypto::Mode::CTR);
cipher.SetIV(iv);
cipher.Transcode(buf, buf_size, buf, Core::Crypto::Op::Decrypt);
}
} // namespace FileSys

View File

@ -0,0 +1,114 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <optional>
#include "common/literals.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree.h"
namespace FileSys {
using namespace Common::Literals;
class AesCtrCounterExtendedStorage : public IReadOnlyStorage {
YUZU_NON_COPYABLE(AesCtrCounterExtendedStorage);
YUZU_NON_MOVEABLE(AesCtrCounterExtendedStorage);
public:
static constexpr size_t BlockSize = 0x10;
static constexpr size_t KeySize = 0x10;
static constexpr size_t IvSize = 0x10;
static constexpr size_t NodeSize = 16_KiB;
class IDecryptor {
public:
virtual ~IDecryptor() {}
virtual void Decrypt(u8* buf, size_t buf_size, const std::array<u8, KeySize>& key,
const std::array<u8, IvSize>& iv) = 0;
};
struct Entry {
enum class Encryption : u8 {
Encrypted = 0,
NotEncrypted = 1,
};
std::array<u8, sizeof(s64)> offset;
Encryption encryption_value;
std::array<u8, 3> reserved;
s32 generation;
void SetOffset(s64 value) {
std::memcpy(this->offset.data(), std::addressof(value), sizeof(s64));
}
s64 GetOffset() const {
s64 value;
std::memcpy(std::addressof(value), this->offset.data(), sizeof(s64));
return value;
}
};
static_assert(sizeof(Entry) == 0x10);
static_assert(alignof(Entry) == 4);
static_assert(std::is_trivial_v<Entry>);
public:
static constexpr s64 QueryHeaderStorageSize() {
return BucketTree::QueryHeaderStorageSize();
}
static constexpr s64 QueryNodeStorageSize(s32 entry_count) {
return BucketTree::QueryNodeStorageSize(NodeSize, sizeof(Entry), entry_count);
}
static constexpr s64 QueryEntryStorageSize(s32 entry_count) {
return BucketTree::QueryEntryStorageSize(NodeSize, sizeof(Entry), entry_count);
}
static Result CreateSoftwareDecryptor(std::unique_ptr<IDecryptor>* out);
public:
AesCtrCounterExtendedStorage()
: m_table(), m_data_storage(), m_secure_value(), m_counter_offset(), m_decryptor() {}
virtual ~AesCtrCounterExtendedStorage() {
this->Finalize();
}
Result Initialize(const void* key, size_t key_size, u32 secure_value, s64 counter_offset,
VirtualFile data_storage, VirtualFile node_storage, VirtualFile entry_storage,
s32 entry_count, std::unique_ptr<IDecryptor>&& decryptor);
void Finalize();
bool IsInitialized() const {
return m_table.IsInitialized();
}
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override;
virtual size_t GetSize() const override {
BucketTree::Offsets offsets;
ASSERT(R_SUCCEEDED(m_table.GetOffsets(std::addressof(offsets))));
return offsets.end_offset;
}
Result GetEntryList(Entry* out_entries, s32* out_entry_count, s32 entry_count, s64 offset,
s64 size);
private:
Result Initialize(const void* key, size_t key_size, u32 secure_value, VirtualFile data_storage,
VirtualFile table_storage);
private:
mutable BucketTree m_table;
VirtualFile m_data_storage;
std::array<u8, KeySize> m_key;
u32 m_secure_value;
s64 m_counter_offset;
std::unique_ptr<IDecryptor> m_decryptor;
};
} // namespace FileSys

View File

@ -0,0 +1,129 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/alignment.h"
#include "common/swap.h"
#include "core/file_sys/fssystem/fssystem_aes_ctr_storage.h"
#include "core/file_sys/fssystem/fssystem_pooled_buffer.h"
#include "core/file_sys/fssystem/fssystem_utility.h"
namespace FileSys {
void AesCtrStorage::MakeIv(void* dst, size_t dst_size, u64 upper, s64 offset) {
ASSERT(dst != nullptr);
ASSERT(dst_size == IvSize);
ASSERT(offset >= 0);
const uintptr_t out_addr = reinterpret_cast<uintptr_t>(dst);
*reinterpret_cast<u64_be*>(out_addr + 0) = upper;
*reinterpret_cast<s64_be*>(out_addr + sizeof(u64)) = static_cast<s64>(offset / BlockSize);
}
AesCtrStorage::AesCtrStorage(VirtualFile base, const void* key, size_t key_size, const void* iv,
size_t iv_size)
: m_base_storage(std::move(base)) {
ASSERT(m_base_storage != nullptr);
ASSERT(key != nullptr);
ASSERT(iv != nullptr);
ASSERT(key_size == KeySize);
ASSERT(iv_size == IvSize);
std::memcpy(m_key.data(), key, KeySize);
std::memcpy(m_iv.data(), iv, IvSize);
m_cipher.emplace(m_key, Core::Crypto::Mode::CTR);
}
size_t AesCtrStorage::Read(u8* buffer, size_t size, size_t offset) const {
// Allow zero-size reads.
if (size == 0) {
return size;
}
// Ensure buffer is valid.
ASSERT(buffer != nullptr);
// We can only read at block aligned offsets.
ASSERT(Common::IsAligned(offset, BlockSize));
ASSERT(Common::IsAligned(size, BlockSize));
// Read the data.
m_base_storage->Read(buffer, size, offset);
// Setup the counter.
std::array<u8, IvSize> ctr;
std::memcpy(ctr.data(), m_iv.data(), IvSize);
AddCounter(ctr.data(), IvSize, offset / BlockSize);
// Decrypt.
m_cipher->SetIV(ctr);
m_cipher->Transcode(buffer, size, buffer, Core::Crypto::Op::Decrypt);
return size;
}
size_t AesCtrStorage::Write(const u8* buffer, size_t size, size_t offset) {
// Allow zero-size writes.
if (size == 0) {
return size;
}
// Ensure buffer is valid.
ASSERT(buffer != nullptr);
// We can only write at block aligned offsets.
ASSERT(Common::IsAligned(offset, BlockSize));
ASSERT(Common::IsAligned(size, BlockSize));
// Get a pooled buffer.
PooledBuffer pooled_buffer;
const bool use_work_buffer = true;
if (use_work_buffer) {
pooled_buffer.Allocate(size, BlockSize);
}
// Setup the counter.
std::array<u8, IvSize> ctr;
std::memcpy(ctr.data(), m_iv.data(), IvSize);
AddCounter(ctr.data(), IvSize, offset / BlockSize);
// Loop until all data is written.
size_t remaining = size;
s64 cur_offset = 0;
while (remaining > 0) {
// Determine data we're writing and where.
const size_t write_size =
use_work_buffer ? std::min(pooled_buffer.GetSize(), remaining) : remaining;
void* write_buf;
if (use_work_buffer) {
write_buf = pooled_buffer.GetBuffer();
} else {
write_buf = const_cast<u8*>(buffer);
}
// Encrypt the data.
m_cipher->SetIV(ctr);
m_cipher->Transcode(buffer, write_size, reinterpret_cast<u8*>(write_buf),
Core::Crypto::Op::Encrypt);
// Write the encrypted data.
m_base_storage->Write(reinterpret_cast<u8*>(write_buf), write_size, offset + cur_offset);
// Advance.
cur_offset += write_size;
remaining -= write_size;
if (remaining > 0) {
AddCounter(ctr.data(), IvSize, write_size / BlockSize);
}
}
return size;
}
size_t AesCtrStorage::GetSize() const {
return m_base_storage->GetSize();
}
} // namespace FileSys

View File

@ -0,0 +1,43 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <optional>
#include "core/crypto/aes_util.h"
#include "core/crypto/key_manager.h"
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/vfs.h"
namespace FileSys {
class AesCtrStorage : public IStorage {
YUZU_NON_COPYABLE(AesCtrStorage);
YUZU_NON_MOVEABLE(AesCtrStorage);
public:
static constexpr size_t BlockSize = 0x10;
static constexpr size_t KeySize = 0x10;
static constexpr size_t IvSize = 0x10;
public:
static void MakeIv(void* dst, size_t dst_size, u64 upper, s64 offset);
public:
AesCtrStorage(VirtualFile base, const void* key, size_t key_size, const void* iv,
size_t iv_size);
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override;
virtual size_t Write(const u8* buffer, size_t size, size_t offset) override;
virtual size_t GetSize() const override;
private:
VirtualFile m_base_storage;
std::array<u8, KeySize> m_key;
std::array<u8, IvSize> m_iv;
mutable std::optional<Core::Crypto::AESCipher<Core::Crypto::Key128>> m_cipher;
};
} // namespace FileSys

View File

@ -0,0 +1,112 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/alignment.h"
#include "common/swap.h"
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fssystem_aes_xts_storage.h"
#include "core/file_sys/fssystem/fssystem_pooled_buffer.h"
#include "core/file_sys/fssystem/fssystem_utility.h"
namespace FileSys {
void AesXtsStorage::MakeAesXtsIv(void* dst, size_t dst_size, s64 offset, size_t block_size) {
ASSERT(dst != nullptr);
ASSERT(dst_size == IvSize);
ASSERT(offset >= 0);
const uintptr_t out_addr = reinterpret_cast<uintptr_t>(dst);
*reinterpret_cast<s64_be*>(out_addr + sizeof(s64)) = offset / block_size;
}
AesXtsStorage::AesXtsStorage(VirtualFile base, const void* key1, const void* key2, size_t key_size,
const void* iv, size_t iv_size, size_t block_size)
: m_base_storage(std::move(base)), m_block_size(block_size), m_mutex() {
ASSERT(m_base_storage != nullptr);
ASSERT(key1 != nullptr);
ASSERT(key2 != nullptr);
ASSERT(iv != nullptr);
ASSERT(key_size == KeySize);
ASSERT(iv_size == IvSize);
ASSERT(Common::IsAligned(m_block_size, AesBlockSize));
std::memcpy(m_key.data() + 0, key1, KeySize);
std::memcpy(m_key.data() + 0x10, key2, KeySize);
std::memcpy(m_iv.data(), iv, IvSize);
m_cipher.emplace(m_key, Core::Crypto::Mode::XTS);
}
size_t AesXtsStorage::Read(u8* buffer, size_t size, size_t offset) const {
// Allow zero-size reads.
if (size == 0) {
return size;
}
// Ensure buffer is valid.
ASSERT(buffer != nullptr);
// We can only read at block aligned offsets.
ASSERT(Common::IsAligned(offset, AesBlockSize));
ASSERT(Common::IsAligned(size, AesBlockSize));
// Read the data.
m_base_storage->Read(buffer, size, offset);
// Setup the counter.
std::array<u8, IvSize> ctr;
std::memcpy(ctr.data(), m_iv.data(), IvSize);
AddCounter(ctr.data(), IvSize, offset / m_block_size);
// Handle any unaligned data before the start.
size_t processed_size = 0;
if ((offset % m_block_size) != 0) {
// Determine the size of the pre-data read.
const size_t skip_size =
static_cast<size_t>(offset - Common::AlignDown(offset, m_block_size));
const size_t data_size = std::min(size, m_block_size - skip_size);
// Decrypt into a pooled buffer.
{
PooledBuffer tmp_buf(m_block_size, m_block_size);
ASSERT(tmp_buf.GetSize() >= m_block_size);
std::memset(tmp_buf.GetBuffer(), 0, skip_size);
std::memcpy(tmp_buf.GetBuffer() + skip_size, buffer, data_size);
m_cipher->SetIV(ctr);
m_cipher->Transcode(tmp_buf.GetBuffer(), m_block_size, tmp_buf.GetBuffer(),
Core::Crypto::Op::Decrypt);
std::memcpy(buffer, tmp_buf.GetBuffer() + skip_size, data_size);
}
AddCounter(ctr.data(), IvSize, 1);
processed_size += data_size;
ASSERT(processed_size == std::min(size, m_block_size - skip_size));
}
// Decrypt aligned chunks.
char* cur = reinterpret_cast<char*>(buffer) + processed_size;
size_t remaining = size - processed_size;
while (remaining > 0) {
const size_t cur_size = std::min(m_block_size, remaining);
m_cipher->SetIV(ctr);
m_cipher->Transcode(cur, cur_size, cur, Core::Crypto::Op::Decrypt);
remaining -= cur_size;
cur += cur_size;
AddCounter(ctr.data(), IvSize, 1);
}
return size;
}
size_t AesXtsStorage::GetSize() const {
return m_base_storage->GetSize();
}
} // namespace FileSys

View File

@ -0,0 +1,42 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <optional>
#include "core/crypto/aes_util.h"
#include "core/crypto/key_manager.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
namespace FileSys {
class AesXtsStorage : public IReadOnlyStorage {
YUZU_NON_COPYABLE(AesXtsStorage);
YUZU_NON_MOVEABLE(AesXtsStorage);
public:
static constexpr size_t AesBlockSize = 0x10;
static constexpr size_t KeySize = 0x20;
static constexpr size_t IvSize = 0x10;
public:
static void MakeAesXtsIv(void* dst, size_t dst_size, s64 offset, size_t block_size);
public:
AesXtsStorage(VirtualFile base, const void* key1, const void* key2, size_t key_size,
const void* iv, size_t iv_size, size_t block_size);
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override;
virtual size_t GetSize() const override;
private:
VirtualFile m_base_storage;
std::array<u8, KeySize> m_key;
std::array<u8, IvSize> m_iv;
const size_t m_block_size;
std::mutex m_mutex;
mutable std::optional<Core::Crypto::AESCipher<Core::Crypto::Key256>> m_cipher;
};
} // namespace FileSys

View File

@ -0,0 +1,146 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/alignment.h"
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/fssystem/fssystem_alignment_matching_storage_impl.h"
#include "core/file_sys/fssystem/fssystem_pooled_buffer.h"
namespace FileSys {
template <size_t DataAlign_, size_t BufferAlign_>
class AlignmentMatchingStorage : public IStorage {
YUZU_NON_COPYABLE(AlignmentMatchingStorage);
YUZU_NON_MOVEABLE(AlignmentMatchingStorage);
public:
static constexpr size_t DataAlign = DataAlign_;
static constexpr size_t BufferAlign = BufferAlign_;
static constexpr size_t DataAlignMax = 0x200;
static_assert(DataAlign <= DataAlignMax);
static_assert(Common::IsPowerOfTwo(DataAlign));
static_assert(Common::IsPowerOfTwo(BufferAlign));
private:
VirtualFile m_base_storage;
s64 m_base_storage_size;
public:
explicit AlignmentMatchingStorage(VirtualFile bs) : m_base_storage(std::move(bs)) {}
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override {
// Allocate a work buffer on stack.
alignas(DataAlignMax) std::array<char, DataAlign> work_buf;
// Succeed if zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
s64 bs_size = this->GetSize();
ASSERT(R_SUCCEEDED(IStorage::CheckAccessRange(offset, size, bs_size)));
return AlignmentMatchingStorageImpl::Read(m_base_storage, work_buf.data(), work_buf.size(),
DataAlign, BufferAlign, offset, buffer, size);
}
virtual size_t Write(const u8* buffer, size_t size, size_t offset) override {
// Allocate a work buffer on stack.
alignas(DataAlignMax) std::array<char, DataAlign> work_buf;
// Succeed if zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
s64 bs_size = this->GetSize();
ASSERT(R_SUCCEEDED(IStorage::CheckAccessRange(offset, size, bs_size)));
return AlignmentMatchingStorageImpl::Write(m_base_storage, work_buf.data(), work_buf.size(),
DataAlign, BufferAlign, offset, buffer, size);
}
virtual size_t GetSize() const override {
return m_base_storage->GetSize();
}
};
template <size_t BufferAlign_>
class AlignmentMatchingStoragePooledBuffer : public IStorage {
YUZU_NON_COPYABLE(AlignmentMatchingStoragePooledBuffer);
YUZU_NON_MOVEABLE(AlignmentMatchingStoragePooledBuffer);
public:
static constexpr size_t BufferAlign = BufferAlign_;
static_assert(Common::IsPowerOfTwo(BufferAlign));
private:
VirtualFile m_base_storage;
s64 m_base_storage_size;
size_t m_data_align;
public:
explicit AlignmentMatchingStoragePooledBuffer(VirtualFile bs, size_t da)
: m_base_storage(std::move(bs)), m_data_align(da) {
ASSERT(Common::IsPowerOfTwo(da));
}
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override {
// Succeed if zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
s64 bs_size = this->GetSize();
ASSERT(R_SUCCEEDED(IStorage::CheckAccessRange(offset, size, bs_size)));
// Allocate a pooled buffer.
PooledBuffer pooled_buffer;
pooled_buffer.AllocateParticularlyLarge(m_data_align, m_data_align);
return AlignmentMatchingStorageImpl::Read(m_base_storage, pooled_buffer.GetBuffer(),
pooled_buffer.GetSize(), m_data_align,
BufferAlign, offset, buffer, size);
}
virtual size_t Write(const u8* buffer, size_t size, size_t offset) override {
// Succeed if zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
s64 bs_size = this->GetSize();
ASSERT(R_SUCCEEDED(IStorage::CheckAccessRange(offset, size, bs_size)));
// Allocate a pooled buffer.
PooledBuffer pooled_buffer;
pooled_buffer.AllocateParticularlyLarge(m_data_align, m_data_align);
return AlignmentMatchingStorageImpl::Write(m_base_storage, pooled_buffer.GetBuffer(),
pooled_buffer.GetSize(), m_data_align,
BufferAlign, offset, buffer, size);
}
virtual size_t GetSize() const override {
return m_base_storage->GetSize();
}
};
} // namespace FileSys

View File

@ -0,0 +1,204 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/alignment.h"
#include "core/file_sys/fssystem/fssystem_alignment_matching_storage_impl.h"
namespace FileSys {
namespace {
template <typename T>
constexpr size_t GetRoundDownDifference(T x, size_t align) {
return static_cast<size_t>(x - Common::AlignDown(x, align));
}
template <typename T>
constexpr size_t GetRoundUpDifference(T x, size_t align) {
return static_cast<size_t>(Common::AlignUp(x, align) - x);
}
template <typename T>
size_t GetRoundUpDifference(T* x, size_t align) {
return GetRoundUpDifference(reinterpret_cast<uintptr_t>(x), align);
}
} // namespace
size_t AlignmentMatchingStorageImpl::Read(VirtualFile base_storage, char* work_buf,
size_t work_buf_size, size_t data_alignment,
size_t buffer_alignment, s64 offset, u8* buffer,
size_t size) {
// Check preconditions.
ASSERT(work_buf_size >= data_alignment);
// Succeed if zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
// Determine extents.
u8* aligned_core_buffer;
s64 core_offset;
size_t core_size;
size_t buffer_gap;
size_t offset_gap;
s64 covered_offset;
const size_t offset_round_up_difference = GetRoundUpDifference(offset, data_alignment);
if (Common::IsAligned(reinterpret_cast<uintptr_t>(buffer) + offset_round_up_difference,
buffer_alignment)) {
aligned_core_buffer = buffer + offset_round_up_difference;
core_offset = Common::AlignUp(offset, data_alignment);
core_size = (size < offset_round_up_difference)
? 0
: Common::AlignDown(size - offset_round_up_difference, data_alignment);
buffer_gap = 0;
offset_gap = 0;
covered_offset = core_size > 0 ? core_offset : offset;
} else {
const size_t buffer_round_up_difference = GetRoundUpDifference(buffer, buffer_alignment);
aligned_core_buffer = buffer + buffer_round_up_difference;
core_offset = Common::AlignDown(offset, data_alignment);
core_size = (size < buffer_round_up_difference)
? 0
: Common::AlignDown(size - buffer_round_up_difference, data_alignment);
buffer_gap = buffer_round_up_difference;
offset_gap = GetRoundDownDifference(offset, data_alignment);
covered_offset = offset;
}
// Read the core portion.
if (core_size > 0) {
base_storage->Read(aligned_core_buffer, core_size, core_offset);
if (offset_gap != 0 || buffer_gap != 0) {
std::memmove(aligned_core_buffer - buffer_gap, aligned_core_buffer + offset_gap,
core_size - offset_gap);
core_size -= offset_gap;
}
}
// Handle the head portion.
if (offset < covered_offset) {
const s64 head_offset = Common::AlignDown(offset, data_alignment);
const size_t head_size = static_cast<size_t>(covered_offset - offset);
ASSERT(GetRoundDownDifference(offset, data_alignment) + head_size <= work_buf_size);
base_storage->Read(reinterpret_cast<u8*>(work_buf), data_alignment, head_offset);
std::memcpy(buffer, work_buf + GetRoundDownDifference(offset, data_alignment), head_size);
}
// Handle the tail portion.
s64 tail_offset = covered_offset + core_size;
size_t remaining_tail_size = static_cast<size_t>((offset + size) - tail_offset);
while (remaining_tail_size > 0) {
const auto aligned_tail_offset = Common::AlignDown(tail_offset, data_alignment);
const auto cur_size =
std::min(static_cast<size_t>(aligned_tail_offset + data_alignment - tail_offset),
remaining_tail_size);
base_storage->Read(reinterpret_cast<u8*>(work_buf), data_alignment, aligned_tail_offset);
ASSERT((tail_offset - offset) + cur_size <= size);
ASSERT((tail_offset - aligned_tail_offset) + cur_size <= data_alignment);
std::memcpy(reinterpret_cast<char*>(buffer) + (tail_offset - offset),
work_buf + (tail_offset - aligned_tail_offset), cur_size);
remaining_tail_size -= cur_size;
tail_offset += cur_size;
}
return size;
}
size_t AlignmentMatchingStorageImpl::Write(VirtualFile base_storage, char* work_buf,
size_t work_buf_size, size_t data_alignment,
size_t buffer_alignment, s64 offset, const u8* buffer,
size_t size) {
// Check preconditions.
ASSERT(work_buf_size >= data_alignment);
// Succeed if zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
// Determine extents.
const u8* aligned_core_buffer;
s64 core_offset;
size_t core_size;
s64 covered_offset;
const size_t offset_round_up_difference = GetRoundUpDifference(offset, data_alignment);
if (Common::IsAligned(reinterpret_cast<uintptr_t>(buffer) + offset_round_up_difference,
buffer_alignment)) {
aligned_core_buffer = buffer + offset_round_up_difference;
core_offset = Common::AlignUp(offset, data_alignment);
core_size = (size < offset_round_up_difference)
? 0
: Common::AlignDown(size - offset_round_up_difference, data_alignment);
covered_offset = core_size > 0 ? core_offset : offset;
} else {
aligned_core_buffer = nullptr;
core_offset = Common::AlignDown(offset, data_alignment);
core_size = 0;
covered_offset = offset;
}
// Write the core portion.
if (core_size > 0) {
base_storage->Write(aligned_core_buffer, core_size, core_offset);
}
// Handle the head portion.
if (offset < covered_offset) {
const s64 head_offset = Common::AlignDown(offset, data_alignment);
const size_t head_size = static_cast<size_t>(covered_offset - offset);
ASSERT((offset - head_offset) + head_size <= data_alignment);
base_storage->Read(reinterpret_cast<u8*>(work_buf), data_alignment, head_offset);
std::memcpy(work_buf + (offset - head_offset), buffer, head_size);
base_storage->Write(reinterpret_cast<u8*>(work_buf), data_alignment, head_offset);
}
// Handle the tail portion.
s64 tail_offset = covered_offset + core_size;
size_t remaining_tail_size = static_cast<size_t>((offset + size) - tail_offset);
while (remaining_tail_size > 0) {
ASSERT(static_cast<size_t>(tail_offset - offset) < size);
const auto aligned_tail_offset = Common::AlignDown(tail_offset, data_alignment);
const auto cur_size =
std::min(static_cast<size_t>(aligned_tail_offset + data_alignment - tail_offset),
remaining_tail_size);
base_storage->Read(reinterpret_cast<u8*>(work_buf), data_alignment, aligned_tail_offset);
std::memcpy(work_buf + GetRoundDownDifference(tail_offset, data_alignment),
buffer + (tail_offset - offset), cur_size);
base_storage->Write(reinterpret_cast<u8*>(work_buf), data_alignment, aligned_tail_offset);
remaining_tail_size -= cur_size;
tail_offset += cur_size;
}
return size;
}
} // namespace FileSys

View File

@ -0,0 +1,21 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
namespace FileSys {
class AlignmentMatchingStorageImpl {
public:
static size_t Read(VirtualFile base_storage, char* work_buf, size_t work_buf_size,
size_t data_alignment, size_t buffer_alignment, s64 offset, u8* buffer,
size_t size);
static size_t Write(VirtualFile base_storage, char* work_buf, size_t work_buf_size,
size_t data_alignment, size_t buffer_alignment, s64 offset,
const u8* buffer, size_t size);
};
} // namespace FileSys

View File

@ -0,0 +1,598 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree_utils.h"
#include "core/file_sys/fssystem/fssystem_pooled_buffer.h"
namespace FileSys {
namespace {
using Node = impl::BucketTreeNode<const s64*>;
static_assert(sizeof(Node) == sizeof(BucketTree::NodeHeader));
static_assert(std::is_trivial_v<Node>);
constexpr inline s32 NodeHeaderSize = sizeof(BucketTree::NodeHeader);
class StorageNode {
private:
class Offset {
public:
using difference_type = s64;
private:
s64 m_offset;
s32 m_stride;
public:
constexpr Offset(s64 offset, s32 stride) : m_offset(offset), m_stride(stride) {}
constexpr Offset& operator++() {
m_offset += m_stride;
return *this;
}
constexpr Offset operator++(int) {
Offset ret(*this);
m_offset += m_stride;
return ret;
}
constexpr Offset& operator--() {
m_offset -= m_stride;
return *this;
}
constexpr Offset operator--(int) {
Offset ret(*this);
m_offset -= m_stride;
return ret;
}
constexpr difference_type operator-(const Offset& rhs) const {
return (m_offset - rhs.m_offset) / m_stride;
}
constexpr Offset operator+(difference_type ofs) const {
return Offset(m_offset + ofs * m_stride, m_stride);
}
constexpr Offset operator-(difference_type ofs) const {
return Offset(m_offset - ofs * m_stride, m_stride);
}
constexpr Offset& operator+=(difference_type ofs) {
m_offset += ofs * m_stride;
return *this;
}
constexpr Offset& operator-=(difference_type ofs) {
m_offset -= ofs * m_stride;
return *this;
}
constexpr bool operator==(const Offset& rhs) const {
return m_offset == rhs.m_offset;
}
constexpr bool operator!=(const Offset& rhs) const {
return m_offset != rhs.m_offset;
}
constexpr s64 Get() const {
return m_offset;
}
};
private:
const Offset m_start;
const s32 m_count;
s32 m_index;
public:
StorageNode(size_t size, s32 count)
: m_start(NodeHeaderSize, static_cast<s32>(size)), m_count(count), m_index(-1) {}
StorageNode(s64 ofs, size_t size, s32 count)
: m_start(NodeHeaderSize + ofs, static_cast<s32>(size)), m_count(count), m_index(-1) {}
s32 GetIndex() const {
return m_index;
}
void Find(const char* buffer, s64 virtual_address) {
s32 end = m_count;
auto pos = m_start;
while (end > 0) {
auto half = end / 2;
auto mid = pos + half;
s64 offset = 0;
std::memcpy(std::addressof(offset), buffer + mid.Get(), sizeof(s64));
if (offset <= virtual_address) {
pos = mid + 1;
end -= half + 1;
} else {
end = half;
}
}
m_index = static_cast<s32>(pos - m_start) - 1;
}
Result Find(VirtualFile storage, s64 virtual_address) {
s32 end = m_count;
auto pos = m_start;
while (end > 0) {
auto half = end / 2;
auto mid = pos + half;
s64 offset = 0;
storage->ReadObject(std::addressof(offset), mid.Get());
if (offset <= virtual_address) {
pos = mid + 1;
end -= half + 1;
} else {
end = half;
}
}
m_index = static_cast<s32>(pos - m_start) - 1;
R_SUCCEED();
}
};
} // namespace
void BucketTree::Header::Format(s32 entry_count_) {
ASSERT(entry_count_ >= 0);
this->magic = Magic;
this->version = Version;
this->entry_count = entry_count_;
this->reserved = 0;
}
Result BucketTree::Header::Verify() const {
R_UNLESS(this->magic == Magic, ResultInvalidBucketTreeSignature);
R_UNLESS(this->entry_count >= 0, ResultInvalidBucketTreeEntryCount);
R_UNLESS(this->version <= Version, ResultUnsupportedVersion);
R_SUCCEED();
}
Result BucketTree::NodeHeader::Verify(s32 node_index, size_t node_size, size_t entry_size) const {
R_UNLESS(this->index == node_index, ResultInvalidBucketTreeNodeIndex);
R_UNLESS(entry_size != 0 && node_size >= entry_size + NodeHeaderSize, ResultInvalidSize);
const size_t max_entry_count = (node_size - NodeHeaderSize) / entry_size;
R_UNLESS(this->count > 0 && static_cast<size_t>(this->count) <= max_entry_count,
ResultInvalidBucketTreeNodeEntryCount);
R_UNLESS(this->offset >= 0, ResultInvalidBucketTreeNodeOffset);
R_SUCCEED();
}
Result BucketTree::Initialize(VirtualFile node_storage, VirtualFile entry_storage, size_t node_size,
size_t entry_size, s32 entry_count) {
// Validate preconditions.
ASSERT(entry_size >= sizeof(s64));
ASSERT(node_size >= entry_size + sizeof(NodeHeader));
ASSERT(NodeSizeMin <= node_size && node_size <= NodeSizeMax);
ASSERT(Common::IsPowerOfTwo(node_size));
ASSERT(!this->IsInitialized());
// Ensure valid entry count.
R_UNLESS(entry_count > 0, ResultInvalidArgument);
// Allocate node.
R_UNLESS(m_node_l1.Allocate(node_size), ResultBufferAllocationFailed);
ON_RESULT_FAILURE {
m_node_l1.Free(node_size);
};
// Read node.
node_storage->Read(reinterpret_cast<u8*>(m_node_l1.Get()), node_size);
// Verify node.
R_TRY(m_node_l1->Verify(0, node_size, sizeof(s64)));
// Validate offsets.
const auto offset_count = GetOffsetCount(node_size);
const auto entry_set_count = GetEntrySetCount(node_size, entry_size, entry_count);
const auto* const node = m_node_l1.Get<Node>();
s64 start_offset;
if (offset_count < entry_set_count && node->GetCount() < offset_count) {
start_offset = *node->GetEnd();
} else {
start_offset = *node->GetBegin();
}
const auto end_offset = node->GetEndOffset();
R_UNLESS(0 <= start_offset && start_offset <= node->GetBeginOffset(),
ResultInvalidBucketTreeEntryOffset);
R_UNLESS(start_offset < end_offset, ResultInvalidBucketTreeEntryOffset);
// Set member variables.
m_node_storage = node_storage;
m_entry_storage = entry_storage;
m_node_size = node_size;
m_entry_size = entry_size;
m_entry_count = entry_count;
m_offset_count = offset_count;
m_entry_set_count = entry_set_count;
m_offset_cache.offsets.start_offset = start_offset;
m_offset_cache.offsets.end_offset = end_offset;
m_offset_cache.is_initialized = true;
// We succeeded.
R_SUCCEED();
}
void BucketTree::Initialize(size_t node_size, s64 end_offset) {
ASSERT(NodeSizeMin <= node_size && node_size <= NodeSizeMax);
ASSERT(Common::IsPowerOfTwo(node_size));
ASSERT(end_offset > 0);
ASSERT(!this->IsInitialized());
m_node_size = node_size;
m_offset_cache.offsets.start_offset = 0;
m_offset_cache.offsets.end_offset = end_offset;
m_offset_cache.is_initialized = true;
}
void BucketTree::Finalize() {
if (this->IsInitialized()) {
m_node_storage = VirtualFile();
m_entry_storage = VirtualFile();
m_node_l1.Free(m_node_size);
m_node_size = 0;
m_entry_size = 0;
m_entry_count = 0;
m_offset_count = 0;
m_entry_set_count = 0;
m_offset_cache.offsets.start_offset = 0;
m_offset_cache.offsets.end_offset = 0;
m_offset_cache.is_initialized = false;
}
}
Result BucketTree::Find(Visitor* visitor, s64 virtual_address) {
ASSERT(visitor != nullptr);
ASSERT(this->IsInitialized());
R_UNLESS(virtual_address >= 0, ResultInvalidOffset);
R_UNLESS(!this->IsEmpty(), ResultOutOfRange);
BucketTree::Offsets offsets;
R_TRY(this->GetOffsets(std::addressof(offsets)));
R_TRY(visitor->Initialize(this, offsets));
R_RETURN(visitor->Find(virtual_address));
}
Result BucketTree::InvalidateCache() {
// Reset our offsets.
m_offset_cache.is_initialized = false;
R_SUCCEED();
}
Result BucketTree::EnsureOffsetCache() {
// If we already have an offset cache, we're good.
R_SUCCEED_IF(m_offset_cache.is_initialized);
// Acquire exclusive right to edit the offset cache.
std::scoped_lock lk(m_offset_cache.mutex);
// Check again, to be sure.
R_SUCCEED_IF(m_offset_cache.is_initialized);
// Read/verify L1.
m_node_storage->Read(reinterpret_cast<u8*>(m_node_l1.Get()), m_node_size);
R_TRY(m_node_l1->Verify(0, m_node_size, sizeof(s64)));
// Get the node.
auto* const node = m_node_l1.Get<Node>();
s64 start_offset;
if (m_offset_count < m_entry_set_count && node->GetCount() < m_offset_count) {
start_offset = *node->GetEnd();
} else {
start_offset = *node->GetBegin();
}
const auto end_offset = node->GetEndOffset();
R_UNLESS(0 <= start_offset && start_offset <= node->GetBeginOffset(),
ResultInvalidBucketTreeEntryOffset);
R_UNLESS(start_offset < end_offset, ResultInvalidBucketTreeEntryOffset);
m_offset_cache.offsets.start_offset = start_offset;
m_offset_cache.offsets.end_offset = end_offset;
m_offset_cache.is_initialized = true;
R_SUCCEED();
}
Result BucketTree::Visitor::Initialize(const BucketTree* tree, const BucketTree::Offsets& offsets) {
ASSERT(tree != nullptr);
ASSERT(m_tree == nullptr || m_tree == tree);
if (m_entry == nullptr) {
m_entry = ::operator new(tree->m_entry_size);
R_UNLESS(m_entry != nullptr, ResultBufferAllocationFailed);
m_tree = tree;
m_offsets = offsets;
}
R_SUCCEED();
}
Result BucketTree::Visitor::MoveNext() {
R_UNLESS(this->IsValid(), ResultOutOfRange);
// Invalidate our index, and read the header for the next index.
auto entry_index = m_entry_index + 1;
if (entry_index == m_entry_set.info.count) {
const auto entry_set_index = m_entry_set.info.index + 1;
R_UNLESS(entry_set_index < m_entry_set_count, ResultOutOfRange);
m_entry_index = -1;
const auto end = m_entry_set.info.end;
const auto entry_set_size = m_tree->m_node_size;
const auto entry_set_offset = entry_set_index * static_cast<s64>(entry_set_size);
m_tree->m_entry_storage->ReadObject(std::addressof(m_entry_set), entry_set_offset);
R_TRY(m_entry_set.header.Verify(entry_set_index, entry_set_size, m_tree->m_entry_size));
R_UNLESS(m_entry_set.info.start == end && m_entry_set.info.start < m_entry_set.info.end,
ResultInvalidBucketTreeEntrySetOffset);
entry_index = 0;
} else {
m_entry_index = -1;
}
// Read the new entry.
const auto entry_size = m_tree->m_entry_size;
const auto entry_offset = impl::GetBucketTreeEntryOffset(
m_entry_set.info.index, m_tree->m_node_size, entry_size, entry_index);
m_tree->m_entry_storage->Read(reinterpret_cast<u8*>(m_entry), entry_size, entry_offset);
// Note that we changed index.
m_entry_index = entry_index;
R_SUCCEED();
}
Result BucketTree::Visitor::MovePrevious() {
R_UNLESS(this->IsValid(), ResultOutOfRange);
// Invalidate our index, and read the header for the previous index.
auto entry_index = m_entry_index;
if (entry_index == 0) {
R_UNLESS(m_entry_set.info.index > 0, ResultOutOfRange);
m_entry_index = -1;
const auto start = m_entry_set.info.start;
const auto entry_set_size = m_tree->m_node_size;
const auto entry_set_index = m_entry_set.info.index - 1;
const auto entry_set_offset = entry_set_index * static_cast<s64>(entry_set_size);
m_tree->m_entry_storage->ReadObject(std::addressof(m_entry_set), entry_set_offset);
R_TRY(m_entry_set.header.Verify(entry_set_index, entry_set_size, m_tree->m_entry_size));
R_UNLESS(m_entry_set.info.end == start && m_entry_set.info.start < m_entry_set.info.end,
ResultInvalidBucketTreeEntrySetOffset);
entry_index = m_entry_set.info.count;
} else {
m_entry_index = -1;
}
--entry_index;
// Read the new entry.
const auto entry_size = m_tree->m_entry_size;
const auto entry_offset = impl::GetBucketTreeEntryOffset(
m_entry_set.info.index, m_tree->m_node_size, entry_size, entry_index);
m_tree->m_entry_storage->Read(reinterpret_cast<u8*>(m_entry), entry_size, entry_offset);
// Note that we changed index.
m_entry_index = entry_index;
R_SUCCEED();
}
Result BucketTree::Visitor::Find(s64 virtual_address) {
ASSERT(m_tree != nullptr);
// Get the node.
const auto* const node = m_tree->m_node_l1.Get<Node>();
R_UNLESS(virtual_address < node->GetEndOffset(), ResultOutOfRange);
// Get the entry set index.
s32 entry_set_index = -1;
if (m_tree->IsExistOffsetL2OnL1() && virtual_address < node->GetBeginOffset()) {
const auto start = node->GetEnd();
const auto end = node->GetBegin() + m_tree->m_offset_count;
auto pos = std::upper_bound(start, end, virtual_address);
R_UNLESS(start < pos, ResultOutOfRange);
--pos;
entry_set_index = static_cast<s32>(pos - start);
} else {
const auto start = node->GetBegin();
const auto end = node->GetEnd();
auto pos = std::upper_bound(start, end, virtual_address);
R_UNLESS(start < pos, ResultOutOfRange);
--pos;
if (m_tree->IsExistL2()) {
const auto node_index = static_cast<s32>(pos - start);
R_UNLESS(0 <= node_index && node_index < m_tree->m_offset_count,
ResultInvalidBucketTreeNodeOffset);
R_TRY(this->FindEntrySet(std::addressof(entry_set_index), virtual_address, node_index));
} else {
entry_set_index = static_cast<s32>(pos - start);
}
}
// Validate the entry set index.
R_UNLESS(0 <= entry_set_index && entry_set_index < m_tree->m_entry_set_count,
ResultInvalidBucketTreeNodeOffset);
// Find the entry.
R_TRY(this->FindEntry(virtual_address, entry_set_index));
// Set count.
m_entry_set_count = m_tree->m_entry_set_count;
R_SUCCEED();
}
Result BucketTree::Visitor::FindEntrySet(s32* out_index, s64 virtual_address, s32 node_index) {
const auto node_size = m_tree->m_node_size;
PooledBuffer pool(node_size, 1);
if (node_size <= pool.GetSize()) {
R_RETURN(
this->FindEntrySetWithBuffer(out_index, virtual_address, node_index, pool.GetBuffer()));
} else {
pool.Deallocate();
R_RETURN(this->FindEntrySetWithoutBuffer(out_index, virtual_address, node_index));
}
}
Result BucketTree::Visitor::FindEntrySetWithBuffer(s32* out_index, s64 virtual_address,
s32 node_index, char* buffer) {
// Calculate node extents.
const auto node_size = m_tree->m_node_size;
const auto node_offset = (node_index + 1) * static_cast<s64>(node_size);
VirtualFile storage = m_tree->m_node_storage;
// Read the node.
storage->Read(reinterpret_cast<u8*>(buffer), node_size, node_offset);
// Validate the header.
NodeHeader header;
std::memcpy(std::addressof(header), buffer, NodeHeaderSize);
R_TRY(header.Verify(node_index, node_size, sizeof(s64)));
// Create the node, and find.
StorageNode node(sizeof(s64), header.count);
node.Find(buffer, virtual_address);
R_UNLESS(node.GetIndex() >= 0, ResultInvalidBucketTreeVirtualOffset);
// Return the index.
*out_index = static_cast<s32>(m_tree->GetEntrySetIndex(header.index, node.GetIndex()));
R_SUCCEED();
}
Result BucketTree::Visitor::FindEntrySetWithoutBuffer(s32* out_index, s64 virtual_address,
s32 node_index) {
// Calculate node extents.
const auto node_size = m_tree->m_node_size;
const auto node_offset = (node_index + 1) * static_cast<s64>(node_size);
VirtualFile storage = m_tree->m_node_storage;
// Read and validate the header.
NodeHeader header;
storage->ReadObject(std::addressof(header), node_offset);
R_TRY(header.Verify(node_index, node_size, sizeof(s64)));
// Create the node, and find.
StorageNode node(node_offset, sizeof(s64), header.count);
R_TRY(node.Find(storage, virtual_address));
R_UNLESS(node.GetIndex() >= 0, ResultOutOfRange);
// Return the index.
*out_index = static_cast<s32>(m_tree->GetEntrySetIndex(header.index, node.GetIndex()));
R_SUCCEED();
}
Result BucketTree::Visitor::FindEntry(s64 virtual_address, s32 entry_set_index) {
const auto entry_set_size = m_tree->m_node_size;
PooledBuffer pool(entry_set_size, 1);
if (entry_set_size <= pool.GetSize()) {
R_RETURN(this->FindEntryWithBuffer(virtual_address, entry_set_index, pool.GetBuffer()));
} else {
pool.Deallocate();
R_RETURN(this->FindEntryWithoutBuffer(virtual_address, entry_set_index));
}
}
Result BucketTree::Visitor::FindEntryWithBuffer(s64 virtual_address, s32 entry_set_index,
char* buffer) {
// Calculate entry set extents.
const auto entry_size = m_tree->m_entry_size;
const auto entry_set_size = m_tree->m_node_size;
const auto entry_set_offset = entry_set_index * static_cast<s64>(entry_set_size);
VirtualFile storage = m_tree->m_entry_storage;
// Read the entry set.
storage->Read(reinterpret_cast<u8*>(buffer), entry_set_size, entry_set_offset);
// Validate the entry_set.
EntrySetHeader entry_set;
std::memcpy(std::addressof(entry_set), buffer, sizeof(EntrySetHeader));
R_TRY(entry_set.header.Verify(entry_set_index, entry_set_size, entry_size));
// Create the node, and find.
StorageNode node(entry_size, entry_set.info.count);
node.Find(buffer, virtual_address);
R_UNLESS(node.GetIndex() >= 0, ResultOutOfRange);
// Copy the data into entry.
const auto entry_index = node.GetIndex();
const auto entry_offset = impl::GetBucketTreeEntryOffset(0, entry_size, entry_index);
std::memcpy(m_entry, buffer + entry_offset, entry_size);
// Set our entry set/index.
m_entry_set = entry_set;
m_entry_index = entry_index;
R_SUCCEED();
}
Result BucketTree::Visitor::FindEntryWithoutBuffer(s64 virtual_address, s32 entry_set_index) {
// Calculate entry set extents.
const auto entry_size = m_tree->m_entry_size;
const auto entry_set_size = m_tree->m_node_size;
const auto entry_set_offset = entry_set_index * static_cast<s64>(entry_set_size);
VirtualFile storage = m_tree->m_entry_storage;
// Read and validate the entry_set.
EntrySetHeader entry_set;
storage->ReadObject(std::addressof(entry_set), entry_set_offset);
R_TRY(entry_set.header.Verify(entry_set_index, entry_set_size, entry_size));
// Create the node, and find.
StorageNode node(entry_set_offset, entry_size, entry_set.info.count);
R_TRY(node.Find(storage, virtual_address));
R_UNLESS(node.GetIndex() >= 0, ResultOutOfRange);
// Copy the data into entry.
const auto entry_index = node.GetIndex();
const auto entry_offset =
impl::GetBucketTreeEntryOffset(entry_set_offset, entry_size, entry_index);
storage->Read(reinterpret_cast<u8*>(m_entry), entry_size, entry_offset);
// Set our entry set/index.
m_entry_set = entry_set;
m_entry_index = entry_index;
R_SUCCEED();
}
} // namespace FileSys

View File

@ -0,0 +1,416 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <mutex>
#include "common/alignment.h"
#include "common/common_funcs.h"
#include "common/common_types.h"
#include "common/literals.h"
#include "core/file_sys/vfs.h"
#include "core/hle/result.h"
namespace FileSys {
using namespace Common::Literals;
class BucketTree {
YUZU_NON_COPYABLE(BucketTree);
YUZU_NON_MOVEABLE(BucketTree);
public:
static constexpr u32 Magic = Common::MakeMagic('B', 'K', 'T', 'R');
static constexpr u32 Version = 1;
static constexpr size_t NodeSizeMin = 1_KiB;
static constexpr size_t NodeSizeMax = 512_KiB;
public:
class Visitor;
struct Header {
u32 magic;
u32 version;
s32 entry_count;
s32 reserved;
void Format(s32 entry_count);
Result Verify() const;
};
static_assert(std::is_trivial_v<Header>);
static_assert(sizeof(Header) == 0x10);
struct NodeHeader {
s32 index;
s32 count;
s64 offset;
Result Verify(s32 node_index, size_t node_size, size_t entry_size) const;
};
static_assert(std::is_trivial_v<NodeHeader>);
static_assert(sizeof(NodeHeader) == 0x10);
struct Offsets {
s64 start_offset;
s64 end_offset;
constexpr bool IsInclude(s64 offset) const {
return this->start_offset <= offset && offset < this->end_offset;
}
constexpr bool IsInclude(s64 offset, s64 size) const {
return size > 0 && this->start_offset <= offset && size <= (this->end_offset - offset);
}
};
static_assert(std::is_trivial_v<Offsets>);
static_assert(sizeof(Offsets) == 0x10);
struct OffsetCache {
Offsets offsets;
std::mutex mutex;
bool is_initialized;
OffsetCache() : offsets{-1, -1}, mutex(), is_initialized(false) {}
};
class ContinuousReadingInfo {
public:
constexpr ContinuousReadingInfo() : m_read_size(), m_skip_count(), m_done() {}
constexpr void Reset() {
m_read_size = 0;
m_skip_count = 0;
m_done = false;
}
constexpr void SetSkipCount(s32 count) {
ASSERT(count >= 0);
m_skip_count = count;
}
constexpr s32 GetSkipCount() const {
return m_skip_count;
}
constexpr bool CheckNeedScan() {
return (--m_skip_count) <= 0;
}
constexpr void Done() {
m_read_size = 0;
m_done = true;
}
constexpr bool IsDone() const {
return m_done;
}
constexpr void SetReadSize(size_t size) {
m_read_size = size;
}
constexpr size_t GetReadSize() const {
return m_read_size;
}
constexpr bool CanDo() const {
return m_read_size > 0;
}
private:
size_t m_read_size;
s32 m_skip_count;
bool m_done;
};
private:
class NodeBuffer {
YUZU_NON_COPYABLE(NodeBuffer);
public:
NodeBuffer() : m_header() {}
~NodeBuffer() {
ASSERT(m_header == nullptr);
}
NodeBuffer(NodeBuffer&& rhs) : m_header(rhs.m_header) {
rhs.m_header = nullptr;
}
NodeBuffer& operator=(NodeBuffer&& rhs) {
if (this != std::addressof(rhs)) {
ASSERT(m_header == nullptr);
m_header = rhs.m_header;
rhs.m_header = nullptr;
}
return *this;
}
bool Allocate(size_t node_size) {
ASSERT(m_header == nullptr);
m_header = ::operator new(node_size, std::align_val_t{sizeof(s64)});
// ASSERT(Common::IsAligned(m_header, sizeof(s64)));
return m_header != nullptr;
}
void Free(size_t node_size) {
if (m_header) {
::operator delete(m_header, std::align_val_t{sizeof(s64)});
m_header = nullptr;
}
}
void FillZero(size_t node_size) const {
if (m_header) {
std::memset(m_header, 0, node_size);
}
}
NodeHeader* Get() const {
return reinterpret_cast<NodeHeader*>(m_header);
}
NodeHeader* operator->() const {
return this->Get();
}
template <typename T>
T* Get() const {
static_assert(std::is_trivial_v<T>);
static_assert(sizeof(T) == sizeof(NodeHeader));
return reinterpret_cast<T*>(m_header);
}
private:
void* m_header;
};
private:
static constexpr s32 GetEntryCount(size_t node_size, size_t entry_size) {
return static_cast<s32>((node_size - sizeof(NodeHeader)) / entry_size);
}
static constexpr s32 GetOffsetCount(size_t node_size) {
return static_cast<s32>((node_size - sizeof(NodeHeader)) / sizeof(s64));
}
static constexpr s32 GetEntrySetCount(size_t node_size, size_t entry_size, s32 entry_count) {
const s32 entry_count_per_node = GetEntryCount(node_size, entry_size);
return Common::DivideUp(entry_count, entry_count_per_node);
}
static constexpr s32 GetNodeL2Count(size_t node_size, size_t entry_size, s32 entry_count) {
const s32 offset_count_per_node = GetOffsetCount(node_size);
const s32 entry_set_count = GetEntrySetCount(node_size, entry_size, entry_count);
if (entry_set_count <= offset_count_per_node) {
return 0;
}
const s32 node_l2_count = Common::DivideUp(entry_set_count, offset_count_per_node);
ASSERT(node_l2_count <= offset_count_per_node);
return Common::DivideUp(entry_set_count - (offset_count_per_node - (node_l2_count - 1)),
offset_count_per_node);
}
public:
BucketTree()
: m_node_storage(), m_entry_storage(), m_node_l1(), m_node_size(), m_entry_size(),
m_entry_count(), m_offset_count(), m_entry_set_count(), m_offset_cache() {}
~BucketTree() {
this->Finalize();
}
Result Initialize(VirtualFile node_storage, VirtualFile entry_storage, size_t node_size,
size_t entry_size, s32 entry_count);
void Initialize(size_t node_size, s64 end_offset);
void Finalize();
bool IsInitialized() const {
return m_node_size > 0;
}
bool IsEmpty() const {
return m_entry_size == 0;
}
Result Find(Visitor* visitor, s64 virtual_address);
Result InvalidateCache();
s32 GetEntryCount() const {
return m_entry_count;
}
Result GetOffsets(Offsets* out) {
// Ensure we have an offset cache.
R_TRY(this->EnsureOffsetCache());
// Set the output.
*out = m_offset_cache.offsets;
R_SUCCEED();
}
public:
static constexpr s64 QueryHeaderStorageSize() {
return sizeof(Header);
}
static constexpr s64 QueryNodeStorageSize(size_t node_size, size_t entry_size,
s32 entry_count) {
ASSERT(entry_size >= sizeof(s64));
ASSERT(node_size >= entry_size + sizeof(NodeHeader));
ASSERT(NodeSizeMin <= node_size && node_size <= NodeSizeMax);
ASSERT(Common::IsPowerOfTwo(node_size));
ASSERT(entry_count >= 0);
if (entry_count <= 0) {
return 0;
}
return (1 + GetNodeL2Count(node_size, entry_size, entry_count)) *
static_cast<s64>(node_size);
}
static constexpr s64 QueryEntryStorageSize(size_t node_size, size_t entry_size,
s32 entry_count) {
ASSERT(entry_size >= sizeof(s64));
ASSERT(node_size >= entry_size + sizeof(NodeHeader));
ASSERT(NodeSizeMin <= node_size && node_size <= NodeSizeMax);
ASSERT(Common::IsPowerOfTwo(node_size));
ASSERT(entry_count >= 0);
if (entry_count <= 0) {
return 0;
}
return GetEntrySetCount(node_size, entry_size, entry_count) * static_cast<s64>(node_size);
}
private:
template <typename EntryType>
struct ContinuousReadingParam {
s64 offset;
size_t size;
NodeHeader entry_set;
s32 entry_index;
Offsets offsets;
EntryType entry;
};
private:
template <typename EntryType>
Result ScanContinuousReading(ContinuousReadingInfo* out_info,
const ContinuousReadingParam<EntryType>& param) const;
bool IsExistL2() const {
return m_offset_count < m_entry_set_count;
}
bool IsExistOffsetL2OnL1() const {
return this->IsExistL2() && m_node_l1->count < m_offset_count;
}
s64 GetEntrySetIndex(s32 node_index, s32 offset_index) const {
return (m_offset_count - m_node_l1->count) + (m_offset_count * node_index) + offset_index;
}
Result EnsureOffsetCache();
private:
mutable VirtualFile m_node_storage;
mutable VirtualFile m_entry_storage;
NodeBuffer m_node_l1;
size_t m_node_size;
size_t m_entry_size;
s32 m_entry_count;
s32 m_offset_count;
s32 m_entry_set_count;
OffsetCache m_offset_cache;
};
class BucketTree::Visitor {
YUZU_NON_COPYABLE(Visitor);
YUZU_NON_MOVEABLE(Visitor);
public:
constexpr Visitor()
: m_tree(), m_entry(), m_entry_index(-1), m_entry_set_count(), m_entry_set{} {}
~Visitor() {
if (m_entry != nullptr) {
::operator delete(m_entry, m_tree->m_entry_size);
m_tree = nullptr;
m_entry = nullptr;
}
}
bool IsValid() const {
return m_entry_index >= 0;
}
bool CanMoveNext() const {
return this->IsValid() && (m_entry_index + 1 < m_entry_set.info.count ||
m_entry_set.info.index + 1 < m_entry_set_count);
}
bool CanMovePrevious() const {
return this->IsValid() && (m_entry_index > 0 || m_entry_set.info.index > 0);
}
Result MoveNext();
Result MovePrevious();
template <typename EntryType>
Result ScanContinuousReading(ContinuousReadingInfo* out_info, s64 offset, size_t size) const;
const void* Get() const {
ASSERT(this->IsValid());
return m_entry;
}
template <typename T>
const T* Get() const {
ASSERT(this->IsValid());
return reinterpret_cast<const T*>(m_entry);
}
const BucketTree* GetTree() const {
return m_tree;
}
private:
Result Initialize(const BucketTree* tree, const BucketTree::Offsets& offsets);
Result Find(s64 virtual_address);
Result FindEntrySet(s32* out_index, s64 virtual_address, s32 node_index);
Result FindEntrySetWithBuffer(s32* out_index, s64 virtual_address, s32 node_index,
char* buffer);
Result FindEntrySetWithoutBuffer(s32* out_index, s64 virtual_address, s32 node_index);
Result FindEntry(s64 virtual_address, s32 entry_set_index);
Result FindEntryWithBuffer(s64 virtual_address, s32 entry_set_index, char* buffer);
Result FindEntryWithoutBuffer(s64 virtual_address, s32 entry_set_index);
private:
friend class BucketTree;
union EntrySetHeader {
NodeHeader header;
struct Info {
s32 index;
s32 count;
s64 end;
s64 start;
} info;
static_assert(std::is_trivial_v<Info>);
};
static_assert(std::is_trivial_v<EntrySetHeader>);
const BucketTree* m_tree;
BucketTree::Offsets m_offsets;
void* m_entry;
s32 m_entry_index;
s32 m_entry_set_count;
EntrySetHeader m_entry_set;
};
} // namespace FileSys

View File

@ -0,0 +1,170 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree_utils.h"
#include "core/file_sys/fssystem/fssystem_pooled_buffer.h"
namespace FileSys {
template <typename EntryType>
Result BucketTree::ScanContinuousReading(ContinuousReadingInfo* out_info,
const ContinuousReadingParam<EntryType>& param) const {
static_assert(std::is_trivial_v<ContinuousReadingParam<EntryType>>);
// Validate our preconditions.
ASSERT(this->IsInitialized());
ASSERT(out_info != nullptr);
ASSERT(m_entry_size == sizeof(EntryType));
// Reset the output.
out_info->Reset();
// If there's nothing to read, we're done.
R_SUCCEED_IF(param.size == 0);
// If we're reading a fragment, we're done.
R_SUCCEED_IF(param.entry.IsFragment());
// Validate the first entry.
auto entry = param.entry;
auto cur_offset = param.offset;
R_UNLESS(entry.GetVirtualOffset() <= cur_offset, ResultOutOfRange);
// Create a pooled buffer for our scan.
PooledBuffer pool(m_node_size, 1);
char* buffer = nullptr;
s64 entry_storage_size = m_entry_storage->GetSize();
// Read the node.
if (m_node_size <= pool.GetSize()) {
buffer = pool.GetBuffer();
const auto ofs = param.entry_set.index * static_cast<s64>(m_node_size);
R_UNLESS(m_node_size + ofs <= static_cast<size_t>(entry_storage_size),
ResultInvalidBucketTreeNodeEntryCount);
m_entry_storage->Read(reinterpret_cast<u8*>(buffer), m_node_size, ofs);
}
// Calculate extents.
const auto end_offset = cur_offset + static_cast<s64>(param.size);
s64 phys_offset = entry.GetPhysicalOffset();
// Start merge tracking.
s64 merge_size = 0;
s64 readable_size = 0;
bool merged = false;
// Iterate.
auto entry_index = param.entry_index;
for (const auto entry_count = param.entry_set.count; entry_index < entry_count; ++entry_index) {
// If we're past the end, we're done.
if (end_offset <= cur_offset) {
break;
}
// Validate the entry offset.
const auto entry_offset = entry.GetVirtualOffset();
R_UNLESS(entry_offset <= cur_offset, ResultInvalidIndirectEntryOffset);
// Get the next entry.
EntryType next_entry = {};
s64 next_entry_offset;
if (entry_index + 1 < entry_count) {
if (buffer != nullptr) {
const auto ofs = impl::GetBucketTreeEntryOffset(0, m_entry_size, entry_index + 1);
std::memcpy(std::addressof(next_entry), buffer + ofs, m_entry_size);
} else {
const auto ofs = impl::GetBucketTreeEntryOffset(param.entry_set.index, m_node_size,
m_entry_size, entry_index + 1);
m_entry_storage->ReadObject(std::addressof(next_entry), ofs);
}
next_entry_offset = next_entry.GetVirtualOffset();
R_UNLESS(param.offsets.IsInclude(next_entry_offset), ResultInvalidIndirectEntryOffset);
} else {
next_entry_offset = param.entry_set.offset;
}
// Validate the next entry offset.
R_UNLESS(cur_offset < next_entry_offset, ResultInvalidIndirectEntryOffset);
// Determine the much data there is.
const auto data_size = next_entry_offset - cur_offset;
ASSERT(data_size > 0);
// Determine how much data we should read.
const auto remaining_size = end_offset - cur_offset;
const size_t read_size = static_cast<size_t>(std::min(data_size, remaining_size));
ASSERT(read_size <= param.size);
// Update our merge tracking.
if (entry.IsFragment()) {
// If we can't merge, stop looping.
if (EntryType::FragmentSizeMax <= read_size || remaining_size <= data_size) {
break;
}
// Otherwise, add the current size to the merge size.
merge_size += read_size;
} else {
// If we can't merge, stop looping.
if (phys_offset != entry.GetPhysicalOffset()) {
break;
}
// Add the size to the readable amount.
readable_size += merge_size + read_size;
ASSERT(readable_size <= static_cast<s64>(param.size));
// Update whether we've merged.
merged |= merge_size > 0;
merge_size = 0;
}
// Advance.
cur_offset += read_size;
ASSERT(cur_offset <= end_offset);
phys_offset += next_entry_offset - entry_offset;
entry = next_entry;
}
// If we merged, set our readable size.
if (merged) {
out_info->SetReadSize(static_cast<size_t>(readable_size));
}
out_info->SetSkipCount(entry_index - param.entry_index);
R_SUCCEED();
}
template <typename EntryType>
Result BucketTree::Visitor::ScanContinuousReading(ContinuousReadingInfo* out_info, s64 offset,
size_t size) const {
static_assert(std::is_trivial_v<EntryType>);
ASSERT(this->IsValid());
// Create our parameters.
ContinuousReadingParam<EntryType> param = {
.offset = offset,
.size = size,
.entry_set = m_entry_set.header,
.entry_index = m_entry_index,
.offsets{},
.entry{},
};
std::memcpy(std::addressof(param.offsets), std::addressof(m_offsets),
sizeof(BucketTree::Offsets));
std::memcpy(std::addressof(param.entry), m_entry, sizeof(EntryType));
// Scan.
R_RETURN(m_tree->ScanContinuousReading<EntryType>(out_info, param));
}
} // namespace FileSys

View File

@ -0,0 +1,110 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/fssystem/fssystem_bucket_tree.h"
namespace FileSys::impl {
class SafeValue {
public:
static s64 GetInt64(const void* ptr) {
s64 value;
std::memcpy(std::addressof(value), ptr, sizeof(s64));
return value;
}
static s64 GetInt64(const s64* ptr) {
return GetInt64(static_cast<const void*>(ptr));
}
static s64 GetInt64(const s64& v) {
return GetInt64(std::addressof(v));
}
static void SetInt64(void* dst, const void* src) {
std::memcpy(dst, src, sizeof(s64));
}
static void SetInt64(void* dst, const s64* src) {
return SetInt64(dst, static_cast<const void*>(src));
}
static void SetInt64(void* dst, const s64& v) {
return SetInt64(dst, std::addressof(v));
}
};
template <typename IteratorType>
struct BucketTreeNode {
using Header = BucketTree::NodeHeader;
Header header;
s32 GetCount() const {
return this->header.count;
}
void* GetArray() {
return std::addressof(this->header) + 1;
}
template <typename T>
T* GetArray() {
return reinterpret_cast<T*>(this->GetArray());
}
const void* GetArray() const {
return std::addressof(this->header) + 1;
}
template <typename T>
const T* GetArray() const {
return reinterpret_cast<const T*>(this->GetArray());
}
s64 GetBeginOffset() const {
return *this->GetArray<s64>();
}
s64 GetEndOffset() const {
return this->header.offset;
}
IteratorType GetBegin() {
return IteratorType(this->GetArray<s64>());
}
IteratorType GetEnd() {
return IteratorType(this->GetArray<s64>()) + this->header.count;
}
IteratorType GetBegin() const {
return IteratorType(this->GetArray<s64>());
}
IteratorType GetEnd() const {
return IteratorType(this->GetArray<s64>()) + this->header.count;
}
IteratorType GetBegin(size_t entry_size) {
return IteratorType(this->GetArray(), entry_size);
}
IteratorType GetEnd(size_t entry_size) {
return IteratorType(this->GetArray(), entry_size) + this->header.count;
}
IteratorType GetBegin(size_t entry_size) const {
return IteratorType(this->GetArray(), entry_size);
}
IteratorType GetEnd(size_t entry_size) const {
return IteratorType(this->GetArray(), entry_size) + this->header.count;
}
};
constexpr inline s64 GetBucketTreeEntryOffset(s64 entry_set_offset, size_t entry_size,
s32 entry_index) {
return entry_set_offset + sizeof(BucketTree::NodeHeader) +
entry_index * static_cast<s64>(entry_size);
}
constexpr inline s64 GetBucketTreeEntryOffset(s32 entry_set_index, size_t node_size,
size_t entry_size, s32 entry_index) {
return GetBucketTreeEntryOffset(entry_set_index * static_cast<s64>(node_size), entry_size,
entry_index);
}
} // namespace FileSys::impl

View File

@ -0,0 +1,963 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/literals.h"
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree.h"
#include "core/file_sys/fssystem/fssystem_compression_common.h"
#include "core/file_sys/fssystem/fssystem_pooled_buffer.h"
#include "core/file_sys/vfs.h"
namespace FileSys {
using namespace Common::Literals;
class CompressedStorage : public IReadOnlyStorage {
YUZU_NON_COPYABLE(CompressedStorage);
YUZU_NON_MOVEABLE(CompressedStorage);
public:
static constexpr size_t NodeSize = 16_KiB;
struct Entry {
s64 virt_offset;
s64 phys_offset;
CompressionType compression_type;
s32 phys_size;
s64 GetPhysicalSize() const {
return this->phys_size;
}
};
static_assert(std::is_trivial_v<Entry>);
static_assert(sizeof(Entry) == 0x18);
public:
static constexpr s64 QueryNodeStorageSize(s32 entry_count) {
return BucketTree::QueryNodeStorageSize(NodeSize, sizeof(Entry), entry_count);
}
static constexpr s64 QueryEntryStorageSize(s32 entry_count) {
return BucketTree::QueryEntryStorageSize(NodeSize, sizeof(Entry), entry_count);
}
private:
class CompressedStorageCore {
YUZU_NON_COPYABLE(CompressedStorageCore);
YUZU_NON_MOVEABLE(CompressedStorageCore);
public:
CompressedStorageCore() : m_table(), m_data_storage() {}
~CompressedStorageCore() {
this->Finalize();
}
public:
Result Initialize(VirtualFile data_storage, VirtualFile node_storage,
VirtualFile entry_storage, s32 bktr_entry_count, size_t block_size_max,
size_t continuous_reading_size_max,
GetDecompressorFunction get_decompressor) {
// Check pre-conditions.
ASSERT(0 < block_size_max);
ASSERT(block_size_max <= continuous_reading_size_max);
ASSERT(get_decompressor != nullptr);
// Initialize our entry table.
R_TRY(m_table.Initialize(node_storage, entry_storage, NodeSize, sizeof(Entry),
bktr_entry_count));
// Set our other fields.
m_block_size_max = block_size_max;
m_continuous_reading_size_max = continuous_reading_size_max;
m_data_storage = data_storage;
m_get_decompressor_function = get_decompressor;
R_SUCCEED();
}
void Finalize() {
if (this->IsInitialized()) {
m_table.Finalize();
m_data_storage = VirtualFile();
}
}
VirtualFile GetDataStorage() {
return m_data_storage;
}
Result GetDataStorageSize(s64* out) {
// Check pre-conditions.
ASSERT(out != nullptr);
// Get size.
*out = m_data_storage->GetSize();
R_SUCCEED();
}
BucketTree& GetEntryTable() {
return m_table;
}
Result GetEntryList(Entry* out_entries, s32* out_read_count, s32 max_entry_count,
s64 offset, s64 size) {
// Check pre-conditions.
ASSERT(offset >= 0);
ASSERT(size >= 0);
ASSERT(this->IsInitialized());
// Check that we can output the count.
R_UNLESS(out_read_count != nullptr, ResultNullptrArgument);
// Check that we have anything to read at all.
R_SUCCEED_IF(size == 0);
// Check that either we have a buffer, or this is to determine how many we need.
if (max_entry_count != 0) {
R_UNLESS(out_entries != nullptr, ResultNullptrArgument);
}
// Get the table offsets.
BucketTree::Offsets table_offsets;
R_TRY(m_table.GetOffsets(std::addressof(table_offsets)));
// Validate arguments.
R_UNLESS(table_offsets.IsInclude(offset, size), ResultOutOfRange);
// Find the offset in our tree.
BucketTree::Visitor visitor;
R_TRY(m_table.Find(std::addressof(visitor), offset));
{
const auto entry_offset = visitor.Get<Entry>()->virt_offset;
R_UNLESS(0 <= entry_offset && table_offsets.IsInclude(entry_offset),
ResultUnexpectedInCompressedStorageA);
}
// Get the entries.
const auto end_offset = offset + size;
s32 read_count = 0;
while (visitor.Get<Entry>()->virt_offset < end_offset) {
// If we should be setting the output, do so.
if (max_entry_count != 0) {
// Ensure we only read as many entries as we can.
if (read_count >= max_entry_count) {
break;
}
// Set the current output entry.
out_entries[read_count] = *visitor.Get<Entry>();
}
// Increase the read count.
++read_count;
// If we're at the end, we're done.
if (!visitor.CanMoveNext()) {
break;
}
// Move to the next entry.
R_TRY(visitor.MoveNext());
}
// Set the output read count.
*out_read_count = read_count;
R_SUCCEED();
}
Result GetSize(s64* out) {
// Check pre-conditions.
ASSERT(out != nullptr);
// Get our table offsets.
BucketTree::Offsets offsets;
R_TRY(m_table.GetOffsets(std::addressof(offsets)));
// Set the output.
*out = offsets.end_offset;
R_SUCCEED();
}
Result OperatePerEntry(s64 offset, s64 size, auto f) {
// Check pre-conditions.
ASSERT(offset >= 0);
ASSERT(size >= 0);
ASSERT(this->IsInitialized());
// Succeed if there's nothing to operate on.
R_SUCCEED_IF(size == 0);
// Get the table offsets.
BucketTree::Offsets table_offsets;
R_TRY(m_table.GetOffsets(std::addressof(table_offsets)));
// Validate arguments.
R_UNLESS(table_offsets.IsInclude(offset, size), ResultOutOfRange);
// Find the offset in our tree.
BucketTree::Visitor visitor;
R_TRY(m_table.Find(std::addressof(visitor), offset));
{
const auto entry_offset = visitor.Get<Entry>()->virt_offset;
R_UNLESS(0 <= entry_offset && table_offsets.IsInclude(entry_offset),
ResultUnexpectedInCompressedStorageA);
}
// Prepare to operate in chunks.
auto cur_offset = offset;
const auto end_offset = offset + static_cast<s64>(size);
while (cur_offset < end_offset) {
// Get the current entry.
const auto cur_entry = *visitor.Get<Entry>();
// Get and validate the entry's offset.
const auto cur_entry_offset = cur_entry.virt_offset;
R_UNLESS(cur_entry_offset <= cur_offset, ResultUnexpectedInCompressedStorageA);
// Get and validate the next entry offset.
s64 next_entry_offset;
if (visitor.CanMoveNext()) {
R_TRY(visitor.MoveNext());
next_entry_offset = visitor.Get<Entry>()->virt_offset;
R_UNLESS(table_offsets.IsInclude(next_entry_offset),
ResultUnexpectedInCompressedStorageA);
} else {
next_entry_offset = table_offsets.end_offset;
}
R_UNLESS(cur_offset < next_entry_offset, ResultUnexpectedInCompressedStorageA);
// Get the offset of the entry in the data we read.
const auto data_offset = cur_offset - cur_entry_offset;
const auto data_size = (next_entry_offset - cur_entry_offset);
ASSERT(data_size > 0);
// Determine how much is left.
const auto remaining_size = end_offset - cur_offset;
const auto cur_size = std::min<s64>(remaining_size, data_size - data_offset);
ASSERT(cur_size <= size);
// Get the data storage size.
s64 storage_size = m_data_storage->GetSize();
// Check that our read remains naively physically in bounds.
R_UNLESS(0 <= cur_entry.phys_offset && cur_entry.phys_offset <= storage_size,
ResultUnexpectedInCompressedStorageC);
// If we have any compression, verify that we remain physically in bounds.
if (cur_entry.compression_type != CompressionType::None) {
R_UNLESS(cur_entry.phys_offset + cur_entry.GetPhysicalSize() <= storage_size,
ResultUnexpectedInCompressedStorageC);
}
// Check that block alignment requirements are met.
if (CompressionTypeUtility::IsBlockAlignmentRequired(cur_entry.compression_type)) {
R_UNLESS(Common::IsAligned(cur_entry.phys_offset, CompressionBlockAlignment),
ResultUnexpectedInCompressedStorageA);
}
// Invoke the operator.
bool is_continuous = true;
R_TRY(
f(std::addressof(is_continuous), cur_entry, data_size, data_offset, cur_size));
// If not continuous, we're done.
if (!is_continuous) {
break;
}
// Advance.
cur_offset += cur_size;
}
R_SUCCEED();
}
public:
using ReadImplFunction = std::function<Result(void*, size_t)>;
using ReadFunction = std::function<Result(size_t, const ReadImplFunction&)>;
public:
Result Read(s64 offset, s64 size, const ReadFunction& read_func) {
// Check pre-conditions.
ASSERT(offset >= 0);
ASSERT(this->IsInitialized());
// Succeed immediately, if we have nothing to read.
R_SUCCEED_IF(size == 0);
// Declare read lambda.
constexpr int EntriesCountMax = 0x80;
struct Entries {
CompressionType compression_type;
u32 gap_from_prev;
u32 physical_size;
u32 virtual_size;
};
std::array<Entries, EntriesCountMax> entries;
s32 entry_count = 0;
Entry prev_entry = {
.virt_offset = -1,
.phys_offset{},
.compression_type{},
.phys_size{},
};
bool will_allocate_pooled_buffer = false;
s64 required_access_physical_offset = 0;
s64 required_access_physical_size = 0;
auto PerformRequiredRead = [&]() -> Result {
// If there are no entries, we have nothing to do.
R_SUCCEED_IF(entry_count == 0);
// Get the remaining size in a convenient form.
const size_t total_required_size =
static_cast<size_t>(required_access_physical_size);
// Perform the read based on whether we need to allocate a buffer.
if (will_allocate_pooled_buffer) {
// Allocate a pooled buffer.
PooledBuffer pooled_buffer;
if (pooled_buffer.GetAllocatableSizeMax() >= total_required_size) {
pooled_buffer.Allocate(total_required_size, m_block_size_max);
} else {
pooled_buffer.AllocateParticularlyLarge(
std::min<size_t>(
total_required_size,
PooledBuffer::GetAllocatableParticularlyLargeSizeMax()),
m_block_size_max);
}
// Read each of the entries.
for (s32 entry_idx = 0; entry_idx < entry_count; ++entry_idx) {
// Determine the current read size.
bool will_use_pooled_buffer = false;
const size_t cur_read_size = [&]() -> size_t {
if (const size_t target_entry_size =
static_cast<size_t>(entries[entry_idx].physical_size) +
static_cast<size_t>(entries[entry_idx].gap_from_prev);
target_entry_size <= pooled_buffer.GetSize()) {
// We'll be using the pooled buffer.
will_use_pooled_buffer = true;
// Determine how much we can read.
const size_t max_size = std::min<size_t>(
required_access_physical_size, pooled_buffer.GetSize());
size_t read_size = 0;
for (auto n = entry_idx; n < entry_count; ++n) {
const size_t cur_entry_size =
static_cast<size_t>(entries[n].physical_size) +
static_cast<size_t>(entries[n].gap_from_prev);
if (read_size + cur_entry_size > max_size) {
break;
}
read_size += cur_entry_size;
}
return read_size;
} else {
// If we don't fit, we must be uncompressed.
ASSERT(entries[entry_idx].compression_type ==
CompressionType::None);
// We can perform the whole of an uncompressed read directly.
return entries[entry_idx].virtual_size;
}
}();
// Perform the read based on whether or not we'll use the pooled buffer.
if (will_use_pooled_buffer) {
// Read the compressed data into the pooled buffer.
auto* const buffer = pooled_buffer.GetBuffer();
m_data_storage->Read(reinterpret_cast<u8*>(buffer), cur_read_size,
required_access_physical_offset);
// Decompress the data.
size_t buffer_offset;
for (buffer_offset = 0;
entry_idx < entry_count &&
((static_cast<size_t>(entries[entry_idx].physical_size) +
static_cast<size_t>(entries[entry_idx].gap_from_prev)) == 0 ||
buffer_offset < cur_read_size);
buffer_offset += entries[entry_idx++].physical_size) {
// Advance by the relevant gap.
buffer_offset += entries[entry_idx].gap_from_prev;
const auto compression_type = entries[entry_idx].compression_type;
switch (compression_type) {
case CompressionType::None: {
// Check that we can remain within bounds.
ASSERT(buffer_offset + entries[entry_idx].virtual_size <=
cur_read_size);
// Perform no decompression.
R_TRY(read_func(
entries[entry_idx].virtual_size,
[&](void* dst, size_t dst_size) -> Result {
// Check that the size is valid.
ASSERT(dst_size == entries[entry_idx].virtual_size);
// We have no compression, so just copy the data
// out.
std::memcpy(dst, buffer + buffer_offset,
entries[entry_idx].virtual_size);
R_SUCCEED();
}));
break;
}
case CompressionType::Zeros: {
// Check that we can remain within bounds.
ASSERT(buffer_offset <= cur_read_size);
// Zero the memory.
R_TRY(read_func(
entries[entry_idx].virtual_size,
[&](void* dst, size_t dst_size) -> Result {
// Check that the size is valid.
ASSERT(dst_size == entries[entry_idx].virtual_size);
// The data is zeroes, so zero the buffer.
std::memset(dst, 0, entries[entry_idx].virtual_size);
R_SUCCEED();
}));
break;
}
default: {
// Check that we can remain within bounds.
ASSERT(buffer_offset + entries[entry_idx].physical_size <=
cur_read_size);
// Get the decompressor.
const auto decompressor =
this->GetDecompressor(compression_type);
R_UNLESS(decompressor != nullptr,
ResultUnexpectedInCompressedStorageB);
// Decompress the data.
R_TRY(read_func(entries[entry_idx].virtual_size,
[&](void* dst, size_t dst_size) -> Result {
// Check that the size is valid.
ASSERT(dst_size ==
entries[entry_idx].virtual_size);
// Perform the decompression.
R_RETURN(decompressor(
dst, entries[entry_idx].virtual_size,
buffer + buffer_offset,
entries[entry_idx].physical_size));
}));
break;
}
}
}
// Check that we processed the correct amount of data.
ASSERT(buffer_offset == cur_read_size);
} else {
// Account for the gap from the previous entry.
required_access_physical_offset += entries[entry_idx].gap_from_prev;
required_access_physical_size -= entries[entry_idx].gap_from_prev;
// We don't need the buffer (as the data is uncompressed), so just
// execute the read.
R_TRY(
read_func(cur_read_size, [&](void* dst, size_t dst_size) -> Result {
// Check that the size is valid.
ASSERT(dst_size == cur_read_size);
// Perform the read.
m_data_storage->Read(reinterpret_cast<u8*>(dst), cur_read_size,
required_access_physical_offset);
R_SUCCEED();
}));
}
// Advance on.
required_access_physical_offset += cur_read_size;
required_access_physical_size -= cur_read_size;
}
// Verify that we have nothing remaining to read.
ASSERT(required_access_physical_size == 0);
R_SUCCEED();
} else {
// We don't need a buffer, so just execute the read.
R_TRY(read_func(total_required_size, [&](void* dst, size_t dst_size) -> Result {
// Check that the size is valid.
ASSERT(dst_size == total_required_size);
// Perform the read.
m_data_storage->Read(reinterpret_cast<u8*>(dst), total_required_size,
required_access_physical_offset);
R_SUCCEED();
}));
}
R_SUCCEED();
};
R_TRY(this->OperatePerEntry(
offset, size,
[&](bool* out_continuous, const Entry& entry, s64 virtual_data_size,
s64 data_offset, s64 read_size) -> Result {
// Determine the physical extents.
s64 physical_offset, physical_size;
if (CompressionTypeUtility::IsRandomAccessible(entry.compression_type)) {
physical_offset = entry.phys_offset + data_offset;
physical_size = read_size;
} else {
physical_offset = entry.phys_offset;
physical_size = entry.GetPhysicalSize();
}
// If we have a pending data storage operation, perform it if we have to.
const s64 required_access_physical_end =
required_access_physical_offset + required_access_physical_size;
if (required_access_physical_size > 0) {
const bool required_by_gap =
!(required_access_physical_end <= physical_offset &&
physical_offset <= Common::AlignUp(required_access_physical_end,
CompressionBlockAlignment));
const bool required_by_continuous_size =
((physical_size + physical_offset) - required_access_physical_end) +
required_access_physical_size >
static_cast<s64>(m_continuous_reading_size_max);
const bool required_by_entry_count = entry_count == EntriesCountMax;
if (required_by_gap || required_by_continuous_size ||
required_by_entry_count) {
// Check that our planned access is sane.
ASSERT(!will_allocate_pooled_buffer ||
required_access_physical_size <=
static_cast<s64>(m_continuous_reading_size_max));
// Perform the required read.
const Result rc = PerformRequiredRead();
if (R_FAILED(rc)) {
R_THROW(rc);
}
// Reset our requirements.
prev_entry.virt_offset = -1;
required_access_physical_size = 0;
entry_count = 0;
will_allocate_pooled_buffer = false;
}
}
// Sanity check that we're within bounds on entries.
ASSERT(entry_count < EntriesCountMax);
// Determine if a buffer allocation is needed.
if (entry.compression_type != CompressionType::None ||
(prev_entry.virt_offset >= 0 &&
entry.virt_offset - prev_entry.virt_offset !=
entry.phys_offset - prev_entry.phys_offset)) {
will_allocate_pooled_buffer = true;
}
// If we need to access the data storage, update our required access parameters.
if (CompressionTypeUtility::IsDataStorageAccessRequired(
entry.compression_type)) {
// If the data is compressed, ensure the access is sane.
if (entry.compression_type != CompressionType::None) {
R_UNLESS(data_offset == 0, ResultInvalidOffset);
R_UNLESS(virtual_data_size == read_size, ResultInvalidSize);
R_UNLESS(entry.GetPhysicalSize() <= static_cast<s64>(m_block_size_max),
ResultUnexpectedInCompressedStorageD);
}
// Update the required access parameters.
s64 gap_from_prev;
if (required_access_physical_size > 0) {
gap_from_prev = physical_offset - required_access_physical_end;
} else {
gap_from_prev = 0;
required_access_physical_offset = physical_offset;
}
required_access_physical_size += physical_size + gap_from_prev;
// Create an entry to access the data storage.
entries[entry_count++] = {
.compression_type = entry.compression_type,
.gap_from_prev = static_cast<u32>(gap_from_prev),
.physical_size = static_cast<u32>(physical_size),
.virtual_size = static_cast<u32>(read_size),
};
} else {
// Verify that we're allowed to be operating on the non-data-storage-access
// type.
R_UNLESS(entry.compression_type == CompressionType::Zeros,
ResultUnexpectedInCompressedStorageB);
// If we have entries, create a fake entry for the zero region.
if (entry_count != 0) {
// We need to have a physical size.
R_UNLESS(entry.GetPhysicalSize() != 0,
ResultUnexpectedInCompressedStorageD);
// Create a fake entry.
entries[entry_count++] = {
.compression_type = CompressionType::Zeros,
.gap_from_prev = 0,
.physical_size = 0,
.virtual_size = static_cast<u32>(read_size),
};
} else {
// We have no entries, so we can just perform the read.
const Result rc =
read_func(static_cast<size_t>(read_size),
[&](void* dst, size_t dst_size) -> Result {
// Check the space we should zero is correct.
ASSERT(dst_size == static_cast<size_t>(read_size));
// Zero the memory.
std::memset(dst, 0, read_size);
R_SUCCEED();
});
if (R_FAILED(rc)) {
R_THROW(rc);
}
}
}
// Set the previous entry.
prev_entry = entry;
// We're continuous.
*out_continuous = true;
R_SUCCEED();
}));
// If we still have a pending access, perform it.
if (required_access_physical_size != 0) {
R_TRY(PerformRequiredRead());
}
R_SUCCEED();
}
private:
DecompressorFunction GetDecompressor(CompressionType type) const {
// Check that we can get a decompressor for the type.
if (CompressionTypeUtility::IsUnknownType(type)) {
return nullptr;
}
// Get the decompressor.
return m_get_decompressor_function(type);
}
bool IsInitialized() const {
return m_table.IsInitialized();
}
private:
size_t m_block_size_max;
size_t m_continuous_reading_size_max;
BucketTree m_table;
VirtualFile m_data_storage;
GetDecompressorFunction m_get_decompressor_function;
};
class CacheManager {
YUZU_NON_COPYABLE(CacheManager);
YUZU_NON_MOVEABLE(CacheManager);
private:
struct AccessRange {
s64 virtual_offset;
s64 virtual_size;
u32 physical_size;
bool is_block_alignment_required;
s64 GetEndVirtualOffset() const {
return this->virtual_offset + this->virtual_size;
}
};
static_assert(std::is_trivial_v<AccessRange>);
public:
CacheManager() = default;
public:
Result Initialize(s64 storage_size, size_t cache_size_0, size_t cache_size_1,
size_t max_cache_entries) {
// Set our fields.
m_storage_size = storage_size;
R_SUCCEED();
}
Result Read(CompressedStorageCore& core, s64 offset, void* buffer, size_t size) {
// If we have nothing to read, succeed.
R_SUCCEED_IF(size == 0);
// Check that we have a buffer to read into.
R_UNLESS(buffer != nullptr, ResultNullptrArgument);
// Check that the read is in bounds.
R_UNLESS(offset <= m_storage_size, ResultInvalidOffset);
// Determine how much we can read.
const size_t read_size = std::min<size_t>(size, m_storage_size - offset);
// Create head/tail ranges.
AccessRange head_range = {};
AccessRange tail_range = {};
bool is_tail_set = false;
// Operate to determine the head range.
R_TRY(core.OperatePerEntry(
offset, 1,
[&](bool* out_continuous, const Entry& entry, s64 virtual_data_size,
s64 data_offset, s64 data_read_size) -> Result {
// Set the head range.
head_range = {
.virtual_offset = entry.virt_offset,
.virtual_size = virtual_data_size,
.physical_size = static_cast<u32>(entry.phys_size),
.is_block_alignment_required =
CompressionTypeUtility::IsBlockAlignmentRequired(
entry.compression_type),
};
// If required, set the tail range.
if (static_cast<s64>(offset + read_size) <=
entry.virt_offset + virtual_data_size) {
tail_range = {
.virtual_offset = entry.virt_offset,
.virtual_size = virtual_data_size,
.physical_size = static_cast<u32>(entry.phys_size),
.is_block_alignment_required =
CompressionTypeUtility::IsBlockAlignmentRequired(
entry.compression_type),
};
is_tail_set = true;
}
// We only want to determine the head range, so we're not continuous.
*out_continuous = false;
R_SUCCEED();
}));
// If necessary, determine the tail range.
if (!is_tail_set) {
R_TRY(core.OperatePerEntry(
offset + read_size - 1, 1,
[&](bool* out_continuous, const Entry& entry, s64 virtual_data_size,
s64 data_offset, s64 data_read_size) -> Result {
// Set the tail range.
tail_range = {
.virtual_offset = entry.virt_offset,
.virtual_size = virtual_data_size,
.physical_size = static_cast<u32>(entry.phys_size),
.is_block_alignment_required =
CompressionTypeUtility::IsBlockAlignmentRequired(
entry.compression_type),
};
// We only want to determine the tail range, so we're not continuous.
*out_continuous = false;
R_SUCCEED();
}));
}
// Begin performing the accesses.
s64 cur_offset = offset;
size_t cur_size = read_size;
char* cur_dst = static_cast<char*>(buffer);
// Determine our alignment.
const bool head_unaligned = head_range.is_block_alignment_required &&
(cur_offset != head_range.virtual_offset ||
static_cast<s64>(cur_size) < head_range.virtual_size);
const bool tail_unaligned = [&]() -> bool {
if (tail_range.is_block_alignment_required) {
if (static_cast<s64>(cur_size + cur_offset) ==
tail_range.GetEndVirtualOffset()) {
return false;
} else if (!head_unaligned) {
return true;
} else {
return head_range.GetEndVirtualOffset() <
static_cast<s64>(cur_size + cur_offset);
}
} else {
return false;
}
}();
// Determine start/end offsets.
const s64 start_offset =
head_range.is_block_alignment_required ? head_range.virtual_offset : cur_offset;
const s64 end_offset = tail_range.is_block_alignment_required
? tail_range.GetEndVirtualOffset()
: cur_offset + cur_size;
// Perform the read.
bool is_burst_reading = false;
R_TRY(core.Read(
start_offset, end_offset - start_offset,
[&](size_t size_buffer_required,
const CompressedStorageCore::ReadImplFunction& read_impl) -> Result {
// Determine whether we're burst reading.
const AccessRange* unaligned_range = nullptr;
if (!is_burst_reading) {
// Check whether we're using head, tail, or none as unaligned.
if (head_unaligned && head_range.virtual_offset <= cur_offset &&
cur_offset < head_range.GetEndVirtualOffset()) {
unaligned_range = std::addressof(head_range);
} else if (tail_unaligned && tail_range.virtual_offset <= cur_offset &&
cur_offset < tail_range.GetEndVirtualOffset()) {
unaligned_range = std::addressof(tail_range);
} else {
is_burst_reading = true;
}
}
ASSERT((is_burst_reading ^ (unaligned_range != nullptr)));
// Perform reading by burst, or not.
if (is_burst_reading) {
// Check that the access is valid for burst reading.
ASSERT(size_buffer_required <= cur_size);
// Perform the read.
Result rc = read_impl(cur_dst, size_buffer_required);
if (R_FAILED(rc)) {
R_THROW(rc);
}
// Advance.
cur_dst += size_buffer_required;
cur_offset += size_buffer_required;
cur_size -= size_buffer_required;
// Determine whether we're going to continue burst reading.
const s64 offset_aligned =
tail_unaligned ? tail_range.virtual_offset : end_offset;
ASSERT(cur_offset <= offset_aligned);
if (offset_aligned <= cur_offset) {
is_burst_reading = false;
}
} else {
// We're not burst reading, so we have some unaligned range.
ASSERT(unaligned_range != nullptr);
// Check that the size is correct.
ASSERT(size_buffer_required ==
static_cast<size_t>(unaligned_range->virtual_size));
// Get a pooled buffer for our read.
PooledBuffer pooled_buffer;
pooled_buffer.Allocate(size_buffer_required, size_buffer_required);
// Perform read.
Result rc = read_impl(pooled_buffer.GetBuffer(), size_buffer_required);
if (R_FAILED(rc)) {
R_THROW(rc);
}
// Copy the data we read to the destination.
const size_t skip_size = cur_offset - unaligned_range->virtual_offset;
const size_t copy_size = std::min<size_t>(
cur_size, unaligned_range->GetEndVirtualOffset() - cur_offset);
std::memcpy(cur_dst, pooled_buffer.GetBuffer() + skip_size, copy_size);
// Advance.
cur_dst += copy_size;
cur_offset += copy_size;
cur_size -= copy_size;
}
R_SUCCEED();
}));
R_SUCCEED();
}
private:
s64 m_storage_size = 0;
};
public:
CompressedStorage() = default;
virtual ~CompressedStorage() {
this->Finalize();
}
Result Initialize(VirtualFile data_storage, VirtualFile node_storage, VirtualFile entry_storage,
s32 bktr_entry_count, size_t block_size_max,
size_t continuous_reading_size_max, GetDecompressorFunction get_decompressor,
size_t cache_size_0, size_t cache_size_1, s32 max_cache_entries) {
// Initialize our core.
R_TRY(m_core.Initialize(data_storage, node_storage, entry_storage, bktr_entry_count,
block_size_max, continuous_reading_size_max, get_decompressor));
// Get our core size.
s64 core_size = 0;
R_TRY(m_core.GetSize(std::addressof(core_size)));
// Initialize our cache manager.
R_TRY(m_cache_manager.Initialize(core_size, cache_size_0, cache_size_1, max_cache_entries));
R_SUCCEED();
}
void Finalize() {
m_core.Finalize();
}
VirtualFile GetDataStorage() {
return m_core.GetDataStorage();
}
Result GetDataStorageSize(s64* out) {
R_RETURN(m_core.GetDataStorageSize(out));
}
Result GetEntryList(Entry* out_entries, s32* out_read_count, s32 max_entry_count, s64 offset,
s64 size) {
R_RETURN(m_core.GetEntryList(out_entries, out_read_count, max_entry_count, offset, size));
}
BucketTree& GetEntryTable() {
return m_core.GetEntryTable();
}
public:
virtual size_t GetSize() const override {
s64 ret{};
m_core.GetSize(&ret);
return ret;
}
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override {
if (R_SUCCEEDED(m_cache_manager.Read(m_core, offset, buffer, size))) {
return size;
} else {
return 0;
}
}
private:
mutable CompressedStorageCore m_core;
mutable CacheManager m_cache_manager;
};
} // namespace FileSys

View File

@ -0,0 +1,43 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/hle/result.h"
namespace FileSys {
enum class CompressionType : u8 {
None = 0,
Zeros = 1,
Two = 2,
Lz4 = 3,
Unknown = 4,
};
using DecompressorFunction = Result (*)(void*, size_t, const void*, size_t);
using GetDecompressorFunction = DecompressorFunction (*)(CompressionType);
constexpr s64 CompressionBlockAlignment = 0x10;
namespace CompressionTypeUtility {
constexpr bool IsBlockAlignmentRequired(CompressionType type) {
return type != CompressionType::None && type != CompressionType::Zeros;
}
constexpr bool IsDataStorageAccessRequired(CompressionType type) {
return type != CompressionType::Zeros;
}
constexpr bool IsRandomAccessible(CompressionType type) {
return type == CompressionType::None;
}
constexpr bool IsUnknownType(CompressionType type) {
return type >= CompressionType::Unknown;
}
} // namespace CompressionTypeUtility
} // namespace FileSys

View File

@ -0,0 +1,36 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/lz4_compression.h"
#include "core/file_sys/fssystem/fssystem_compression_configuration.h"
namespace FileSys {
namespace {
Result DecompressLz4(void* dst, size_t dst_size, const void* src, size_t src_size) {
auto result = Common::Compression::DecompressDataLZ4(dst, dst_size, src, src_size);
R_UNLESS(static_cast<size_t>(result) == dst_size, ResultUnexpectedInCompressedStorageC);
R_SUCCEED();
}
constexpr DecompressorFunction GetNcaDecompressorFunction(CompressionType type) {
switch (type) {
case CompressionType::Lz4:
return DecompressLz4;
default:
return nullptr;
}
}
} // namespace
const NcaCompressionConfiguration& GetNcaCompressionConfiguration() {
static const NcaCompressionConfiguration configuration = {
.get_decompressor = GetNcaDecompressorFunction,
};
return configuration;
}
} // namespace FileSys

View File

@ -0,0 +1,12 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/fssystem/fssystem_nca_file_system_driver.h"
namespace FileSys {
const NcaCompressionConfiguration& GetNcaCompressionConfiguration();
}

View File

@ -0,0 +1,65 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/crypto/aes_util.h"
#include "core/crypto/key_manager.h"
#include "core/file_sys/fssystem/fssystem_crypto_configuration.h"
namespace FileSys {
namespace {
void GenerateKey(void* dst_key, size_t dst_key_size, const void* src_key, size_t src_key_size,
s32 key_type) {
if (key_type == static_cast<s32>(KeyType::ZeroKey)) {
std::memset(dst_key, 0, dst_key_size);
return;
}
if (key_type == static_cast<s32>(KeyType::InvalidKey) ||
key_type < static_cast<s32>(KeyType::ZeroKey) ||
key_type >= static_cast<s32>(KeyType::NcaExternalKey)) {
std::memset(dst_key, 0xFF, dst_key_size);
return;
}
const auto& instance = Core::Crypto::KeyManager::Instance();
if (key_type == static_cast<s32>(KeyType::NcaHeaderKey1) ||
key_type == static_cast<s32>(KeyType::NcaHeaderKey2)) {
const s32 key_index = static_cast<s32>(KeyType::NcaHeaderKey2) == key_type;
const auto key = instance.GetKey(Core::Crypto::S256KeyType::Header);
std::memcpy(dst_key, key.data() + key_index * 0x10, std::min(dst_key_size, key.size() / 2));
return;
}
const s32 key_generation =
std::max(key_type / NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexCount, 1) - 1;
const s32 key_index = key_type % NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexCount;
Core::Crypto::AESCipher<Core::Crypto::Key128> cipher(
instance.GetKey(Core::Crypto::S128KeyType::KeyArea, key_generation, key_index),
Core::Crypto::Mode::ECB);
cipher.Transcode(reinterpret_cast<const u8*>(src_key), src_key_size,
reinterpret_cast<u8*>(dst_key), Core::Crypto::Op::Decrypt);
}
} // namespace
const NcaCryptoConfiguration& GetCryptoConfiguration() {
static const NcaCryptoConfiguration configuration = {
.header_1_sign_key_moduli{},
.header_1_sign_key_public_exponent{},
.key_area_encryption_key_source{},
.header_encryption_key_source{},
.header_encrypted_encryption_keys{},
.generate_key = GenerateKey,
.verify_sign1{},
.is_plaintext_header_available{},
.is_available_sw_key{},
};
return configuration;
}
} // namespace FileSys

View File

@ -0,0 +1,12 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/fssystem/fssystem_nca_file_system_driver.h"
namespace FileSys {
const NcaCryptoConfiguration& GetCryptoConfiguration();
}

View File

@ -0,0 +1,127 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/file_sys/fssystem/fssystem_hierarchical_integrity_verification_storage.h"
#include "core/file_sys/vfs_offset.h"
namespace FileSys {
HierarchicalIntegrityVerificationStorage::HierarchicalIntegrityVerificationStorage()
: m_data_size(-1) {
for (size_t i = 0; i < MaxLayers - 1; i++) {
m_verify_storages[i] = std::make_shared<IntegrityVerificationStorage>();
}
}
Result HierarchicalIntegrityVerificationStorage::Initialize(
const HierarchicalIntegrityVerificationInformation& info,
HierarchicalStorageInformation storage, int max_data_cache_entries, int max_hash_cache_entries,
s8 buffer_level) {
// Validate preconditions.
ASSERT(IntegrityMinLayerCount <= info.max_layers && info.max_layers <= IntegrityMaxLayerCount);
// Set member variables.
m_max_layers = info.max_layers;
// Initialize the top level verification storage.
m_verify_storages[0]->Initialize(storage[HierarchicalStorageInformation::MasterStorage],
storage[HierarchicalStorageInformation::Layer1Storage],
static_cast<s64>(1) << info.info[0].block_order, HashSize,
false);
// Ensure we don't leak state if further initialization goes wrong.
ON_RESULT_FAILURE {
m_verify_storages[0]->Finalize();
m_data_size = -1;
};
// Initialize the top level buffer storage.
m_buffer_storages[0] = m_verify_storages[0];
R_UNLESS(m_buffer_storages[0] != nullptr, ResultAllocationMemoryFailedAllocateShared);
// Prepare to initialize the level storages.
s32 level = 0;
// Ensure we don't leak state if further initialization goes wrong.
ON_RESULT_FAILURE_2 {
m_verify_storages[level + 1]->Finalize();
for (; level > 0; --level) {
m_buffer_storages[level].reset();
m_verify_storages[level]->Finalize();
}
};
// Initialize the level storages.
for (; level < m_max_layers - 3; ++level) {
// Initialize the verification storage.
auto buffer_storage =
std::make_shared<OffsetVfsFile>(m_buffer_storages[level], info.info[level].size, 0);
m_verify_storages[level + 1]->Initialize(
std::move(buffer_storage), storage[level + 2],
static_cast<s64>(1) << info.info[level + 1].block_order,
static_cast<s64>(1) << info.info[level].block_order, false);
// Initialize the buffer storage.
m_buffer_storages[level + 1] = m_verify_storages[level + 1];
R_UNLESS(m_buffer_storages[level + 1] != nullptr,
ResultAllocationMemoryFailedAllocateShared);
}
// Initialize the final level storage.
{
// Initialize the verification storage.
auto buffer_storage =
std::make_shared<OffsetVfsFile>(m_buffer_storages[level], info.info[level].size, 0);
m_verify_storages[level + 1]->Initialize(
std::move(buffer_storage), storage[level + 2],
static_cast<s64>(1) << info.info[level + 1].block_order,
static_cast<s64>(1) << info.info[level].block_order, true);
// Initialize the buffer storage.
m_buffer_storages[level + 1] = m_verify_storages[level + 1];
R_UNLESS(m_buffer_storages[level + 1] != nullptr,
ResultAllocationMemoryFailedAllocateShared);
}
// Set the data size.
m_data_size = info.info[level + 1].size;
// We succeeded.
R_SUCCEED();
}
void HierarchicalIntegrityVerificationStorage::Finalize() {
if (m_data_size >= 0) {
m_data_size = 0;
for (s32 level = m_max_layers - 2; level >= 0; --level) {
m_buffer_storages[level].reset();
m_verify_storages[level]->Finalize();
}
m_data_size = -1;
}
}
size_t HierarchicalIntegrityVerificationStorage::Read(u8* buffer, size_t size,
size_t offset) const {
// Validate preconditions.
ASSERT(m_data_size >= 0);
// Succeed if zero-size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
// Read the data.
return m_buffer_storages[m_max_layers - 2]->Read(buffer, size, offset);
}
size_t HierarchicalIntegrityVerificationStorage::GetSize() const {
return m_data_size;
}
} // namespace FileSys

View File

@ -0,0 +1,164 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/alignment.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/fssystem/fs_types.h"
#include "core/file_sys/fssystem/fssystem_alignment_matching_storage.h"
#include "core/file_sys/fssystem/fssystem_integrity_verification_storage.h"
#include "core/file_sys/vfs_offset.h"
namespace FileSys {
struct HierarchicalIntegrityVerificationLevelInformation {
Int64 offset;
Int64 size;
s32 block_order;
std::array<u8, 4> reserved;
};
static_assert(std::is_trivial_v<HierarchicalIntegrityVerificationLevelInformation>);
static_assert(sizeof(HierarchicalIntegrityVerificationLevelInformation) == 0x18);
static_assert(alignof(HierarchicalIntegrityVerificationLevelInformation) == 0x4);
struct HierarchicalIntegrityVerificationInformation {
u32 max_layers;
std::array<HierarchicalIntegrityVerificationLevelInformation, IntegrityMaxLayerCount - 1> info;
HashSalt seed;
s64 GetLayeredHashSize() const {
return this->info[this->max_layers - 2].offset;
}
s64 GetDataOffset() const {
return this->info[this->max_layers - 2].offset;
}
s64 GetDataSize() const {
return this->info[this->max_layers - 2].size;
}
};
static_assert(std::is_trivial_v<HierarchicalIntegrityVerificationInformation>);
struct HierarchicalIntegrityVerificationMetaInformation {
u32 magic;
u32 version;
u32 master_hash_size;
HierarchicalIntegrityVerificationInformation level_hash_info;
};
static_assert(std::is_trivial_v<HierarchicalIntegrityVerificationMetaInformation>);
struct HierarchicalIntegrityVerificationSizeSet {
s64 control_size;
s64 master_hash_size;
std::array<s64, IntegrityMaxLayerCount - 2> layered_hash_sizes;
};
static_assert(std::is_trivial_v<HierarchicalIntegrityVerificationSizeSet>);
class HierarchicalIntegrityVerificationStorage : public IReadOnlyStorage {
YUZU_NON_COPYABLE(HierarchicalIntegrityVerificationStorage);
YUZU_NON_MOVEABLE(HierarchicalIntegrityVerificationStorage);
public:
using GenerateRandomFunction = void (*)(void* dst, size_t size);
class HierarchicalStorageInformation {
public:
enum {
MasterStorage = 0,
Layer1Storage = 1,
Layer2Storage = 2,
Layer3Storage = 3,
Layer4Storage = 4,
Layer5Storage = 5,
DataStorage = 6,
};
private:
std::array<VirtualFile, DataStorage + 1> m_storages;
public:
void SetMasterHashStorage(VirtualFile s) {
m_storages[MasterStorage] = s;
}
void SetLayer1HashStorage(VirtualFile s) {
m_storages[Layer1Storage] = s;
}
void SetLayer2HashStorage(VirtualFile s) {
m_storages[Layer2Storage] = s;
}
void SetLayer3HashStorage(VirtualFile s) {
m_storages[Layer3Storage] = s;
}
void SetLayer4HashStorage(VirtualFile s) {
m_storages[Layer4Storage] = s;
}
void SetLayer5HashStorage(VirtualFile s) {
m_storages[Layer5Storage] = s;
}
void SetDataStorage(VirtualFile s) {
m_storages[DataStorage] = s;
}
VirtualFile& operator[](s32 index) {
ASSERT(MasterStorage <= index && index <= DataStorage);
return m_storages[index];
}
};
public:
HierarchicalIntegrityVerificationStorage();
virtual ~HierarchicalIntegrityVerificationStorage() override {
this->Finalize();
}
Result Initialize(const HierarchicalIntegrityVerificationInformation& info,
HierarchicalStorageInformation storage, int max_data_cache_entries,
int max_hash_cache_entries, s8 buffer_level);
void Finalize();
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override;
virtual size_t GetSize() const override;
bool IsInitialized() const {
return m_data_size >= 0;
}
s64 GetL1HashVerificationBlockSize() const {
return m_verify_storages[m_max_layers - 2]->GetBlockSize();
}
VirtualFile GetL1HashStorage() {
return std::make_shared<OffsetVfsFile>(
m_buffer_storages[m_max_layers - 3],
Common::DivideUp(m_data_size, this->GetL1HashVerificationBlockSize()), 0);
}
public:
static constexpr s8 GetDefaultDataCacheBufferLevel(u32 max_layers) {
return static_cast<s8>(16 + max_layers - 2);
}
protected:
static constexpr s64 HashSize = 256 / 8;
static constexpr size_t MaxLayers = IntegrityMaxLayerCount;
private:
static GenerateRandomFunction s_generate_random;
static void SetGenerateRandomFunction(GenerateRandomFunction func) {
s_generate_random = func;
}
private:
friend struct HierarchicalIntegrityVerificationMetaInformation;
private:
std::array<std::shared_ptr<IntegrityVerificationStorage>, MaxLayers - 1> m_verify_storages;
std::array<VirtualFile, MaxLayers - 1> m_buffer_storages;
s64 m_data_size;
s32 m_max_layers;
};
} // namespace FileSys

View File

@ -0,0 +1,80 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/alignment.h"
#include "common/scope_exit.h"
#include "core/file_sys/fssystem/fssystem_hierarchical_sha256_storage.h"
namespace FileSys {
namespace {
s32 Log2(s32 value) {
ASSERT(value > 0);
ASSERT(Common::IsPowerOfTwo(value));
s32 log = 0;
while ((value >>= 1) > 0) {
++log;
}
return log;
}
} // namespace
Result HierarchicalSha256Storage::Initialize(VirtualFile* base_storages, s32 layer_count,
size_t htbs, void* hash_buf, size_t hash_buf_size) {
// Validate preconditions.
ASSERT(layer_count == LayerCount);
ASSERT(Common::IsPowerOfTwo(htbs));
ASSERT(hash_buf != nullptr);
// Set size tracking members.
m_hash_target_block_size = static_cast<s32>(htbs);
m_log_size_ratio = Log2(m_hash_target_block_size / HashSize);
// Get the base storage size.
m_base_storage_size = base_storages[2]->GetSize();
{
auto size_guard = SCOPE_GUARD({ m_base_storage_size = 0; });
R_UNLESS(m_base_storage_size <= static_cast<s64>(HashSize)
<< m_log_size_ratio << m_log_size_ratio,
ResultHierarchicalSha256BaseStorageTooLarge);
size_guard.Cancel();
}
// Set hash buffer tracking members.
m_base_storage = base_storages[2];
m_hash_buffer = static_cast<char*>(hash_buf);
m_hash_buffer_size = hash_buf_size;
// Read the master hash.
std::array<u8, HashSize> master_hash{};
base_storages[0]->ReadObject(std::addressof(master_hash));
// Read and validate the data being hashed.
s64 hash_storage_size = base_storages[1]->GetSize();
ASSERT(Common::IsAligned(hash_storage_size, HashSize));
ASSERT(hash_storage_size <= m_hash_target_block_size);
ASSERT(hash_storage_size <= static_cast<s64>(m_hash_buffer_size));
base_storages[1]->Read(reinterpret_cast<u8*>(m_hash_buffer),
static_cast<size_t>(hash_storage_size), 0);
R_SUCCEED();
}
size_t HierarchicalSha256Storage::Read(u8* buffer, size_t size, size_t offset) const {
// Succeed if zero-size.
if (size == 0) {
return size;
}
// Validate that we have a buffer to read into.
ASSERT(buffer != nullptr);
// Read the data.
return m_base_storage->Read(buffer, size, offset);
}
} // namespace FileSys

View File

@ -0,0 +1,44 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <mutex>
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/vfs.h"
namespace FileSys {
class HierarchicalSha256Storage : public IReadOnlyStorage {
YUZU_NON_COPYABLE(HierarchicalSha256Storage);
YUZU_NON_MOVEABLE(HierarchicalSha256Storage);
public:
static constexpr s32 LayerCount = 3;
static constexpr size_t HashSize = 256 / 8;
public:
HierarchicalSha256Storage() : m_mutex() {}
Result Initialize(VirtualFile* base_storages, s32 layer_count, size_t htbs, void* hash_buf,
size_t hash_buf_size);
virtual size_t GetSize() const override {
return m_base_storage->GetSize();
}
virtual size_t Read(u8* buffer, size_t length, size_t offset) const override;
private:
VirtualFile m_base_storage;
s64 m_base_storage_size;
char* m_hash_buffer;
size_t m_hash_buffer_size;
s32 m_hash_target_block_size;
s32 m_log_size_ratio;
std::mutex m_mutex;
};
} // namespace FileSys

View File

@ -0,0 +1,119 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fssystem_indirect_storage.h"
namespace FileSys {
Result IndirectStorage::Initialize(VirtualFile table_storage) {
// Read and verify the bucket tree header.
BucketTree::Header header;
table_storage->ReadObject(std::addressof(header));
R_TRY(header.Verify());
// Determine extents.
const auto node_storage_size = QueryNodeStorageSize(header.entry_count);
const auto entry_storage_size = QueryEntryStorageSize(header.entry_count);
const auto node_storage_offset = QueryHeaderStorageSize();
const auto entry_storage_offset = node_storage_offset + node_storage_size;
// Initialize.
R_RETURN(this->Initialize(
std::make_shared<OffsetVfsFile>(table_storage, node_storage_size, node_storage_offset),
std::make_shared<OffsetVfsFile>(table_storage, entry_storage_size, entry_storage_offset),
header.entry_count));
}
void IndirectStorage::Finalize() {
if (this->IsInitialized()) {
m_table.Finalize();
for (auto i = 0; i < StorageCount; i++) {
m_data_storage[i] = VirtualFile();
}
}
}
Result IndirectStorage::GetEntryList(Entry* out_entries, s32* out_entry_count, s32 entry_count,
s64 offset, s64 size) {
// Validate pre-conditions.
ASSERT(offset >= 0);
ASSERT(size >= 0);
ASSERT(this->IsInitialized());
// Clear the out count.
R_UNLESS(out_entry_count != nullptr, ResultNullptrArgument);
*out_entry_count = 0;
// Succeed if there's no range.
R_SUCCEED_IF(size == 0);
// If we have an output array, we need it to be non-null.
R_UNLESS(out_entries != nullptr || entry_count == 0, ResultNullptrArgument);
// Check that our range is valid.
BucketTree::Offsets table_offsets;
R_TRY(m_table.GetOffsets(std::addressof(table_offsets)));
R_UNLESS(table_offsets.IsInclude(offset, size), ResultOutOfRange);
// Find the offset in our tree.
BucketTree::Visitor visitor;
R_TRY(m_table.Find(std::addressof(visitor), offset));
{
const auto entry_offset = visitor.Get<Entry>()->GetVirtualOffset();
R_UNLESS(0 <= entry_offset && table_offsets.IsInclude(entry_offset),
ResultInvalidIndirectEntryOffset);
}
// Prepare to loop over entries.
const auto end_offset = offset + static_cast<s64>(size);
s32 count = 0;
auto cur_entry = *visitor.Get<Entry>();
while (cur_entry.GetVirtualOffset() < end_offset) {
// Try to write the entry to the out list.
if (entry_count != 0) {
if (count >= entry_count) {
break;
}
std::memcpy(out_entries + count, std::addressof(cur_entry), sizeof(Entry));
}
count++;
// Advance.
if (visitor.CanMoveNext()) {
R_TRY(visitor.MoveNext());
cur_entry = *visitor.Get<Entry>();
} else {
break;
}
}
// Write the output count.
*out_entry_count = count;
R_SUCCEED();
}
size_t IndirectStorage::Read(u8* buffer, size_t size, size_t offset) const {
// Validate pre-conditions.
ASSERT(this->IsInitialized());
ASSERT(buffer != nullptr);
// Succeed if there's nothing to read.
if (size == 0) {
return 0;
}
const_cast<IndirectStorage*>(this)->OperatePerEntry<true, true>(
offset, size,
[=](VirtualFile storage, s64 data_offset, s64 cur_offset, s64 cur_size) -> Result {
storage->Read(reinterpret_cast<u8*>(buffer) + (cur_offset - offset),
static_cast<size_t>(cur_size), data_offset);
R_SUCCEED();
});
return size;
}
} // namespace FileSys

View File

@ -0,0 +1,294 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree.h"
#include "core/file_sys/fssystem/fssystem_bucket_tree_template_impl.h"
#include "core/file_sys/vfs.h"
#include "core/file_sys/vfs_offset.h"
namespace FileSys {
class IndirectStorage : public IReadOnlyStorage {
YUZU_NON_COPYABLE(IndirectStorage);
YUZU_NON_MOVEABLE(IndirectStorage);
public:
static constexpr s32 StorageCount = 2;
static constexpr size_t NodeSize = 16_KiB;
struct Entry {
std::array<u8, sizeof(s64)> virt_offset;
std::array<u8, sizeof(s64)> phys_offset;
s32 storage_index;
void SetVirtualOffset(const s64& ofs) {
std::memcpy(this->virt_offset.data(), std::addressof(ofs), sizeof(s64));
}
s64 GetVirtualOffset() const {
s64 offset;
std::memcpy(std::addressof(offset), this->virt_offset.data(), sizeof(s64));
return offset;
}
void SetPhysicalOffset(const s64& ofs) {
std::memcpy(this->phys_offset.data(), std::addressof(ofs), sizeof(s64));
}
s64 GetPhysicalOffset() const {
s64 offset;
std::memcpy(std::addressof(offset), this->phys_offset.data(), sizeof(s64));
return offset;
}
};
static_assert(std::is_trivial_v<Entry>);
static_assert(sizeof(Entry) == 0x14);
struct EntryData {
s64 virt_offset;
s64 phys_offset;
s32 storage_index;
void Set(const Entry& entry) {
this->virt_offset = entry.GetVirtualOffset();
this->phys_offset = entry.GetPhysicalOffset();
this->storage_index = entry.storage_index;
}
};
static_assert(std::is_trivial_v<EntryData>);
public:
IndirectStorage() : m_table(), m_data_storage() {}
virtual ~IndirectStorage() {
this->Finalize();
}
Result Initialize(VirtualFile table_storage);
void Finalize();
bool IsInitialized() const {
return m_table.IsInitialized();
}
Result Initialize(VirtualFile node_storage, VirtualFile entry_storage, s32 entry_count) {
R_RETURN(
m_table.Initialize(node_storage, entry_storage, NodeSize, sizeof(Entry), entry_count));
}
void SetStorage(s32 idx, VirtualFile storage) {
ASSERT(0 <= idx && idx < StorageCount);
m_data_storage[idx] = storage;
}
template <typename T>
void SetStorage(s32 idx, T storage, s64 offset, s64 size) {
ASSERT(0 <= idx && idx < StorageCount);
m_data_storage[idx] = std::make_shared<OffsetVfsFile>(storage, size, offset);
}
Result GetEntryList(Entry* out_entries, s32* out_entry_count, s32 entry_count, s64 offset,
s64 size);
virtual size_t GetSize() const override {
BucketTree::Offsets offsets{};
m_table.GetOffsets(std::addressof(offsets));
return offsets.end_offset;
}
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override;
public:
static constexpr s64 QueryHeaderStorageSize() {
return BucketTree::QueryHeaderStorageSize();
}
static constexpr s64 QueryNodeStorageSize(s32 entry_count) {
return BucketTree::QueryNodeStorageSize(NodeSize, sizeof(Entry), entry_count);
}
static constexpr s64 QueryEntryStorageSize(s32 entry_count) {
return BucketTree::QueryEntryStorageSize(NodeSize, sizeof(Entry), entry_count);
}
protected:
BucketTree& GetEntryTable() {
return m_table;
}
VirtualFile& GetDataStorage(s32 index) {
ASSERT(0 <= index && index < StorageCount);
return m_data_storage[index];
}
template <bool ContinuousCheck, bool RangeCheck, typename F>
Result OperatePerEntry(s64 offset, s64 size, F func);
private:
struct ContinuousReadingEntry {
static constexpr size_t FragmentSizeMax = 4_KiB;
IndirectStorage::Entry entry;
s64 GetVirtualOffset() const {
return this->entry.GetVirtualOffset();
}
s64 GetPhysicalOffset() const {
return this->entry.GetPhysicalOffset();
}
bool IsFragment() const {
return this->entry.storage_index != 0;
}
};
static_assert(std::is_trivial_v<ContinuousReadingEntry>);
private:
mutable BucketTree m_table;
std::array<VirtualFile, StorageCount> m_data_storage;
};
template <bool ContinuousCheck, bool RangeCheck, typename F>
Result IndirectStorage::OperatePerEntry(s64 offset, s64 size, F func) {
// Validate preconditions.
ASSERT(offset >= 0);
ASSERT(size >= 0);
ASSERT(this->IsInitialized());
// Succeed if there's nothing to operate on.
R_SUCCEED_IF(size == 0);
// Get the table offsets.
BucketTree::Offsets table_offsets;
R_TRY(m_table.GetOffsets(std::addressof(table_offsets)));
// Validate arguments.
R_UNLESS(table_offsets.IsInclude(offset, size), ResultOutOfRange);
// Find the offset in our tree.
BucketTree::Visitor visitor;
R_TRY(m_table.Find(std::addressof(visitor), offset));
{
const auto entry_offset = visitor.Get<Entry>()->GetVirtualOffset();
R_UNLESS(0 <= entry_offset && table_offsets.IsInclude(entry_offset),
ResultInvalidIndirectEntryOffset);
}
// Prepare to operate in chunks.
auto cur_offset = offset;
const auto end_offset = offset + static_cast<s64>(size);
BucketTree::ContinuousReadingInfo cr_info;
while (cur_offset < end_offset) {
// Get the current entry.
const auto cur_entry = *visitor.Get<Entry>();
// Get and validate the entry's offset.
const auto cur_entry_offset = cur_entry.GetVirtualOffset();
R_UNLESS(cur_entry_offset <= cur_offset, ResultInvalidIndirectEntryOffset);
// Validate the storage index.
R_UNLESS(0 <= cur_entry.storage_index && cur_entry.storage_index < StorageCount,
ResultInvalidIndirectEntryStorageIndex);
// If we need to check the continuous info, do so.
if constexpr (ContinuousCheck) {
// Scan, if we need to.
if (cr_info.CheckNeedScan()) {
R_TRY(visitor.ScanContinuousReading<ContinuousReadingEntry>(
std::addressof(cr_info), cur_offset,
static_cast<size_t>(end_offset - cur_offset)));
}
// Process a base storage entry.
if (cr_info.CanDo()) {
// Ensure that we can process.
R_UNLESS(cur_entry.storage_index == 0, ResultInvalidIndirectEntryStorageIndex);
// Ensure that we remain within range.
const auto data_offset = cur_offset - cur_entry_offset;
const auto cur_entry_phys_offset = cur_entry.GetPhysicalOffset();
const auto cur_size = static_cast<s64>(cr_info.GetReadSize());
// If we should, verify the range.
if constexpr (RangeCheck) {
// Get the current data storage's size.
s64 cur_data_storage_size = m_data_storage[0]->GetSize();
R_UNLESS(0 <= cur_entry_phys_offset &&
cur_entry_phys_offset <= cur_data_storage_size,
ResultInvalidIndirectEntryOffset);
R_UNLESS(cur_entry_phys_offset + data_offset + cur_size <=
cur_data_storage_size,
ResultInvalidIndirectStorageSize);
}
// Operate.
R_TRY(func(m_data_storage[0], cur_entry_phys_offset + data_offset, cur_offset,
cur_size));
// Mark as done.
cr_info.Done();
}
}
// Get and validate the next entry offset.
s64 next_entry_offset;
if (visitor.CanMoveNext()) {
R_TRY(visitor.MoveNext());
next_entry_offset = visitor.Get<Entry>()->GetVirtualOffset();
R_UNLESS(table_offsets.IsInclude(next_entry_offset), ResultInvalidIndirectEntryOffset);
} else {
next_entry_offset = table_offsets.end_offset;
}
R_UNLESS(cur_offset < next_entry_offset, ResultInvalidIndirectEntryOffset);
// Get the offset of the entry in the data we read.
const auto data_offset = cur_offset - cur_entry_offset;
const auto data_size = (next_entry_offset - cur_entry_offset);
ASSERT(data_size > 0);
// Determine how much is left.
const auto remaining_size = end_offset - cur_offset;
const auto cur_size = std::min<s64>(remaining_size, data_size - data_offset);
ASSERT(cur_size <= size);
// Operate, if we need to.
bool needs_operate;
if constexpr (!ContinuousCheck) {
needs_operate = true;
} else {
needs_operate = !cr_info.IsDone() || cur_entry.storage_index != 0;
}
if (needs_operate) {
const auto cur_entry_phys_offset = cur_entry.GetPhysicalOffset();
if constexpr (RangeCheck) {
// Get the current data storage's size.
s64 cur_data_storage_size = m_data_storage[cur_entry.storage_index]->GetSize();
// Ensure that we remain within range.
R_UNLESS(0 <= cur_entry_phys_offset &&
cur_entry_phys_offset <= cur_data_storage_size,
ResultIndirectStorageCorrupted);
R_UNLESS(cur_entry_phys_offset + data_offset + cur_size <= cur_data_storage_size,
ResultIndirectStorageCorrupted);
}
R_TRY(func(m_data_storage[cur_entry.storage_index], cur_entry_phys_offset + data_offset,
cur_offset, cur_size));
}
cur_offset += cur_size;
}
R_SUCCEED();
}
} // namespace FileSys

View File

@ -0,0 +1,30 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/file_sys/fssystem/fssystem_integrity_romfs_storage.h"
namespace FileSys {
Result IntegrityRomFsStorage::Initialize(
HierarchicalIntegrityVerificationInformation level_hash_info, Hash master_hash,
HierarchicalIntegrityVerificationStorage::HierarchicalStorageInformation storage_info,
int max_data_cache_entries, int max_hash_cache_entries, s8 buffer_level) {
// Set master hash.
m_master_hash = master_hash;
m_master_hash_storage = std::make_shared<ArrayVfsFile<sizeof(Hash)>>(m_master_hash.value);
R_UNLESS(m_master_hash_storage != nullptr,
ResultAllocationMemoryFailedInIntegrityRomFsStorageA);
// Set the master hash storage.
storage_info[0] = m_master_hash_storage;
// Initialize our integrity storage.
R_RETURN(m_integrity_storage.Initialize(level_hash_info, storage_info, max_data_cache_entries,
max_hash_cache_entries, buffer_level));
}
void IntegrityRomFsStorage::Finalize() {
m_integrity_storage.Finalize();
}
} // namespace FileSys

View File

@ -0,0 +1,42 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/fssystem/fssystem_hierarchical_integrity_verification_storage.h"
#include "core/file_sys/fssystem/fssystem_nca_header.h"
#include "core/file_sys/vfs_vector.h"
namespace FileSys {
constexpr inline size_t IntegrityLayerCountRomFs = 7;
constexpr inline size_t IntegrityHashLayerBlockSize = 16_KiB;
class IntegrityRomFsStorage : public IReadOnlyStorage {
public:
IntegrityRomFsStorage() {}
virtual ~IntegrityRomFsStorage() override {
this->Finalize();
}
Result Initialize(
HierarchicalIntegrityVerificationInformation level_hash_info, Hash master_hash,
HierarchicalIntegrityVerificationStorage::HierarchicalStorageInformation storage_info,
int max_data_cache_entries, int max_hash_cache_entries, s8 buffer_level);
void Finalize();
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override {
return m_integrity_storage.Read(buffer, size, offset);
}
virtual size_t GetSize() const override {
return m_integrity_storage.GetSize();
}
private:
HierarchicalIntegrityVerificationStorage m_integrity_storage;
Hash m_master_hash;
std::shared_ptr<ArrayVfsFile<sizeof(Hash)>> m_master_hash_storage;
};
} // namespace FileSys

View File

@ -0,0 +1,91 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/alignment.h"
#include "core/file_sys/fssystem/fssystem_integrity_verification_storage.h"
namespace FileSys {
constexpr inline u32 ILog2(u32 val) {
ASSERT(val > 0);
return static_cast<u32>((sizeof(u32) * 8) - 1 - std::countl_zero<u32>(val));
}
void IntegrityVerificationStorage::Initialize(VirtualFile hs, VirtualFile ds, s64 verif_block_size,
s64 upper_layer_verif_block_size, bool is_real_data) {
// Validate preconditions.
ASSERT(verif_block_size >= HashSize);
// Set storages.
m_hash_storage = hs;
m_data_storage = ds;
// Set verification block sizes.
m_verification_block_size = verif_block_size;
m_verification_block_order = ILog2(static_cast<u32>(verif_block_size));
ASSERT(m_verification_block_size == 1ll << m_verification_block_order);
// Set upper layer block sizes.
upper_layer_verif_block_size = std::max(upper_layer_verif_block_size, HashSize);
m_upper_layer_verification_block_size = upper_layer_verif_block_size;
m_upper_layer_verification_block_order = ILog2(static_cast<u32>(upper_layer_verif_block_size));
ASSERT(m_upper_layer_verification_block_size == 1ll << m_upper_layer_verification_block_order);
// Validate sizes.
{
s64 hash_size = m_hash_storage->GetSize();
s64 data_size = m_data_storage->GetSize();
ASSERT(((hash_size / HashSize) * m_verification_block_size) >= data_size);
}
// Set data.
m_is_real_data = is_real_data;
}
void IntegrityVerificationStorage::Finalize() {
m_hash_storage = VirtualFile();
m_data_storage = VirtualFile();
}
size_t IntegrityVerificationStorage::Read(u8* buffer, size_t size, size_t offset) const {
// Succeed if zero size.
if (size == 0) {
return size;
}
// Validate arguments.
ASSERT(buffer != nullptr);
// Validate the offset.
s64 data_size = m_data_storage->GetSize();
ASSERT(offset <= static_cast<size_t>(data_size));
// Validate the access range.
ASSERT(R_SUCCEEDED(IStorage::CheckAccessRange(
offset, size, Common::AlignUp(data_size, static_cast<size_t>(m_verification_block_size)))));
// Determine the read extents.
size_t read_size = size;
if (static_cast<s64>(offset + read_size) > data_size) {
// Determine the padding sizes.
s64 padding_offset = data_size - offset;
size_t padding_size = static_cast<size_t>(
m_verification_block_size - (padding_offset & (m_verification_block_size - 1)));
ASSERT(static_cast<s64>(padding_size) < m_verification_block_size);
// Clear the padding.
std::memset(static_cast<u8*>(buffer) + padding_offset, 0, padding_size);
// Set the new in-bounds size.
read_size = static_cast<size_t>(data_size - offset);
}
// Perform the read.
return m_data_storage->Read(buffer, read_size, offset);
}
size_t IntegrityVerificationStorage::GetSize() const {
return m_data_storage->GetSize();
}
} // namespace FileSys

View File

@ -0,0 +1,65 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <optional>
#include "core/file_sys/fssystem/fs_i_storage.h"
#include "core/file_sys/fssystem/fs_types.h"
namespace FileSys {
class IntegrityVerificationStorage : public IReadOnlyStorage {
YUZU_NON_COPYABLE(IntegrityVerificationStorage);
YUZU_NON_MOVEABLE(IntegrityVerificationStorage);
public:
static constexpr s64 HashSize = 256 / 8;
struct BlockHash {
std::array<u8, HashSize> hash;
};
static_assert(std::is_trivial_v<BlockHash>);
public:
IntegrityVerificationStorage()
: m_verification_block_size(0), m_verification_block_order(0),
m_upper_layer_verification_block_size(0), m_upper_layer_verification_block_order(0) {}
virtual ~IntegrityVerificationStorage() override {
this->Finalize();
}
void Initialize(VirtualFile hs, VirtualFile ds, s64 verif_block_size,
s64 upper_layer_verif_block_size, bool is_real_data);
void Finalize();
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override;
virtual size_t GetSize() const override;
s64 GetBlockSize() const {
return m_verification_block_size;
}
private:
static void SetValidationBit(BlockHash* hash) {
ASSERT(hash != nullptr);
hash->hash[HashSize - 1] |= 0x80;
}
static bool IsValidationBit(const BlockHash* hash) {
ASSERT(hash != nullptr);
return (hash->hash[HashSize - 1] & 0x80) != 0;
}
private:
VirtualFile m_hash_storage;
VirtualFile m_data_storage;
s64 m_verification_block_size;
s64 m_verification_block_order;
s64 m_upper_layer_verification_block_size;
s64 m_upper_layer_verification_block_order;
bool m_is_real_data;
};
} // namespace FileSys

View File

@ -0,0 +1,61 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/fssystem/fs_i_storage.h"
namespace FileSys {
class MemoryResourceBufferHoldStorage : public IStorage {
YUZU_NON_COPYABLE(MemoryResourceBufferHoldStorage);
YUZU_NON_MOVEABLE(MemoryResourceBufferHoldStorage);
public:
MemoryResourceBufferHoldStorage(VirtualFile storage, size_t buffer_size)
: m_storage(std::move(storage)), m_buffer(::operator new(buffer_size)),
m_buffer_size(buffer_size) {}
virtual ~MemoryResourceBufferHoldStorage() {
// If we have a buffer, deallocate it.
if (m_buffer != nullptr) {
::operator delete(m_buffer);
}
}
bool IsValid() const {
return m_buffer != nullptr;
}
void* GetBuffer() const {
return m_buffer;
}
public:
virtual size_t Read(u8* buffer, size_t size, size_t offset) const override {
// Check pre-conditions.
ASSERT(m_storage != nullptr);
return m_storage->Read(buffer, size, offset);
}
virtual size_t GetSize() const override {
// Check pre-conditions.
ASSERT(m_storage != nullptr);
return m_storage->GetSize();
}
virtual size_t Write(const u8* buffer, size_t size, size_t offset) override {
// Check pre-conditions.
ASSERT(m_storage != nullptr);
return m_storage->Write(buffer, size, offset);
}
private:
VirtualFile m_storage;
void* m_buffer;
size_t m_buffer_size;
};
} // namespace FileSys

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,364 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "core/file_sys/fssystem/fssystem_compression_common.h"
#include "core/file_sys/fssystem/fssystem_nca_header.h"
#include "core/file_sys/vfs.h"
namespace FileSys {
class CompressedStorage;
class AesCtrCounterExtendedStorage;
class IndirectStorage;
class SparseStorage;
struct NcaCryptoConfiguration;
using KeyGenerationFunction = void (*)(void* dst_key, size_t dst_key_size, const void* src_key,
size_t src_key_size, s32 key_type);
using VerifySign1Function = bool (*)(const void* sig, size_t sig_size, const void* data,
size_t data_size, u8 generation);
struct NcaCryptoConfiguration {
static constexpr size_t Rsa2048KeyModulusSize = 2048 / 8;
static constexpr size_t Rsa2048KeyPublicExponentSize = 3;
static constexpr size_t Rsa2048KeyPrivateExponentSize = Rsa2048KeyModulusSize;
static constexpr size_t Aes128KeySize = 128 / 8;
static constexpr size_t Header1SignatureKeyGenerationMax = 1;
static constexpr s32 KeyAreaEncryptionKeyIndexCount = 3;
static constexpr s32 HeaderEncryptionKeyCount = 2;
static constexpr u8 KeyAreaEncryptionKeyIndexZeroKey = 0xFF;
static constexpr size_t KeyGenerationMax = 32;
std::array<const u8*, Header1SignatureKeyGenerationMax + 1> header_1_sign_key_moduli;
std::array<u8, Rsa2048KeyPublicExponentSize> header_1_sign_key_public_exponent;
std::array<std::array<u8, Aes128KeySize>, KeyAreaEncryptionKeyIndexCount>
key_area_encryption_key_source;
std::array<u8, Aes128KeySize> header_encryption_key_source;
std::array<std::array<u8, Aes128KeySize>, HeaderEncryptionKeyCount>
header_encrypted_encryption_keys;
KeyGenerationFunction generate_key;
VerifySign1Function verify_sign1;
bool is_plaintext_header_available;
bool is_available_sw_key;
};
static_assert(std::is_trivial_v<NcaCryptoConfiguration>);
struct NcaCompressionConfiguration {
GetDecompressorFunction get_decompressor;
};
static_assert(std::is_trivial_v<NcaCompressionConfiguration>);
constexpr inline s32 KeyAreaEncryptionKeyCount =
NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexCount *
NcaCryptoConfiguration::KeyGenerationMax;
enum class KeyType : s32 {
ZeroKey = -2,
InvalidKey = -1,
NcaHeaderKey1 = KeyAreaEncryptionKeyCount + 0,
NcaHeaderKey2 = KeyAreaEncryptionKeyCount + 1,
NcaExternalKey = KeyAreaEncryptionKeyCount + 2,
SaveDataDeviceUniqueMac = KeyAreaEncryptionKeyCount + 3,
SaveDataSeedUniqueMac = KeyAreaEncryptionKeyCount + 4,
SaveDataTransferMac = KeyAreaEncryptionKeyCount + 5,
};
constexpr inline bool IsInvalidKeyTypeValue(s32 key_type) {
return key_type < 0;
}
constexpr inline s32 GetKeyTypeValue(u8 key_index, u8 key_generation) {
if (key_index == NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexZeroKey) {
return static_cast<s32>(KeyType::ZeroKey);
}
if (key_index >= NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexCount) {
return static_cast<s32>(KeyType::InvalidKey);
}
return NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexCount * key_generation + key_index;
}
class NcaReader {
YUZU_NON_COPYABLE(NcaReader);
YUZU_NON_MOVEABLE(NcaReader);
public:
NcaReader();
~NcaReader();
Result Initialize(VirtualFile base_storage, const NcaCryptoConfiguration& crypto_cfg,
const NcaCompressionConfiguration& compression_cfg);
VirtualFile GetSharedBodyStorage();
u32 GetMagic() const;
NcaHeader::DistributionType GetDistributionType() const;
NcaHeader::ContentType GetContentType() const;
u8 GetHeaderSign1KeyGeneration() const;
u8 GetKeyGeneration() const;
u8 GetKeyIndex() const;
u64 GetContentSize() const;
u64 GetProgramId() const;
u32 GetContentIndex() const;
u32 GetSdkAddonVersion() const;
void GetRightsId(u8* dst, size_t dst_size) const;
bool HasFsInfo(s32 index) const;
s32 GetFsCount() const;
const Hash& GetFsHeaderHash(s32 index) const;
void GetFsHeaderHash(Hash* dst, s32 index) const;
void GetFsInfo(NcaHeader::FsInfo* dst, s32 index) const;
u64 GetFsOffset(s32 index) const;
u64 GetFsEndOffset(s32 index) const;
u64 GetFsSize(s32 index) const;
void GetEncryptedKey(void* dst, size_t size) const;
const void* GetDecryptionKey(s32 index) const;
bool HasValidInternalKey() const;
bool HasInternalDecryptionKeyForAesHw() const;
bool IsSoftwareAesPrioritized() const;
void PrioritizeSoftwareAes();
bool IsAvailableSwKey() const;
bool HasExternalDecryptionKey() const;
const void* GetExternalDecryptionKey() const;
void SetExternalDecryptionKey(const void* src, size_t size);
void GetRawData(void* dst, size_t dst_size) const;
NcaHeader::EncryptionType GetEncryptionType() const;
Result ReadHeader(NcaFsHeader* dst, s32 index) const;
GetDecompressorFunction GetDecompressor() const;
bool GetHeaderSign1Valid() const;
void GetHeaderSign2(void* dst, size_t size) const;
private:
NcaHeader m_header;
std::array<std::array<u8, NcaCryptoConfiguration::Aes128KeySize>,
NcaHeader::DecryptionKey_Count>
m_decryption_keys;
VirtualFile m_body_storage;
VirtualFile m_header_storage;
std::array<u8, NcaCryptoConfiguration::Aes128KeySize> m_external_decryption_key;
bool m_is_software_aes_prioritized;
bool m_is_available_sw_key;
NcaHeader::EncryptionType m_header_encryption_type;
bool m_is_header_sign1_signature_valid;
GetDecompressorFunction m_get_decompressor;
};
class NcaFsHeaderReader {
YUZU_NON_COPYABLE(NcaFsHeaderReader);
YUZU_NON_MOVEABLE(NcaFsHeaderReader);
public:
NcaFsHeaderReader() : m_fs_index(-1) {
std::memset(std::addressof(m_data), 0, sizeof(m_data));
}
Result Initialize(const NcaReader& reader, s32 index);
bool IsInitialized() const {
return m_fs_index >= 0;
}
void GetRawData(void* dst, size_t dst_size) const;
NcaFsHeader::HashData& GetHashData();
const NcaFsHeader::HashData& GetHashData() const;
u16 GetVersion() const;
s32 GetFsIndex() const;
NcaFsHeader::FsType GetFsType() const;
NcaFsHeader::HashType GetHashType() const;
NcaFsHeader::EncryptionType GetEncryptionType() const;
NcaPatchInfo& GetPatchInfo();
const NcaPatchInfo& GetPatchInfo() const;
const NcaAesCtrUpperIv GetAesCtrUpperIv() const;
bool IsSkipLayerHashEncryption() const;
Result GetHashTargetOffset(s64* out) const;
bool ExistsSparseLayer() const;
NcaSparseInfo& GetSparseInfo();
const NcaSparseInfo& GetSparseInfo() const;
bool ExistsCompressionLayer() const;
NcaCompressionInfo& GetCompressionInfo();
const NcaCompressionInfo& GetCompressionInfo() const;
bool ExistsPatchMetaHashLayer() const;
NcaMetaDataHashDataInfo& GetPatchMetaDataHashDataInfo();
const NcaMetaDataHashDataInfo& GetPatchMetaDataHashDataInfo() const;
NcaFsHeader::MetaDataHashType GetPatchMetaHashType() const;
bool ExistsSparseMetaHashLayer() const;
NcaMetaDataHashDataInfo& GetSparseMetaDataHashDataInfo();
const NcaMetaDataHashDataInfo& GetSparseMetaDataHashDataInfo() const;
NcaFsHeader::MetaDataHashType GetSparseMetaHashType() const;
private:
NcaFsHeader m_data;
s32 m_fs_index;
};
class NcaFileSystemDriver {
YUZU_NON_COPYABLE(NcaFileSystemDriver);
YUZU_NON_MOVEABLE(NcaFileSystemDriver);
public:
struct StorageContext {
bool open_raw_storage;
VirtualFile body_substorage;
std::shared_ptr<SparseStorage> current_sparse_storage;
VirtualFile sparse_storage_meta_storage;
std::shared_ptr<SparseStorage> original_sparse_storage;
void* external_current_sparse_storage;
void* external_original_sparse_storage;
VirtualFile aes_ctr_ex_storage_meta_storage;
VirtualFile aes_ctr_ex_storage_data_storage;
std::shared_ptr<AesCtrCounterExtendedStorage> aes_ctr_ex_storage;
VirtualFile indirect_storage_meta_storage;
std::shared_ptr<IndirectStorage> indirect_storage;
VirtualFile fs_data_storage;
VirtualFile compressed_storage_meta_storage;
std::shared_ptr<CompressedStorage> compressed_storage;
VirtualFile patch_layer_info_storage;
VirtualFile sparse_layer_info_storage;
VirtualFile external_original_storage;
};
private:
enum class AlignmentStorageRequirement {
CacheBlockSize = 0,
None = 1,
};
public:
static Result SetupFsHeaderReader(NcaFsHeaderReader* out, const NcaReader& reader,
s32 fs_index);
public:
NcaFileSystemDriver(std::shared_ptr<NcaReader> reader) : m_original_reader(), m_reader(reader) {
ASSERT(m_reader != nullptr);
}
NcaFileSystemDriver(std::shared_ptr<NcaReader> original_reader,
std::shared_ptr<NcaReader> reader)
: m_original_reader(original_reader), m_reader(reader) {
ASSERT(m_reader != nullptr);
}
Result OpenStorageWithContext(VirtualFile* out, NcaFsHeaderReader* out_header_reader,
s32 fs_index, StorageContext* ctx);
Result OpenStorage(VirtualFile* out, NcaFsHeaderReader* out_header_reader, s32 fs_index) {
// Create a storage context.
StorageContext ctx{};
// Open the storage.
R_RETURN(OpenStorageWithContext(out, out_header_reader, fs_index, std::addressof(ctx)));
}
public:
Result CreateStorageByRawStorage(VirtualFile* out, const NcaFsHeaderReader* header_reader,
VirtualFile raw_storage, StorageContext* ctx);
private:
Result OpenStorageImpl(VirtualFile* out, NcaFsHeaderReader* out_header_reader, s32 fs_index,
StorageContext* ctx);
Result OpenIndirectableStorageAsOriginal(VirtualFile* out,
const NcaFsHeaderReader* header_reader,
StorageContext* ctx);
Result CreateBodySubStorage(VirtualFile* out, s64 offset, s64 size);
Result CreateAesCtrStorage(VirtualFile* out, VirtualFile base_storage, s64 offset,
const NcaAesCtrUpperIv& upper_iv,
AlignmentStorageRequirement alignment_storage_requirement);
Result CreateAesXtsStorage(VirtualFile* out, VirtualFile base_storage, s64 offset);
Result CreateSparseStorageMetaStorage(VirtualFile* out, VirtualFile base_storage, s64 offset,
const NcaAesCtrUpperIv& upper_iv,
const NcaSparseInfo& sparse_info);
Result CreateSparseStorageCore(std::shared_ptr<SparseStorage>* out, VirtualFile base_storage,
s64 base_size, VirtualFile meta_storage,
const NcaSparseInfo& sparse_info, bool external_info);
Result CreateSparseStorage(VirtualFile* out, s64* out_fs_data_offset,
std::shared_ptr<SparseStorage>* out_sparse_storage,
VirtualFile* out_meta_storage, s32 index,
const NcaAesCtrUpperIv& upper_iv, const NcaSparseInfo& sparse_info);
Result CreateSparseStorageMetaStorageWithVerification(
VirtualFile* out, VirtualFile* out_verification, VirtualFile base_storage, s64 offset,
const NcaAesCtrUpperIv& upper_iv, const NcaSparseInfo& sparse_info,
const NcaMetaDataHashDataInfo& meta_data_hash_data_info);
Result CreateSparseStorageWithVerification(
VirtualFile* out, s64* out_fs_data_offset,
std::shared_ptr<SparseStorage>* out_sparse_storage, VirtualFile* out_meta_storage,
VirtualFile* out_verification, s32 index, const NcaAesCtrUpperIv& upper_iv,
const NcaSparseInfo& sparse_info, const NcaMetaDataHashDataInfo& meta_data_hash_data_info,
NcaFsHeader::MetaDataHashType meta_data_hash_type);
Result CreateAesCtrExStorageMetaStorage(VirtualFile* out, VirtualFile base_storage, s64 offset,
NcaFsHeader::EncryptionType encryption_type,
const NcaAesCtrUpperIv& upper_iv,
const NcaPatchInfo& patch_info);
Result CreateAesCtrExStorage(VirtualFile* out,
std::shared_ptr<AesCtrCounterExtendedStorage>* out_ext,
VirtualFile base_storage, VirtualFile meta_storage,
s64 counter_offset, const NcaAesCtrUpperIv& upper_iv,
const NcaPatchInfo& patch_info);
Result CreateIndirectStorageMetaStorage(VirtualFile* out, VirtualFile base_storage,
const NcaPatchInfo& patch_info);
Result CreateIndirectStorage(VirtualFile* out, std::shared_ptr<IndirectStorage>* out_ind,
VirtualFile base_storage, VirtualFile original_data_storage,
VirtualFile meta_storage, const NcaPatchInfo& patch_info);
Result CreatePatchMetaStorage(VirtualFile* out_aes_ctr_ex_meta, VirtualFile* out_indirect_meta,
VirtualFile* out_verification, VirtualFile base_storage,
s64 offset, const NcaAesCtrUpperIv& upper_iv,
const NcaPatchInfo& patch_info,
const NcaMetaDataHashDataInfo& meta_data_hash_data_info);
Result CreateSha256Storage(VirtualFile* out, VirtualFile base_storage,
const NcaFsHeader::HashData::HierarchicalSha256Data& sha256_data);
Result CreateIntegrityVerificationStorage(
VirtualFile* out, VirtualFile base_storage,
const NcaFsHeader::HashData::IntegrityMetaInfo& meta_info);
Result CreateIntegrityVerificationStorageForMeta(
VirtualFile* out, VirtualFile* out_verification, VirtualFile base_storage, s64 offset,
const NcaMetaDataHashDataInfo& meta_data_hash_data_info);
Result CreateIntegrityVerificationStorageImpl(
VirtualFile* out, VirtualFile base_storage,
const NcaFsHeader::HashData::IntegrityMetaInfo& meta_info, s64 layer_info_offset,
int max_data_cache_entries, int max_hash_cache_entries, s8 buffer_level);
Result CreateRegionSwitchStorage(VirtualFile* out, const NcaFsHeaderReader* header_reader,
VirtualFile inside_storage, VirtualFile outside_storage);
Result CreateCompressedStorage(VirtualFile* out, std::shared_ptr<CompressedStorage>* out_cmp,
VirtualFile* out_meta, VirtualFile base_storage,
const NcaCompressionInfo& compression_info);
public:
Result CreateCompressedStorage(VirtualFile* out, std::shared_ptr<CompressedStorage>* out_cmp,
VirtualFile* out_meta, VirtualFile base_storage,
const NcaCompressionInfo& compression_info,
GetDecompressorFunction get_decompressor);
private:
std::shared_ptr<NcaReader> m_original_reader;
std::shared_ptr<NcaReader> m_reader;
};
} // namespace FileSys

View File

@ -0,0 +1,20 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/file_sys/fssystem/fssystem_nca_header.h"
namespace FileSys {
u8 NcaHeader::GetProperKeyGeneration() const {
return std::max(this->key_generation, this->key_generation_2);
}
bool NcaPatchInfo::HasIndirectTable() const {
return this->indirect_size != 0;
}
bool NcaPatchInfo::HasAesCtrExTable() const {
return this->aes_ctr_ex_size != 0;
}
} // namespace FileSys

View File

@ -0,0 +1,338 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/common_funcs.h"
#include "common/common_types.h"
#include "common/literals.h"
#include "core/file_sys/errors.h"
#include "core/file_sys/fssystem/fs_types.h"
namespace FileSys {
using namespace Common::Literals;
struct Hash {
static constexpr std::size_t Size = 256 / 8;
std::array<u8, Size> value;
};
static_assert(sizeof(Hash) == Hash::Size);
static_assert(std::is_trivial_v<Hash>);
using NcaDigest = Hash;
struct NcaHeader {
enum class ContentType : u8 {
Program = 0,
Meta = 1,
Control = 2,
Manual = 3,
Data = 4,
PublicData = 5,
Start = Program,
End = PublicData,
};
enum class DistributionType : u8 {
Download = 0,
GameCard = 1,
Start = Download,
End = GameCard,
};
enum class EncryptionType : u8 {
Auto = 0,
None = 1,
};
enum DecryptionKey {
DecryptionKey_AesXts = 0,
DecryptionKey_AesXts1 = DecryptionKey_AesXts,
DecryptionKey_AesXts2 = 1,
DecryptionKey_AesCtr = 2,
DecryptionKey_AesCtrEx = 3,
DecryptionKey_AesCtrHw = 4,
DecryptionKey_Count,
};
struct FsInfo {
u32 start_sector;
u32 end_sector;
u32 hash_sectors;
u32 reserved;
};
static_assert(sizeof(FsInfo) == 0x10);
static_assert(std::is_trivial_v<FsInfo>);
static constexpr u32 Magic0 = Common::MakeMagic('N', 'C', 'A', '0');
static constexpr u32 Magic1 = Common::MakeMagic('N', 'C', 'A', '1');
static constexpr u32 Magic2 = Common::MakeMagic('N', 'C', 'A', '2');
static constexpr u32 Magic3 = Common::MakeMagic('N', 'C', 'A', '3');
static constexpr u32 Magic = Magic3;
static constexpr std::size_t Size = 1_KiB;
static constexpr s32 FsCountMax = 4;
static constexpr std::size_t HeaderSignCount = 2;
static constexpr std::size_t HeaderSignSize = 0x100;
static constexpr std::size_t EncryptedKeyAreaSize = 0x100;
static constexpr std::size_t SectorSize = 0x200;
static constexpr std::size_t SectorShift = 9;
static constexpr std::size_t RightsIdSize = 0x10;
static constexpr std::size_t XtsBlockSize = 0x200;
static constexpr std::size_t CtrBlockSize = 0x10;
static_assert(SectorSize == (1 << SectorShift));
// Data members.
std::array<u8, HeaderSignSize> header_sign_1;
std::array<u8, HeaderSignSize> header_sign_2;
u32 magic;
DistributionType distribution_type;
ContentType content_type;
u8 key_generation;
u8 key_index;
u64 content_size;
u64 program_id;
u32 content_index;
u32 sdk_addon_version;
u8 key_generation_2;
u8 header1_signature_key_generation;
std::array<u8, 2> reserved_222;
std::array<u32, 3> reserved_224;
std::array<u8, RightsIdSize> rights_id;
std::array<FsInfo, FsCountMax> fs_info;
std::array<Hash, FsCountMax> fs_header_hash;
std::array<u8, EncryptedKeyAreaSize> encrypted_key_area;
static constexpr u64 SectorToByte(u32 sector) {
return static_cast<u64>(sector) << SectorShift;
}
static constexpr u32 ByteToSector(u64 byte) {
return static_cast<u32>(byte >> SectorShift);
}
u8 GetProperKeyGeneration() const;
};
static_assert(sizeof(NcaHeader) == NcaHeader::Size);
static_assert(std::is_trivial_v<NcaHeader>);
struct NcaBucketInfo {
static constexpr size_t HeaderSize = 0x10;
Int64 offset;
Int64 size;
std::array<u8, HeaderSize> header;
};
static_assert(std::is_trivial_v<NcaBucketInfo>);
struct NcaPatchInfo {
static constexpr size_t Size = 0x40;
static constexpr size_t Offset = 0x100;
Int64 indirect_offset;
Int64 indirect_size;
std::array<u8, NcaBucketInfo::HeaderSize> indirect_header;
Int64 aes_ctr_ex_offset;
Int64 aes_ctr_ex_size;
std::array<u8, NcaBucketInfo::HeaderSize> aes_ctr_ex_header;
bool HasIndirectTable() const;
bool HasAesCtrExTable() const;
};
static_assert(std::is_trivial_v<NcaPatchInfo>);
union NcaAesCtrUpperIv {
u64 value;
struct {
u32 generation;
u32 secure_value;
} part;
};
static_assert(std::is_trivial_v<NcaAesCtrUpperIv>);
struct NcaSparseInfo {
NcaBucketInfo bucket;
Int64 physical_offset;
u16 generation;
std::array<u8, 6> reserved;
s64 GetPhysicalSize() const {
return this->bucket.offset + this->bucket.size;
}
u32 GetGeneration() const {
return static_cast<u32>(this->generation) << 16;
}
const NcaAesCtrUpperIv MakeAesCtrUpperIv(NcaAesCtrUpperIv upper_iv) const {
NcaAesCtrUpperIv sparse_upper_iv = upper_iv;
sparse_upper_iv.part.generation = this->GetGeneration();
return sparse_upper_iv;
}
};
static_assert(std::is_trivial_v<NcaSparseInfo>);
struct NcaCompressionInfo {
NcaBucketInfo bucket;
std::array<u8, 8> resreved;
};
static_assert(std::is_trivial_v<NcaCompressionInfo>);
struct NcaMetaDataHashDataInfo {
Int64 offset;
Int64 size;
Hash hash;
};
static_assert(std::is_trivial_v<NcaMetaDataHashDataInfo>);
struct NcaFsHeader {
static constexpr size_t Size = 0x200;
static constexpr size_t HashDataOffset = 0x8;
struct Region {
Int64 offset;
Int64 size;
};
static_assert(std::is_trivial_v<Region>);
enum class FsType : u8 {
RomFs = 0,
PartitionFs = 1,
};
enum class EncryptionType : u8 {
Auto = 0,
None = 1,
AesXts = 2,
AesCtr = 3,
AesCtrEx = 4,
AesCtrSkipLayerHash = 5,
AesCtrExSkipLayerHash = 6,
};
enum class HashType : u8 {
Auto = 0,
None = 1,
HierarchicalSha256Hash = 2,
HierarchicalIntegrityHash = 3,
AutoSha3 = 4,
HierarchicalSha3256Hash = 5,
HierarchicalIntegritySha3Hash = 6,
};
enum class MetaDataHashType : u8 {
None = 0,
HierarchicalIntegrity = 1,
};
union HashData {
struct HierarchicalSha256Data {
static constexpr size_t HashLayerCountMax = 5;
static const size_t MasterHashOffset;
Hash fs_data_master_hash;
s32 hash_block_size;
s32 hash_layer_count;
std::array<Region, HashLayerCountMax> hash_layer_region;
} hierarchical_sha256_data;
static_assert(std::is_trivial_v<HierarchicalSha256Data>);
struct IntegrityMetaInfo {
static const size_t MasterHashOffset;
u32 magic;
u32 version;
u32 master_hash_size;
struct LevelHashInfo {
u32 max_layers;
struct HierarchicalIntegrityVerificationLevelInformation {
static constexpr size_t IntegrityMaxLayerCount = 7;
Int64 offset;
Int64 size;
s32 block_order;
std::array<u8, 4> reserved;
};
std::array<
HierarchicalIntegrityVerificationLevelInformation,
HierarchicalIntegrityVerificationLevelInformation::IntegrityMaxLayerCount - 1>
info;
struct SignatureSalt {
static constexpr size_t Size = 0x20;
std::array<u8, Size> value;
};
SignatureSalt seed;
} level_hash_info;
Hash master_hash;
} integrity_meta_info;
static_assert(std::is_trivial_v<IntegrityMetaInfo>);
std::array<u8, NcaPatchInfo::Offset - HashDataOffset> padding;
};
u16 version;
FsType fs_type;
HashType hash_type;
EncryptionType encryption_type;
MetaDataHashType meta_data_hash_type;
std::array<u8, 2> reserved;
HashData hash_data;
NcaPatchInfo patch_info;
NcaAesCtrUpperIv aes_ctr_upper_iv;
NcaSparseInfo sparse_info;
NcaCompressionInfo compression_info;
NcaMetaDataHashDataInfo meta_data_hash_data_info;
std::array<u8, 0x30> pad;
bool IsSkipLayerHashEncryption() const {
return this->encryption_type == EncryptionType::AesCtrSkipLayerHash ||
this->encryption_type == EncryptionType::AesCtrExSkipLayerHash;
}
Result GetHashTargetOffset(s64* out) const {
switch (this->hash_type) {
case HashType::HierarchicalIntegrityHash:
case HashType::HierarchicalIntegritySha3Hash:
*out = this->hash_data.integrity_meta_info.level_hash_info
.info[this->hash_data.integrity_meta_info.level_hash_info.max_layers - 2]
.offset;
R_SUCCEED();
case HashType::HierarchicalSha256Hash:
case HashType::HierarchicalSha3256Hash:
*out =
this->hash_data.hierarchical_sha256_data
.hash_layer_region[this->hash_data.hierarchical_sha256_data.hash_layer_count -
1]
.offset;
R_SUCCEED();
default:
R_THROW(ResultInvalidNcaFsHeader);
}
}
};
static_assert(sizeof(NcaFsHeader) == NcaFsHeader::Size);
static_assert(std::is_trivial_v<NcaFsHeader>);
static_assert(offsetof(NcaFsHeader, patch_info) == NcaPatchInfo::Offset);
inline constexpr const size_t NcaFsHeader::HashData::HierarchicalSha256Data::MasterHashOffset =
offsetof(NcaFsHeader, hash_data.hierarchical_sha256_data.fs_data_master_hash);
inline constexpr const size_t NcaFsHeader::HashData::IntegrityMetaInfo::MasterHashOffset =
offsetof(NcaFsHeader, hash_data.integrity_meta_info.master_hash);
struct NcaMetaDataHashData {
s64 layer_info_offset;
NcaFsHeader::HashData::IntegrityMetaInfo integrity_meta_info;
};
static_assert(sizeof(NcaMetaDataHashData) ==
sizeof(NcaFsHeader::HashData::IntegrityMetaInfo) + sizeof(s64));
static_assert(std::is_trivial_v<NcaMetaDataHashData>);
} // namespace FileSys

View File

@ -0,0 +1,531 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "core/file_sys/fssystem/fssystem_aes_xts_storage.h"
#include "core/file_sys/fssystem/fssystem_nca_file_system_driver.h"
#include "core/file_sys/vfs_offset.h"
namespace FileSys {
namespace {
constexpr inline u32 SdkAddonVersionMin = 0x000B0000;
constexpr inline size_t Aes128KeySize = 0x10;
constexpr const std::array<u8, Aes128KeySize> ZeroKey{};
constexpr Result CheckNcaMagic(u32 magic) {
// Verify the magic is not a deprecated one.
R_UNLESS(magic != NcaHeader::Magic0, ResultUnsupportedSdkVersion);
R_UNLESS(magic != NcaHeader::Magic1, ResultUnsupportedSdkVersion);
R_UNLESS(magic != NcaHeader::Magic2, ResultUnsupportedSdkVersion);
// Verify the magic is the current one.
R_UNLESS(magic == NcaHeader::Magic3, ResultInvalidNcaSignature);
R_SUCCEED();
}
} // namespace
NcaReader::NcaReader()
: m_body_storage(), m_header_storage(), m_is_software_aes_prioritized(false),
m_is_available_sw_key(false), m_header_encryption_type(NcaHeader::EncryptionType::Auto),
m_get_decompressor() {
std::memset(std::addressof(m_header), 0, sizeof(m_header));
std::memset(std::addressof(m_decryption_keys), 0, sizeof(m_decryption_keys));
std::memset(std::addressof(m_external_decryption_key), 0, sizeof(m_external_decryption_key));
}
NcaReader::~NcaReader() {}
Result NcaReader::Initialize(VirtualFile base_storage, const NcaCryptoConfiguration& crypto_cfg,
const NcaCompressionConfiguration& compression_cfg) {
// Validate preconditions.
ASSERT(base_storage != nullptr);
ASSERT(m_body_storage == nullptr);
// Create the work header storage storage.
VirtualFile work_header_storage;
// We need to be able to generate keys.
R_UNLESS(crypto_cfg.generate_key != nullptr, ResultInvalidArgument);
// Generate keys for header.
using AesXtsStorageForNcaHeader = AesXtsStorage;
constexpr std::array<s32, NcaCryptoConfiguration::HeaderEncryptionKeyCount>
HeaderKeyTypeValues = {
static_cast<s32>(KeyType::NcaHeaderKey1),
static_cast<s32>(KeyType::NcaHeaderKey2),
};
std::array<std::array<u8, NcaCryptoConfiguration::Aes128KeySize>,
NcaCryptoConfiguration::HeaderEncryptionKeyCount>
header_decryption_keys;
for (size_t i = 0; i < NcaCryptoConfiguration::HeaderEncryptionKeyCount; i++) {
crypto_cfg.generate_key(header_decryption_keys[i].data(),
AesXtsStorageForNcaHeader::KeySize,
crypto_cfg.header_encrypted_encryption_keys[i].data(),
AesXtsStorageForNcaHeader::KeySize, HeaderKeyTypeValues[i]);
}
// Create the header storage.
std::array<u8, AesXtsStorageForNcaHeader::IvSize> header_iv = {};
work_header_storage = std::make_unique<AesXtsStorageForNcaHeader>(
base_storage, header_decryption_keys[0].data(), header_decryption_keys[1].data(),
AesXtsStorageForNcaHeader::KeySize, header_iv.data(), AesXtsStorageForNcaHeader::IvSize,
NcaHeader::XtsBlockSize);
// Check that we successfully created the storage.
R_UNLESS(work_header_storage != nullptr, ResultAllocationMemoryFailedInNcaReaderA);
// Read the header.
work_header_storage->ReadObject(std::addressof(m_header), 0);
// Validate the magic.
if (const Result magic_result = CheckNcaMagic(m_header.magic); R_FAILED(magic_result)) {
// Try to use a plaintext header.
base_storage->ReadObject(std::addressof(m_header), 0);
R_UNLESS(R_SUCCEEDED(CheckNcaMagic(m_header.magic)), magic_result);
// Configure to use the plaintext header.
auto base_storage_size = base_storage->GetSize();
work_header_storage = std::make_shared<OffsetVfsFile>(base_storage, base_storage_size, 0);
R_UNLESS(work_header_storage != nullptr, ResultAllocationMemoryFailedInNcaReaderA);
// Set encryption type as plaintext.
m_header_encryption_type = NcaHeader::EncryptionType::None;
}
// Verify the header sign1.
if (crypto_cfg.verify_sign1 != nullptr) {
const u8* sig = m_header.header_sign_1.data();
const size_t sig_size = NcaHeader::HeaderSignSize;
const u8* msg =
static_cast<const u8*>(static_cast<const void*>(std::addressof(m_header.magic)));
const size_t msg_size =
NcaHeader::Size - NcaHeader::HeaderSignSize * NcaHeader::HeaderSignCount;
m_is_header_sign1_signature_valid = crypto_cfg.verify_sign1(
sig, sig_size, msg, msg_size, m_header.header1_signature_key_generation);
if (!m_is_header_sign1_signature_valid) {
LOG_WARNING(Common_Filesystem, "Invalid NCA header sign1");
}
}
// Validate the sdk version.
R_UNLESS(m_header.sdk_addon_version >= SdkAddonVersionMin, ResultUnsupportedSdkVersion);
// Validate the key index.
R_UNLESS(m_header.key_index < NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexCount ||
m_header.key_index == NcaCryptoConfiguration::KeyAreaEncryptionKeyIndexZeroKey,
ResultInvalidNcaKeyIndex);
// Check if we have a rights id.
constexpr const std::array<u8, NcaHeader::RightsIdSize> ZeroRightsId{};
if (std::memcmp(ZeroRightsId.data(), m_header.rights_id.data(), NcaHeader::RightsIdSize) == 0) {
// If we don't, then we don't have an external key, so we need to generate decryption keys.
crypto_cfg.generate_key(
m_decryption_keys[NcaHeader::DecryptionKey_AesCtr].data(), Aes128KeySize,
m_header.encrypted_key_area.data() + NcaHeader::DecryptionKey_AesCtr * Aes128KeySize,
Aes128KeySize, GetKeyTypeValue(m_header.key_index, m_header.GetProperKeyGeneration()));
crypto_cfg.generate_key(
m_decryption_keys[NcaHeader::DecryptionKey_AesXts1].data(), Aes128KeySize,
m_header.encrypted_key_area.data() + NcaHeader::DecryptionKey_AesXts1 * Aes128KeySize,
Aes128KeySize, GetKeyTypeValue(m_header.key_index, m_header.GetProperKeyGeneration()));
crypto_cfg.generate_key(
m_decryption_keys[NcaHeader::DecryptionKey_AesXts2].data(), Aes128KeySize,
m_header.encrypted_key_area.data() + NcaHeader::DecryptionKey_AesXts2 * Aes128KeySize,
Aes128KeySize, GetKeyTypeValue(m_header.key_index, m_header.GetProperKeyGeneration()));
crypto_cfg.generate_key(
m_decryption_keys[NcaHeader::DecryptionKey_AesCtrEx].data(), Aes128KeySize,
m_header.encrypted_key_area.data() + NcaHeader::DecryptionKey_AesCtrEx * Aes128KeySize,
Aes128KeySize, GetKeyTypeValue(m_header.key_index, m_header.GetProperKeyGeneration()));
// Copy the hardware speed emulation key.
std::memcpy(m_decryption_keys[NcaHeader::DecryptionKey_AesCtrHw].data(),
m_header.encrypted_key_area.data() +
NcaHeader::DecryptionKey_AesCtrHw * Aes128KeySize,
Aes128KeySize);
}
// Clear the external decryption key.
std::memset(m_external_decryption_key.data(), 0, m_external_decryption_key.size());
// Set software key availability.
m_is_available_sw_key = crypto_cfg.is_available_sw_key;
// Set our decompressor function getter.
m_get_decompressor = compression_cfg.get_decompressor;
// Set our storages.
m_header_storage = std::move(work_header_storage);
m_body_storage = std::move(base_storage);
R_SUCCEED();
}
VirtualFile NcaReader::GetSharedBodyStorage() {
ASSERT(m_body_storage != nullptr);
return m_body_storage;
}
u32 NcaReader::GetMagic() const {
ASSERT(m_body_storage != nullptr);
return m_header.magic;
}
NcaHeader::DistributionType NcaReader::GetDistributionType() const {
ASSERT(m_body_storage != nullptr);
return m_header.distribution_type;
}
NcaHeader::ContentType NcaReader::GetContentType() const {
ASSERT(m_body_storage != nullptr);
return m_header.content_type;
}
u8 NcaReader::GetHeaderSign1KeyGeneration() const {
ASSERT(m_body_storage != nullptr);
return m_header.header1_signature_key_generation;
}
u8 NcaReader::GetKeyGeneration() const {
ASSERT(m_body_storage != nullptr);
return m_header.GetProperKeyGeneration();
}
u8 NcaReader::GetKeyIndex() const {
ASSERT(m_body_storage != nullptr);
return m_header.key_index;
}
u64 NcaReader::GetContentSize() const {
ASSERT(m_body_storage != nullptr);
return m_header.content_size;
}
u64 NcaReader::GetProgramId() const {
ASSERT(m_body_storage != nullptr);
return m_header.program_id;
}
u32 NcaReader::GetContentIndex() const {
ASSERT(m_body_storage != nullptr);
return m_header.content_index;
}
u32 NcaReader::GetSdkAddonVersion() const {
ASSERT(m_body_storage != nullptr);
return m_header.sdk_addon_version;
}
void NcaReader::GetRightsId(u8* dst, size_t dst_size) const {
ASSERT(dst != nullptr);
ASSERT(dst_size >= NcaHeader::RightsIdSize);
std::memcpy(dst, m_header.rights_id.data(), NcaHeader::RightsIdSize);
}
bool NcaReader::HasFsInfo(s32 index) const {
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
return m_header.fs_info[index].start_sector != 0 || m_header.fs_info[index].end_sector != 0;
}
s32 NcaReader::GetFsCount() const {
ASSERT(m_body_storage != nullptr);
for (s32 i = 0; i < NcaHeader::FsCountMax; i++) {
if (!this->HasFsInfo(i)) {
return i;
}
}
return NcaHeader::FsCountMax;
}
const Hash& NcaReader::GetFsHeaderHash(s32 index) const {
ASSERT(m_body_storage != nullptr);
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
return m_header.fs_header_hash[index];
}
void NcaReader::GetFsHeaderHash(Hash* dst, s32 index) const {
ASSERT(m_body_storage != nullptr);
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
ASSERT(dst != nullptr);
std::memcpy(dst, std::addressof(m_header.fs_header_hash[index]), sizeof(*dst));
}
void NcaReader::GetFsInfo(NcaHeader::FsInfo* dst, s32 index) const {
ASSERT(m_body_storage != nullptr);
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
ASSERT(dst != nullptr);
std::memcpy(dst, std::addressof(m_header.fs_info[index]), sizeof(*dst));
}
u64 NcaReader::GetFsOffset(s32 index) const {
ASSERT(m_body_storage != nullptr);
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
return NcaHeader::SectorToByte(m_header.fs_info[index].start_sector);
}
u64 NcaReader::GetFsEndOffset(s32 index) const {
ASSERT(m_body_storage != nullptr);
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
return NcaHeader::SectorToByte(m_header.fs_info[index].end_sector);
}
u64 NcaReader::GetFsSize(s32 index) const {
ASSERT(m_body_storage != nullptr);
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
return NcaHeader::SectorToByte(m_header.fs_info[index].end_sector -
m_header.fs_info[index].start_sector);
}
void NcaReader::GetEncryptedKey(void* dst, size_t size) const {
ASSERT(m_body_storage != nullptr);
ASSERT(dst != nullptr);
ASSERT(size >= NcaHeader::EncryptedKeyAreaSize);
std::memcpy(dst, m_header.encrypted_key_area.data(), NcaHeader::EncryptedKeyAreaSize);
}
const void* NcaReader::GetDecryptionKey(s32 index) const {
ASSERT(m_body_storage != nullptr);
ASSERT(0 <= index && index < NcaHeader::DecryptionKey_Count);
return m_decryption_keys[index].data();
}
bool NcaReader::HasValidInternalKey() const {
for (s32 i = 0; i < NcaHeader::DecryptionKey_Count; i++) {
if (std::memcmp(ZeroKey.data(), m_header.encrypted_key_area.data() + i * Aes128KeySize,
Aes128KeySize) != 0) {
return true;
}
}
return false;
}
bool NcaReader::HasInternalDecryptionKeyForAesHw() const {
return std::memcmp(ZeroKey.data(), this->GetDecryptionKey(NcaHeader::DecryptionKey_AesCtrHw),
Aes128KeySize) != 0;
}
bool NcaReader::IsSoftwareAesPrioritized() const {
return m_is_software_aes_prioritized;
}
void NcaReader::PrioritizeSoftwareAes() {
m_is_software_aes_prioritized = true;
}
bool NcaReader::IsAvailableSwKey() const {
return m_is_available_sw_key;
}
bool NcaReader::HasExternalDecryptionKey() const {
return std::memcmp(ZeroKey.data(), this->GetExternalDecryptionKey(), Aes128KeySize) != 0;
}
const void* NcaReader::GetExternalDecryptionKey() const {
return m_external_decryption_key.data();
}
void NcaReader::SetExternalDecryptionKey(const void* src, size_t size) {
ASSERT(src != nullptr);
ASSERT(size == sizeof(m_external_decryption_key));
std::memcpy(m_external_decryption_key.data(), src, sizeof(m_external_decryption_key));
}
void NcaReader::GetRawData(void* dst, size_t dst_size) const {
ASSERT(m_body_storage != nullptr);
ASSERT(dst != nullptr);
ASSERT(dst_size >= sizeof(NcaHeader));
std::memcpy(dst, std::addressof(m_header), sizeof(NcaHeader));
}
GetDecompressorFunction NcaReader::GetDecompressor() const {
ASSERT(m_get_decompressor != nullptr);
return m_get_decompressor;
}
NcaHeader::EncryptionType NcaReader::GetEncryptionType() const {
return m_header_encryption_type;
}
Result NcaReader::ReadHeader(NcaFsHeader* dst, s32 index) const {
ASSERT(dst != nullptr);
ASSERT(0 <= index && index < NcaHeader::FsCountMax);
const s64 offset = sizeof(NcaHeader) + sizeof(NcaFsHeader) * index;
m_header_storage->ReadObject(dst, offset);
R_SUCCEED();
}
bool NcaReader::GetHeaderSign1Valid() const {
return m_is_header_sign1_signature_valid;
}
void NcaReader::GetHeaderSign2(void* dst, size_t size) const {
ASSERT(dst != nullptr);
ASSERT(size == NcaHeader::HeaderSignSize);
std::memcpy(dst, m_header.header_sign_2.data(), size);
}
Result NcaFsHeaderReader::Initialize(const NcaReader& reader, s32 index) {
// Reset ourselves to uninitialized.
m_fs_index = -1;
// Read the header.
R_TRY(reader.ReadHeader(std::addressof(m_data), index));
// Set our index.
m_fs_index = index;
R_SUCCEED();
}
void NcaFsHeaderReader::GetRawData(void* dst, size_t dst_size) const {
ASSERT(this->IsInitialized());
ASSERT(dst != nullptr);
ASSERT(dst_size >= sizeof(NcaFsHeader));
std::memcpy(dst, std::addressof(m_data), sizeof(NcaFsHeader));
}
NcaFsHeader::HashData& NcaFsHeaderReader::GetHashData() {
ASSERT(this->IsInitialized());
return m_data.hash_data;
}
const NcaFsHeader::HashData& NcaFsHeaderReader::GetHashData() const {
ASSERT(this->IsInitialized());
return m_data.hash_data;
}
u16 NcaFsHeaderReader::GetVersion() const {
ASSERT(this->IsInitialized());
return m_data.version;
}
s32 NcaFsHeaderReader::GetFsIndex() const {
ASSERT(this->IsInitialized());
return m_fs_index;
}
NcaFsHeader::FsType NcaFsHeaderReader::GetFsType() const {
ASSERT(this->IsInitialized());
return m_data.fs_type;
}
NcaFsHeader::HashType NcaFsHeaderReader::GetHashType() const {
ASSERT(this->IsInitialized());
return m_data.hash_type;
}
NcaFsHeader::EncryptionType NcaFsHeaderReader::GetEncryptionType() const {
ASSERT(this->IsInitialized());
return m_data.encryption_type;
}
NcaPatchInfo& NcaFsHeaderReader::GetPatchInfo() {
ASSERT(this->IsInitialized());
return m_data.patch_info;
}
const NcaPatchInfo& NcaFsHeaderReader::GetPatchInfo() const {
ASSERT(this->IsInitialized());
return m_data.patch_info;
}
const NcaAesCtrUpperIv NcaFsHeaderReader::GetAesCtrUpperIv() const {
ASSERT(this->IsInitialized());
return m_data.aes_ctr_upper_iv;
}
bool NcaFsHeaderReader::IsSkipLayerHashEncryption() const {
ASSERT(this->IsInitialized());
return m_data.IsSkipLayerHashEncryption();
}
Result NcaFsHeaderReader::GetHashTargetOffset(s64* out) const {
ASSERT(out != nullptr);
ASSERT(this->IsInitialized());
R_RETURN(m_data.GetHashTargetOffset(out));
}
bool NcaFsHeaderReader::ExistsSparseLayer() const {
ASSERT(this->IsInitialized());
return m_data.sparse_info.generation != 0;
}
NcaSparseInfo& NcaFsHeaderReader::GetSparseInfo() {
ASSERT(this->IsInitialized());
return m_data.sparse_info;
}
const NcaSparseInfo& NcaFsHeaderReader::GetSparseInfo() const {
ASSERT(this->IsInitialized());
return m_data.sparse_info;
}
bool NcaFsHeaderReader::ExistsCompressionLayer() const {
ASSERT(this->IsInitialized());
return m_data.compression_info.bucket.offset != 0 && m_data.compression_info.bucket.size != 0;
}
NcaCompressionInfo& NcaFsHeaderReader::GetCompressionInfo() {
ASSERT(this->IsInitialized());
return m_data.compression_info;
}
const NcaCompressionInfo& NcaFsHeaderReader::GetCompressionInfo() const {
ASSERT(this->IsInitialized());
return m_data.compression_info;
}
bool NcaFsHeaderReader::ExistsPatchMetaHashLayer() const {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_data_info.size != 0 && this->GetPatchInfo().HasIndirectTable();
}
NcaMetaDataHashDataInfo& NcaFsHeaderReader::GetPatchMetaDataHashDataInfo() {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_data_info;
}
const NcaMetaDataHashDataInfo& NcaFsHeaderReader::GetPatchMetaDataHashDataInfo() const {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_data_info;
}
NcaFsHeader::MetaDataHashType NcaFsHeaderReader::GetPatchMetaHashType() const {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_type;
}
bool NcaFsHeaderReader::ExistsSparseMetaHashLayer() const {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_data_info.size != 0 && this->ExistsSparseLayer();
}
NcaMetaDataHashDataInfo& NcaFsHeaderReader::GetSparseMetaDataHashDataInfo() {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_data_info;
}
const NcaMetaDataHashDataInfo& NcaFsHeaderReader::GetSparseMetaDataHashDataInfo() const {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_data_info;
}
NcaFsHeader::MetaDataHashType NcaFsHeaderReader::GetSparseMetaHashType() const {
ASSERT(this->IsInitialized());
return m_data.meta_data_hash_type;
}
} // namespace FileSys

View File

@ -0,0 +1,61 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/alignment.h"
#include "core/file_sys/fssystem/fssystem_pooled_buffer.h"
namespace FileSys {
namespace {
constexpr size_t HeapBlockSize = BufferPoolAlignment;
static_assert(HeapBlockSize == 4_KiB);
// A heap block is 4KiB. An order is a power of two.
// This gives blocks of the order 32KiB, 512KiB, 4MiB.
constexpr s32 HeapOrderMax = 7;
constexpr s32 HeapOrderMaxForLarge = HeapOrderMax + 3;
constexpr size_t HeapAllocatableSizeMax = HeapBlockSize * (static_cast<size_t>(1) << HeapOrderMax);
constexpr size_t HeapAllocatableSizeMaxForLarge =
HeapBlockSize * (static_cast<size_t>(1) << HeapOrderMaxForLarge);
} // namespace
size_t PooledBuffer::GetAllocatableSizeMaxCore(bool large) {
return large ? HeapAllocatableSizeMaxForLarge : HeapAllocatableSizeMax;
}
void PooledBuffer::AllocateCore(size_t ideal_size, size_t required_size, bool large) {
// Ensure preconditions.
ASSERT(m_buffer == nullptr);
// Check that we can allocate this size.
ASSERT(required_size <= GetAllocatableSizeMaxCore(large));
const size_t target_size =
std::min(std::max(ideal_size, required_size), GetAllocatableSizeMaxCore(large));
// Dummy implementation for allocate.
if (target_size > 0) {
m_buffer =
reinterpret_cast<char*>(::operator new(target_size, std::align_val_t{HeapBlockSize}));
m_size = target_size;
// Ensure postconditions.
ASSERT(m_buffer != nullptr);
}
}
void PooledBuffer::Shrink(size_t ideal_size) {
ASSERT(ideal_size <= GetAllocatableSizeMaxCore(true));
// Shrinking to zero means that we have no buffer.
if (ideal_size == 0) {
::operator delete(m_buffer, std::align_val_t{HeapBlockSize});
m_buffer = nullptr;
m_size = ideal_size;
}
}
} // namespace FileSys

View File

@ -0,0 +1,95 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/common_funcs.h"
#include "common/common_types.h"
#include "common/literals.h"
#include "core/hle/result.h"
namespace FileSys {
using namespace Common::Literals;
constexpr inline size_t BufferPoolAlignment = 4_KiB;
constexpr inline size_t BufferPoolWorkSize = 320;
class PooledBuffer {
YUZU_NON_COPYABLE(PooledBuffer);
public:
// Constructor/Destructor.
constexpr PooledBuffer() : m_buffer(), m_size() {}
PooledBuffer(size_t ideal_size, size_t required_size) : m_buffer(), m_size() {
this->Allocate(ideal_size, required_size);
}
~PooledBuffer() {
this->Deallocate();
}
// Move and assignment.
explicit PooledBuffer(PooledBuffer&& rhs) : m_buffer(rhs.m_buffer), m_size(rhs.m_size) {
rhs.m_buffer = nullptr;
rhs.m_size = 0;
}
PooledBuffer& operator=(PooledBuffer&& rhs) {
PooledBuffer(std::move(rhs)).Swap(*this);
return *this;
}
// Allocation API.
void Allocate(size_t ideal_size, size_t required_size) {
return this->AllocateCore(ideal_size, required_size, false);
}
void AllocateParticularlyLarge(size_t ideal_size, size_t required_size) {
return this->AllocateCore(ideal_size, required_size, true);
}
void Shrink(size_t ideal_size);
void Deallocate() {
// Shrink the buffer to empty.
this->Shrink(0);
ASSERT(m_buffer == nullptr);
}
char* GetBuffer() const {
ASSERT(m_buffer != nullptr);
return m_buffer;
}
size_t GetSize() const {
ASSERT(m_buffer != nullptr);
return m_size;
}
public:
static size_t GetAllocatableSizeMax() {
return GetAllocatableSizeMaxCore(false);
}
static size_t GetAllocatableParticularlyLargeSizeMax() {
return GetAllocatableSizeMaxCore(true);
}
private:
static size_t GetAllocatableSizeMaxCore(bool large);
private:
void Swap(PooledBuffer& rhs) {
std::swap(m_buffer, rhs.m_buffer);
std::swap(m_size, rhs.m_size);
}
void AllocateCore(size_t ideal_size, size_t required_size, bool large);
private:
char* m_buffer;
size_t m_size;
};
} // namespace FileSys

Some files were not shown because too many files have changed in this diff Show More