[3.0] Implement name-related improvements identified in previous Vulkan PR

[Exanite]

Summary of the PR

This PR focuses on implementing the naming-related improvements identified in #2457.
Depending on the scope of the tasks, some of these tasks may be pushed to another PR.

This PR also contains the changes of multiple other PRs.
See each individual PR for more information.

Included PRs:

• [3.0] Regenerate bindings on Windows for Name Improvements PR Exanite/Silk.NET#19
• [3.0] Implement new name affix category approach Exanite/Silk.NET#21
• [3.0] Update feature/name-affixes-categories with changes from develop/3.0 Exanite/Silk.NET#23
• [3.0] Reorder KhronosVendor suffixes to be last Exanite/Silk.NET#24
• [3.0] Format files I changed using CSharpier Exanite/Silk.NET#25
• [3.0] Heavily expand testing suite; Rewrite NameUtils.Prettify; Fix a lot of subtle naming bugs Exanite/Silk.NET#28
• [3.0] Change acronym threshold to 2 for all bindings Exanite/Silk.NET#29

Important changes:

• Handle types are now named as -HandleEXT instead of -EXTHandle.
  • Eg: AccelerationStructureHandleKHR
• Function pointer delegate types are now named as -DelegateEXT instead of EXTDelegate.
  • Eg: pfn struct BufferCallbackSOFT and pfn delegate BufferCallbackDelegateSOFT
• Functions with data type suffixes following a number such as alGetSourcei64vDirectSOFT are now trimmed and prettified as GetSourcei64vDirectSOFT instead of GetSourcei64VDirectSOFT. Note the capitalization of v. This is because affixes are added after prettification.
• Vendor suffixes are now kept strictly uppercased.
• OpenGL vendor suffix trimming is now generalized and can be configured and used for any Khronos API.
  • Notably, TrimEnumMemberImpliedVendors trims enum member vendor suffixes if it matches the enum type vendor suffix.
• Struct properties are now prettified. They were previously ignored.
• NativeName is a new attribute that saves the name output by ClangSharp. This for the most part isn't used by the generator, but can be useful for the end user.
• NameAffix is a new attribute that allows prefixes/suffixes to be declared upfront and reordered. The idea is that mods add the attribute to identifiers and PrettifyNames applies the affixes during trimming and prettification.
• A large portion of the trimming/PrettifyNames code has been rewritten.
  • NameTrimmer has been kept mostly the same.
  • INameTrimmer.Order is used to order the trimmers instead of the version.
  • PrettifyNames.Visitor's functionality is mostly the same, but also gathers name affix data.
  • MixKhronosData.Trim is almost all replaced by the name affix system. This is implemented in MixKhronosData.RewriterPhase3 (yes, there are 3 rewriters, but this keeps things simpler without adding much time cost).
  • Most usages of tuples have been replaced with record structs.
  • Most nullable collections (eg: List<string>?) have been made non-nullable. Most of these get allocated anyway and it makes the code more maintainable. If there is any performance issue (mainly concerning the Microsoft job), I will take the time to optimize it.
  • Prettification (as in calling the Prettify extension method) is now implemented as another trimmer. Prettification now happens before name conflict resolution and before name affixes are added.
• Handle structs now have constructors since they were previously not possible to construct (field is readonly).
• The StripAttributes mod can be used to remove attributes from the final bindings.
  • Currently the following are removed: NativeTypeName, NameAffix, Transformed

Related issues, Discord discussions, or proposals

Previous Vulkan PR (initial bindings generation): #2457
Discord thread: https://discord.com/channels/521092042781229087/1376331581198827520/1442651368207941643

This PR also contains the changes of multiple other PRs.
See each individual PR for more information.

Included PRs:

• [3.0] Regenerate bindings on Windows for Name Improvements PR Exanite/Silk.NET#19
• [3.0] Implement new name affix category approach Exanite/Silk.NET#21
• [3.0] Update feature/name-affixes-categories with changes from develop/3.0 Exanite/Silk.NET#23
• [3.0] Reorder KhronosVendor suffixes to be last Exanite/Silk.NET#24
• [3.0] Format files I changed using CSharpier Exanite/Silk.NET#25
• [3.0] Heavily expand testing suite; Rewrite NameUtils.Prettify; Fix a lot of subtle naming bugs Exanite/Silk.NET#28
• [3.0] Change acronym threshold to 2 for all bindings Exanite/Silk.NET#29

Further Comments

Tasks not part of this PR

These are the tasks from my previous PR. These have been sorted and trimmed down to ensure that this PR remains focused. These may be added to this PR.

If you want to see the original, unmodified set of tasks, please see #2457.

These will also be part of a future PR.

• Apply BoolType transformation to VkBool32 in structs. They currently are only handled for functions.
• Add default field values to structs where it makes sense (eg: SType)
• Ensure SupportedApiProfiles attributes are correct
  • Properly resolve API profiles for Flags/FlagBits types
  • Prefer name from NativeName for resolving API profiles

Future PRs

• Update Curin's branch to use my new LocationTransformation code. The new renamer should be replaced with this. See the NameUtils.cs and Renamer.cs files on Curin's branch. Renamer.cs should be removed. NameUtils.cs should have the methods updated to use my LocationTransformation code instead (this should show up as a merge conflict).
  • PR opened and ready for review/merge
  • PR merged
  • Old renamer code removed
• Consider opening a ClangSharp PR to fix issue with SDL_MAX_SINT64 generation on Linux
  • More context: https://discord.com/channels/521092042781229087/1376331581198827520/1443401810822828113
  • public const nint SDL_MAX_SINT64 = unchecked(0x7FFFFFFFFFFFFFFF); does not compile
• Reimplement the struct chaining API
• Fix issues when running multiple jobs
  • Figure out why the headers are getting mixed up when running multiple jobs at the same time.
    • Seems to be related to TransformFunctions.
  • Figure out why the SDL handle structs are getting named as Silk.NET.SDL.ConditionHandle.gen.cs instead of ConditionHandle.gen.cs when running multiple jobs at the same time.
• Maybe rework how using statements are added. Not sure why, but using statements are randomly added/removed between Windows/Linux. This causes noisy diffs.
• Consider splitting ExtractNestedTyping and TransformHandles into a new set of Extract- mods.
  • This this mainly for maintainability for ExtractNestedTyping, but for TransformHandles it is useful for AddApiProfiles to be executed after all types are extracted, but not yet transformed.
  • Also consider making AddApiProfiles work off of NativeName attributes.
• Optimize/refactor name overrides in PrettifyNames
• Consider reworking data type trimming regexes.
  • This was discussed more here: [3.0] Implement name-related improvements identified in previous Vulkan PR #2503 (comment)
  • The summary is that the current set of 2 regexes doesn't elegantly handle the case where the ending of a word should not be trimmed, but the word itself can have data type suffixes.
  • For example: alSourceRewindv and alSourceRewind. The d in Rewind can be mistaken as a data type suffix, so we add it to the EndingsNotToTrim regex. However, the dv in Rewindv also has the same problem.
  • We likely need something more akin to WordsNotToTrimInto than EndingsNotToTrim.
• Optimize NativeName attributes.
  • This should involve renaming StripAttributes to CleanupAttributes and adding an option in its config to optimize NativeName attributes.
  • Perksey seems to really want this so considering prioritizing this: [3.0] Implement name-related improvements identified in previous Vulkan PR #2503 (comment)
• Consider replacing LenientUnderscore and updating codebase to use NameSplitter
• Consider implementing a IsProbablyABitmask function based on BuildTool's IsProbablyABitmask for identifying flags enums
  • Original comment: [3.0] Implement name-related improvements identified in previous Vulkan PR #2503 (comment)
• Improve handling of nested struct names
  • Eg: PerformanceCounterDescriptionArmName should be PerformanceCounterDescriptionARMName
  • Conversation here: https://discord.com/channels/521092042781229087/587346162802229298/1460017726813900975
• Address INameTrimmer.Order/Version issue (we don't want magic numbers) and consider removing INameTrimmer entirely.

Done during later PRs:

• Prevent OES being singularizing into O in ArrayParameterTransformer.TransformArrayParameterRewriter.
  • Eg: glFeedbackBufferxOES
  • Discussed solution is to expose the name affix API so that TransformArrayParameterRewriter can singularize the base name instead of the full name.
• Change native name of GLEnum to GLenum. Same for ALCenum and ALenum.

Completed Todos

Newest groups at the top. Order within a group is chronological.

Note that I also wrote a summary at the top. The summary is more comprehensive, but this section can be useful getting additional context. Also see my commit descriptions if more context is needed.

Prepare to merge.

• Finish self code review
• Generate bindings on Windows for consistency
• Temporarily remove generated files from PR during code review
  • This needs to be done since there are too many generated files and they hide some of the manually changed files.
  • Revert the commit after code review: [3.0] Regenerate bindings on Windows for Name Improvements PR Exanite/Silk.NET#19
• Review the generated bindings as a second code review
• Consider updating OpenAL submodule and regenerating
• Merge name affix config PR into this PR
• Fix broken test cases (I know there is at least one)

Acronym threshold change and prettify improvements

• Acronym related name improvements discussed here: https://discord.com/channels/521092042781229087/587346162802229298/1446979314330501172
  • Re-read discussion and organize into tasks.
  • Change default acronym threshold to 2 for all bindings
  • Change default acronym threshold to 2 to be the default if not specified in the PrettifyNames config.
  • Handle Mat4X4 case. The "X" should be lowercased.
  • Update existing names to match framework design guidelines
    • EGL as Egl
    • Name overrides in generator config

INameTrimmer.Order

• Add priority system to INameTrimmer.
  • I implemented the new name affixer system using trimmers, so this introduced 3 new trimmers in order to separate each stage properly. There is now a new INameTrimmer.Order property and internal TrimmerOrder enum.

Fix misc issues I noticed

• VkVendorId members are not trimmed properly
• Figure out why the members of the enum AttribMask in OpenGL are not getting rewritten.
  • Initial investigation shows that the fieldSymbol.ConstantValue is null, but why it is null is unclear. Maybe a Roslyn bug? Example member: DepthBufferBit = unchecked((uint)0x00000100). Note that similar expressions in Vulkan are fine, eg: unchecked((ulong)0x00000002UL). Maybe it is because the literal type and cast type are different.
  • Wait, are these the enums that we generate?
  • Yup, they are. MixKhronosData was outputting subtly invalid type names. I changed it to use ParseTypeName(baseType) instead of IdentifierName(baseType).

Fix issues introduced by name affix system

• Identify and fix issues introduced by name affixation. Also just identify and fix naming issues in general.
  • Data type trimming in OpenAL seems to be broken. Probably OpenGL as well.
  • Check if I need to reimplement vendor suffix removal for OpenGL. I probably do.
    • Discuss: I decided to change the vendor suffix removal for enum members so that it removes the member suffix if it matches the type suffix. This prevents cases where a member of a different suffix is added, causing all members to have their suffixes restored.
    • Discuss: The above change made it so that the member suffix removal code is usable for Vulkan so I enabled it for Vulkan as well. See PresentModeKHR to see how it looks (the KHR suffixed members have their suffixes removed, but EXT remains the same).
  • Reapply some of my MixKhronosData code removals to the current develop/3.0 branch and see what changes would be made. This is to ensure I don't accidentally remove functionality when deleting code.
• Ensure other APIs are configured consistently with Vulkan. Eg: Ensure affix priorities are set, etc.

Name affix system

• Properly trim prefixed/suffixed names, such as PFNVkDebugUtilsMessengerCallbackEXT and BufferTHandle
  • PFNVkDebugUtilsMessengerCallbackEXT prefix
  • PFNVkDebugUtilsMessengerCallbackDelegateEXT suffix
  • AccelerationStructureHandleKHR suffix
  • Extract vendor extensions from names and store into NameSuffix attributes.
  • Revert "hacky" THandle fix.
• Consider renaming handle types as HandleEXT instead of EXTHandle for readability.
  • This is fairly doable. The above todos track this.

Name metadata

• Add NativeName attribute
• Ensure NativeName attribute is added by all mods that generate new types/members
  • Technically not strictly required, but nice from a API consistency standpoint.
  • MixKhronosData
    • Enum members
    • Enum declarations
      • Vulkan (uses FlagBits version of enum names since it is more helpful when googling)
      • Discuss validity in other Khronos APIs (some APIs don't have actual enums)
        • Eg: GLEnum and the other OpenGL enums
  • ExtractNestedTyping
    • Function pointers
    • Others?
  • TransformHandles

Name prettification

• Change naming convention to not capitalize acronyms (i.e., strictly pascal cased)
  • See: https://discord.com/channels/521092042781229087/1376331581198827520/1395097379329282089
  • Update generated method class names in the rsp files to be strictly pascal case
• Fix StdVideoEncodeH264SliceHeaderFlags property names. They currently aren't getting prettified.

Fix bugs from original PR

• Enums containing negative values should not have an unsigned backing type.

Handle struct improvements

• Handle structs don't have an easy way of being constructed (field is readonly and no constructor).