Skip to content

Linker parsing updates#32

Open
johnwparent wants to merge 10 commits intospack:mainfrom
johnwparent:linker-parsing-updates
Open

Linker parsing updates#32
johnwparent wants to merge 10 commits intospack:mainfrom
johnwparent:linker-parsing-updates

Conversation

@johnwparent
Copy link
Copy Markdown
Collaborator

@johnwparent johnwparent commented Jan 31, 2026

Linker Parsing: Overhaul linker CLI processing

  • Stop disabiguating between obj, lib, and lo files, they're treated the same by the linker and we don't need extra logic for each, and we need to be able to process them in order.
  • Carefully parses the command line and .def file to determine the intended output name and ensures the internal modeling of that name in the compiler wrapper is consistent
  • Adds behavior to allow for extremely long command lines (Paraviews are 75kb) by composing arguments into an RSP file
  • Adds logic to expand the contents of an RSP file so we can inspect the full command line to better reason about naming and eventual length
  • Small bug fixes of QOL improvements

Based on #31

Adds support for

* Path canonicalization
* Making paths absolute
* Add path padding character to padding signature
* Stripping quotes
* Backslash escaping
* Corrects regex "match" bug
* Adds error handling to SFN name processing
* Adds file creation to sfn processing so sfns can be generated for "non existent" files
* Adds method to validate a path is the proper length
* Updates path padding to leverage the above api additions
* Corrects a missing return statement
* Adds new error types for current and future use

Signed-off-by: John Parent <john.parent@kitware.com>
Signed-off-by: John Parent <john.parent@kitware.com>
Ensures child stderr pipes are handled gracefully

Corrects an issue where the error pipes were not copied on assign or copy or included in the process start info or closed.

Signed-off-by: John Parent <john.parent@kitware.com>
Some packages install themselves as readonly, or with other file permissions that conflict with what we're doing with this compiler wrapper. Thus, we need the ability to override (and restore) those settings.

For this compiler wrapper to perform it's duties, it needs to be able to read and write the various binaries artifacts from a build. We thus need to be able to:
1. Obtain and store current permissions
2. Grant ourselves the required permissions
3. Do our reading/writing
4. Restore previous permissions

On Windows, file access is handled by the read/write bit, as on other platforms, but also by Access Control Lists (ACLs) which are populated by Discretionary Access Control Lists (DACLs) which are the atomic units responsible for granting rights.

This PR adds an interface to inspect, store, update, and restore the read/write bits and DACLs of a given file. This functionality is wrapped up in a RAII based class that essentially provides scoped permissions to a file for scope of the permissions object.

Note: this assumes the user creating the permissions is the user driving this wrapper, or this user has admin rights, otherwise this will fail (gracefully, but still a failure)
Signed-off-by: John Parent <john.parent@kitware.com>
Previously we were searching a set of prefixes for DLLs during relocation. If we found a dll that matched the dll we were looking for (based on file name) we performed relocation based on that path. This is both dangerous and extraneous.

This is dangerous as mutli config layouts may have the same binary with the same name in mutliple different paths for different configs or variations. Since we were previously only checking the filename, this could lead to a false positive detection and bad relocation not detected until runtime.

This is extraneous as we should never need to search. We have the dll locations before and after relocation, whether from the stage to install prefix or from buildcache to buildcache, so rather than a filesystem search, we can have a linear time operation where we search through a list of relocation old->new prefix mappings.

Spack core will set an environment variable of the structure:
old_prefix|new_prefix and the compiler wrapper now composes a map out of that list and then PE files looking to relocate their internal dll references get a constant time lookup.

Signed-off-by: John Parent <john.parent@kitware.com>
    * Adds api calls for obtaining the PE names stored in coff files
      Supports both short and long names, or detecting both. Typically only one is defined in an import library, so whichever is returned first.

    * Adds access guards to coff reading

Signed-off-by: John Parent <john.parent@kitware.com>
We're no longer modifying binaries on BC push

Signed-off-by: John Parent <john.parent@kitware.com>
* Makes all PE paths absolute
* Removes rename logic that dealt with the spack sigil and BC pushes
* Uses new relocation logic to avoid FS search and instead use Spack env variable and util support added in prior PR
* Adds access rights scoping for read and write operations
* General code cleanup

Signed-off-by: John Parent <john.parent@kitware.com>
Reduce the degree to which we decompose an incoming command line in the toolchain. The odering and context of CLI arguments is highly fragile and important to the contents and naming of build artifacts, in particular w.r.t. maintaining user expectations and logic regarding name reasoning.

Instead toolchain is now a thin wrapper to pass the toolchain commands through the compiler wrapper itself, preserving inputs and order and injecting Spack libs, includes, etc safely.

We leave command line parsing to specialized tools designed for the linker/compiler/etc.

Signed-off-by: John Parent <john.parent@kitware.com>
* Stop disabiguating between obj, lib, and lo files, they're treated the same by the linker and we don't need extra logic for each, and we need to be able to process them in order.
* Carefully parses the command line and .def file to determine the intended output name and ensures the internal modeling of that name in the compiler wrapper is consistent
* Adds behavior to allow for extremely long command lines (Paraviews are 75kb) by composing arguments into an RSP file
* Adds logic to expand the contents of an RSP file so we can inspect the full command line to better reason about naming and eventual length
* Small bug fixes of QOL improvements

Signed-off-by: John Parent <john.parent@kitware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant