[feat] Git sparse checkout for sourcesdir#3629
[feat] Git sparse checkout for sourcesdir#3629victorusu wants to merge 4 commits intoreframe-hpc:developfrom
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #3629 +/- ##
===========================================
+ Coverage 91.25% 91.32% +0.06%
===========================================
Files 62 62
Lines 13587 13532 -55
===========================================
- Hits 12399 12358 -41
+ Misses 1188 1174 -14 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| #: Default value is now conditionally set to either ``'src'`` or | ||
| #: :class:`None`. | ||
| sourcesdir = variable(str, type(None), value='src') | ||
| sourcesdir = variable(str, typ.Dict, type(None), value='src') |
There was a problem hiding this comment.
| sourcesdir = variable(str, typ.Dict, type(None), value='src') | |
| sourcesdir = variable(str, typ.Dict[str, object], type(None), value='src') |
| srcdir = self.sourcesdir | ||
| if isinstance(srcdir, dict): | ||
| if 'url' not in srcdir: | ||
| raise ReframeError(f'{srcdir} misses the url key') | ||
|
|
||
| url = srcdir['url'] | ||
| if not osext.is_url(url): | ||
| raise ReframeError(f'The dictionary syntax only supports ' | ||
| 'git repositories') | ||
|
|
||
| self._clone_to_stagedir(url, | ||
| files=srcdir[url]['files'] if 'files' | ||
| in srcdir[url] else None, | ||
| opts=srcdir[url]['opts'] if 'opts' | ||
| in srcdir[url] else None) | ||
| elif osext.is_url(srcdir): | ||
| self._clone_to_stagedir(srcdir) |
There was a problem hiding this comment.
This is somewhat ugly. Ideally, I would see the keys of the sourcesdir correspond the arguments of the target function. You could do something along these lines:
if not isinstance(self.sourcesdir, dict):
# Convert it to dict passing the default values
self.sourcesdir = {'url': self.sourcesdir, ...}
try:
self._clone_to_stagedir(**self.sourcesdir)
except TypeError as err:
raise ReframeSyntaxError('invalid syntax for sourcesdir') from err|
|
||
|
|
||
| def git_clone(url, targetdir=None, opts=None, timeout=5): | ||
| def git_clone(url, targetdir=None, opts=None, timeout=5, files=None): |
There was a problem hiding this comment.
Same here, fine tune the options to match those from _clone_stagedir(). Technically, it's vice-versa: _clone_stagedir() should follow the options of this one and sourcesdir the options in _clone_stagedir().
This expands the
sourcesdirsyntax to include a dictionary format. In this case, the giturlis passed via theurlkey.Additional command options can be added via the
optskey. This capability was previously available in thegit_clonefunction, but not accessible to thesourcesdirgit clone interface.The
git sparse-checkoutpath is available when one defines thefileskeys with a list of files to checkout.In this case, instead of emitting a
git clone ...command the framework will emit the following set of commandswhere,
${file list}is the list of files defined in thefileslist.An example test is this.
Please note that I haven't added any unit tests because the framework does not include any unit tests for the original
git_clonefunction.closes #3627
closes #3053