NewValidation: Not Giving Lifecycle for F1527/F1528/F1529

[raminderchouhan]

(use upvote 👍 for attentions)

Validation Type

[X ] - Fault

[ ] - Config

[ ] - Bug

[ ] - Other

What needs to be validated

Worked with Customer , script gave errors Check16 : APIC Disk Space Usage
Cleared the NON-SVC files and disk space was reduced but the error still showed on the next run of the script.

###For a config validation, describe the exact configuration to be validated
The request is to either ignore the State “Retaining” of the error or more preferably, show the current State and Severity of the Fault if present in the system so we can safely ignore it.
Example:

Worked with Customer, reduced the /var/log/dme/log to 30 percent

GUI does not have F1527/F1528/F1529 any more

moquery shows state retaining and severity Cleared

But the re-run of the script still shows the Mount Point /data/log with Usage 75 percent.

Why it needs to be validated

I explained to customer there is not error anymore but customer did not want to proceed with upgrade till the check is cleared, and also per the customer, if the script is run by a person who is following the MOP will not proceed with the process if the Error still shows , even though TAC assured them, so having the check against either to ignore and does not produce the ERROR, or showing what is the current state and the severity and a text that so and so state and severity is considered good to proceed

Additional context

Add any other context about the feature request here.

[MrKhamba]

@raminderchouhan - Thanks for raising the validation check-

Following are my findings regarding the script and the checks I want to be resolved -

1. Check#16 is defined under checkmanager class as the 16th entry(zero indexed)

2. So, the check in question is the apic_disk_space_faults_check function

3. The apic_disk_space_faults_check function does NOT proactively check disk utilization by running commands like df -h or du -sh * on the APIC nodes. Instead, it reactively checks for existing fault instances that the ACI APIC system itself has raised due to disk space issues – I will explain it how :

4. @check_wrapper decorator defines the check_title, explicitly stating that this check is related to faults F1527, F1528, and F1529 and it initializes result to FAIL_UF, meaning it assumes failure unless proven otherwise.

5. It sets up specific recommended_action messages based on the affected mount point (/firmware, /techsupport) and it also adds a version-specific note to the default_action for older ACI versions (pre-4.0(1h) or pre-3.2(6i)), highlighting a known bug (CSCvn13119) which might contribute to disk filling. This demonstrates a nuanced, experience-based approach.

6. This is the critical line. The icurl function (which wraps subprocess.check_output to call the icurl command-line tool or a similar API client) queries the ACI APIC's Management Information Model (MIM).It specifically asks for objects of class faultInst (fault instance). The query-target-filter is what makes this check dependent on F1527 (and F1528, F1529). It filters the faultInst objects to only return those where the code attribute is F1527 OR F1528 OR F1529.
   faultInsts = icurl('class',
   'faultInst.json?query-target-filter=or(eq(faultInst.code,"F1527"),eq(faultInst.code,"F1528"),eq(faultInst.code,"F1529"))')

7. For each faultInst returned by the icurl query:
   7.1 It extracts the fault code (e.g., "F1527").
   7.2 It attempts to parse the fault's dn and descr using the previously defined regular expressions.
   7.3 If successful, it populates the data list with the structured information: fault code, pod ID, node ID, mount point, current usage percentage, and the appropriate recommended action.
   7.4 If parsing fails (e.g., the regex doesn't match the fault description's format), it stores the raw fault data in unformatted_data for later review

Conclusion : apic_disk_space_faults_check is entirely dependent on fault F1527 (along with F1528 and F1529).
Explicit Filtering: The icurl query's query-target-filter explicitly targets faultInst objects where the code is "F1527", "F1528", or "F1529".

Reactive, Not Proactive: This check does not actively measure disk space. It relies on the APIC's internal monitoring mechanisms to detect high disk utilization and then raise the corresponding equipment-full faults (F1527, F1528, F1529).

Implication: If an APIC device experiences high disk usage but, for some reason, fails to generate these specific fault codes (e.g., a bug in older software, a custom configuration that suppresses them), this script check would report PASS even though there might be a critical disk space issue in our case it’s doing the opposite where even when we bring down the usage of the filesystem it keeps on reporting as Upgrade Failure.