Opened 4 weeks ago
#63617 new defect (bug)
Account for when artifacts expire or go missing during performance testing
Reported by: |
|
Owned by: | |
---|---|---|---|
Milestone: | Awaiting Review | Priority: | normal |
Severity: | normal | Version: | |
Component: | Build/Test Tools | Keywords: | needs-patch |
Focuses: | Cc: |
Description
The performance testing workflow attempts to download the build artifact for the previous commit for the purposes of comparing metrics for a given commit. When artifacts expire or go missing, the workflow results in a failure.
This is most common when committing code to older branches (especially those only receiving security updates) because of the long periods of time between commits. An example of this can be seen in a recent commit to the 6.5
branch. This is not the only scenario though. Artifacts can also be deleted both manually and programmatically.
When artifacts go missing, the branch should be taken into account to raise an appropriate level of concern.
For example, it should continue to fail when this happens on trunk
or the most recent release's branch as this may indicate a problem with the workflow or a service outage. But if the branch is older than that, the workflow can probably succeed with a warning or some type of notice as it's likely due to an expired artifact