Add list of resolved vulnerabilities to the PR Comment / Logs

[david-wiggs]

This is a first attempt at #717 without any changes to the API

[Copilot]

Pull request overview

This PR adds functionality to track and display resolved vulnerabilities when vulnerable dependencies are removed from a project. The feature provides positive feedback to developers about the security improvements their changes introduce, without requiring any API changes. Resolved vulnerabilities are identified by examining removed dependencies that had known vulnerabilities.

Key changes:

• New resolved-vulnerabilities output containing JSON data about resolved vulnerabilities
• Visual feedback in PR summaries and logs highlighting resolved vulnerabilities with positive messaging
• Support for both HTML (Action summaries) and Markdown (PR comments) formatting

Reviewed changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/schemas.ts	Adds ResolvedVulnerability and ResolvedVulnerabilities schemas with fields for tracking resolved vulnerability details
src/resolved-vulnerabilities.ts	New module implementing getResolvedVulnerabilities() to extract vulnerabilities from removed dependencies
src/summary.ts	Adds functions to display resolved vulnerabilities in summaries, with separate HTML/Markdown formatting; updates existing summary functions to include resolved vulnerabilities
src/main.ts	Integrates resolved vulnerabilities feature by computing them from changes and displaying them when vulnerability_check is enabled
tests/resolved-vulnerabilities.test.ts	Comprehensive test suite for the getResolvedVulnerabilities() function covering various scenarios
tests/summary.test.ts	Updates all existing tests to pass the new emptyResolvedVulnerabilities parameter
scripts/create_summary.ts	Updates summary creation script to include the new resolved vulnerabilities parameter
docs/examples.md	Adds documentation and example workflow for accessing the resolved-vulnerabilities output
README.md	Documents the new resolved-vulnerabilities output in the action outputs section
dist/index.js	Compiled distribution file reflecting all source code changes

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Resolved vulnerabilities are shown in the summary list even when config.vulnerability_check is disabled. This is inconsistent with the treatment of other vulnerability information. Consider wrapping this section with a check for config.vulnerability_check:

...(config.vulnerability_check && resolvedVulnerabilities.length > 0
  ? [
      `${icons.check} <strong>${resolvedVulnerabilities.length}</strong> vulnerability(ies) resolved 🎉`
    ]
  : []),

[Copilot]

Resolved vulnerabilities are shown in the markdown summary list even when config.vulnerability_check is disabled. This is inconsistent with the treatment of other vulnerability information. Consider wrapping this section with a check for config.vulnerability_check:

...(config.vulnerability_check && resolvedVulnerabilities.length > 0
  ? [
      `${icons.check} **${resolvedVulnerabilities.length}** vulnerability(ies) resolved 🎉`
    ]
  : []),

[Copilot]

The documentation states that resolved vulnerabilities are found "by removing or upgrading packages," but the implementation doesn't distinguish between these two scenarios. When a vulnerable package is completely removed (not upgraded to a safer version), treating this as a "resolved vulnerability" may be misleading. The vulnerability isn't technically resolved—the functionality may have been removed entirely or moved elsewhere.

Consider either:

1. Updating the documentation to be more precise about what "resolved" means in this context
2. Enhancing the logic to differentiate between packages that were upgraded (removed old version + added new version) vs. completely removed

[ahpook]

@david-wiggs thanks for this PR, this seems like useful functionality and i understand from the community comments on #717 that it's a desirable feature. a couple of requests:

1. please drop the dist/* files from your PR, we'll generate them at release time and including them cause irresolvable merge conflicts
2. would it be possible to gate this feature behind a configuration option, which defaults to 'off'? this would allow users to opt-in to try it out in this feature series and if it works out well, we could flip it to default on in the next major version.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 11 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (2)

src/main.ts:148

• resolvedVulnerabilities is computed from the full changes list before scope/allow-list filtering. This can surface “resolved vulnerabilities” for scopes that the action is configured to ignore via fail-on-scopes (and for advisories that may be allowed via allow-ghsas). To keep the output consistent with what the action evaluates, consider deriving resolved vulnerabilities from scopedChanges (and potentially after allow-list filtering, depending on intended semantics).

    // Extract resolved vulnerabilities from removed dependencies
    const resolvedVulnerabilities = getResolvedVulnerabilities(changes)

    const scopedChanges = filterChangesByScopes(config.fail_on_scopes, changes)

    const filteredChanges = filterAllowedAdvisories(
      config.allow_ghsas,
      scopedChanges
    )

src/summary.ts:123

• Same as above: the markdown status list includes the resolved-vulnerabilities line without checking config.vulnerability_check, which can surface vulnerability-related messaging when vulnerability checks are disabled.

  const summaryListMarkdown: string[] = [
    // Add resolved vulnerabilities as positive feedback first
    ...(resolvedVulnerabilities.length > 0
      ? [
          `${icons.check} **${resolvedVulnerabilities.length}** vulnerability(ies) resolved 🎉`
        ]
      : []),
    ...(config.vulnerability_check

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The resolved-vulnerabilities output is only set when the list is non-empty. This makes downstream workflow conditions like steps.review.outputs.resolved-vulnerabilities != '[]' unreliable (unset outputs become empty strings). Consider always setting the output when config.vulnerability_check is enabled (use '[]' when empty) for consistency with vulnerable-changes and to simplify consumer logic.

Suggested change

	if (config.vulnerability_check && resolvedVulnerabilities.length > 0) {
	core.setOutput('resolved-vulnerabilities', JSON.stringify(resolvedVulnerabilities))
	summary.addResolvedVulnerabilitiesToSummary(resolvedVulnerabilities)
	printResolvedVulnerabilitiesBlock(resolvedVulnerabilities)
	if (config.vulnerability_check) {
	core.setOutput(
	'resolved-vulnerabilities',
	JSON.stringify(resolvedVulnerabilities)
	)
	if (resolvedVulnerabilities.length > 0) {
	summary.addResolvedVulnerabilitiesToSummary(resolvedVulnerabilities)
	printResolvedVulnerabilitiesBlock(resolvedVulnerabilities)
	}

[Copilot]

The “resolved vulnerabilities” status line is added even when config.vulnerability_check is false (it only checks resolvedVulnerabilities.length). This can cause the summary to mention vulnerabilities despite the user disabling vulnerability checks. Consider gating these list entries on config.vulnerability_check as well.

This issue also appears on line 116 of the same file.

[Copilot]

New resolved-vulnerabilities summary rendering is not covered by tests. There are existing Jest tests for summary formatting in this repo; adding at least one test that asserts the heading/table (and pluralization) for a non-empty resolvedVulnerabilities list would help prevent regressions.

[Copilot]

The workflow example checks steps.review.outputs.resolved-vulnerabilities != '[]', but the action currently does not set this output at all when there are zero resolved vulnerabilities. That makes the condition true for the empty-string case and will cause jq to fail. Either adjust the example to handle unset/empty outputs, or (preferably) ensure the action always sets the output to '[]' when enabled.

[Copilot]

ComparisonResponseSchema now includes a resolved_vulnerabilities field, but the implementation derives resolved vulnerabilities locally from changes and doesn’t read this property. If the API response does not actually include resolved_vulnerabilities, this schema can be misleading; if it does, the action should prefer using it to avoid the false-positive cases around upgrades. Consider either removing this field from the schema or wiring it through and using it as the source of truth.

Suggested change

	snapshot_warnings: z.string(),
	resolved_vulnerabilities: ResolvedVulnerabilitiesSchema.optional().default([])
	snapshot_warnings: z.string()

[Copilot]

getResolvedVulnerabilities() treats any vulnerability on a change_type: 'removed' package as “resolved”. This can produce false positives during upgrades (the old version is removed, but the same GHSA may still exist on the added version or elsewhere in the diff). Consider filtering out advisories that still appear in non-removed changes (e.g., match by ecosystem/manifest/package_name/advisory_ghsa_id), or rename the concept/output to “vulnerabilities on removed dependencies” to avoid overstating remediation.

[Copilot]

README documents a new resolved-vulnerabilities action output, but action.yml does not declare it under outputs. For GitHub Actions consumers and Marketplace docs, outputs should be listed in action.yml as well.

Suggested change

- `resolved-vulnerabilities` holds information about vulnerabilities that have been resolved by removing or upgrading packages in a JSON format.