Your Dashboard Says 14,000 Secrets. The Number That Matters Is 525.
Every secret scanner hands you a big number, and almost nobody can act on it. When we verified each finding against the service that issued it, a five-figure detection count became a three-figure inventory of credentials that actually work. This is what that collapse means for how you prioritize, what you suppress, and what you tell your board.
On this page(7)
Table of Contents
Introduction
Open your secret scanning dashboard and read the number at the top. Then ask what you would actually do with it.
Ours said roughly 14,000. That was the count of credential findings across every connected source: repositories, cloud storage, collaboration tools, all of it. The number was honestly produced and technically correct. It is also the number most scanners put in front of you, and the one that ends up in board decks and budget requests.
Then we asked something narrower of each of those findings. Not "does this look like a credential" but "does this credential work right now?" Every candidate went to the service that issued it, and we read the response body rather than the status line.
525 came back alive.
That gap is not a detector bug. Both numbers are correct. They answer different questions, and only one of them describes exposure you can act on. Choosing the wrong one quietly shapes everything downstream: what your team works on this quarter, what you tell an auditor, and what you believe your risk to be.
A count of candidates is not an inventory
A detection is a claim about a string. It says this sequence of characters has the shape of a credential. That is a useful claim and a cheap one to produce at scale.
An inventory is a claim about access. It says this credential exists, it works, and someone or something can use it against a live system today. That claim costs far more to produce, because the only way to make it honestly is to try the credential.
Most security programs treat the first number as if it were the second. You can see why. It arrives for free, it is large enough to justify attention, and it goes up and to the right when you connect more sources. The trouble starts when you try to work it. Fourteen thousand findings cannot be triaged by hand, so they get sampled, or sorted by a severity heuristic, or quietly deferred. Meanwhile the subset that would actually let an attacker in is sitting somewhere in that pile, indistinguishable from the noise.
We wrote about the mechanics of this before, in why secret scanners produce false positives and how active validation fixes them. This piece is about what happens to your numbers, and your decisions, once that validation is actually running.
Where the other 13,000 went
The collapse is not one effect. It is several, and they are worth separating because each one implies a different lesson.
Some findings were never credentials. A UUID shaped like a token, a commit SHA, a base64 fragment, a placeholder lifted from vendor documentation. The detector was wrong about what the string was.
Some were credentials that had already been revoked or had expired. Real keys, correctly identified, with no remaining power. These are the ones teams most often waste time on, because the string is genuinely sensitive-looking and the instinct is to treat it as an incident.
Some were duplicates of the same underlying secret. One credential committed to four config files across three branches is one credential and four findings. Counting findings instead of credentials inflates the number without adding a single unit of risk.
And some were scoped to nothing that matters: sandbox keys, local test fixtures, demo tokens against a service with no production data behind it.
None of those four categories is a failure of detection. They are the expected residue of a system designed to catch everything that might be a secret. The failure is only in stopping there and calling the total a risk figure.
Ubiquity is not innocence
Here is the finding that changed how we think about suppression.
The most common single secret in our data appeared in more than 40,000 places. A Slack incoming webhook URL, copied and re-copied across repositories, scripts, and config files until it was effectively everywhere.
Every instinct says to suppress it. A string that common is almost certainly a sample from a tutorial, or a shared fixture, or something long since rotated. Frequency looks like a proxy for harmlessness, and treating it that way would have cut a huge fraction of the finding count in one move.
It was live. Posting to it would have put a message in a real channel in a real workspace.
That is the argument against every frequency-based suppression rule we have ever been tempted to write. Commonness tells you something about how a string spread. It tells you nothing about whether the credential behind it still works. The only thing that retires a finding is verification, and a rule that suppresses by count will eventually suppress the one that mattered, precisely because it spread the furthest.
This cuts against the usual advice to tune noisy detections down. Tuning is how you fix a detection problem. It is not how you fix a validation problem, and reaching for it here trades a number you dislike for exposure you cannot see.
The bucket that keeps the number honest
There is a way to get a small, satisfying number that is worse than the big one.
If verification has two outcomes, valid and invalid, then every credential you failed to check gets filed as invalid. The validator was missing. The network dropped. The rule was renamed. A service has no verifier written for it yet. In a two-state system all of that reads as "dead," your count drops, and your dashboard looks excellent while live credentials sit inside the resolved pile.
Verification needs three outcomes. Valid, invalid, and indeterminate, and indeterminate must never collapse into invalid. When a service tells you a credential is bad, you can act on that. When you simply could not check, you know nothing, and the only correct behavior is to keep the finding open, surface it, and retry.
This matters twice over. It keeps the count honest, and it is the guardrail on automated remediation. Any pipeline that revokes or deletes on its own must refuse to act on an inconclusive result, because the cost of being wrong is no longer a wasted triage hour. It is a production outage caused by your security tooling. We hold to a simple rule internally: never auto-revoke on indeterminate.
So the real reporting shape is not one number. It is three: verified live, verified dead, and could not verify. The third one is a measure of your own coverage, and watching it is how you find out which services you are blind to.
What you do with 525
A three-figure number behaves completely differently from a five-figure one, and not only because it is smaller.
It is assignable. Five hundred credentials can be given owners, and the ones with no identifiable owner become their own priority queue rather than a rounding error. That is usually where the sharpest risk sits, as we found when we looked at credentials with no identifiable owner.
It is schedulable. You can rotate a few hundred credentials on a real timeline with real owners attached. Fourteen thousand findings produce a program that never finishes and a team that stops believing the backlog is meaningful.
It is defensible. When an auditor or a board member asks what your credential exposure is, "525 credentials confirmed working, with owners assigned to 60% and rotation scheduled" is a position you can hold. "About 14,000 findings" invites the only sensible follow-up, which is what fraction of those are real, and you will not have an answer.
And it is a baseline you can move. A detection count goes up when you connect a new source, which makes it useless as a progress metric. A verified-live count goes down when you fix things. It is the only one of the two that rewards the work.
The number to ask for
If you take one operational change from this, make it a question you ask your tooling and your vendors: of everything you have flagged, how many have you confirmed are live, and how did you confirm it?
Push on the second half. Verification that reads only an HTTP status code will confirm dead keys as live and live keys as dead, because plenty of APIs answer 200 to a request carrying a credential they do not recognize. A real check reads the response body and knows what an authenticated response looks like for that specific service. That work is unglamorous, it has to be maintained per service, and it is the entire difference between a number you report and a number you trust.
At Cremit, this is the line we build on. Argus validates findings against the issuing service with body-level checks, keeps indeterminate as a first-class outcome, and never auto-revokes on it. The result is not a smaller dashboard for its own sake. It is an inventory small enough and true enough that a team can finish it.
Your scanner is probably not lying to you. It is just answering a question you did not mean to ask.
Get the next one in your inbox
Monthly NHI research brief from the Cremit team. One email, high signal.