One of the most important things we learned while developing CBITS Lens is that a SharePoint storage scan has two very different workloads.
The first is understanding the current files across the environment. The second is looking behind those files at historical versions.
Both matter, but they do not need to be approached in exactly the same way.
Version history is too important to ignore
A document library can look reasonable when you consider only the latest copy of each file. Behind those files, SharePoint may be retaining many historical versions, particularly for large or frequently edited documents.
If an analysis ignores that layer, it can miss a meaningful part of the storage picture.
So Lens includes version history in its analysis. The development question was how to do that without making every customer wait for an exhaustive crawl of every historical copy in a large tenant.
Scan broadly where precision matters most
Lens scans the active file inventory across the environment so it can calculate current-file totals and age-based storage buckets. That gives the report a concrete foundation for questions such as how much current content has not been modified in one, two, three, or five years.
Version history is handled differently. Lens measures a subset of historical versions and uses those observations to estimate the broader version footprint.
This is not about pretending sampled data is exact. The report distinguishes the measured current-file inventory from the version-history estimate so the user can understand how each number was produced.
We improved the sample instead of simply making it bigger
During development, we moved beyond treating every file as equally informative for version analysis. File size matters: a version pattern observed on a tiny document does not necessarily tell you much about a library dominated by very large files.
Lens therefore uses size-aware sampling so the version estimate better reflects the mix of files in the environment. We also track the number of files and historical versions measured as part of the analysis.
The result is a method designed to spend scan time where it adds information rather than treating more API calls as an end in themselves.
Validate the estimator against real SharePoint data
Sampling only becomes useful when you test it against known results. As we developed Lens, we compared sampled Graph measurements with SharePoint version totals in real, large document libraries and refined the approach based on those comparisons.
That validation work gave us confidence that we could dramatically reduce the amount of version-history scanning while still producing a useful planning estimate.
It also reinforced an important product decision: Lens should be transparent about what is directly measured and what is estimated.
Performance is part of usability
A technically complete scanner that takes too long to finish is not automatically a better product. For an analysis tool, the useful question is how much work is required to reach a decision-quality answer.
By combining a broad current-file inventory with targeted version-history measurement, Lens can surface both sides of the SharePoint storage problem without brute-forcing every possible data point.
That balance — depth where it matters, sampling where it makes sense, and clear reporting about the difference — is one of the design choices we are most pleased with in Lens.