Crash-free is not the same as healthy
A title can sit at 99.7% crash-free sessions and still be the sickest app in the portfolio. Here is how that happens, and what we read instead.
Most weekly packs we are handed open with crash-free session rate. It is a comforting figure. It is also a poor way to rank a portfolio. A staff rostering app used for four short sessions a day can look calmer than a consumer checkout app used once a fortnight, even when the rostering app is quietly dropping a tenth of clock-ins.
Crash-free counts sessions that did not end in a crash. It says nothing about the session that never started because the previous version failed to update. It says nothing about the webview that froze without a crash report. It says nothing about the login SDK that three of your titles share, so the ‘healthy’ apps are one library bump away from a bad Tuesday.
When we read a portfolio we keep crash-free on the table, then sit three other figures beside it. Time-to-first-failure after a release, counted in hours, not days. Support tickets tagged to a version, even when the ticket does not mention a crash. And day-7 return for the titles that should see people come back — which is not every title; a parking app is allowed to be episodic.
Shared libraries are the other trap. If five apps embed the same payments webview, a crash cluster that looks modest in each title is a portfolio event. Ranking apps in isolation hides that. The heat table we draw for a Portfolio Health Review forces shared clusters into a row of their own, so nobody ‘owns’ the failure until the library owner is named.
If your pack still leads with a single crash-free number per title, ask for the release marker and the sampled-versus-full note on the extract. Sampled crash logs from a vendor console are not a census. We will not rank a sampled title against a fully instrumented one without saying so in the brief.