Journal · sampling
Replay sampling that does not hide rage taps
Most replay tools default to a random slice. Random is fair to users and unfair to problems. Calm power users dominate the draw. Rage taps, OCR retries, and dead-end searches are rare enough to vanish in a 5% sample even when they ruin a release.
Bias toward struggle, then cap it
In Replay Triage Clinic we teach a two-bucket sample. Bucket A is still random, so you remember what ordinary use looks like. Bucket B over-samples sessions with error events, rapid repeat taps, search reformulations, or payment method switches.
Without a cap, Bucket B becomes a horror reel. Analysts burn out. PMs stop watching. Cap struggle sessions at half the weekly review. You need both textures in the same hour.
Rage taps are not all equal
A double tap on a map pin is often curiosity. A triple tap on a disabled primary button is struggle. Your detector probably cannot tell them apart. That is why a human still watches a sample. Automate the draw, not the judgement.
Log the warrant: “repeat tap on Confirm after the button greyed out during 3DS.” If you cannot write the warrant, you do not yet have a finding.
Do not sample only from iOS
Replay coverage is often skewed toward newer phones. If your Android share is large in upcountry Thailand and your recordings are not, write that limitation on every readout. Otherwise you will ship polish for the devices in the room and miss the devices in the wild.
Sampling is part of session quality evaluation, not a prelude you rush. A beautiful score on a biased sample is still a vanity metric, just slower to draw.