Resilience Bites #10 - LinkedIn Rewind (week 16)
April 19th, 2025
Hi everyone,
Welcome to this week's LinkedIn Rewind.
This week, I discussed the challenges of building resilient systems, the prevention paradox, where successfully prevented threats are often questioned as real risks. I also addressed the importance of regularly validating runbooks, avoiding checkbox approaches to architectural reviews, implementing adaptive capacity within teams, setting realistic availability goals, and balancing system performance with resilience against real-world chaos.
Finally, I explored how the "Dogs Not Barking" principle, the idea that the absence of evidence can sometimes be evidence itself, has implications for cyber resilience.
Let’s jump right into it!
The Linkedin Rewind
The prevention paradox might be the most significant obstacle organizations face when building resilient systems.
And yet, hardly anyone talks about it.
When we successfully prevent a threat before it happens, people often wonder if the threat was even real in the first place.
[…]
When was the last time you discovered your critical runbook was outdated... during an actual incident?
Runbooks don't stay accurate by themselves. You've got to validate them regularly, or they'll be useless when you really need them.
One good practice for keeping your runbooks in shape is to have your on-call rotation team (secondary, for example) go through and test the runbooks once a week. Call that a mini GameDay :)
[…]
Continue reading on LinkedIn
Problems with architectural or operational reviews
Have you ever noticed how architectural reviews or operational reviews often end up in checkbox exercises?
Do you have monitoring? Check.
Is there redundancy? Check.
Does it scale? Check.
Yet somehow, despite all those checkmarks, systems still fail in unexpected ways, and people are surprised.
[…]
“Dogs Not Barking” and the implication on Cyber Resilience
I recently discussed the "Dogs Not Barking" principle from Sherlock Holmes (link in comments).
"Dogs Not Barking" is the idea that the absence of evidence can be evidence itself.
As my friend Kennedy Torkura from Mitigant pointed out in the post's comment section, this concept has powerful implications for cyber resilience.
While Jake Burns doesn't explicitly mention it, creating adaptive human systems that can evolve with changing conditions isn't just good for cloud migration; it's actually critical for long-term organizational resilience.
Here is why.
When organizations invest in building adaptive capacity within their teams rather than seeking external solutions or hiring new folks, they improve overall resilience.
[…]
Unrealistic availability goals
I've seen way too many organizations chase unrealistic availability goals only to find themselves trapped in an expensive, complex, and ultimately impossible pursuit.
This obsession with the infamous "nines" often comes from the lack of understanding about the relation between availability, cost, and complexity.
Take a look at this table. Let it sink in for a minute.
[…]
Balancing performance and real-world chaos
What good is setting records at Nürburgring if your car can't survive the road?
This is the exact challenge we face with software systems: balancing performance with real-world chaos.
To address this challenge, engineers use two key methodologies that serve distinct purposes.
[…]