How to make a realistically destructible stone wall?

I’m trying to create a stone wall, that when shot with heavy rocks, is knocked down. What I’ve got so far is a health system, where each “block” or chunk of wall has a health value. As it gets battered, and the health goes down, it shifts to a more damaged looking chunk, with an animation of rocks falling down, and stone crumbling. When it reaches 0 health, that block of wall collapses entirely.

The primary problem I’m facing is how to handle the structural integrity as a whole, I’m not sure when to let above chunks fall down. When there is only a small hole in the wall, the above layer would not instantly collapse. When a chunk is floating in midair, the surrounding chunks destroyed, it would collapse. I’m not sure what the correct midpoint is, and I’m finding a huge lack of information online about the physics behind stone walls being demolished with flung heavy rocks.

I’m not set in stone (get it?) on my current implementation, and so I ask, how would you make a realistically destructible stone wall?

This might help: Unity 3D Student - 3d Modelling