Creating rubble is just a matter of having separate models for the destroyed versions of buildings or whatever. These models have the same rendering overhead as any other model. The main way to keep them efficient is to use a less detailed (ie, lower polygon count) model for the rubble. Depending on the type of game, it might also be possible to make the rubble objects disappear completely when the player gets a certain distance away.