I mostly see the phrase "tumbling down" used in relation to something that ended quickly, that when it came down, it created a ripple or domino effect, or that many things are ending or in a bad state at once.
For example, your phrase "the world around you is tumbling down" would mean something like many things are going wrong in your life at the same time. Maybe you lost your job, got divorced, and the dog died.