I agree with what Carolina said - the kids had us climbing the walls (because we were angry or worried etc.), or they were 'driving us up the wall' meaning they were annoying us. But I have also heard and seen people using it to refer to the kids themselves, for example when they are bored. Think of the summer holidays when kids have nothing to do and lots of energy. In this case you could also say that the kids were climbing the walls (out of boredom).

Hello "To be climbing the walls" or "to be going up the wall" is the reaction that YOU would have to your children being noisy. These idioms are used when we want to describe a negative reaction, that had the person feel like they were going crazy. So in this case, you could say, "the children were so noisy they had us climbing the walls/going up the wall." Best of luck with your English! Carolina.