All three could be fine, but have slightly different meanings.
You would use the first one to emphasize that the statement is true. For example, if someone else said "He didn't go back down", and they were wrong, you could say, "He DID go back down."
The other two are similar, but not exactly the same.
"Go back down" means you return to the bottom - you go up, then you go back down. "Go down again" could mean the same thing, but also could mean "go down a second time" - you went down yesterday, and today you go down again.
I hope this helps!