"All three of us" means "we three, together, as a group" My mother, father and I will visit you. All three of us will visit you tomorrow. (Not just one of us, or two of us....) "Only three of us" = not more than three. In my family there are only three of us - my two brothers and me.

They have different meanings. "All three of us" connotates that there are only 3 people. "Only Three of us" connotates that there are more than 3 people and the number of people can be very large.