To me they sound exactly the same! Maybe I am wrong though. A large number of soldiers attacked the building. Large numbers of soldiers attacked the building.

"A large number of " is talking about a specific group (definite reference). Whereas "Large numbers of" is not talking about a specific group (indefinite reference) Over there is a large number of soldiers. You cant say "Over there is large numbers of soldiers" It is the same with a lot of things... The library has huge collections of music. The library has a huge collection of music I really like. Although meaning is very similar, I wouldn't worry too much.