It depends on who they are talking to, for example if they are talking to you then they are saying it to you but if they are talking to a group they will refer it to everyone. But usually when someone refers to a group they will say "all of you are beautiful" or "you are all beautiful" !