It is only you ;) No, it could be you and anyone who is more used to speaking and listening to American English and less exposed to British English,specially that the American English is more prevalent in most media. The differences are yet not so big , mainly that of accent and some vocabulary specially informal expressions and slang.

Honestly, neither one is harder to understand than the other. It comes down to how well you recognise the style, which depends on how much US/UK English you listen to.

Yes, it's only you!

Yeah, it's YOU.