No, 'crowd around' doesn't work for what you want to say. How you word it, depends on the particular nuance you want to give. If you're not mocking them, then you could say "If you're good, don't rush all at once. Let's fight one-on-one." This says no more than don't all come at once. If you want to mock them a little and shame them into just attacking you one by one, then "If you're good, don't be cowards. Let's fight one-on-one".

It would be "don't gang up on me." The phrase "don't be a group of curs" is technically correct but sounds funny, because the word "cur" is very outdated.