I think it depends on who you speak to. It'd be better if you could speak correct English as you'd be judged by it. It is a sad fact that you'd be regarded as less educated if you speak broken English. However, you shouldn't be discouraged by it. It is better that you can communicate in multiple languages whether or not they are imperfect. Considre this, a perfect English speaker who is monolingual and might disregard you as less educated due to your broken English is far behind you. So, I'd say it is much better that you master the language in all aspects, but you're not less than anyhone who is monolingual since you, at least, are trying to understand another language and culture.

That's absolutely right. but i think, the basic grammar is still usefull.

Correct grammar usages are important. But maybe it is not what have made feel upset about. People can express their thoughts using simple sentence structures. When you get used to chatting with foreigners, you will feel less nervous. Chatting with foreigners also helps to improve our oral, listenning and grammar usages.