It's fine if you can't speak grammatically correct in your conversation as long as you have the confidence to say it and the other parties understand what you mean. I'm an English teacher who teaches younger learners and most of them can't speak or even understand English. However, in that school, we must speak English no matter what. It takes a lot of efforts to make them understand what we say and teach them how to speak English. But after some months, some of them can already speak English. Not perfectly but they have the confidence which can improve day by day. So don't give up and keep trying! Do your best! :)