The phrase "I was expecting that someone turn up." is correct. The second sentence "I was expecting that someone turns up" would be understood, but is not natural phrasing. You could say, I was expecting that someone might turn up. If you wanted to use "turns", you could say: I hope that someone turns up. Or I expect that someone turns up. (Although this last sentence is a little weird to a native speaker. One might say I expect that someone will turn up, instead - since you are talking about something that will happen in the future.

I was expecting that someone would turn up...or I was expecting that someone might turn up. Hey, I forgot to tell you, it would be more natural sounding to say, "I was expecting that someone might show up." The reason I'd use "show up" over "turn up" is because sometimes the news talks about how someone "turns up" dead. Turn up is ok but usually people would say, "show up" there.