Actually, from what I've read, "talking turkey" long ago referred to the opposite - chit-chat and pleasant conversation. When people in the U.S. eat turkey, it is usually at some big meal, so the talk would come from people who are full and happy. It's meaning today is "skip the small talk, let's get on to what is important" It may come from that fact that turkeys are a sound you would hear at any farm, like the mooing of the cows - something essential hear if you were running the farm. This is only a guess.