Your sentence is wrong. It should be, "You are way over your head". "Way over" means too much. See the note usage at the end of http://www.thefreedictionary.com/on+the+way

Right. The only time you should see "in way" together would be in a preposition-adverb: "You are in way over your head." The reason is that "in" is a preposition and "way" is an adverb that enhances "over." Without "way," you have "You are in over your head" which means the same thing, but is less intense. In a different sense, "in the way" means to obstruct, or occupy an area since "way" also means "path."

No. The expression is "To be in over one's head". Literally, it means that you're IN water (or whatever) that's deeper than you're tall. Now you should see where that "IN" is coming from. "Way" (without IN) means very, to a great degree.

Way as an adverb can mean much or very.