I'm not quite sure what your question is about a "that-clause", but in your example, "that" introduces an independent clause which can be treated as a separate sentence. So as far as placing "yesterday" goes, you can set "I'm terribly sorry that" aside. Yes, the best place for "yesterday" is after "gratitude". That's because certain adverbs, including those of time, naturally go towards the end of the clause. Now you also have another adverbial phrase relating to time, which is "when we parted after our last class". This would more naturally follow "yesterday" because it is more specific. You could move "yesterday" to the very end, but then it would be taken to modify "parted" rather than "didn't show", which is not really critical in this example, but you perhaps could invent analogous examples where it would make a difference. You could also put "yesterday" at the front of the clause if you particularly wanted to emphasize it.

Adverbs of time are usually put at the end of the sentence.If you don't want to put emphasis on the time, you can also put the adverb of time at the beginning of the sentence.

I think it sounds awkward the way it is at the top because "that" needs to be directly followed by the clause's subject. We never separate that from the subject of the clause it starts.