Since in American English "project" also means "housing development",1 and 2 are correct, but Katya's objection is right with respect to British English (and perhaps Australian English and other varieties I am not familiar with), because "project" does not have that extended meaning in those varieties and therefore a project cannot properly be "under construction"; only the resulting building can. Anyway, to the extent 1 and 2 are accepted, you are right: there is no reason not to accept 5 as well, although its non-restrictive relative clause is a bit too "light" and, for that reason, the sentence does not sound completely natural. So is the restrictive clause in 1, by the way, which makes 2, in its turn, sound more natural than 1: since it is possible to express the same content by means of a restrictive prepositional phrase, the full clause that "which" introduces sounds a bit "heavy" and excessive. Be that as it may, in 5 there is no incompatibility between "which" and the comma; fdmaxey must have made a mistake there. It is the so-called 'relative' "that" (not a relative, actually) that cannot introduce non-restrictive relative clauses (and so cannot be preceded by comma). On the contrary, "which" often introduces non-restrictive relative clauses and whenever it does it MUST be preceded by comma, of course. Sentence 6, which complies with Katya's observation, only needs the indefinite article "a", whose absence after "is" I suppose is a mere oversight of yours, to be completely correct and perhaps the only really appropriate choice if you are writing British English. Sentences 3 and 4, finally, are out for the reasons fdmaxey says.

1 and 2 are correct 3 is wrong because 'which' needs a verb 4 is wrong because no commas are needed, especially between the main subject and the verb. 5 Wrong, because 'which' does not take a comma. 6 Wrong, because you left out the indefinite article 'a' between 'is' and 'project'

I would say the correct one is the last one: number 6. "Which" as a pronoun should refer to the subject is mentioned last in the sentence. In this case the building is under construction, not a project. :) Hope, it helps. :)