They are the same. Also, I don’t agree with Karen’s comments that they are “filler” words. They serve a useful function — similar to “however,” “nevertheless” or “despite X” — of introducing a contrary idea that outweighs the previously-mentioned considerations. Example: “It is true that country X has made great strides both economically and politically. That being said, its standard of living still lags behind most of its neighbors, and civil rights abuses continue to be serious.”

They're the same :) although you would also use a commar after 'having said that'.

Those phrases are, in almost all cases, “filler” phrases that serve little to no purpose in a sentence. If you find yourself writing them, try building your sentence without them. You may find a stronger, clearer way to make your point.