having said that
A phrase used to negate something that you just said so that you can then say what you actually think.
Larry: The book looks great, Jason. Having said that, I would consider it more of a pamphlet than a book. It's looking a little thin.
having said that
A phrase used to completely negate what you said before. It is often used by an individual who is weary of revealing their true feelings on a subject matter.
Jeb: I'm not a racist but having said that I think all non-white people are dumb.
Jeff: So you are saying you are a racist...
Jeb: Yes. Absolutely.
Jeff: So you are saying you are a racist...
Jeb: Yes. Absolutely.
shouldn't have said that
The split second after you've said something that you really shouldn't have said, specifically if it results in something bad.
Guy #1: You motherfucker!
Guy #2: What did you say?
Guy #1: Uh oh, shouldn't have said that.
Guy #2: What did you say?
Guy #1: Uh oh, shouldn't have said that.