I read Stefan's line more as "and I won't always be there to protect you," but I think it was basically the writers admitting that they'd written things to the point where Damon's going to do whatever he wants to whoever he wants, and no one will actually stop him. I mean, we reached the point where the only reason everyone wasn't ganging up to kill him around midseason one and the only narrative reason not to is "But he's a main characters!"

Have you seen the new trailer, where Damon is manhandling Elena and threatening her with physical violence? Because it is so charming, let me tell you.