Your comments

The problem is only if code block on some side is deleted, so there's no line numbers for it.

I think it would be good if borders around the current difference had some other color.

There's another advantage of such design. Look at my pictures above. Lines 14—18 are all surrounded by same borders, so it's unclear, which lines are different: it's some mess of the borders. If two (or one, if block is deleted) borders around current difference were in specific color, the picture would be more clear.