When everyone was absent, my thoughts were with me.

 X: "I will not prune even a millimeter of my thoughts for anyone’s sake, under any circumstance."

Y: "But that is so arrogant."

X: "When everyone was absent, they were with me. And when you leave, too, only they will remain with me once again."