"when you use that to create an abstraction and arbitrarily apply it to external objects, that is when we have a problem"
Property rights are not arbitrary. There are things like, homestading principle, trade, inheritance etc...