You did the best thing because since your plan is to make the place a farm land not to build house and since people have dominate they place, it will not be of help to if you continue on your plan there.

Because the people might take advantage of your farm, they will just enter at anytime to take anything they want.

So you have really done the Best thing and that is this disadvantage of not deployed the place before you travel out, you can do the farming and you travel out without the help of someone to care proper care of it.