It's very likely that the pot was left in the tree. Yep, Opuntias aren't native to your part of the states, they originated further South in desert and semi-desert areas. They are extremely invasive and grow all over where I live, which is why I thought of a bird spreading it around