I think everything you said is spot on. A building is simply the gathering place for the "called out ones".