Maybe ask the workers if it's okay to film. Then set up all your gear and even some lights and make it really obvious you're filming a video there. That way people won't care as much