I suspect that you may also benefit from eliminating the other things that I have listed. Jack Kruse specifically lists them as out in his Epi Paleo Rx, which I have partially based my current diet regime on. Not because I think Kruse is all wise or anything, but because he said it, and I already had proven it to myself 3 years ago when I first got chronic allergic asthma so bad I didn't sleep for 3 days when it first started.
I don't really have any significant problems with dust in general, but some kinds of dust are bad. I do sweep and mop my floor pretty regularly.