That 40 number is out of date. In a past hard fork, it changed to 10.

In my opinion, you shouldn't really worry about your voting power decreasing, just make sure your VP is always under 100. Any time your voting power reaches 100, you're wasting VP and influence.