Height does not matter. In leelikesbikes.com´s book he talks about how Peaty and Lopes descend about equally, using their different physical statures in quite differing styles. Peaty powers through the corner at the apex or something, and Lopes just scoots around smoothly and sprints up to speed. I think that´s a bad example, but any time you try to generalize this broadly about DH racing, it is simply not true. Just know how to use what you got and be happy, it really doesn´t matter too much.
Robbie Bourdon is a midged, but Cedric if I remember, we looked eye-to-eye, around 5'11". Peat is a big boy. 6 foot something, but also he is pretty heavy I thought, around 230, if I'm not mistaken. I generally think more "pro freeriders" and small, Richy Schley is a munchkin. Racers on the other hand varry in all shapes and sizes.