I have a wooden desktop as a mousepad, and it works fine with my optical Logitech Wheel Mouse, even when I move it real fast. In school we had those Pro Mices, and they worked fine on the wooden mousepad.
So I guess there is something wrong with your optical eye, maybe it has scratches..