I'm guessing you are using iLife 05, yes? I've personally never had a problem with the quality(as of yet...knock on wood). I'll take a try at helping if I can.
What is the source for your audio. Imported sound clips from web, camera, cd or iTunes music etc?
The first thing I would check is...