While I still hate phones going off, people talking is indeed worse.
What I find astounding is that several times I've had people talking constantly and very animatedly (throwing arms around etc.) while sitting in front of me. The thing is, this has happened when these people are in the second or third row! Like WTF, if you're going to be an asshole and talk, go up the back at LEAST, so the lecturer doesn't have to see you and the people at the front obviously because they want to learn don't have to listen to your crap.