It irritates me that many "believers" are so ignorant and become judgmental as a consequence of it. I don't have a problem with religion directly, but when a person starts to say things like, "well gay people shouldn't be gay, because that's not what I believe in" then it starts to become a huge issue. These people seem to think the world revolves around them, and that their belief is the only one that's right (no matter what kind of scientific evidence/basic common logic they're provided with) and that is my major concern. It is also true (though I suppose debatable like all things) that most major wars were started based on religion and differences of beliefs.
If someone wants to believe something I'm absolutely fine with that, provided they don't try to share it with everyone else, and don't go around promoting hate and discrimination. But, like someone else already mentioned, the problem lies within the people themselves, and not the actual religion. There are -plenty- of people out there who don't believe in any particular religion but discriminate against ethnicity/sexual orientation/gender. It's a problem of the ages and I doubt it will ever see a full end.