Well yes, but I would amend that to say that the problem is that they selectively choose to believe and follow certain teachings and not others.

The books and letters in the Bible were written by many different authors over the course of about a thousand years, sometimes using oral source material even older than that.

The Bible does not, to put it mildly, cohere. Even the gospels, written over the course of only several decades, are wildly self-contradicting.

So, you can use the Bible, and many people do, to say just about anything you want it to say.

This is a big problem with Evangelical Protestant Christianity, from one direction. Being Protestants, they believe in sola scriptura, that the Bible is the only appropriate foundation for Christian belief. Add to that that Evangelical Christians believe in infallibility and inerrancy. In other words, they take the Bible literally and believe that their views from the Bible are the only views possible to have. Of course, they can't even agree among themselves what those views should be ...

The Catholic problem with respect to LGBT people and women's equality and all that is a different one.

Sola scriptura arose as a reaction against the Catholic Church's belief in continuing revelation, of constructing dogma and doctrine from tradition as much as from the Bible.

You could show a Catholic theologian that a particular clobber passage in the Bible doesn't actually say anything against homosexuality, and they would be deeply unimpressed even if they agreed with you, which many of them actually do already.

Evangelical Protestant scholars are far more likely to deny textual scholarship and other historical critical work than Catholic scholars, who are actually some of the people doing good historical critical work textual work.

They just don't care if their scholarship shows that certain passages were not composed to condemn homosexuality. They've constructed entire philosophies to do that all on their own, extra biblically.

So yes, belief in the Bible is a problem, but it's far from the only problem.