Possibly your frustration with Newman is due to the fact that Newman is playing against character. Typical Newman plays a strong character only rarely showing weakness and most often the hero. In his role of Brick he certainly is no hero. Newman plays Brick as Williams wrote the role, a weak, guilt ridden, drunk. Only in the closing scenes of the movie do we see Brick defending Maggie, even thou he believes she is lying leading to some sort of reconciliation.
I think both Taylor and Newman were very good. However, Burl Ives as Big Daddy was wonderful casting. He played the same role on Broadway. Ben Gazzara, who played Brick in the Broadway production would have been a better choice than Newman but Newman was a better box office draw.
You would get some argument here on Taylor not being sexy. In her youth she had a near perfect figure, 34-21-25. Taylor was never known as the hottest actress in Hollywood but rather the hottest actress who consistently gave great performances (6 Oscar nominations and two win) .