Well, you have a problem then, because the church is Christ. All the people who make up the church are the body of Christ. If you were invited to the White House for a state dinner, would you "dress up"? When you go to Mass and receive Jesus in communion, don't you dress respectively...