When a minor party gets a significant number of first place votes
And that demonstrates why you're entirely off base. Whatever you think "significant" means, the fact is that smaller parties will NEVER be in the running with rank choice voting, because their votes will only be hijacked by whichever major party candidate the voter indicateds with their number 2 vote.
Your entire argument relies on the absurd premise that these smaller parties are more popular than the major parties. If what you say is true, then the solution you are seeking wouldn't be necessary in the first place. If, for example, the Green Party were more popular on the left than the Democrat party, then voter registration would reflect that. Mass organization would
inevitably result, and this would ultimately translate into some form of regular electoral success and the eventual usurpation of the Democratic Party's position in the duopoly by the Green Party.
But that's not the case. The Democratic and Republican parties remain the two most popular choices among the electorate. All that will happen with rank choice voting is that votes for smaller parties will be ABSORBED by the larger two parties, and allow each party to avoid spoiler effects. Rank choice voting in 2000 wouldn't have done squat to give Nader a chance to win. All it would have done is to give Nader's votes to Gore.
The smaller parties will become nothing more than farm teams for the two main parties, and THAT will be the final nail in their coffins. People will hasten their disillusionment with smaller parties, and the ONLY people to retain interest in them will be zealots stubbornly clinging to a niche ideology.
The simple fact is that there's nothing we can do prevent our system from trending toward two major parties. The solution is to set a low bar to entry for smaller parties to enter the scene with alternative ideas, so that major parties can't simply rely on pre-existing size to suffocate out competition.