And how it "knows" what the temperature of a distant piece of matter might be.
Just like a rock knows which direction is down. Amazing that you guys think that matter and energy must be intelligent in order to obey the laws of physics.
Lets take a piece of matter at, say 100K, that happens to be floating in a gas cloud of 200K. We'll call this piece of matter, Chunk A. Per your contention, it cannot radiate in any direction.
Now another chunk of matter approaches. It is 98K. We'll call that Chunk B. Your contention says that A will radiate selectively in the direction of B (and you put no range limitations on this but I'll save that for another day). As B shoots past A, the beam of photons you believe A is able to emit in the direction of B will rotate to not only keep it in aim, but TO LEAD IT, since photons travel at a high, but finite velocity. When asked to suggest any mechanism that would allow or enforce such behavior, you tell us the "Second Law" (as if it were the Zeroth Commandment) and that we simply do not (and perhaps can not) know how this works.
Science, on the other hand, states that Chunk A is constantly emitting photons in all directions and of a spectrum appropriate to its temperature (per SB). Chunk B comes by, also emitting photons in all directions per its temperature. The net flow between the two results in an excess of photons (and the energy they contain) going from A to B. The result is the same as your scenario, but no insane voodoo is required.
It is a perfect violation of Occam's razor to reject a fully understood and mathematically symmetrical explanation for an observed phenomenon (radiative heat transfer) in favor of one which reeks of inexplainable actions and multiple violations of basic physical principles which you attempt to justify with "we don't know". I can see few causes for your choices here, but the most likely are:
1) You realized that you could use this to argue against the greenhouse effect
2) You are contrarian to the point of insanity
3) You are trolling
4) You are extraordinarily stupid.
And nothing prevents reality from being composed of a blend of these reasons.