"Fact" in relation to a subjective relationship again. The doctor means that Charles Gould is too idealistic for the moral degradation of Costaguana. Unbeknownst to him, he is right in the opposite sense: Charles Gould is now too amoral for the climate of passionate idealism that is Costaguana.
Ironically, Charles Gould will no longer live in Costaguana; he will
live in the Occidental Republic, the nation of the material interests that is
much more amenable to his amorality.