Checked at every attempt to escape their dreadful bargain, the People turn on the Devil (the man with the "evil eye") as the instigator of the trap: Nostromo is virtually accusing Monygham of setting up this whole outcome from the beginning. He is angered that the Devil should mention "the treasure" -- personal integrity -- while setting up a society in which the People will have none, in which they will be deceivers. The line, "You have sworn my ruin" makes no sense applied to Monygham except as a superstitious raving, but it makes complete sense applied to the Devil in the context of this ongoing metaphor.