There are several layers to this line, reflecting from the symbolism of the novel itself. Nostromo's hidden silver on the Great Isabel stands symbolically for the hidden treasure of intrinsic worth in the subjective personality of the People. This sailor, on the other hand, seems devoid of that sort of worth. So the line, on the surface directed from the thief to the sailor, can be read as if it was directed from the thief to Conrad himself, with a rough meaning of, "My story, as such, did not provide the thematic heart of your novel, did it?"