The implication is that he does not like children; certainly that they have been concealed from him on purpose. It is a reminder that the Goulds have no children of their own.