Though "no family in the world can compare with the Morrises," they are exactly what Peter desires for himself. Getting along with the wife, having smart, well-behaved children, and two cars. Or in today's terms, perfect suburban simplicity. The fact that his choices in women to be with are a) an excellent hostess who rejected him decades ago and is already married, or b) a 24-year-old Indian woman who is also already married, (and to a major in the Indian army, no less) merely serves to emphasize what the novel has been saying throughout, and what all the other characters have known for years: that he has trouble with women.
Amusingly, one of his more prominent contradictions, his love/disdain for the upper English society, appears once again as an aspect of the Morrises that he loves ("they don't care a hang for the upper classes"), along with the fact that they "like what they like." I'm sure Peter would enjoy that quality in himself, if only he could ever figure out what it actually was that he liked.
