I’m having a hard time understanding the application of the grammar to this sentence from a quiz

首相とも あろう 人にお 目めに か か れ る とは光こう栄えいで す 。 It's an honor to be recognized by the Prime Minister.

I checked some other sites and they also seem to include that the second part should be undesirable or not aligned with expectations.

"~のような素晴らしい人が・・・"

話す人が高く評価している人が、ふさわしくない行動をして驚いた、がっかりしたと言いたい時に使います。

The explanation is correct, but I think the "despite" in the header feels awkward, as it would often be structurally unnatural to translate the Japanese examples into English that way. I think the prime minister example above feels odd because most of the examples of ～ともあろう seem to be cases of A being well regarded and B being a disappointing or undesirable outcome. However, ～ともあろう is more broadly used to express surprise or disappointment in light of how the subject is regarded: that is, despite my expectation of A, B resulted. A is often, but not necessarily, a person, group, or position of high status, respect, or responsibility, while B expresses a surprising or undesirable outcome. The key is really the contrast with the expectation set by A’s standing. A doesn't have to be positive and B doesn't have to be negative. For example, 彼女ともあろう人が、こんなに寛大な対応をしてくれるなんて、さすがだね。(It's impressive that she, of all people, would respond so generously.) In this case, the expectation of A is negative, and B is contrary to expectations but positive.