My attempt at understanding this:

Both imparfait and passé composé of devoir reflect a degree of obligation. The “different meaning” is more in whether the “obligation” was met or not, or remains uncertain.

In English, “I was supposed to ...” reflects an obligation and the inference that ‘you didn’t . . .’, unless it is qualified by an affirmative statement, eg “and I did”. Also, in English “I had to but...” is often used in this same context.

In French, “I was supposed to.. “ and the likelihood that it didn’t happen is conveyed with devoir in the imparfait.

Likewise in English, “I had to ...” raises an inference that you did, unless a statement that you didn’t follows. In French that meaning of “I had to and did” is conveyed by devoir in passé composé.

In English “I had to” is sometimes also used figuratively - eg “the room was so small that I had to close the fridge to open the cupboard”. You may or may not have ever actually closed the fridge to open the cupboard -  that ‘uncertainty’ would be expressed with imparfait in French. If there was indeed a defined occurrence of this, the description would usually be expanded in English, and in French passé composé +/- more information would convey that it was not figurative.

The context determines whether the story teller expresses using imparfait or passé composé, and the use of passé composé or imparfait informs the ‘story receiver’ of the context.

Undoubtedly there are likely still gaps in this, some deliberately left out and others just plain flaws, but it has taken me further than just looking for ‘had to’ as the trigger!

Finally, the example is of the form “supposed to/had to but (excuse) ....” and although not stated explicitly, it is likely “aller la voir” did not happen, so imparfait would be used.

Ok, I think I get it now.