First of all, he is currently a family father (and the heir to the heir).
His mother in law, the grandmother of his children, the woman who allegedly catered to all his needs for a decade, including cheese toast, became 64.
His SIL threw a party for her.
And he stays home because he doesn't like his SIL's in-laws (i. e., a very small part of the other guests)? I think that would be really selfish, and as you said, to meet them at least once a year for 4-5 hours can't be that much of a burden.
Security issues couldn't have been the problem, because if it's safe enough for Kate, then it should be safe enough for him as well (I am sure her RPOs stayed close as long as she was there, and it's not that Pipster & Hubby live in a problematic area of London ...).
But yeah, it's Willnot. I could totally imagine that he refused to come with Kate just because he didn't want to grace them with his presence.
I think he is an entitled prat but stayed away due to his Felipe-type problem not because of his Entitled Pratness: as heir to heir he can't be seen to be fraternising with a family member accused of a serious crime/s - David Matthews. Uncle Andrew knows all about that!
Edit: assuming David was at the party