Hi MarkAtCATIC,
By "the ribbon bar Delete X", do you mean the button below under Message?

I tried to test in my Outlook 2016 (Version 1705. build 8201.2207) by clicking the button (Or pree Ctrl + D), both the attachment and the message got deleted as expected.
As far as I know, the attachment will be deleted separately if you select the attachment and then click the Remove Attachment button as follows :

Please feel free to let me know if I have misunderstood anything.
Best regards,
Yuki Sun
Please remember to mark the replies as answers if they helped.
If you have feedback for TechNet Subscriber Support, contact
tnmff@microsoft.com.