No quick answer is forthcoming so I am working it out:
Remove the credential
Find your TenantID : Install-Module AzureRM : login-AzureRMAccount
Find the KeyID : Get-MsolServicePrincipalCredential -AppPrincipalId 981f26a1-7f43-403b-a875-f8b09b8cd720 and put * in ReturnKeyValues
Remove the MsolServicePrincipalCredential :
Remove-MsolServicePrincipalCredential -KeyIds @("keyID you found above") -TenantID yourTenantID
-AppPrincipalId 981f26a1-7f43-403b-a875-f8b09b8cd720
Check which certificate you are using
New-AdfsAzureMfaTenantCertificate -TenantID yourTenantID | outfile mycert.cer
Use the certificate snapin to remove the certificate on the ADFS farm.
Not found out how to remove it in Azure yet.
CarolChi