Hi,
Just to understand. Bascially they want to show the password reset functionality, only after failed login attempt and if you are a member of a specific AD group?
If you have Office365 or Azure Active Directory, you could look into the password reset functionality available there. I know that it has filtering on AD groups. If you do not already have it, any other third party password reset would properly do the same.
The custom redirection is a bit more tricky, I'd guess that would require some javascript development on the side.