SSPR Odd Issue RRS feed

  • Question

  • I have run into a very strange issue that I am uncertain how to fix.

    I have one 2008 R2 server running SQL 2008 R2/FIM Service/FIM Sync Service.  One 2008 R2 server running the pwdreg/pwdreset portals.

    I have 19 MAs, one for the FIMMA and one for each domain in the forest for the static 'domain' attribute.  Everything works as expected.  The users are imported into the MV and then into FIM from the ADMAs.  All users can register with the registration portal.  Only two domains are immediately able to use the reset portal.  All of the other users in the other 16 domains receive an error for which the event logs states 'Password Reset Activity could not find Mv record for user'.  I have verified the users with this issue are in the MV, all attributes flowed correctly. 

    Here comes the strange part.  Once I log into the FIM portal as that user, they are then able to reset their password.  We have thousands of users with new student accounts added almost daily.  It is not possible to each morning log in using their default passwords into the portal just so they can then register/reset their own passwords later.  Again, this does not happen for two of the domains.  All delegated permissions are the same across the board as noted by the successful pwd reset after the account has logged into the FIM portal. 

    What could possibly be causing this?

    Tuesday, July 29, 2014 2:58 PM

All replies

  • Have you looked in AD and compared a newly created account in the "good" domain to the "bad" domain before going to the portal?  Perhaps the problem in at the AD level. Are you certain all key attributes are going into the portal such as accountName, domain and ojectID?  Are you certain that all of the "bad" accounts are in the MV and joined to the FIM MA portal entry?  I had this happen to me the other day and it turns out my admin ID was filtered and got that message.

    Good luck!

    If this post has been useful please click the green arrow to the left or click Propose as answer

    Tuesday, July 29, 2014 3:44 PM
  • With it being 16 out of the 18 did not work immediately I did go through the 'start from scratch' path with each and every one.  They are all setup the exact same.  I also had a few test accounts in several of the domains and looked in the MV and compared each the working and non working.  They are also the same.  Looked in the portal and the non working users are listed in the correct Sets.  Doing a MV search using domain + accountname finds them.  this has been strange....  It just makes no sense as to why after the non working account logs into the FIM user portal they are then immediately able to reset their password.  But prior to that, no go.
    Tuesday, July 29, 2014 4:01 PM
  • Hmm..have you tried turning on verbose logging for the FIM portal and service?  Another thought is to find a "bad" account and log into a machine in the domain where the FIM Service/Portal lives.  Perhaps there is some sort of auth problem between the domains and logging into to a machine might shed light on this.  The bottom line is that I would up the logging and see if I could get some more info.  If the portal works for one account and everyone is joined in the MV correctly, then it should work for all.


    If this post has been useful please click the green arrow to the left or click Propose as answer

    Tuesday, July 29, 2014 5:33 PM
  • My thoughts exactly :)
    Tuesday, July 29, 2014 5:51 PM
  • Extra logging only tells me that it is unable to resolve the gate.  then the next event is the no mv user found.  But still, once this failed user logs into the portal, then they are able to reset their password and those events are not logged.
    Thursday, July 31, 2014 6:34 PM
  • Take a look at this as well as this.  

    If this post has been useful please click the green arrow to the left or click Propose as answer

    Thursday, July 31, 2014 9:48 PM
  • Thank you for the links.  The first I had already verified was not the issue.  The second was a new one I had not seen, but all permissions are correct.  I have even verified them on the individual users that tried and failed to make sure inheritance was checked and working.  I have verified that the users are in the correct Sets.  With the fact that the user is able to reset their password after they log into the portal proves those permissions and memberships are correct.  I just do not understand why they are unable to use the reset portal until after they log into the FIM portal itself.  We actually do not want to give this to our students. 
    Friday, August 1, 2014 11:05 AM
  • Just a follow up on this issue.  As this was never resolved we have decided to direct our students to the FIM user portal where only the register for password reset link is displayed to them.  Then they are able to go through the whole process successfully.  It is a work around but they only have to do it once.
    Wednesday, October 29, 2014 3:01 PM
  • We've had an issue with AAM and cross forest logons for the portal so it got me thinking about this issue again.    From all of the other forums and posts I have read about users having the same issue, all of my settings check out.  Users attributes populated, etc.  So what changes on the account when the user logs into the user portal for the first time that would then allow them to be able to use the password reset portal?  I have watched an account and verified that I cannot actually see any differences before or after the user portal login.  But every single time the user fails to reset their password, logging into the user portal allows it to work immediately.  So I guess my real question would be, does anyone know what logging into the user portal does to the users account that would then allow the password portal to work?
    Thursday, November 6, 2014 3:56 PM
  • I suppose you have the correct kerberos authenction configure. If so, when the user login to the FIM portal, it will request several kerberos tickets from DCs. And after that when you try to login to the SSPR site, it can leverage the cached tickets for authentication. So which domain are these fim components and service accounts deployed? Any relatation between the working and non working domains? probably capturing some network trace can help understand what is the tricky part.

    Friday, November 7, 2014 7:14 AM
  • I think you may be right about the cached token.  I was working with the user portal yesterday and while using the AAM URL users in the 'users' forest cannot logon.  But if they use the FQDN of the portal server they can, then after they can also use the SSPR portal and the AAM url for the user portal.  I think I am almost there just need to figure out where the setting is that is stopping this from working correctly.  The two domains that worked with the SSPR right from the start, no longer work immediately.  They are now following the rest which may be a result of all of the changes I am making to get this to work.

    I'll break down what I have been concentrating on.  The applicationhost file was changed to this:

                        <windowsAuthentication enabled="true" useKernelMode="false" useAppPoolCredentials="true">
                                <clear />
                                <add value="Negotiate" />
                                <add value="NTLM" />

    The webconfig file was changed to this:

    <resourceManagementClient requireKerberos="true" resourceManagementServiceBaseAddress="http://fimsyncserver-internal-FQDN:5725"  <--?????

    SPNs registered are FIMService\portalalias on fim sa acct  HTTP\portalalias on sharepoint app pool acct (both NetBIOS and full).  On both the fim service and sharepoint app pool accounts in AD delegations set for the fimservice\portalalias spn.

    In IIS on portal server Sharepoint-80 authentication enabled for Windows Auth and ASP.NET. On Windows Auth properties disabled kernel mode and providers are as above(negotiate and NTLM).  The four app pools are set to run as:  Sharepoint-80 = sharepoint service account with the AD SPNs, Token service = fim admin account services installed under, Two numeric named app pools = fim admin account services.

    The fim sa and sharepoint app pool accounts have been denied the security policies as listed in the deployment document.

    I enabled logging and this is the error displayed which keeps me thinking Kerberos may not be working as it should.  The exception is what puzzles me to where the issue lies.

    [Win32Exception (0x80004005): Security Support Provider Interface (SSPI) authentication failed. The server may not be running in an account with identity 'FIMService/internalFQDN'. If the server is running in a service account (Network Service for example), specify the account's ServicePrincipalName as the identity in the EndpointAddress for the server. If the server is running in a user account, specify the account's UserPrincipalName as the identity in the EndpointAddress for the server.]

    I know it has something to do with Kerberos and the cross forest logons but just cannot put my finger on it.

    Friday, November 7, 2014 12:23 PM
  • Let me also mention just in case this may be an issue, that the SSPR URL is an external DNS zone (as well as the portal alias).  So the A records are queried not from the AD DNS servers but from external DNS servers.  Not sure if that would make a different when resetting the password on accounts located in the users forest.
    Friday, November 7, 2014 1:43 PM
  • Hmm, okay something else I just found are the SPNs for the registration and reset portal are assigned to the server object in AD (running on same server) and not the fim accounts the app pools run under.  So I'm going to switch those. 
    Friday, November 7, 2014 1:56 PM
  • Well that made no difference.  Still getting the 3000 error and

    Error while performing the password reset operation: PWUnrecoverableError

    in the event log

    Friday, November 7, 2014 3:19 PM
  • And I should probably note just for sake being that we have 23 AD MAs, one for each domain so that a custom string populates the FIM domain attribute correctly in the sync rules.
    Friday, November 7, 2014 8:58 PM
  • Hi Jen,

    Please confirm the following in your environment as you have multiple domains connected:

    1. In FIM Service's confguration file (Microsoft.ResourceManagement.exe.config) you have FQDN of FIM Synchronization Service Active node.
    2. In the same file you have FIMService's address pointing to FQDN of FIMService host with port 5725 specified.
    3. In FIM Portal's web.config file you have FQDN of FIM Service pointed.
    4. In Password Registration and Reset portal's web.config files you have FQDN of FIM Service
    5. SPN for FIM Portal is registered as HTTP/FQDNFIMPortal
    6. SPN for FIM Password reset is registered as HTTP/FQDNFIMSSPR
    7. SPN for FIM Password register is registered as HTTP/FQDNFIMRegister
    8. SPN for FIM Service is registered as FIMService/FIMServiceFQDN
    9. You have specified delegation between FIM Portal and FIM Service accounts (FIM Portal -delegation-> FIM Service  and FIMService -delegation->FIMService)

    Make sure you are using FQDNs in all those places, not a short names.

    Also make sure you have correct value of active FIM Synch node pointed from FIMService's config file.

    If you found my post helpful, please give it a Helpful vote. If it answered your question, remember to mark it as an Answer.

    Sunday, November 9, 2014 11:40 AM
  • I do know right off hand that the FQDN of the FIM sync server is not in those files, it is the NetBIOS name.  I will make that change now and reboot all machines.  I did however register the SPNs with short and long.  I'll report back shortly.
    Monday, November 10, 2014 11:50 AM
  • Still getting the 3000 error and the below logged on the reset portal server. 

    System.Web.HttpUnhandledException: ScriptManager_AsyncPostBackError ---> System.InvalidProgramException: Error while performing the password reset operation: PWUnrecoverableError

       at Microsoft.IdentityManagement.CredentialManagement.Portal.Reset.AttemptToResetPassword()

    Monday, November 10, 2014 12:09 PM
  • So I'm kind of talking out loud in hopes that a light will go off for someone and they will be able to say, there's your problem.

    It just seems odd that users receive the 3000 error when trying to reset their password unless they logon to the portal first.  If this is Kerberos caching I'm confused as the reset portal should not require users to be on a domain joined machine therefore only the QA gate is used for authentication.  So why would it then work after they logon to the portal (immediately).  Obviously if they logon to the portal then wait an hour, the reset site does not work.  It's strange that these are the symptoms and the event logs are recording MVuser not found.  I have verified the users 3 attribs in the connector space and metaverse as this does work immediately after the portal logon.  The web.config file is set to SecurityContextAssertion" value=" nonspecified".  The reset portal app pool account has the registered SPNs both short and long name and a delegation to the fimservice account. Kerberos delegation set as any protocol.

    Tuesday, November 11, 2014 12:53 PM
  • Complete shot in the dark, but are the other domains that FIM is pointing to RODC DC's? (Read Only Domain Controllers)? 
    Tuesday, November 11, 2014 5:11 PM
  • No, I have not installed any read only DCs in either forest.
    Tuesday, November 11, 2014 5:12 PM
  • I know this is old, but I thought I would share if others suffer the same pain I had this evening.  This same error (Password Reset Activity could not find Mv record for user.) also occurs when the AD MA account is locked out.


    Tuesday, January 24, 2017 3:43 AM