none
ISSUE: Unbind Request Received starting IP-DLC conection Event ID:276 0x8a000008 and followed by Event ID 585, 134, 585, 533, 276 RRS feed

  • Question

  • I have My HOSTSYS connection change from ACTIVE to PENDING in a loop. IP-DLC connection was working until MS11-082 was installed. I have changed the configuration back to SNADLC connection and the PU stay up. I don't know what is causing this. I need to have syetem converted to use IP-DLC and MS11-082 installed for the next patching cycle. Could anyone help ?

    EventID 276

    Abnormal UNBIND request received

    Sense code = 0x08a00008

    Local LU name = AUCBA001.PN0228LS

    Partner LU name = AUCBA001.CDRMD1

    Mode name = CPSVRMGR

    UNBIND RU :

    32fe08a0 00086018 f0a7bc62 464a2be9

    0fc1e4c3 c2c1f0f0 f14bc3c4 d9d4c4f1

    351708a0 00080000 0fc1e4c3 c2c1f0f0

    f14bc3c4 d9d4c4f1 00f100bc 62464a2b

    e91004c1 e4c3c2c1 f0f0f14b c3c4d9d4

    c4f10011 c1e4c3c2 c1f0f0f1 4bd7d5f0

    f2f2f8d3 e26018f0 a7bc6246 4a2be90f

    c1e4c3c2 c1f0f0f1 4bc3c4d9 d4c4f12b

    36010134 461a8015 11c1e4c3 c2c1f0f0

    f14bd7d5 f0f2f2f8 d3e22180 00000103

    88801583 11c1e4c3 c2c1f0f0 f14bd7d5

    f0f2f2f8 d3e2002c 0a0708e2 d5c1e2e5

    c3d4c7e2 00620880 02020400 00000003

    88801583 11c1e4c3 c2c1f0f0 f14bd7d5

    f0f2f2f8 d3e2002c 0a02087b c9d5e3c5

    d9404000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    Cause:

    Abnormal UNBIND request received. This may indicate a configuration error, or a protocol error. This error may occur during normal shutdown of the CPSVRMGR pipe when it is no longer required by the node. Note that this message may be logged during normal node shutdown.

    Effect:

    The session will fail with the specified sense code.

    Action:

    If the sense code indicates a configuration error, check for inconsistencies between the configuration at the local LU and the configuration at the partner LU. If the configuration is consistent and the problem results in unexpected session loss, contact support with details of the problem.

    EventID 585

    CPSVRMGR pipe session failure

    DLUS partner = AUCBA001.CDRMD1

    Sense Code = 0x08a00008

    Cause:

    CPSVRMGR pipe failed to specified DLUS. This error may occur during normal deactivation of the CPSVRMGR pipe when the DLUS no longer requires it. Note that this message could be logged during normal node shutdown.

    Effect:

    Any PUs using the specified DLUS are deactivated (that is, DACTPU(cold) is sent. DLUR may attempt to contact one or more backup DLUSs, if configured.

    Action:

    If a pipe with backup DLUS is not initiated automatically manually restart any required PUs

    EventID 134

    Conversation ended by session outage

    TP name = DLURSENDTP

    TP identifier = 00000000 0000000e

    Conversation identifier = 0x48df0004

    Local LU (Alias) = AUCBA001.PN0228LS (PN0228LS)

    Partner LU (Alias) = AUCBA001.CDRMD1 (@I000053)

    Mode name = CPSVRMGR

    Session identifier = c8bbf56c 938c8d7f

    Cause:

    The session being used by a conversation has been deactivated because of a session outage, causing the conversation to fail. Note that this message could be logged during normal node shutdown.

    Effect:

    The conversation will be terminated, either by an APPC primary_rc of NAP_CONV_FAILURE_RETRY, or a CPI-C return_code of CM_RESOURCE_FAILURE_RETRY.

    Action:

    This log gives information on which TPs and conversation have been affected by a session outage. Other, more specific, problem or exception logs give more information on the reason for the session outage. Use the Session identifier to correlate this log with other related logs.

    EventID 585

    CPSVRMGR pipe session failure

    DLUS partner = AUCBA001.CDRMD1

    Sense Code = 0x08a00008

    Cause:

    CPSVRMGR pipe failed to specified DLUS. This error may occur during normal deactivation of the CPSVRMGR pipe when the DLUS no longer requires it. Note that this message could be logged during normal node shutdown.

    Effect:

    Any PUs using the specified DLUS are deactivated (that is, DACTPU(cold) is sent. DLUR may attempt to contact one or more backup DLUSs, if configured.

    Action:

    If a pipe with backup DLUS is not initiated automatically manually restart any required PUs

    EventID 533

    Locate search failed: LU not found

    Sense code = 0x08400007

    Origin CP name = AUCBA001.PN0228LS

    Origin LU name = AUCBA001.PN0228LS

    Destination LU name = AUCBA001.CDRMD1

    Cause:

    A network search for which this node was the originator or the network node server failed to locate the target LU. This may be caused by the target LU name being incorrect, the target system being inoperative, or by link errors in the backbone of the network. Note that this message could be logged during normal node shutdown.

    Effect:

    Session activation will fail with the specified sense code.

    Action:

    If the target LU name is correct, check that the system the LU is defined on is active. If the system is active, check the topology of the network (using the QUERY_NN_TOPOLOGY_* verbs) to ensure that the target system (or its network node server) is reachable from this node.

    EventID 276

    Abnormal UNBIND request received

    Sense code = 0x08390003

    Local LU name = AUCBA001.PN0228LS

    Partner LU name = AUCBA001.CDRMD1

    Mode name = CPSVCMG

    UNBIND RU :

    320f0000 0000601a c8bbf56c 938c8e04

    11c1e4c3 c2c1f0f0 f14bd7d5 f0f2f2f8

    d3e23517 08390003 00000fc1 e4c3c2c1

    f0f0f14b c3c4d9d4 c4f100bc 62464a3c

    781004c1 e4c3c2c1 f0f0f14b c3c4d9d4

    c4f10011 c1e4c3c2 c1f0f0f1 4bd7d5f0

    f2f2f8d3 e26018f0 a7bc6246 4a3c780f

    c1e4c3c2 c1f0f0f1 4bc3c4d9 d4c4f12b

    66020228 461a8015 11c1e4c3 c2c1f0f0

    f14be5d9 d5d3d6c3 c1d3a180 00000104

    91c9d708 a504400a 139a0a3c 46168001

    11c1e4c3 c2c1f0f0 f14bd7d5 f0f2f2f8

    d3e22104 91c9d708 a504400a 4f38e603

    88801583 11c1e4c3 c2c1f0f0 f14bd7d5

    f0f2f2f8 d3e2002c 0a0708e2 d5c1e2e5

    c3d4c700 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    00000000 00000000 00000000 00000000

    Cause:

    Abnormal UNBIND request received. This may indicate a configuration error, or a protocol error. This error may occur during normal shutdown of the CPSVRMGR pipe when it is no longer required by the node. Note that this message may be logged during normal node shutdown.

    Effect:

    The session will fail with the specified sense code.

    Action:

    If the sense code indicates a configuration error, check for inconsistencies between the configuration at the local LU and the configuration at the partner LU. If the configuration is consistent and the problem results in unexpected session loss, contact support with details of the problem.

    Wednesday, July 18, 2012 5:45 AM

Answers

  • Stephen,

    The mainframe upgrade went ahead on Monday and as far as I can see there are no further issues and it is unlikely to know the real cause of the problem.

    • Marked as answer by Peter M Lee Wednesday, August 8, 2012 5:01 AM
    Wednesday, August 8, 2012 5:01 AM

All replies

  • This appears to match the description of a support case that was just opened so I assume that you are addressing this via the support case.

    This type of issue usually requires some HIS and network traces that capture the data flow when the problem occurs.

    If this is the same issue as the support case that was opened, I will add an update here once the problem is resolved.

    Thanks...


    Stephen Jackson - MSFT

    Thursday, July 19, 2012 2:15 PM
  • Yes, the case was raised by me but is held up by financial matters.

    For those interested, the system is Windows Server 2008 SP2 32 bit with HIS2009.

    An interesting aspect is that the PU would become active eventually with MS11-082 installed. The time it takes the PU to become active varies. If the server reboots, the problem comes back again. 

    Removing MS11-082 doesn't resolve the issue as thou the patch has not fully uninstalled.

    MS11-082 only mentioned the following registry key

    For Microsoft Host Integration Server 2009 (32-bit systems):
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\Microsoft Host Integration Server 2009 Hotfix [See KB article 2579598 for details]GDR

    Apart from this registry key and view installed updates. I don't know what was updated by this patch.

    This patch is supposed to prevent DOS attack but is now causing service disruption.

    Enterprise Extender and MS11-082 Deployment are on hold until this is resolved.

    Friday, July 20, 2012 4:37 AM
  • Peter

    -

    What do your colleagues responsible for VTAM think about this problem?

    Perhaps your VTAM colleages can investigate using the NetView Session Manager traces. I assume they have the necessary knowledge of APPN, Enterprise Extender (EE) - aka IP-DLC - and the use of Dependent LU Requester (DLUR)/Dependent LU Server (DLUS) to deal with the interpretation of what can be seen with this tool.

    If your VTAM colleagues do not have access to NetView Session Monitor, they are going to have to suffer the drudgery of taking VTAM buffer traces.

    -

    Your VTAM colleagues should also investigate the output of the following VTAM commands:

    DISPLAY NET,CPCP

    or

    DISPLAY NET,ADJCP,E,ID=nnn where nnn is the CP name defined to the IP-DLC Link Service logic

    They should be able to confirm that the adjacent CP is fully active as an APPN End Node. There should also be a Rapid Transport Protocol (RTP) connection.

    DISPLAY NET,CLSTRS

    They should find that the list includes the names of both the PU statement which represents the adjacent link station for the EE connection and the PU statement which represents the adjacent link station for the DLUR-DLUS connection. I'm pretty sure that the former will show that it is active and that the letter will - according to what I can deduce from your post - not.

    DISPLAY NET,DLUS

    When this shows a fully active pair of CP LU to CP LU sessions using mode name CPSVRMGR, you will either have succeeded in setting up your configuration or be close. I suspect that, currently, the output of this command will indicate that you have not - yet - succeeded!

    DISPLAY NET,EE,LIST=DETAIL

    The evidence from your post is that you have succeeded in setting up an EE connection. The output of this command will be able to verify that.

    DISPLAY NET,E,ID=nnn where nnn is the name of the PU statement representing the EE adjacent link station

    The named resource should be active. I suspect that currently it is.

    DISPLAY NET,E,ID=nnn where nnn is the name of the PU statement representing the DLUR/DLUS adjacent link station

    The named resource should be active. I suspect that currently it is not.

    DISPLAY NET,SESSIONS,LIST=ALL

    Eventually you should find all your desired sessions listed.

    For more information on these commands and the messages they generate, you should refer to the following manuals:

    z/OS Communications Server SNA Operation
    http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/F1A1B7B0/

    z/OS Communications Server SNA Messages
    http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/F1A1C6B1/

    I have assumed that your configuration is the most straightforward and that you have an EE conection between the APPN node supported by the IP-DLC Link Service and a VTAM APPN node, that the VTAM APPN node is acting as a Network Node Server and also as a Dependent LU Server.

    -

    Of course, I have assumed that you have properly understood how to migrate from using the SNA DLC to using IP-DLC.

    I have expended some effort in "improving" the Microsoft-supplied information on this topic. No doubt you have tried to use the "White Paper". In addition you should read the second of my posts dated October 24, 2011, in the thread '"Starter" HIS/VTAM IP-DLC/Enterprise Extender Notes' of October 24, 2011.

    If there is anything that you do not or, more importantly, your VTAM-trained colleague does not understand about my post or, after having followed my directions, your configuration still does not work, please post again.

    In order to be sure you have compatible definitions, please post the VTAM definitions for the XCA major node, the switched minor node for the Enterprise Extender connection (or the model minor node replacing the switched minor node), a PU statement, and the switched minor node for the DLUR-DLUS connection, a PU statement normally followed by LU statements (which may be essentially the same statements as used for the DLC SNA connection).

    These definitions should correspond to the Microsoft HIS definitions as clarified by my post '"Starter" HIS/VTAM IP-DLC/Enterprise Extender Notes' of October 24, 2011.

    Hoping that you are working with a test environment and that the number of messages will be limited, please also post the messages generated by the following commands:

    DISPLAY NET,CPCP
    DISPLAY NET,ADJCP,E,ID=nnn where nnn is the CP name defined to the IP-DLC Link Service logic
    DISPLAY NET,CLSTRS
    DISPLAY NET,DLUS
    DISPLAY NET,E,ID=nnn where nnn is the name of the PU statement representing the EE adjacent link station
    DISPLAY NET,E,ID=nnn where nnn is the name of the PU statement representing the DLUR/DLUS adjacent link station

    -

    Chris Mason

    Saturday, July 21, 2012 4:17 PM
  • My colleague from VTAM thinks the problem is that a connection is started and LU-LU sessions are started and then everything dies because of a topology error.

    I have two servers in the test environment. Two of them have the same problem and then one of them before midday changed to repeated EventID 319 and 731. Rebooting the server has Pending Failed on both HOSTSYS and PEERSYS.

    EventID 319

    UNBIND request received in response to a BIND request

    Sense code = 0x08570004

    Local LU name = AUCBA001.PN0282LS

    Partner LU name = AUCBA001.CDRMD1

    Mode name = CPSVCMG

    UNBIND RU :

    320f0000 0000601a f093f56c 9e43cd50

    11c1e4c3 c2c1f0f0 f14bd7d5 f0f2f8f2

    d3e2351a 08570004 80038126 810fc1e4

    c3c2c1f0 f0f14bc3 c4d9d4c4 f100

    BIND RU :

    31001307 b0b050b3 07879797 87070602

    00000000 00004014 23000011 c1e4c3c2

    c1f0f0f1 4bd7d5f0 f2f8f2d3 e2320008

    02c3d7e2 e5c3d4c7 09030193 f56c9e43

    cd501204 c1e4c3c2 c1f0f0f1 4bd7d5f0

    f2f8f2d3 e20a1300 5cde5854 76632091

    000fc1e4 c3c2c1f0 f0f14bc3 c4d9d4c4

    f12c0904 07c3d7e2 e5c3d4c7 601af093

    f56c9e43 cd5011c1 e4c3c2c1 f0f0f14b

    d7d5f0f2 f8f2d3e2

    Cause:

    UNBIND request received in response to a BIND request. This may indicate a configuration error, or a protocol error.

    Common sense codes which typically indicate a configuration error or a normal race condition include

    0805xxxx - the session could not be activated as session

    activation limits have been reached

    08060014 - the partner LU is not known

    0806xxxx - the BIND specified a resource which is not known

    080Fxxxx - security authorization failed

    0821xxxx - the BIND supplied an invalid session parameter

    0835xxxx - parameter error in BIND RU at offset xxxx

    Other sense codes include

    0812xxxx - session activation failed due to resource shortage

    at the remote node

    083Bxxxx - invalid PCID in BIND RU

    0852xxxx - duplicate session activation request

    0861xxxx - invalid COS name in BIND RU

    088Cxxxx - control vector or subfield missing from BIND RU

    0895xxxx - BIND RU contained a control vector that was in

    error

    0896xxxx - BIND RU contained a control vector that was too

    long

    Effect:

    Session activation will fail with the specified sense code.

    Action:

    If the sense code indicates a configuration error, check for inconsistencies between the configuration at the local LU and the configuration at the partner LU. If the configuration is consistent and the problem persists, contact support with details of the problem.

    EventID 731

    CP capabilities exchange failed because of contention winner CP-CP

    session failure

    Sense code = 0x08570004

    Adjacent CP name = AUCBA001.CDRMD1

    Cause:

    CP capabilities exchange failed because of contention winner CP-CP session failure.

    Effect:

    Contention loser CP-CP session will be deactivated. SNAP APPN will attempt to reactivate CP-CP sessions with this adjacent CP.

    Action:

    This log flags the fact that a CP-CP session failed. Other logs give more details on the reason for the session failure (e.g. insufficient resources, link failure).

    Monday, July 23, 2012 7:10 AM
  • I have sent the following to my contact. Microsoft Support should receive the following information tomorrow.

    Application and System eventlogs, COM.CFG on two servers

    Enabled and captured Snaserver, SNAIP1 traces and captured wireshark traces on two servers

    Firewall is not enabled.

    I installed CU3 (kb2667532) but it didn't fix the issues on both servers.

    Tuesday, July 24, 2012 1:36 PM
  • Peter,

    I've looked at the last set of traces that were provided (maybe not the ones that you mentioned above) and the issues that I saw were all related to connecting to and communicating with the PU that was setup for the IP-DLC link service on that HIS Server. In this case, almost all of the event messages pointed to issues realted to PU PN8011L2. I see that the message above is for a different PU but it looks to be the same errors that are happening.

    One suggestion that was passed back was to have one of the failing HIS Servers use a PU for the SNAIP1 link service that a working HIS Server is using. You'd shutdown the working HIS Server and then re-configure the SNAIP1 link service on one of the failing HIS Servers to use the "working" PU to see if the problem still occurs. If it doesn't then there may be an issue with some of the link service PUs. If the problem still occurs, then maybe the problem is related to some sort of network path issue where all the traffic is not correctly passing through the network and back.

    Matching network traces from the HIS Server network segment and the network segment where the mainframe resides may also help to show if there are some frames being dropped along the path somewhere.

    Thanks...


    Stephen Jackson - MSFT

    Tuesday, July 24, 2012 9:48 PM
  • Peter

    > My colleague from VTAM ...

    Will your VTAM colleague be posting the VTAM definitions and VTAM DISPLAY command output for which I asked?

    > My colleague from VTAM thinks the problem is that a connection is started ...

    I suspect this means that he or she has noted that the Enterprise Extender (IP-DLC) connection is in place and that and that RTP "pipes" have become active. This is all good news - although confirmation from the DISPLAY command output would be helpful.

    > My colleague from VTAM thinks ... LU-LU sessions are started ...

    What evidence does he or she have that LU-LU sessions have been stated? Again, the output from the DISPLAY commands I mentioned in my first response would be helpful.

    When two APPN nodes establish a connection, the CP LU of one may initiate the establishment of a CP LU to CP LU session with the CP LU of the other using mode name CPSVCMG and vice versa. These sessions using mode name CPSVCMG are initiated when both nodes have been configured to initiate these sessions. Note that it is required that these sessions are in place in order to permit the operation of APPN mechanisms between the two nodes.

    In case you are wondering why initiating these sessions is optional, you should note that, as long as there is a concatenation of such sessions between any two nodes in an APPN network, APPN mechanisms can operate. Thus the existence of these sessions between any two nodes is not always required.

    > My colleague from VTAM thinks ... and then everything dies because of a topology error.

    Again, does he or she have any evidence, perhaps unsolicited VTAM messages, which indicate what this "topology error" might be. If there were indeed an APPN "topology error", there would surely be some VTAM messages describing it.

    Something from your initial post worried me:

    >...> IP-DLC connection was working until MS11-082 was installed.

    If everything, including LU-LU sessions where one of the LUs was an SSCP-dependent LU as supported by the SNA DLC Link Service, was working before you installed some maintenance, I may be wasting my time and you have a problem which should be handled by whoever supplied the maintenance, presumably Microsoft.

    I guess it all depends on what you mean by "working".

    Chris Mason

    Wednesday, July 25, 2012 12:53 AM
  • Stephen,

    One of the PU’s that my customer are having issues with is PN8011LS.  I think you have mistaken the ‘S’ for a ‘2’.

    My customer has already tried to reconfigure two of their servers with ones that were working.  One of them worked OK for about 2 hours and then started to experience the same issue and the other one didn’t work from the start.

    Note: My customer is looking after their CRM HIS Server Farm and they experienced the same problem on their test servers only. And I am looking after my customer's legacy HIS Servers and the network is the same.

    Wednesday, July 25, 2012 2:46 AM
  • Chris,

    Pretty much everything you said if you weren’t able to establish a connection at all.  That isn’t our original problem.  Now that I have two problems on two servers. I have already engaged Microsoft Support to look at the traces requested.

    When I said working, I meant PU is active and LU are in sessions.

    Wednesday, July 25, 2012 3:02 AM
  • Stephen,

    I removed CU3, MS11-082, HIS2009 and installed HIS2009 again.

    I cannot get the PU working. It is still showing EventID 713 and 319 in the eventlog. So it looks like MS11-082 is not the cause of problem.

    Thursday, July 26, 2012 1:19 PM
  • Peter,

    I looked at another set of traces today after the VRN setting was removed. The issue was still happening but the errors (save for 2 that I'm looking into) are from the host. The last update was that the connections came up immediately when Mainframe support changed the ownership of the server from one LPAR to the other. The other change was that HIS was reconfigued to use CDRMD3 instead of CDRMD1 for Primary NNS (effectively using  CDRMD3 both Primary and Backup NNS).

    I've asked for traces from this scenario to compare against the ones with the errors.

    Thanks... 


    Stephen Jackson - MSFT

    Friday, July 27, 2012 9:48 PM
  • Stephen,

    The mainframe upgrade went ahead on Monday and as far as I can see there are no further issues and it is unlikely to know the real cause of the problem.

    • Marked as answer by Peter M Lee Wednesday, August 8, 2012 5:01 AM
    Wednesday, August 8, 2012 5:01 AM
  • Peter,

    I was informed of this as well. Since the only changes that were made were done on the mainframe side, the odds are that there was something on the mainframe side causing the problem. You are correct that we are unlikely to know the exact cause of the problem. I did request (and have received) a new set of traces from one of the servers that had the problem after the updates were made. I will have a look at these to see if anything sticks out when compared to the failing traces.

    Thanks...


    Stephen Jackson - MSFT

    Wednesday, August 8, 2012 2:31 PM