Cluster Failover Failed to bring secondary node online
-
Thursday, March 08, 2012 10:24 AM
Hi,
i have 2 node cluster on MS SQL Server 2008 R2 on MS Windows 2008 R2. when i perform cluster test and move resource from node 1 to node 2 than it fails, following error wriiten in cluster event log.
The Cluster service failed to bring clustered service or application completely online or offline.
One or more resources may be in a failed state. This may impact the availability of the clustered service or application.
Cluster resource 'SQL IP Address 1 (Cluster-XXX)' in clustered service or application 'Cluster-XXX' failed.
Cluster resource 'IP Address XX.XX.XX.XX' in clustered service or application 'Cluster-XXXXX' failed.
Cluster disk resource 'Cluster Disk 3' indicates corruption for volume '\\?\Volume{f576887de3-5192-11e1-b66c-0016447f7a355}'. Chkdsk is being run to repair problems. The disk will be unavailable until Chkdsk completes. Chkdsk output will be logged to file 'C:\Windows\Cluster\Reports\ChkDsk_ResCluster Disk 3_Disk3Part1.log'.
Chkdsk may also write information to the Application Event Log.
than i revert back resource to Node 1 and working. please guide why fail occur what configuration cause problema and how to fix it.
thx.
iffi
All Replies
-
Thursday, March 08, 2012 11:07 AM
Hi,
You might have better luck in the Windows forums. Also, please post the contents of ChkDsk_ResCluster Disk 3_Disk3Part1.log
Thanks,
Andrew Bainbridge
SQL Server DBA
Please click "Propose As Answer" if a post solves your problem, or "Vote As Helpful" if a post has been useful to you -
Thursday, March 08, 2012 11:31 PMIt might also be good to see the cluster log.
-
Friday, March 09, 2012 5:17 AM
hi,
here is log. pls advice
The type of the file system is NTFS.
Volume label is New Volume.
CHKDSK is verifying files (stage 1 of 3)...
0 percent complete. (0 of 19968 file records processed)
Deleted corrupt attribute list entry
with type code 128 in file 344.
Deleted corrupt attribute list entry
with type code 48 in file 344.
Deleting corrupt attribute record (48, "")
from file record segment 344.
Deleting corrupt attribute record (128, "")
from file record segment 427.
Deleting corrupt attribute record (128, "")
from file record segment 410.
Deleted corrupt attribute list entry
with type code 128 in file 462.
Deleted corrupt attribute list entry
with type code 48 in file 462.
Deleting corrupt attribute record (48, "")
from file record segment 462.
Deleting corrupt attribute record (128, "")
from file record segment 724.
Deleted corrupt attribute list entry
with type code 128 in file 498.
Deleting corrupt attribute record (128, "")
from file record segment 405.
1 percent complete. (1997 of 19968 file records processed)
2 percent complete. (3994 of 19968 file records processed)
3 percent complete. (5991 of 19968 file records processed)
4 percent complete. (7988 of 19968 file records processed)
5 percent complete. (9984 of 19968 file records processed)
6 percent complete. (11981 of 19968 file records processed)
7 percent complete. (13978 of 19968 file records processed)
8 percent complete. (15975 of 19968 file records processed)
9 percent complete. (17972 of 19968 file records processed)
19968 file records processed.
File verification completed.
Deleting orphan file record segment 405.
22 large file records processed.
0 bad file records processed.
0 EA records processed.
0 reparse records processed.
CHKDSK is verifying indexes (stage 2 of 3)...
11 percent complete. (292 of 20202 index entries processed)
Correcting minor file name errors in file 462.
12 percent complete. (591 of 20202 index entries processed)
13 percent complete. (890 of 20202 index entries processed)
14 percent complete. (1189 of 20202 index entries processed)
15 percent complete. (1488 of 20202 index entries processed)
16 percent complete. (1787 of 20202 index entries processed)
17 percent complete. (2086 of 20202 index entries processed)
18 percent complete. (2385 of 20202 index entries processed)
19 percent complete. (2684 of 20202 index entries processed)
20 percent complete. (2983 of 20202 index entries processed)
21 percent complete. (3282 of 20202 index entries processed)
22 percent complete. (3581 of 20202 index entries processed)
23 percent complete. (3880 of 20202 index entries processed)
24 percent complete. (4178 of 20202 index entries processed)
25 percent complete. (4477 of 20202 index entries processed)
26 percent complete. (4776 of 20202 index entries processed)
27 percent complete. (5075 of 20202 index entries processed)
28 percent complete. (5374 of 20202 index entries processed)
29 percent complete. (5673 of 20202 index entries processed)
30 percent complete. (5972 of 20202 index entries processed)
31 percent complete. (6271 of 20202 index entries processed)
32 percent complete. (6570 of 20202 index entries processed)
33 percent complete. (6869 of 20202 index entries processed)
34 percent complete. (7168 of 20202 index entries processed)
35 percent complete. (7467 of 20202 index entries processed)
36 percent complete. (7766 of 20202 index entries processed)
37 percent complete. (8065 of 20202 index entries processed)
38 percent complete. (8364 of 20202 index entries processed)
39 percent complete. (8663 of 20202 index entries processed)
40 percent complete. (8962 of 20202 index entries processed)
41 percent complete. (9261 of 20202 index entries processed)
42 percent complete. (9560 of 20202 index entries processed)
43 percent complete. (9859 of 20202 index entries processed)
44 percent complete. (10158 of 20202 index entries processed)
45 percent complete. (10456 of 20202 index entries processed)
46 percent complete. (10755 of 20202 index entries processed)
47 percent complete. (11054 of 20202 index entries processed)
48 percent complete. (11353 of 20202 index entries processed)
49 percent complete. (11652 of 20202 index entries processed)
50 percent complete. (11951 of 20202 index entries processed)
51 percent complete. (12250 of 20202 index entries processed)
52 percent complete. (12549 of 20202 index entries processed)
53 percent complete. (12848 of 20202 index entries processed)
54 percent complete. (13147 of 20202 index entries processed)
55 percent complete. (13446 of 20202 index entries processed)
56 percent complete. (13745 of 20202 index entries processed)
57 percent complete. (14044 of 20202 index entries processed)
58 percent complete. (14343 of 20202 index entries processed)
59 percent complete. (14642 of 20202 index entries processed)
60 percent complete. (14941 of 20202 index entries processed)
61 percent complete. (15240 of 20202 index entries processed)
62 percent complete. (15539 of 20202 index entries processed)
63 percent complete. (15838 of 20202 index entries processed)
64 percent complete. (16137 of 20202 index entries processed)
65 percent complete. (16436 of 20202 index entries processed)
66 percent complete. (16734 of 20202 index entries processed)
67 percent complete. (17033 of 20202 index entries processed)
68 percent complete. (17332 of 20202 index entries processed)
69 percent complete. (17631 of 20202 index entries processed)
70 percent complete. (17930 of 20202 index entries processed)
71 percent complete. (18229 of 20202 index entries processed)
72 percent complete. (18528 of 20202 index entries processed)
73 percent complete. (18827 of 20202 index entries processed)
74 percent complete. (19126 of 20202 index entries processed)
75 percent complete. (19425 of 20202 index entries processed)
76 percent complete. (19724 of 20202 index entries processed)
Correcting error in index $I30 for file 141.
Sorting index $I30 in file 141.
Deleting index entry WS2B2D~1.MDF in index $I30 of file 141.
Deleting index entry WSS_Content_Finance.mdf in index $I30 of file 141.
76 percent complete. (20012 of 20202 index entries processed)
Deleting index entry ERRORLOG.1 in index $I30 of file 263.
Deleting index entry SQLFT0001200005.LOG.3 in index $I30 of file 263.
Deleting index entry SQLFT0~1.3 in index $I30 of file 263.
77 percent complete. (20023 of 20202 index entries processed)
77 percent complete. (20033 of 20202 index entries processed)
77 percent complete. (20036 of 20202 index entries processed)
77 percent complete. (20039 of 20202 index entries processed)
77 percent complete. (20042 of 20202 index entries processed)
77 percent complete. (20045 of 20202 index entries processed)
77 percent complete. (20049 of 20202 index entries processed)
77 percent complete. (20052 of 20202 index entries processed)
77 percent complete. (20056 of 20202 index entries processed)
77 percent complete. (20063 of 20202 index entries processed)
77 percent complete. (20074 of 20202 index entries processed)
20202 index entries processed.
Index verification completed.
CHKDSK is scanning unindexed files for reconnect to their original directory.
Recovering orphaned file SharePoint_Config.mdf (356) into directory file 141.
Recovering orphaned file SharePoint_Config_log.LDF (357) into directory file 141.
77 percent complete. (4 of 16 unindexed files scanned)
Recovering orphaned file SharePoint_AdminContent_6d6322df-967f-40c5-9201-91bc2d913f5d.mdf (358) into directory file 141.
Recovering orphaned file SharePoint_AdminContent_6d6322df-967f-40c5-9201-91bc2d913f5d_log.LDF (359) into directory file 141.
Recovering orphaned file log_122.trc (410) into directory file 263.
Recovering orphaned file 4TESTS~1.KST (411) into directory file 500.
Recovering orphaned file 4.TestSuiteSK.kstore (411) into directory file 500.
Recovering orphaned file SEARCH~2.MDF (448) into directory file 141.
Recovering orphaned file SEARCH~2.LDF (456) into directory file 141.
77 percent complete. (11 of 16 unindexed files scanned)
Recovering orphaned file WS2B2D~1.MDF (462) into directory file 141.
Recovering orphaned file SEARCH~4.MDF (465) into directory file 141.
Recovering orphaned file SEARCH~4.LDF (476) into directory file 141.
Recovering orphaned file SEARCH~3.MDF (491) into directory file 141.
Recovering orphaned file SEARCH~3.LDF (494) into directory file 141.
16 unindexed files scanned.
Recovering orphaned file SECURE~1.MDF (509) into directory file 141.
Recovering orphaned file Secure_Store_Service_DB_9a2f9f51-355c-40e9-97cc-e32c4c0aba2a.mdf (509) into directory file 141.
Recovering orphaned file SECURE~1.LDF (510) into directory file 141.
Recovering orphaned file Secure_Store_Service_DB_9a2f9f51-355c-40e9-97cc-e32c4c0aba2a_log.LDF (510) into directory file 141.
CHKDSK is recovering remaining unindexed files.
2 unindexed files recovered.
CHKDSK is verifying security descriptors (stage 3 of 3)...
Inserting an index entry with Id 317 into index $SDH of file 9.
Repairing the security file record segment.
Deleting an index entry with Id 305 from index $SII of file 9.
Deleting an index entry with Id 306 from index $SII of file 9.
Deleting an index entry with Id 307 from index $SII of file 9.
Deleting an index entry with Id 308 from index $SII of file 9.
Deleting an index entry with Id 309 from index $SII of file 9.
Deleting an index entry with Id 311 from index $SII of file 9.
Deleting an index entry with Id 313 from index $SII of file 9.
Deleting an index entry with Id 315 from index $SII of file 9.
Deleting an index entry with Id 316 from index $SII of file 9.
Deleting an index entry with Id 319 from index $SII of file 9.
Deleting an index entry with Id 328 from index $SII of file 9.
Deleting an index entry with Id 329 from index $SII of file 9.
Deleting an index entry with Id 330 from index $SII of file 9.
Deleting an index entry with Id 331 from index $SII of file 9.
Deleting an index entry with Id 332 from index $SII of file 9.
Deleting an index entry with Id 333 from index $SII of file 9.
Deleting an index entry with Id 334 from index $SII of file 9.
Deleting an index entry with Id 335 from index $SII of file 9.
Deleting an index entry with Id 336 from index $SII of file 9.
Deleting an index entry with Id 338 from index $SII of file 9.
Deleting an index entry with Id 341 from index $SII of file 9.
Deleting an index entry with Id 343 from index $SII of file 9.
Deleting an index entry with Id 347 from index $SII of file 9.
Deleting an index entry with Id 349 from index $SII of file 9.
Deleting an index entry with Id 350 from index $SII of file 9.
Deleting an index entry with Id 356 from index $SII of file 9.
Deleting an index entry with Id 357 from index $SII of file 9.
Deleting an index entry with Id 358 from index $SII of file 9.
Deleting an index entry with Id 359 from index $SII of file 9.
Deleting an index entry with Id 360 from index $SII of file 9.
Deleting an index entry with Id 363 from index $SII of file 9.
Deleting an index entry with Id 366 from index $SII of file 9.
Deleting an index entry with Id 367 from index $SII of file 9.
Deleting an index entry with Id 368 from index $SII of file 9.
Deleting an index entry with Id 369 from index $SII of file 9.
Deleting an index entry with Id 370 from index $SII of file 9.
Deleting an index entry with Id 372 from index $SII of file 9.
Deleting an index entry with Id 375 from index $SII of file 9.
Deleting an index entry with Id 382 from index $SII of file 9.
Deleting an index entry with Id 392 from index $SII of file 9.
Deleting an index entry with Id 393 from index $SII of file 9.
Deleting an index entry with Id 394 from index $SII of file 9.
Deleting an index entry with Id 399 from index $SII of file 9.
Deleting an index entry with Id 400 from index $SII of file 9.
Deleting an index entry with Id 401 from index $SII of file 9.
Deleting an index entry with Id 409 from index $SDH of file 9.
Deleting an index entry with Id 407 from index $SDH of file 9.
Deleting an index entry with Id 410 from index $SDH of file 9.
Deleting an index entry with Id 406 from index $SDH of file 9.
Deleting an index entry with Id 289 from index $SDH of file 9.
Deleting an index entry with Id 408 from index $SDH of file 9.
Deleting an index entry with Id 405 from index $SDH of file 9.
77 percent complete. (0 of 19968 file SDs/SIDs processed)
Replacing invalid security id with default security id for file 148.
Replacing invalid security id with default security id for file 209.
Replacing invalid security id with default security id for file 263.
Replacing invalid security id with default security id for file 343.
Replacing invalid security id with default security id for file 344.
Replacing invalid security id with default security id for file 345.
Replacing invalid security id with default security id for file 346.
Replacing invalid security id with default security id for file 348.
Replacing invalid security id with default security id for file 350.
Replacing invalid security id with default security id for file 351.
78 percent complete. (352 of 19968 file SDs/SIDs processed)
Replacing invalid security id with default security id for file 352.
Replacing invalid security id with default security id for file 353.
Replacing invalid security id with default security id for file 365.
Replacing invalid security id with default security id for file 367.
Replacing invalid security id with default security id for file 368.
Replacing invalid security id with default security id for file 369.
Replacing invalid security id with default security id for file 370.
Replacing invalid security id with default security id for file 372.
Replacing invalid security id with default security id for file 373.
Replacing invalid security id with default security id for file 374.
Replacing invalid security id with default security id for file 375.
Replacing invalid security id with default security id for file 376.
Replacing invalid security id with default security id for file 377.
Replacing invalid security id with default security id for file 378.
Replacing invalid security id with default security id for file 380.
Replacing invalid security id with default security id for file 394.
Replacing invalid security id with default security id for file 399.
Replacing invalid security id with default security id for file 401.
Replacing invalid security id with default security id for file 402.
Replacing invalid security id with default security id for file 414.
Replacing invalid security id with default security id for file 458.
Replacing invalid security id with default security id for file 512.
Replacing invalid security id with default security id for file 939.
79 percent complete. (1249 of 19968 file SDs/SIDs processed)
Replacing invalid security id with default security id for file 1363.
Replacing invalid security id with default security id for file 1364.
Replacing invalid security id with default security id for file 1366.
Replacing invalid security id with default security id for file 1367.
Replacing invalid security id with default security id for file 1369.
Replacing invalid security id with default security id for file 1370.
Replacing invalid security id with default security id for file 1379.
Replacing invalid security id with default security id for file 1380.
Replacing invalid security id with default security id for file 1382.
Replacing invalid security id with default security id for file 1383.
Replacing invalid security id with default security id for file 1385.
Replacing invalid security id with default security id for file 1386.
Replacing invalid security id with default security id for file 1388.
Replacing invalid security id with default security id for file 1389.
Replacing invalid security id with default security id for file 1390.
Replacing invalid security id with default security id for file 1392.
Replacing invalid security id with default security id for file 1393.
Replacing invalid security id with default security id for file 1398.
Replacing invalid security id with default security id for file 1402.
Replacing invalid security id with default security id for file 1404.
Replacing invalid security id with default security id for file 1405.
80 percent complete. (2146 of 19968 file SDs/SIDs processed)
81 percent complete. (3043 of 19968 file SDs/SIDs processed)
82 percent complete. (3940 of 19968 file SDs/SIDs processed)
83 percent complete. (4838 of 19968 file SDs/SIDs processed)
84 percent complete. (5735 of 19968 file SDs/SIDs processed)
85 percent complete. (6632 of 19968 file SDs/SIDs processed)
86 percent complete. (7529 of 19968 file SDs/SIDs processed)
87 percent complete. (8426 of 19968 file SDs/SIDs processed)
88 percent complete. (9323 of 19968 file SDs/SIDs processed)
89 percent complete. (10220 of 19968 file SDs/SIDs processed)
90 percent complete. (11117 of 19968 file SDs/SIDs processed)
91 percent complete. (12014 of 19968 file SDs/SIDs processed)
92 percent complete. (12911 of 19968 file SDs/SIDs processed)
93 percent complete. (13808 of 19968 file SDs/SIDs processed)
94 percent complete. (14705 of 19968 file SDs/SIDs processed)
95 percent complete. (15602 of 19968 file SDs/SIDs processed)
96 percent complete. (16499 of 19968 file SDs/SIDs processed)
97 percent complete. (17396 of 19968 file SDs/SIDs processed)
98 percent complete. (18293 of 19968 file SDs/SIDs processed)
99 percent complete. (19190 of 19968 file SDs/SIDs processed)
19968 file SDs/SIDs processed.
Cleaning up 3 unused index entries from index $SII of file 9.
Cleaning up 3 unused index entries from index $SDH of file 9.
Cleaning up 3 unused security descriptors.
Security descriptor verification completed.
Inserting data attribute into file 344.
Inserting data attribute into file 410.
Inserting data attribute into file 462.
Inserting data attribute into file 498.
121 data files processed.
Correcting errors in the master file table's (MFT) BITMAP attribute.
CHKDSK discovered free space marked as allocated in the volume bitmap.
Windows has made corrections to the file system.
1610826751 KB total disk space.
195241284 KB in 10477 files.
13588 KB in 120 indexes.
0 KB in bad sectors.
135111 KB in use by the system.
65536 KB occupied by the log file.
1415436768 KB available on disk.
4096 bytes in each allocation unit.
402706687 total allocation units on disk.
353859192 allocation units available on disk.
iffi
-
Monday, March 12, 2012 7:40 AMModerator
Hi imughal,
I’m sorry that I cannot find information about this error from the log you posted above. It might be something wrong with the Windows cluster. Please pay attention to this similar thread: I get an error from Cluster IP Address won't come back online.
As suggested, it might be helpful to ask for on High Availability (Clustering) forum.
Stephanie Lv
TechNet Community Support
- Marked As Answer by Stephanie LvModerator Thursday, March 15, 2012 9:26 AM
.png)
