Answered by:
Lync 2013 Edge replication is not working after migration from 2010 !

Question
-
Edge stopped replicating with FE since 29 December out of no where! I can't see anything on the Edge side but on Front end in control panel it shows in topology that it's not replicating.
on even viewer I can see the following event
File transfer failed for some replica machines. Microsoft Lync Server 2013, File Transfer Agent will continuously attempt to replicate to these machines.
While this condition persists, configuration changes will not be delivered to these replica machines.
Replica file transfer failures:
edge.mydomain.com: Https request failed.
Cause: Possible issues with transferring files to the replicas listed above.
Resolution:
Check the accessibility of file shares or https web services listed above.any suggestions?
Thanks
Mohammed JH
Tuesday, January 8, 2013 12:20 AM
Answers
-
I have finally fixed it, this week i had a migration project and Edge also failed and the only way to fix it was registry trick provided by Microsoft support.
Here is the fix below
Mohammed JH
- Marked as answer by moh10ly Friday, December 20, 2013 2:53 PM
Friday, December 20, 2013 2:53 PM
All replies
-
Can you run a telnet test on port 4443 from the FE server to the Edge server? Are all Edge Lync services started?
Tim Harrington | Lync: MCM/MVP | Blog: http://HowDoUC.blogspot.com | Twitter: @twharrington
Tuesday, January 8, 2013 1:11 AM -
Hi Tim,
Yes all services are started and 4443 is open. When I run the command Get-CsManagementStoreReplicationStatus
I get the following
UpToDate : False
ReplicaFqdn : edge.domain.com
LastStatusReport : 12/29/2012 9:58:36 AM
LastUpdateCreation : 1/8/2013 8:06:20 AM
ProductVersion : 5.0.8308.0I can ping to both edge and Lync front end. how do I troubleshoot this furthermore?
thanks
Mohammed JH
Tuesday, January 8, 2013 7:14 AM -
Tuesday, January 8, 2013 8:24 AM
-
Hi Oguzhan,
I have tried adding a route and i'm able to ping both Edge NIC IPs from Lync front. and Invoke command doesn't show any reference to port 4443 on my Reverse Proxy firewall. nor the Edge firewall.
All Lync services on Edge and FE are up.
I'm suspecting it's the xds shared folder. I'm following Scenario 3 in this link but I can't replace the user with local administrator as it gives me access denied.
The user i'm using for Lync FE is a domain admin, but xds showing no current owner on it.
The only thing I can remember that I have done between 28 and today is reinstalling the Edge server as we have some issues with some federated partners and most of the errors are indicating TLS issues but I have regenerated certificates couple of times and installed them but to no avail.
Mohammed JH
Tuesday, January 8, 2013 8:46 AM -
Sounds like you have some fundamental deployment issues. You should not be able to ping both interfaces of your edge server from internal. Your Edge server should be deployed on two different subnets (multi-homed) with strict firewall and routing enforced. Also, Lync replication does not have anything to do with the Reverse proxy, it is strictly between the FE and the internal interface of your Edge server. Also, you should not have to adjust the share permissions on the folder.
Read this article and let me know if this doesn't clear it up: http://blog.schertz.name/2012/07/lync-edge-server-best-practices/
Tim Harrington | Lync: MCM/MVP | Blog: http://HowDoUC.blogspot.com | Twitter: @twharrington
Wednesday, January 9, 2013 3:06 AM -
The file RtcReplicaRoot is not shared.
The CMS master uses a directory structure shared with other Lync Server components in network share defined in the topology document. The name of the top level directory is the Service Id of the CMS master. The tree structure is the following: <Lync Server FileStore>\<CMS Service Id>\CMSFileStore\xds-master, where <Lync Server FileStore> is the name of the directory selected to be the FileStore used by the CentralMgmt service in the topology document and CMS Service Id is the Service Id of the CentralMgmt service in the topology document .
You should check the Lync share file’s permission.
For the replication for CMS, check the following blog.
http://blogs.technet.com/b/jenstr/archive/2010/10/13/what-is-central-management-store-cms.aspx
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Wednesday, January 9, 2013 6:25 AM -
Hi Tim,
Yes, it is not pingable, but first when I deployed Lync I have created a static route as I have seen some RTP traffic sent from Edge's DMZ NIC to Front End's NIC. this traffic was being denied by the firewall. so I allowed it and create a static route to enable ping between both interfaces. but later on I removed it and now i'm unable to ping.
Btw, before the migration in Lync 2010 the replication status was ok, Even after the migration it still was ok but since I have federation issue, I decided to re-install/deploy the edge. So I removed it from the Front end and uninstalled Edge app from the edge server, restarted it and redeployed Lync.
Unfortunately my federation issue wasn't related to Edge at all. it was firewall related. so it has now been resolved but I have this replication issue.
Mohammed JH
- Edited by moh10ly Wednesday, January 9, 2013 1:31 PM
Wednesday, January 9, 2013 12:36 PM -
Lisa,
Not sure if you're talking about the RtcReplicaRoot folder or file, but there's a folder called RtcReplicaRoot in the root C drive and yes it is not shared. do I have to share it in order for replication to work ?
Inside the RtcReplicaRoot there's another folder called "xds-replica" now this one is shared with two permissions "network service" and "RTC local Config Replicator" with full permission.
In the security tab, it says Object name: C:\RtcReplicaRoot\xds-replica
You must have read permission to view the properties of this object. click advanced to continue
Clicking advanced will take me to the snapshot that I previously shared with you. on top if I choose to change the owner I get a windows security message which attached below
Clicking Yes will give me an error saying "An error occured while applying security information to: c:\rtcreplicaroot\cds-replica
failed to enumerate objects in the container. Access is denied.
Mohammed JH
Wednesday, January 9, 2013 12:46 PM -
Hi again Lisa and Tim,
Thanks for article, it seems that the issue could be related to more than one thing in my case .. as the error I have posted shows that there's an issue with HTTPS request from the Edge server. According to the article you sent the Edge server runs the https request to copy the replica files from the FE and sync the changes. but in my case it's failing.
My Lync Front End Server is sitting behind a TMG Firewall, and I have seen some 4443 requests benign denied from TMG's IP internal IP to FE's IP. I think Edge is trying to send the request to Lync FE but TMG is refusing to transmit the request even though there's a rule created to allow traffic.
I have created another rule that enables connection on port 4443 between all internal servers and localhost "TMG" but haven't noticed any change yet.
how do I push replication to Edge?
--
"The MASTER generates the data package containing new changes to CMS and stores a copy in each to-replicadirectory for every replica.
The data package must be copied to all replicas. All Lync Server 2010 servers, except the Edge Server, uses the Windows file copy SMB[4] protocol mechanism to push the data package from the CMS master to the replica.
For Edge servers the file copy is performed over an HTTPS channel. The Edge Server runs a Web Service (https://<edgeserverfqdn>:Port/ReplicationWebService) on the port specified for the ManagementServices in the topology document. The default port is 4443. The Web Service, ReplicationWebService, does not require IIS to be installed on the Edge Server. It is implemented as part of the REPLICA service. The certificate used for the HTTPS channel is the internal or default certificate created for the Edge Server"
--
Update:
On Edge server I can see two errors related to replication service. first one is 3010:
Failed to connect to back-end database. Microsoft Lync Server 2013, Replica Replicator Agent will continuously attempt to reconnect to the back-end. While this condition persists, no replication will be done.
The Connection string :
Data Source = (local)\rtclocal;
Database = xds;
Max Pool Size = 5;
Connection Timeout = 60;
Connection Reset = false;
Enlist = false;
Integrated Security = true;
Pooling = true;
Cause: Possible issues with back-end database.
Resolution:
Ensure the back-end is functioning correctly.Second error is :3033
Failed to register with back-end database. Microsoft Lync Server 2013, Replica Replicator Agent will continuously attempt to reconnect to the back-end. While this condition persists, no replication will be done.
The Connection string: Data Source = (local)\rtclocal;
Database = xds;
Max Pool Size = 5;
Connection Timeout = 60;
Connection Reset = false;
Enlist = false;
Integrated Security = true;
Pooling = true;
Exception: [2000] Could not connect to SQL server : [Exception=System.Data.SqlClient.SqlException (0x80131904): A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: SSL Provider, error: 0 - The wait operation timed out.) ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.Rtc.Common.Data.DBCore.PerformSprocContextExecution(SprocContext sprocContext)
ClientConnectionId:88d37252-568a-4944-8173-b16649ca20ab]
Cause: Possible issues with back-end database.
Resolution:
Ensure the back-end is functioning correctly.
Mohammed JH
- Edited by moh10ly Wednesday, January 9, 2013 2:09 PM
Wednesday, January 9, 2013 1:24 PM -
Lisa,
Not sure if you're talking about the RtcReplicaRoot folder or file, but there's a folder called RtcReplicaRoot in the root C drive and yes it is not shared. do I have to share it in order for replication to work ?
Inside the RtcReplicaRoot there's another folder called "xds-replica" now this one is shared with two permissions "network service" and "RTC local Config Replicator" with full permission.
In the security tab, it says Object name: C:\RtcReplicaRoot\xds-replica
You must have read permission to view the properties of this object. click advanced to continue
Clicking advanced will take me to the snapshot that I previously shared with you. on top if I choose to change the owner I get a windows security message which attached below
Clicking Yes will give me an error saying "An error occured while applying security information to: c:\rtcreplicaroot\cds-replica
failed to enumerate objects in the container. Access is denied.
Mohammed JH
That is the right behavior.
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Monday, January 14, 2013 1:16 PM -
You Lync edge server fails to replicate as the HTTPS fails. I think you should check the port for HTTPs is not blocked.
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Monday, January 14, 2013 1:24 PM -
You Lync edge server fails to replicate as the HTTPS fails. I think you should check the port for HTTPs is not blocked.
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Monday, January 14, 2013 1:24 PM -
Hi Lisa, I have checked the port couple times and its not blocked.. I can telnet from any machine to the Edge internal IP.
Mohammed JH
Monday, January 14, 2013 5:51 PM -
I was able to telnet to the IP addresses of the Lync 2013 edge servers from the Front End Servers, but was unable to do so using fqdn.
Added the host entries of the edge servers in the internal DNS.
Invoked the replication and then verified the replciation.
Worked successfully.
Friday, November 22, 2013 7:01 AM -
I have finally fixed it, this week i had a migration project and Edge also failed and the only way to fix it was registry trick provided by Microsoft support.
Here is the fix below
Mohammed JH
- Marked as answer by moh10ly Friday, December 20, 2013 2:53 PM
Friday, December 20, 2013 2:53 PM