My guess is that you are experiencing clock drift. Try syncing the server's clock to the encoder's clock.
WMS must deliver packets based on the presentation time that is stamped on each video packet. If the system clock that WMS is running on begins to drift then very gradually the times that it sends packets to the client will cause a bit of a temporal pile-up
Microsoft is conducting an online survey to understand your opinion of the Technet Web site. If you choose to participate, the online survey will be presented to you when you leave the Technet Web site.