none
Offline Servers Slow Launching Apps

    Question

  • So I have a group of virtual servers housed on 3 VMware ESXi servers which was in an open data center during build but has since been moved to a secured environment which has no internet access.  Since the servers have been moved to this new environment I have noticed they have been very sluggish.

    Performance on the servers in minimal using under 5% CPU and around 12% memory.

    Application perform without problem once launched, however the initial launch process takes a very long time to complete.  Even an RDP to the individual server from within the environment takes too long to fully initiate but once the connection is established it whips through, gets stucj on Securing remote connection.

    It almost seems like there is a delay in any type of authentication.  Checking to see if user has access to launch app, logging on, executing administrative powershell.  All of this takes way to long to launch but once launched runs perfect.

    Site has 2 domain controllers on same subnet and even same physical hardware.  Domain appears to be functioning properly and I can find nothing is the logs to suggest an issue.

    If anyone has any suggestion to try please let me know.  Soon this will be a production environment and this issue has to go.


    Thursday, June 14, 2018 3:19 PM

All replies

  • It can be caused by some queries on DNS. Because AD servers asking root level server or some pre-configured upper level (provider) DNS server for queries which cannot resolve.

    And DNS usually may cause similar problems. (you are talking that servers are out of internet)

    So what about to allow just DNS queries and nothing else?

    Microsoft is very often asking for tons of telemetry servers and much more stuff like this.

    And second what about storage? Try to take a look on queue, latency, throughput etc. But DNS is my bet.

    Thursday, June 14, 2018 4:39 PM
  • The SAN array is was under used at this point as well as network traffic on the 10gb connections.  I would have to jump towards DNS however these systems are contained within a secure environment and will not have any external communication or ability to query DNS for those domains, they control the firewall.

    Even if they were just able to query the domain names they would still attempt to connect would that not cause the same delay I am seeing?

    Thursday, June 14, 2018 5:36 PM
  • may be wireshark or similar will be helpful to inspect what exactly apps are doing

    from your description it seems that they waiting for something on network (source of DNS idea)

    and also try this one: https://docs.microsoft.com/en-us/sysinternals/downloads/procmon

    this can show when the app is thinking and waiting, very helpful utility

    SAN was mentioned only as another idea (I don´t expect problem here because these problems are usually reported in some way)

    Thursday, June 14, 2018 9:04 PM
  • how many vm's on each of the 3 HOSTS? how many users does this air-gapped network support? how are 'applications' delivered to the user? 

    you are basically handing us an box and asking us to guess why it's hissing without opening it.

    Thursday, June 14, 2018 9:25 PM
  • I will try a Wireshark when I am able to get the software transferred and will try to get procmon at the same time.

    Each host has only 3 - 4 server a piece.  We had no issues until the hosts were moved to the secure data center so it should not be a hardware/capacity issue.

    I agree with you Michal they seem to be trying to communicate with something either for approval of the request or confirmation the connection is allowed.  Both of which they should have no problem.  They have all communication with the DCs and local firewalls allow the connections, no hardware firewalls in place internal.

    Friday, June 15, 2018 2:23 PM