Code Coverage for Azure Services

Code Coverage for Azure Services

Overview:
This document describes how to perform code coverage of Azure services by running them on dev fabric. It first describes the issues faced and then the methodology using which code coverage was obtained for Azure services.
Issues faced:
Code coverage for Azure services could not be done within Visual Studio.
Un Instrumented Dlls

·         Deployed the Azure service on dev fabric

·         Copied the symbol files in the same folder where the dll exists at \Users\...\AppData\Local\dftmp\s0\deployment(N)\res\...\AspNetTemp\aspNetTemp\...

·         Opened the coverage tab in testrunconfig and added the dll from \Users\...\AppData\Local\dftmp\s0\deployment(N)\res\...\AspNetTemp\aspNetTemp\...

·         Got an error, Error VSP1024: Unable to open file \Users\...\AppData\Local\dftmp\s0\deployment(N)\res\...\AspNetTemp\aspNetTemp\...' for writing

Reason: This is because VS is trying to instrument a dll which is already in use.
Instrumented Dlls

·         Instrumented the dlls

·         Deployed the Azure service on dev fabric

·         Opened the coverage tab in testrunconfig and added the dll from \Users\...\AppData\Local\dftmp\s0\deployment(N)\res\...\AspNetTemp\aspNetTemp\...

·         Got an error, Error VSP1018: VSInstr does not support processing binaries that are already instrumented

Reason: This is because VS is trying to instrument an already instrumented dll.
Code Coverage Methodology
Since the Azure services for which we want to obtain the coverage are running on dev fabric which is a 64-bit process, we should use Visual Studio 64 bit tools. So either launch VS 64-bit command prompt or open command prompt and change directory to \Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\x64
Steps:

·         Open command prompt in privileged mode.

·         Change directory to \Program Files\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\x64

·         Instrument your dlls for the Azure service by running the command vsinstr /coverage “…\abc.dll”

·         Start coverage monitor by running the command vsperfcmd /start:coverage /output:”Test.coverage”

·         Deploy the instrumented Azure service on dev fabric.

o   Go to Programs->Windows Azure SDK v1.2->Windows Azure SDK Command Prompt 
o   Type csrun /devfabric:start
o   Type csrun /devstore:start
o   Type csrun /run:"...\abc.csx";"...\ServiceConfiguration.cscfg" 

·         Run your tests pointing the instrumented Azure service deployed on dev fabric.

·         Stop the coverage monitor by running the command vsperfcmd /shutdown

o   If you see the error Waiting for the process …\WaWebHost.exe to shutdown, shutdown the dev fabric by using the command /devfabric:shutdown

·         Open the file Test.coverage to find your code coverage results.

Note: The important step here is to start the coverage monitor before deploying the Azure service on dev fabric.
 
 
 
 
 
Sort by: Published Date | Most Recent | Most Useful
Comments
  • Very helpful article! I have a basic question here, when you say "run your tests pointing to instrumented Azure service deployed on dev fabric", does it mean that reference the instrumented dlls in the tests etc? In my case, I have a WCF service running as a webrole in dev fabric. I am POSTing data to my WCF service using its localhost address as POST url...it works fine, but I don't get any code coverage on this. Am I doing something wrong, should I reference its dll etc?

    Note: I think my setup is fine as I do get some basic code coverage numbers (on standard actions like run etc).

  • You don't need to reference the dll's. If you are not getting the code coverage, the profilemonitor might not be running of the dll's deployed on dev fabric might not be instrumented.

    Do you see an error that profile monitor does not appear to be running when you run the command "vsperfcmd /shutdown"

    You mentioned that you are getting basic code coverage on standart actions. So you should make sure that your tests are poiting to the right endpoint where the instrumented dll's are deployed. You can even try sending request to your dev fabric endpoint using fiddler, etc and see if you get code coverage numbers.

  • Yes, I am getting basic code coverage numbers on standard events (e.g. OnStart) etc. Also, I can successfully send request (HTTP POST) to my dev fabric endpoint using fiddler but I don't get any coverage on this. Under IIS, I see different port number than dev fabric for my deployed Azure service and I tried  sending request to that IIS endpoint as well, it works but still no coverage on my POST request. Do I need to start/stop IIS (w3svc or/and w3wp) as well to in addition to dev fabric to catch my HTTP POST request's coverage?

  • I think I have found the issue. I need to provide extra parameters "/cs /user:Everyone" with the vsperfcmd command. The "/cs" for cross-session profiling (on Vista and above, w3wp runs in a different session than the interactive session) and "/user:Everyone" to allow processes running as other users (e.g. NETWORK SERVICE) to connect to the monitor to send data  

  • Some troubleshooting points:

    If the 64 bit version of vsperfcmd is not used, you will see this error in the eventlog while deploying the instrumented build on Dev Fabric:

    Event 1425, VSPERF - Unable to connect to monitor.  Make sure the monitor is started and in the correct mode.

    If the vsperfcmd coverage monitor is not running, you will see this error in the eventlog while deploying the instrumented build on Dev Fabric:

    Event 1306, VSPERF - Profile Monitor does not appear to be running.

Page 1 of 1 (5 items)