Overview

Microsoft Surface Simulator is a Microsoft Windows application. You can use Surface Simulator together with Microsoft Visual C# 2008 Express Edition (or Microsoft Visual Studio 2008) to test Microsoft Surface applications on a separate workstation. Surface Simulator runs the applications and enables you to use a mouse or mice to simulate finger, blob, and tagged-object contacts, like the contacts that a user uses on a Microsoft Surface unit.

ImportantImportant
Surface Simulator does not support the use of raw image input. If your application requires the use of raw image APIs like TryGetRawImage and UpdateRawImage, you must test it on an actual Microsoft Surface unit.

Surface Simulator replicates the user interface and behavior of a Microsoft Surface unit that is in user mode. Surface Simulator has access points, Launcher, and the loading screen. When you start an application in Surface Simulator, the application displays like it is on a Microsoft Surface unit.

You can use Surface Simulator to evaluate how an application and its user interface respond to basic input. For example, if you simulate a painting application and if you touch multiple colors, one at a time, and then add the colors to a mixing bucket, you can test the logic of the application and how well it mixes the colors by using the touch-based interface.

Surface Simulator runs with the appearance and functionality of a Microsoft Surface unit in user mode (the way that it appears to users). You can switch applications by using Launcher and the access points that display on the Launcher screen and the applications.

Surface Shell is always turned on when you use Surface Simulator, so you see the attract application and Launcher (if you have touched an access point on the attract application to start Launcher).

Starting Surface Simulator

Microsoft Surface Simulator uses a 1024 × 768 simulation region to display Microsoft Surface applications. In this simulation region, Surface Simulator interprets mouse inputs as types of contacts (fingers, blobs, gestures, or tagged objects).

noteNote
You cannot use Surface Simulator in a Remote Desktop or virtual environment.

You do not have to have Microsoft Visual C# 2008 Express Edition (or Microsoft Visual Studio 2008) running when you start Surface Simulator. However, if Visual C# 2008 (or Visual Studio 2008) is running when you start Surface Simulator, make sure that you start Surface Simulator before you start a Microsoft Surface application from Visual C# 2008 (or Visual Studio 2008).

You must run Surface Simulator at the same privilege level as Visual C# 2008 (or Visual Studio 2008). If you start Visual C# 2008 (or Visual Studio 2008) as Administrator, you must also start Surface Simulator as Administrator. If you start Visual C# 2008 (or Visual Studio 2008) as a User, you must also start Surface Simulator as a User.

To start Surface Simulator, click Start, click All Programs, click Microsoft Surface SDK 1.0 SP1, click Tools, and then click Surface Simulator.

You can start a Microsoft Surface application from one of the following locations:

  • From Launcher in Surface Simulator (as a user accesses it on a Microsoft Surface unit). To start the application from Launcher, the application must be registered with the unit.

  • From outside Surface Simulator:

    • From Visual C# 2008 (or Visual Studio 2008), by pressing F5. This method is useful if you are debugging the application.

    • From the application executable file in Windows Explorer.

    • From the application executable file at a command prompt.
ImportantImportant
  • If you start the application outside of Surface Simulator while Launcher is open, Launcher displays on top of the application. You must click an access point to open the application.
  • If you start more than one application from outside Surface Simulator, you must use the Windows taskbar to switch between the applications.

Surface Simulator User Interface

This section describes the types of Microsoft Surface contacts you can use in Surface Simulator and describes Surface Simulator's interface.

Surface Simulator enables you to use a mouse to simulate three types of Microsoft Surface input:

  • Finger. The contact of a finger.

  • Blob. The contact of an object.

  • Tagged object. The contact of an object that is marked with a byte tag or identity tag.

Parts of Surface Simulator

The following illustrations show the Surface Simulator interface with an attract application.

Surface Simulator - Tools tab labeled

 

Surface Simulator - Record tab labeled

Surface Simulator interface includes the following pieces. (The numbers in the previous illustrations correspond to the numbers in the following list.)

  1. The Contact Selector button enables you to select and manipulate one or more contacts. Manipulations include move, resize, stretch, and rotate.

    Surface Simulator - Contact Selector
  2. The Finger button simulates a finger that contacts the application and triggers finger contact events.

    Surface Simulator - Finger button
  3. The Blob button simulates an arbitrary ellipse shape that contacts the application and triggers blob contact events. You can resize the Blob button to simulate shapes of different sizes.

    Surface Simulator - Blob button
  4. The Tag button simulates a tagged object that contacts the application and that uses user-specified hexadecimal values. This button triggers tagged-object contact events according to the specified hexadecimal values.

    Surface Simulator - Identity Tag Series and Identi
  5. The Tag Value data entry control enables you to enter a hexadecimal value (00 through FF) to assign to tagged-object contacts that you create by using the Tag button.

    Surface Simulator - Tag Value box
  6. The Identity Tag button enables you to simulate an identity tag contact that is placed on a Microsoft Surface screen. Enter identity tag series and value properties in the adjacent Series and Value data entry controls.

    Surface Simulator - Identity Tag button
  7. The Identity Tag Series and Identity Tag Value controls enable you to assign hexadecimal values (0000000000000000 through 7FFFFFFFFFFFFFFF) to the series and value of an identity tag contact that you create by using the Identity Tag button.

    4daa8aac-6c0a-4e98-8685-09fe9d66d9ef
  8. The Remove All Contacts button removes all contacts from the simulation region.

    Remove All Contacts button
  9. The Hide/Show Contacts button determines whether the current contacts on the simulation region are visible. You can click the button to switch from showing the contacts to hiding the contacts. You can still put new contacts on the active area of the simulated screen when existing contacts are invisible.

    Surface Simulator - Hide-Show Contacts button
  10. The simulation region is the area of the Surface Simulator user interface where the application appears. In this area, you can test the application by using finger, blob, and tagged object contacts. When you first start Surface Simulator, the active area displays the default attract application.

  11. The Surface Simulator access points that appear in each corner of the active area are Microsoft Surface controls that enable you to move between applications and Launcher. These access points and Launcher work the same way that they do on Microsoft Surface units.

    71d06639-1e0c-43ee-8efb-a3fb10392f46
  12. The recording panel enables you to record interactions and then play them for testing purposes. For more information about how to record and play interactions, see Recording and Playing Interactions below.

    Surface Simulator - Recording Panel interface
  13. The Minimize and Exit buttons minimize or close Surface Simulator and all Surface applications, respectively.

    Surface Simulator - Minimize and Exit buttons
  14. The Help button opens the Surface Simulator section of the Surface SDK documentation.

    Surface Simulator - Help button
  15. The Surface Simulator window is a movable window that encloses the Surface Simulator user interface and the application that you are testing. The window size is fixed to 1224 × 868, but you can minimize and restore the window.

How to Use Surface Simulator

This section describes how to work with Microsoft Surface contacts in Surface Simulator.

Adding Contacts to Surface Simulator

In the upper-left corner of the Surface Simulator window, you can use the buttons to simulate the different types of Microsoft Surface input (finger, blob, and tagged object). A contact is an instance of an input type that is interacting with the simulated region.

To simulate contact input, do the following:

  1. Select a contact tool by clicking one of the buttons on the toolbar in the upper-left area of the Surface Simulator window (for example, the Finger button).

  2. Move the cursor over the simulation region. The cursor displays the pointer that corresponds to the tool that you selected.

  3. Press the left mouse button to add the contact and then release the mouse button to remove the contact. Or, to put a contact on the screen and leave it there, press-and-hold the left mouse button and then right-click.

You can also click the Contact Selector button and then select a contact on the simulated Microsoft Surface screen and drag it across the screen.

To simulate more than one contact, you can connect additional mice to your computer's USB ports.

noteNote
For additional USB ports, you can connect a USB hub to the computer.

 

Contact States

A Surface Simulator contact can appear in one of the following states:

  • White with a red shadow indicates that a contact is selected for manipulation (but is still on the simulated Microsoft Surface screen).

  • Red indicates that the contact is placed on the screen.

  • White indicates that the contact is hovering and is not yet placed on the screen.

549ad58b-d58d-436b-ac23-ecb2bca2f07b

Using Surface Simulator Controls and Contacts

This section describes how to use Surface Simulator controls and contacts.

Contact Selector

Click the Contact Selector button (or press CTRL+S) to choose the Contact Selector tool. You can then select contacts that are already placed in the simulation region by doing one of the following:

  • Place the cursor directly over an already placed contact and then click it.

  • Drag a selection region around one or more contacts that have already been placed.

You can remove selected contacts by using the DELETE key.

For examples of multitouch gestures that you can simulate by using Contact Selector, see Common Microsoft Surface Actions in Surface Simulator.

Surface Simulator - Contact Selector

Finger Contact

Click the Finger button (or press CTRL+F) and then click in the active area to simulate a finger contact in the application. You can also click the Finger button and then drag it in the active area to simulate a dragging gesture across the application.

Press-and-hold the left mouse button and then right-click to place a finger contact, which leaves a finger pressed in that location of the application. You might use this procedure to press multiple controls at the same time (for example, buttons or pieces of content), to press-and-hold a control, or to anchor part of an interface. For example, you might use this procedure to hold a corner of a photo and then use the mouse to rotate the photo by interacting with the other corner.

Surface Simulator - Finger button

Blob Contact

Click the Blob button (or press CTRL+B) to simulate an arbitrary ellipse shape. This button works the same way as the Finger button. You might want to use blob input when users are pressing different parts of their hands on the surface. For example, if a user leaves a handprint on the screen, the application might need to receive blob input.

Surface Simulator - Blob button

Tagged-Object Contact (Byte Tag)

To simulate a byte tag

  1. Click the Tag button (or press CTRL+T).

  2. Click Tag Value, and enter the hexadecimal value you want.

  3. Click the active area to simulate an object that is touching the screen and lifting up.

  4. Press-and-hold in the active area, and then right-click to place the tagged object.

You can leave unlimited tagged-object contacts on the application. The Shopping Cart sample application demonstrates the code to use tagged-object contacts.

4daa8aac-6c0a-4e98-8685-09fe9d66d9ef

Tag Value

Click the Tag Value box and enter a valid hexadecimal value ("00" through "FF") for a tagged object.

noteNote
You can select the tagged-object contact before or after you enter your hexadecimal value.

 

Surface Simulator - Tag Value box

ImportantImportant
The updated hexadecimal value is applied to:
  • The cursor that clicked the Tag Value box before changing it.
  • All cursors that subsequently select the Tag button.

 

Tagged-Object Contact (Identity Tag)

To simulate an identity tag

  1. Click the Identity Tag button (or press CTRL+I).

  2. Click the Series box, and enter the hexadecimal series value that you want ("0000000000000000" through "7FFFFFFFFFFFFFFF").

  3. Click the Value box, and enter the hexadecimal value that you want ("0000000000000000" through "7FFFFFFFFFFFFFFF").

  4. Click the active area to simulate an object that is touching the screen and lifting up.

  5. Press-and-hold in the active area, and then right-click to place the tagged object.


58356fbe-2309-4d5f-97e9-5ef0f3e6efec

Identity Tag Series and Identity Tag Value

Click the Identity Tag Series or Identity Tag Value boxes, and enter a valid hexadecimal value ("0000000000000000" through "7FFFFFFFFFFFFFFF") for a tagged object.

noteNote
You can select the tagged-object contact before or after you enter your hexadecimal series and value.

 

Surface Simulator - Identity Tag Series and Identi

ImportantImportant
The updated series and value are applied to:
  • The cursor that clicked Identity Tag Series or Identity Tag Value before changing it.
  • All cursors that subsequently select the Identity Tag button.

 

Hide/Show and Remove Contacts

Click the Remove All Contacts button to remove all contacts that you have placed in the active area of the Surface Simulator window. You can also press the Delete key to remove all currently selected contacts. Click the Hide/Show Contacts button to hide all your placed contacts or to show all the contacts that you have hidden. These features enable you to clear any extra contacts from the user interface.

Surface Simulator - Hide-Show and Remove Contacts

Common Microsoft Surface Actions in Surface Simulator

The following table shows the corresponding mouse actions in Surface Simulator for common actions on a Microsoft Surface unit. Some of these actions are also listed in the preceding sections.

Surface action Mouse action

Select a contact type (finger, blob, or tagged object).

Click the appropriate icon in the upper-left corner of the Surface Simulator window.

Simulate a tagged object by using a byte tag with a specific value.

  1. Click Tag Value, and enter the appropriate hexadecimal value.
  2. Click the Tag button.
  3. Click in the simulation region.

Simulate a tagged object by using an identity tag with a specific series and value.

  1. Click the Identity Tag Series box, and enter the appropriate hexadecimal value.
  2. Click the Identity Tag Value box, and enter the appropriate hexadecimal value.
  3. Click the Identity Tag button.
  4. Click in the simulation region.

Modify the size or orientation of a blob contact before placing it.

  1. Click the Blob button.
  2. With the cursor over the simulation region, right-click to display manipulation handles and enter the tool edit mode.
  3. Drag the handles to resize or rotate the blob.
  4. Click anywhere to remove the manipulation handles and leave tool edit mode.
  5. Clicking the simulation region to add a blob contact with the modified properties.

Modify the size or orientation of a placed blob contact.

  1. With the Contact Selector tool, click a placed blob, or with another tool right-click a placed blob. Manipulation handles are displayed for the blob.
  2. Drag the handles to resize, rotate, or move the blob.

Touch the screen with finger.

Click the Finger button, and then click the active area.

Press a finger on the screen and hold indefinitely (place a finger contact).

Press-and-hold the Finger contact, and then right-click.

Drag a finger on the screen.

Press-and-hold, and then drag the Finger contact.

Drag two fingers on the screen.

Do one of the following:

  • Connect two mice, click each contact, and drag them both.
  • Place two contacts, select them by using the Contact Selector button, and drag the selection.

Lift a finger off the screen.

Move the mouse over a placed contact until it becomes active, press-and-hold the contact, and then right-click it.

Rotate a finger.

Select a contact, and then rotate the mouse wheel up or down.

Touch the screen with a blob.

Click the Blob button, and then click the active area.

Touch the screen with a tagged object.

Click the Tagged Object button, and then click the active area.

Lift a tagged object off the screen.

  1. Move the cursor over a placed tagged object.
  2. Press the left mouse button.
  3. Press the right mouse button before you release the left mouse button.

Drag two fingers across the screen.

Place the first finger on the simulated screen, place the second finger on the simulated screen, use the Contact Selector button to select both finger contacts, and then drag the contacts by using the symbol in the center of the selection area.

Surface Simulator - Drag two fingers apart

Rotate two fingers around a central point.

Place the first finger on the simulator screen, place the second finger on the simulated screen, use the Contact Selector button to select both finger contacts, and then rotate the contacts by using the double-arrow pointer to the right of the selection area.

Surface Simulator - Rotate two fingers example

Drag two fingers apart (also known as a zoom gesture).

Place the first finger on the simulated screen, place the second finger on the simulated screen, use the Contact Selector to select both finger contacts, and then drag the contacts apart by clicking and holding one corner on the selection area and dragging it.

Gestures - Drag two fingers apart

Remove all contacts.

Click the Remove All Contacts button in the upper-right corner of the active area. You can also press the DELETE key to remove all currently selected contacts.

Show or hide contacts.

Click the Hide/Show Contacts button in the upper-right corner of the active area.


Recording and Playing Interactions

Surface Simulator supports recording and playback. You can record a sequence of Microsoft Surface interactions for rapid, iterative testing. These recordings are saved as scripts that you can then play later.

ImportantImportant
Do not open the script file for a recording and alter it.

 

Recording Panel

When you want to record interactions on a computer, you can use the recording panel that appears at the bottom of Surface Simulator window.

Surface Simulator - Recording Panel interface

You can also record interactions on a Microsoft Surface unit by using the recording panel that appears at the top of the Microsoft Surface screen on top of an application. To record and play interactions on a unit, log on the unit as an administrator and start Surface Simulator.

Surace Simulator - Recording panel on Data Visuali

When you use the recording panel on a Microsoft Surface unit, you can use the following additional features:

  • When your mouse is not moving, the recording panel disappears so that it does not obstruct your recording.
    noteNote
    Move the mouse to make the recording panel reappear.

     

  • You can click the Close button (Surface Simulator - Close button) to close the recording panel.

    noteNote
    The recording panel does not respond to touch input. You must use the mouse instead.

     

Recording Interactions

To record interactions in Surface Simulator

  1. Click the Record button (Surface Simulator - Record button) to start recording.

    noteNote
    If you record a script when another script is already loaded, Surface Simulator starts a new recording. Surface Simulator does not erase or overwrite the previously loaded script.

     

  2. Complete gestures that you want to test by using the Surface Simulator controls.

  3. Click the Pause button (Surface Simulator - Pause button) to pause the recording, interact with the application, or change the interface so that the script moves to a new task when you play it.

  4. Click the Stop button (Surface Simulator - Stop button) to name and save your recorded script.

 

Playing Interactions

To play interactions in Surface Simulator

  1. Click the Open button (Surface Simulator - Open button), and then select a previously recorded script.

  2. Click the Play button (Surface Simulator - Play button) to start playing the script.

    ImportantImportant
    • Do not use any Surface Simulator controls (such, as Finger, Blob, Tag, or Contact Selector) when you are playing a recorded script. These controls are disabled.
    • Do not play a script if you have altered the interface.
    • Play a script only on top of the application that the script was recorded with.

     

  3. Click the Pause button (Surface Simulator - Pause button) to pause the playback or to physically interact with the application.

  4. Click the Stop button (Surface Simulator - Stop button) to stop recording. You must stop the script before you can load a new script.

    noteNote
    To play interactions by using the Surface Simulator automation API, see SimulatedContactScript.

     

When to Record Interactions

Surface Simulator's recording feature is useful for the following areas:

  • Repeating test cases. You can record interactions to automate test cases.

  • Tracking contacts. You can track contacts more easily by using Surface Simulator than by using a Microsoft Surface unit. However, you can create more realistic testing on a computer by using scripts that you recorded on the unit.

  • Reproducing bugs. You can record actions that show the process of uncovering a bug. You can then easily send the script file to other developers to review.

The following procedure demonstrates an example of how you can use the recording and play features to create a test case that can be repeated:

  1. Click the Record button (Surface Simulator - Record button) to start recording.

  2. Interact with the application by using Surface Simulator controls:

    1. Place three blob contacts on three movable pieces of content, such as photos.

    2. Use the Contact Selector button to select all three placed contacts.

    3. Drag any of the selection manipulation handles to transform the positions of the selected contacts.
  3. Click the Stop button (Surface Simulator - Stop button) to finish and save the recording.

  4. Make any changes to your code and reopen your application in Surface Simulator.

  5. Click the Open button (Surface Simulator - Open button), and then select the recorded script.

  6. Click the Play button (Surface Simulator - Play button) to run the same test case.