Since the release to market of the first Microsoft Surface device (Aka Surface V1) and the today Samsung SUR40 with Microsoft PixelSense technology, I have been working on building different types of application for such devices. By looking a bit behind (even if I am not used to do that in general) I have explore and validate different form of approach and rules on such technology , in order to bring to ending customers what they were really expecting to get.

Those rules and guide lines that I have experiment during all those past years, this is something that I will describe here. It has perfectly work in all my projects and could be apply or adjust in order to cover why not yours.

Understanding customer objectives

One of the most important rule in my mind is to really understand main objectives of your customers, on having a dedicated PixelSense application. At first it might sounds an evidence but when you look deeper it is the key point entry. This understanding will indicate the type of application that you are going to build and set up. Depending on customers objectives the application interaction and design goal will be totally different and you will be able to setup a compromise between the “Wow” effect and application performance.

Always remember basic questions before anything else

  • What is the purpose of integrating the device in a selected location?
  • Does the location is suitable to receive the device? ( lighting condition, traffic,..)
  • Who will be your application users?
  • What type of application will be more suitable
  • How the environment will be ? ( colors, style, branding,..)

Learn from those who have tried before

Do not jump too quickly into programming your design, but instead raise your head and takes some time to explore what is already existing. To help developers, the Surface team has written an important set of documents to puts you in the right track. The meeting point is the Microsoft PixelSense Design and Development. The Surface SDK 2.0 is the set of tools and sample that will help you building an application but keep in mind that any good programmer can use and understand an SDK but designing a great application is another story and this is the point where you will spend most of your time in a project.

Forget what you have previously learned on designing traditional application

For some, it is difficult to jump to a new design approach and they always refer to what they have done before and wish to get the same result. You cannot imagine how many times I had to explain to customers or design agency (most of them coming from the web) that the design approach of an application in PixelSense is totally different. This is something where you need to explain again and again until the point you manage to convince them and established a full trust relationship, relying on the experience you provide.

“It takes time to build a great experience, so start with simple thing that you can validate easily”

The Wow factor against application performance

Who have never start to think of his application by already imagine the type of different animation that could be done in order to bring to the application the famous Wow effect ?

Let me stop a while on this simple question: Does “Wow effect” really means animation?

Absolutely not, and this is one of an important point that comes to me when designing an application.

“What my customer is really expecting when he was telling me “Make it wow”

In all of my first PixelSense project, I was permanently thinking of that 3 words and with the time, they disappears from my though for the simple reason that this famous wow factor will come naturally by bringing to live customers mains objectives.

The success of an application will be identify when your customer will start to use your application and will naturally pronounced one word: “Wow”.

Points that make me talk about this are essentially based on followings:

  • Too much animation will kill your application
  • Reserved animation for content transition and make it fast enough for rich experience. For example an opacity rendering should not go over 0.5 s.
  • Do not overload your application for unnecessary things that brings no added value for the general concept
  • Always gives priority to application performance

Avoid using “Forever” storyboard on TagVisualizer

This is one of the rule I recently try to explain to the final customer on one of my last project. As every potential customers, they took major of their ideas from different videos they could catch on the web and explore different application type that can exist, until one of them could be apply to their own use.

So far nothing special, instead a normal behavior when you are new to this world of interaction. The main part of the application was to interact fully with different objects and bring respective content. Nothing special either in that sense except one single request that makes me start to doubt a bit: Each object visual was requested to be animated during the whole presentation of content.

We start to build the requested animation which was based on quite many different layer of successive animated object in different time line. To reach the customer demand we had to set up no more than 8 different storyboard to reach the goal. So far so good we get the animation in place and it behaves as customer expected to get it:

  • The object his place on the table
  • Different circle object appears around the object and rotate asynchronously
  • So particles moves around illustrating user activities during the whole time object was on the table

That was working quite nicely until the time we place more similar object on the table. At that time we could easily notify the weak response of the application during content manipulation. My first doubt start to come true and I could verify this by exploring the CPU usage which reach over 85%.

After some different testing, by simply modifying the storyboard animation to run only for few seconds, the CPU usage drop to 45%

Of course there is not only the fact of stopping a storyboard with make such measure possible but that was the goal of this single test only. Having effect on storyboard consume a lot of resource as well and can be avoided whenever possible.

Always bring alternative to object interaction

Interacting with object is the must have feature for a PixelSense application offering better user experience in a natural way. Objects gets build according to the impact and image to be deliver to the audience and they are in close relation to direct information content attached.

I have been face to a personal bad experience in an earlier application during an event where we bring some nice object specially built to interact with the application I was showing. At the beginning it was cool until the time you let visitors playing around the table and suddenly discover that some objects are not anymore present.

“A missing object should not avoid accessing to it’s initial content”

I do not know what I get in mind when I was preparing the application, but I get a strange feeling that I had to be able to access object content without the object. And this is what I have done and saved my demo at that time.

Do not hesitate to disable unwanted touch input if needed

After releasing an application and deploy it to the customer location, I always spend time to clearly look at users and the way they interact with the application in order to bring fine tuning during validation process on site. During that phase, I write down all that works as well as all that does not work.

I was of course expected different type of people around my application but also people wearing different type of clothes.

“Why am I talking about clothes here as we are talking about designing an application? “

Well, have you ever present the result of your application to your manager and ask him to try it? I am sure yes and at that time you have also notice that his long tie touching the screen disturbed his interaction with requested content. This is what could happen in real as well, as you are not expected to wear only tee-shirt to use your application correctly.

At this time you need to adjust you application in the environment it is used. For such you simply need from your main application windows to disable all touch input that your application do not need.

Use a single SurfaceWindow for better portability and flexibility

In PixelSense application design and WPF in general we should forget the multi window approach and going more to a single Window showing different type of content as illustrated below:



This architecture will offer flexibility and easier portability. With such approach we have a single main input entry for the whole application and content type gets displayed based on different criteria and directly activated through data binding. The power of the MVVM pattern here takes all it’s importance.

Let’s keep In mind as well that the Surface SDK 2.0 is compatible with Windows 7 platform. By having one single input window for the whole application the compatibility change of the application for Windows 7 compatibility will be resume only by providing a Surface Window for PixelSense app and a standard WPF Windows for Window7.

The only change to switch between those environments will be done from the App.xaml.cs file by overriding the App_Startup as shown below:

Void App_Startup (object sender, StartupEventArgs e)
if (InteractiveSurface.PrimarySurfaceDevice.IsTagRecognitionSupported == true)
//define Surface environment
StartupUri = new Uri (@”/PropertySense;component/SurfaceMainWindow.xaml”, UriKind.Relative);
//define Windows 7 environment
StartupUri = new Uri (@”/PropertySense;component/MainWindow.xaml”, UriKind.Relative);

Drag & Drop operation are far from being natural if not shown

During whole application design where drag and drop was in place, I have notice how complicated it is to make such operation intuitive and evident for users when they are in front of your application for the first time.

By this I really mean that you let users manipulate your application and simply see them, without showing or saying anything. We make this simple test where we provide a simple view of photos equivalent as the one below:




How do you think user does in front of such photos collection?

First they see an horizontal slider so they understand easily that sliding from left to right can give access to more content. The next thing they most all do is touching a picture and wait. All users were doing exactly the same thing until I decide to show them that they can drag.

"Always implement the Tap gesture to your component"

From that simple test and great feedback results, you need to start to reconsider that interaction. It does not mean that the drag/ drop interaction is bad in this condition but more how to have it more intuitive to users. I start to explore different approach as:

  • Add to the interaction the Tap gesture on all element in the list
  • Display a Notification message to the user at a certain time
  • Display text information inside the visual element being manipulated
  • Simulate the drag operation when no activities on the list

From those possible solution, I came from the fact that simulating the interaction with a small storyboard sequence was the most effective way to bring users to repeat that gesture

This scenario keeps me remember all time that people never do what you are expecting them to do in some situation.

Users are the main source of inspiration

When you have completed the deployment of your application, after a certain time of use, always collect feedback of users manipulating your application. If you have the possibility to go at the location, do not hesitate and simply open your eyes. You will surely see something that you will have never think of or never seen.

“By looking at others using your app, you are already thinking of your next application design”

Hoping that I could help you a bit in your reflection based on my successful deployment so far where those rules gets extracted. Of course they engaged only my own personal opinion based on proven scenarios.

Provide a way to clean your screen

When you build a Surface application for Microsoft PixelSense device and have in mind the multi user approach, cleaning the screen is not necessary the first thing you are thinking of. This comes once again when you look at users the way they use the app. In all application I have made I could notice that users always open content, but most of the time never close it after they have seen it. This is the reason why after some time of user your main application screen is full of content, the the next user coming to use your app will be confuse on where to start.

Base on that fact we need to try to follow those simple aspect of your content :

    - Always provide a close button to be able to close manually your visual content
    - Provide an inactivity time out on your content in order that after a certain time of no use, content will be closed automatically