The default install of IIS on Windows Vista Beta 2 is missing several components the Virtual Server needs to run. Most importantly it does not support CGI applications or Windows authentication. You can manually select these components under the Windows component installer. you will also need to select to install everything under the 'IIS 6 Management Compatibility' section under 'Web Management tools' (and the already mentioned CGI and Windows authentication support). In order to access Virtual Server, you will need to launch Internet Explorer 'as Administrator' and manually go to the Virtual Server website (http://localhost/VirtualServer/vswebapp.exe)
To sum it up and make things easier, install everything under Web management and World Wide Web Services in IIS. then install virtual server.