I just finished a long troubleshooting session with Microsoft support and they were able to confirm that this is a BUG in the integration between SP and Outlook. If you change the number of "assign to" people to just one person or group it will synchronize OK. We will have to wait on a hotfix from MS on this problem. I hope this helps other people from wasting their time troubleshooting.