I'm not sure if this was an issue in 2005 that has been corrected in a newer version. I've tested it using 2008 R2 and can't reproduce the same behavior. As a work-around, you can explictly set the Series Value Fill property using an expression.
My example uses the AdventureWorks2008 database. The stcked column chart Series is grouped on the Category field. The following expression is used on the Fill property to set the color based on a set of anticipated values:
Microsoft is conducting an online survey to understand your opinion of the Technet Web site. If you choose to participate, the online survey will be presented to you when you leave the Technet Web site.