locked
Why Microsoft Remove SIS features from Exchange 2010 RRS feed

  • Question

  • Dear Microsoft,

    Why have you remove SIS (Single Instance Storage) features from Exchange 2010? My all customers are complaing that due to removal of this features my disk space will consume more space because SIS just keep pointer record for all attachment size which is send to multiple users in Exchange 2010.

    I have gone through technet document and on that you have mentioned that only message header will be compress and not for attachment compression.  Is there any specific reason for removing this features from Exchange 2010 ?



    One more complain from my all customer that Microsoft Documentation is so bad because they are just giving the text based document not attaching the screenshot.

    So i would like to suggest Microsoft documentation team please preapre such types of document on which screenshot also included becasue most of world wide Consultant, Architect and engineers are like such types of document on which  screenshot Included.

    Please change your documentation format its my request so that everybody can learn Microsoft Technology very easily and can Implement product their own company. It will increase Microsoft business productivity. Because due to Microsoft documentation format no one can learn Microsoft product himself and due to this reason they are unaware on most of the technology and Microsoft loosing their business.


    Cheers,






    Subhash Tiwari |MCA|MCSE|MCSA|MCDBA|MCTS|MCITP | Blog-> http://subhashtiwari.spaces.live.com
    Saturday, February 6, 2010 8:29 AM

All replies

  • SIS was actually removed (broken?) with one of the Exchange 2007 Service packs.

    For Exchange 2010, you might have noticed the drastic drop in the disk IO load.  For Exchange 2007, the big factor was the increased cache availability as a result of moving to a 64 bit platform.  For Exchange 2010, the reduction came about through a redesign of the database schema. a change in the size of the database pages as well as a change in the way the tables are organised.

    The basic changes are documented here  http://technet.microsoft.com/en-us/library/bb125040.aspx this post also discusses the changes in a little more detail http://social.technet.microsoft.com/Forums/en/exchange2010/thread/eff4ef8c-7776-432e-ad2d-cab507c1f767 and this post also expands on it http://www.enowconsulting.com/ese/2010_01_01_archive.html

    hth

    Gordon
    www.longneckconsulting.com
    Saturday, February 6, 2010 11:29 AM
  • One thing you have to know that Exchnage 2007 uses Random method when reading or Writing Data , but Exchange 2010 uses sequential method to read and write data ...
    Saturday, February 6, 2010 1:44 PM
  • Why have you remove SIS (Single Instance Storage) features from Exchange 2010? My all customers are complaing that due to removal of this features my disk space will consume more space because SIS just keep pointer record for all attachment size which is send to multiple users in Exchange 2010.

    I have gone through technet document and on that you have mentioned that only message header will be compress and not for attachment compression.  Is there any specific reason for removing this features from Exchange 2010 ?

    Hi Subhash,

    Please keep in mind during the design phase of an Exchange storage subsystem, SIS was never a consideration or variable used in any calculation. If any user were to modify the SIS'd message at all it would then store a second altered copy in their mailbox. There was never a guarantee every user wouldn't do this and then you would run out of space. Therefore the removal of SIS and the addition of message body compression shouldn't change the way we design the storage system :)

    Can you provide some detailed examples of bloated database sizes your customers have seen?
    Brian Day, Overall Exchange & AD Geek
    MCSA 2000/2003, CCNA
    MCTS: Microsoft Enterprise Server 2010, Configuration
    MCITP: Enterprise Messaging Administrator 2010
    LMNOP
    Saturday, February 6, 2010 2:32 PM
  • Storage is cheap. Disk I/O is a bottle neck. By not dealing with SIS there is a performance gain for the individual mailbox. Plus when there is SIS there has be logic around when and where to store the object as different mailboxes are moved, or delete the object. By not having that logic Exchange eliminates the opportunity for either the object living indefinatly, or being removed too soon. The reason it was removed is because mailbox performance improves.

    Saturday, February 6, 2010 6:01 PM
  • Hello Brian,

    Thanks for giving me feedback for removing SIS from Exchnage 2010.

    Regarding your question on the example, I have 10 Enterprise customer who have more that 25000 users and all of them HR dept, Legal dept, Markt. dept. etc. sending mail to internal All Employee with having attchment more than 10 MB size to everyday but no one modifying that attachment.

    And you can understand if suppose that 100 users will send 10 MB attachment to all 25000 users then how much space it will consume. I hope you can understand my objective. Due to this reason my customer complaing that why Microsoft remove this features.  Its a major concern.


    Cheers,

    Subhash tiwari
    Sunday, February 7, 2010 5:25 AM
  • To me, email doesn't sound like the solution to that problem. It sounds like the customers need file shares. A solution such as SharePoint should help with that.

    Sunday, February 7, 2010 6:22 AM
  • Even though file shares can be used, there's always the uninformed user that sends email containing a 2MB attachment with funny video.wmv to 2000 users anyway.

    There are 3rd party applications that can re-introduce sis in exchange 2010. These applications perform data deduplication
    eg.  enterprisevault from symantec

    See last comment here

    http://www.networkcomputing.com/servers-storage/exchange-2010-changes-storage-demands.php
    Sunday, February 7, 2010 12:16 PM
  • Hello Jader / Anand,

    Thanks for your suggestion. But my customer main objective is that if they are not planning to buy SharePoint & Symantec EnterpriseVault because again they will have to pay some license cost for this solution. Meanwhile on Exchange 2003 /2007 have inbuilt features for reducing the disk space cost.

    I don't think its a best practice for removing a wonderfull features which can give first impression to the Enterprise customer for pitching Exchange 2010.


    Dear Microsoft,

    I would like to suggest you that please before removing any features from any product atleast you guys think about that product features value and business impact because due to your small mistake another vendor can defaet your product and win a big deal.

    I hope you guys will try to reintroduce this features on your next release of Exchange Server version.

    Cheers,


    Subhash Tiwari |MCA|MCSE|MCSA|MCDBA|MCTS|MCITP | Blog-> http://subhashtiwari.spaces.live.com
    Monday, February 8, 2010 5:47 AM
  • Regarding your question on the example, I have 10 Enterprise customer who have more that 25000 users and all of them HR dept, Legal dept, Markt. dept. etc. sending mail to internal All Employee with having attchment more than 10 MB size to everyday but no one modifying that attachment.

    And you can understand if suppose that 100 users will send 10 MB attachment to all 25000 users then how much space it will consume. I hope you can understand my objective. Due to this reason my customer complaing that why Microsoft remove this features.  Its a major concern.

    If the user's quotas were properly defined before the system is deployed how would this matter? The storage system would have been designed to handle X users by Y amount of space. SIS really shouldn't matter in that case.

    Do you have some customers with examples of DB sizes showing far larger than the same DB on 2003 or 2007? I'd love to see some empirical data to help us understand your concern. MS seemed rather confident the lack of SIS was made up for if not even done better by compressing the other areas of the DB.
    Brian Day, Overall Exchange & AD Geek
    MCSA 2000/2003, CCNA
    MCTS: Microsoft Enterprise Server 2010, Configuration
    MCITP: Enterprise Messaging Administrator 2010
    LMNOP
    Monday, February 8, 2010 5:56 AM
  • Hello Brian,


    See right now my all customers are using Exchange 2003 /2007 now we are proposing the solution to them but they are asking from us that why Microsoft have removed this features is there any specific reason and for the same we don't have any answer.


    And regaridng the SIS features i wold like to ask you one thing that if suppose that i am sending one email to 5000 users  which having 10 MB attachment then how much space it will covered on the DB i mean on the database without SIS and how much users mailbox size will get increase. Please explain it to me for both Exchange 2003/2007 and 2010.

    See in Exchange 2003 / 2007 SIS it does as per below...

    If suppose that any user will send a mail to 100 users with attachment 10 MB size then it just calculating 10 MB size on DB and for another users its just creating a reference pointer for that attachment. So its not calculating 10 MB size on DB for each and every users.

    So my customers saying that if suppose that i will send a mail on Exchange 2010 then it will consume each mailbox 10 MB size on DB  and overall size would be 100 * 10 = 1000 MB. Due to this reason we are helpless to explain it to customer end.


    Please revert.

    Cheers,
    Subhash Tiwari |MCA|MCSE|MCSA|MCDBA|MCTS|MCITP | Blog-> http://subhashtiwari.spaces.live.com
    Monday, February 8, 2010 10:00 AM
  • I understand your situation and I am also concerned about this. If the exchange store does not compress attachments then I don't how exchange 2010 can actually save hard drive space without SIS.

    See the following-

    http://techrepublic.com.com/5208-13400-0.html?forumID=102&threadID=323711&messageID=3223083&tag=content;leftCol


    3rd QA on this page:
    http://technet.microsoft.com/en-us/magazine/2009.09.exchangeqa.aspx
    "The Exchange Product Group compressed the databases (more specifically, the message headers and text or HTML bodies)."


    http://www.kapie.com/blog/2009/08/14/Exchange2010StoreCompression.aspx
    "Store Compression actually only compresses the email headers and any text or html body text"

    Monday, February 8, 2010 10:50 AM
  • So my customers saying that if suppose that i will send a mail on Exchange 2010 then it will consume each mailbox 10 MB size on DB  and overall size would be 100 * 10 = 1000 MB. Due to this reason we are helpless to explain it to customer end.



    We may have to agree to disagree on this issue as I do not see it as important as you do. :)

    To me if the DB grew by 1000MB due to sending a 10MB file to 1000 people I wouldn't care because that means that there was enough space left in all of the mailbox quotas for the files to fit and that would mean I am still within my designed DB size and backup strategy.

    SIS was never supposed to be a design variable in previous versions of Exchange, so the number crunching for 2010 should be no different than it was before.


    I'm still very interested in seeing real-life data from any customer(s) you have actually suffering from this. What did their DBs look like on 2003/2007 and what do they now look like on 2010? Did they not plan accordingly thinking their users would never fill their quotas? Seeing some real life data may help all of us understand and plan accordingly.
    Brian Day, Overall Exchange & AD Geek
    MCSA 2000/2003, CCNA
    MCTS: Microsoft Enterprise Server 2010, Configuration
    MCITP: Enterprise Messaging Administrator 2010
    LMNOP
    Monday, February 8, 2010 2:45 PM
  • Hello Brian,

    For your kind information no one Enterprise customer will provide console to outside company person because every EA customer have their own policy. I hope you can understand because you are working in a one of the top UK based bank and you know this kind of policy. So it can't be possible to provide you console to see and rectify this issue.

    If you want to rectify this issue you can install it on your lab  Exchange 2003, 2007 and 2010 and create dummy users and try to send email with 10 MB attachment on all version of Exchange and see the difference about the DB size.

    Regarding the SIS and Mailbox Quota actually you are thinking users mailbox quota level but i am thinking DB size level and you are saying that you wouldn't care but why? Because if suppose that my DB size increasing everyday 1 GB on Exchange 2010 as compare to Exchange 2003 / 2007 then its really bad impression on product.

    So i would like to suggest you that you think about the DB level don't think about the individual users Mailbox Quota level.


    Cheers,



    Subhash Tiwari |MCA|MCSE|MCSA|MCDBA|MCTS|MCITP | Blog-> http://subhashtiwari.spaces.live.com
    Tuesday, February 9, 2010 6:00 AM
  • I'm sure that Exchange is flattered that your customers use Exchange as a file storage and delivery solution, but that's not what Exchange is optimized for.
    Recent post on the Exchange team blog: http://msexchangeteam.com/archive/2010/02/22/454051.aspx
    • Proposed as answer by jader3rd Sunday, March 21, 2010 10:08 PM
    Friday, February 26, 2010 5:44 AM
  • Just adding my thoughts after the fact here ...  It seems that SIS is a very good idea.  However, IMHO, a better solution would be file-system-based attachment storage.  In this scenario, the attachment gets stored on a share somewhere and a "pointer" to the attachment is stored in the database.  This technology is now available in both SQL Server and SharePoint Server so it would seem the rest of Microsoft is moving towards this type of attachment/BLOB storage mechanism. 

    This would also allow the bulk of the volume of the Exchange dB to be off-loaded to Tier 3 storage.  Disk is cheap, but not when it's Tier 1 (which is what you need to use for Exchange dB storage if you want the speed and reliability). 

    What about it Microsoft Exchange Team? 

    Does anyone know if there is a product out there that already does this?

    Thursday, May 27, 2010 5:34 PM
  • You raise an interesting point. However, let me pose this to you.
    In SharePoint (which is what it boils down to frankly) if a person uploads a document it can be blobbed out. If it's checked out, edited and checked in as a new version it's a partial copy. If someone else uploads to a separate area it's a new file. No SIS involved there. A new blob entirely. Now take Exchange. Someone sends an email in 2007 with an att so a bunch of people in the same store. Sure, it's single instanced but the structure of 2007 means that a lot of traversal needs to be done to get the attachment. In 2010 it's all right there. Yes there are multiple copies of it but at least the IO plummets because the storage doesn't have to go far to get it. Pro's and Con's on both sides. I work for a storage vendor that is the player in primary deduplication so we're super-fine with it.
     
    Sadly, and I like your idea, it's not something that's achievable at the moment in Exchange or SharePoint.
     
    And then of course there's the whole 'tier' argument in storage. That's old fashioned thinking. Our solution does Exchange on SATA and we employ, if necessary, a card in front of the storage on the controller to accelerate performance. In 2010 on SATA this is actually rarely, if ever, necessary. So it's a tier 1 application, sure, you can have that, but tiering your storage is a couple of years out of date.
     
    All your idea is going to do is to result in small EDB files and a ton of identical data on CIFS shares. Now, I like that as an idea. A lot. But then you're suddenly, if you use your thinking, putting a load of Exchange information onto CIFS shares. Are you now upgrading your file servers (or NAS) to tier 1? That's a fresh argument.
     
    Excellent debate though.
    "Ian Miller" wrote in message news:6c5a404b-ac53-48df-a3e8-af6765261223...

    Just adding my thoughts after the fact here ...  It seems that SIS is a very good idea.  However, IMHO, a better solution would be file-system-based attachment storage.  In this scenario, the attachment gets stored on a share somewhere and a "pointer" to the attachment is stored in the database.  This technology is now available in both SQL Server and SharePoint Server so it would seem the rest of Microsoft is moving towards this type of attachment/BLOB storage mechanism. 

    This would also allow the bulk of the volume of the Exchange dB to be off-loaded to Tier 3 storage.  Disk is cheap, but not when it's Tier 1 (which is what you need to use for Exchange dB storage if you want the speed and reliability). 

    What about it Microsoft Exchange Team? 

    Does anyone know if there is a product out there that already does this?


    Mark Arnold, Exchange MVP.
    Thursday, May 27, 2010 6:45 PM
  • I'm curious why Microsoft never moved exchange to sql? I've heard from the development team since exchange 2000 that it was going to move from Jet to Sql but here we are with 2010 and still no move to Sql? 

    With it moving to Sql I would think it would open all kinds of possibilities and feature sets. 

    To me, and forgive me since I don't know your inner company workings or the "Grand" scheme of things, but supporting different databases seems counter intuitive imho for a software company.  Also it leaves people with a view that maybe Sql isn't all its cracked up, otherwise why wouldn't microsoft move exchange (their premier product) to it? Are there concerns doing so would open it up to some competitor stealing it away?

     

    Tuesday, January 25, 2011 6:19 PM
  • Subash,

    I hate to be cynical, but do you have 25,000 users in a single storage group? From what I remember the Storage Group was the end limit for the SIS. It's not organization wide for the SIS.

    Really though, we're sort of cursed with the idea that people tend to use Exchange as a document database and not an Email, contacts and calendaring collaboration system. I would strongly suggest talking to the people in charge of the organization about how people use the system and see if they can curve some of the bad user habits.

    /Peter McCollough


    Peter McCollough, Messaging Consultant with Humandata AB, Stockholm, Sweden email me at peter AT humandata DOT se
    Friday, July 15, 2011 12:05 PM
  • Subash,

    I hate to be cynical, but do you have 25,000 users in a single storage group? From what I remember the Storage Group was the end limit for the SIS. It's not organization wide for the SIS.

    Really though, we're sort of cursed with the idea that people tend to use Exchange as a document database and not an Email, contacts and calendaring collaboration system. I would strongly suggest talking to the people in charge of the organization about how people use the system and see if they can curve some of the bad user habits.

    /Peter McCollough


    Peter McCollough, Messaging Consultant with Humandata AB, Stockholm, Sweden email me at peter AT humandata DOT se

    Actually, SIS was per mailbox store, not per storage group.
    Friday, July 15, 2011 12:08 PM
  • Alaskan,

    I've heard people ask this kind of question over and over. Exchange has gotten to a point where ESE works really well for what it does. The database is simple, smooth and efficient. SQL is a lot more feature rich, but a lot more complicated to setup and administer. Also, just think about the idea of what a database structure for Exchange would look like in SQL. Not to mention BLOB storage in SQL is the biggest thing we'd sort of be talking about in context of this post and that again is a sort of convulted mess in SQL.

    I like ESE. Each tool works for what it's supposed to work for, not everything.

     Just my thoughts on it.

    /Peter McCollough


    Peter McCollough, Messaging Consultant with Humandata AB, Stockholm, Sweden email me at peter AT humandata DOT se
    Friday, July 15, 2011 12:14 PM
  • Well then the question is whether the 25,000 are in one database and next being that if 25,000 edit the same piece of mail and save their own revision, is Exchange smart enough to notice just the changes to the email to save or does it save the whole piece of mail again 25,000 times?


    Peter McCollough, Messaging Consultant with Humandata AB, Stockholm, Sweden email me at peter AT humandata DOT se
    Monday, July 18, 2011 12:00 PM
  • I know this is an old thread now, but times have moved on....

    Windows Server 2012 using Data Deduplication feature might be of help

    you can store the databases on very cheap "laptop or desktop class" hard drives as they don't have to be high performance disks due to the reduced I/O. Server class disks can easily be 4 times as expensive than these cheaper disks and of lower capacity. example we can get a 500GB laptop drive for around £50, yet a 140GB HP SAS 15k drive is over £200....

    use the combination of cheap storage and possibly investigate data deduplication technologies which may help you justify the removal of SIS.

    I also thought it was a shame to remove SIS but as others have mentioned on this thread, there is no guarantee SIS will be used anyway if users modify their files. in fact I would go a step further and reiterate what others have said and persuade your customers at the very least to use a network folder share.... or even the free SharePoint version to gain further benefits of file sharing and collaborating.

    Using email systems to share and distribute files is and always has been an appropriate place to store files, emails belong in an email system and files belong on a file system (hence the names of the systems). we have actually blocked being able to save email files to the file system, I only wish we could do the same for the email system but reducing the attachment size lower than 10MB isn't always practical for us if sending information outside the business.

    Hope that helps

    Steve

    Wednesday, September 26, 2012 2:14 PM