locked
Column total not showing properly RRS feed

  • Question

  • I have a number column, in my view it's setup to show the "sum" of this # column.  Oddly enough showing "Sum = 9.09E-13" when it should be showing "0.00".  Check it out:

     

    If you total this it's supposed to come to "0".  Here's my column setup:

     

     

    Any ideas why this might be happening?
    Tuesday, August 10, 2010 8:48 PM

Answers

  • My guess is that "Number of decimal places:" is only a ornamental limitation, meaning that it only comes into play when the value is being displayed, not when it is stored.

    So, you've made the error of:
    1. imported or converted your data from another data source.
    2. calculated it on the fly and stored it.

    Try upping your decimal places to 6 or so, and see what's really behind the curtain.  (Export to csv or txt if you can't get a read on it.) 

    Steve Clark, MCTS 70-542
    Twin-Soft.com

    • Marked as answer by Stanfford Liu Friday, August 20, 2010 3:00 AM
    Tuesday, August 10, 2010 9:16 PM

All replies

  • What was the source of the data? Imported or copy/paste from Excel?  Or was it hand typed?

    My guess is that it was copy and pasted, or imported and the numbers were something like 1.23333333333 that got displayed as 1.23, or 1.0000000000001 that got displayed as 1.00.   (E-13 is a VERY tiny number)

    If the data was not hand typed, could the source (Excel?) be updated to do rounding (not formatting) before the import or copy/paste?

     


    Mike Smith TechTrainingNotes.blogspot.com
    Tuesday, August 10, 2010 9:08 PM
  • My guess is that "Number of decimal places:" is only a ornamental limitation, meaning that it only comes into play when the value is being displayed, not when it is stored.

    So, you've made the error of:
    1. imported or converted your data from another data source.
    2. calculated it on the fly and stored it.

    Try upping your decimal places to 6 or so, and see what's really behind the curtain.  (Export to csv or txt if you can't get a read on it.) 

    Steve Clark, MCTS 70-542
    Twin-Soft.com

    • Marked as answer by Stanfford Liu Friday, August 20, 2010 3:00 AM
    Tuesday, August 10, 2010 9:16 PM