locked
different expression sequence different result RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote
    Could anyone answer me that why below 2 expressions have different result ? Thanks in advance.

    1.SELECT FLOOR(CAST(91200 AS FLOAT)/160000*100)
       return 56
    2.SELECT FLOOR(CAST(91200 AS FLOAT)*100/160000)
       return 57

    Doesn't it should be 57 for both since 91200/160000=0.57  and 0.57*100=57 ?
    Wednesday, October 14, 2009 6:47 AM

Answers

  • Question
    Sign in to vote
    1
    Sign in to vote
    Hi Daphne

    Since float is an approximate datatype, you may get rounding errors like this. In this case the rounding error is magnified due to the boundary between 56.999999999999993 and 57.

    If you use decimal you'll get consistent results.

    HTH

    Ewan
    If you have found this post helpful, please click the 'Vote as Helpful' link (the green triangle and number on the top-left).

    If this post answers your question, click the 'Mark As Answered' link below. It helps others who experience the same issue in future to find the solution.
    • Marked as answer by Zongqing Li Tuesday, October 20, 2009 8:58 AM
    Wednesday, October 14, 2009 7:29 AM