SYSTEM AND METHOD FOR ROUNDING RECIPROCAL SQUARE ROOT RESULTS OF INPUT FLOATING POINT NUMBERS

摘要

Methods and systems for determining whether an infinitely precise result of a reciprocal square root operation performed on an input floating point number is greater than a particular number in a first floating point precision. The method includes calculating the square of the particular number in a second lower floating point precision; calculating an error in the calculated square due to the second floating point precision; calculating a first delta value in the first floating point precision by calculating the square multiplied by the input floating point number less one; calculating a second delta value by calculating the error multiplied by the input floating point number plus the first delta value; and outputting an indication of whether the infinitely precise result of the reciprocal square root operation is greater than the particular number based on the second delta term.

主权项

1. A system configured to determine whether an infinitely precise result of a reciprocal square root operation performed on an input floating point number is greater than, less than, or equal to a particular number in a first floating point precision, the system comprising:
one or more fused multiply add components configured to:
(a) calculate a square of the particular number in a second floating point precision that is less than the first floating point precision;(b) calculate a rounding error in the calculated square;(c) calculate a first delta value in the first floating point precision by calculating the square multiplied by the input floating point number less one; and(d) calculate a second delta value in the first floating point precision by calculating the rounding error multiplied by the input floating point number plus the first delta value; and an output module configured to output an indication of whether the infinitely precise result of the reciprocal square root operation is greater than, less than or equal to the particular number based on a sign of the second delta value.