py/obj: Fix REPR_C bias toward zero.

Current implementation of REPR_C works by clearing the two lower bits of
the mantissa to zero.  As this happens after each floating point operation,
this tends to bias floating point numbers towards zero, causing decimals
like .9997 instead of rounded numbers.  This is visible in test cases
involving repeated computations, such as `tests/misc/rge_sm.py` for
instance.

The suggested fix fills in the missing bits by copying the previous two
bits.  Although this cannot recreate missing information, it fixes the bias
by inserting plausible values for the lost bits, at a relatively low cost.

Some float tests involving irrational numbers have to be softened in case
of REPR_C, as the 30 bits are not always enough to fulfill the expectations
of the original test, and the change may randomly affect the last digits.
Such cases have been made explicit by testing for REPR_C or by adding a
clear comment.

The perf_test fft code was also missing a call to round() before casting a
log_2 operation to int, which was causing a failure due to a last-decimal
change.

Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>

1 files changed, 4 insertions, 0 deletions

diff --git a/py/obj.h b/py/obj.h
index a1df661ff..6c2153697 100644
--- a/py/obj.h
+++ b/py/obj.h
@@ -206,6 +206,10 @@ static inline mp_float_t mp_obj_float_get(mp_const_obj_t o) {
         mp_float_t f;
         mp_uint_t u;
     } num = {.u = ((mp_uint_t)o - 0x80800000u) & ~3u};
+    // Rather than always truncating toward zero, which creates a strong
+    // bias, copy the two previous bits to fill in the two missing bits.
+    // This appears to be a pretty good heuristic.
+    num.u |= (num.u >> 2) & 3u;
     return num.f;
 }
 static inline mp_obj_t mp_obj_new_float(mp_float_t f) {