Обсуждение: modulo operator bug?

Поиск
Список
Период
Сортировка

modulo operator bug?

От
Kemal Bayram
Дата:
from psql:

kemal=# select 0.99 % 0.50;
 ?column?
----------
     0.04
(1 row)

Am I missing something fundemental here or is this indeed a bug?

Re: modulo operator bug?

От
Tom Lane
Дата:
Kemal Bayram <kemal@corenetworks.co.uk> writes:
> from psql:
> kemal=# select 0.99 % 0.50;
>  ?column?
> ----------
>      0.04
> (1 row)

> Am I missing something fundemental here or is this indeed a bug?

Looks like a bug to me too.  mod_var shouldn't be using tmp.rscale,
I think.

            regards, tom lane

Re: modulo operator bug?

От
Tom Lane
Дата:
> Kemal Bayram <kemal@corenetworks.co.uk> writes:
>> from psql:
>> kemal=# select 0.99 % 0.50;
>> ?column?
>> ----------
>> 0.04
>> (1 row)

>> Am I missing something fundemental here or is this indeed a bug?

> Looks like a bug to me too.  mod_var shouldn't be using tmp.rscale,
> I think.

OK, fixed for 7.1.  If you need a fix in 7.0.*, the attached patch should
apply to 7.0.*, although the line numbers will be different.

            regards, tom lane


*** src/backend/utils/adt/numeric.c.orig    Wed Dec  6 21:47:35 2000
--- src/backend/utils/adt/numeric.c    Wed Mar 14 11:50:37 2001
***************
*** 3355,3370 ****
      init_var(&tmp);

      /* ----------
!      * We do it by fiddling around with global_rscale and truncating
!      * the result of the division.
       * ----------
       */
      save_global_rscale = global_rscale;
      global_rscale = var2->rscale + 2;

      div_var(var1, var2, &tmp);
      tmp.rscale = var2->rscale;
-     tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + tmp.rscale + 1));

      global_rscale = var2->rscale;
      mul_var(var2, &tmp, &tmp);
--- 3355,3373 ----
      init_var(&tmp);

      /* ----------
!      * We do this using the equation
!      *        mod(x,y) = x - trunc(x/y)*y
!      * We fiddle a bit with global_rscale to control result precision.
       * ----------
       */
      save_global_rscale = global_rscale;
      global_rscale = var2->rscale + 2;

      div_var(var1, var2, &tmp);
+
+     /* do trunc() by forgetting digits to the right of the decimal point */
+     tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + 1));
      tmp.rscale = var2->rscale;

      global_rscale = var2->rscale;
      mul_var(var2, &tmp, &tmp);