Re: Arbitrary precision modulo operation

Поиск
Список
Период
Сортировка
От Paul Tillotson
Тема Re: Arbitrary precision modulo operation
Дата
Msg-id 408F227E.9060800@shentel.net
обсуждение исходный текст
Ответ на Re: Arbitrary precision modulo operation  (Alvaro Herrera <alvherre@dcc.uchile.cl>)
Ответы Re: Arbitrary precision modulo operation  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-general
Alvaro Herrera wrote:

>On Mon, Apr 26, 2004 at 12:48:45PM -0700, Dann Corbit wrote:
>
>
>>Maple output:
>>y := 123456789012345678901234567890 mod 123;
>>                               y := 117
>>
>>
>
>PgSQL 7.3.6 gives the right answer (117), 7.4 gets it wrong (-6). Most
>likely a bug was introduced when NUMERIC was rewritten.  Strange it
>hasn't been noticed before.
>
>
>
mod(x, y) is computed as x - trunc(x / y) * y in the mod_var() function
(I think).

However, it appears that the division operator itself is rounding up,
such that the trunc() function  (which ought to round down) does no good
as a round up has already occurred.

Thus, the value of (x / y) is 1 too large, and so  x % y is actually
giving you (x % y) - y, a negative number.  I tried looking at how the
division actually works, but it is over my head at least for the 30
minute perusal.

Regards,
Paul Tillotson

-----------------------------------------------------------------------

[paul@pjt4 paul]$ bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
111111111111111111 / 6
18518518518518518

[paul@pjt4 bin]$ ./psql -U postgres template1
Welcome to psql 7.4.2, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help on internal slash commands
       \g or terminate with semicolon to execute query
       \q to quit

template1=# select 111111111111111111::numeric / 6;
     ?column?
-------------------
 18518518518518519
(1 row)

template1=# select 111111111111111111 / 6;
     ?column?
-------------------
 18518518518518518
(1 row)

template1=# select version();

version

---------------------------------------------------------------------------------------------------------
 PostgreSQL 7.4.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.2
20031022 (Red Hat Linux 3.3.2-1)
(1 row)






В списке pgsql-general по дате отправления:

Предыдущее
От: Paul Tillotson
Дата:
Сообщение: Re: Question
Следующее
От: Tom Lane
Дата:
Сообщение: Re: Cannot open relation pg_cast_source_target_index