Обсуждение: How to update unicode mapping table?

Hi, hackers

When I try to update the unicode mapping table through *.xml in
src/backend/utils/mb/Unicode/, it doesn't update the *.map files.
I find the make cannot go to this directory, what can I do to update
the mapping tables?

-- 
Regrads,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.

On 25.09.23 08:02, Japin Li wrote:
> When I try to update the unicode mapping table through *.xml in
> src/backend/utils/mb/Unicode/, it doesn't update the *.map files.
> I find the make cannot go to this directory, what can I do to update
> the mapping tables?

This is done by "make update-unicode".

Peter Eisentraut <peter@eisentraut.org> writes:
> On 25.09.23 08:02, Japin Li wrote:
>> When I try to update the unicode mapping table through *.xml in
>> src/backend/utils/mb/Unicode/, it doesn't update the *.map files.
>> I find the make cannot go to this directory, what can I do to update
>> the mapping tables?

> This is done by "make update-unicode".

On a slightly related note, I noticed while preparing 3aff1d3fd
that src/backend/utils/mb/Unicode/Makefile seems a little screwy.
If you go into that directory and type "make distclean", you'll
find it removes some files that are in git:

[postgres@sss1 Unicode]$ make distclean
rm -f 8859-10.TXT 8859-13.TXT 8859-14.TXT 8859-15.TXT 8859-16.TXT 8859-2.TXT 8859-3.TXT 8859-4.TXT 8859-5.TXT
8859-6.TXT8859-7.TXT 8859-8.TXT 8859-9.TXT BIG5.TXT CNS11643.TXT CP1250.TXT CP1251.TXT CP1252.TXT CP1253.TXT CP1254.TXT
CP1255.TXTCP1256.TXT CP1257.TXT CP1258.TXT CP866.TXT CP874.TXT CP932.TXT CP936.TXT CP950.TXT JIS0212.TXT JOHAB.TXT
KOI8-R.TXTKOI8-U.TXT KSX1001.TXT euc-jis-2004-std.txt gb-18030-2000.xml sjis-0213-2004-std.txt windows-949-2000.xml 
[postgres@sss1 Unicode]$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    euc-jis-2004-std.txt
        deleted:    gb-18030-2000.xml
        deleted:    sjis-0213-2004-std.txt

no changes added to commit (use "git add" and/or "git commit -a")

This seems wrong.  If you "make maintainer-clean", that removes even more:

$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    big5_to_utf8.map
        deleted:    euc-jis-2004-std.txt
        deleted:    euc_cn_to_utf8.map
        deleted:    euc_jis_2004_to_utf8.map
        deleted:    euc_jp_to_utf8.map
        deleted:    euc_kr_to_utf8.map
        deleted:    euc_tw_to_utf8.map
        deleted:    gb-18030-2000.xml
        deleted:    gb18030_to_utf8.map
        deleted:    gbk_to_utf8.map
        deleted:    iso8859_10_to_utf8.map
        deleted:    iso8859_13_to_utf8.map
        deleted:    iso8859_14_to_utf8.map
        deleted:    iso8859_15_to_utf8.map
        deleted:    iso8859_16_to_utf8.map
        deleted:    iso8859_2_to_utf8.map
        deleted:    iso8859_3_to_utf8.map
        deleted:    iso8859_4_to_utf8.map
        deleted:    iso8859_5_to_utf8.map
        deleted:    iso8859_6_to_utf8.map
        deleted:    iso8859_7_to_utf8.map
        deleted:    iso8859_8_to_utf8.map
        deleted:    iso8859_9_to_utf8.map
        deleted:    johab_to_utf8.map
        deleted:    koi8r_to_utf8.map
        deleted:    koi8u_to_utf8.map
        deleted:    shift_jis_2004_to_utf8.map
        deleted:    sjis-0213-2004-std.txt
        deleted:    sjis_to_utf8.map
        deleted:    uhc_to_utf8.map
        deleted:    utf8_to_big5.map
        deleted:    utf8_to_euc_cn.map
        deleted:    utf8_to_euc_jis_2004.map
        deleted:    utf8_to_euc_jp.map
        deleted:    utf8_to_euc_kr.map
        deleted:    utf8_to_euc_tw.map
        deleted:    utf8_to_gb18030.map
        deleted:    utf8_to_gbk.map
        deleted:    utf8_to_iso8859_10.map
        deleted:    utf8_to_iso8859_13.map
        deleted:    utf8_to_iso8859_14.map
        deleted:    utf8_to_iso8859_15.map
        deleted:    utf8_to_iso8859_16.map
        deleted:    utf8_to_iso8859_2.map
        deleted:    utf8_to_iso8859_3.map
        deleted:    utf8_to_iso8859_4.map
        deleted:    utf8_to_iso8859_5.map
        deleted:    utf8_to_iso8859_6.map
        deleted:    utf8_to_iso8859_7.map
        deleted:    utf8_to_iso8859_8.map
        deleted:    utf8_to_iso8859_9.map
        deleted:    utf8_to_johab.map
        deleted:    utf8_to_koi8r.map
        deleted:    utf8_to_koi8u.map
        deleted:    utf8_to_shift_jis_2004.map
        deleted:    utf8_to_sjis.map
        deleted:    utf8_to_uhc.map
        deleted:    utf8_to_win1250.map
        deleted:    utf8_to_win1251.map
        deleted:    utf8_to_win1252.map
        deleted:    utf8_to_win1253.map
        deleted:    utf8_to_win1254.map
        deleted:    utf8_to_win1255.map
        deleted:    utf8_to_win1256.map
        deleted:    utf8_to_win1257.map
        deleted:    utf8_to_win1258.map
        deleted:    utf8_to_win866.map
        deleted:    utf8_to_win874.map
        deleted:    win1250_to_utf8.map
        deleted:    win1251_to_utf8.map
        deleted:    win1252_to_utf8.map
        deleted:    win1253_to_utf8.map
        deleted:    win1254_to_utf8.map
        deleted:    win1255_to_utf8.map
        deleted:    win1256_to_utf8.map
        deleted:    win1257_to_utf8.map
        deleted:    win1258_to_utf8.map
        deleted:    win866_to_utf8.map
        deleted:    win874_to_utf8.map

no changes added to commit (use "git add" and/or "git commit -a")

To undo the mess, I tried "make", which did rebuild all those files
but now I have

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        8859-10.TXT
        8859-13.TXT
        8859-14.TXT
        8859-15.TXT
        8859-16.TXT
        8859-2.TXT
        8859-3.TXT
        8859-4.TXT
        8859-5.TXT
        8859-6.TXT
        8859-7.TXT
        8859-8.TXT
        8859-9.TXT
        BIG5.TXT
        CNS11643.TXT
        CP1250.TXT
        CP1251.TXT
        CP1252.TXT
        CP1253.TXT
        CP1254.TXT
        CP1255.TXT
        CP1256.TXT
        CP1257.TXT
        CP1258.TXT
        CP866.TXT
        CP874.TXT
        CP932.TXT
        CP936.TXT
        CP950.TXT
        JIS0212.TXT
        JOHAB.TXT
        KOI8-R.TXT
        KOI8-U.TXT
        KSX1001.TXT
        windows-949-2000.xml

nothing added to commit but untracked files present (use "git add" to track)

So there doesn't seem to be any clean way to regenerate the fileset
present in git.  Maybe these targets aren't supposed to be invoked
here, but then why have a Makefile here at all?  Alternatively,
maybe we have files in git that shouldn't be there (very likely due
to the fact that this directory also lacks a .gitignore file).

            regards, tom lane

On Tue, 26 Sep 2023 at 05:58, Peter Eisentraut <peter@eisentraut.org> wrote:
> On 25.09.23 08:02, Japin Li wrote:
>> When I try to update the unicode mapping table through *.xml in
>> src/backend/utils/mb/Unicode/, it doesn't update the *.map files.
>> I find the make cannot go to this directory, what can I do to update
>> the mapping tables?
>
> This is done by "make update-unicode".

Thanks, it seems works.

-- 
Regrads,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.

On Tue, 26 Sep 2023 at 06:20, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Peter Eisentraut <peter@eisentraut.org> writes:
>> On 25.09.23 08:02, Japin Li wrote:
>>> When I try to update the unicode mapping table through *.xml in
>>> src/backend/utils/mb/Unicode/, it doesn't update the *.map files.
>>> I find the make cannot go to this directory, what can I do to update
>>> the mapping tables?
>
>> This is done by "make update-unicode".
>
> On a slightly related note, I noticed while preparing 3aff1d3fd
> that src/backend/utils/mb/Unicode/Makefile seems a little screwy.
>
> So there doesn't seem to be any clean way to regenerate the fileset
> present in git.  Maybe these targets aren't supposed to be invoked
> here, but then why have a Makefile here at all?  Alternatively,
> maybe we have files in git that shouldn't be there (very likely due
> to the fact that this directory also lacks a .gitignore file).
>

I find those files do not removed when using VPATH build.

-- 
Regrads,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.

On 26.09.23 00:20, Tom Lane wrote:
> On a slightly related note, I noticed while preparing 3aff1d3fd
> that src/backend/utils/mb/Unicode/Makefile seems a little screwy.
> If you go into that directory and type "make distclean", you'll
> find it removes some files that are in git:

Since this is only used during update-unicode, whose purpose is to 
overwrite files in git, this might not be totally wrong.

Maybe the target names distclean and maintainer-clean are inappropriate, 
since they suggest some standard semantics.  Maybe the current semantics 
are also not that useful, I'm not sure.  I guess among the things you'd 
want are "delete all intermediate downloaded files", which seemingly 
none of these do.