ETL tool for json and postgresql

Поиск
Список
Период
Сортировка
От Jayson Hreczuck
Тема ETL tool for json and postgresql
Дата
Msg-id CAHg7=h48Bag0iJ=U3zsRO1bnnTO5v4=FTTgh--wY2d73cFE6HQ@mail.gmail.com
обсуждение исходный текст
Список pgsql-admin
Good Morning,

This is probably not the best place to ask this, but I'd figure I would give it a shot.

We are slowly moving over to a hybrid nosql model, and many of our large sets of data are being stored in json format, specifically using the jsonb data type.

The issue is that the etl tool that I've developed to parse this data into a datawarehouse is very manual and pretty cumbersome - essentially using the jsonb postgresql query commands and cron or pgagent.  It's unable to effectively traverse the element levels, and really isn't very good at handling dynamic changes.

I am wondering what people use for a tool to do this work.  We are a smaller shop and do not need a large solution.  Would prefer something that integrates well with postgresql and can be easily modified to change data mappings, and possibly support additional filetypes outside of json.

The end result would be a table or set of tables that can be ingested by a visualization tool like Tableau or PowerBI.

Thanks!
-Jayson

--


Jayson Hreczuck | Principal Database Administrator

C: +1 617.571.9861130 Turner St, Ste 530, Waltham, MA 02453

В списке pgsql-admin по дате отправления:

Предыдущее
От: Priancka Chatz
Дата:
Сообщение: Re: auto vacuum restart
Следующее
От: Laurenz Albe
Дата:
Сообщение: Re: PostgreSQL community vs PostgreSQL EDB comparison and Difference document needed!!!!