Обсуждение: Interested in GSoC projects on pgAdmin 4

Hi

[Moving pgsql-hackers to BCC to prevent further cross-posting]

On Sat, Mar 2, 2019 at 12:36 AM Haoran Yu <haleyyew@gmail.com> wrote:

Dear PostgreSQL community:

I am a MSc student in computer science working on data management research, and I will likely graduate this summer. I also was a participant of GSoC in 2017 working with the NRNB organization on data standards conversion.

Cool.

 

I have been a user of pgAdmin 4 briefly, and I am interested in learning more about the project and contribute to it. The projects web page lists 3 potential projects, but I don't know which one I'm suitable for. Are there any suggestions on how to get started on exploring more about the Query Tool in pgAdmin? For example, use cases with some sample data would be nice! I also checked out the pgadmin4 repository from git, and I'll start exploring the code shortly. 

The choice of which project to work on is entirely down to your own personal interests. You can even propose something else if you like, though the listed projects are ones that are likely to be accepted as they're known to be valuable.

The first project (Query Tool Graphing) has a simple use case of allowing any user to quickly render a graph of their data. More specific use cases can be discussed as part of the project, but quite simply the idea is to allow users a quick and easy way to visualise their data. It would probably help to install PostGIS in a database, and then load the test data we used to play with it and see how that works (see https://www.postgresql.org/message-id/CAA7HE_cU7bmQv1kdPB3hiKYGJLaOVVft_XxqcD6ueJpAGfqykQ%40mail.gmail.com - specifically, the Google Drive link). The GIS viewer is similar to what I have in mind for this feature, except that instead of drawing maps, we'd draw different types of graphs.

The second project is about supporting bytea in the Query Tool. Right now bytea data isn't rendered when you run a query - we show a placeholder instead. The use case here is for users that store media files in bytea columns; we want to be able to automatically detect different file types and allow them to be viewed (or listened to) in the tool. When running in Edit more, the user should be able to add or replace data in a row by uploading from the browser. I don't have any sample data for this.

The final project listed is a long-term design goal of pgAdmin 4 (and probably the hardest project). In pgAdmin 3 we had separate Query Tool and View/Edit data tools. In pgAdmin 4, we made them into the same tool, but running in two separate modes. The use case here is to prevent the need for the user to choose what mode to open the tool in (Query Tool vs. View/Edit Data), and to automatically detect whether any query would produce an updateable resultset. This would allow the tool to offer all features at all times, and simple enable/disable in-place editing of the query results if there's no way to automatically generate an update/insert/delete statement. This one is potentially hard as it will likely require some amount of parsing of the query string to make that determination. You can simply play with any test data to get a feel for this one.

Hope this helps.

Regards, Dave.

 

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Hi

On Mon, Apr 1, 2019 at 3:12 AM Haoran Yu <haleyyew@gmail.com> wrote:

Dear PostgreSQL community,

I have submitted a proposal for the project pgAdmin 4 bytea support. The project discusses storing media content (images, audio, video) as bytea. However, I have a quick question. What does bytea data look like typically when storing media content? What I had in mind is, media contents that uses MIME type, which are rendered as part of HTML. For example, the following is rendered as a red dot:

'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==’

This string is decoded to bytea, and I stored it in a bytea column.

What are some other examples of using bytea to store media content, not necessarily using the MIME type? Is there a way to detect the type of these media (audio, image) stored in bytea?

When I have stored small media items in bytea columns in the past, I just stored the data. I vaguely recall I did store the mime type in another column, but that may not be the case in all scenarios (e.g. when a system is designed to store only PNGs). I think you should assume it's raw data only, and try to determine the file type by examining the data;

e.g 

PNG files have an 8 byte signature: http://www.libpng.org/pub/png/spec/1.2/PNG-Structure.html

MPEG files have identifying information in the frame header that you may be able to use: http://mpgedit.org/mpgedit/mpeg_format/MP3Format.html

JPEG images have identifying markers: https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format

etc.

 

Another question I had is, I read that there are performance-related issues for storing media in bytea. Are there practical ways to store bytea data that does not face performance-related issues? For example, storing large media content using multiple bytea parts, and reassembling them together once retrieved from the database?

Not that I'm aware of. For larger objects, most people store them externally (which of course loses ACID properties). There are certainly applications for storing smaller objects directly in the database though - and some folks have done work in the past with index types and operators/functions for finding and comparing images for example, so there are also benefits other than ACID to storing data in this way.

BTW; for pgAdmin related GSoC questions, you'd do better to ask on pgadmin-hackers@postgresql.org.

 

--

Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company