Обсуждение: Slow Query
We’re trying to show a list of active conversations. Each conversation (named a spool in the database) has multiple threads, kind of like Slack channels. And the messages are stored in each thread. We want to return the 30 most recent conversations with recency determined as the most recent message in any thread of the conversation you are a participant of (you may not be a participant of certain threads in a conversation so it’s important those don’t leak sensitive data).
We found that as the number of threads increases, the query slowed down dramatically. We think the issue has to do with the fact that there is no easy way to go from a thread you are a participant to its most recent message, however, it is possible the issue is elsewhere. We’ve provided the full query and a simplified query of where we think the issue is, along with the EXPLAIN ANALYZE BUFFERS result. We figure this is not exactly an uncommon use case, so it’s likely that we are overlooking the potential for some missing indices or a better way to write the query. We appreciate the help and any advice!
______
SELECT
handle.handle,
case.closed,
notification.read,
notification2.time,
message.message,
message.time,
message.author,
location.geo
FROM
spool
JOIN handle ON handle.id = spool.id
JOIN thread ON thread.spool = spool.id
JOIN message ON message.thread = thread.id
LEFT JOIN location ON location.id = spool.location
LEFT JOIN case ON case.id = spool.id
LEFT JOIN notification ON notification.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND
notification.id = (
SELECT
FROM
notification AS notification3
JOIN notification_thread ON notification_thread.id = notification3.id
JOIN thread AS thread2 ON thread2.id = notification_thread.thread
WHERE
thread2.spool = spool.id
AND notification3.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND notification3.time <= '2020-09-30 16:32:38.054558'
ORDER BY
notification3.time DESC
LIMIT 1
)
LEFT JOIN notification AS notification2 ON notification2.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND notification2.id = (
SELECT
FROM
notification AS notification3
JOIN notification_thread ON notification_thread.id = notification3.id
JOIN thread AS thread2 ON thread2.id = notification_thread.thread
WHERE
thread2.spool = spool.id
AND notification3.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND notification3.time > '2020-09-30 16:32:38.054558'
ORDER BY
notification3.time DESC
LIMIT 1
)
WHERE
message.time = (
SELECT
MAX ( message2.time )
FROM
message AS message2
JOIN thread AS thread2 ON thread2.id = message2.thread
JOIN participant ON participant.thread = thread2.id
JOIN identity ON identity.id = participant.identity
LEFT JOIN relation ON relation.to = identity.id
AND relation.from = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND relation.manages = TRUE
WHERE
NOT message2.draft
AND ( identity.id = 'b16690e4-a3c5-4868-945e-c2458c27a525' OR NOT relation.to IS NULL )
AND thread2.spool = spool.id
LIMIT 1
)
AND notification.id IS NOT NULL
ORDER BY
message.time DESC
LIMIT 31;
Is there no index on thread.spool? What about notification.user? How about message.time (without thread as a leading column). Those would all seem very significant. Your row counts are very low to have a query perform so badly. Work_mem could probably be increased above 4MB, but it isn't hurting this query in particular.My primary concern is that the query is rather chaotic at a glance. It would be great to re-write and remove the unneeded keywords, double quotes, totally worthless parentheses, etc. Something like the below may help you see the crux of the query and what could be done and understand how many rows might be coming out of those subqueries. I re-ordered some joins and there might be syntax errors, but give it a shot once you've added the indexes suggested above.SELECT
handle.handle,
case.closed,
notification.read,
notification2.time,
message.message,
message.time,
message.author,
location.geo
FROM
spool
JOIN handle ON handle.id = spool.id
JOIN thread ON thread.spool = spool.id
JOIN message ON message.thread = thread.id
LEFT JOIN location ON location.id = spool.location
LEFT JOIN case ON case.id = spool.id
LEFT JOIN notification ON notification.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND
notification.id = (
SELECT
FROM
notification AS notification3
JOIN notification_thread ON notification_thread.id = notification3.id
JOIN thread AS thread2 ON thread2.id = notification_thread.thread
WHERE
thread2.spool = spool.id
AND notification3.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND notification3.time <= '2020-09-30 16:32:38.054558'
ORDER BY
notification3.time DESC
LIMIT 1
)
LEFT JOIN notification AS notification2 ON notification2.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND notification2.id = (
SELECT
FROM
notification AS notification3
JOIN notification_thread ON notification_thread.id = notification3.id
JOIN thread AS thread2 ON thread2.id = notification_thread.thread
WHERE
thread2.spool = spool.id
AND notification3.user = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND notification3.time > '2020-09-30 16:32:38.054558'
ORDER BY
notification3.time DESC
LIMIT 1
)
WHERE
message.time = (
SELECT
MAX ( message2.time )
FROM
message AS message2
JOIN thread AS thread2 ON thread2.id = message2.thread
JOIN participant ON participant.thread = thread2.id
JOIN identity ON identity.id = participant.identity
LEFT JOIN relation ON relation.to = identity.id
AND relation.from = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND relation.manages = TRUE
WHERE
NOT message2.draft
AND ( identity.id = 'b16690e4-a3c5-4868-945e-c2458c27a525' OR NOT relation.to IS NULL )
AND thread2.spool = spool.id
LIMIT 1
)
AND notification.id IS NOT NULL
ORDER BY
message.time DESC
LIMIT 31;
Вложения
WHERE message.time = greatest( sub1.time, sub2.time )
/* sub1.time */
MAX ( message2.time )
FROM
message AS message2
JOIN thread AS thread2 ON thread2.id = message2.thread
JOIN participant ON participant.thread = thread2.id
WHERE
NOT message2.draft
AND participant.identity = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND thread2.spool = spool.id
)
/* sub2.time */
select
MAX ( message2.time )
FROM
message AS message2
JOIN thread AS thread2 ON thread2.id = message2.thread
JOIN participant ON participant.thread = thread2.id
JOIN relation ON relation.to = participant.identity
AND relation.from = 'b16690e4-a3c5-4868-945e-c2458c27a525'
AND relation.manages = TRUE
WHERE
NOT message2.draft
AND thread2.spool = spool.id