PostgreSQL Source Code  git master
async.c
Go to the documentation of this file.
1 /*-------------------------------------------------------------------------
2  *
3  * async.c
4  * Asynchronous notification: NOTIFY, LISTEN, UNLISTEN
5  *
6  * Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
7  * Portions Copyright (c) 1994, Regents of the University of California
8  *
9  * IDENTIFICATION
10  * src/backend/commands/async.c
11  *
12  *-------------------------------------------------------------------------
13  */
14 
15 /*-------------------------------------------------------------------------
16  * Async Notification Model as of 9.0:
17  *
18  * 1. Multiple backends on same machine. Multiple backends listening on
19  * several channels. (Channels are also called "conditions" in other
20  * parts of the code.)
21  *
22  * 2. There is one central queue in disk-based storage (directory pg_notify/),
23  * with actively-used pages mapped into shared memory by the slru.c module.
24  * All notification messages are placed in the queue and later read out
25  * by listening backends.
26  *
27  * There is no central knowledge of which backend listens on which channel;
28  * every backend has its own list of interesting channels.
29  *
30  * Although there is only one queue, notifications are treated as being
31  * database-local; this is done by including the sender's database OID
32  * in each notification message. Listening backends ignore messages
33  * that don't match their database OID. This is important because it
34  * ensures senders and receivers have the same database encoding and won't
35  * misinterpret non-ASCII text in the channel name or payload string.
36  *
37  * Since notifications are not expected to survive database crashes,
38  * we can simply clean out the pg_notify data at any reboot, and there
39  * is no need for WAL support or fsync'ing.
40  *
41  * 3. Every backend that is listening on at least one channel registers by
42  * entering its PID into the array in AsyncQueueControl. It then scans all
43  * incoming notifications in the central queue and first compares the
44  * database OID of the notification with its own database OID and then
45  * compares the notified channel with the list of channels that it listens
46  * to. In case there is a match it delivers the notification event to its
47  * frontend. Non-matching events are simply skipped.
48  *
49  * 4. The NOTIFY statement (routine Async_Notify) stores the notification in
50  * a backend-local list which will not be processed until transaction end.
51  *
52  * Duplicate notifications from the same transaction are sent out as one
53  * notification only. This is done to save work when for example a trigger
54  * on a 2 million row table fires a notification for each row that has been
55  * changed. If the application needs to receive every single notification
56  * that has been sent, it can easily add some unique string into the extra
57  * payload parameter.
58  *
59  * When the transaction is ready to commit, PreCommit_Notify() adds the
60  * pending notifications to the head of the queue. The head pointer of the
61  * queue always points to the next free position and a position is just a
62  * page number and the offset in that page. This is done before marking the
63  * transaction as committed in clog. If we run into problems writing the
64  * notifications, we can still call elog(ERROR, ...) and the transaction
65  * will roll back.
66  *
67  * Once we have put all of the notifications into the queue, we return to
68  * CommitTransaction() which will then do the actual transaction commit.
69  *
70  * After commit we are called another time (AtCommit_Notify()). Here we
71  * make the actual updates to the effective listen state (listenChannels).
72  *
73  * Finally, after we are out of the transaction altogether, we check if
74  * we need to signal listening backends. In SignalBackends() we scan the
75  * list of listening backends and send a PROCSIG_NOTIFY_INTERRUPT signal
76  * to every listening backend (we don't know which backend is listening on
77  * which channel so we must signal them all). We can exclude backends that
78  * are already up to date, though. We don't bother with a self-signal
79  * either, but just process the queue directly.
80  *
81  * 5. Upon receipt of a PROCSIG_NOTIFY_INTERRUPT signal, the signal handler
82  * sets the process's latch, which triggers the event to be processed
83  * immediately if this backend is idle (i.e., it is waiting for a frontend
84  * command and is not within a transaction block. C.f.
85  * ProcessClientReadInterrupt()). Otherwise the handler may only set a
86  * flag, which will cause the processing to occur just before we next go
87  * idle.
88  *
89  * Inbound-notify processing consists of reading all of the notifications
90  * that have arrived since scanning last time. We read every notification
91  * until we reach either a notification from an uncommitted transaction or
92  * the head pointer's position. Then we check if we were the laziest
93  * backend: if our pointer is set to the same position as the global tail
94  * pointer is set, then we move the global tail pointer ahead to where the
95  * second-laziest backend is (in general, we take the MIN of the current
96  * head position and all active backends' new tail pointers). Whenever we
97  * move the global tail pointer we also truncate now-unused pages (i.e.,
98  * delete files in pg_notify/ that are no longer used).
99  *
100  * An application that listens on the same channel it notifies will get
101  * NOTIFY messages for its own NOTIFYs. These can be ignored, if not useful,
102  * by comparing be_pid in the NOTIFY message to the application's own backend's
103  * PID. (As of FE/BE protocol 2.0, the backend's PID is provided to the
104  * frontend during startup.) The above design guarantees that notifies from
105  * other backends will never be missed by ignoring self-notifies.
106  *
107  * The amount of shared memory used for notify management (NUM_ASYNC_BUFFERS)
108  * can be varied without affecting anything but performance. The maximum
109  * amount of notification data that can be queued at one time is determined
110  * by slru.c's wraparound limit; see QUEUE_MAX_PAGE below.
111  *-------------------------------------------------------------------------
112  */
113 
114 #include "postgres.h"
115 
116 #include <limits.h>
117 #include <unistd.h>
118 #include <signal.h>
119 
120 #include "access/parallel.h"
121 #include "access/slru.h"
122 #include "access/transam.h"
123 #include "access/xact.h"
124 #include "catalog/pg_database.h"
125 #include "commands/async.h"
126 #include "funcapi.h"
127 #include "libpq/libpq.h"
128 #include "libpq/pqformat.h"
129 #include "miscadmin.h"
130 #include "storage/ipc.h"
131 #include "storage/lmgr.h"
132 #include "storage/proc.h"
133 #include "storage/procarray.h"
134 #include "storage/procsignal.h"
135 #include "storage/sinval.h"
136 #include "tcop/tcopprot.h"
137 #include "utils/builtins.h"
138 #include "utils/hashutils.h"
139 #include "utils/memutils.h"
140 #include "utils/ps_status.h"
141 #include "utils/snapmgr.h"
142 #include "utils/timestamp.h"
143 
144 
145 /*
146  * Maximum size of a NOTIFY payload, including terminating NULL. This
147  * must be kept small enough so that a notification message fits on one
148  * SLRU page. The magic fudge factor here is noncritical as long as it's
149  * more than AsyncQueueEntryEmptySize --- we make it significantly bigger
150  * than that, so changes in that data structure won't affect user-visible
151  * restrictions.
152  */
153 #define NOTIFY_PAYLOAD_MAX_LENGTH (BLCKSZ - NAMEDATALEN - 128)
154 
155 /*
156  * Struct representing an entry in the global notify queue
157  *
158  * This struct declaration has the maximal length, but in a real queue entry
159  * the data area is only big enough for the actual channel and payload strings
160  * (each null-terminated). AsyncQueueEntryEmptySize is the minimum possible
161  * entry size, if both channel and payload strings are empty (but note it
162  * doesn't include alignment padding).
163  *
164  * The "length" field should always be rounded up to the next QUEUEALIGN
165  * multiple so that all fields are properly aligned.
166  */
167 typedef struct AsyncQueueEntry
168 {
169  int length; /* total allocated length of entry */
170  Oid dboid; /* sender's database OID */
171  TransactionId xid; /* sender's XID */
172  int32 srcPid; /* sender's PID */
175 
176 /* Currently, no field of AsyncQueueEntry requires more than int alignment */
177 #define QUEUEALIGN(len) INTALIGN(len)
178 
179 #define AsyncQueueEntryEmptySize (offsetof(AsyncQueueEntry, data) + 2)
180 
181 /*
182  * Struct describing a queue position, and assorted macros for working with it
183  */
184 typedef struct QueuePosition
185 {
186  int page; /* SLRU page number */
187  int offset; /* byte offset within page */
188 } QueuePosition;
189 
190 #define QUEUE_POS_PAGE(x) ((x).page)
191 #define QUEUE_POS_OFFSET(x) ((x).offset)
192 
193 #define SET_QUEUE_POS(x,y,z) \
194  do { \
195  (x).page = (y); \
196  (x).offset = (z); \
197  } while (0)
198 
199 #define QUEUE_POS_EQUAL(x,y) \
200  ((x).page == (y).page && (x).offset == (y).offset)
201 
202 /* choose logically smaller QueuePosition */
203 #define QUEUE_POS_MIN(x,y) \
204  (asyncQueuePagePrecedes((x).page, (y).page) ? (x) : \
205  (x).page != (y).page ? (y) : \
206  (x).offset < (y).offset ? (x) : (y))
207 
208 /* choose logically larger QueuePosition */
209 #define QUEUE_POS_MAX(x,y) \
210  (asyncQueuePagePrecedes((x).page, (y).page) ? (y) : \
211  (x).page != (y).page ? (x) : \
212  (x).offset > (y).offset ? (x) : (y))
213 
214 /*
215  * Struct describing a listening backend's status
216  */
217 typedef struct QueueBackendStatus
218 {
219  int32 pid; /* either a PID or InvalidPid */
220  Oid dboid; /* backend's database OID, or InvalidOid */
221  QueuePosition pos; /* backend has read queue up to here */
223 
224 /*
225  * Shared memory state for LISTEN/NOTIFY (excluding its SLRU stuff)
226  *
227  * The AsyncQueueControl structure is protected by the AsyncQueueLock.
228  *
229  * When holding the lock in SHARED mode, backends may only inspect their own
230  * entries as well as the head and tail pointers. Consequently we can allow a
231  * backend to update its own record while holding only SHARED lock (since no
232  * other backend will inspect it).
233  *
234  * When holding the lock in EXCLUSIVE mode, backends can inspect the entries
235  * of other backends and also change the head and tail pointers.
236  *
237  * AsyncCtlLock is used as the control lock for the pg_notify SLRU buffers.
238  * In order to avoid deadlocks, whenever we need both locks, we always first
239  * get AsyncQueueLock and then AsyncCtlLock.
240  *
241  * Each backend uses the backend[] array entry with index equal to its
242  * BackendId (which can range from 1 to MaxBackends). We rely on this to make
243  * SendProcSignal fast.
244  */
245 typedef struct AsyncQueueControl
246 {
247  QueuePosition head; /* head points to the next free location */
248  QueuePosition tail; /* the global tail is equivalent to the pos of
249  * the "slowest" backend */
250  TimestampTz lastQueueFillWarn; /* time of last queue-full msg */
251  QueueBackendStatus backend[FLEXIBLE_ARRAY_MEMBER];
252  /* backend[0] is not used; used entries are from [1] to [MaxBackends] */
254 
256 
257 #define QUEUE_HEAD (asyncQueueControl->head)
258 #define QUEUE_TAIL (asyncQueueControl->tail)
259 #define QUEUE_BACKEND_PID(i) (asyncQueueControl->backend[i].pid)
260 #define QUEUE_BACKEND_DBOID(i) (asyncQueueControl->backend[i].dboid)
261 #define QUEUE_BACKEND_POS(i) (asyncQueueControl->backend[i].pos)
262 
263 /*
264  * The SLRU buffer area through which we access the notification queue
265  */
267 
268 #define AsyncCtl (&AsyncCtlData)
269 #define QUEUE_PAGESIZE BLCKSZ
270 #define QUEUE_FULL_WARN_INTERVAL 5000 /* warn at most once every 5s */
271 
272 /*
273  * slru.c currently assumes that all filenames are four characters of hex
274  * digits. That means that we can use segments 0000 through FFFF.
275  * Each segment contains SLRU_PAGES_PER_SEGMENT pages which gives us
276  * the pages from 0 to SLRU_PAGES_PER_SEGMENT * 0x10000 - 1.
277  *
278  * It's of course possible to enhance slru.c, but this gives us so much
279  * space already that it doesn't seem worth the trouble.
280  *
281  * The most data we can have in the queue at a time is QUEUE_MAX_PAGE/2
282  * pages, because more than that would confuse slru.c into thinking there
283  * was a wraparound condition. With the default BLCKSZ this means there
284  * can be up to 8GB of queued-and-not-read data.
285  *
286  * Note: it's possible to redefine QUEUE_MAX_PAGE with a smaller multiple of
287  * SLRU_PAGES_PER_SEGMENT, for easier testing of queue-full behaviour.
288  */
289 #define QUEUE_MAX_PAGE (SLRU_PAGES_PER_SEGMENT * 0x10000 - 1)
290 
291 /*
292  * listenChannels identifies the channels we are actually listening to
293  * (ie, have committed a LISTEN on). It is a simple list of channel names,
294  * allocated in TopMemoryContext.
295  */
296 static List *listenChannels = NIL; /* list of C strings */
297 
298 /*
299  * State for pending LISTEN/UNLISTEN actions consists of an ordered list of
300  * all actions requested in the current transaction. As explained above,
301  * we don't actually change listenChannels until we reach transaction commit.
302  *
303  * The list is kept in CurTransactionContext. In subtransactions, each
304  * subtransaction has its own list in its own CurTransactionContext, but
305  * successful subtransactions attach their lists to their parent's list.
306  * Failed subtransactions simply discard their lists.
307  */
308 typedef enum
309 {
314 
315 typedef struct
316 {
318  char channel[FLEXIBLE_ARRAY_MEMBER]; /* nul-terminated string */
319 } ListenAction;
320 
321 static List *pendingActions = NIL; /* list of ListenAction */
322 
323 static List *upperPendingActions = NIL; /* list of upper-xact lists */
324 
325 /*
326  * State for outbound notifies consists of a list of all channels+payloads
327  * NOTIFYed in the current transaction. We do not actually perform a NOTIFY
328  * until and unless the transaction commits. pendingNotifies is NULL if no
329  * NOTIFYs have been done in the current (sub) transaction.
330  *
331  * We discard duplicate notify events issued in the same transaction.
332  * Hence, in addition to the list proper (which we need to track the order
333  * of the events, since we guarantee to deliver them in order), we build a
334  * hash table which we can probe to detect duplicates. Since building the
335  * hash table is somewhat expensive, we do so only once we have at least
336  * MIN_HASHABLE_NOTIFIES events queued in the current (sub) transaction;
337  * before that we just scan the events linearly.
338  *
339  * The list is kept in CurTransactionContext. In subtransactions, each
340  * subtransaction has its own list in its own CurTransactionContext, but
341  * successful subtransactions add their entries to their parent's list.
342  * Failed subtransactions simply discard their lists. Since these lists
343  * are independent, there may be notify events in a subtransaction's list
344  * that duplicate events in some ancestor (sub) transaction; we get rid of
345  * the dups when merging the subtransaction's list into its parent's.
346  *
347  * Note: the action and notify lists do not interact within a transaction.
348  * In particular, if a transaction does NOTIFY and then LISTEN on the same
349  * condition name, it will get a self-notify at commit. This is a bit odd
350  * but is consistent with our historical behavior.
351  */
352 typedef struct Notification
353 {
354  uint16 channel_len; /* length of channel-name string */
355  uint16 payload_len; /* length of payload string */
356  /* null-terminated channel name, then null-terminated payload follow */
357  char data[FLEXIBLE_ARRAY_MEMBER];
358 } Notification;
359 
360 typedef struct NotificationList
361 {
362  List *events; /* list of Notification structs */
363  HTAB *hashtab; /* hash of NotificationHash structs, or NULL */
365 
366 #define MIN_HASHABLE_NOTIFIES 16 /* threshold to build hashtab */
367 
368 typedef struct NotificationHash
369 {
370  Notification *event; /* => the actual Notification struct */
372 
373 static NotificationList *pendingNotifies = NULL; /* current list, if any */
374 
375 static List *upperPendingNotifies = NIL; /* list of upper-xact lists */
376 
377 /*
378  * Inbound notifications are initially processed by HandleNotifyInterrupt(),
379  * called from inside a signal handler. That just sets the
380  * notifyInterruptPending flag and sets the process
381  * latch. ProcessNotifyInterrupt() will then be called whenever it's safe to
382  * actually deal with the interrupt.
383  */
384 volatile sig_atomic_t notifyInterruptPending = false;
385 
386 /* True if we've registered an on_shmem_exit cleanup */
387 static bool unlistenExitRegistered = false;
388 
389 /* True if we're currently registered as a listener in asyncQueueControl */
390 static bool amRegisteredListener = false;
391 
392 /* has this backend sent notifications in the current transaction? */
393 static bool backendHasSentNotifications = false;
394 
395 /* GUC parameter */
396 bool Trace_notify = false;
397 
398 /* local function prototypes */
399 static bool asyncQueuePagePrecedes(int p, int q);
400 static void queue_listen(ListenActionKind action, const char *channel);
401 static void Async_UnlistenOnExit(int code, Datum arg);
402 static void Exec_ListenPreCommit(void);
403 static void Exec_ListenCommit(const char *channel);
404 static void Exec_UnlistenCommit(const char *channel);
405 static void Exec_UnlistenAllCommit(void);
406 static bool IsListeningOn(const char *channel);
407 static void asyncQueueUnregister(void);
408 static bool asyncQueueIsFull(void);
409 static bool asyncQueueAdvance(volatile QueuePosition *position, int entryLength);
411 static ListCell *asyncQueueAddEntries(ListCell *nextNotify);
412 static double asyncQueueUsage(void);
413 static void asyncQueueFillWarning(void);
414 static bool SignalBackends(void);
415 static void asyncQueueReadAllNotifications(void);
416 static bool asyncQueueProcessPageEntries(volatile QueuePosition *current,
417  QueuePosition stop,
418  char *page_buffer,
419  Snapshot snapshot);
420 static void asyncQueueAdvanceTail(void);
421 static void ProcessIncomingNotify(void);
422 static bool AsyncExistsPendingNotify(Notification *n);
424 static uint32 notification_hash(const void *key, Size keysize);
425 static int notification_match(const void *key1, const void *key2, Size keysize);
426 static void ClearPendingActionsAndNotifies(void);
427 
428 /*
429  * We will work on the page range of 0..QUEUE_MAX_PAGE.
430  */
431 static bool
433 {
434  int diff;
435 
436  /*
437  * We have to compare modulo (QUEUE_MAX_PAGE+1)/2. Both inputs should be
438  * in the range 0..QUEUE_MAX_PAGE.
439  */
440  Assert(p >= 0 && p <= QUEUE_MAX_PAGE);
441  Assert(q >= 0 && q <= QUEUE_MAX_PAGE);
442 
443  diff = p - q;
444  if (diff >= ((QUEUE_MAX_PAGE + 1) / 2))
445  diff -= QUEUE_MAX_PAGE + 1;
446  else if (diff < -((QUEUE_MAX_PAGE + 1) / 2))
447  diff += QUEUE_MAX_PAGE + 1;
448  return diff < 0;
449 }
450 
451 /*
452  * Report space needed for our shared memory area
453  */
454 Size
456 {
457  Size size;
458 
459  /* This had better match AsyncShmemInit */
460  size = mul_size(MaxBackends + 1, sizeof(QueueBackendStatus));
461  size = add_size(size, offsetof(AsyncQueueControl, backend));
462 
464 
465  return size;
466 }
467 
468 /*
469  * Initialize our shared memory area
470  */
471 void
473 {
474  bool found;
475  int slotno;
476  Size size;
477 
478  /*
479  * Create or attach to the AsyncQueueControl structure.
480  *
481  * The used entries in the backend[] array run from 1 to MaxBackends; the
482  * zero'th entry is unused but must be allocated.
483  */
484  size = mul_size(MaxBackends + 1, sizeof(QueueBackendStatus));
485  size = add_size(size, offsetof(AsyncQueueControl, backend));
486 
487  asyncQueueControl = (AsyncQueueControl *)
488  ShmemInitStruct("Async Queue Control", size, &found);
489 
490  if (!found)
491  {
492  /* First time through, so initialize it */
493  int i;
494 
495  SET_QUEUE_POS(QUEUE_HEAD, 0, 0);
496  SET_QUEUE_POS(QUEUE_TAIL, 0, 0);
497  asyncQueueControl->lastQueueFillWarn = 0;
498  /* zero'th entry won't be used, but let's initialize it anyway */
499  for (i = 0; i <= MaxBackends; i++)
500  {
504  }
505  }
506 
507  /*
508  * Set up SLRU management of the pg_notify data.
509  */
510  AsyncCtl->PagePrecedes = asyncQueuePagePrecedes;
512  AsyncCtlLock, "pg_notify", LWTRANCHE_ASYNC_BUFFERS);
513  /* Override default assumption that writes should be fsync'd */
514  AsyncCtl->do_fsync = false;
515 
516  if (!found)
517  {
518  /*
519  * During start or reboot, clean out the pg_notify directory.
520  */
522 
523  /* Now initialize page zero to empty */
524  LWLockAcquire(AsyncCtlLock, LW_EXCLUSIVE);
526  /* This write is just to verify that pg_notify/ is writable */
527  SimpleLruWritePage(AsyncCtl, slotno);
528  LWLockRelease(AsyncCtlLock);
529  }
530 }
531 
532 
533 /*
534  * pg_notify -
535  * SQL function to send a notification event
536  */
537 Datum
539 {
540  const char *channel;
541  const char *payload;
542 
543  if (PG_ARGISNULL(0))
544  channel = "";
545  else
546  channel = text_to_cstring(PG_GETARG_TEXT_PP(0));
547 
548  if (PG_ARGISNULL(1))
549  payload = "";
550  else
551  payload = text_to_cstring(PG_GETARG_TEXT_PP(1));
552 
553  /* For NOTIFY as a statement, this is checked in ProcessUtility */
555 
556  Async_Notify(channel, payload);
557 
558  PG_RETURN_VOID();
559 }
560 
561 
562 /*
563  * Async_Notify
564  *
565  * This is executed by the SQL notify command.
566  *
567  * Adds the message to the list of pending notifies.
568  * Actual notification happens during transaction commit.
569  * ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
570  */
571 void
572 Async_Notify(const char *channel, const char *payload)
573 {
574  size_t channel_len;
575  size_t payload_len;
576  Notification *n;
577  MemoryContext oldcontext;
578 
579  if (IsParallelWorker())
580  elog(ERROR, "cannot send notifications from a parallel worker");
581 
582  if (Trace_notify)
583  elog(DEBUG1, "Async_Notify(%s)", channel);
584 
585  channel_len = channel ? strlen(channel) : 0;
586  payload_len = payload ? strlen(payload) : 0;
587 
588  /* a channel name must be specified */
589  if (channel_len == 0)
590  ereport(ERROR,
591  (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
592  errmsg("channel name cannot be empty")));
593 
594  /* enforce length limits */
595  if (channel_len >= NAMEDATALEN)
596  ereport(ERROR,
597  (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
598  errmsg("channel name too long")));
599 
600  if (payload_len >= NOTIFY_PAYLOAD_MAX_LENGTH)
601  ereport(ERROR,
602  (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
603  errmsg("payload string too long")));
604 
605  /*
606  * We must construct the Notification entry, even if we end up not using
607  * it, in order to compare it cheaply to existing list entries.
608  *
609  * The notification list needs to live until end of transaction, so store
610  * it in the transaction context.
611  */
613 
615  channel_len + payload_len + 2);
616  n->channel_len = channel_len;
617  n->payload_len = payload_len;
618  strcpy(n->data, channel);
619  if (payload)
620  strcpy(n->data + channel_len + 1, payload);
621  else
622  n->data[channel_len + 1] = '\0';
623 
624  /* Now check for duplicates */
626  {
627  /* It's a dup, so forget it */
628  pfree(n);
629  MemoryContextSwitchTo(oldcontext);
630  return;
631  }
632 
633  if (pendingNotifies == NULL)
634  {
635  /* First notify event in current (sub)xact */
636  pendingNotifies = (NotificationList *) palloc(sizeof(NotificationList));
637  pendingNotifies->events = list_make1(n);
638  /* We certainly don't need a hashtable yet */
639  pendingNotifies->hashtab = NULL;
640  }
641  else
642  {
643  /* Append more events to existing list */
645  }
646 
647  MemoryContextSwitchTo(oldcontext);
648 }
649 
650 /*
651  * queue_listen
652  * Common code for listen, unlisten, unlisten all commands.
653  *
654  * Adds the request to the list of pending actions.
655  * Actual update of the listenChannels list happens during transaction
656  * commit.
657  */
658 static void
659 queue_listen(ListenActionKind action, const char *channel)
660 {
661  MemoryContext oldcontext;
662  ListenAction *actrec;
663 
664  /*
665  * Unlike Async_Notify, we don't try to collapse out duplicates. It would
666  * be too complicated to ensure we get the right interactions of
667  * conflicting LISTEN/UNLISTEN/UNLISTEN_ALL, and it's unlikely that there
668  * would be any performance benefit anyway in sane applications.
669  */
671 
672  /* space for terminating null is included in sizeof(ListenAction) */
673  actrec = (ListenAction *) palloc(offsetof(ListenAction, channel) +
674  strlen(channel) + 1);
675  actrec->action = action;
676  strcpy(actrec->channel, channel);
677 
678  pendingActions = lappend(pendingActions, actrec);
679 
680  MemoryContextSwitchTo(oldcontext);
681 }
682 
683 /*
684  * Async_Listen
685  *
686  * This is executed by the SQL listen command.
687  */
688 void
689 Async_Listen(const char *channel)
690 {
691  if (Trace_notify)
692  elog(DEBUG1, "Async_Listen(%s,%d)", channel, MyProcPid);
693 
694  queue_listen(LISTEN_LISTEN, channel);
695 }
696 
697 /*
698  * Async_Unlisten
699  *
700  * This is executed by the SQL unlisten command.
701  */
702 void
703 Async_Unlisten(const char *channel)
704 {
705  if (Trace_notify)
706  elog(DEBUG1, "Async_Unlisten(%s,%d)", channel, MyProcPid);
707 
708  /* If we couldn't possibly be listening, no need to queue anything */
709  if (pendingActions == NIL && !unlistenExitRegistered)
710  return;
711 
712  queue_listen(LISTEN_UNLISTEN, channel);
713 }
714 
715 /*
716  * Async_UnlistenAll
717  *
718  * This is invoked by UNLISTEN * command, and also at backend exit.
719  */
720 void
722 {
723  if (Trace_notify)
724  elog(DEBUG1, "Async_UnlistenAll(%d)", MyProcPid);
725 
726  /* If we couldn't possibly be listening, no need to queue anything */
727  if (pendingActions == NIL && !unlistenExitRegistered)
728  return;
729 
731 }
732 
733 /*
734  * SQL function: return a set of the channel names this backend is actively
735  * listening to.
736  *
737  * Note: this coding relies on the fact that the listenChannels list cannot
738  * change within a transaction.
739  */
740 Datum
742 {
743  FuncCallContext *funcctx;
744 
745  /* stuff done only on the first call of the function */
746  if (SRF_IS_FIRSTCALL())
747  {
748  /* create a function context for cross-call persistence */
749  funcctx = SRF_FIRSTCALL_INIT();
750  }
751 
752  /* stuff done on every call of the function */
753  funcctx = SRF_PERCALL_SETUP();
754 
755  if (funcctx->call_cntr < list_length(listenChannels))
756  {
757  char *channel = (char *) list_nth(listenChannels,
758  funcctx->call_cntr);
759 
760  SRF_RETURN_NEXT(funcctx, CStringGetTextDatum(channel));
761  }
762 
763  SRF_RETURN_DONE(funcctx);
764 }
765 
766 /*
767  * Async_UnlistenOnExit
768  *
769  * This is executed at backend exit if we have done any LISTENs in this
770  * backend. It might not be necessary anymore, if the user UNLISTENed
771  * everything, but we don't try to detect that case.
772  */
773 static void
775 {
778 }
779 
780 /*
781  * AtPrepare_Notify
782  *
783  * This is called at the prepare phase of a two-phase
784  * transaction. Save the state for possible commit later.
785  */
786 void
788 {
789  /* It's not allowed to have any pending LISTEN/UNLISTEN/NOTIFY actions */
790  if (pendingActions || pendingNotifies)
791  ereport(ERROR,
792  (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
793  errmsg("cannot PREPARE a transaction that has executed LISTEN, UNLISTEN, or NOTIFY")));
794 }
795 
796 /*
797  * PreCommit_Notify
798  *
799  * This is called at transaction commit, before actually committing to
800  * clog.
801  *
802  * If there are pending LISTEN actions, make sure we are listed in the
803  * shared-memory listener array. This must happen before commit to
804  * ensure we don't miss any notifies from transactions that commit
805  * just after ours.
806  *
807  * If there are outbound notify requests in the pendingNotifies list,
808  * add them to the global queue. We do that before commit so that
809  * we can still throw error if we run out of queue space.
810  */
811 void
813 {
814  ListCell *p;
815 
816  if (!pendingActions && !pendingNotifies)
817  return; /* no relevant statements in this xact */
818 
819  if (Trace_notify)
820  elog(DEBUG1, "PreCommit_Notify");
821 
822  /* Preflight for any pending listen/unlisten actions */
823  foreach(p, pendingActions)
824  {
825  ListenAction *actrec = (ListenAction *) lfirst(p);
826 
827  switch (actrec->action)
828  {
829  case LISTEN_LISTEN:
831  break;
832  case LISTEN_UNLISTEN:
833  /* there is no Exec_UnlistenPreCommit() */
834  break;
835  case LISTEN_UNLISTEN_ALL:
836  /* there is no Exec_UnlistenAllPreCommit() */
837  break;
838  }
839  }
840 
841  /* Queue any pending notifies (must happen after the above) */
842  if (pendingNotifies)
843  {
844  ListCell *nextNotify;
845 
846  /*
847  * Make sure that we have an XID assigned to the current transaction.
848  * GetCurrentTransactionId is cheap if we already have an XID, but not
849  * so cheap if we don't, and we'd prefer not to do that work while
850  * holding AsyncQueueLock.
851  */
852  (void) GetCurrentTransactionId();
853 
854  /*
855  * Serialize writers by acquiring a special lock that we hold till
856  * after commit. This ensures that queue entries appear in commit
857  * order, and in particular that there are never uncommitted queue
858  * entries ahead of committed ones, so an uncommitted transaction
859  * can't block delivery of deliverable notifications.
860  *
861  * We use a heavyweight lock so that it'll automatically be released
862  * after either commit or abort. This also allows deadlocks to be
863  * detected, though really a deadlock shouldn't be possible here.
864  *
865  * The lock is on "database 0", which is pretty ugly but it doesn't
866  * seem worth inventing a special locktag category just for this.
867  * (Historical note: before PG 9.0, a similar lock on "database 0" was
868  * used by the flatfiles mechanism.)
869  */
870  LockSharedObject(DatabaseRelationId, InvalidOid, 0,
872 
873  /* Now push the notifications into the queue */
875 
876  nextNotify = list_head(pendingNotifies->events);
877  while (nextNotify != NULL)
878  {
879  /*
880  * Add the pending notifications to the queue. We acquire and
881  * release AsyncQueueLock once per page, which might be overkill
882  * but it does allow readers to get in while we're doing this.
883  *
884  * A full queue is very uncommon and should really not happen,
885  * given that we have so much space available in the SLRU pages.
886  * Nevertheless we need to deal with this possibility. Note that
887  * when we get here we are in the process of committing our
888  * transaction, but we have not yet committed to clog, so at this
889  * point in time we can still roll the transaction back.
890  */
891  LWLockAcquire(AsyncQueueLock, LW_EXCLUSIVE);
893  if (asyncQueueIsFull())
894  ereport(ERROR,
895  (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
896  errmsg("too many notifications in the NOTIFY queue")));
897  nextNotify = asyncQueueAddEntries(nextNotify);
898  LWLockRelease(AsyncQueueLock);
899  }
900  }
901 }
902 
903 /*
904  * AtCommit_Notify
905  *
906  * This is called at transaction commit, after committing to clog.
907  *
908  * Update listenChannels and clear transaction-local state.
909  */
910 void
912 {
913  ListCell *p;
914 
915  /*
916  * Allow transactions that have not executed LISTEN/UNLISTEN/NOTIFY to
917  * return as soon as possible
918  */
919  if (!pendingActions && !pendingNotifies)
920  return;
921 
922  if (Trace_notify)
923  elog(DEBUG1, "AtCommit_Notify");
924 
925  /* Perform any pending listen/unlisten actions */
926  foreach(p, pendingActions)
927  {
928  ListenAction *actrec = (ListenAction *) lfirst(p);
929 
930  switch (actrec->action)
931  {
932  case LISTEN_LISTEN:
933  Exec_ListenCommit(actrec->channel);
934  break;
935  case LISTEN_UNLISTEN:
936  Exec_UnlistenCommit(actrec->channel);
937  break;
938  case LISTEN_UNLISTEN_ALL:
940  break;
941  }
942  }
943 
944  /* If no longer listening to anything, get out of listener array */
945  if (amRegisteredListener && listenChannels == NIL)
947 
948  /* And clean up */
950 }
951 
952 /*
953  * Exec_ListenPreCommit --- subroutine for PreCommit_Notify
954  *
955  * This function must make sure we are ready to catch any incoming messages.
956  */
957 static void
959 {
960  QueuePosition head;
961  QueuePosition max;
962  int i;
963 
964  /*
965  * Nothing to do if we are already listening to something, nor if we
966  * already ran this routine in this transaction.
967  */
969  return;
970 
971  if (Trace_notify)
972  elog(DEBUG1, "Exec_ListenPreCommit(%d)", MyProcPid);
973 
974  /*
975  * Before registering, make sure we will unlisten before dying. (Note:
976  * this action does not get undone if we abort later.)
977  */
979  {
981  unlistenExitRegistered = true;
982  }
983 
984  /*
985  * This is our first LISTEN, so establish our pointer.
986  *
987  * We set our pointer to the global tail pointer and then move it forward
988  * over already-committed notifications. This ensures we cannot miss any
989  * not-yet-committed notifications. We might get a few more but that
990  * doesn't hurt.
991  *
992  * In some scenarios there might be a lot of committed notifications that
993  * have not yet been pruned away (because some backend is being lazy about
994  * reading them). To reduce our startup time, we can look at other
995  * backends and adopt the maximum "pos" pointer of any backend that's in
996  * our database; any notifications it's already advanced over are surely
997  * committed and need not be re-examined by us. (We must consider only
998  * backends connected to our DB, because others will not have bothered to
999  * check committed-ness of notifications in our DB.) But we only bother
1000  * with that if there's more than a page worth of notifications
1001  * outstanding, otherwise scanning all the other backends isn't worth it.
1002  *
1003  * We need exclusive lock here so we can look at other backends' entries.
1004  */
1005  LWLockAcquire(AsyncQueueLock, LW_EXCLUSIVE);
1006  head = QUEUE_HEAD;
1007  max = QUEUE_TAIL;
1008  if (QUEUE_POS_PAGE(max) != QUEUE_POS_PAGE(head))
1009  {
1010  for (i = 1; i <= MaxBackends; i++)
1011  {
1013  max = QUEUE_POS_MAX(max, QUEUE_BACKEND_POS(i));
1014  }
1015  }
1019  LWLockRelease(AsyncQueueLock);
1020 
1021  /* Now we are listed in the global array, so remember we're listening */
1022  amRegisteredListener = true;
1023 
1024  /*
1025  * Try to move our pointer forward as far as possible. This will skip over
1026  * already-committed notifications. Still, we could get notifications that
1027  * have already committed before we started to LISTEN.
1028  *
1029  * Note that we are not yet listening on anything, so we won't deliver any
1030  * notification to the frontend. Also, although our transaction might
1031  * have executed NOTIFY, those message(s) aren't queued yet so we can't
1032  * see them in the queue.
1033  *
1034  * This will also advance the global tail pointer if possible.
1035  */
1036  if (!QUEUE_POS_EQUAL(max, head))
1038 }
1039 
1040 /*
1041  * Exec_ListenCommit --- subroutine for AtCommit_Notify
1042  *
1043  * Add the channel to the list of channels we are listening on.
1044  */
1045 static void
1046 Exec_ListenCommit(const char *channel)
1047 {
1048  MemoryContext oldcontext;
1049 
1050  /* Do nothing if we are already listening on this channel */
1051  if (IsListeningOn(channel))
1052  return;
1053 
1054  /*
1055  * Add the new channel name to listenChannels.
1056  *
1057  * XXX It is theoretically possible to get an out-of-memory failure here,
1058  * which would be bad because we already committed. For the moment it
1059  * doesn't seem worth trying to guard against that, but maybe improve this
1060  * later.
1061  */
1063  listenChannels = lappend(listenChannels, pstrdup(channel));
1064  MemoryContextSwitchTo(oldcontext);
1065 }
1066 
1067 /*
1068  * Exec_UnlistenCommit --- subroutine for AtCommit_Notify
1069  *
1070  * Remove the specified channel name from listenChannels.
1071  */
1072 static void
1073 Exec_UnlistenCommit(const char *channel)
1074 {
1075  ListCell *q;
1076 
1077  if (Trace_notify)
1078  elog(DEBUG1, "Exec_UnlistenCommit(%s,%d)", channel, MyProcPid);
1079 
1080  foreach(q, listenChannels)
1081  {
1082  char *lchan = (char *) lfirst(q);
1083 
1084  if (strcmp(lchan, channel) == 0)
1085  {
1086  listenChannels = foreach_delete_current(listenChannels, q);
1087  pfree(lchan);
1088  break;
1089  }
1090  }
1091 
1092  /*
1093  * We do not complain about unlistening something not being listened;
1094  * should we?
1095  */
1096 }
1097 
1098 /*
1099  * Exec_UnlistenAllCommit --- subroutine for AtCommit_Notify
1100  *
1101  * Unlisten on all channels for this backend.
1102  */
1103 static void
1105 {
1106  if (Trace_notify)
1107  elog(DEBUG1, "Exec_UnlistenAllCommit(%d)", MyProcPid);
1108 
1109  list_free_deep(listenChannels);
1110  listenChannels = NIL;
1111 }
1112 
1113 /*
1114  * ProcessCompletedNotifies --- send out signals and self-notifies
1115  *
1116  * This is called from postgres.c just before going idle at the completion
1117  * of a transaction. If we issued any notifications in the just-completed
1118  * transaction, send signals to other backends to process them, and also
1119  * process the queue ourselves to send messages to our own frontend.
1120  *
1121  * The reason that this is not done in AtCommit_Notify is that there is
1122  * a nonzero chance of errors here (for example, encoding conversion errors
1123  * while trying to format messages to our frontend). An error during
1124  * AtCommit_Notify would be a PANIC condition. The timing is also arranged
1125  * to ensure that a transaction's self-notifies are delivered to the frontend
1126  * before it gets the terminating ReadyForQuery message.
1127  *
1128  * Note that we send signals and process the queue even if the transaction
1129  * eventually aborted. This is because we need to clean out whatever got
1130  * added to the queue.
1131  *
1132  * NOTE: we are outside of any transaction here.
1133  */
1134 void
1136 {
1137  MemoryContext caller_context;
1138  bool signalled;
1139 
1140  /* Nothing to do if we didn't send any notifications */
1142  return;
1143 
1144  /*
1145  * We reset the flag immediately; otherwise, if any sort of error occurs
1146  * below, we'd be locked up in an infinite loop, because control will come
1147  * right back here after error cleanup.
1148  */
1150 
1151  /*
1152  * We must preserve the caller's memory context (probably MessageContext)
1153  * across the transaction we do here.
1154  */
1155  caller_context = CurrentMemoryContext;
1156 
1157  if (Trace_notify)
1158  elog(DEBUG1, "ProcessCompletedNotifies");
1159 
1160  /*
1161  * We must run asyncQueueReadAllNotifications inside a transaction, else
1162  * bad things happen if it gets an error.
1163  */
1165 
1166  /* Send signals to other backends */
1167  signalled = SignalBackends();
1168 
1169  if (listenChannels != NIL)
1170  {
1171  /* Read the queue ourselves, and send relevant stuff to the frontend */
1173  }
1174  else if (!signalled)
1175  {
1176  /*
1177  * If we found no other listening backends, and we aren't listening
1178  * ourselves, then we must execute asyncQueueAdvanceTail to flush the
1179  * queue, because ain't nobody else gonna do it. This prevents queue
1180  * overflow when we're sending useless notifies to nobody. (A new
1181  * listener could have joined since we looked, but if so this is
1182  * harmless.)
1183  */
1185  }
1186 
1188 
1189  MemoryContextSwitchTo(caller_context);
1190 
1191  /* We don't need pq_flush() here since postgres.c will do one shortly */
1192 }
1193 
1194 /*
1195  * Test whether we are actively listening on the given channel name.
1196  *
1197  * Note: this function is executed for every notification found in the queue.
1198  * Perhaps it is worth further optimization, eg convert the list to a sorted
1199  * array so we can binary-search it. In practice the list is likely to be
1200  * fairly short, though.
1201  */
1202 static bool
1203 IsListeningOn(const char *channel)
1204 {
1205  ListCell *p;
1206 
1207  foreach(p, listenChannels)
1208  {
1209  char *lchan = (char *) lfirst(p);
1210 
1211  if (strcmp(lchan, channel) == 0)
1212  return true;
1213  }
1214  return false;
1215 }
1216 
1217 /*
1218  * Remove our entry from the listeners array when we are no longer listening
1219  * on any channel. NB: must not fail if we're already not listening.
1220  */
1221 static void
1223 {
1224  bool advanceTail;
1225 
1226  Assert(listenChannels == NIL); /* else caller error */
1227 
1228  if (!amRegisteredListener) /* nothing to do */
1229  return;
1230 
1231  LWLockAcquire(AsyncQueueLock, LW_SHARED);
1232  /* check if entry is valid and oldest ... */
1233  advanceTail = (MyProcPid == QUEUE_BACKEND_PID(MyBackendId)) &&
1235  /* ... then mark it invalid */
1238  LWLockRelease(AsyncQueueLock);
1239 
1240  /* mark ourselves as no longer listed in the global array */
1241  amRegisteredListener = false;
1242 
1243  /* If we were the laziest backend, try to advance the tail pointer */
1244  if (advanceTail)
1246 }
1247 
1248 /*
1249  * Test whether there is room to insert more notification messages.
1250  *
1251  * Caller must hold at least shared AsyncQueueLock.
1252  */
1253 static bool
1255 {
1256  int nexthead;
1257  int boundary;
1258 
1259  /*
1260  * The queue is full if creating a new head page would create a page that
1261  * logically precedes the current global tail pointer, ie, the head
1262  * pointer would wrap around compared to the tail. We cannot create such
1263  * a head page for fear of confusing slru.c. For safety we round the tail
1264  * pointer back to a segment boundary (compare the truncation logic in
1265  * asyncQueueAdvanceTail).
1266  *
1267  * Note that this test is *not* dependent on how much space there is on
1268  * the current head page. This is necessary because asyncQueueAddEntries
1269  * might try to create the next head page in any case.
1270  */
1271  nexthead = QUEUE_POS_PAGE(QUEUE_HEAD) + 1;
1272  if (nexthead > QUEUE_MAX_PAGE)
1273  nexthead = 0; /* wrap around */
1274  boundary = QUEUE_POS_PAGE(QUEUE_TAIL);
1275  boundary -= boundary % SLRU_PAGES_PER_SEGMENT;
1276  return asyncQueuePagePrecedes(nexthead, boundary);
1277 }
1278 
1279 /*
1280  * Advance the QueuePosition to the next entry, assuming that the current
1281  * entry is of length entryLength. If we jump to a new page the function
1282  * returns true, else false.
1283  */
1284 static bool
1285 asyncQueueAdvance(volatile QueuePosition *position, int entryLength)
1286 {
1287  int pageno = QUEUE_POS_PAGE(*position);
1288  int offset = QUEUE_POS_OFFSET(*position);
1289  bool pageJump = false;
1290 
1291  /*
1292  * Move to the next writing position: First jump over what we have just
1293  * written or read.
1294  */
1295  offset += entryLength;
1296  Assert(offset <= QUEUE_PAGESIZE);
1297 
1298  /*
1299  * In a second step check if another entry can possibly be written to the
1300  * page. If so, stay here, we have reached the next position. If not, then
1301  * we need to move on to the next page.
1302  */
1304  {
1305  pageno++;
1306  if (pageno > QUEUE_MAX_PAGE)
1307  pageno = 0; /* wrap around */
1308  offset = 0;
1309  pageJump = true;
1310  }
1311 
1312  SET_QUEUE_POS(*position, pageno, offset);
1313  return pageJump;
1314 }
1315 
1316 /*
1317  * Fill the AsyncQueueEntry at *qe with an outbound notification message.
1318  */
1319 static void
1321 {
1322  size_t channellen = n->channel_len;
1323  size_t payloadlen = n->payload_len;
1324  int entryLength;
1325 
1326  Assert(channellen < NAMEDATALEN);
1327  Assert(payloadlen < NOTIFY_PAYLOAD_MAX_LENGTH);
1328 
1329  /* The terminators are already included in AsyncQueueEntryEmptySize */
1330  entryLength = AsyncQueueEntryEmptySize + payloadlen + channellen;
1331  entryLength = QUEUEALIGN(entryLength);
1332  qe->length = entryLength;
1333  qe->dboid = MyDatabaseId;
1334  qe->xid = GetCurrentTransactionId();
1335  qe->srcPid = MyProcPid;
1336  memcpy(qe->data, n->data, channellen + payloadlen + 2);
1337 }
1338 
1339 /*
1340  * Add pending notifications to the queue.
1341  *
1342  * We go page by page here, i.e. we stop once we have to go to a new page but
1343  * we will be called again and then fill that next page. If an entry does not
1344  * fit into the current page, we write a dummy entry with an InvalidOid as the
1345  * database OID in order to fill the page. So every page is always used up to
1346  * the last byte which simplifies reading the page later.
1347  *
1348  * We are passed the list cell (in pendingNotifies->events) containing the next
1349  * notification to write and return the first still-unwritten cell back.
1350  * Eventually we will return NULL indicating all is done.
1351  *
1352  * We are holding AsyncQueueLock already from the caller and grab AsyncCtlLock
1353  * locally in this function.
1354  */
1355 static ListCell *
1357 {
1358  AsyncQueueEntry qe;
1359  QueuePosition queue_head;
1360  int pageno;
1361  int offset;
1362  int slotno;
1363 
1364  /* We hold both AsyncQueueLock and AsyncCtlLock during this operation */
1365  LWLockAcquire(AsyncCtlLock, LW_EXCLUSIVE);
1366 
1367  /*
1368  * We work with a local copy of QUEUE_HEAD, which we write back to shared
1369  * memory upon exiting. The reason for this is that if we have to advance
1370  * to a new page, SimpleLruZeroPage might fail (out of disk space, for
1371  * instance), and we must not advance QUEUE_HEAD if it does. (Otherwise,
1372  * subsequent insertions would try to put entries into a page that slru.c
1373  * thinks doesn't exist yet.) So, use a local position variable. Note
1374  * that if we do fail, any already-inserted queue entries are forgotten;
1375  * this is okay, since they'd be useless anyway after our transaction
1376  * rolls back.
1377  */
1378  queue_head = QUEUE_HEAD;
1379 
1380  /* Fetch the current page */
1381  pageno = QUEUE_POS_PAGE(queue_head);
1382  slotno = SimpleLruReadPage(AsyncCtl, pageno, true, InvalidTransactionId);
1383  /* Note we mark the page dirty before writing in it */
1384  AsyncCtl->shared->page_dirty[slotno] = true;
1385 
1386  while (nextNotify != NULL)
1387  {
1388  Notification *n = (Notification *) lfirst(nextNotify);
1389 
1390  /* Construct a valid queue entry in local variable qe */
1392 
1393  offset = QUEUE_POS_OFFSET(queue_head);
1394 
1395  /* Check whether the entry really fits on the current page */
1396  if (offset + qe.length <= QUEUE_PAGESIZE)
1397  {
1398  /* OK, so advance nextNotify past this item */
1399  nextNotify = lnext(pendingNotifies->events, nextNotify);
1400  }
1401  else
1402  {
1403  /*
1404  * Write a dummy entry to fill up the page. Actually readers will
1405  * only check dboid and since it won't match any reader's database
1406  * OID, they will ignore this entry and move on.
1407  */
1408  qe.length = QUEUE_PAGESIZE - offset;
1409  qe.dboid = InvalidOid;
1410  qe.data[0] = '\0'; /* empty channel */
1411  qe.data[1] = '\0'; /* empty payload */
1412  }
1413 
1414  /* Now copy qe into the shared buffer page */
1415  memcpy(AsyncCtl->shared->page_buffer[slotno] + offset,
1416  &qe,
1417  qe.length);
1418 
1419  /* Advance queue_head appropriately, and detect if page is full */
1420  if (asyncQueueAdvance(&(queue_head), qe.length))
1421  {
1422  /*
1423  * Page is full, so we're done here, but first fill the next page
1424  * with zeroes. The reason to do this is to ensure that slru.c's
1425  * idea of the head page is always the same as ours, which avoids
1426  * boundary problems in SimpleLruTruncate. The test in
1427  * asyncQueueIsFull() ensured that there is room to create this
1428  * page without overrunning the queue.
1429  */
1430  slotno = SimpleLruZeroPage(AsyncCtl, QUEUE_POS_PAGE(queue_head));
1431  /* And exit the loop */
1432  break;
1433  }
1434  }
1435 
1436  /* Success, so update the global QUEUE_HEAD */
1437  QUEUE_HEAD = queue_head;
1438 
1439  LWLockRelease(AsyncCtlLock);
1440 
1441  return nextNotify;
1442 }
1443 
1444 /*
1445  * SQL function to return the fraction of the notification queue currently
1446  * occupied.
1447  */
1448 Datum
1450 {
1451  double usage;
1452 
1453  LWLockAcquire(AsyncQueueLock, LW_SHARED);
1454  usage = asyncQueueUsage();
1455  LWLockRelease(AsyncQueueLock);
1456 
1457  PG_RETURN_FLOAT8(usage);
1458 }
1459 
1460 /*
1461  * Return the fraction of the queue that is currently occupied.
1462  *
1463  * The caller must hold AsyncQueueLock in (at least) shared mode.
1464  */
1465 static double
1467 {
1468  int headPage = QUEUE_POS_PAGE(QUEUE_HEAD);
1469  int tailPage = QUEUE_POS_PAGE(QUEUE_TAIL);
1470  int occupied;
1471 
1472  occupied = headPage - tailPage;
1473 
1474  if (occupied == 0)
1475  return (double) 0; /* fast exit for common case */
1476 
1477  if (occupied < 0)
1478  {
1479  /* head has wrapped around, tail not yet */
1480  occupied += QUEUE_MAX_PAGE + 1;
1481  }
1482 
1483  return (double) occupied / (double) ((QUEUE_MAX_PAGE + 1) / 2);
1484 }
1485 
1486 /*
1487  * Check whether the queue is at least half full, and emit a warning if so.
1488  *
1489  * This is unlikely given the size of the queue, but possible.
1490  * The warnings show up at most once every QUEUE_FULL_WARN_INTERVAL.
1491  *
1492  * Caller must hold exclusive AsyncQueueLock.
1493  */
1494 static void
1496 {
1497  double fillDegree;
1498  TimestampTz t;
1499 
1500  fillDegree = asyncQueueUsage();
1501  if (fillDegree < 0.5)
1502  return;
1503 
1504  t = GetCurrentTimestamp();
1505 
1506  if (TimestampDifferenceExceeds(asyncQueueControl->lastQueueFillWarn,
1508  {
1509  QueuePosition min = QUEUE_HEAD;
1510  int32 minPid = InvalidPid;
1511  int i;
1512 
1513  for (i = 1; i <= MaxBackends; i++)
1514  {
1515  if (QUEUE_BACKEND_PID(i) != InvalidPid)
1516  {
1517  min = QUEUE_POS_MIN(min, QUEUE_BACKEND_POS(i));
1518  if (QUEUE_POS_EQUAL(min, QUEUE_BACKEND_POS(i)))
1519  minPid = QUEUE_BACKEND_PID(i);
1520  }
1521  }
1522 
1523  ereport(WARNING,
1524  (errmsg("NOTIFY queue is %.0f%% full", fillDegree * 100),
1525  (minPid != InvalidPid ?
1526  errdetail("The server process with PID %d is among those with the oldest transactions.", minPid)
1527  : 0),
1528  (minPid != InvalidPid ?
1529  errhint("The NOTIFY queue cannot be emptied until that process ends its current transaction.")
1530  : 0)));
1531 
1532  asyncQueueControl->lastQueueFillWarn = t;
1533  }
1534 }
1535 
1536 /*
1537  * Send signals to all listening backends (except our own).
1538  *
1539  * Returns true if we sent at least one signal.
1540  *
1541  * Since we need EXCLUSIVE lock anyway we also check the position of the other
1542  * backends and in case one is already up-to-date we don't signal it.
1543  * This can happen if concurrent notifying transactions have sent a signal and
1544  * the signaled backend has read the other notifications and ours in the same
1545  * step.
1546  *
1547  * Since we know the BackendId and the Pid the signalling is quite cheap.
1548  */
1549 static bool
1551 {
1552  bool signalled = false;
1553  int32 *pids;
1554  BackendId *ids;
1555  int count;
1556  int i;
1557  int32 pid;
1558 
1559  /*
1560  * Identify all backends that are listening and not already up-to-date. We
1561  * don't want to send signals while holding the AsyncQueueLock, so we just
1562  * build a list of target PIDs.
1563  *
1564  * XXX in principle these pallocs could fail, which would be bad. Maybe
1565  * preallocate the arrays? But in practice this is only run in trivial
1566  * transactions, so there should surely be space available.
1567  */
1568  pids = (int32 *) palloc(MaxBackends * sizeof(int32));
1569  ids = (BackendId *) palloc(MaxBackends * sizeof(BackendId));
1570  count = 0;
1571 
1572  LWLockAcquire(AsyncQueueLock, LW_EXCLUSIVE);
1573  for (i = 1; i <= MaxBackends; i++)
1574  {
1575  pid = QUEUE_BACKEND_PID(i);
1576  if (pid != InvalidPid && pid != MyProcPid)
1577  {
1579 
1580  if (!QUEUE_POS_EQUAL(pos, QUEUE_HEAD))
1581  {
1582  pids[count] = pid;
1583  ids[count] = i;
1584  count++;
1585  }
1586  }
1587  }
1588  LWLockRelease(AsyncQueueLock);
1589 
1590  /* Now send signals */
1591  for (i = 0; i < count; i++)
1592  {
1593  pid = pids[i];
1594 
1595  /*
1596  * Note: assuming things aren't broken, a signal failure here could
1597  * only occur if the target backend exited since we released
1598  * AsyncQueueLock; which is unlikely but certainly possible. So we
1599  * just log a low-level debug message if it happens.
1600  */
1601  if (SendProcSignal(pid, PROCSIG_NOTIFY_INTERRUPT, ids[i]) < 0)
1602  elog(DEBUG3, "could not signal backend with PID %d: %m", pid);
1603  else
1604  signalled = true;
1605  }
1606 
1607  pfree(pids);
1608  pfree(ids);
1609 
1610  return signalled;
1611 }
1612 
1613 /*
1614  * AtAbort_Notify
1615  *
1616  * This is called at transaction abort.
1617  *
1618  * Gets rid of pending actions and outbound notifies that we would have
1619  * executed if the transaction got committed.
1620  */
1621 void
1623 {
1624  /*
1625  * If we LISTEN but then roll back the transaction after PreCommit_Notify,
1626  * we have registered as a listener but have not made any entry in
1627  * listenChannels. In that case, deregister again.
1628  */
1629  if (amRegisteredListener && listenChannels == NIL)
1631 
1632  /* And clean up */
1634 }
1635 
1636 /*
1637  * AtSubStart_Notify() --- Take care of subtransaction start.
1638  *
1639  * Push empty state for the new subtransaction.
1640  */
1641 void
1643 {
1644  MemoryContext old_cxt;
1645 
1646  /* Keep the list-of-lists in TopTransactionContext for simplicity */
1648 
1649  upperPendingActions = lcons(pendingActions, upperPendingActions);
1650 
1651  Assert(list_length(upperPendingActions) ==
1653 
1654  pendingActions = NIL;
1655 
1656  upperPendingNotifies = lcons(pendingNotifies, upperPendingNotifies);
1657 
1658  Assert(list_length(upperPendingNotifies) ==
1660 
1661  pendingNotifies = NULL;
1662 
1663  MemoryContextSwitchTo(old_cxt);
1664 }
1665 
1666 /*
1667  * AtSubCommit_Notify() --- Take care of subtransaction commit.
1668  *
1669  * Reassign all items in the pending lists to the parent transaction.
1670  */
1671 void
1673 {
1674  List *parentPendingActions;
1675  NotificationList *parentPendingNotifies;
1676 
1677  parentPendingActions = linitial_node(List, upperPendingActions);
1678  upperPendingActions = list_delete_first(upperPendingActions);
1679 
1680  Assert(list_length(upperPendingActions) ==
1682 
1683  /*
1684  * Mustn't try to eliminate duplicates here --- see queue_listen()
1685  */
1686  pendingActions = list_concat(parentPendingActions, pendingActions);
1687 
1688  parentPendingNotifies = (NotificationList *) linitial(upperPendingNotifies);
1689  upperPendingNotifies = list_delete_first(upperPendingNotifies);
1690 
1691  Assert(list_length(upperPendingNotifies) ==
1693 
1694  if (pendingNotifies == NULL)
1695  {
1696  /* easy, no notify events happened in current subxact */
1697  pendingNotifies = parentPendingNotifies;
1698  }
1699  else if (parentPendingNotifies == NULL)
1700  {
1701  /* easy, subxact's list becomes parent's */
1702  }
1703  else
1704  {
1705  /*
1706  * Formerly, we didn't bother to eliminate duplicates here, but now we
1707  * must, else we fall foul of "Assert(!found)", either here or during
1708  * a later attempt to build the parent-level hashtable.
1709  */
1710  NotificationList *childPendingNotifies = pendingNotifies;
1711  ListCell *l;
1712 
1713  pendingNotifies = parentPendingNotifies;
1714  /* Insert all the subxact's events into parent, except for dups */
1715  foreach(l, childPendingNotifies->events)
1716  {
1717  Notification *childn = (Notification *) lfirst(l);
1718 
1719  if (!AsyncExistsPendingNotify(childn))
1720  AddEventToPendingNotifies(childn);
1721  }
1722  }
1723 }
1724 
1725 /*
1726  * AtSubAbort_Notify() --- Take care of subtransaction abort.
1727  */
1728 void
1730 {
1731  int my_level = GetCurrentTransactionNestLevel();
1732 
1733  /*
1734  * All we have to do is pop the stack --- the actions/notifies made in
1735  * this subxact are no longer interesting, and the space will be freed
1736  * when CurTransactionContext is recycled.
1737  *
1738  * This routine could be called more than once at a given nesting level if
1739  * there is trouble during subxact abort. Avoid dumping core by using
1740  * GetCurrentTransactionNestLevel as the indicator of how far we need to
1741  * prune the list.
1742  */
1743  while (list_length(upperPendingActions) > my_level - 2)
1744  {
1745  pendingActions = linitial_node(List, upperPendingActions);
1746  upperPendingActions = list_delete_first(upperPendingActions);
1747  }
1748 
1749  while (list_length(upperPendingNotifies) > my_level - 2)
1750  {
1751  pendingNotifies = (NotificationList *) linitial(upperPendingNotifies);
1752  upperPendingNotifies = list_delete_first(upperPendingNotifies);
1753  }
1754 }
1755 
1756 /*
1757  * HandleNotifyInterrupt
1758  *
1759  * Signal handler portion of interrupt handling. Let the backend know
1760  * that there's a pending notify interrupt. If we're currently reading
1761  * from the client, this will interrupt the read and
1762  * ProcessClientReadInterrupt() will call ProcessNotifyInterrupt().
1763  */
1764 void
1766 {
1767  /*
1768  * Note: this is called by a SIGNAL HANDLER. You must be very wary what
1769  * you do here.
1770  */
1771 
1772  /* signal that work needs to be done */
1773  notifyInterruptPending = true;
1774 
1775  /* make sure the event is processed in due course */
1776  SetLatch(MyLatch);
1777 }
1778 
1779 /*
1780  * ProcessNotifyInterrupt
1781  *
1782  * This is called just after waiting for a frontend command. If a
1783  * interrupt arrives (via HandleNotifyInterrupt()) while reading, the
1784  * read will be interrupted via the process's latch, and this routine
1785  * will get called. If we are truly idle (ie, *not* inside a transaction
1786  * block), process the incoming notifies.
1787  */
1788 void
1790 {
1792  return; /* not really idle */
1793 
1794  while (notifyInterruptPending)
1796 }
1797 
1798 
1799 /*
1800  * Read all pending notifications from the queue, and deliver appropriate
1801  * ones to my frontend. Stop when we reach queue head or an uncommitted
1802  * notification.
1803  */
1804 static void
1806 {
1807  volatile QueuePosition pos;
1808  QueuePosition oldpos;
1809  QueuePosition head;
1810  Snapshot snapshot;
1811  bool advanceTail;
1812 
1813  /* page_buffer must be adequately aligned, so use a union */
1814  union
1815  {
1816  char buf[QUEUE_PAGESIZE];
1817  AsyncQueueEntry align;
1818  } page_buffer;
1819 
1820  /* Fetch current state */
1821  LWLockAcquire(AsyncQueueLock, LW_SHARED);
1822  /* Assert checks that we have a valid state entry */
1824  pos = oldpos = QUEUE_BACKEND_POS(MyBackendId);
1825  head = QUEUE_HEAD;
1826  LWLockRelease(AsyncQueueLock);
1827 
1828  if (QUEUE_POS_EQUAL(pos, head))
1829  {
1830  /* Nothing to do, we have read all notifications already. */
1831  return;
1832  }
1833 
1834  /* Get snapshot we'll use to decide which xacts are still in progress */
1835  snapshot = RegisterSnapshot(GetLatestSnapshot());
1836 
1837  /*----------
1838  * Note that we deliver everything that we see in the queue and that
1839  * matches our _current_ listening state.
1840  * Especially we do not take into account different commit times.
1841  * Consider the following example:
1842  *
1843  * Backend 1: Backend 2:
1844  *
1845  * transaction starts
1846  * NOTIFY foo;
1847  * commit starts
1848  * transaction starts
1849  * LISTEN foo;
1850  * commit starts
1851  * commit to clog
1852  * commit to clog
1853  *
1854  * It could happen that backend 2 sees the notification from backend 1 in
1855  * the queue. Even though the notifying transaction committed before
1856  * the listening transaction, we still deliver the notification.
1857  *
1858  * The idea is that an additional notification does not do any harm, we
1859  * just need to make sure that we do not miss a notification.
1860  *
1861  * It is possible that we fail while trying to send a message to our
1862  * frontend (for example, because of encoding conversion failure).
1863  * If that happens it is critical that we not try to send the same
1864  * message over and over again. Therefore, we place a PG_TRY block
1865  * here that will forcibly advance our backend position before we lose
1866  * control to an error. (We could alternatively retake AsyncQueueLock
1867  * and move the position before handling each individual message, but
1868  * that seems like too much lock traffic.)
1869  *----------
1870  */
1871  PG_TRY();
1872  {
1873  bool reachedStop;
1874 
1875  do
1876  {
1877  int curpage = QUEUE_POS_PAGE(pos);
1878  int curoffset = QUEUE_POS_OFFSET(pos);
1879  int slotno;
1880  int copysize;
1881 
1882  /*
1883  * We copy the data from SLRU into a local buffer, so as to avoid
1884  * holding the AsyncCtlLock while we are examining the entries and
1885  * possibly transmitting them to our frontend. Copy only the part
1886  * of the page we will actually inspect.
1887  */
1888  slotno = SimpleLruReadPage_ReadOnly(AsyncCtl, curpage,
1890  if (curpage == QUEUE_POS_PAGE(head))
1891  {
1892  /* we only want to read as far as head */
1893  copysize = QUEUE_POS_OFFSET(head) - curoffset;
1894  if (copysize < 0)
1895  copysize = 0; /* just for safety */
1896  }
1897  else
1898  {
1899  /* fetch all the rest of the page */
1900  copysize = QUEUE_PAGESIZE - curoffset;
1901  }
1902  memcpy(page_buffer.buf + curoffset,
1903  AsyncCtl->shared->page_buffer[slotno] + curoffset,
1904  copysize);
1905  /* Release lock that we got from SimpleLruReadPage_ReadOnly() */
1906  LWLockRelease(AsyncCtlLock);
1907 
1908  /*
1909  * Process messages up to the stop position, end of page, or an
1910  * uncommitted message.
1911  *
1912  * Our stop position is what we found to be the head's position
1913  * when we entered this function. It might have changed already.
1914  * But if it has, we will receive (or have already received and
1915  * queued) another signal and come here again.
1916  *
1917  * We are not holding AsyncQueueLock here! The queue can only
1918  * extend beyond the head pointer (see above) and we leave our
1919  * backend's pointer where it is so nobody will truncate or
1920  * rewrite pages under us. Especially we don't want to hold a lock
1921  * while sending the notifications to the frontend.
1922  */
1923  reachedStop = asyncQueueProcessPageEntries(&pos, head,
1924  page_buffer.buf,
1925  snapshot);
1926  } while (!reachedStop);
1927  }
1928  PG_CATCH();
1929  {
1930  /* Update shared state */
1931  LWLockAcquire(AsyncQueueLock, LW_SHARED);
1933  advanceTail = QUEUE_POS_EQUAL(oldpos, QUEUE_TAIL);
1934  LWLockRelease(AsyncQueueLock);
1935 
1936  /* If we were the laziest backend, try to advance the tail pointer */
1937  if (advanceTail)
1939 
1940  PG_RE_THROW();
1941  }
1942  PG_END_TRY();
1943 
1944  /* Update shared state */
1945  LWLockAcquire(AsyncQueueLock, LW_SHARED);
1947  advanceTail = QUEUE_POS_EQUAL(oldpos, QUEUE_TAIL);
1948  LWLockRelease(AsyncQueueLock);
1949 
1950  /* If we were the laziest backend, try to advance the tail pointer */
1951  if (advanceTail)
1953 
1954  /* Done with snapshot */
1955  UnregisterSnapshot(snapshot);
1956 }
1957 
1958 /*
1959  * Fetch notifications from the shared queue, beginning at position current,
1960  * and deliver relevant ones to my frontend.
1961  *
1962  * The current page must have been fetched into page_buffer from shared
1963  * memory. (We could access the page right in shared memory, but that
1964  * would imply holding the AsyncCtlLock throughout this routine.)
1965  *
1966  * We stop if we reach the "stop" position, or reach a notification from an
1967  * uncommitted transaction, or reach the end of the page.
1968  *
1969  * The function returns true once we have reached the stop position or an
1970  * uncommitted notification, and false if we have finished with the page.
1971  * In other words: once it returns true there is no need to look further.
1972  * The QueuePosition *current is advanced past all processed messages.
1973  */
1974 static bool
1976  QueuePosition stop,
1977  char *page_buffer,
1978  Snapshot snapshot)
1979 {
1980  bool reachedStop = false;
1981  bool reachedEndOfPage;
1982  AsyncQueueEntry *qe;
1983 
1984  do
1985  {
1986  QueuePosition thisentry = *current;
1987 
1988  if (QUEUE_POS_EQUAL(thisentry, stop))
1989  break;
1990 
1991  qe = (AsyncQueueEntry *) (page_buffer + QUEUE_POS_OFFSET(thisentry));
1992 
1993  /*
1994  * Advance *current over this message, possibly to the next page. As
1995  * noted in the comments for asyncQueueReadAllNotifications, we must
1996  * do this before possibly failing while processing the message.
1997  */
1998  reachedEndOfPage = asyncQueueAdvance(current, qe->length);
1999 
2000  /* Ignore messages destined for other databases */
2001  if (qe->dboid == MyDatabaseId)
2002  {
2003  if (XidInMVCCSnapshot(qe->xid, snapshot))
2004  {
2005  /*
2006  * The source transaction is still in progress, so we can't
2007  * process this message yet. Break out of the loop, but first
2008  * back up *current so we will reprocess the message next
2009  * time. (Note: it is unlikely but not impossible for
2010  * TransactionIdDidCommit to fail, so we can't really avoid
2011  * this advance-then-back-up behavior when dealing with an
2012  * uncommitted message.)
2013  *
2014  * Note that we must test XidInMVCCSnapshot before we test
2015  * TransactionIdDidCommit, else we might return a message from
2016  * a transaction that is not yet visible to snapshots; compare
2017  * the comments at the head of heapam_visibility.c.
2018  *
2019  * Also, while our own xact won't be listed in the snapshot,
2020  * we need not check for TransactionIdIsCurrentTransactionId
2021  * because our transaction cannot (yet) have queued any
2022  * messages.
2023  */
2024  *current = thisentry;
2025  reachedStop = true;
2026  break;
2027  }
2028  else if (TransactionIdDidCommit(qe->xid))
2029  {
2030  /* qe->data is the null-terminated channel name */
2031  char *channel = qe->data;
2032 
2033  if (IsListeningOn(channel))
2034  {
2035  /* payload follows channel name */
2036  char *payload = qe->data + strlen(channel) + 1;
2037 
2038  NotifyMyFrontEnd(channel, payload, qe->srcPid);
2039  }
2040  }
2041  else
2042  {
2043  /*
2044  * The source transaction aborted or crashed, so we just
2045  * ignore its notifications.
2046  */
2047  }
2048  }
2049 
2050  /* Loop back if we're not at end of page */
2051  } while (!reachedEndOfPage);
2052 
2053  if (QUEUE_POS_EQUAL(*current, stop))
2054  reachedStop = true;
2055 
2056  return reachedStop;
2057 }
2058 
2059 /*
2060  * Advance the shared queue tail variable to the minimum of all the
2061  * per-backend tail pointers. Truncate pg_notify space if possible.
2062  */
2063 static void
2065 {
2066  QueuePosition min;
2067  int i;
2068  int oldtailpage;
2069  int newtailpage;
2070  int boundary;
2071 
2072  LWLockAcquire(AsyncQueueLock, LW_EXCLUSIVE);
2073  min = QUEUE_HEAD;
2074  for (i = 1; i <= MaxBackends; i++)
2075  {
2076  if (QUEUE_BACKEND_PID(i) != InvalidPid)
2077  min = QUEUE_POS_MIN(min, QUEUE_BACKEND_POS(i));
2078  }
2079  oldtailpage = QUEUE_POS_PAGE(QUEUE_TAIL);
2080  QUEUE_TAIL = min;
2081  LWLockRelease(AsyncQueueLock);
2082 
2083  /*
2084  * We can truncate something if the global tail advanced across an SLRU
2085  * segment boundary.
2086  *
2087  * XXX it might be better to truncate only once every several segments, to
2088  * reduce the number of directory scans.
2089  */
2090  newtailpage = QUEUE_POS_PAGE(min);
2091  boundary = newtailpage - (newtailpage % SLRU_PAGES_PER_SEGMENT);
2092  if (asyncQueuePagePrecedes(oldtailpage, boundary))
2093  {
2094  /*
2095  * SimpleLruTruncate() will ask for AsyncCtlLock but will also release
2096  * the lock again.
2097  */
2098  SimpleLruTruncate(AsyncCtl, newtailpage);
2099  }
2100 }
2101 
2102 /*
2103  * ProcessIncomingNotify
2104  *
2105  * Deal with arriving NOTIFYs from other backends as soon as it's safe to
2106  * do so. This used to be called from the PROCSIG_NOTIFY_INTERRUPT
2107  * signal handler, but isn't anymore.
2108  *
2109  * Scan the queue for arriving notifications and report them to my front
2110  * end.
2111  *
2112  * NOTE: since we are outside any transaction, we must create our own.
2113  */
2114 static void
2116 {
2117  /* We *must* reset the flag */
2118  notifyInterruptPending = false;
2119 
2120  /* Do nothing else if we aren't actively listening */
2121  if (listenChannels == NIL)
2122  return;
2123 
2124  if (Trace_notify)
2125  elog(DEBUG1, "ProcessIncomingNotify");
2126 
2127  set_ps_display("notify interrupt", false);
2128 
2129  /*
2130  * We must run asyncQueueReadAllNotifications inside a transaction, else
2131  * bad things happen if it gets an error.
2132  */
2134 
2136 
2138 
2139  /*
2140  * Must flush the notify messages to ensure frontend gets them promptly.
2141  */
2142  pq_flush();
2143 
2144  set_ps_display("idle", false);
2145 
2146  if (Trace_notify)
2147  elog(DEBUG1, "ProcessIncomingNotify: done");
2148 }
2149 
2150 /*
2151  * Send NOTIFY message to my front end.
2152  */
2153 void
2154 NotifyMyFrontEnd(const char *channel, const char *payload, int32 srcPid)
2155 {
2157  {
2159 
2160  pq_beginmessage(&buf, 'A');
2161  pq_sendint32(&buf, srcPid);
2162  pq_sendstring(&buf, channel);
2164  pq_sendstring(&buf, payload);
2165  pq_endmessage(&buf);
2166 
2167  /*
2168  * NOTE: we do not do pq_flush() here. For a self-notify, it will
2169  * happen at the end of the transaction, and for incoming notifies
2170  * ProcessIncomingNotify will do it after finding all the notifies.
2171  */
2172  }
2173  else
2174  elog(INFO, "NOTIFY for \"%s\" payload \"%s\"", channel, payload);
2175 }
2176 
2177 /* Does pendingNotifies include a match for the given event? */
2178 static bool
2180 {
2181  if (pendingNotifies == NULL)
2182  return false;
2183 
2184  if (pendingNotifies->hashtab != NULL)
2185  {
2186  /* Use the hash table to probe for a match */
2187  if (hash_search(pendingNotifies->hashtab,
2188  &n,
2189  HASH_FIND,
2190  NULL))
2191  return true;
2192  }
2193  else
2194  {
2195  /* Must scan the event list */
2196  ListCell *l;
2197 
2198  foreach(l, pendingNotifies->events)
2199  {
2200  Notification *oldn = (Notification *) lfirst(l);
2201 
2202  if (n->channel_len == oldn->channel_len &&
2203  n->payload_len == oldn->payload_len &&
2204  memcmp(n->data, oldn->data,
2205  n->channel_len + n->payload_len + 2) == 0)
2206  return true;
2207  }
2208  }
2209 
2210  return false;
2211 }
2212 
2213 /*
2214  * Add a notification event to a pre-existing pendingNotifies list.
2215  *
2216  * Because pendingNotifies->events is already nonempty, this works
2217  * correctly no matter what CurrentMemoryContext is.
2218  */
2219 static void
2221 {
2222  Assert(pendingNotifies->events != NIL);
2223 
2224  /* Create the hash table if it's time to */
2225  if (list_length(pendingNotifies->events) >= MIN_HASHABLE_NOTIFIES &&
2226  pendingNotifies->hashtab == NULL)
2227  {
2228  HASHCTL hash_ctl;
2229  ListCell *l;
2230 
2231  /* Create the hash table */
2232  MemSet(&hash_ctl, 0, sizeof(hash_ctl));
2233  hash_ctl.keysize = sizeof(Notification *);
2234  hash_ctl.entrysize = sizeof(NotificationHash);
2235  hash_ctl.hash = notification_hash;
2236  hash_ctl.match = notification_match;
2237  hash_ctl.hcxt = CurTransactionContext;
2238  pendingNotifies->hashtab =
2239  hash_create("Pending Notifies",
2240  256L,
2241  &hash_ctl,
2243 
2244  /* Insert all the already-existing events */
2245  foreach(l, pendingNotifies->events)
2246  {
2247  Notification *oldn = (Notification *) lfirst(l);
2248  NotificationHash *hentry;
2249  bool found;
2250 
2251  hentry = (NotificationHash *) hash_search(pendingNotifies->hashtab,
2252  &oldn,
2253  HASH_ENTER,
2254  &found);
2255  Assert(!found);
2256  hentry->event = oldn;
2257  }
2258  }
2259 
2260  /* Add new event to the list, in order */
2261  pendingNotifies->events = lappend(pendingNotifies->events, n);
2262 
2263  /* Add event to the hash table if needed */
2264  if (pendingNotifies->hashtab != NULL)
2265  {
2266  NotificationHash *hentry;
2267  bool found;
2268 
2269  hentry = (NotificationHash *) hash_search(pendingNotifies->hashtab,
2270  &n,
2271  HASH_ENTER,
2272  &found);
2273  Assert(!found);
2274  hentry->event = n;
2275  }
2276 }
2277 
2278 /*
2279  * notification_hash: hash function for notification hash table
2280  *
2281  * The hash "keys" are pointers to Notification structs.
2282  */
2283 static uint32
2284 notification_hash(const void *key, Size keysize)
2285 {
2286  const Notification *k = *(const Notification *const *) key;
2287 
2288  Assert(keysize == sizeof(Notification *));
2289  /* We don't bother to include the payload's trailing null in the hash */
2290  return DatumGetUInt32(hash_any((const unsigned char *) k->data,
2291  k->channel_len + k->payload_len + 1));
2292 }
2293 
2294 /*
2295  * notification_match: match function to use with notification_hash
2296  */
2297 static int
2298 notification_match(const void *key1, const void *key2, Size keysize)
2299 {
2300  const Notification *k1 = *(const Notification *const *) key1;
2301  const Notification *k2 = *(const Notification *const *) key2;
2302 
2303  Assert(keysize == sizeof(Notification *));
2304  if (k1->channel_len == k2->channel_len &&
2305  k1->payload_len == k2->payload_len &&
2306  memcmp(k1->data, k2->data,
2307  k1->channel_len + k1->payload_len + 2) == 0)
2308  return 0; /* equal */
2309  return 1; /* not equal */
2310 }
2311 
2312 /* Clear the pendingActions and pendingNotifies lists. */
2313 static void
2315 {
2316  /*
2317  * We used to have to explicitly deallocate the list members and nodes,
2318  * because they were malloc'd. Now, since we know they are palloc'd in
2319  * CurTransactionContext, we need not do that --- they'll go away
2320  * automatically at transaction exit. We need only reset the list head
2321  * pointers.
2322  */
2323  pendingActions = NIL;
2324  pendingNotifies = NULL;
2325 }
uint64 call_cntr
Definition: funcapi.h:66
#define DatumGetUInt32(X)
Definition: postgres.h:486
struct QueueBackendStatus QueueBackendStatus
#define NIL
Definition: pg_list.h:65
static void usage(void)
Definition: pg_standby.c:593
#define QUEUE_TAIL
Definition: async.c:258
char data[NAMEDATALEN+NOTIFY_PAYLOAD_MAX_LENGTH]
Definition: async.c:173
bool XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot)
Definition: snapmgr.c:2241
#define DEBUG1
Definition: elog.h:25
int MyProcPid
Definition: globals.c:40
int errhint(const char *fmt,...)
Definition: elog.c:974
static void queue_listen(ListenActionKind action, const char *channel)
Definition: async.c:659
static SlruCtlData AsyncCtlData
Definition: async.c:266
BackendId MyBackendId
Definition: globals.c:81
struct NotificationHash NotificationHash
List * events
Definition: async.c:362
#define pq_flush()
Definition: libpq.h:39
MemoryContext TopTransactionContext
Definition: mcxt.c:49
#define QUEUE_BACKEND_PID(i)
Definition: async.c:259
int page
Definition: async.c:186
#define HASH_CONTEXT
Definition: hsearch.h:93
#define HASH_ELEM
Definition: hsearch.h:87
uint32 TransactionId
Definition: c.h:507
Snapshot RegisterSnapshot(Snapshot snapshot)
Definition: snapmgr.c:865
static ListCell * lnext(const List *l, const ListCell *c)
Definition: pg_list.h:321
MemoryContext hcxt
Definition: hsearch.h:78
bool SlruScanDirCbDeleteAll(SlruCtl ctl, char *filename, int segpage, void *data)
Definition: slru.c:1359
#define DEBUG3
Definition: elog.h:23
TimestampTz GetCurrentTimestamp(void)
Definition: timestamp.c:1569
void AsyncShmemInit(void)
Definition: async.c:472
int64 TimestampTz
Definition: timestamp.h:39
#define SRF_IS_FIRSTCALL()
Definition: funcapi.h:283
struct NotificationList NotificationList
char * pstrdup(const char *in)
Definition: mcxt.c:1161
void CommitTransactionCommand(void)
Definition: xact.c:2895
void SimpleLruTruncate(SlruCtl ctl, int cutoffPage)
Definition: slru.c:1174
#define PG_RETURN_FLOAT8(x)
Definition: fmgr.h:356
char data[FLEXIBLE_ARRAY_MEMBER]
Definition: async.c:357
#define AsyncCtl
Definition: async.c:268
static MemoryContext MemoryContextSwitchTo(MemoryContext context)
Definition: palloc.h:109
static void Exec_UnlistenAllCommit(void)
Definition: async.c:1104
int offset
Definition: async.c:187
void set_ps_display(const char *activity, bool force)
Definition: ps_status.c:331
Size entrysize
Definition: hsearch.h:73
MemoryContext CurTransactionContext
Definition: mcxt.c:50
static List * listenChannels
Definition: async.c:296
List * list_concat(List *list1, const List *list2)
Definition: list.c:515
void AtPrepare_Notify(void)
Definition: async.c:787
int errcode(int sqlerrcode)
Definition: elog.c:570
Datum pg_notification_queue_usage(PG_FUNCTION_ARGS)
Definition: async.c:1449
bool IsTransactionOrTransactionBlock(void)
Definition: xact.c:4627
#define MemSet(start, val, len)
Definition: c.h:955
#define INFO
Definition: elog.h:33
static double asyncQueueUsage(void)
Definition: async.c:1466
void pq_sendstring(StringInfo buf, const char *str)
Definition: pqformat.c:197
void Async_Listen(const char *channel)
Definition: async.c:689
static void ClearPendingActionsAndNotifies(void)
Definition: async.c:2314
static void asyncQueueNotificationToEntry(Notification *n, AsyncQueueEntry *qe)
Definition: async.c:1320
Notification * event
Definition: async.c:370
void * hash_search(HTAB *hashp, const void *keyPtr, HASHACTION action, bool *foundPtr)
Definition: dynahash.c:906
static bool asyncQueueAdvance(volatile QueuePosition *position, int entryLength)
Definition: async.c:1285
#define QUEUE_HEAD
Definition: async.c:257
bool TransactionIdDidCommit(TransactionId transactionId)
Definition: transam.c:125
static void Exec_UnlistenCommit(const char *channel)
Definition: async.c:1073
unsigned int Oid
Definition: postgres_ext.h:31
#define PG_PROTOCOL_MAJOR(v)
Definition: pqcomm.h:104
void SetLatch(Latch *latch)
Definition: latch.c:436
#define linitial_node(type, l)
Definition: pg_list.h:198
bool TimestampDifferenceExceeds(TimestampTz start_time, TimestampTz stop_time, int msec)
Definition: timestamp.c:1668
Size SimpleLruShmemSize(int nslots, int nlsns)
Definition: slru.c:145
void list_free_deep(List *list)
Definition: list.c:1387
static bool IsListeningOn(const char *channel)
Definition: async.c:1203
#define SRF_PERCALL_SETUP()
Definition: funcapi.h:287
static bool unlistenExitRegistered
Definition: async.c:387
static bool asyncQueueIsFull(void)
Definition: async.c:1254
void pq_beginmessage(StringInfo buf, char msgtype)
Definition: pqformat.c:87
static bool AsyncExistsPendingNotify(Notification *n)
Definition: async.c:2179
static void asyncQueueFillWarning(void)
Definition: async.c:1495
signed int int32
Definition: c.h:346
#define PG_GETARG_TEXT_PP(n)
Definition: fmgr.h:303
void LWLockRelease(LWLock *lock)
Definition: lwlock.c:1726
#define foreach_delete_current(lst, cell)
Definition: pg_list.h:368
#define list_make1(x1)
Definition: pg_list.h:227
QueuePosition pos
Definition: async.c:221
#define NAMEDATALEN
#define SRF_RETURN_NEXT(_funcctx, _result)
Definition: funcapi.h:289
static void pq_sendint32(StringInfo buf, uint32 i)
Definition: pqformat.h:145
void PreCommit_Notify(void)
Definition: async.c:812
#define NUM_ASYNC_BUFFERS
Definition: async.h:21
#define QUEUE_POS_OFFSET(x)
Definition: async.c:191
Definition: dynahash.c:208
int SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
Definition: procsignal.c:180
unsigned short uint16
Definition: c.h:357
void pfree(void *pointer)
Definition: mcxt.c:1031
static NotificationList * pendingNotifies
Definition: async.c:373
#define linitial(l)
Definition: pg_list.h:195
#define AsyncQueueEntryEmptySize
Definition: async.c:179
#define ERROR
Definition: elog.h:43
static List * pendingActions
Definition: async.c:321
void PreventCommandDuringRecovery(const char *cmdname)
Definition: utility.c:276
void ProcessNotifyInterrupt(void)
Definition: async.c:1789
static void * list_nth(const List *list, int n)
Definition: pg_list.h:277
void * ShmemInitStruct(const char *name, Size size, bool *foundPtr)
Definition: shmem.c:372
int SimpleLruReadPage(SlruCtl ctl, int pageno, bool write_ok, TransactionId xid)
Definition: slru.c:375
static void Async_UnlistenOnExit(int code, Datum arg)
Definition: async.c:774
void AtSubCommit_Notify(void)
Definition: async.c:1672
static bool backendHasSentNotifications
Definition: async.c:393
int MaxBackends
Definition: globals.c:135
TransactionId GetCurrentTransactionId(void)
Definition: xact.c:423
static char * buf
Definition: pg_test_fsync.c:68
#define QUEUE_PAGESIZE
Definition: async.c:269
#define SET_QUEUE_POS(x, y, z)
Definition: async.c:193
static void AddEventToPendingNotifies(Notification *n)
Definition: async.c:2220
int errdetail(const char *fmt,...)
Definition: elog.c:860
void before_shmem_exit(pg_on_exit_callback function, Datum arg)
Definition: ipc.c:333
#define InvalidTransactionId
Definition: transam.h:31
static ListCell * list_head(const List *l)
Definition: pg_list.h:125
unsigned int uint32
Definition: c.h:358
void SimpleLruWritePage(SlruCtl ctl, int slotno)
Definition: slru.c:578
MemoryContext CurrentMemoryContext
Definition: mcxt.c:38
#define NOTIFY_PAYLOAD_MAX_LENGTH
Definition: async.c:153
static bool asyncQueuePagePrecedes(int p, int q)
Definition: async.c:432
static AsyncQueueControl * asyncQueueControl
Definition: async.c:255
static void asyncQueueAdvanceTail(void)
Definition: async.c:2064
static void Exec_ListenCommit(const char *channel)
Definition: async.c:1046
#define ereport(elevel, rest)
Definition: elog.h:141
#define IsParallelWorker()
Definition: parallel.h:60
MemoryContext TopMemoryContext
Definition: mcxt.c:44
void UnregisterSnapshot(Snapshot snapshot)
Definition: snapmgr.c:907
#define QUEUE_FULL_WARN_INTERVAL
Definition: async.c:270
List * lappend(List *list, void *datum)
Definition: list.c:321
struct AsyncQueueControl AsyncQueueControl
#define WARNING
Definition: elog.h:40
#define MIN_HASHABLE_NOTIFIES
Definition: async.c:366
struct QueuePosition QueuePosition
Size mul_size(Size s1, Size s2)
Definition: shmem.c:492
void AtSubAbort_Notify(void)
Definition: async.c:1729
static List * upperPendingActions
Definition: async.c:323
ListenActionKind
Definition: async.c:308
static void ProcessIncomingNotify(void)
Definition: async.c:2115
HTAB * hash_create(const char *tabname, long nelem, HASHCTL *info, int flags)
Definition: dynahash.c:316
uintptr_t Datum
Definition: postgres.h:367
static void asyncQueueUnregister(void)
Definition: async.c:1222
static bool amRegisteredListener
Definition: async.c:390
void AtAbort_Notify(void)
Definition: async.c:1622
Size add_size(Size s1, Size s2)
Definition: shmem.c:475
int BackendId
Definition: backendid.h:21
#define QUEUEALIGN(len)
Definition: async.c:177
Oid MyDatabaseId
Definition: globals.c:85
int SimpleLruReadPage_ReadOnly(SlruCtl ctl, int pageno, TransactionId xid)
Definition: slru.c:467
Size keysize
Definition: hsearch.h:72
void LockSharedObject(Oid classid, Oid objid, uint16 objsubid, LOCKMODE lockmode)
Definition: lmgr.c:1004
HashCompareFunc match
Definition: hsearch.h:75
#define QUEUE_BACKEND_DBOID(i)
Definition: async.c:260
QueuePosition head
Definition: async.c:247
static bool asyncQueueProcessPageEntries(volatile QueuePosition *current, QueuePosition stop, char *page_buffer, Snapshot snapshot)
Definition: async.c:1975
#define InvalidOid
Definition: postgres_ext.h:36
static void asyncQueueReadAllNotifications(void)
Definition: async.c:1805
void AtSubStart_Notify(void)
Definition: async.c:1642
int GetCurrentTransactionNestLevel(void)
Definition: xact.c:842
#define PG_RETURN_VOID()
Definition: fmgr.h:339
Datum pg_notify(PG_FUNCTION_ARGS)
Definition: async.c:538
List * lcons(void *datum, List *list)
Definition: list.c:453
#define PG_CATCH()
Definition: elog.h:310
HTAB * hashtab
Definition: async.c:363
static int notification_match(const void *key1, const void *key2, Size keysize)
Definition: async.c:2298
#define PG_ARGISNULL(n)
Definition: fmgr.h:204
static ListCell * asyncQueueAddEntries(ListCell *nextNotify)
Definition: async.c:1356
#define Assert(condition)
Definition: c.h:732
void ProcessCompletedNotifies(void)
Definition: async.c:1135
#define lfirst(lc)
Definition: pg_list.h:190
bool SlruScanDirectory(SlruCtl ctl, SlruScanCallback callback, void *data)
Definition: slru.c:1382
static uint32 notification_hash(const void *key, Size keysize)
Definition: async.c:2284
uint16 channel_len
Definition: async.c:354
static void Exec_ListenPreCommit(void)
Definition: async.c:958
void StartTransactionCommand(void)
Definition: xact.c:2794
uint16 payload_len
Definition: async.c:355
#define HASH_COMPARE
Definition: hsearch.h:90
Datum hash_any(register const unsigned char *k, register int keylen)
Definition: hashfn.c:148
struct AsyncQueueEntry AsyncQueueEntry
Size AsyncShmemSize(void)
Definition: async.c:455
size_t Size
Definition: c.h:466
void Async_UnlistenAll(void)
Definition: async.c:721
static int list_length(const List *l)
Definition: pg_list.h:169
struct Notification Notification
bool LWLockAcquire(LWLock *lock, LWLockMode mode)
Definition: lwlock.c:1122
#define PG_RE_THROW()
Definition: elog.h:331
bool Trace_notify
Definition: async.c:396
Snapshot GetLatestSnapshot(void)
Definition: snapmgr.c:381
void Async_Notify(const char *channel, const char *payload)
Definition: async.c:572
Datum pg_listening_channels(PG_FUNCTION_ARGS)
Definition: async.c:741
TransactionId xid
Definition: async.c:171
char * text_to_cstring(const text *t)
Definition: varlena.c:204
#define AccessExclusiveLock
Definition: lockdefs.h:45
void AtCommit_Notify(void)
Definition: async.c:911
void HandleNotifyInterrupt(void)
Definition: async.c:1765
void * palloc(Size size)
Definition: mcxt.c:924
int errmsg(const char *fmt,...)
Definition: elog.c:784
void pq_endmessage(StringInfo buf)
Definition: pqformat.c:298
#define elog(elevel,...)
Definition: elog.h:226
char channel[FLEXIBLE_ARRAY_MEMBER]
Definition: async.c:318
int i
ListenActionKind action
Definition: async.c:317
#define CStringGetTextDatum(s)
Definition: builtins.h:83
int32 srcPid
Definition: async.c:172
void * arg
static List * upperPendingNotifies
Definition: async.c:375
struct Latch * MyLatch
Definition: globals.c:54
volatile sig_atomic_t notifyInterruptPending
Definition: async.c:384
#define PG_FUNCTION_ARGS
Definition: fmgr.h:188
#define SLRU_PAGES_PER_SEGMENT
Definition: slru.h:33
TimestampTz lastQueueFillWarn
Definition: async.c:250
CommandDest whereToSendOutput
Definition: postgres.c:90
#define QUEUE_MAX_PAGE
Definition: async.c:289
#define PG_TRY()
Definition: elog.h:301
QueuePosition tail
Definition: async.c:248
#define QUEUE_BACKEND_POS(i)
Definition: async.c:261
Definition: pg_list.h:50
ProtocolVersion FrontendProtocol
Definition: globals.c:28
int SimpleLruZeroPage(SlruCtl ctl, int pageno)
Definition: slru.c:263
#define PG_END_TRY()
Definition: elog.h:317
#define QUEUE_POS_PAGE(x)
Definition: async.c:190
#define QUEUE_POS_EQUAL(x, y)
Definition: async.c:199
#define offsetof(type, field)
Definition: c.h:655
#define QUEUE_POS_MIN(x, y)
Definition: async.c:203
void Async_Unlisten(const char *channel)
Definition: async.c:703
List * list_delete_first(List *list)
Definition: list.c:857
void NotifyMyFrontEnd(const char *channel, const char *payload, int32 srcPid)
Definition: async.c:2154
HashValueFunc hash
Definition: hsearch.h:74
#define SRF_RETURN_DONE(_funcctx)
Definition: funcapi.h:307
static bool SignalBackends(void)
Definition: async.c:1550
#define HASH_FUNCTION
Definition: hsearch.h:89
#define InvalidPid
Definition: miscadmin.h:32
void SimpleLruInit(SlruCtl ctl, const char *name, int nslots, int nlsns, LWLock *ctllock, const char *subdir, int tranche_id)
Definition: slru.c:165
#define SRF_FIRSTCALL_INIT()
Definition: funcapi.h:285
#define QUEUE_POS_MAX(x, y)
Definition: async.c:209