Projectspg-queue

pg-queue

Postgres-native job queue, pub/sub, and cache.

I was already running Postgres and did not want to add Redis just for background jobs. So I used what Postgres already does.


How it works

FeatureMechanismReplaces
Job queueSELECT ... FOR UPDATE SKIP LOCKEDRedis lists, Celery, Sidekiq
Pub/subLISTEN / NOTIFYRedis pub/sub, NATS
CacheUNLOGGED table with TTL per rowRedis GET/SET/EXPIRE
Request-responseQueue + NOTIFY with correlation IDRedis streams, RPC frameworks

Why it works

  • SKIP LOCKED lets multiple workers grab jobs at the same time without blocking each other
  • LISTEN/NOTIFY pushes events to subscribers the instant a row is inserted: no polling
  • UNLOGGED tables skip write-ahead logging for cache-tier speed (data is not crash-safe, which is fine for a cache)
  • Zero new infrastructure. Your Postgres already does all of this

Where it is used

Primary consumer: Testudo. The transactional outbox bridge between the execution engine and the DLT settlement layer.


Related: Testudo | Projects

Built with LogoFlowershow