Changelog

All notable changes to this project will be documented in this file.

Unreleased

1.18.0 – 2025-05-29

Fixed

Added

  • Add repr to MessageProxy class. (#690, @karolinepauls)

  • Add --worker-fork-timeout command-line argument to configure time to wait for the worker processes to come online after forking. (#706, @guedesfelipe)

  • Log a warning when added a duplicate middleware class, since this can lead to unexpected behavior. (#709, @synweap15)

Changed

  • Extend version limit on redis-py to version 6.X. (#711, @dbowring)

Removed

  • Remove support for end-of-life Python 3.8. (#662, @amureki)

Documentation

  • Add Podcatcher sponsor.

  • Add link to dramatiq-workflow. (#667, @pencil)

  • Update Sentry docs to reference sentry-sdk. (#675, @DHUKK)

1.17.1 – 2024-10-26

Fixed

Added

  • The Retries middleware now supports an on_retries_exhausted target actor to execute when retries on a message have been exhausted. (#630, @dbowring)

Changed

  • The actor decorator now checks for duplicate actor names. (#640, #641, @z0z0r4)

  • The Retries middleware now tracks when a message was last requeued on retry. (#629, @kuba-lilz)

1.17.0 – 2024-05-09

Added

Changed

  • The =watch= extras now require Watchdog version 4.0+.

Fixed

1.16.0 – 2024-01-25

Fixed

  • The CurrentMessage middleware now works under AsyncIO. (#586, #593, @pahrohfit)

  • Improved logging behavior under different buffer modes. (#596, @5tefan)

Added

  • CLI watcher now supports setting include and exclude patterns (#594, @nhairs)

1.15.0 – 2023-10-23

Fixed

Added

Changed

  • Filesystem watcher no longer reloads dramatiq on file open events. (#552, @seanpile)

  • The version bound on redis-py has been increased to include version 5.0. (#567, @scott-8)

1.14.2 – 2023-03-25

Fixed

  • Restored namedtuple instance methods on Message. (#538)

1.14.1 – 2023-02-25

Fixed

  • Added missing py.typed file to distributions. (#531)

1.14.0 – 2023-02-05

Removed

  • Dropped Python 3.6 support as it reached end-of-life

Changed

  • Added Python 3.11 support to CI builds. (#511, @FinnLidbetter)

  • Improved typing support. Message is now a dataclass, but it should be compatible with the previous namedtuple-based implementation. (#512, #513, #515, #516, @orsinium)

1.13.0 – 2022-04-02

Fixed

Changed

  • Typing support has been improved. (#482, @staticdev)

  • The default broker now falls back to Redis if the RabbitMQ extras are not installed, in an attempt to make the getting started process easier. (#483, #486, @kurtmckee)

1.12.3 – 2022-01-16

Fixed

1.12.2 – 2022-01-14

Fixed

  • An issue where stopping the process too quickly after boot could lead to an AttributeError. (#463, #464, @FinnLidbetter)

1.12.1 – 2021-12-19

Fixed

  • Actors and messages can now specify 0 backoff. (@FinnLidbetter, #438)

  • An issue where Redis message ids could be put back onto the queue after ack/nack. (#442, #444)

Removed

  • Dropped Python 3.5 support as it reached end-of-life

1.12.0 – 2021-10-23

Added

Changed

  • The watchdog library is no longer being pinned to a specific version. (#428)

  • The redis broker now limits unpacks to half the size of the Lua stack. (#433, #434, @ethervoid)

Fixed

1.11.0 – 2021-05-22

Added

Changed

  • The RabbitMQ broker moves messages that fail to decode to the DLQ. (#375, @thomazthz)

Fixed

  • The Redis broker is now more defensive in how it handles re-enqueueing messages. This should fix a potential race condition where a worker could hit its heartbeat timeout but still end up re-enqueueing messages (thus ending up with the same message id enqueued multiple times). (#266, #395)

  • A code path that could lead to an unbound variable error has now been fixed. (#382)

  • Deleting the connection off of a RabbitMQ broker now correctly closes it and its associated channel (if any) before removing it from the broker. (#381, #384)

1.10.0 – 2020-12-21

Added

  • The RabbitMQ broker dead message TTL can now be configured via the dramatiq_dead_message_ttl environment variable. (#354, @evstratbg)

  • The CLI now supports referencing a callable to set up the broker on worker startup. (#350)

  • The --worker-shutdown-timeout flag. (#330, @mic47)

Changed

  • The CLI raises an error when the --watch flag is set on unsupported platforms. (#326, #328, @CaselIT)

Fixed

  • The CLI now returns code 1 when one of the workers is killed by an unhandled signal. (#334, @omegacoleman)

  • The results middleware now gracefully handles actor-not-found errors during nack. (#336, #337, @AndreCimander)

  • A memory bloat issue with tasks that raise exceptions. (#351)

  • CI on Windows. (#371, @gdvalle)

1.9.0 – 2020-06-08

Added

  • A custom Redis connection can now be passed to the Redis broker via the new client keyword argument. (#274, @davidt99)

  • Message priority can now be changed in before_enqueue hooks. (#313, @thomazthz)

  • Support for storing actor exceptions. (#156)

  • Support for silent Retries. (#295)

  • Support for expected exceptions via the throws actor option. (#303, @takhs91)

  • Support for changing the consumer class in the RabbitMQ and Redis brokers. (#316, @AndreCimander)

Changed

Fixed

  • A race condition during command line startup where the wrong exit codes could be returned when subprocesses failed. (#286)

  • A race condition between worker processes and fork processes during boot. (#297)

  • A logging race condition on Linux. (#171, #286)

  • fileno has been added to StreamablePipe. (#291, @takhs91)

1.8.1 – 2020-02-02

Fixed

  • An issue where an IndexError would be raised when multiple middlewre containing fork functions were defined. (#271)

1.8.0 – 2020-02-02

Added

  • Support for forking and running arbitrary functions (so-called “fork functions”). (#127, #230)

  • The --fork-function flag.

  • The --skip-logging flag. (#263, @whalesalad)

Fixed

Changed

  • The RabbitmqBroker now creates its queues lazily. (#163, #270, @timdrijvers)

  • The Prometheus middleware no longer depends on file locking to start its exposition server. Instead, it uses the new fork functions functionality to start the server in a separate, unique process. The middleware no longer takes any parameters. While this would normally be a breaking change, it appears those parameters were previously ignored anyway. (#127, #230)

1.7.0 – 2019-09-22

Added

Changed

  • Uncaught exceptions within workers are now logged as errors rather than warnings. (#221, @th0th)

1.6.1 – 2019-07-24

Added

Changed

  • Updated allowed version range for prometheus-client. (#219, @robinro)

1.6.0 – 2019-05-02

Added

  • dramatiq_queue_prefetch environment variable to control the number of messages to prefetch per worker process. (#183, #184, @xelhark)

  • The RabbitMQ broker now retries the queue declaration process if an error occurs. (#179, @davidt99)

  • Support for accessing nested broker instances from the CLI. (#191, @bersace)

  • Support for eagerly raising actor exceptions in the joining thread with the StubBroker. (#195, #203)

  • Support for accessing the current message from an actor via CurrentMessage. (#208)

Changed

  • Pinned pika version >1.0,<2.0. (#202)

Fixed

  • An issue where workers would fail and never recover after the connection to Redis was severed. (#207)

  • pipe_ignore has been fixed to apply to the next message in line within a pipeline. (#194, @metheoryt)

1.5.0 – 2019-02-18

Added

Changed

  • Pika 0.13 is now required.

Fixed

  • Consumers are now stopped after workers finish running their tasks. (#160, @brownan)

  • Worker logging on Python 3.7 is no longer delayed.

1.4.3 – 2019-01-08

Fixed

  • Changed license classifier to the correct license. This is why you shouldn’t publish changed before you’ve had coffee, folks!

1.4.2 – 2019-01-08

Fixed

  • License classifier in PyPI package. There were no source code changes for this release.

1.4.1 – 2018-12-30

Added

Fixed

  • Workers wait for RMQ messages to be acked upon shutdown. (#148)

  • Pipelines no longer continue when a message is failed. (#151, @davidt99)

  • Log files now work under Windows. (#141, @ryansm1)

1.4.0 – 2018-11-25

Added

Changed

  • cli.main now takes an optional argument namespace so that users may define their own entrypoints. (#140, @maerteijn)

  • Actor “message received” and “completed in x ms” log messages are now logged with the DEBUG level instead of INFO level. This improves throughput and makes logging much less verbose.

  • The TimeLimit middleware no longer uses signals to trigger time limit handling. Instead it uses a background thread per worker process.

  • Dramatiq now shuts itself down if any of the workers die unexpectedly (for example, if one of them is killed by the OOM killer).

  • Windows is now supported (with some caveats)! (#119, @ryansm1)

Fixed

  • Allow pipe_ignore option to be set at the actor level. (#100)

  • Result encoder now defaults to the global encoder. (#108, @xdmiodz)

  • Dot characters are now allowed in queue names. (#111)

  • Tests are now run on Windows. (#113, @ryansm1)

1.3.0 – 2018-07-05

Changed

  • Upgraded prometheus_client to 0.2.x.

  • Bumped pika to version 0.12. Because of this change, the interrupt method on Broker and its usages within Worker have been dropped.

  • There is no longer a max message delay.

Fixed

  • Brokers can now be passed an empty list of middleware. (#90)

  • Potential stack overflow when restarting Consumer threads. (#89)

1.2.0 – 2018-05-24

Added

Changed

Fixed

  • StubBroker.join and Worker.join are now more reliable.

  • Module import path is now prepended to search path rather than appended. This fixes an issue where importing modules with the same name as modules from site-packages would end up importing the modules from site-packages. (#88)

  • Prometheus middleware no longer wipes the prometheus data directory on startup. This fixes an issue with exporting application metrics along with worker metrics.

Deprecated

  • requeue_{deadline,interval} parameters to RedisBroker. These two parameters no longer have any effect.

1.1.0 – 2018-04-17

Added

  • confirm_delivery parameter to RabbitmqBroker.

  • dead_message_ttl, requeue_deadline and requeue_interval parameters to RedisBroker.

  • url parameter to Redis rate limiter backend.

  • url parameter to Redis result backend.

  • timeout parameter to all the brokers’ join methods. (#57)

  • flush and flush_all methods to RedisBroker. (#62)

  • flush and flush_all methods to RabbitmqBroker. (#62)

Changed

  • Cleaned up command line argument descriptions.

Deprecated

  • URLRabbitmqBroker is deprecated. The RabbitmqBroker takes a url parameter so use that instead. URLRabbitmqBroker will be removed in version 2.0.

Fixed

  • rabbitmq and watch extra dependencies are only installed when they are explicitly required now. (#60, @rpkilby)

  • signal handling from the master process on FreeBSD 10.3. (#66)

  • reloading now uses sys.executable when exec’ing workers that were started with python -m dramatiq.

  • an issue that caused logging to fail when non-utf-8 characters were printed to stdout/err. (#63)

  • an issue with potentially drifting keys in the WindowRateLimiter. (#69, @gdvalle)

1.0.0 – 2018-03-31

Added

Changed

  • Dramatiq is now licensed under the LGPL.

Fixed

  • Passing time_limit in send_with_options. (#44)

0.20.0 – 2018-03-17

Added

  • --queues CLI argument. (#35)

Changed

  • Unhandled errors within workers now print the full stack trace. (#42)

0.19.1 – 2018-03-08

Fixed

0.19.0 – 2018-01-17

Added

Changed

  • RateLimitExceeded errors no longer log the full stack trace when raised within workers.

  • Consumer connection errors no longer dump a stack trace.

  • Consumers now wait exactly 3 seconds between retries after a connection error, rather than using exponential backoff.

0.18.0 – 2018-01-06

Added

  • pip install dramatiq[all] installs all deps.

  • --path command line argument. (#27)

Changed

  • pip install dramatiq now installs RabbitMQ and watch deps.

0.17.0 – 2017-12-30

Added

Fixed

  • Pinned pika version 0.11 to avoid an issue where passing heartbeat to RabbitmqBroker in get_broker would raise a TypeError. (#23, @chen2aaron)

0.16.0 – 2017-12-25

Added

Changed

  • Redis result backend is now considerably more resource-efficient (it no longer polls).

  • sys.std{err,out} are now redirected to stderr and line-buffered.

Fixed

  • TimeLimit middleware now uses a monotonic clock.

0.15.1 – 2017-12-08

Fixed

  • Autoreload now works under gevent.

0.15.0 – 2017-11-24

Added

  • Support for Results.

  • pool parameter to the Memcached rate limiter backend.

  • client parameter to the Redis rate limiter backend.

  • --watch-use-polling command line argument.

Fixed

  • Fixed bad file descriptor issue during RMQ broker shutdown under gevent.

0.14.0 – 2017-11-21

Added

Fixed

0.13.1 – 2017-11-17

Changed

  • Connection and import errors that occur during process boot now log stack traces (@rakanalh).

  • Added support for Python 3.5 (#7 by @jssuzanne).

0.13.0 – 2017-11-15

Added

0.12.1 – 2017-11-15

Fixed

  • An AssertionError after starting the consumer if RabbitMQ is not running (#10).

0.12.0 – 2017-11-14

Added

Fixed

  • Pending interrupt messages are now removed from pika’s queue before cancel is called. This fixes an issue where an AtrributeError was sometimes raised on worker shutdown.

  • Pika connection reset logs from the main thread are now hidden.

  • Distribution of dramatiq-gevent (#2).

0.11.0 – 2017-11-09

Added

Changed

  • Consumer reconnect backoff factor has been lowered from 10s to 100ms.

  • URLRabbitmqBroker is now a factory function that creates instances of RabbitmqBroker.

Fixed

  • Worker processes no longer use a spinlock to consume messages.

  • Consumers now use the same idle timeout as workers.

  • StubBroker no longer declares dead letter queues.

0.10.2 – 2017-11-06

Changed

  • pika is now pinned to >=0.10,<0.12.

0.10.1 – 2017-11-04

Added

  • More benchmarks.

Fixed

0.10.0 – 2017-10-30

Added

Changed

  • Implementation of the window rate limiter has been streamlined.

  • Redis requeue is now more efficient.

  • RabbitMQ enqueue is now resilient to disconnects.

Fixed

0.9.0 – 2017-10-20

Changed

  • Messages are no longer assigned new ids when they are re-enqueued. This makes tracking messages using middleware significantly easier.

  • The RedisBroker now assigns its own internal message ids.

0.8.0 – 2017-10-19

Changed

  • RabbitmqBroker no longer takes a ConnectionParameters param as input. Instead, it builds one based on kwargs.

  • exec is now used to reload the main process on source code changes when the --watch flag is enabled.

0.7.1 – 2017-10-08

Fixed

  • Lua files are now properly distributed with the package.

0.7.0 – 2017-09-13

Changed

  • Reworked scheduled messages to improve fairness. Messages are now re-enqueued on the broker once they hit their eta.

  • prometheus-client has been pinned to version 0.0.20.

0.6.1 – 2017-07-20

Fixed

  • A race condition with calls to cas in the memcached rate limiter backend.

0.6.0 – 2017-07-09

Added

0.5.2 – 2017-06-29

Changed

  • Changed the default max retries value from None to 20, meaning tasks are now retried for up to about 30 days before they’re dead-lettered by default.

0.5.1 – 2017-06-28

Removed

  • Dropped RabbitMQ heartbeat to avoid spurious disconnects.

0.5.0 – 2017-06-27

Added

  • Added dramatiq-gevent script.

Changed

  • Capped prefetch counts to 65k.