Changelog

All notable changes to this project will be documented in this file.

Unreleased

2.0.0 – 2025-11-18

Breaking Changes

Major Breaking Changes

These are breaking changes we believe are most likely to effect your project.

  • The fail_fast argument to StubBroker.join now defaults to True. This means that calling StubBroker.join, will by default, re-raise any Exceptions that caused messages to get dead-lettered (i.e. any uncaught Exceptions in your actor functions). You may need to explicitly catch these exception in your tests (e.g. with unittest.TestCase.assertRaises() or pytest.raises()).

    Alternatively, you can revert to the old behavior by passing fail_fast_default=False to StubBroker.

    However, we think the new default behavior is best, because it makes exceptions happening in your actor functions obvious in your tests. Previsouly, exceptions in your actor functions could pass silently, and potentially unnoticed unless you checked the side-effects of the actor. (#739, #758, @LincolnPuzey)

  • The Prometheus middleware is no longer in the default middleware list. To keep exporting the Prometheus statistics, you must now install the prometheus extra (e.g. pip install 'dramatiq[prometheus]') and add the Prometheus middleware (see Customizing Middleware). If you are not using the Promotheus statistics, no action is needed. (#95, #345, #688, @azmeuk)

  • The backend argument to the Results middleware is now required. Previously, not supplying this argument would result in a non-functional Results middleware. (#728, @LincolnPuzey)

Minor Breaking Changes

These are changes that while technically breaking, we believe are unlikely to effect your project.

  • Removed URLRabbitmqBroker. This has been deprecated since version 1.1.0. Use RabbitmqBroker with url as a keyword argument instead. (#786, @LincolnPuzey)

  • Removed the dev installable extra from the project metadata. Instead the development dependencies are defined in a PEP-735 [dependency-groups] table in pyproject.toml. These can be installed in development environments with pip install --group dev. (#766, @LincolnPuzey)

  • The keys argument to RateLimiterBackend.incr_and_sum must now be a callable that returns a list of keys. This is only relevant if you have written custom code (such as a custom Rate Limiter) that uses a RateLimiterBackend. (#741, #772, @mikeroll)

  • The deprecated requeue_deadline and requeue_interval arguments of RedisBroker have been removed. These have been deprecated and have had no effect since version 1.2.0. (#782, @mikeroll)

  • RedisBroker: The code for compatibility with pre-v1.2.0 acks has been removed. If you are using the RedisBroker, you must first upgrade Dramatiq to a version >=1.2.0, <2.0.0, and run it for some time, before upgrading to a version >=2.0.0. This is to allow Dramatiq to migrate the pre-v1.2.0 ack data structures in redis to the v1.2.0+ versions. This migration code is what has been removed in version 2.0.0. (#771, @mikeroll)

Fixed

  • Fixed the way that backoff time is calculated for retries so that the first backoff is not less than min_backoff. This means that each retry backoff is now, on average, twice as long. To avoid this effect, you can halve your min_backoff (which should now be correctly observed). (#651, #721, @LincolnPuzey)

  • Fixed the RabbitMQ broker making 1 more retry than configured when declaring queues or enqueueing messages fail due to connection errors. 1 fewer retries are now made. This fixes a regression introduced by #669 released in 1.18.0. (#734, @LincolnPuzey)

  • When adding middleware to a Broker that already has actors declared, call the after_declare_actor middleware hook with the correct argument. (#743, @jenstroeger)

  • Made dramatiq robust against non-numeric values for the eta option. This should only be necessary when manually enqueuing messages. (#759, #761, @gurelkaynak)

  • Fixed edge case where the StubBroker.join would try to raise None as an exception. (#763, @LincolnPuzey)

Added

  • Add Python 3.14 to test matrix and project classifiers. (#751, @LincolnPuzey) Things to be aware of when upgrading to Python 3.14:

    • Python 3.14 changes the default multiprocessing start method to forkserver on some platforms. Dramatiq Workers use multiprocessing and will be effected by this change. forkserver should be less bug-prone than the old default fork. However, if you run into weird issues, using the existing --use-spawn flag when starting Dramatiq to set the start method to spawn, might solve them.

    • The free-threaded build of Python is now officially supported by Python. Dramatiq is not yet unit-tested with free-threaded Python, but we hope to do so soon. In the meantime, if you have any success or problems running Dramatiq with free-threaded Python, we would love to hear about it.

  • Added type annotations for the external API of the Worker and Broker classes. (#727, #731, #744, @jenstroeger)

  • Added type annotations for the external API of the Middleware class and its subclasses. (#521, #735, @jenstroeger)

  • Added message_datetime property to the Message class to retrieve message_timestamp as an aware datetime.datetime instance. (#736, @karolinepauls)

  • Added dramatiq_worker_timeout environment variable. (#773, @ksoviero-zengrc)

Changed

  • Promoted ConsumerThread and WorkerThread classes to public names, since they are used in the Middleware interface type hints. The previous names _ConsumerThread and _WorkerThread are still available for backwards compatibility. (#760, @synweap15)

  • Increased the minimum redis-py library version to 4.0.0. (#738, #764, @LincolnPuzey)

Removed

Packaging

  • The project now defines a PEP-518 build backend in the pyproject.toml file. (#750, @LincolnPuzey)

Documentation

1.18.0 – 2025-05-29

Fixed

Added

  • Add repr to MessageProxy class. (#690, @karolinepauls)

  • Add --worker-fork-timeout command-line argument to configure time to wait for the worker processes to come online after forking. (#706, @guedesfelipe)

  • Log a warning when added a duplicate middleware class, since this can lead to unexpected behavior. (#709, @synweap15)

Changed

  • Extend version limit on redis-py to version 6.X. (#711, @dbowring)

Removed

  • Remove support for end-of-life Python 3.8. (#662, @amureki)

Documentation

  • Add Podcatcher sponsor.

  • Add link to dramatiq-workflow. (#667, @pencil)

  • Update Sentry docs to reference sentry-sdk. (#675, @DHUKK)

1.17.1 – 2024-10-26

Fixed

Added

  • The Retries middleware now supports an on_retries_exhausted target actor to execute when retries on a message have been exhausted. (#630, @dbowring)

Changed

  • The actor decorator now checks for duplicate actor names. (#640, #641, @z0z0r4)

  • The Retries middleware now tracks when a message was last requeued on retry. (#629, @kuba-lilz)

1.17.0 – 2024-05-09

Added

Changed

  • The =watch= extras now require Watchdog version 4.0+.

Fixed

1.16.0 – 2024-01-25

Fixed

  • The CurrentMessage middleware now works under AsyncIO. (#586, #593, @pahrohfit)

  • Improved logging behavior under different buffer modes. (#596, @5tefan)

Added

  • CLI watcher now supports setting include and exclude patterns (#594, @nhairs)

1.15.0 – 2023-10-23

Fixed

Added

Changed

  • Filesystem watcher no longer reloads dramatiq on file open events. (#552, @seanpile)

  • The version bound on redis-py has been increased to include version 5.0. (#567, @scott-8)

1.14.2 – 2023-03-25

Fixed

  • Restored namedtuple instance methods on Message. (#538)

1.14.1 – 2023-02-25

Fixed

  • Added missing py.typed file to distributions. (#531)

1.14.0 – 2023-02-05

Removed

  • Dropped Python 3.6 support as it reached end-of-life

Changed

  • Added Python 3.11 support to CI builds. (#511, @FinnLidbetter)

  • Improved typing support. Message is now a dataclass, but it should be compatible with the previous namedtuple-based implementation. (#512, #513, #515, #516, @orsinium)

1.13.0 – 2022-04-02

Fixed

Changed

  • Typing support has been improved. (#482, @staticdev)

  • The default broker now falls back to Redis if the RabbitMQ extras are not installed, in an attempt to make the getting started process easier. (#483, #486, @kurtmckee)

1.12.3 – 2022-01-16

Fixed

1.12.2 – 2022-01-14

Fixed

  • An issue where stopping the process too quickly after boot could lead to an AttributeError. (#463, #464, @FinnLidbetter)

1.12.1 – 2021-12-19

Fixed

  • Actors and messages can now specify 0 backoff. (@FinnLidbetter, #438)

  • An issue where Redis message ids could be put back onto the queue after ack/nack. (#442, #444)

Removed

  • Dropped Python 3.5 support as it reached end-of-life

1.12.0 – 2021-10-23

Added

Changed

  • The watchdog library is no longer being pinned to a specific version. (#428)

  • The redis broker now limits unpacks to half the size of the Lua stack. (#433, #434, @ethervoid)

Fixed

1.11.0 – 2021-05-22

Added

Changed

  • The RabbitMQ broker moves messages that fail to decode to the DLQ. (#375, @thomazthz)

Fixed

  • The Redis broker is now more defensive in how it handles re-enqueueing messages. This should fix a potential race condition where a worker could hit its heartbeat timeout but still end up re-enqueueing messages (thus ending up with the same message id enqueued multiple times). (#266, #395)

  • A code path that could lead to an unbound variable error has now been fixed. (#382)

  • Deleting the connection off of a RabbitMQ broker now correctly closes it and its associated channel (if any) before removing it from the broker. (#381, #384)

1.10.0 – 2020-12-21

Added

  • The RabbitMQ broker dead message TTL can now be configured via the dramatiq_dead_message_ttl environment variable. (#354, @evstratbg)

  • The CLI now supports referencing a callable to set up the broker on worker startup. (#350)

  • The --worker-shutdown-timeout flag. (#330, @mic47)

Changed

  • The CLI raises an error when the --watch flag is set on unsupported platforms. (#326, #328, @CaselIT)

Fixed

  • The CLI now returns code 1 when one of the workers is killed by an unhandled signal. (#334, @omegacoleman)

  • The results middleware now gracefully handles actor-not-found errors during nack. (#336, #337, @AndreCimander)

  • A memory bloat issue with tasks that raise exceptions. (#351)

  • CI on Windows. (#371, @gdvalle)

1.9.0 – 2020-06-08

Added

  • A custom Redis connection can now be passed to the Redis broker via the new client keyword argument. (#274, @davidt99)

  • Message priority can now be changed in before_enqueue hooks. (#313, @thomazthz)

  • Support for storing actor exceptions. (#156)

  • Support for silent Retries. (#295)

  • Support for expected exceptions via the throws actor option. (#303, @takhs91)

  • Support for changing the consumer class in the RabbitMQ and Redis brokers. (#316, @AndreCimander)

Changed

Fixed

  • A race condition during command line startup where the wrong exit codes could be returned when subprocesses failed. (#286)

  • A race condition between worker processes and fork processes during boot. (#297)

  • A logging race condition on Linux. (#171, #286)

  • fileno has been added to StreamablePipe. (#291, @takhs91)

1.8.1 – 2020-02-02

Fixed

  • An issue where an IndexError would be raised when multiple middlewre containing fork functions were defined. (#271)

1.8.0 – 2020-02-02

Added

  • Support for forking and running arbitrary functions (so-called “fork functions”). (#127, #230)

  • The --fork-function flag.

  • The --skip-logging flag. (#263, @whalesalad)

Fixed

Changed

  • The RabbitmqBroker now creates its queues lazily. (#163, #270, @timdrijvers)

  • The Prometheus middleware no longer depends on file locking to start its exposition server. Instead, it uses the new fork functions functionality to start the server in a separate, unique process. The middleware no longer takes any parameters. While this would normally be a breaking change, it appears those parameters were previously ignored anyway. (#127, #230)

1.7.0 – 2019-09-22

Added

Changed

  • Uncaught exceptions within workers are now logged as errors rather than warnings. (#221, @th0th)

1.6.1 – 2019-07-24

Added

Changed

  • Updated allowed version range for prometheus-client. (#219, @robinro)

1.6.0 – 2019-05-02

Added

  • dramatiq_queue_prefetch environment variable to control the number of messages to prefetch per worker process. (#183, #184, @xelhark)

  • The RabbitMQ broker now retries the queue declaration process if an error occurs. (#179, @davidt99)

  • Support for accessing nested broker instances from the CLI. (#191, @bersace)

  • Support for eagerly raising actor exceptions in the joining thread with the StubBroker. (#195, #203)

  • Support for accessing the current message from an actor via CurrentMessage. (#208)

Changed

  • Pinned pika version >1.0,<2.0. (#202)

Fixed

  • An issue where workers would fail and never recover after the connection to Redis was severed. (#207)

  • pipe_ignore has been fixed to apply to the next message in line within a pipeline. (#194, @metheoryt)

1.5.0 – 2019-02-18

Added

Changed

  • Pika 0.13 is now required.

Fixed

  • Consumers are now stopped after workers finish running their tasks. (#160, @brownan)

  • Worker logging on Python 3.7 is no longer delayed.

1.4.3 – 2019-01-08

Fixed

  • Changed license classifier to the correct license. This is why you shouldn’t publish changed before you’ve had coffee, folks!

1.4.2 – 2019-01-08

Fixed

  • License classifier in PyPI package. There were no source code changes for this release.

1.4.1 – 2018-12-30

Added

Fixed

  • Workers wait for RMQ messages to be acked upon shutdown. (#148)

  • Pipelines no longer continue when a message is failed. (#151, @davidt99)

  • Log files now work under Windows. (#141, @ryansm1)

1.4.0 – 2018-11-25

Added

Changed

  • cli.main now takes an optional argument namespace so that users may define their own entrypoints. (#140, @maerteijn)

  • Actor “message received” and “completed in x ms” log messages are now logged with the DEBUG level instead of INFO level. This improves throughput and makes logging much less verbose.

  • The TimeLimit middleware no longer uses signals to trigger time limit handling. Instead it uses a background thread per worker process.

  • Dramatiq now shuts itself down if any of the workers die unexpectedly (for example, if one of them is killed by the OOM killer).

  • Windows is now supported (with some caveats)! (#119, @ryansm1)

Fixed

  • Allow pipe_ignore option to be set at the actor level. (#100)

  • Result encoder now defaults to the global encoder. (#108, @xdmiodz)

  • Dot characters are now allowed in queue names. (#111)

  • Tests are now run on Windows. (#113, @ryansm1)

1.3.0 – 2018-07-05

Changed

  • Upgraded prometheus_client to 0.2.x.

  • Bumped pika to version 0.12. Because of this change, the interrupt method on Broker and its usages within Worker have been dropped.

  • There is no longer a max message delay.

Fixed

  • Brokers can now be passed an empty list of middleware. (#90)

  • Potential stack overflow when restarting Consumer threads. (#89)

1.2.0 – 2018-05-24

Added

Changed

Fixed

  • StubBroker.join and Worker.join are now more reliable.

  • Module import path is now prepended to search path rather than appended. This fixes an issue where importing modules with the same name as modules from site-packages would end up importing the modules from site-packages. (#88)

  • Prometheus middleware no longer wipes the prometheus data directory on startup. This fixes an issue with exporting application metrics along with worker metrics.

Deprecated

  • requeue_{deadline,interval} parameters to RedisBroker. These two parameters no longer have any effect.

1.1.0 – 2018-04-17

Added

  • confirm_delivery parameter to RabbitmqBroker.

  • dead_message_ttl, requeue_deadline and requeue_interval parameters to RedisBroker.

  • url parameter to Redis rate limiter backend.

  • url parameter to Redis result backend.

  • timeout parameter to all the brokers’ join methods. (#57)

  • flush and flush_all methods to RedisBroker. (#62)

  • flush and flush_all methods to RabbitmqBroker. (#62)

Changed

  • Cleaned up command line argument descriptions.

Deprecated

  • URLRabbitmqBroker is deprecated. The RabbitmqBroker takes a url parameter so use that instead. URLRabbitmqBroker will be removed in version 2.0.

Fixed

  • rabbitmq and watch extra dependencies are only installed when they are explicitly required now. (#60, @rpkilby)

  • signal handling from the master process on FreeBSD 10.3. (#66)

  • reloading now uses sys.executable when exec’ing workers that were started with python -m dramatiq.

  • an issue that caused logging to fail when non-utf-8 characters were printed to stdout/err. (#63)

  • an issue with potentially drifting keys in the WindowRateLimiter. (#69, @gdvalle)

1.0.0 – 2018-03-31

Added

Changed

  • Dramatiq is now licensed under the LGPL.

Fixed

  • Passing time_limit in send_with_options. (#44)

0.20.0 – 2018-03-17

Added

  • --queues CLI argument. (#35)

Changed

  • Unhandled errors within workers now print the full stack trace. (#42)

0.19.1 – 2018-03-08

Fixed

0.19.0 – 2018-01-17

Added

Changed

  • RateLimitExceeded errors no longer log the full stack trace when raised within workers.

  • Consumer connection errors no longer dump a stack trace.

  • Consumers now wait exactly 3 seconds between retries after a connection error, rather than using exponential backoff.

0.18.0 – 2018-01-06

Added

  • pip install dramatiq[all] installs all deps.

  • --path command line argument. (#27)

Changed

  • pip install dramatiq now installs RabbitMQ and watch deps.

0.17.0 – 2017-12-30

Added

Fixed

  • Pinned pika version 0.11 to avoid an issue where passing heartbeat to RabbitmqBroker in get_broker would raise a TypeError. (#23, @chen2aaron)

0.16.0 – 2017-12-25

Added

Changed

  • Redis result backend is now considerably more resource-efficient (it no longer polls).

  • sys.std{err,out} are now redirected to stderr and line-buffered.

Fixed

  • TimeLimit middleware now uses a monotonic clock.

0.15.1 – 2017-12-08

Fixed

  • Autoreload now works under gevent.

0.15.0 – 2017-11-24

Added

  • Support for Results.

  • pool parameter to the Memcached rate limiter backend.

  • client parameter to the Redis rate limiter backend.

  • --watch-use-polling command line argument.

Fixed

  • Fixed bad file descriptor issue during RMQ broker shutdown under gevent.

0.14.0 – 2017-11-21

Added

Fixed

0.13.1 – 2017-11-17

Changed

  • Connection and import errors that occur during process boot now log stack traces (@rakanalh).

  • Added support for Python 3.5 (#7 by @jssuzanne).

0.13.0 – 2017-11-15

Added

0.12.1 – 2017-11-15

Fixed

  • An AssertionError after starting the consumer if RabbitMQ is not running (#10).

0.12.0 – 2017-11-14

Added

Fixed

  • Pending interrupt messages are now removed from pika’s queue before cancel is called. This fixes an issue where an AtrributeError was sometimes raised on worker shutdown.

  • Pika connection reset logs from the main thread are now hidden.

  • Distribution of dramatiq-gevent (#2).

0.11.0 – 2017-11-09

Added

Changed

  • Consumer reconnect backoff factor has been lowered from 10s to 100ms.

  • URLRabbitmqBroker is now a factory function that creates instances of RabbitmqBroker.

Fixed

  • Worker processes no longer use a spinlock to consume messages.

  • Consumers now use the same idle timeout as workers.

  • StubBroker no longer declares dead letter queues.

0.10.2 – 2017-11-06

Changed

  • pika is now pinned to >=0.10,<0.12.

0.10.1 – 2017-11-04

Added

  • More benchmarks.

Fixed

0.10.0 – 2017-10-30

Added

Changed

  • Implementation of the window rate limiter has been streamlined.

  • Redis requeue is now more efficient.

  • RabbitMQ enqueue is now resilient to disconnects.

Fixed

0.9.0 – 2017-10-20

Changed

  • Messages are no longer assigned new ids when they are re-enqueued. This makes tracking messages using middleware significantly easier.

  • The RedisBroker now assigns its own internal message ids.

0.8.0 – 2017-10-19

Changed

  • RabbitmqBroker no longer takes a ConnectionParameters param as input. Instead, it builds one based on kwargs.

  • exec is now used to reload the main process on source code changes when the --watch flag is enabled.

0.7.1 – 2017-10-08

Fixed

  • Lua files are now properly distributed with the package.

0.7.0 – 2017-09-13

Changed

  • Reworked scheduled messages to improve fairness. Messages are now re-enqueued on the broker once they hit their eta.

  • prometheus-client has been pinned to version 0.0.20.

0.6.1 – 2017-07-20

Fixed

  • A race condition with calls to cas in the memcached rate limiter backend.

0.6.0 – 2017-07-09

Added

0.5.2 – 2017-06-29

Changed

  • Changed the default max retries value from None to 20, meaning tasks are now retried for up to about 30 days before they’re dead-lettered by default.

0.5.1 – 2017-06-28

Removed

  • Dropped RabbitMQ heartbeat to avoid spurious disconnects.

0.5.0 – 2017-06-27

Added

  • Added dramatiq-gevent script.

Changed

  • Capped prefetch counts to 65k.