Celery

Celery is a task queue implementation for Python web applications used to asynchronously execute work outside the HTTP request-response cycle.

Celery task queue project logo.

Celery is an implementation of the task queue concept. Learn more in the web development chapter or view the table of contents for all topics.

Why is Celery useful?

Task queues and the Celery implementation in particular are one of the trickier parts of a Python web application stack to understand.

If you are a junior developer it can be unclear why moving work outside the HTTP request-response cycle is important. In short, you want your WSGI server to respond to incoming requests as quickly as possible because each request ties up a worker process until the response is finished. Moving work off those workers by spinning up asynchronous jobs as tasks in a queue is a straightforward way to improve WSGI server response times.

What's the difference between Celeryd and Celerybeat?

Celery can be used to run batch jobs in the background on a regular schedule. A key concept in Celery is the difference between the Celery daemon (celeryd), which executes tasks, Celerybeat, which is a scheduler. Think of Celeryd as a tunnel-vision set of one or more workers that handle whatever tasks you put in front of them. Each worker will perform a task and when the task is completed will pick up the next one. The cycle will repeat continously, only waiting idly when there are no more tasks to put in front of them.

Celerybeat on the other hand is like a boss who keeps track of when tasks should be executed. Your application can tell Celerybeat to execute a task at time intervals, such as every 5 seconds or once a week. Celerybeat can also be instructed to run tasks on a specific date or time, such as 5:03pm every Sunday. When the interval or specific time is hit, Celerybeat will hand the job over to Celeryd to execute on the next available worker.

Celery tutorials and advice

Celery is a powerful tool that can be difficult to wrap your mind around at first. Be sure to read up on task queue concepts then dive into these specific Celery tutorials.

Celery with web frameworks

Celery is typically used with a web framework such as Django, Flask or Pyramid. These resources show you how to integrate the Celery task queue with the web framework of your choice.

Celery deployment resources

Celery and its broker run separately from your web and WSGI servers so it adds some additional complexity to your deployments. The following resources walk you through how to handle deployments and get the right configuration settings in place.

您想了解有关任务队列或其他主题的更多信息吗?

如何在 HTTP 请求 - 响应周期之外执行代码?

我已经创建了一个 Python Web 应用,那我该如何部署呢?

有哪些工具可用于监控已部署的 Web 应用程序?

Sign up for a monthly email with Full Stack Python tutorials. No spam ever.

Sponsored By

Rollbar logo

Fix errors in your Python code before your users see them by monitoring with Rollbar.

Learn More Python

Introduction to Ansible video course logo.

Deploy web apps with the Ansible configuration management tool.


TestDriven logo

Build microservices with Docker, Flask & React in this great course.

Full Stack Python

Full Stack Python is an open book that explains concepts in plain language and provides helpful resources for those topics.
Updates via newsletter, Twitter & Facebook.