The Circus package is composed of a high-level get_arbiter() function and many classes. In most cases, using the high-level function should be enough, as it creates everything that is needed for Circus to run.
You can subclass Circus’ classes if you need more granularity than what is offered by the configuration.
get_arbiter() is just a convenience on top of the various circus classes. It creates a arbiter (class Arbiter) instance with the provided options, which in turn runs a single Watcher with a single Process.
Example:
from circus import get_arbiter
arbiter = get_arbiter({"cmd": "myprogram", "numprocesses": 3})
try:
arbiter.start()
finally:
arbiter.stop()
Circus provides a series of classes you can use to implement your own process manager:
Wraps a process.
Options:
Return the pid
Return the stdout stream
Return the stdout stream
Sends a signal sig to the process.
Terminate the process.
Return the age of the process in seconds.
Return process info.
The info returned is a mapping with these keys:
Return a list of children pids.
Return True is the given pid is a child of that process.
Send signal signum to child pid.
Send signal signum to all children.
Return the process status as a constant
Example:
>>> from circus.process import Process
>>> process = Process('Top', 'top', shell=True)
>>> process.age()
3.0107998847961426
>>> process.info()
'Top: 6812 N/A tarek Zombie N/A N/A N/A N/A N/A'
>>> process.status
1
>>> process.stop()
>>> process.status
2
>>> process.info()
'No such process (stopped?)'
Class managing a list of processes for a given command.
Options:
name: name given to the watcher. Used to uniquely identify it.
cmd: the command to run. May contain $WID, which will be replaced by wid.
args: the arguments for the command to run. Can be a list or a string. If args is a string, it’s splitted using shlex.split(). Defaults to None.
numprocesses: Number of processes to run.
working_dir: the working directory to run the command in. If not provided, will default to the current working directory.
shell: if True, will run the command in the shell environment. False by default. warning: this is a security hazard.
uid: if given, is the user id or name the command should run with. The current uid is the default.
gid: if given, is the group id or name the command should run with. The current gid is the default.
send_hup: if True, a process reload will be done by sending the SIGHUP signal. Defaults to False.
env: a mapping containing the environment variables the command will run with. Optional.
rlimits: a mapping containing rlimit names and values that will be set before the command runs.
stdout_stream: a mapping that defines the stream for the process stdout. Defaults to None.
Optional. When provided, stdout_stream is a mapping containing up to three keys:
This mapping will be used to create a stream callable of the specified class. Each entry received by the callable is a mapping containing:
stderr_stream: a mapping that defines the stream for the process stderr. Defaults to None.
Optional. When provided, stderr_stream is a mapping containing up to three keys: - class: the stream class. Defaults to circus.stream.FileStream - filename: the filename, if using a FileStream - refresh_time: the delay between two stream checks. Defaults
to 0.3 seconds.
This mapping will be used to create a stream callable of the specified class.
Each entry received by the callable is a mapping containing:
priority – integer that defines a priority for the watcher. When the Arbiter do some operations on all watchers, it will sort them with this field, from the bigger number to the smallest. (default: 0)
singleton – If True, this watcher has a single process. (default:False)
use_sockets – If True, the processes will inherit the file descriptors, thus can reuse the sockets opened by circusd. (default: False)
on_demand – If True, the processes will be started only at the first connection to the socket (default: False)
copy_env – If True, the environment in which circus is running run will be reproduced for the workers. (default: False)
copy_path – If True, circusd sys.path is sent to the process through PYTHONPATH. You must activate copy_env for copy_path to work. (default: False)
max_age: If set after around max_age seconds, the process is replaced with a new one. (default: 0, Disabled)
max_age_variance: The maximum number of seconds that can be added to max_age. This extra value is to avoid restarting all processes at the same time. A process will live between max_age and max_age + max_age_variance seconds.
hooks: callback functions for hooking into the watcher startup and shutdown process. hooks is a dict where each key is the hook name and each value is a 2-tuple with the name of the callable or the callabled itself and a boolean flag indicating if an exception occuring in the hook should not be ignored. Possible values for the hook name: before_start, after_start, befor_spawn, before_stop, after_stop.
options – extra options for the worker. All options found in the configuration file for instance, are passed in this mapping – this can be used by plugins for watcher-specific options.
respawn – If set to False, the processes handled by a watcher will not be respawned automatically. (default: True)
virtualenv – The root directory of a virtualenv. If provided, the watcher will load the environment for its execution. (default: None)
close_child_stdout: If True, closes the stdout after the fork. default: False.
close_child_stderr: If True, closes the stderr after the fork. default: False.
Publish a message on the event publisher channel
Reap all the processes for this watcher.
Manage processes.
Reap & manage processes.
Spawn processes.
Spawn process.
Kill process.
Kill all the processes of this watcher.
Send signal to a child.
Stop.
Start.
Restart.
Class used to control a list of watchers.
Options:
watchers – a list of Watcher objects
endpoint – the controller ZMQ endpoint
pubsub_endpoint – the pubsub endpoint
statsd – If True, a circusd-stats process is run (default: False)
stats_endpoint – the stats endpoint.
statsd_close_outputs – if True sends the circusd-stats stdout/stderr to /dev/null (default: False)
multicast_endpoint – the multicast endpoint for circusd cluster auto-discovery (default: udp://237.219.251.97:12027) Multicast addr should be between 224.0.0.0 to 239.255.255.255 and the same for the all cluster.
check_delay – the delay between two controller points (default: 1 s)
prereload_fn – callable that will be executed on each reload (default: None)
context – if provided, the zmq context to reuse. (default: None)
to reuse. (default: None)
plugins – a list of plugins. Each item is a mapping with:
- use – Fully qualified name that points to the plugin class
- every other value is passed to the plugin in the config option
sockets – a mapping of sockets. Each key is the socket name, and each value a CircusSocket class. (default: None)
warmup_delay – a delay in seconds between two watchers startup. (default: 0)
httpd – If True, a circushttpd process is run (default: False)
httpd_host – the circushttpd host (default: localhost)
httpd_port – the circushttpd port (default: 8080)
httpd_close_outputs – if True, sends circushttpd stdout/stderr to /dev/null. (default: False)
debug – if True, adds a lot of debug info in the stdout (default: False)
proc_name – the arbiter process name
instance on the cluster.
Starts all the watchers.
The start command is an infinite loop that waits for any command from a client and that watches all the processes and restarts them if needed.
Reloads everything.
Run the prereload_fn() callable if any, then gracefuly reload all watchers.
Return the number of processes running across all watchers.
Return the number of watchers.
Return the watcher name.
Adds a watcher.
Options: