Ring allows you to create a Ring like topology easily on a computer cluster.
RingNode represents a node in the
rank: The id assigned to this node. Each node will be assigned a unique id called
rank. Rank 0 is the control node of the
Ring(processes, func, initializer, initargs=None)
Create a ring of nodes (processes). Each node runs a copy of the same function.
processes: Number of ring nodes in this Ring.
func: the target function that each ring node will run.
initializer: the initialization function that each ring node runs. This is used to setup the custom ring structure like PyTorch's torch.distributed, Horovod, etc.
initargs: positional arguments that are passed to initializer. Currently this is not used.
Start this Ring. This will start the ring 0 process on the same machine and start all the other ring nodes with Fiber processes.