

fiber.experimental.ring

Ring allows you to create a Ring like topology easily on a computer cluster.

RingNode

RingNode(self, rank)

A RingNode represents a node in the Ring.

Arguments:

• rank: The id assigned to this node. Each node will be assigned a unique id called rank. Rank 0 is the control node of the Ring.

  Ring

Ring(self, processes, func, initializer, initargs=None)

Create a ring of nodes (processes). Each node runs a copy of the same function.

Arguments:

• processes: Number of ring nodes in this Ring.
• func: the target function that each ring node will run.
• initializer: the initialization function that each ring node runs. This is used to setup the custom ring structure like PyTorch's torch.distributed, Horovod, etc.
• initargs: positional arguments that are passed to initializer. Currently this is not used.

  run

Ring.run()

Start this Ring. This will start the ring 0 process on the same machine and start all the other ring nodes with Fiber processes.