fiber.experimental.ring

Ring allows you to create a Ring like topology easily on a computer cluster.

RingNode

RingNode(rank)

A RingNode represents a node in the Ring.

Arguments:

  • rank: The id assigned to this node. Each node will be assigned a unique id called rank. Rank 0 is the control node of the Ring.

    Ring

Ring(processes, func, initializer, initargs=None)

Create a ring of nodes (processes). Each node runs a copy of the same function.

Arguments:

  • processes: Number of ring nodes in this Ring.
  • func: the target function that each ring node will run.
  • initializer: the initialization function that each ring node runs. This is used to setup the custom ring structure like PyTorch's torch.distributed, Horovod, etc.
  • initargs: positional arguments that are passed to initializer. Currently this is not used.

    run

Ring.run()

Start this Ring. This will start the ring 0 process on the same machine and start all the other ring nodes with Fiber processes.