Platforms and Backends
Currently the following platforms are supported:
- Operating system: Linux, MacOS (local backend only)
- Python: 3.6+
- Supported cluster management systems:
- Kubernetes (Tested with Google Kubernetes Engine on Google cloud)
We are interested in supporting other cluster management systems like Slurm, if you want to contribute to it please let us know.
Different fiber backends¶
Fiber provides 3 built-in backends currently:
This backend is the default backend used by Fiber. It starts new processes
locally on your computer. The child process shares the same running environment
as the parent process. When using this backend, Fiber works the same as
local backend works on Linux and MacOS.
This backend starts new processes with docker containers. This backend still
works locally but compared to
docker backend runs child
processes in a completed isolated docker environment. You need to build a
docker image for your project before running processes with this backend.
Examples of using this backend can be found
backend is usually used as a way to test your containerized application before
you run everything on a large scale on a computer cluster.
docker backend currently only works on Linux and doesn't work on MacOS,
we will add support for it soon.
This backend is the backend to use when you want to run your application on Kubernetes cluster. When starting new processes, it creates a new Kubernetes pod for it. Each process also runs in an isolated environment. You also need to build a docker image and push it to the appropriate container image registry for your Kubernetes platform. Example usage of this backend can be found here.
See how to configure Fiber to use each backend here
Automatic backend selection¶
Fiber also supports automatically detect which backend to use when no backend
is explicitly set. When fiber runs on a non-cluster environment, it uses
backend by default. When it runs inside a Kubernetes pod, it will select
kubernetes as the backend to use.