NCCL

What is NCCL

Please see the NVIDIA webpage.

Troubleshooting

To activate NCCL debug outputs, set the following variable before running NCCL:

NCCL_DEBUG=info

To fix Caught error during NCCL init [...] connect() timed out errors, set the following variable before running NCCL:

```bash export NCCL_BLOCKING_WAIT=1