This is also recommended to ensure your exact architecture is used. It will also make the build faster when you only build for a few architectures. You can also change it just for a specific run with: Unique directory and DeepSpeed will use it to save and load CUDA extensions.

So in each virtual environment you can point it to a TORCH_EXTENSIONS_DIR environment variable. Therefore, if you need to you can override the default location with the help of the Python or cuda versions) and then the loading of a CUDA extension built by another environment willįail. Torch_extensions directory, but different virtual environments may use different setups (e.g., different If you use multiple virtual environments this could be a problem, since by default there is only one The next time the same program is executed these now precompiledĮxtensions will be loaded form that directory.

When the code using DeepSpeed is used for the first time it’ll automatically build only the CUDAĮxtensions, required for the run, and by default it’ll place them under The nodes listed in your hostfile (either given via -hostfile, or defaults to This will build a python wheel locally and copy it to all For installs spanning multiple nodes we find it useful to install DeepSpeed

Advanced installer tutorial