Troubleshoot issues with ONNX Runtime generate() API
Installation issues
Windows Conda import error
ImportError: DLL load failed while importing onnxruntime_genai: A dynamic link library (DLL) initialization routine failed.
If you see this issue in a Conda environment on Windows, you need to upgrade the C++ runtime for Visual Studio
. In the conda environment, run the following command:
conda install conda-forge::vs2015_runtime
The onnxruntime-genai Python package should run without error after this extra step.
Windows CUDA import error
After CUDA toolkit installation completed on windows, ensure that the CUDA_PATH
system environment variable has been set to the path where the toolkit was installed. This variable will be used when importing the onnxruntime_genai python module on Windows. Unset or incorrectly set CUDA_PATH
variable may lead to a DLL load failed while importing onnxruntime_genai
.