discover: fall back to standard CUDA when the JetPack runner is absent (#16949)
* discover: use the SBSA CUDA build on JetPack 7 (L4T r38+) JetPack 7 supports SBSA-based CUDA, so the standard cuda_v13 build — shipped in the base linux-arm64 package, and given the Orin arch (CC 8.7) in #16628 — runs on these devices. JETSON_JETPACK=7 previously selected a nonexistent jetpack7 runner, so runner.go skipped every CUDA library and discovery fell back to CPU. The L4T releases JetPack 7 uses (r38 on Thor, r39 on Orin) also hit the unrecognized branch, and install.sh warned the version was unsupported. Map JetPack 7+ (L4T r38 and newer) to cuda_v13 (returned as "" from cudaJetpack); no Jetson-specific download is needed, so install.sh no longer warns. Fixes #16602 * discover: fall back to standard CUDA when the JetPack runner is absent Per review, drop the L4T-version mapping (in cudaJetpack and install.sh) and instead clear the jetpack override in runner.go when the detected cuda_jetpack runner isn't installed. Normal discovery then selects the standard cuda_v13 build, which supports Orin (CC 8.7) on JetPack 7.
This commit is contained in:
@@ -67,6 +67,15 @@ func GPUDevices(ctx context.Context, runners []ml.FilteredRunnerDiscovery) []ml.
|
||||
requested := envconfig.LLMLibrary()
|
||||
jetpack := cudaJetpack()
|
||||
|
||||
// If the detected JetPack runner isn't installed, clear the override so
|
||||
// normal discovery can select a standard CUDA build (e.g. cuda_v13,
|
||||
// which supports Orin on JetPack 7).
|
||||
if jetpack != "" {
|
||||
if _, ok := libDirs[filepath.Join(ml.LibOllamaPath, "cuda_"+jetpack)]; !ok {
|
||||
jetpack = ""
|
||||
}
|
||||
}
|
||||
|
||||
// For our initial discovery pass, we gather all the known GPUs through
|
||||
// all the libraries that were detected. This pass may include GPUs that
|
||||
// are enumerated, but not actually supported.
|
||||
|
||||
Reference in New Issue
Block a user