discover: fall back to standard CUDA when the JetPack runner is absent (#16949)

* discover: use the SBSA CUDA build on JetPack 7 (L4T r38+)

JetPack 7 supports SBSA-based CUDA, so the standard cuda_v13 build — shipped
in the base linux-arm64 package, and given the Orin arch (CC 8.7) in #16628
— runs on these devices.

JETSON_JETPACK=7 previously selected a nonexistent jetpack7 runner, so
runner.go skipped every CUDA library and discovery fell back to CPU. The L4T
releases JetPack 7 uses (r38 on Thor, r39 on Orin) also hit the unrecognized
branch, and install.sh warned the version was unsupported. Map JetPack 7+
(L4T r38 and newer) to cuda_v13 (returned as "" from cudaJetpack); no
Jetson-specific download is needed, so install.sh no longer warns.

Fixes #16602

* discover: fall back to standard CUDA when the JetPack runner is absent

Per review, drop the L4T-version mapping (in cudaJetpack and install.sh) and
instead clear the jetpack override in runner.go when the detected cuda_jetpack
runner isn't installed. Normal discovery then selects the standard cuda_v13
build, which supports Orin (CC 8.7) on JetPack 7.
This commit is contained in:
Kevin Park
2026-07-02 11:34:35 -04:00
committed by GitHub
parent cecd265d3a
commit 624cada952
+9
View File
@@ -67,6 +67,15 @@ func GPUDevices(ctx context.Context, runners []ml.FilteredRunnerDiscovery) []ml.
requested := envconfig.LLMLibrary() requested := envconfig.LLMLibrary()
jetpack := cudaJetpack() jetpack := cudaJetpack()
// If the detected JetPack runner isn't installed, clear the override so
// normal discovery can select a standard CUDA build (e.g. cuda_v13,
// which supports Orin on JetPack 7).
if jetpack != "" {
if _, ok := libDirs[filepath.Join(ml.LibOllamaPath, "cuda_"+jetpack)]; !ok {
jetpack = ""
}
}
// For our initial discovery pass, we gather all the known GPUs through // For our initial discovery pass, we gather all the known GPUs through
// all the libraries that were detected. This pass may include GPUs that // all the libraries that were detected. This pass may include GPUs that
// are enumerated, but not actually supported. // are enumerated, but not actually supported.