From 624cada952c87bcf1f0e7d00b669f0cc483103eb Mon Sep 17 00:00:00 2001 From: Kevin Park Date: Thu, 2 Jul 2026 11:34:35 -0400 Subject: [PATCH] discover: fall back to standard CUDA when the JetPack runner is absent (#16949) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * discover: use the SBSA CUDA build on JetPack 7 (L4T r38+) JetPack 7 supports SBSA-based CUDA, so the standard cuda_v13 build — shipped in the base linux-arm64 package, and given the Orin arch (CC 8.7) in #16628 — runs on these devices. JETSON_JETPACK=7 previously selected a nonexistent jetpack7 runner, so runner.go skipped every CUDA library and discovery fell back to CPU. The L4T releases JetPack 7 uses (r38 on Thor, r39 on Orin) also hit the unrecognized branch, and install.sh warned the version was unsupported. Map JetPack 7+ (L4T r38 and newer) to cuda_v13 (returned as "" from cudaJetpack); no Jetson-specific download is needed, so install.sh no longer warns. Fixes #16602 * discover: fall back to standard CUDA when the JetPack runner is absent Per review, drop the L4T-version mapping (in cudaJetpack and install.sh) and instead clear the jetpack override in runner.go when the detected cuda_jetpack runner isn't installed. Normal discovery then selects the standard cuda_v13 build, which supports Orin (CC 8.7) on JetPack 7. --- discover/runner.go | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/discover/runner.go b/discover/runner.go index f0cfe8fc..2c533c33 100644 --- a/discover/runner.go +++ b/discover/runner.go @@ -67,6 +67,15 @@ func GPUDevices(ctx context.Context, runners []ml.FilteredRunnerDiscovery) []ml. requested := envconfig.LLMLibrary() jetpack := cudaJetpack() + // If the detected JetPack runner isn't installed, clear the override so + // normal discovery can select a standard CUDA build (e.g. cuda_v13, + // which supports Orin on JetPack 7). + if jetpack != "" { + if _, ok := libDirs[filepath.Join(ml.LibOllamaPath, "cuda_"+jetpack)]; !ok { + jetpack = "" + } + } + // For our initial discovery pass, we gather all the known GPUs through // all the libraries that were detected. This pass may include GPUs that // are enumerated, but not actually supported.