Apache Spark 4.2.0 - High - Remote - File write - Connect archives hash-fragment dest
abraxaslabs.tech · github.com/abraxas · @abraxas_null · [email protected] · spark-connect-archive-fragment
Class: File write Reach: Remote
Apache Spark 4.2.0 (32f72996) - Apache Software Foundation
Spark Connect AddArtifacts names of the form archives/<file>#<fragment> split on the hash / number-sign character. The zip bytes go through concatenatePaths. The fragment does not. SparkContext.addFile(..., isArchive=true) then does new File(root, uri.getFragment), deleteRecursively(dest), and unpack on the driver. Executors build the same dest. SPARK-33530 documented #name as a directory name under SparkFiles. A fragment of ../<outside> is the dest.
Connect archives/<file>#../<path> unpacks and deleteRecursivelys outside the session SparkFiles dir as the Spark UID.
| ID | no CVE yet |
| Class | File write |
| Reach | Remote (Spark Connect client; AddArtifacts with archive=True) |
| CWE | CWE-22 |
| CVSS | High: 8.1 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:H/A:H |
| Product | Apache Spark |
| Affected | 4.2.0 (32f7299601108917fb01920a54e084595b7b3bf8) Connect archives/ hash-fragment dest |
| Auth | Connect client. Default Connect has no PSK (spark.connect.authenticate.token unset). PR:L if PSK / network ACL; PR:N on default open Connect. |
| License | GNU Affero GPL v3.0 |
| Lab | 127.0.0.1 only |
I pinned tag v4.2.0 peel 32f72996 (2026-07-11, v4.2.0-rc6). The lab is a Java gadget of the pinned StagedArtifact hash split, ArtifactUtils.concatenatePaths, new File(root, fragment), deleteRecursively, and dest-contained zip unpack. Loopback HTTP on 127.0.0.1:18930. Official PySpark already copies a URI fragment into the archive name.
This is not RCE. Zip members that walk out of dest are rejected (Hadoop FileUtil.unZip / HADOOP-13649 class). The leftover is the dest itself.
| Method | Sink | Privilege |
|---|---|---|
Connect AddArtifacts name archives/<file>#../<outside> | SparkContext.addFile dest new File(root, uri.getFragment) then Utils.unpack (SparkContext.scala:1891-1900) | Integrity (lab-proven): write archive members as the Spark UID outside session SparkFiles. Lab dest canonical /lab/oracle/SPARK-CONNECT-ARCHIVE-FRAGMENT-WITNESS, uid 1000, dest-outside=yes. |
| Same dest, before unpack | Utils.deleteRecursively(dest) (SparkContext.scala:1899) | Availability (lab-proven): dest canary gone. Lab deleteRecursively=yes; canary.txt under dest did not survive. Sibling /lab/oracle/canary-keep.txt kept. |
| Same primitive on each executor that fetches the archive | Executor dest new File(root, sourceURI.getFragment) then deleteRecursively + unpack (Executor.scala:1468-1477) | Integrity / availability (inferred): same dest construction on executors. Lab is the driver-side gadget. |
Artifact name files/../../... or archives/../../... | ArtifactUtils.concatenatePaths (ArtifactUtils.scala:26-39) | Closed (lab-proven): INVALID_ARTIFACT_PATH. |
Unix absolute fragment /abs | new File(root, "/abs") | Closed on Unix (lab-proven): parent is not ignored; dest joins under root. Lab dest /lab/root/lab/oracle/abs-..., outside=false, abs canary kept. |
Own-session jars/ / classes/ add | session URLClassLoader | Closed as a leftover: own-session JAR load is by design. Not this finding. |
| Cross-session jar load | Connect isolation spark.sql.artifact.isolation.enabled=true | Closed: isolated sessions read addedJars(sessionUUID). |
Lab-proven vs inferred: hash-fragment dest outside SparkFiles, dest canary deleted, sibling canary kept, files/ and archives/ name concat rejected, Unix absolute fragment joins under root, uid 1000, dest-contained zip unpack: lab. Executor fetch of the same dest: inferred (same constructor). Overwrite of a later-loaded class on the driver classpath: inferred chain, not demonstrated, not inflated to RCE.
Requires: a Connect client that can AddArtifacts with archive=True. Official PySpark spark.addArtifact(..., archive=True) already puts a non-empty URI fragment into the name (python/pyspark/sql/connect/client/artifact.py:223-236). Default Connect has no PSK.
| Surface | Lab |
|---|---|
Artifact name .. / absolute through concatenatePaths | files-concat=rejected INVALID_ARTIFACT_PATH |
archives/ name walk archives/../../... | same INVALID_ARTIFACT_PATH |
Unix new File(root, "/abs") | dest under root; abs oracle canary kept |
| Zip member zip-slip under a contained dest | gadget rejects members that escape dest (HADOOP-13649 class). Not this leftover. |
files/ path concat (no hash split) | fragment is not taken; name still concatenatePaths |
| Cross-session jar isolation | default on for Connect |
| Unauth SQL / Python on open Connect | by-design job submit (SECURITY.md FAQ). Not this leftover. |
| Job submit / ML model load | by design |
StagedArtifact splits only names that start with archives plus the server File.separator, and only on #:
val splits = name.split("#")
if (splits.length == 2) {
(splits(0), Some(splits(1)))
}
splits(0) is staged via concatenatePaths. splits(1) is kept as fragment. ArtifactManager attaches that fragment to the session file-server URI and calls SparkContext.addArchive. Dest is new File(root, uri.getFragment). SPARK-33530 tests sc.addArchive(s"${jarFile.getAbsolutePath}#foo") as a directory name under SparkFiles.
Hunt first pass claimed a Unix absolute fragment ignores root (java.io.File parent-ignore for absolute children). On Unix, new File(root, "/abs") strips the leading slash and joins under root. Parent is ignored only on Windows. The High walk on this pin is #../.
cd lab
./run.sh
Loopback 127.0.0.1:18930. Compose project spark-connect-archive-fragment. Image eclipse-temurin:21-jdk-jammy, runtime uid 1000. Witness SPARK-CONNECT-ARCHIVE-FRAGMENT-WITNESS. Last line:
SUCCESS SPARK-CONNECT-ARCHIVE-FRAGMENT how=gadget dest-outside=yes deleteRecursively=yes files-concat=rejected uid=1000 SPARK-CONNECT-ARCHIVE-FRAGMENT-WITNESS
CONTROL: archives/payload.zip unpacks under /lab/root. ATTACK: archives/payload-frag.zip#../oracle/SPARK-CONNECT-ARCHIVE-FRAGMENT-WITNESS dest canonical /lab/oracle/..., canary deleted, pwned.txt written as uid 1000. Unix absolute fragment joins under root. Stack tears down on EXIT.
Pass the archive fragment through the same stay-under-root check as the artifact name (concatenatePaths, or reject .. / absolute). Keep SPARK-33530 #name as a single directory component under SparkFiles. Do not deleteRecursively a dest that failed that check. Executors use the same constructor; fix both.