abraxas/spark-proxyuser-classpathApache Spark 4.2.0 - High - Remote - Privilege leftover - proxy-user defaultExtraClassPath
abraxaslabs.tech · github.com/abraxas · @abraxas_null · [email protected] · spark-proxyuser-classpath
Class: Privilege leftover (leftover of CVE-2023-22946 / SPARK-41958) Reach: Remote
Apache Spark 4.2.0 (32f72996) - Apache Software Foundation
CVE-2023-22946 / SPARK-41958 still defaults spark.submit.proxyUser.allowCustomClasspathInClusterMode to false and still runs childClasspath.clear() on --proxy-user cluster submit. That is the public YARN --jars path. Spark 4.0 then added spark.driver.defaultExtraClassPath. The launcher always prepends that key to the spark-submit JVM -cp, including cluster mode. SPARK-41958 never sees it.
A Livy-style tenant can still load a staged jar inside the gateway spark-submit JVM. Hadoop UserGroupInformation.doAs does not change that Unix UID.
| ID | no CVE yet |
| Class | Privilege leftover (leftover of CVE-2023-22946 / SPARK-41958) |
| Reach | Remote (Livy-style --proxy-user job submit over the gateway) |
| CWE | CWE-269, CWE-427 |
| CVSS | High: 8.8 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H |
| Product | Apache Spark |
| Affected | Apache Spark 4.2.0 (32f7299601108917fb01920a54e084595b7b3bf8) |
| Auth | authenticated tenant of a Livy / Kyuubi / Oozie-style gateway that forwards --conf and staged --jars into spark-submit --proxy-user ... --deploy-mode cluster |
| License | GNU Affero GPL v3.0 |
| Lab | 127.0.0.1:18950 only. Witness file. Gadget of the cited launcher / submit walks. |
Hold a tenant account on a multi-tenant Spark gateway (Livy, Kyuubi, Oozie, and friends) that runs spark-submit --proxy-user <tenant> --deploy-mode cluster as a superuser UID. Stage a jar the gateway JVM can open. Send --conf spark.driver.defaultExtraClassPath=<staged jar>. The SPARK-41958 gate can stay false. The constructed spark-submit -cp still starts with that jar. Classes in it load as the gateway Unix UID.
--conf spark.submit.proxyUser.allowCustomClasspathInClusterMode=true is a second walk: .internal() is docs-only, --conf still writes the key, and YARN cluster then keeps --jars on childClasspath. Public CVE-2023-22946 remains the YARN default --jars path that clear() already covers. This leftover is ungated defaultExtraClassPath.
Hadoop impersonation after a clean submit is still the proxy user for the YARN AM / k8s driver. This leftover is the gateway JVM only.
| Method | Sink | Recon | Integrity | Availability | Provenance |
|---|---|---|---|---|---|
Livy-style submit --proxy-user + yarn/k8s/standalone cluster + --conf spark.driver.defaultExtraClassPath=<staged local jar>, SPARK-41958 flag left false | SparkSubmitCommandBuilder.buildSparkSubmitCommand SparkSubmitCommandBuilder.java:317-327 -cp first entry | List/read files the gateway UID can read (keytab dir, spark-defaults, local staging). Hadoop identity of later RPCs is still the proxy user. | Load tenant classes inside the spark-submit JVM. Unix UID stays the gateway (Hadoop proxy superuser). Lab: defaultExtraClassPath-on-cp=yes childClasspath-cleared=yes lab.WitnessMain wrote SPARK-PROXYUSER-CLASSPATH-WITNESS. Inferred from the public CVE class: Hadoop Configuration resource overlay / FileSystem impl lookup as that UID. | Submit JVM crash if the overlay throws. Not the class. | Lab-proven load + witness write as constructed -cp. Hadoop overlay inferred. Livy/Kyuubi --conf passthrough inferred (those trees are not this pin). |
Same submit with --conf spark.submit.proxyUser.allowCustomClasspathInClusterMode=true plus --jars / primary resource | SparkSubmit.prepareSubmitEnvironment SparkSubmit.scala:333-336,738-743,917-924 then runMain :980-987 | Same recon as walk A, on the child URLClassLoader rather than the system loader. | Keep --jars on childClasspath so addJarToClasspath runs. Lab: conf-flag-keeps-jars=yes. | Same. | Lab-proven childClasspath keep. Not a re-file of the public YARN --jars path that clear() covers when the flag stays false. |
Lab-proven vs inferred: constructed -cp contains defaultExtraClassPath in cluster mode; childClasspath.clear() still runs on the default flag; spark.driver.extraClassPath is omitted in cluster; --jars with flag false is cleared; a witness class on that -cp writes the oracle. Hadoop config-class execution is the public CVE-2023-22946 mechanism, not re-run here. Livy/Kyuubi forwarding --conf is the architecture those projects document.
| Method | Sink | Stays closed |
|---|---|---|
--driver-class-path / spark.driver.extraClassPath in cluster mode | SparkSubmitCommandBuilder.java:318-319 | Not added to the spark-submit -cp. Lab: extraClassPath-omitted=yes. Goes to the cluster driver, which YARN/K8s start as the proxy user. |
Default cluster + --proxy-user + --jars without flipping the flag and without defaultExtraClassPath | SparkSubmit.scala:917-918 childClasspath.clear() | YARN/standalone/k8s cluster child classpath emptied. Lab: childClasspath-cleared=yes on the control walk. Public CVE-2023-22946 remains this path. |
--proxy-user client mode driver on the gateway | docs/security.md Proxy User section | Documented: client-mode driver JVM is the superuser. Vendor says not a CVE. Livy/Kyuubi are told to force cluster mode. |
K8s cluster-mode driver container (spark.kubernetes.submitInDriver=true) adding . to classpath | SparkSubmit.scala:420-431 SPARK-43540 | Second spark-submit inside the driver pod, not the Livy host. Pod UID is the container user. |
spark.driver.extraJavaOptions in cluster mode | SparkSubmitCommandBuilder.java:341-357 | Not applied to the spark-submit JVM. SPARK_SUBMIT_OPTS is gateway env, not tenant --conf. |
| YARN AM / k8s driver of the submitted app | Hadoop impersonation after a clean submit | Still the proxy user for cluster launch. This leftover is the gateway JVM only. |
| Unauth job submit / Connect SQL / ML model load | SECURITY.md FAQ / docs/ml-security.md | By design. Not this leftover. |
