Apache Spark 4.2.0 - High - Remote - Privilege leftover - standalone REST ignores spark.authenticate
abraxaslabs.tech · github.com/abraxas · @abraxas_null · [email protected] · spark-rest-auth-bypass
Class: Privilege leftover Reach: Remote
Apache Spark 4.2.0 (32f72996) - Apache Software Foundation
Standalone Master REST is default-on port 6066 and never reads spark.authenticate. CVE-2018-11770 promised REST off by default and a fail-closed if the RPC secret was set while REST was on. This pin reversed both. MASTER_REST_SERVER_ENABLED is createWithDefault(true). Filters are only MASTER_REST_SERVER_FILTERS, default Nil. An unauthenticated POST /v1/submissions/create runs attacker main as the Worker UID. The Worker injects _SPARK_AUTH_SECRET, so the 2018 "driver without executors" cap does not hold.
This is a leftover of 11770, not a new CVE-2020-9480 RCE. RPC 7077 still installs AuthServerBootstrap. Job submit on a cluster with spark.authenticate=false is the documented default and is not this bug.
| ID | no CVE yet |
| Class | Privilege leftover (leftover of CVE-2018-11770) |
| Reach | Remote (unauthenticated HTTP to standalone Master 6066) |
| CWE | CWE-306, CWE-287 |
| CVSS | High: 8.8 CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H |
| Product | Apache Spark |
| Affected | 4.2.0 (32f7299601108917fb01920a54e084595b7b3bf8) standalone Master REST |
| Auth | none on REST. Operator set spark.authenticate=true. REST left at default. |
| License | GNU Affero GPL v3.0 |
| Lab | 127.0.0.1:18900 only. Official spark:4.2.0 image. |
I pinned tag v4.2.0, peel 32f7299601108917fb01920a54e084595b7b3bf8 (2026-07-11, v4.2.0-rc6). Not master. Not a 4.3 RC.
Reach standalone Master REST on 6066. No Authorization header. spark.authenticate is on. spark.master.rest.filters is unset. At least one Worker is live.
| Method | Sink | Privilege |
|---|---|---|
Unauth POST /v1/submissions/create | RequestSubmitDriver (Master.scala:402-420) then Worker DriverWrapper classForName(mainClass).main | Integrity (lab-proven): run attacker mainClass as the Worker UID. Lab: driver RUNNING, witness file. |
Same create; Worker CommandUtils.buildLocalCommand | driver env _SPARK_AUTH_SECRET | Integrity (lab-proven): driver can SASL-join the authenticated RPC cluster. Lab: injected=yes. |
Unauth GET /v1/submissions/status/{id} | RequestDriverStatus (Master.scala:489-503) | Recon (lab-proven for the created id): driver state / worker host. |
Unauth GET /v1/submissions/readyz | RequestReadyz | Recon (lab-proven): Master not STANDBY. |
Unauth POST /v1/submissions/kill/{id} | RequestKillDriver (Master.scala:422-453) | Integrity + availability (inferred): stop another tenant's standalone driver. Same unauth servlet wiring; not exercised. |
Unauth POST /v1/submissions/killall | RequestKillAllDrivers (Master.scala:455-478) | Availability (inferred): end every waiting/running standalone driver. Not exercised. |
Unauth POST /v1/submissions/clear | RequestClearCompletedDriversAndApps | Integrity (inferred): drop completed driver/app bookkeeping. Not exercised. |
Lab-proven vs inferred: create, secret inject, own-id status, and readyz are lab. Kill / killall / clear are inferred from the same unauth servlet map.
Requires: standalone Master (not YARN/K8s). Operator set spark.authenticate=true. REST left at default (spark.master.rest.enabled true, spark.master.rest.filters unset). Network path to Master 6066. At least one Worker. Not the documented-open cluster (spark.authenticate=false).
| Surface | Why |
|---|---|
RPC 7077 with spark.authenticate=true | AuthServerBootstrap still required. CVE-2020-9480 receive-path stays closed. Not a new 9480 RCE. |
| JWSFilter when set | spark.master.rest.filters=org.apache.spark.ui.JWSFilter plus secretKey. Tests 403 without JWT. Opt-in. Default Nil. |
UI doAs | still requires admin. REST does not inherit UI filters. |
| CVE-2022-33891 UI shell | id -Gn argv leftover closed on this pin. |
| YARN / Kubernetes submit | this REST server is standalone Master only. |
Default spark.authenticate=false job submit | FAQ: by design. Not this leftover. |
| Mesos REST | removed in Spark 4.0. |
CLI spark-submit default | still uses RPC 7077 (useRest string-defaults false). --kill / --status on spark:// always use REST. The leftover is the server listener. |
CVE-2018-11770's code fix (SPARK-25088) was: default REST off, and require Master fail-closed if spark.authenticate.secret is set while REST is on. This pin:
package.scala:2045-2048. Docs: "Since Spark 4.1, Spark Master daemon provides REST API by default."Master.scala. Test SPARK-57451: Allows REST server and spark.authenticate.secret to be enabled together (MasterSuite.scala:253-261) asserts start must succeed.NETWORK_AUTH_ENABLED / AUTH_SECRET. addFilters applies only spark.master.rest.filters default Nil.An operator who followed CVE-2018-17190 / the FAQ ("enable spark.authenticate") still has unauth HTTP job submit on 6066.