abraxaslabs.tech  ·  github.com/abraxas  ·  @abraxas_null  ·  [email protected]  ·  spark-rpc-stream-unauth

spark-rpc-stream-unauth

Class: Privilege leftover / File read Reach: Remote

Apache Spark 4.2.0 (32f72996) - Apache Software Foundation

CVE-2020-9480 closed unauthenticated RequestSubmitDriver / RegisterApplication on the RPC control plane. AbstractAuthRpcHandler gates receive() and receiveStream(). It does not gate getStreamManager(). TransportRequestHandler still dispatches StreamRequest onto that manager before SASL. NettyStreamManager.openStream does new File(dir, fname) with no stay-under-root check.

Unauthenticated StreamRequest classes/../../path reads Spark-UID files. Unix absolute children stay under the registered dir (lab 404). files/ is a map get (404). receive() / receiveStream() stay 401. This is not 9480 RCE.

ID	no CVE yet
Class	Privilege leftover / File read (ungated RPC data plane leftover of CVE-2020-9480)
Reach	Remote (unauthenticated TCP to a driver RpcEnv / ESS / BlockManager with spark.authenticate=true)
CWE	CWE-862, CWE-22
CVSS	High: 7.5 CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N
Product	Apache Spark
Affected	4.2.0 (32f7299601108917fb01920a54e084595b7b3bf8). Operator-enabled spark.authenticate=true. Host-file read needs a registered directory (spark.repl.class.outputDir /classes and/or Connect artifacts).
Auth	none on StreamRequest / getStreamManager()
License	GNU Affero GPL v3.0
Lab	127.0.0.1 only

What an attacker can do

TCP to a Spark RPC port that has spark.authenticate=true and a registered directory. No shared secret. No spark.network.crypto.enabled. Send a transport StreamRequest (message type 6) with streamId classes/../../<path>. TransportRequestHandler.processStreamRequest never calls RpcHandler.receive. It calls getStreamManager().openStream. The 9480 wrapper returns the real manager. new File(dir, "../../oracle/secret") canonicalizes outside the REPL/artifact root and comes back as FileSegmentManagedBuffer.

That is a host-file read as the Spark UID. It is not job submit. It is not RequestSubmitDriver. It is not a shell.

Confidentiality (lab-proven)

Method	Sink	Privilege
Unauth StreamRequest classes/../../<path>	NettyStreamManager.openStream new File(dir, fname) then FileSegmentManagedBuffer	Read Spark-UID files the driver can open. Lab-proven: opened /opt/repl-classes/../../oracle/secret, canonical /oracle/secret, body SPARK-RPC-STREAM-UNAUTH-WITNESS, x-auth-gated=false, x-authenticated=false.
Unauth StreamRequest classes/hello.txt (in-dir control)	same join, stays under registered dir	Read a file that is actually under /classes. Lab-proven 200 IN-DIR-OK, no witness. Confirms the file server works without SASL even for a contained name.

Confidentiality (inferred)

Method	Sink	Privilege
Unauth ChunkFetchRequest / StreamRequest of a live ESS streamId	OneForOneStreamManager.getChunk / openStream. checkAuthorization skips when client.getClientId() == null	Read another app's in-flight shuffle / disk-RDD bytes. Availability: drain advances curChunk. Needs a live streamId (random start, then +1). Gadget did not speak live ESS.
Unauth MergedBlockMetaRequest	AbstractAuthRpcHandler.getMergedBlockMetaReqHandler() also ungated	Read merged-shuffle chunk bitmaps for a chosen appId when ESS has merged shuffle. Same null-clientId skip. Not exercised.

Closed on this leftover (lab-proven)

Method	Sink	Stays closed
Unauth StreamRequest classes//oracle/secret (Unix absolute child)	new File(dir, "/oracle/secret")	404. Opened path is /opt/repl-classes/oracle/secret. Unix File concatenates; it does not ignore the parent. Hunt claim that an absolute child drops the root is wrong on Unix.
Unauth StreamRequest files//oracle/secret	files map get	404. files/ and jars/ are basename lookups, not joins.
Unauth receive() / OneWayMessage	AbstractAuthRpcHandler.receive	401 Unauthenticated call to receive(). x-auth-gated=true.
Unauth receiveStream() / UploadStream	AbstractAuthRpcHandler.receiveStream	401 Unauthenticated call to receiveStream().
Authed receiveStream() with the lab secret	same gate after doAuthChallenge	200 receiveStream-ok. Control that the secret still opens the 9480 path.

Lab-proven vs inferred: unauth .. file read, Unix absolute 404, files/ 404, gated receive / receiveStream are lab. ESS shuffle drain is inferred.

Adjacent closed (do not inflate):

• CVE-2020-9480 RCE walk: unauth RequestSubmitDriver / RegisterApplication via OneWayMessage / UploadStream / RpcRequest. Still needs the shared secret.
• Default spark.authenticate=false unauth shuffle and job submit (SECURITY.md FAQ, documented trusted-network default). Not this leftover.
• Authenticated app A fetching app B (AppIsolationSuite / SPARK-57889 non-null clientId).
• CVE-2021-38296 handshake key recovery.
• CVE-2025-55039 Moderate CTR default.
• Standalone master RpcEnv typically has no /classes or artifacts registered, so this host-file walk is driver / REPL / Connect, not master:7077.
• Does not execute the file. Does not write. Does not dump spark.authenticate.secret.

How I found it

I pinned apache/spark v4.2.0 (32f7299601108917fb01920a54e084595b7b3bf8), not master and not a 4.3 RC. Twenty-eight source hunts. Job submit on an open cluster is by design. ML model load is executable code. The public Jackson History RCE, UI id -Gn shell, PySpark local impersonation, and 9480 receive-path are closed on this peel.

The RPC-auth hunt was supposed to be a re-file check of 9480. AuthServerBootstrap still wraps the handler. Unauth receive still throws. Then I read getStreamManager(). Three lines. It returns the delegate. TransportRequestHandler.handle routes StreamRequest around receive. Default StreamManager.checkAuthorization is a no-op. SPARK-57889 added a string-id check on StreamRequest and kept the null-clientId skip that ESS uses for "auth off," which is also "this connection never authenticated."

SparkContext registers /classes when spark.repl.class.outputDir is set. Connect ArtifactManager registers artifacts. NettyStreamManager.openStream splits ftype/fname. Non-files/jars types do new File(dir, fname) with no canonical-prefix check.

Wrong turn, recorded because it wasted a day of hunt confidence: JavaDoc says an absolute child ignores the parent. On Unix, UnixFileSystem.resolve concatenates. new File("/opt/repl-classes", "/oracle/secret") is /opt/repl-classes/oracle/secret. Lab 404. The High walk is classes/../../oracle/secret.