abraxaslabs.tech  ·  github.com/abraxas  ·  @abraxas_null  ·  [email protected]  ·  spark-connect-copy-local

spark-connect-copy-local

Class: File write (leftover of SPARK-58531; static on 4.3.0 only) Reach: Remote

Apache Spark 4.2.0 (32f72996) - Apache Software Foundation

spark.sql.artifact.copyFromLocalToFs.allowDestLocal is still buildConf (session-settable) on this pin. SPARK-58531 moved it to StaticSQLConf in 4.3.0. A Connect session can SET the testing-only local-dest gate, then copyFromLocalToFs / AddArtifacts forward_to_fs/<abs> overwrites a driver-local path as the Spark UID.

Connect session SET then forward_to_fs writes driver-local files. Dest must be LocalFileSystem. Not RCE.

ID	no CVE yet (leftover of SPARK-58531)
Class	File write (leftover of SPARK-58531; static on 4.3.0 only)
Reach	Remote (Spark Connect Config + AddArtifacts)
CWE	CWE-22, CWE-732
CVSS	High: 8.1 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:H/A:H
Product	Apache Spark (ASF)
Affected	Apache Spark 4.2.0 (32f7299601108917fb01920a54e084595b7b3bf8). 4.3.0 is static.
Auth	Connect client that can ConfigRequest.SET and AddArtifacts. Default Connect has no PSK.
License	GNU Affero GPL v3.0
Lab	127.0.0.1:18920 only

What an attacker can do

A Spark Connect client who can reach gRPC (default bind port 15002, no PSK unless the operator set spark.connect.authenticate.token) can flip a testing-only gate the driver was started with off, then overwrite a chosen local path on the driver as the Spark UID.

Method	Sink	Privilege
gRPC Config SET spark.sql.artifact.copyFromLocalToFs.allowDestLocal=true (or SQL SET)	SparkConnectConfigHandler.handleSet -> RuntimeConfig.set -> SQLConf.setConfString	Integrity (session conf, lab-proven): flips the per-session local-dest gate. Does not write a file by itself. Does not change other sessions. Lab: SET HTTP 200, modifiable=True.
gRPC AddArtifacts name forward_to_fs + absolute dest, CRC ok	ArtifactManager.uploadArtifactToFs -> LocalFileSystem.copyFromLocalFile(delSrc=false, overwrite=true)	Integrity (file overwrite, lab-proven): write/overwrite a driver-local path as the Spark UID, outside artifactRootDirectory/<sessionUUID>/. Lab: dest /oracle/SPARK-CONNECT-COPY-LOCAL-WITNESS, uid 185, content WITNESS.
Same AddArtifacts to HDFS/S3/ABFS dest	documented copyFromLocalToFs	Integrity (cluster FS, by design): not this leftover. Dest that is not LocalFileSystem never consults the testing flag.

Lab-proven vs inferred: session SET of the buildConf key, then LocalFileSystem dest write as uid 185 outside the artifact sandbox, is lab. Spark UID overwrite of a path the process can write (driver work dir, a world-writable drop, an already-executable script the operator later runs) is inferred. No RIP control. Do not inflate File write to RCE.

Requires: Spark Connect listening. Client that can SET session SQL conf and AddArtifacts. Dest must be Hadoop LocalFileSystem. Default Connect has no PSK; if the operator set a token, the leftover still needs that shared Bearer. It is a bypass of the local-dest control, not a PSK bypass.

Adjacent surface that stays closed

Surface	Lab
Copy without SET	400 _LEGACY_ERROR_TEMP_3161, dest absent
SET the old static spark.connect.copyFromLocalToFs.allowDestLocal	400 CANNOT_MODIFY_STATIC_CONFIG
Artifact name .. / absolute staging (ArtifactUtils.concatenatePaths)	INVALID_ARTIFACT_PATH
Non-forward_to_fs artifacts	stay under artifactRootDirectory/<sessionUUID>/
HDFS/cloud dest without the flag	documented API, not this leftover
Unauth SQL / Python UDF / Scala UdfPacket on open Connect	vendor FAQ job submit; DROP
SPARK-52122 ML DefaultParamsReader	allowlist + safeMLClassLoader on this pin

No shell. No heap dump. No Connect PSK skip. The dest still has to be LocalFileSystem. The write is as the Spark UID, not as root.

How I found it

uploadArtifactToFs refuses a LocalFileSystem dest unless the testing flag is on, with an explicit comment that otherwise a user overwrites an arbitrary file on the driver:

if (fs.isInstanceOf[LocalFileSystem]) {
  val allowDestLocalConf =
    session.sessionState.conf.getConf(SQLConf.ARTIFACT_COPY_FROM_LOCAL_TO_FS_ALLOW_DEST_LOCAL)
      .getOrElse(
        session.conf.get("spark.connect.copyFromLocalToFs.allowDestLocal").contains("true"))
  if (!allowDestLocalConf) {
    throw new SparkUnsupportedOperationException("_LEGACY_ERROR_TEMP_3161")
  }
}
fs.copyFromLocalFile(false, true, new FSPath(localPath.toString), destFSPath)

The 3.5 key is static, default false. The 4.0 replacement is buildConf optional:

val ARTIFACT_COPY_FROM_LOCAL_TO_FS_ALLOW_DEST_LOCAL =
  buildConf("spark.sql.artifact.copyFromLocalToFs.allowDestLocal")
    .internal()
    ...
    .booleanConf
    .createOptional

buildConf does not call registerStaticConfigKey. isModifiable is containsConfigKey && !isStaticConfigKey. RuntimeConfig.set only rejects static keys. Internal is still settable. After SET, getConf(ARTIFACT_...) returns Some(true) and the getOrElse fallback to the static old conf is skipped.

SPARK-58531 / PR 57730 "Convert spark.sql.artifact.copyFromLocalToFs.allowDestLocal to a static conf" is resolved 2026-08-05, Fix Version 4.3.0. JIRA: the conf "controls whether a caller can write to an arbitrary path on the driver's local filesystem, so it is meant to be set by whoever starts the driver." 4.2.0 is listed Affects. This pin still has the runtime conf.

Wrong turns already recorded: treating Connect SQL as a finding (FAQ); SET of the old static key (CANNOT_MODIFY_STATIC_CONFIG); claiming a Unix absolute dest ignores the parent (it joins under root); inflating File write to RCE; pinning 4.3.0 RC because the JIRA is resolved.

Lab

cd lab
./run.sh

Loopback 127.0.0.1:18920. Gadget of the session-conf gate and LocalFileSystem dest copy (pin v4.2.0 32f72996). Server UID 185. Control no-SET: 400 _LEGACY_ERROR_TEMP_3161, dest absent. Control SET of the static old key: CANNOT_MODIFY_STATIC_CONFIG. Session SET of the buildConf key then copy: dest /oracle/SPARK-CONNECT-COPY-LOCAL-WITNESS outside /opt/spark/work/artifacts/<sessionUUID>/, uid 185, content WITNESS.