Jev IDS

Intrusion detection on a stream of network flows. For each flow, Jev IDS picks a few labeled flows from what it has already seen and shows them to TypeSafe's Jev in one request. Jev answers whether the flow is an attack and which kind, in under a quarter of a second, with no text to parse.

An intrusion detection system (IDS) watches the traffic of a network and raises an alarm when it suspects an attack. Jev IDS does this by showing an AI model, Jev, a few flows whose category is already known and asking it to judge the new one. Two components decide what Jev sees. The KController decides how many examples of each kind of traffic to show, and the Selector decides which ones. In a real network the confirmation that a flow was an attack arrives late, so Jev IDS was measured on CIC-IDS2017, a public benchmark of network attacks, with that confirmation arriving 1 to 200 flows later. With the confirmation one flow late, Jev IDS and the strongest classical detector tested, a Random Forest, are level. From a delay of 25 flows on, each of the three runs of Jev IDS scored a higher F1 than every classical detector, and the lead grows with the delay. Most of the lead comes from zero-day attacks, the ones the system had never seen. With a delay of 200 flows, Jev IDS caught 54% of them and the Random Forest 29%.

Read the loop · The request template · Design log · Glossary

The problem

A flow is the summary of one conversation between two computers. It records how many packets and bytes went each way, how long the conversation lasted, which port it used and dozens of other measures. In CIC-IDS2017, the dataset used here, each line is one flow, described by 83 columns of measures and by a label that says whether it is normal traffic or which attack it belongs to.

An IDS receives each flow and decides whether it is benign traffic or part of an attack. When it decides attack, it raises an alarm.

Two facts make the task hard. The first is that attacks appear that the system has never seen, the zero-day attacks defined below. A detector that learned only from older examples tends to take a zero-day attack for normal traffic. The second is that the right answer arrives late. A security analyst confirms that a flow was an attack only some time later, when the damage shows or someone investigates. Until then the system keeps judging new flows without knowing whether it got the earlier ones right.

Zero-day attack. In network security, a zero-day attack is one the defenders do not yet know when it happens, so they had zero days to prepare. Here, the zero-day attacks are the three attack names of the test that never appear in the system's starting memory (How it was tested). The system meets them for the first time during the test, with no example of them.

This delay is called D and is counted in flows. With D = 1, the label of a flow comes back just before the next flow is judged. With D = 200, it comes back only after 200 more flows have been judged.

How it works

Jev is an AI model from TypeSafe, of a kind called a System One Model. It reads a state, a block of information, and answers typed questions about it with probabilities, so there is no generated text to parse. Jev IDS puts into the state the instructions, the 83 column names, a description of each category, a few flows whose category is already known (the examples) and the flow to judge. Then it asks two questions. is_attack comes back as a probability from 0 to 1, called p_attack. category comes back as one of six categories with a confidence. The categories are benign traffic (BENIGN) and five families of attack, listed in How it was tested.

The figure shows one real flow of the test, the first flow of the zero-day attack DoS Slowhttptest, as Jev answered it with D = 200. The top bar is p_attack, 0.80, which is above the cut of 0.58, so the verdict is attack. The lower bar splits the probability among the six categories, with 0.91 of it on DoS, and the confidence of 0.89 says how concentrated that split is.

Around that request runs a loop over the stream of flows. The figure below draws it, and the six steps after it follow one flow through it.

The top row is one step. The flow under test goes to Jev in one request, and the answer becomes a verdict and one row of results. The bottom row is what Jev IDS adds around Jev. The memory feeds the KController and the Selector, which choose the examples that go into the request. The dashed line is the label of each judged flow, which comes back D flows later to the memory and to the KController.

The memory holds the labeled flows a step may draw examples from. These are the flows of the pool, prepared before the test, that ended before the flow under test, and every judged flow whose label has come back.