Last week we announced a new service offering called the DOSarrest Traffic Analyzer (DTA), essentially a cloud netflow/jflow/sflow ingestion and analyzer platform that leverages our Big Data platform we developed inhouse a few years ago and are now making available to you. For this blog, I’ll detail some examples of reporting the DTA can provide you.
I'll take a recent case of a mixed network downstream of DOSarrest, with a focus on hosting that received a decent sized and sustained DDoS. The customer is sending Jflow from three flow sources, with a sample rate 1:2000 (pretty standard for larger traffic networks). During the attack, we were able to see a number of unique vectors that provided detail for the customer’s netops team to take appropriate action. Here’s a summary of what they were able to surmise from our reports:
1) Size of the attack – Nothing too novel here, as we showed the attack topping out at 35 Gb/sec, based on a 10 minute average (100+ Gb/s on a 1 minute average which you will see later) and it was relatively easy to see from their standard SNMP tools. We are able to aggregate the data from all 3 routers so they do have the ability to see how big the attack was on each router. It should be noted that the customer had the ability to focus the reports to specific ranges, as they operate a mixed traffic backbone, so quickly being able to discard other networks in their analysis was vital for them, instead of looking at SNMP counters and surmising which vertical of theirs was being attacked. Some example graphs that illustrate what they saw:
2) Identifying the Targeted IP – With our DTA this becomes much easier and faster. By simply reviewing the report for their netblocks, it was easy enough to see which IP's were being targeted, even when the attack vector was mixed with multiple targets and changing volumes per target. In the example below, you can see over the 6 hours the attack was occurring, the targets changed in bps and pps with smaller attacks to adjacent IP's (probably in an attempt to create more alerts in the customer NOC so as to delay resolution) helping the customer to localize who was being targeted and effected
3) Identifying the Source IP/Networks/ASN/Countries – With some DDoS attacks, the source IP’s and networks can come from a few IP’s or contiguous set of blocks. In this particular case the attack was too widespread to be seen effectively in a summary graph, but I’ll detail what was seen here so as to demonstrate how easy it is to see which sources are involved and if it provides possible mitigation strategies.
Typically, when you are under an attack, you look for the top sources to see if there is something obvious that can be blocked. Here’s a graph that shows the top 10 sources off the bat. As you can see the total of these 10 is only about 1 Gb/s collectively, so the attack is more widespread in order to be able to generate 100+ Gb/s. However, you can see one IP dominates in that set (202.80.208.220), and can provide opportunities for further follow-up and investigation:
Aside from looking at the individual source addresses, you can also group individual addresses into their respective BGP CIDR nets to see if there is a particularly big set of networks involved in the attack. Again, this particular attack was too widespread across multiple addresses and networks, so the top 10 BGP CIDR played a very small part in the overall amount of traffic, but a more detailed report can be generated by the support team at any time:
Looking at the source Autonomous System Number, or ASN, can sometimes provide insight to see if a particular network has the majority of the bots involved in the attack in their network. If one network dominates, you can effectively block that whole ASN by leveraging any half decent IP Location database to create an ACL. Here's an example of what the top 10 ASN spread looked like during the attack. Again, you can see this attack was fairly spread out and the top 10 only represented approximately 3 Gb/s of the attack:
Source Country is also an interesting vector for analysis, as patterns may emerge from those reports, and allow you to make a strategy based on that (eg. Block all Chinese based netblocks). In this particular situation, the countries of origin were dominated by China and Romania, as you can see here:
A country based ACL could have mitigated the majority of the attack, but other vectors that were presented in DTA showed cleaner and easier solutions to implement, detailed in the next section.
4) Source and Destination Ports – Having the ability to index and group on source and destination offers a number of possibilities to be able to block a DDoS. Earlier it was mentioned that this attack had multiple attack vectors within it, which we will isolate here. In the first stages of the attack, from 4:45 AM to about 9 AM, you can see that the attack was a widespread, probably spoofed protocol attack on TCP 80 and 443. There is a pattern to the source ports in the initial stages of the attack from 4:47 AM to 7:47
After 8:47 a new source port pattern emerged in the botnet that looked like so:
It was these source port profiles that were used to mitigate the attack (the mitigation platform is not restricted to just the top 10 entries, btw). After 9 AM the TCP 80/443 attack abated. The attackers, probably frustrated from not being able to successfully bring the site down with the TCP 80/443 attack, then decided to launch a UDP based attack at 9:25 AM, pulsating with 4 major peaks until it the attack finally stopped at 9:35 AM. Despite the relatively large size, this part of the attack was the simplest for our mitigation to block as it was all coming from an NTP UDP source port (123), as you can see here:
Hopefully you can see the value of this capability in reviewing an attack in real time, as well as for a post mortem. In the next blog I’ll be detailing how the platform can be used for traffic engineering purposes.
This is an illustration of how a network engineer could mitigate a DDoS attack using our DTA platform.
In this particular case the customer is actually using our DCD service as well, which automatically stopped the attack. But using the DTA tools the customer in question could follow along in real-time and watch the mitigation in action.
Jag Bains
CTO, DOSarrest Internet Security
