What are Autonomous Systems and why they matter (a lot) for your security

If you have ever set up a server, you know that as soon as your server is online, it starts to be scanned by bots looking for vulnerabilities to exploit to make your server join a botnet for later attacks or crypto mining.

Today I come with good news: whether you want to protect your sites against scraping or stop vulnerability scanners, it's pretty easy to block most of these annoying bots without impacting your legitimate users and customers.

But for that, we first need to understand how internet works.

What is an Autonomous System

As you may know, what we call "the internet" is not simply a big network of computers connected together, but a network of networks. Each of these subnetworks is called an Autonomous System (AS), a set of routers controlling a specific set of IP addresses, or routes. An AS is generally controlled by an organization such as an Internet Service Provider (ISP) or a cloud provider.

Each AS is given a unique Autonomous System Number (ASN) which are written in the form AS123.

For example, Digital Ocean operates AS14061. As we can see on ipinfo.io/AS14061, this AS is composed of a lot of subnetworks such as 104.131.0.0/18 or 104.248.128.0/20.

Every AS advertise their routes using the BGP Protocol. By doing so, AS are telling others AS "hey, you can reach the IP address X.X.X.X here!".

Sometimes, malicious network operators falsely announce the ownership of groups of IP addresses with the goal of redirecting internet traffic to servers they control. This attack is called BGP hijacking, such as in December 2017 when a Russian network operator, AS39523, hijacked the routes of many famous websites such as Google, Facebook, Apple, Microsoft and many more.

But what matters most for us is to identify which ASNs are the source of most hostile bots, and block them or present them a challenge.

Cloudflare offers a free tool to know the bot vs. human traffic distribution for a given ASN: Cloudflare Radar. Here, we can learn that roughly 85% of the traffic coming from Digital Ocean is from bot, so we may save a lot of time and energy by presenting a challenge to filter out bots.

How to block bad AS

Blocking individual IP addresses is a cat-and-mouse game that you will always lose. Attackers only have to click a few buttons to launch a new server and get a fresh IP address. That's why instead, you should consider blocking the few AS that are the sources of most attacks. The Pareto principle, again and again :)

By blocking some AS instead of IP addresses we are saying: I want to be sure that the traffic coming from theses datacenters are humans and not malicious bots.

Cloudflare is a new-generation cloud platform whose principal products are a cloud firewall and a Content Delivery Network (CDN). Today, it's the simplest solution available to protect your websites and APIs and block malicious requests coming from bad AS.

On your Cloudflare dashboard, go to Security > WAF to setup Web Application Firewall (WAF) rules.

Use the field ASN Num with the is in operator and then paste your list of ASNs that you want block or challenge.

However, I do not suggest to simply block the requests coming from "bad" AS, but instead to present a challenge so legitimate users using a VPN (there are tons of legitimate reasons to use a VPN) can still access your website, at the cost of a minor inconvenience. This is why I recommend the Managed Challenge action.

You can find on Github some lists of AS known for being the source of frequent attacks.

Please note that you want some URLs to be reached by bots, such as you RSS feed, so you will need to exclude these special pages from the rule.

Finally, I recommend using terraform (zone_custom_firewall > rules) to set up your custom Cloudflare WAF rules instead of the web interface. This way, you can write comments about which ASN corresponds to which organization and why you decided to block it. Imagine the confusion when your rules look like (ip.geoip.asnum in {123 456 789 1234 5678 ...) and you need to update them a few months later but you don't have any indication about what these ASNs correspond to and why you blocked them in the first place.