SSH Bruteforce attack detection with network flow analysis

Abstract

Secure Socket Shell (SSH) exposes an interface for remote login. This provides convenience to system administrators for managing systems remotely and for other users it facilitates remote access to servers and applications over an unsecured network. SSH requires users to authenticate before they are allowed to access system and this can be done either through a key or by using passwords depending on the configuration. Using key based authentication is secure but it requires maintaining public key for every user and poses serious scalability issues. Thus, password based authentication methods are predominantly used in practice. SSH servers which use password based authentication method are vulnerable to password guessing attacks known as bruteforcing where an adversary tries many pass words to gain access. These login attempts are recorded in a log file by the operating systems. We perform a log analysis case study with logs collected from a production level SSH server in our university campus network. With analysis, we find different types of failed logins, origin of attacks, common usernames used, etc. Our analysis showed that many sources persistently try to login showing recurrent attempts across weeks. Owing to the scalability issues of log analysis for attack detection, we propose network based bruteforce attack detection methods. In the first place, our proposed methods separate network flows corresponding to failed and successful login cases. This is done using few statistical features derived from network flows. We propose two bruteforce attack detection methods using network flows. First one models the failed login cases generated due to users forgetting their passwords or misspelling as a probability distribution and detects unusually low probability events as bruteforcing attacks. In the second method, we build a Petri-Net based model to detect and classify the bruteforcing attacks. The classification marks the attack as either single source, single domain, and/or distributed attack. We evaluate our proposed model with data generated from a testbed setup and also form production level servers.