Anomalous Behaviour Detection in Large Scale Internet Flow Datasets

Internet Service is provided by a large number of commercial enterprises generally in competition with each other. Cooperation required for global connectivity is generally at odds with the need to be a profitable commercial enterprise which often occurs at the expense of one’s competitors - the same people with whom one needs to cooperate with. How this “competitive cooperation” is achieved in practice and how we might improve things provides an interesting study of how good technical research can be shaped and challenged by commercial realities. Though Internet routing infrastructure has worked well over the years there have been instances of intentional throttling as a result of competitive cooperation between network operators around the globe. There is also opportunity for malicious activities that could damage the routing infrastructure in the future. To prevent future errors and malicious activity it is important to increase the security of the Internet’s routing infrastructure by figuring out such intentions and through the deployment of secure routing protocols.