Extraction automatique de contenu sur des forums hébergeant des communautés criminelles

Abstract

Once isolated by geographical borders, criminal communities now take advantage of the anonymity provided by certain networks, such as the TOR network, part of the Dark Web, to cooperate, sell and share their knowledge. For an organization, analyzing these exchanges on the multiple forums present on this network allows detecting trends and thus preventing future attacks. Therefore, this thesis proposes a new approach to generalize the extraction of forum topics and their attributes (title, author and publication date). This approach is based on the hypothesis that it is possible to perform content extraction on the Web using natural language processing (NLP) tools. In order to extract forum topics, two sub-objectives are defined. The first one is to use sequence labeling to identify records and their attributes on Web pages. The second one is to proceed to the extraction of the identified content. To achieve this, a method is defined to transform a Web page into a sequence composed of HTML tags and text. It is then possible to proceed to sequence labeling using a BiLSTM-CRF model. The sequence is then reconstructed into a Web page in order to proceed to the extraction of the forum topics. For this purpose, extraction algorithms have been designed, taking advantage of HTML page’s graph structure. Following the experiments (hyperparameters tuning, vocabulary size adjustment) it is possible to confirm the hypothesis of this thesis. Indeed, the good results on the test set (macro F1 of 99,5 %), as well as the performances in industrial context, demonstrate that the proposed solution was able to generalize the structure of the forums. Consequently, it is possible to extract forum topics from forums that were not used during the training process.

Date	24 Mar 2022
Original language	French
Awarding Institution	École de technologie supérieure
Supervisor	Sylvie Ratté (Supervisor) & Pierre André Ménard (Co-supervisor)