A regular expression generator based on CSS selectors for efficient extraction from HTML pages
Yükleniyor...
Dosyalar
Tarih
2020
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Turkiye Klinikleri
Erişim Hakkı
info:eu-repo/semantics/openAccess
Özet
Cascading style sheets (CSS) selectors are patterns used to select HTML elements. They are often preferred in web data extraction because they are easy to prepare and have short expressions. In order to be able to extract data from web pages by using these patterns, a document object model (DOM) tree is constructed by an HTML parser for a web page. The construction process of this tree and the extraction process using this tree increase time and memory costs depending on the number of HTML elements and their hierarchies. For reducing these costs, regular expressions can be considered as a solution. However, preparing regular expression patterns is a laborious task. In this study, a heuristic approach, namely Regex Generator (REGEXN), that automatically generates these patterns through CSS selectors is introduced and the performance gains are analyzed on a web crawler. The analysis shows that regular expression patterns generated by this approach can significantly reduce the average extraction time results from 743.31 ms to 1.03 ms when compared with the extraction process from a DOM tree. Similarly, the average memory usage drops from 1054.01 B to 1.59 B. Moreover, REGEXN can be easily adapted to the existing frameworks and tools in this task. © TÜBİTAK
Açıklama
Anahtar Kelimeler
Computational efficiency, Heuristic algorithms, Regular expressions, Web data extraction, Extraction, Heuristic methods, HTML, Pattern matching, Trees (mathematics), Web crawler, Websites, XML, Cascading style sheets, Construction process, Document object model, Extraction process, Heuristic approach, Performance Gain, Regular expressions, Web data extraction, Data mining
Kaynak
Turkish Journal of Electrical Engineering and Computer Sciences
WoS Q Değeri
Q4
Scopus Q Değeri
Q3
Cilt
28
Sayı
6
