A Novel Web Scraping Approach Using the Additional Information Obtained from Web Pages

Basit Öğe Kaydı
dc.authorscopusid54783608800
dc.contributor.authorUzun, Erdinç
dc.date.accessioned2022-05-11T14:02:59Z
dc.date.available2022-05-11T14:02:59Z
dc.date.issued2020
dc.departmentFakülteler, Çorlu Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümü
dc.description.abstractWeb scraping is a process of extracting valuable and interesting text information from web pages. Most of the current studies targeting this task are mostly about automated web data extraction. In the extraction process, these studies first create a DOM tree and then access the necessary data through this tree. The construction process of this tree increases the time cost depending on the data structure of the DOM Tree. In the current web scraping literature, it is observed that time efficiency is ignored. This study proposes a novel approach, namely UzunExt, which extracts content quickly using the string methods and additional information without creating a DOM Tree. The string methods consist of the following consecutive steps: searching for a given pattern, then calculating the number of closing HTML elements for this pattern, and finally extracting content for the pattern. In the crawling process, our approach collects the additional information, including the starting position for enhancing the searching process, the number of inner tag for improving the extraction process, and tag repetition for terminating the extraction process. The string methods of this novel approach are about 60 times faster than extracting with the DOM-based method. Moreover, using these additional information improves extraction time by 2.35 times compared to using only the string methods. Furthermore, this approach can easily be adapted to other DOM-based studies/parsers in this task to enhance their time efficiencies. © 2013 IEEE.
dc.identifier.doi10.1109/ACCESS.2020.2984503
dc.identifier.endpage61740
dc.identifier.issn2169-3536
dc.identifier.scopus2-s2.0-85083419796
dc.identifier.scopusqualityQ1
dc.identifier.startpage61726
dc.identifier.urihttps://doi.org/10.1109/ACCESS.2020.2984503
dc.identifier.urihttps://hdl.handle.net/20.500.11776/4566
dc.identifier.volume8
dc.identifier.wosWOS:000527413400002
dc.identifier.wosqualityQ2
dc.indekslendigikaynakWeb of Science
dc.indekslendigikaynakScopus
dc.institutionauthorUzun, Erdinç
dc.language.isoen
dc.publisherInstitute of Electrical and Electronics Engineers Inc.
dc.relation.ispartofIEEE Access
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanıen_US
dc.rightsinfo:eu-repo/semantics/openAccess
dc.subjectalgorithm design and analysis
dc.subjectComputational efficiency
dc.subjectdocument object model
dc.subjectweb crawling and scraping
dc.subjectEfficiency
dc.subjectExtraction
dc.subjectInformation use
dc.subjectTrees (mathematics)
dc.subjectWebsites
dc.subjectConstruction process
dc.subjectExtraction process
dc.subjectExtraction time
dc.subjectString methods
dc.subjectText information
dc.subjectTime efficiencies
dc.subjectWeb data extraction
dc.subjectWeb scrapings
dc.subjectData mining
dc.titleA Novel Web Scraping Approach Using the Additional Information Obtained from Web Pages
dc.typeArticle

Dosyalar

Orijinal paket
Listeleniyor 1 - 1 / 1
Küçük Resim
İsim:
4566.pdf
Boyut:
4.95 MB
Biçim:
Adobe Portable Document Format
Açıklama:
Tam Metin / Full Text
İndir

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu
Scopus İndeksli Yayınlar Koleksiyonu
Çorlu Mühendislik Fakültesi Koleksiyonu