Scraping refers to the use of automated tools to collect large amounts of data from a target application in order to reuse that data elsewhere.
Scraping can range from benign to malicious, depending on the source, objective, and frequency of the requests. For example, a search engine bot that respects scraping rates defined in the site’s robot.txt will likely be viewed as acceptable, whereas daily price scraping from a competitor is likely unwanted.
Scrapers were increasing the airline’s infrastructure costs and affecting the airline’s ability to manage revenue, so the security team sought out Shape.
Using automated tools, off-the-shelf scripts, or even scraping-as-a-service providers, attackers can easily create scripts to discover and scrape website content including prices, promotions, articles, and metadata.
Scraping campaigns can range from brazen to stealth, depending on the attacker’s skillset and aims. Execution of the scraping script may be distributed amongst hundreds or thousands of servers in order to blend in with traffic patterns of the enterprise’s entire user population.
The extracted data may be sold, used for price-comparison sites, or even used to create imitation sites for fraudulent purposes.