Data Scraping: Legal and Practical Considerations

The law governing data scraping remains ambiguous, offering no definitive answers about its permissibility. Despite this, the industry operates on best practices shaped by evolving case law and regulatory principles. This article outlines actionable insights to minimize risks while engaging in data scraping.

At Legally Wired, we offer insights that look beyond the basics of the law. Each article concludes with actionable insights to help you assess potential risks and practical ways to reduce them, empowering you to make confident, informed, proactive choices for your business.

Legal Landscape

Currently, no specific legislation directly addresses the legality of data scraping. However, significant legal developments—such as the Van Buren v. United States and HiQ Labs v. LinkedIn cases (2019–2022)—shed light on its permissibility under the Computer Fraud and Abuse Act (CFAA). This federal statute prohibits accessing computer systems without authorization or exceeding authorized access.

Key Takeaways from Case Law:

Scraping Public Data Isn’t Prohibited: The courts established that scraping publicly available data is not inherently a violation of the CFAA. Accessing publicly accessible websites doesn’t qualify as unauthorized access.

Example: A startup collects publicly visible product prices from online retailers to provide price comparison services. Since the data is publicly accessible, this activity would not be considered unauthorized under the CFAA.

Meaning of ‘Unauthorized Access’: Unauthorized access applies only when information that is explicitly restricted (e.g., behind paywalls, authentication, or authorization mechanisms) is accessed without permission. Website owners are responsible for signalling their intent to restrict access through such barriers.

Example: If you scrape user email addresses from a private forum that requires login credentials, this may constitute unauthorized access.

While these rulings affirm that data scraping of public information is not unlawful under the CFAA, this does not absolve scrapers from other obligations, particularly regarding privacy and copyright laws. These principles impose additional compliance requirements, which are critical to understand.

Data Privacy

Data scraping often involves handling personal or sensitive information, which is governed by strict privacy laws. Even if the data originates from public sources, compliance with regulations like GDPR and CCPA is non-negotiable.

Key Privacy Considerations:

Laws Still Apply: Privacy laws, including the EU GDPR, California Consumer Privacy Act (CCPA), and HIPAA, mandate obligations for handling personal data, irrespective of its source. Public availability does not exempt businesses from compliance.

Example: Scraping job titles and locations from public LinkedIn profiles to build an HR analytics platform might seem harmless, but if the data includes names or email addresses, compliance with GDPR and CCPA is still required.

Anonymization is Key: Scraping should anonymize or redact personal information wherever possible. This minimizes the risk of inadvertently violating privacy laws.

Example: A company scrapes product reviews from an e-commerce platform. Instead of storing reviewer names and photos, it focuses on the content of the reviews for sentiment analysis.

Copyright Considerations

In most jurisdictions, copyright law extends protection to original content, with limited exceptions for factual data. This creates an inherent risk when scraping data, as much of it may qualify as copyrighted material.

When is Scraping Safe?

Analysis vs. Reproduction: Using scraped data purely for analytics, research, or internal insights is generally low-risk, provided that the results do not reproduce the original content.

Example: A tech company uses scraped data to identify trends in movie ratings across different genres but does not publish the reviews in their original form.

Avoiding Violation: Direct reproduction or redistribution of copyrighted data, whether in reports, publications, or other formats, could trigger copyright infringement claims.

Example: Republishing an entire database of articles scraped from news websites on your blog would likely violate copyright laws.

The growing use of scraped data in training AI models highlights the urgency for clearer legal boundaries. Legal disputes in this area show how easily scraping activities can cross into infringement territory.

Contractual Considerations

Another important aspect of data scraping involves the agreement between the website owner and the user. Many websites include terms of use prohibiting scraping, even if technical barriers to access are absent.

The implication of such conditions in the terms of use, is at odds with the case law established on the matter (see above).

There is no straight answer on whether terms of use automatically apply on data scraping activities of publicly available data. Hence, practically, you should consider the following:

Express Agreement: Have you explicitly agreed to the terms of use (e.g., by clicking “I Agree”)?Example: A freelancer creating a market analysis tool scrapes data from a site that requires users to accept its terms prohibiting scraping before accessing the platform. The freelancer may risk a breach of contract claim.

Visibility of Restriction: Was the anti-scraping clause prominently displayed?Example: If a website buries its no-scraping clause deep in a lengthy terms of use document, and the user did not actively agree, enforcement of that restriction might be less certain.

If the answers to the above considerations are in the negative - that is, you have not expressly agreed by way of affirmative action to such terms or that the restriction against scraping data was hidden deep within the website/ terms of use, you might have a chance of not being held in breach of a private contract.

Enforcement of such terms in light of the case law in hiQ v LinkedIn remains untested and hence there are no clear answers yet.

Best Practices for Data Scraping

To stay compliant while leveraging data scraping, adopt these best practices:

Identify Privacy Risks: Assess whether your activity involves personal data, and ensure compliance with relevant privacy laws.

Example: If you scrape contact information for sales leads, confirm consent or anonymize data to avoid violating privacy rules.
Check Authorization: Avoid scraping data that requires explicit authorization or is disallowed by mechanisms like robots.txt.
Example: Scraping a directory that requires login credentials without permission could expose you to legal action.
Document Your Activity: Maintain a library of scraped data with sources, dates, and purposes for accountability.
Example: An e-commerce startup logs all scraped data to ensure transparency and compliance audits.
Transform Data to Avoid Copyright Issues: Ensure that scraped data is transformed into new insights or analyses. Avoid using it to compete directly with the data source.Example: A travel app aggregates flight prices to identify trends but does not replicate the exact format or layout of the scraped websites