Scraping factual data off the web and re-organize to release under proprietary license?
<p>Lots of bits and pieces of information and chaotic data spread across the web and it does cost an arm and a leg to gather them together to form something larger and consistent that would prove to be much more useful.</p>

<p>Is it legally viable to <strong>scrape</strong> these <strong>factual data</strong> / information from different websites, compile and re-organize them as databases / data sets that will then be licensed under a <strong>proprietary license</strong>?</p>

<p>Thus far after some research, I find factual data can't be <strong>copyrighted</strong>, but the particular compilation / structure of data can. Does this mean I can do whatever with the scraped factual data as long as I have my own unique <strong>data organization / compilation / structure</strong>?</p>

<p>For example, I want to make a business information database off hundreds of different websites and then radically re-organize / re-structure / re-index these different data sets into one database that is better than any of the sources. Can I proprietorially <strong>license</strong> this particular database I made?</p>

<p>I know I can never own the data as they are straight facts. Right? But can I <strong>OWN</strong> the particular data organization and legally profit from it?</p>

<p>We are operating in US, but the data could be from websites on other continents.</p>

<p>From your question(s), as well as your various comments, I understand you to have two general inquiries:</p>

<p><strong>1. Is there any infringement of copyright laws if you use things like the titles of books, games, apps, names, address (and any other number of things) which you will then put into datasets that will be licensed for proprietary commercial purposes?</strong> </p>

<p>You may freely put titles, names of people, places or things into datasets without fear that you are infringing on copyright or any other laws. That is clear.</p>

<p>Copyright law does not protect names, titles, short phrases or expressions. Even if a name, title, or short phrase is novel or distinctive it cannot be protected by copyright.  So, there is no point in discussing the doctrine of fair use in this context, because <em>Fair Use</em> is a defense, or a legal safe harbor that is merely an exception to copyright infringement allowing people to use a copyrighted works under specific circumstances. </p>

<p>As I understand your intended endeavor, you will not be infringing on any copyrights to the extent that you are merely using factual data, like names of copyrighted things for the purpose of creating a dataset or an application to help access it. This is why I say you need not concern yourself with the test for Fair Use with regard to this issue.</p>

<p>The Copyright Office states clearly, despite what people may think, that there are no exclusive rights in brief combinations of words such as:</p>

<p>• Names of products or services
• Names of businesses, organizations, or groups (including the names of performing groups)
• Pseudonyms of individuals (including pen or stage names)
• Titles of works
• Catchwords, catchphrases, mottoes, slogans, or short advertising expressions
• Listings of ingredients, as in recipes, labels, or formulas. When a recipe or formula is accompanied by an explanation or directions, the text directions may be copyrightable, but the recipe or formula itself remains uncopyrightable. </p>

<p>Hence, these things are not registrable under a copyright.</p>

<p>While something may be potentially attached to or included in copyrighted material, is not in and of itself subject to the protections of these laws. If <em>it</em> (whatever it is) cannot be registered for a copyright, it is not copyrightable. Because copyright registration/notices have been optional since 1989, when the U.S. attached itself to the Berne Convention, whereby copyright protection is automatic as soon as a work is “fixed in a tangible medium of expression” (written down, recorded, painted, etc.) it’s protected. No notice is required. Registration only becomes required for litigation or enforcement purposes. But this is really extraneous to your inquiry anyway, as far as it applies to the actual data. When you get into copying whole databases for your purpose, that analysis is different. </p>

<p><strong>2. You want to "scrub" the internet for information that you intend to put into your proprietary datasets and use for commercial purposes, some or most of which is already in a database or some organized form, and you want to know if there is some sort of copyright or duty owned to the person who originally databased the materials?</strong></p>

<p>Since ideas, procedures, principles, discoveries, and devices are all specifically excluded from copyright protection, if you want to compile this type of information from the internet for the purpose of creating datasets, or searchable databases, this is permissible. </p>

<p>That said, <em>there are</em> protections for existing databases under copyright law, provided under the concept of a "compilation copyright". <em>A compilation copyright protects the collection and creative assembling of data or other materials</em>.  Compilation copyrights protect the collection and assembling of data or other materials, such that databases are generally protected by copyright law as compilations.</p>

<p>Under the Copyright Act, a compilation is defined as a "collection and assembling of preexisting materials or of data that are selected in such a way that the resulting work as a whole constitutes an original work of authorship." 17. U.S.C. § 101. </p>

<p>The preexisting materials or data may be protected by copyright since the selections of materials and the form they take in an existing database may be original enough to be subject to a copyright. However, the data itself is merely information and is not protectable.</p>

<p>The Copyright Act specifically states that the copyright in a compilation extends only to the compilation itself, and not to the underlying materials or data. 17 U.S.C. § 103(b). As a result, "compilation copyrights" can't be used to place protection upon those things that are otherwise not protectable.</p>

<p>In the case of <em>Feist Publications, Inc. v. Rural Telephone Service Company, Inc.</em>, the U.S. Supreme Court ruled that a compilation work such as a database must contain a minimum level of creativity in order to be protectable under the Copyright Act. <em>Feist</em> makes clear that even a copyright protected database does not hold the right to prevent an individual from extracting factual data from the database (so long as you're not copying the entire database as a whole). If you take an already compiled and copyrighted dataset in its entirety, you must obtain a license for its use. However, if you are merely amassing great amounts of data to then put into your own dataset, that you are free to do.</p>

<p>The big issue will be (and you seem to realize this) <em>where</em> you will amass this data from. Some websites have specific licenses in place that say you cannot use or rework their content. However, many times these websites simply throw these license requirements out there for users to see, despite the fact that they may not be (and some would argue) are <em>not</em> enforceable.</p>

<p>The courts have heard arguments that "contracts" (the end-user licenses) that protect databases and information on websites is beyond the protection available through copyright law should be "preempted" by the Copyright Act itself.</p>

<p>The preemption argument goes like this: Federal law controlling something that is subject to interstate commerce or use, should be controlled by the federal laws. So,since the federal government has enacted the Copyright Act to govern any protections to any original works, states should be (arguably are) prohibited from having contradictory laws. </p>

<p>Because of the ability of a federal statute to preempt state law, and the fact that the Copyright Act at 17 U.S.C. § 301 sets forth specific preemptions, no state may create rights that are equivalent to any of the exclusive rights provided under the Act.</p>

<p>It is this concept of preemption that prevents copyright protection from varying depending upon the state where a work of authorship is created. Arguably, the same is true for the internet, and supposed contractual relationship created through licenses that dictates how non-copyrightable material may be used.</p>

<p>In the case of <em>ProCD, Incorporated v. Matthew Zeidenberg and Silken Mountain Web Services, Inc</em>. the court examined whether an end-user of a CD ROM phone database was subject to the license, when they extracted a large portion of the database and made it available over the Internet.</p>

<p>The database was almost the same as the type of data in the Feist case-The lower court rejected all copyright claims and found that the shrinkwrap license that controlled the end user's right to use the data was both unenforceable (as a shrink wrap license) and preempted by the Copyright Act. As a result, there was no relief available to the creator of the phone database and the end-user was free to extract the data and use it as he saw fit. However, on appeal this decision was reversed (7th circuit).</p>

<p>The appellate court did acknowledge that the database (on the CD) was not original enough to be protected by copyright (finding no copyright infringement by the end-user); However, they did find the end-user was breach of contract, since the shrink-wrap license prohibited the end-user's conduct.</p>

<p>What this tells us is that these licenses (on websites) may or may not be enforceable. While the 7th Circuit found a contract right pursuant to the license, despite the preemption argument, another appellate court that is more liberal may find otherwise. Also, this was a disk, not the internet, which is the "wild west" of information, largely unregulated and unlitigated  as it pertains to the legality and enforceability of (some) regulations that do exist. License agreements for site use on the internet are everywhere. If you take a database from some site that has a license saying you cannot take their work and add to it, or whatever, and you do add it to other databases that are not licensed and then make your own dataset - chances are you are NOT going to be infringing on anyone's copyright. That said, you may be in breach of contract (the license) if they find out about it, and sue you (using it doesn't put you in breach; only getting sued and having a court determine you're in breach puts you in breach. It may be a distinction worth contemplation, but that is up to you). The <em>safest</em>, bet would be to get a license from them to rework the materials. If the material is generic enough, and will be changed enough, that you are creating your own new (copyrightable) work - I'm not sure how they would know you "scrubbed the data in contravention of their license agreement ( I have NO CLUE if there is coding or metadata attached to it such that it's identifiable in that way. I have not tech background and do not endorse taking what's not yours). But if they can and do know, they could cause problems for you.</p>

<p>Lastly, I will just say that the internet is littered with sites that claim copyrights, or impose unenforceable licenses on material that is ripe for public use. Just because it says it's theirs does not make it so. The inverse is also true. Just because a site does not claim copyright to something, does not mean it is in the public domain.</p>

<p>I would recommend either sticking to public domain/use sites for your scrubbing endeavors, or seeking permissions from the sites who impose licensing requirements. Short of that, I would recommend (as I already have) seeking an formal legal opinion to say that you are not imposing on anyone's copyrights (this could only be done once you showed an attorney every place you took material from, as well as what the material is), and that the licenses from sites with generalized information that may try to limit use, are unenforceable. I would do this before you invest a lot of time or money into something that is largely based on the accumulation of other peoples work product.</p>

<p>I wish there was an answer certain, but there just isn't without seeing everything in the end.</p>

<p>It's actually impossible to answer w/out knowing where you're intending to do this.  In the US, collection criteria &amp; presentation <em>might</em> make it copyrightable.  In some other places (e.g., the UK &amp; Europe) databases are protected by <a href="http://en.wikipedia.org/wiki/Sui_generis_database_right" rel="nofollow"><em>sui generis</em> database rights</a>.</p>

<p>You'd want to track when &amp; where you got each bit of data, and what licenses it was provided under, to reduce the chances of a protracted lawsuit.  (<em>When</em> is important, as websites may change their license terms, so you'll want to grab a snapshot of the license terms on each day that you scraped them.)</p>

<p>Also note that some "facts" are written as sentences or presented as graphs &ndash; in the US, those are copyrightable, as are photographs establishing facts.  Take the case of TV listings: The time a show is scheduled on a given channel on is a fact and not copyrightable in the US, but the summary of the episode is a creative work, and copyrightable.  Creative filtering (selection) or presentation may be copyrightable; that issue was left open in <a href="http://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co." rel="nofollow">Feist v. Rural</a>.</p>

<h1>Short Answer</h1>

<blockquote>
  <p>You don't need to license data to your users to generate revenue if you adopt any of the following business models.</p>
  
  <ul>
  <li><p><strong>Software as a Service (SaaS)</strong></p></li>
  <li><p><strong>Pay-per-use Web Application</strong></p></li>
  <li><p><strong>Data API</strong></p></li>
  </ul>
</blockquote>

<h2>Explanation</h2>

<p>It sounds like you're describing either one of the above business models in your question. In all cases, the value of your reorganized datasets will be in the way your app or service allows the user to interact with it to do useful things. So I think you might not really want or need to license the datasets at all — just charge for the service and the use of your software, app or API.</p>

<h3>Implementing a Data API Model</h3>

<p>If you prefer to just focus on the data and let your users build their own apps, then building a <strong>data API</strong> might be your best option. With this product, you would allow your users send <strong>RESTful</strong> HTTP requests to your data service.</p>

<p>The service would respond (most likely) with a <strong>JSON-P</strong> callback to supply the data. You would need to thoroughly document all the methods available from your API along with example calls in various common languages. (Start with <strong>cURL</strong> as that is relatively universal.)</p>

<p>In this model, as with the others, you would issue authorization credentials to all your users (this is what they would pay for). Then with each data call, they would supply these credentials which would in turn be verified server-side before sending the response.</p>

<p><hr>
<strong><em>Disclaimer</strong>: I am not an attorney so don't follow my advice. Consult a real attorney as that is the only way to make sure you stay out of trouble.</em></p>

copyright