Is it legal to share a script that scrapes a website and presents in a different format?
<p>Most news sites and tech blogs provide only a summary in their rss feeds. If I write a script that scrapes their website and extracts some useful data, can I share the script as open source. Does it depend on each and every website and needs prior permission? If scraping is not allowed without permission, who is guilty, the person who wrote the script or the one who used them? </p>

<p>Note 1: I am not redistributing website's content, just sharing the script that helps to read the content in different form. I am specifically interested in Indian and US Law.</p>

<p>Note 2: The script may be specific to particular website and may access data that is accessible only with user's login credentials.</p>

<p>Note 3: script means a program or software application</p>

<p>E.g nytimes offer subscription offer at $3.5 a week for web+smartphone access. But for tablet access, it is $5.00 a week. Can I sell an app for tablet that parses the page from the web interface and converts it to tablet friendly format. The app will use login credentials from the user to login using a virtual browser to download the content and format it such that it is better readable in tablet? Do I have to get permission from nytimes to sell the app? I don't care about this special case but in general can the provider legally restrict how the content is consumed by the end user.</p>

<p><strong>Check the website's terms of service.</strong>  Check to see if you're violating these terms, and check to see if the script you are making enables other people to violate them.  Courts don't often look kindly on actions whose sole purpose is enabling someone else to do something that is prohibited.  If you're making a script that helps people do something they're allowed to do, in a way that's better for at least somebody and makes nobody worse off, that's often a different story.</p>

<p>Major websites will generally indicate whether or not you're allowed to do this.  </p>

<p>Some sites are fairly strict about prohibiting scraping (e.g. Craigslist, which <a href="http://blog.padmapper.com/2012/06/22/bye-bye-craigslist/" rel="nofollow">at one point shut down Padmapper</a>'s alternative more-useful presentation of their content). </p>

<p>Others, like Wikipedia, much more actively encourage reusing content from their sites as long as you meet <a href="https://en.wikipedia.org/wiki/Wikipedia:Text_of_Creative_Commons_Attribution-ShareAlike_3.0_Unported_License" rel="nofollow">certain conditions such as a link back to the original source</a>.</p>

<p>Without too much detail knowledge of both US and Indian law, I can't really think that this wouldn't be allowed. In some jurisdictions, distribution of software programs may be restricted or prohibited if any use of such programs would very likely, or automatically, lead to copyright violations (e.g. "hacker tools"). However, this is a very exceptional scenario and doesn't apply here for various reasons. </p>

<p>The owner of a website publishes content on the Internet with the intention that users can access it (either all users or only those who have subscribed to the content etc.). The website owner then cannot prescribe how exactly users access the content. For example, each different browser may display a website differently, with some browsers like Safari already now providing text-only views, and in the end your script is not very different from a browser.</p>

<p>The only critical situation I can imagine is when somebody uses your script to scrape content from a website and then republishes it. Even then this "somebody" would be responsible and liable, not your script. Think again of the browser analogy: If I download a copyright-protected movie from a website via Mozilla Firefox and then put it on my website, I can't imagine a legal principle that Mozilla should be liable. Neither are you.</p>

<p>Such scripts are all over github, so even if it were illegal, it is de facto legal.</p>

<p>However, there's a more obvious proof of its legality; every web browser on earth does exactly what you're describing. As does every RSS reader. Every network / traffic analysis tool. Every screen reader. Every network proxy, cache, and (depending on your perspective) router and switch.</p>

<p>I'll admit that I'm unfamiliar with the legal precedent, but I do know the precedent in reality, and reality is overwhelmingly in your favor.</p>

united-states internet india