R web scraping of news site: how to pull main content
<p>I have a code below. It goes to 3 distinct news pages and pulls their content. But for each page, I have to search for a different customized html tag. </p>

<ol>
<li>Is there a way to get information from any news page by searching
for only 1 tag? That will allow me to scrape information from many
pages </li>
<li>How can i search for tags matching certain criteria: for
    example <code>html_nodes like ("#main*")</code> or <code>html_nodes like ("*main*")</code></li>
</ol>

<p><code>library(MASS)     # base R</code></p>

<pre><code>library(XML)
library(xml2)
library(rvest)    # R web scraping


    # get the page using rvest fcns
    page2 &lt;- read_html("http://seekingalpha.com/article/3974491-3-things-will-save-intels-future");

      page.all2 &lt;- page2 %&gt;%
      html_nodes("#main_container") %&gt;%
      #html_nodes(class="*main*") %&gt;%
      html_text()
     page.all2

    page5 &lt;- read_html("http://www.fool.com/investing/general/2016/05/09/intel-corporations-strange-choice.aspx");
    page.all5 &lt;- page5 %&gt;%
      html_nodes("#article-1") %&gt;%
      html_text()
      page.all5

    page1 &lt;- read_html("http://www.valuewalk.com/2016/05/intel-no-2-explains-company-headed/ ");
    page.all1 &lt;- page1 %&gt;%
      html_nodes("#main") %&gt;%
      html_text()
    page.all1

    # get a URL (1 URL per row in column link);
</code></pre>