error when reading .txt file from URL in R
<p>I'm having some issues with reading data from the web into R. </p>

<p>The data that I wish to download are stored <a href="http://data.giss.nasa.gov/gistemp/station_data/" rel="nofollow">here</a>, where on this page you can insert the station number and then be directed to the dataset. For example, if I type station number 615071800000 into the box I will be redirected to another page where I can then select the data and ask to view the txt file in the browser. </p>

<p>As I have a large number of stations that I want data from I decided to write a script in R to do this. This is an example below:</p>

<pre><code>site_id &lt;- c('615071800000','629065900000','617106090000','617107080000',
             '615071900000','646066100000','646066420010','646067000000')
for (i in 1:length(site_id)){
  web_str &lt;- 'http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_'
  str &lt;- paste(web_str,site_id[i],'_14_0/station.txt', sep = '')

  # read data from web
  dat &lt;- read.table(str, sep = "", header = TRUE,
                    fill = TRUE, na.string = c(999.9,999.90))
}
</code></pre>

<p>These are all valid site id's and the link provided and used to generate 'str' should lead me to the data. However, when I run the code on this example set, it fails. Running this example, I get:</p>

<pre><code>Error in file(file, "rt") : cannot open the connection
In addition: Warning message:
In file(file, "rt") : cannot open: HTTP status was '404 Not Found'
</code></pre>

<p>which for me occurs at loop i == 7</p>

<p>So, I copy the 'str' for i = 7 into my browser</p>

<p><a href="http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_646066420010_14_0/station.txt" rel="nofollow">http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_646066420010_14_0/station.txt</a></p>

<p>but the page is empty and I get a message 'Not found' on the top of the page. </p>

<p>However, if I go back to the main page (i.e. the first link provided here) and type the station number into the field, it takes me to the data, I then copy the link for that page into R:</p>

<p><a href="http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_646066420010_14_0/station.txt" rel="nofollow">http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_646066420010_14_0/station.txt</a></p>

<p>which turns out to be exactly the same string as I tried previously. Why did this not work? Even more strange is that if I now try to run the code again, R now returns the same error at i == 8, that is it's found the previous file but not the next one. What's going on here? Is this an issue with the website, such as the txt files are only generated when you manually search for the stations on that first page? Is there a workaround for this, or does anyone have a suggestion to how I can make this work? Or can someone verfiy that this is also an issue on another machine. </p>