requests library and http error
<p>I'm currently using the python requests library to interact with an external api which uses json.
Each endpoint works via a method (of the api class) and uses the collect_data method.</p>

<p>However I want the scraper to continue running whenever it encounters a http error (and ideally output this to a log).<br>
What's the best way to do this as currently it just breaks when I use http.raise_for_status()</p>

<p>It seems like I should be using a try/except in someway but not sure how best to do this here?</p>

<pre><code>def scrape_full_address(self, house_no, postcode):
        address_path = '/api/addresses'
        address_url = self.api_source + address_path
        payload = {
            'houseNo': house_no,
            'postcode': postcode,
        }
        return self.collect_data(url=address_url, method='get', payload=payload)


def collect_data(self, url, method, payload=None):
        if method == 'get':
            data = None
            params = payload
        elif method == 'post':
            params = None
            data = payload
        response = getattr(requests, method)(url=url, params=params, json=data, headers=self.headers)
        if response.status_code == 200:
            return response.json()
        else:
            return response.raise_for_status()  
</code></pre>