Defining Additional Methods in a Scrapy Class
<p>How is a Scrapy class executed and how do you include additional methods to a spider class?</p>

<p>For example, from the docs:</p>

<pre><code>import scrapy

class DmozSpider(scrapy.Spider):
    name = "dmoz"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        filename = response.url.split("/")[-2] + '.html'
        with open(filename, 'wb') as f:
            f.write(response.body)
</code></pre>

<p>If I wanted to define some methods for polling a database or something else, how might I go about that, and why?</p>