Scrapy Crawler does not working when there is no link pagination on root page
<p>My scrapy crawler don't work as expected all the times.</p>

<p>I scrap some categorie from a webshop which contains pagination inside each categorie page but sometimes it does not contain the link pagination because there is only one page</p>

<p>My rule works only if link pagination are inside the page i selected  but i don't know how to make it works when the root page of the category is the only one.</p>

<p>here is my rule.</p>

<pre><code>rules = [
    Rule(LinkExtractor(allow=(),restrict_xpaths=('//a[@class="button button_gris button_medium"]',)), callback='parse_item', follow=True)
</code></pre>

<p>Any idea how to make it to scrape also the page not containing pagination link which i target.</p>

<p>Note:</p>

<p>I launch the crawl spider using a start url populated by mysqlDB like this for example where the arg table used to select my link table where start url is used.</p>

<pre><code>scrapy crawl mg_bot_crawler -a table=mini_four 
</code></pre>

<p>thanks for your help</p>