Two scrapy item loaders in a single parse method? how to merge them?
<p>I'm parsing a page but id like to divide it into sections, the page has info about multiple meetings. Some of the info is common to all meetings, but not everything. So i made an item loader for the general info and one for the specific info. However I'd like this parser to return all the info pertaining to a meeting (i.e: the general and specific). Here's the parse method of my code:</p>

<pre><code>def parse(self, response):

    general_loader = ItemLoader(item=ProductItem(), response=response)
    general_loader.default_input_processor = MapCompose(unicode.strip)
    general_loader.default_output_processor = Join("  &amp;  ")

    for field, xpath in self.general_item_fields.iteritems():
        general_loader.add_xpath(field, xpath)

    for meeting in response.xpath(self.meeting_xpath):
        specific_loader = ItemLoader(item=ProductItem(), response=meeting)
        specific_loader.default_input_processor = MapCompose(unicode.strip)
        specific_loader.default_output_processor = Join("  &amp;  ")

        for field, xpath in self.specific_item_fields.iteritems():
            specific_loader.add_xpath(field, xpath)
        yield general_loader.load_item().update(specific_loader.load_item())
</code></pre>

<p>The variables specific_item_fields and general_item_fields are dictionaries with the attribute of the meeting and it's xpath.</p>

<p>So what I'm trying to do here is use meeting as the response for a second ItemLoader that i called specific_loader. And since general_loader.load_item() seems to returns a dictionary, I tried updating or merging it with the specific_loader.load_item() dictionary. </p>

<p>Here's where I'm stuck:</p>

<ul>
<li>The update method is not working on the load_item and I can't seem to merge these two things.</li>
<li>Apparently i can't use a response.xpath() (I'm using meeting here)element for the loader response?</li>
<li>Finally there must be a better way to implement this, I've tried nested loaders and they seem very promising but meeting changes. It cycles through the response.xpath(self.meeting_xpath) list so how could i use nested loaders?</li>
</ul>

<p>Thanks in advance for any pointers or advice, I'm kinda lost :)</p>