<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Render site pages
dpp runs the knesset data pipelines periodically on our server.
This notebook shows how to run pipelines that render pages for the static website at https
Step1:  Run the build pipeline
This pipeline aggregates the relevant data and allows to filter for quicker development cycles.
You can uncomment and modify the filter step in committees/dist/knesset.source-spec.yaml under the build pipeline to change the filter.
The build pipeline can take a few minutes to process for the first time.
Step2:  Download some protocol files for rendering
upgrade to latest dataflows library
Step3:  Restart the kernel if an upgrade was done
Choose some session IDs to download protocol files for
Step4:  Delete dist hash files
Step5:  Render pages
Should run the render pipelines in the following order
Step6:  Rendered meetings stats
Step7:  Committees and homepage
Step8:  Members / Factions
<ASSISTANT_TASK:>
Python Code:
!{'cd /pipelines; KNESSET_LOAD_FROM_URL=1 dpp run --concurrency 4 '\
  './committees/kns_committee,'\
  './people/committee-meeting-attendees,'\
  './members/mk_individual'}
Explanation: Render site pages
dpp runs the knesset data pipelines periodically on our server.
This notebook shows how to run pipelines that render pages for the static website at https://oknesset.org
Load the source data
Download the source data, can take a few minutes.
End of explanation
!{'cd /pipelines; dpp run --verbose ./committees/dist/build'}
Explanation: Run the build pipeline
This pipeline aggregates the relevant data and allows to filter for quicker development cycles.
You can uncomment and modify the filter step in committees/dist/knesset.source-spec.yaml under the build pipeline to change the filter.
The build pipeline can take a few minutes to process for the first time.
End of explanation
!{'pip install --upgrade dataflows'}
Explanation: Download some protocol files for rendering
upgrade to latest dataflows library
End of explanation
session_ids = [2063122, 2063126]
from dataflows import Flow, load, printer, filter_rows
sessions_data = Flow(
    load('/pipelines/data/committees/kns_committeesession/datapackage.json'),
    filter_rows(lambda row: row['CommitteeSessionID'] in session_ids),
    printer(tablefmt='html')
).results()
import os
import subprocess
import sys
for session in sessions_data[0][0]:
    for attr in ['text_parsed_filename', 'parts_parsed_filename']:
        pathpart = 'meeting_protocols_text' if attr == 'text_parsed_filename' else 'meeting_protocols_parts'
        url = 'https://production.oknesset.org/pipelines/data/committees/{}/{}'.format(pathpart, session[attr])
        filename = '/pipelines/data/committees/{}/{}'.format(pathpart, session[attr])
        os.makedirs(os.path.dirname(filename), exist_ok=True)
        cmd = 'curl -s -o {} {}'.format(filename, url)
        print(cmd, file=sys.stderr)
        subprocess.check_call(cmd, shell=True)
Explanation: Restart the kernel if an upgrade was done
Choose some session IDs to download protocol files for:
End of explanation
%%bash
find /pipelines/data/committees/dist -type f -name '*.hash' -delete
Explanation: Delete dist hash files
End of explanation
!{'cd /pipelines; dpp run ./committees/dist/render_meetings'}
Explanation: Render pages
Should run the render pipelines in the following order:
Meetings:
End of explanation
from dataflows import Flow, load, printer, filter_rows, add_field
def add_filenames():
    
    def _add_filenames(row):
        for ext in ['html', 'json']:
            row['rendered_'+ext] = '/pipelines/data/committees/dist/dist/meetings/{}/{}/{}.{}'.format(
                str(row['CommitteeSessionID'])[0], str(row['CommitteeSessionID'])[1], str(row['CommitteeSessionID']), ext)
    
    return Flow(
        add_field('rendered_html', 'string'),
        add_field('rendered_json', 'string'),
        _add_filenames
    )
rendered_meetings = Flow(
    load('/pipelines/data/committees/dist/rendered_meetings_stats/datapackage.json'), 
    add_filenames(),
    filter_rows(lambda row: row['CommitteeSessionID'] in session_ids),
    printer(tablefmt='html')
).results()[0][0]
Explanation: Rendered meetings stats
End of explanation
!{'cd /pipelines; dpp run ./committees/dist/render_committees'}
Explanation: Committees and homepage
End of explanation
!{'cd /pipelines; dpp run ./committees/dist/create_members,./committees/dist/build_positions,./committees/dist/create_factions'}
Explanation: Members / Factions
End of explanation
<END_TASK>