{"action":"created","comment":{"url":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments/39603683","id":39603683,"diff_hunk":"@@ -0,0 +1,200 @@\n+Understand the CSV Product Export\n+=================================\n+\n+We'll discuss here how the csv product export works.\n+\n+It's a good start to understand the overall architecture and how to re-use or replace some parts.\n+\n+.. note::\n+\n+  The import part has been widely re-worked in 1.4, please note that the export part remains the same than in 1.3 (all classes and services are in the BaseConnectorBundle).\n+\n+Definition of the Job\n+---------------------\n+\n+The product export is defined in ``src\\Pim\\Bundle\\BaseConnectorBundle\\Resources\\config\\batch_jobs.yml``.\n+\n+.. code-block:: yaml\n+\n+    connector:\n+        name: Akeneo CSV Connector\n+        jobs:\n+            csv_product_export:\n+                title: pim_base_connector.jobs.csv_product_export.title\n+                type:  export\n+                steps:\n+                    export:\n+                        title:     pim_base_connector.jobs.csv_product_export.export.title\n+                        services:\n+                            reader:    pim_base_connector.reader.doctrine.product\n+                            processor: pim_base_connector.processor.product_to_flat_array\n+                            writer:    pim_base_connector.writer.file.csv_product\n+                        parameters:\n+                            batch_size: 10\n+\n+With the ``type`` parameter, we can see that this job is an export.\n+\n+We can also count a single step named ``export``.\n+\n+Product Export Step\n+-------------------\n+\n+The purpose of this step is to read products from database, to transform product objects to array and write lines in a csv file.\n+\n+This step is a default step, an ``Akeneo\\Bundle\\BatchBundle\\Step\\ItemStep`` is instanciated and injected.\n+\n+.. code-block:: yaml\n+\n+    [...]\n+    export:\n+        title:         pim_base_connector.jobs.csv_product_export.export.title\n+        services:\n+            reader:    pim_base_connector.reader.doctrine.product\n+            processor: pim_base_connector.processor.product_to_flat_array\n+            writer:    pim_base_connector.writer.file.csv_product\n+        parameters:\n+            batch_size: 10\n+    [...]\n+\n+An ``ItemStep`` always contains 3 elements, a ``Akeneo\\Bundle\\BatchBundle\\Item\\ItemReaderInterface``, a ``Akeneo\\Bundle\\BatchBundle\\Item\\ItemProcessorInterface`` and a ``Akeneo\\Bundle\\BatchBundle\\Item\\ItemWriterInterface``.\n+\n+We provide here specific implementations for these elements, the services are declared with aliases ``pim_base_connector.reader.doctrine.product``, ``pim_base_connector.processor.product_to_flat_array``, ``pim_base_connector.writer.file.csv_product``.\n+\n+Product Reader\n+--------------\n+\n+This element reads products from database and returns objects one by one.\n+\n+The service is defined in ``src\\Pim\\Bundle\\BaseConnectorBundle\\Resources\\config\\readers.yml``.\n+\n+.. code-block:: yaml\n+\n+    services:\n+        pim_base_connector.reader.doctrine.product:\n+            class: %pim_base_connector.reader.doctrine.product.class%\n+            arguments:\n+                - '@pim_catalog.repository.product'\n+                - '@pim_catalog.manager.channel'\n+                - '@pim_catalog.manager.completeness'\n+                - '@pim_transform.converter.metric'\n+                - '@pim_catalog.object_manager.product'\n+\n+You can notice that the class parameter is not defined in the same file.\n+\n+Depending on the storage you're using to store your product data, a specific file will be loaded (orm.yml or mongodb-odm.yml).\n+\n+For ORM, the parameter is defined in ``src\\Pim\\Bundle\\BaseConnectorBundle\\Resources\\config\\storage_driver\\doctrine\\orm.yml``.\n+\n+.. code-block:: yaml\n+\n+    parameters:\n+        pim_base_connector.reader.doctrine.product.class: Pim\\Bundle\\BaseConnectorBundle\\Reader\\Doctrine\\ORMProductReader\n+\n+For MongoDBODM, the parameter is defined in ``src\\Pim\\Bundle\\BaseConnectorBundle\\Resources\\config\\storage_driver\\doctrine\\mongodb-odm.yml``.\n+\n+.. code-block:: yaml\n+\n+    parameters:\n+        pim_base_connector.reader.doctrine.product.class: Pim\\Bundle\\BaseConnectorBundle\\Reader\\Doctrine\\ODMProductReader\n+\n+The reader will only return products that are complete for the selected channel, classified in a category and enabled.\n+\n+Product Processor - Overview\n+----------------------------\n+\n+This element receives product objects one by one, transform each product object to an array and returns the array\n+\n+The service is defined in ``src\\Pim\\Bundle\\BaseConnectorBundle\\Resources\\config\\processors.yml``.\n+\n+.. code-block:: yaml\n+\n+    parameters:\n+        pim_base_connector.processor.product_to_flat_array.class: Pim\\Bundle\\BaseConnectorBundle\\Processor\\ProductToFlatArrayProcessor\n+\n+    services:\n+        pim_base_connector.processor.product_to_flat_array:\n+            class: %pim_base_connector.processor.product_to_flat_array.class%\n+            arguments:\n+                - '@pim_serializer'\n+                - '@pim_catalog.manager.channel'\n+                - ['pim_catalog_file', 'pim_catalog_image']\n+\n+The class ``Pim\\Bundle\\BaseConnectorBundle\\Processor\\ProductToFlatArrayProcessor`` mainly delegates the transformation to the service ``pim_serializer``.\n+\n+We can see here that we normalize each product to the ``flat`` format (=csv format).\n+\n+.. code-block:: php\n+\n+    $data['product'] = $this->serializer->normalize($product, 'flat', $this->getNormalizerContext());\n+\n+This service ``pim_serializer`` is declared in ``src\\Pim\\Bundle\\TransformerBundle\\Resources\\config\\serializer\\serializer.yml`` and uses the Symfony ``Serializer`` class.\n+\n+We register several normalizers to normalize any kind of objects to a flat array, these normalizers are defined in ``src\\Pim\\Bundle\\TransformerBundle\\Resources\\config\\serializer\\flat.yml``.\n+\n+As any product does not have values for all attributes, depending on the product, the normalized array will contain differents keys, for instance,","path":"cookbook/import_export/product-export.rst","position":134,"original_position":134,"commit_id":"7046fe580448489127913e7bb395385b8ea0feec","original_commit_id":"7046fe580448489127913e7bb395385b8ea0feec","user":{"login":"grena","id":301169,"avatar_url":"https://avatars.githubusercontent.com/u/301169?v=3","gravatar_id":"","url":"https://api.github.com/users/grena","html_url":"https://github.com/grena","followers_url":"https://api.github.com/users/grena/followers","following_url":"https://api.github.com/users/grena/following{/other_user}","gists_url":"https://api.github.com/users/grena/gists{/gist_id}","starred_url":"https://api.github.com/users/grena/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/grena/subscriptions","organizations_url":"https://api.github.com/users/grena/orgs","repos_url":"https://api.github.com/users/grena/repos","events_url":"https://api.github.com/users/grena/events{/privacy}","received_events_url":"https://api.github.com/users/grena/received_events","type":"User","site_admin":false},"body":"`As a product may not have values for all attributes`","created_at":"2015-09-16T08:16:22Z","updated_at":"2015-09-16T08:16:22Z","html_url":"https://github.com/akeneo/pim-docs/pull/117#discussion_r39603683","pull_request_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/117","_links":{"self":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments/39603683"},"html":{"href":"https://github.com/akeneo/pim-docs/pull/117#discussion_r39603683"},"pull_request":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/117"}}},"pull_request":{"url":"https://api.github.com/repos/akeneo/pim-docs/pulls/117","id":45077974,"html_url":"https://github.com/akeneo/pim-docs/pull/117","diff_url":"https://github.com/akeneo/pim-docs/pull/117.diff","patch_url":"https://github.com/akeneo/pim-docs/pull/117.patch","issue_url":"https://api.github.com/repos/akeneo/pim-docs/issues/117","number":117,"state":"open","locked":false,"title":"PIM-4446: add a chapter to explain the product export","user":{"login":"nidup","id":2104359,"avatar_url":"https://avatars.githubusercontent.com/u/2104359?v=3","gravatar_id":"","url":"https://api.github.com/users/nidup","html_url":"https://github.com/nidup","followers_url":"https://api.github.com/users/nidup/followers","following_url":"https://api.github.com/users/nidup/following{/other_user}","gists_url":"https://api.github.com/users/nidup/gists{/gist_id}","starred_url":"https://api.github.com/users/nidup/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nidup/subscriptions","organizations_url":"https://api.github.com/users/nidup/orgs","repos_url":"https://api.github.com/users/nidup/repos","events_url":"https://api.github.com/users/nidup/events{/privacy}","received_events_url":"https://api.github.com/users/nidup/received_events","type":"User","site_admin":false},"body":"","created_at":"2015-09-15T17:27:11Z","updated_at":"2015-09-16T08:16:22Z","closed_at":null,"merged_at":null,"merge_commit_sha":"45936e5469cc9520fe7871cc7eb14707fb82d615","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/117/commits","review_comments_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/117/comments","review_comment_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/akeneo/pim-docs/issues/117/comments","statuses_url":"https://api.github.com/repos/akeneo/pim-docs/statuses/7046fe580448489127913e7bb395385b8ea0feec","head":{"label":"akeneo:PIM-4446","ref":"PIM-4446","sha":"7046fe580448489127913e7bb395385b8ea0feec","user":{"login":"akeneo","id":2510186,"avatar_url":"https://avatars.githubusercontent.com/u/2510186?v=3","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"repo":{"id":12193604,"name":"pim-docs","full_name":"akeneo/pim-docs","owner":{"login":"akeneo","id":2510186,"avatar_url":"https://avatars.githubusercontent.com/u/2510186?v=3","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/akeneo/pim-docs","description":"Akeneo PIM documentation","fork":false,"url":"https://api.github.com/repos/akeneo/pim-docs","forks_url":"https://api.github.com/repos/akeneo/pim-docs/forks","keys_url":"https://api.github.com/repos/akeneo/pim-docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/akeneo/pim-docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/akeneo/pim-docs/teams","hooks_url":"https://api.github.com/repos/akeneo/pim-docs/hooks","issue_events_url":"https://api.github.com/repos/akeneo/pim-docs/issues/events{/number}","events_url":"https://api.github.com/repos/akeneo/pim-docs/events","assignees_url":"https://api.github.com/repos/akeneo/pim-docs/assignees{/user}","branches_url":"https://api.github.com/repos/akeneo/pim-docs/branches{/branch}","tags_url":"https://api.github.com/repos/akeneo/pim-docs/tags","blobs_url":"https://api.github.com/repos/akeneo/pim-docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/akeneo/pim-docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/akeneo/pim-docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/akeneo/pim-docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/akeneo/pim-docs/statuses/{sha}","languages_url":"https://api.github.com/repos/akeneo/pim-docs/languages","stargazers_url":"https://api.github.com/repos/akeneo/pim-docs/stargazers","contributors_url":"https://api.github.com/repos/akeneo/pim-docs/contributors","subscribers_url":"https://api.github.com/repos/akeneo/pim-docs/subscribers","subscription_url":"https://api.github.com/repos/akeneo/pim-docs/subscription","commits_url":"https://api.github.com/repos/akeneo/pim-docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/akeneo/pim-docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/akeneo/pim-docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/akeneo/pim-docs/issues/comments{/number}","contents_url":"https://api.github.com/repos/akeneo/pim-docs/contents/{+path}","compare_url":"https://api.github.com/repos/akeneo/pim-docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/akeneo/pim-docs/merges","archive_url":"https://api.github.com/repos/akeneo/pim-docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/akeneo/pim-docs/downloads","issues_url":"https://api.github.com/repos/akeneo/pim-docs/issues{/number}","pulls_url":"https://api.github.com/repos/akeneo/pim-docs/pulls{/number}","milestones_url":"https://api.github.com/repos/akeneo/pim-docs/milestones{/number}","notifications_url":"https://api.github.com/repos/akeneo/pim-docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/akeneo/pim-docs/labels{/name}","releases_url":"https://api.github.com/repos/akeneo/pim-docs/releases{/id}","created_at":"2013-08-18T10:17:30Z","updated_at":"2015-09-11T13:18:14Z","pushed_at":"2015-09-15T17:31:40Z","git_url":"git://github.com/akeneo/pim-docs.git","ssh_url":"git@github.com:akeneo/pim-docs.git","clone_url":"https://github.com/akeneo/pim-docs.git","svn_url":"https://github.com/akeneo/pim-docs","homepage":"","size":3355,"stargazers_count":13,"watchers_count":13,"language":"CSS","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":28,"mirror_url":null,"open_issues_count":8,"forks":28,"open_issues":8,"watchers":13,"default_branch":"master"}},"base":{"label":"akeneo:master","ref":"master","sha":"53c9860f83bda98ef92bd2b3b523b957ea5331e8","user":{"login":"akeneo","id":2510186,"avatar_url":"https://avatars.githubusercontent.com/u/2510186?v=3","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"repo":{"id":12193604,"name":"pim-docs","full_name":"akeneo/pim-docs","owner":{"login":"akeneo","id":2510186,"avatar_url":"https://avatars.githubusercontent.com/u/2510186?v=3","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/akeneo/pim-docs","description":"Akeneo PIM documentation","fork":false,"url":"https://api.github.com/repos/akeneo/pim-docs","forks_url":"https://api.github.com/repos/akeneo/pim-docs/forks","keys_url":"https://api.github.com/repos/akeneo/pim-docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/akeneo/pim-docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/akeneo/pim-docs/teams","hooks_url":"https://api.github.com/repos/akeneo/pim-docs/hooks","issue_events_url":"https://api.github.com/repos/akeneo/pim-docs/issues/events{/number}","events_url":"https://api.github.com/repos/akeneo/pim-docs/events","assignees_url":"https://api.github.com/repos/akeneo/pim-docs/assignees{/user}","branches_url":"https://api.github.com/repos/akeneo/pim-docs/branches{/branch}","tags_url":"https://api.github.com/repos/akeneo/pim-docs/tags","blobs_url":"https://api.github.com/repos/akeneo/pim-docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/akeneo/pim-docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/akeneo/pim-docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/akeneo/pim-docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/akeneo/pim-docs/statuses/{sha}","languages_url":"https://api.github.com/repos/akeneo/pim-docs/languages","stargazers_url":"https://api.github.com/repos/akeneo/pim-docs/stargazers","contributors_url":"https://api.github.com/repos/akeneo/pim-docs/contributors","subscribers_url":"https://api.github.com/repos/akeneo/pim-docs/subscribers","subscription_url":"https://api.github.com/repos/akeneo/pim-docs/subscription","commits_url":"https://api.github.com/repos/akeneo/pim-docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/akeneo/pim-docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/akeneo/pim-docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/akeneo/pim-docs/issues/comments{/number}","contents_url":"https://api.github.com/repos/akeneo/pim-docs/contents/{+path}","compare_url":"https://api.github.com/repos/akeneo/pim-docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/akeneo/pim-docs/merges","archive_url":"https://api.github.com/repos/akeneo/pim-docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/akeneo/pim-docs/downloads","issues_url":"https://api.github.com/repos/akeneo/pim-docs/issues{/number}","pulls_url":"https://api.github.com/repos/akeneo/pim-docs/pulls{/number}","milestones_url":"https://api.github.com/repos/akeneo/pim-docs/milestones{/number}","notifications_url":"https://api.github.com/repos/akeneo/pim-docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/akeneo/pim-docs/labels{/name}","releases_url":"https://api.github.com/repos/akeneo/pim-docs/releases{/id}","created_at":"2013-08-18T10:17:30Z","updated_at":"2015-09-11T13:18:14Z","pushed_at":"2015-09-15T17:31:40Z","git_url":"git://github.com/akeneo/pim-docs.git","ssh_url":"git@github.com:akeneo/pim-docs.git","clone_url":"https://github.com/akeneo/pim-docs.git","svn_url":"https://github.com/akeneo/pim-docs","homepage":"","size":3355,"stargazers_count":13,"watchers_count":13,"language":"CSS","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":28,"mirror_url":null,"open_issues_count":8,"forks":28,"open_issues":8,"watchers":13,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/117"},"html":{"href":"https://github.com/akeneo/pim-docs/pull/117"},"issue":{"href":"https://api.github.com/repos/akeneo/pim-docs/issues/117"},"comments":{"href":"https://api.github.com/repos/akeneo/pim-docs/issues/117/comments"},"review_comments":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/117/comments"},"review_comment":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/117/commits"},"statuses":{"href":"https://api.github.com/repos/akeneo/pim-docs/statuses/7046fe580448489127913e7bb395385b8ea0feec"}}}}