{"action":"created","comment":{"url":"https://api.github.com/repos/servo/blog.servo.org/pulls/comments/35132791","id":35132791,"diff_hunk":"@@ -0,0 +1,55 @@\n+# Servo build system overview\n+\n+[Servo](https://github.com/servo/servo) is a new web browser engine. It is one of the largest Rust-based projects, but the total Rust code is still dwarfed by the size of the code provided in native C and C++ libraries. This post is an overview of how we have structured our build system in order to integrate the Cargo build system, with its \"many small and distributed dependencies\" model with our needs to provide many additional features not often found in smaller Rust-only projects.\n+\n+## Mach\n+\n+[Mach](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/mach) is a python driver program that provides a frontend to Servo's build that both reduces the number of steps to build and integrates our different testing tools into a single frontend harness. Similar to its purpose in the Firefox build, we use it to centralize and simplify the number of commands that a developer has to perform.\n+\n+### mach bootstrap\n+\n+The steps that mach will handle before issuing a normal `cargo build` command are:\n+* Downloading the correct versions of the cargo and rustc tools. Servo uses most of the unstable features in Rust, which change pretty frequently. We also test the edges of feature compatibility and so are the first ones to notice many changes that did not at first seem as if they would break anyone. Further, we build a custom version of the tools that additionally supports cross-compilation targeting Android (and ARM in the near future). A random local install of the Rust toolchain is pretty unlikely to work with Servo.\n+\n+* Updating git submodules. Some of Servo's dependencies cannot be downloaded as Cargo dependencies because they need to be directly referenced in the build process, and Cargo adds a hash that makes it difficult to locate those files. For such code, we add them as submodules.\n+\n+### mach build & run\n+\n+The build itself also verifies that the user has explicitly requested either a `dev` or `release` build --- the Servo dev build is debuggable but quite slow, and it's not clear which build should be the default.\n+\n+Additionally, there's the question of _which_ `cargo build` to run. Servo has three different \"toplevel\" Cargo.toml files.\n+* `components/servo/Cargo.toml` is used to build an executable binary named `servo` and is used on Linux and OSX. There are also horrible linker hacks in place that will cause an Android-targeted build to instead produce a file named `servo` that is actually an APK file that can be loaded onto Android devices.\n+* `ports/gonk/Cargo.toml` produces a binary that can run on the Firefox OS Boot2Gecko mobile platform.\n+* `ports/cef/Cargo.toml` produces a shared library that can be loaded within the Chromium Embedding Framework to provide a hostable web rendering engine.\n+\n+The presence of these three different toplevel binaries and the curious directory structure means that mach also provides a `run` command that will execute the correct binary with any provided arguments.\n+\n+### mach test\n+\n+Servo has several testing tools that can be executed via mach.\n+\n+* `mach tidy` will verify that there are no trivial syntactic errors in source files. It checks for valid license headers in each file, no tab characters, no trailing whitespaces, etc.\n+\n+* `mach test-ref` will run the Servo-specific reference tests. These tests render a pair of web pages that implement the same final layout using different CSS features to images. If the images are not pixel-identically, the test fails.\n+\n+* `mach test-wpt` runs the cross-browser W3C Web Platform Tests, which primarily test DOM features.\n+\n+* `mach test-css` runs the cross-browser CSS WG refernce tests, which are a version of the reference tests that are intended to work across many browsers.\n+\n+* `mach test-unit` runs the Rust unit tests embedded in Servo crates. We do not have many of these, except for basic tests of per-crate functionality.\n+\n+## cargo\n+\n+While the code that we have written for Servo is primiarly in Rust, we estimate that at least 2/3 of the code that will run inside of Servo will be written in C/C++, even when we ship. From the SpiderMonkey JavaScript engine to the Skia and Azure/Moz2D graphics pipeline to WebRTC, media extensions, and proprietary video codecs, there is a huge portion of the browser that is integrated and wrapped into Servo, rather than rewritten. For each of these projects, we have a crate that has a `build.rs` file that performs the custom build steps to produce a static library and then produce a Rust `rlib` file to link into Servo.\n+\n+The rest of Servo is a significant amount of code (~150k lines of Rust; ~250k if you include autogenerated DOM bindings), but follows the standard conventions of Cargo and Rust as far as producing crates. For the many crates within the Servo repo, we simply have a `Cargo.toml` file next to a `lib.rs` that defines the module structure. When we break them out into a separate GitHub repository, though, we follow the convention of a toplevel `Cargo.toml` file with a `src` directory that holds all of the Rust code.\n+\n+![Servo's dependency graph](https://raw.githubusercontent.com/maxsnew/cargo-dot/master/etc/servo.png \"Servo dependency graph\")\n+\n+## Things we'd like to fix in the future\n+\n+It would be great if there was a single Cargo.toml file and it was at the toplevel of the Servo repo. It's confusing to people familiar with Rust projects, who go looking for a Cargo.toml file and can't find them.","path":"_posts/2015-07-21-build.md","position":51,"original_position":51,"commit_id":"22d47343134dc3221b074b6249a77fe0a366c29b","original_commit_id":"22d47343134dc3221b074b6249a77fe0a366c29b","user":{"login":"SimonSapin","id":291359,"avatar_url":"https://avatars.githubusercontent.com/u/291359?v=3","gravatar_id":"","url":"https://api.github.com/users/SimonSapin","html_url":"https://github.com/SimonSapin","followers_url":"https://api.github.com/users/SimonSapin/followers","following_url":"https://api.github.com/users/SimonSapin/following{/other_user}","gists_url":"https://api.github.com/users/SimonSapin/gists{/gist_id}","starred_url":"https://api.github.com/users/SimonSapin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SimonSapin/subscriptions","organizations_url":"https://api.github.com/users/SimonSapin/orgs","repos_url":"https://api.github.com/users/SimonSapin/repos","events_url":"https://api.github.com/users/SimonSapin/events{/privacy}","received_events_url":"https://api.github.com/users/SimonSapin/received_events","type":"User","site_admin":false},"body":"Link to https://github.com/servo/servo/issues/5973 ?","created_at":"2015-07-21T17:56:08Z","updated_at":"2015-07-21T17:56:08Z","html_url":"https://github.com/servo/blog.servo.org/pull/20#discussion_r35132791","pull_request_url":"https://api.github.com/repos/servo/blog.servo.org/pulls/20","_links":{"self":{"href":"https://api.github.com/repos/servo/blog.servo.org/pulls/comments/35132791"},"html":{"href":"https://github.com/servo/blog.servo.org/pull/20#discussion_r35132791"},"pull_request":{"href":"https://api.github.com/repos/servo/blog.servo.org/pulls/20"}}},"pull_request":{"url":"https://api.github.com/repos/servo/blog.servo.org/pulls/20","id":40485071,"html_url":"https://github.com/servo/blog.servo.org/pull/20","diff_url":"https://github.com/servo/blog.servo.org/pull/20.diff","patch_url":"https://github.com/servo/blog.servo.org/pull/20.patch","issue_url":"https://api.github.com/repos/servo/blog.servo.org/issues/20","number":20,"state":"open","locked":false,"title":"Add a post on the build system","user":{"login":"larsbergstrom","id":475847,"avatar_url":"https://avatars.githubusercontent.com/u/475847?v=3","gravatar_id":"","url":"https://api.github.com/users/larsbergstrom","html_url":"https://github.com/larsbergstrom","followers_url":"https://api.github.com/users/larsbergstrom/followers","following_url":"https://api.github.com/users/larsbergstrom/following{/other_user}","gists_url":"https://api.github.com/users/larsbergstrom/gists{/gist_id}","starred_url":"https://api.github.com/users/larsbergstrom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/larsbergstrom/subscriptions","organizations_url":"https://api.github.com/users/larsbergstrom/orgs","repos_url":"https://api.github.com/users/larsbergstrom/repos","events_url":"https://api.github.com/users/larsbergstrom/events{/privacy}","received_events_url":"https://api.github.com/users/larsbergstrom/received_events","type":"User","site_admin":false},"body":"r? @metajack \r\n\r\nWrote this up from some notes I was already preparing on Servo's build system for @alexcrichton anyway :-)\n\n<!-- Reviewable:start -->\n[<img src=\"https://reviewable.io/review_button.png\" height=40 alt=\"Review on Reviewable\"/>](https://reviewable.io/reviews/servo/blog.servo.org/20)\n<!-- Reviewable:end -->\n","created_at":"2015-07-21T17:50:43Z","updated_at":"2015-07-21T17:56:08Z","closed_at":null,"merged_at":null,"merge_commit_sha":"503ea863ee9f9712ba9597bdbcfbdc2d62925b05","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/servo/blog.servo.org/pulls/20/commits","review_comments_url":"https://api.github.com/repos/servo/blog.servo.org/pulls/20/comments","review_comment_url":"https://api.github.com/repos/servo/blog.servo.org/pulls/comments{/number}","comments_url":"https://api.github.com/repos/servo/blog.servo.org/issues/20/comments","statuses_url":"https://api.github.com/repos/servo/blog.servo.org/statuses/22d47343134dc3221b074b6249a77fe0a366c29b","head":{"label":"larsbergstrom:gh-pages","ref":"gh-pages","sha":"22d47343134dc3221b074b6249a77fe0a366c29b","user":{"login":"larsbergstrom","id":475847,"avatar_url":"https://avatars.githubusercontent.com/u/475847?v=3","gravatar_id":"","url":"https://api.github.com/users/larsbergstrom","html_url":"https://github.com/larsbergstrom","followers_url":"https://api.github.com/users/larsbergstrom/followers","following_url":"https://api.github.com/users/larsbergstrom/following{/other_user}","gists_url":"https://api.github.com/users/larsbergstrom/gists{/gist_id}","starred_url":"https://api.github.com/users/larsbergstrom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/larsbergstrom/subscriptions","organizations_url":"https://api.github.com/users/larsbergstrom/orgs","repos_url":"https://api.github.com/users/larsbergstrom/repos","events_url":"https://api.github.com/users/larsbergstrom/events{/privacy}","received_events_url":"https://api.github.com/users/larsbergstrom/received_events","type":"User","site_admin":false},"repo":{"id":39460822,"name":"blog.servo.org","full_name":"larsbergstrom/blog.servo.org","owner":{"login":"larsbergstrom","id":475847,"avatar_url":"https://avatars.githubusercontent.com/u/475847?v=3","gravatar_id":"","url":"https://api.github.com/users/larsbergstrom","html_url":"https://github.com/larsbergstrom","followers_url":"https://api.github.com/users/larsbergstrom/followers","following_url":"https://api.github.com/users/larsbergstrom/following{/other_user}","gists_url":"https://api.github.com/users/larsbergstrom/gists{/gist_id}","starred_url":"https://api.github.com/users/larsbergstrom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/larsbergstrom/subscriptions","organizations_url":"https://api.github.com/users/larsbergstrom/orgs","repos_url":"https://api.github.com/users/larsbergstrom/repos","events_url":"https://api.github.com/users/larsbergstrom/events{/privacy}","received_events_url":"https://api.github.com/users/larsbergstrom/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/larsbergstrom/blog.servo.org","description":"The Servo blog","fork":true,"url":"https://api.github.com/repos/larsbergstrom/blog.servo.org","forks_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/forks","keys_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/keys{/key_id}","collaborators_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/teams","hooks_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/hooks","issue_events_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/issues/events{/number}","events_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/events","assignees_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/assignees{/user}","branches_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/branches{/branch}","tags_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/tags","blobs_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/git/refs{/sha}","trees_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/git/trees{/sha}","statuses_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/statuses/{sha}","languages_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/languages","stargazers_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/stargazers","contributors_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/contributors","subscribers_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/subscribers","subscription_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/subscription","commits_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/commits{/sha}","git_commits_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/git/commits{/sha}","comments_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/comments{/number}","issue_comment_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/issues/comments{/number}","contents_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/contents/{+path}","compare_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/compare/{base}...{head}","merges_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/merges","archive_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/downloads","issues_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/issues{/number}","pulls_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/pulls{/number}","milestones_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/milestones{/number}","notifications_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/labels{/name}","releases_url":"https://api.github.com/repos/larsbergstrom/blog.servo.org/releases{/id}","created_at":"2015-07-21T17:48:11Z","updated_at":"2015-07-21T17:48:11Z","pushed_at":"2015-07-21T17:49:38Z","git_url":"git://github.com/larsbergstrom/blog.servo.org.git","ssh_url":"git@github.com:larsbergstrom/blog.servo.org.git","clone_url":"https://github.com/larsbergstrom/blog.servo.org.git","svn_url":"https://github.com/larsbergstrom/blog.servo.org","homepage":"http://blog.servo.org","size":1042,"stargazers_count":0,"watchers_count":0,"language":"CSS","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"gh-pages"}},"base":{"label":"servo:gh-pages","ref":"gh-pages","sha":"d8eba4d831cd97c8ee8303215c80c932072fe8d5","user":{"login":"servo","id":2566135,"avatar_url":"https://avatars.githubusercontent.com/u/2566135?v=3","gravatar_id":"","url":"https://api.github.com/users/servo","html_url":"https://github.com/servo","followers_url":"https://api.github.com/users/servo/followers","following_url":"https://api.github.com/users/servo/following{/other_user}","gists_url":"https://api.github.com/users/servo/gists{/gist_id}","starred_url":"https://api.github.com/users/servo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/servo/subscriptions","organizations_url":"https://api.github.com/users/servo/orgs","repos_url":"https://api.github.com/users/servo/repos","events_url":"https://api.github.com/users/servo/events{/privacy}","received_events_url":"https://api.github.com/users/servo/received_events","type":"Organization","site_admin":false},"repo":{"id":23540436,"name":"blog.servo.org","full_name":"servo/blog.servo.org","owner":{"login":"servo","id":2566135,"avatar_url":"https://avatars.githubusercontent.com/u/2566135?v=3","gravatar_id":"","url":"https://api.github.com/users/servo","html_url":"https://github.com/servo","followers_url":"https://api.github.com/users/servo/followers","following_url":"https://api.github.com/users/servo/following{/other_user}","gists_url":"https://api.github.com/users/servo/gists{/gist_id}","starred_url":"https://api.github.com/users/servo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/servo/subscriptions","organizations_url":"https://api.github.com/users/servo/orgs","repos_url":"https://api.github.com/users/servo/repos","events_url":"https://api.github.com/users/servo/events{/privacy}","received_events_url":"https://api.github.com/users/servo/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/servo/blog.servo.org","description":"The Servo blog","fork":false,"url":"https://api.github.com/repos/servo/blog.servo.org","forks_url":"https://api.github.com/repos/servo/blog.servo.org/forks","keys_url":"https://api.github.com/repos/servo/blog.servo.org/keys{/key_id}","collaborators_url":"https://api.github.com/repos/servo/blog.servo.org/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/servo/blog.servo.org/teams","hooks_url":"https://api.github.com/repos/servo/blog.servo.org/hooks","issue_events_url":"https://api.github.com/repos/servo/blog.servo.org/issues/events{/number}","events_url":"https://api.github.com/repos/servo/blog.servo.org/events","assignees_url":"https://api.github.com/repos/servo/blog.servo.org/assignees{/user}","branches_url":"https://api.github.com/repos/servo/blog.servo.org/branches{/branch}","tags_url":"https://api.github.com/repos/servo/blog.servo.org/tags","blobs_url":"https://api.github.com/repos/servo/blog.servo.org/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/servo/blog.servo.org/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/servo/blog.servo.org/git/refs{/sha}","trees_url":"https://api.github.com/repos/servo/blog.servo.org/git/trees{/sha}","statuses_url":"https://api.github.com/repos/servo/blog.servo.org/statuses/{sha}","languages_url":"https://api.github.com/repos/servo/blog.servo.org/languages","stargazers_url":"https://api.github.com/repos/servo/blog.servo.org/stargazers","contributors_url":"https://api.github.com/repos/servo/blog.servo.org/contributors","subscribers_url":"https://api.github.com/repos/servo/blog.servo.org/subscribers","subscription_url":"https://api.github.com/repos/servo/blog.servo.org/subscription","commits_url":"https://api.github.com/repos/servo/blog.servo.org/commits{/sha}","git_commits_url":"https://api.github.com/repos/servo/blog.servo.org/git/commits{/sha}","comments_url":"https://api.github.com/repos/servo/blog.servo.org/comments{/number}","issue_comment_url":"https://api.github.com/repos/servo/blog.servo.org/issues/comments{/number}","contents_url":"https://api.github.com/repos/servo/blog.servo.org/contents/{+path}","compare_url":"https://api.github.com/repos/servo/blog.servo.org/compare/{base}...{head}","merges_url":"https://api.github.com/repos/servo/blog.servo.org/merges","archive_url":"https://api.github.com/repos/servo/blog.servo.org/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/servo/blog.servo.org/downloads","issues_url":"https://api.github.com/repos/servo/blog.servo.org/issues{/number}","pulls_url":"https://api.github.com/repos/servo/blog.servo.org/pulls{/number}","milestones_url":"https://api.github.com/repos/servo/blog.servo.org/milestones{/number}","notifications_url":"https://api.github.com/repos/servo/blog.servo.org/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/servo/blog.servo.org/labels{/name}","releases_url":"https://api.github.com/repos/servo/blog.servo.org/releases{/id}","created_at":"2014-09-01T11:37:43Z","updated_at":"2015-05-23T08:24:13Z","pushed_at":"2015-07-21T17:50:48Z","git_url":"git://github.com/servo/blog.servo.org.git","ssh_url":"git@github.com:servo/blog.servo.org.git","clone_url":"https://github.com/servo/blog.servo.org.git","svn_url":"https://github.com/servo/blog.servo.org","homepage":"http://blog.servo.org","size":1042,"stargazers_count":2,"watchers_count":2,"language":"CSS","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":7,"mirror_url":null,"open_issues_count":2,"forks":7,"open_issues":2,"watchers":2,"default_branch":"gh-pages"}},"_links":{"self":{"href":"https://api.github.com/repos/servo/blog.servo.org/pulls/20"},"html":{"href":"https://github.com/servo/blog.servo.org/pull/20"},"issue":{"href":"https://api.github.com/repos/servo/blog.servo.org/issues/20"},"comments":{"href":"https://api.github.com/repos/servo/blog.servo.org/issues/20/comments"},"review_comments":{"href":"https://api.github.com/repos/servo/blog.servo.org/pulls/20/comments"},"review_comment":{"href":"https://api.github.com/repos/servo/blog.servo.org/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/servo/blog.servo.org/pulls/20/commits"},"statuses":{"href":"https://api.github.com/repos/servo/blog.servo.org/statuses/22d47343134dc3221b074b6249a77fe0a366c29b"}}}}