{"action":"created","comment":{"url":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/comments/40619178","id":40619178,"diff_hunk":"@@ -0,0 +1,257 @@\n+# ForwardDiff.jl: Automatic Differentiation in Julia\n+\n+**Jarrett Revels**\n+\n+This summer, I've had the good fortune to be able to participate in the first ever **Julia Summer of Code (JSoC)**, generously sponsored by the Gordon and Betty Moore Foundation. My JSoC project was to explore the use of Julia for **automatic differentiation (AD)**, a topic with a wide array of applications in the field of optimization.\n+\n+Under the mentorship of Miles Lubin and Theodore Papamarkou, I completed a major overhaul of [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl), a Julia package for **calculating derivatives, gradients, Jacobians, Hessians, and higher-order derivatives** of native Julia functions (or any callable Julia type, really).\n+\n+By the end of this post, you'll hopefully know a little bit about how ForwardDiff.jl works, why it's useful, and why Julia is uniquely well-suited for AD compared to other languages.\n+\n+## Seeing ForwardDiff.jl In Action\n+\n+Before we get down to the nitty-gritty details, it might be helpful to see a simple example that illustrates various methods from ForwardDiff.jl's API.\n+\n+The below is a somewhat contrived example, but works well enough as an introduction to the package. First, we define a target function we'd like to differentiate, then use ForwardDiff.jl to calculate some derivatives of the function at a given input:\n+\n+```julia\n+julia> using ForwardDiff\n+\n+julia> f(x::Vector) = sum(sin, x) + prod(tan, x) * sum(sqrt, x);\n+\n+julia> x = rand(5)\n+5-element Array{Float64,1}:\n+ 0.986403\n+ 0.140913\n+ 0.294963\n+ 0.837125\n+ 0.650451\n+\n+julia> g = ForwardDiff.gradient(f); # g = ∇f\n+\n+julia> g(x)\n+5-element Array{Float64,1}:\n+ 1.01358\n+ 2.50014\n+ 1.72574\n+ 1.10139\n+ 1.2445\n+\n+julia> j = ForwardDiff.jacobian(g); # j = J(∇f)\n+\n+julia> j(x)\n+5x5 Array{Float64,2}:\n+ 0.585111  3.48083  1.7706    0.994057  1.03257\n+ 3.48083   1.06079  5.79299   3.25245   3.37871\n+ 1.7706    5.79299  0.423981  1.65416   1.71818\n+ 0.994057  3.25245  1.65416   0.251396  0.964566\n+ 1.03257   3.37871  1.71818   0.964566  0.140689\n+\n+julia> ForwardDiff.hessian(f, x) # H(f)(x) == J(∇f)(x), as expected \n+5x5 Array{Float64,2}:\n+ 0.585111  3.48083  1.7706    0.994057  1.03257\n+ 3.48083   1.06079  5.79299   3.25245   3.37871\n+ 1.7706    5.79299  0.423981  1.65416   1.71818\n+ 0.994057  3.25245  1.65416   0.251396  0.964566\n+ 1.03257   3.37871  1.71818   0.964566  0.140689\n+```\n+\n+Tada! Okay, that's not *too* exciting - I could've just done the same thing with Calculus.jl! What makes ForwardDiff.jl different? Read on to find out...\n+\n+## How ForwardDiff.jl Works\n+\n+### What is Automatic Differentiation?\n+\n+In broad terms, [automatic differentiation](https://en.wikipedia.org/wiki/Automatic_differentiation) describes a class of algorithms for **automatically taking exact derivatives of user-provided functions**. In addition to producing more accurate results, AD methods are also often **faster than other common differentiation methods** (such as [finite differencing](https://en.wikipedia.org/wiki/Numerical_differentiation)).\n+\n+There are two main flavors of AD: **forward mode**, and **reverse mode**. As you might've guessed, this post only discusses forward mode, which is the kind of AD implemented by ForwardDiff.jl.\n+\n+### `ForwardDiffNumber` Types\n+\n+ForwardDiff.jl implements several new number types, all of which are subtypes of `ForwardDiffNumber{N,T,C} <: Number`.\n+\n+Elementary numerical functions on these types are then overloaded to evaluate both **the original function and its derivative(s)**, returning the results in the form of a new `ForwardDiffNumber`. Thus, we can pass these number types into a general function $f$ (which is assumed to be composed of the overloaded elementary functions), and **the derivative information is naturally propogated** at each step of the calculation by way of the chain rule.","path":"blog/_posts/2015-09-27-auto-diff-in-julia.md","position":null,"original_position":73,"commit_id":"eba36dee09e05900d005b8e5fedd99ba6dd09776","original_commit_id":"454573d74b9cf5d88346ecd131ec5c9e815651d9","user":{"login":"jrevels","id":3277443,"avatar_url":"https://avatars.githubusercontent.com/u/3277443?v=3","gravatar_id":"","url":"https://api.github.com/users/jrevels","html_url":"https://github.com/jrevels","followers_url":"https://api.github.com/users/jrevels/followers","following_url":"https://api.github.com/users/jrevels/following{/other_user}","gists_url":"https://api.github.com/users/jrevels/gists{/gist_id}","starred_url":"https://api.github.com/users/jrevels/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jrevels/subscriptions","organizations_url":"https://api.github.com/users/jrevels/orgs","repos_url":"https://api.github.com/users/jrevels/repos","events_url":"https://api.github.com/users/jrevels/events{/privacy}","received_events_url":"https://api.github.com/users/jrevels/received_events","type":"User","site_admin":false},"body":"fixed, thanks for catching that!","created_at":"2015-09-28T23:33:26Z","updated_at":"2015-09-28T23:33:26Z","html_url":"https://github.com/JuliaLang/julialang.github.com/pull/270#discussion_r40619178","pull_request_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270","_links":{"self":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/comments/40619178"},"html":{"href":"https://github.com/JuliaLang/julialang.github.com/pull/270#discussion_r40619178"},"pull_request":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270"}}},"pull_request":{"url":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270","id":46106686,"html_url":"https://github.com/JuliaLang/julialang.github.com/pull/270","diff_url":"https://github.com/JuliaLang/julialang.github.com/pull/270.diff","patch_url":"https://github.com/JuliaLang/julialang.github.com/pull/270.patch","issue_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/issues/270","number":270,"state":"open","locked":false,"title":"JSoC blog post about ForwardDiff.jl","user":{"login":"jrevels","id":3277443,"avatar_url":"https://avatars.githubusercontent.com/u/3277443?v=3","gravatar_id":"","url":"https://api.github.com/users/jrevels","html_url":"https://github.com/jrevels","followers_url":"https://api.github.com/users/jrevels/followers","following_url":"https://api.github.com/users/jrevels/following{/other_user}","gists_url":"https://api.github.com/users/jrevels/gists{/gist_id}","starred_url":"https://api.github.com/users/jrevels/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jrevels/subscriptions","organizations_url":"https://api.github.com/users/jrevels/orgs","repos_url":"https://api.github.com/users/jrevels/repos","events_url":"https://api.github.com/users/jrevels/events{/privacy}","received_events_url":"https://api.github.com/users/jrevels/received_events","type":"User","site_admin":false},"body":"@mlubin @ViralBShah This PR adds my JSoC blog post about ForwardDiff.jl. Let me know if the formatting (or anything else) requires updating.","created_at":"2015-09-27T19:13:01Z","updated_at":"2015-09-28T23:33:26Z","closed_at":null,"merged_at":null,"merge_commit_sha":"0d6985e0b799764f5b5fc737553ce2d20fb33e51","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270/commits","review_comments_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270/comments","review_comment_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/comments{/number}","comments_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/issues/270/comments","statuses_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/statuses/eba36dee09e05900d005b8e5fedd99ba6dd09776","head":{"label":"jrevels:master","ref":"master","sha":"eba36dee09e05900d005b8e5fedd99ba6dd09776","user":{"login":"jrevels","id":3277443,"avatar_url":"https://avatars.githubusercontent.com/u/3277443?v=3","gravatar_id":"","url":"https://api.github.com/users/jrevels","html_url":"https://github.com/jrevels","followers_url":"https://api.github.com/users/jrevels/followers","following_url":"https://api.github.com/users/jrevels/following{/other_user}","gists_url":"https://api.github.com/users/jrevels/gists{/gist_id}","starred_url":"https://api.github.com/users/jrevels/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jrevels/subscriptions","organizations_url":"https://api.github.com/users/jrevels/orgs","repos_url":"https://api.github.com/users/jrevels/repos","events_url":"https://api.github.com/users/jrevels/events{/privacy}","received_events_url":"https://api.github.com/users/jrevels/received_events","type":"User","site_admin":false},"repo":{"id":43260486,"name":"julialang.github.com","full_name":"jrevels/julialang.github.com","owner":{"login":"jrevels","id":3277443,"avatar_url":"https://avatars.githubusercontent.com/u/3277443?v=3","gravatar_id":"","url":"https://api.github.com/users/jrevels","html_url":"https://github.com/jrevels","followers_url":"https://api.github.com/users/jrevels/followers","following_url":"https://api.github.com/users/jrevels/following{/other_user}","gists_url":"https://api.github.com/users/jrevels/gists{/gist_id}","starred_url":"https://api.github.com/users/jrevels/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jrevels/subscriptions","organizations_url":"https://api.github.com/users/jrevels/orgs","repos_url":"https://api.github.com/users/jrevels/repos","events_url":"https://api.github.com/users/jrevels/events{/privacy}","received_events_url":"https://api.github.com/users/jrevels/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jrevels/julialang.github.com","description":"Julia Project web site.","fork":true,"url":"https://api.github.com/repos/jrevels/julialang.github.com","forks_url":"https://api.github.com/repos/jrevels/julialang.github.com/forks","keys_url":"https://api.github.com/repos/jrevels/julialang.github.com/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jrevels/julialang.github.com/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jrevels/julialang.github.com/teams","hooks_url":"https://api.github.com/repos/jrevels/julialang.github.com/hooks","issue_events_url":"https://api.github.com/repos/jrevels/julialang.github.com/issues/events{/number}","events_url":"https://api.github.com/repos/jrevels/julialang.github.com/events","assignees_url":"https://api.github.com/repos/jrevels/julialang.github.com/assignees{/user}","branches_url":"https://api.github.com/repos/jrevels/julialang.github.com/branches{/branch}","tags_url":"https://api.github.com/repos/jrevels/julialang.github.com/tags","blobs_url":"https://api.github.com/repos/jrevels/julialang.github.com/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jrevels/julialang.github.com/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jrevels/julialang.github.com/git/refs{/sha}","trees_url":"https://api.github.com/repos/jrevels/julialang.github.com/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jrevels/julialang.github.com/statuses/{sha}","languages_url":"https://api.github.com/repos/jrevels/julialang.github.com/languages","stargazers_url":"https://api.github.com/repos/jrevels/julialang.github.com/stargazers","contributors_url":"https://api.github.com/repos/jrevels/julialang.github.com/contributors","subscribers_url":"https://api.github.com/repos/jrevels/julialang.github.com/subscribers","subscription_url":"https://api.github.com/repos/jrevels/julialang.github.com/subscription","commits_url":"https://api.github.com/repos/jrevels/julialang.github.com/commits{/sha}","git_commits_url":"https://api.github.com/repos/jrevels/julialang.github.com/git/commits{/sha}","comments_url":"https://api.github.com/repos/jrevels/julialang.github.com/comments{/number}","issue_comment_url":"https://api.github.com/repos/jrevels/julialang.github.com/issues/comments{/number}","contents_url":"https://api.github.com/repos/jrevels/julialang.github.com/contents/{+path}","compare_url":"https://api.github.com/repos/jrevels/julialang.github.com/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jrevels/julialang.github.com/merges","archive_url":"https://api.github.com/repos/jrevels/julialang.github.com/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jrevels/julialang.github.com/downloads","issues_url":"https://api.github.com/repos/jrevels/julialang.github.com/issues{/number}","pulls_url":"https://api.github.com/repos/jrevels/julialang.github.com/pulls{/number}","milestones_url":"https://api.github.com/repos/jrevels/julialang.github.com/milestones{/number}","notifications_url":"https://api.github.com/repos/jrevels/julialang.github.com/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jrevels/julialang.github.com/labels{/name}","releases_url":"https://api.github.com/repos/jrevels/julialang.github.com/releases{/id}","created_at":"2015-09-27T19:06:30Z","updated_at":"2015-09-27T19:06:31Z","pushed_at":"2015-09-28T23:33:05Z","git_url":"git://github.com/jrevels/julialang.github.com.git","ssh_url":"git@github.com:jrevels/julialang.github.com.git","clone_url":"https://github.com/jrevels/julialang.github.com.git","svn_url":"https://github.com/jrevels/julialang.github.com","homepage":"http://julialang.org/","size":9227,"stargazers_count":0,"watchers_count":0,"language":"TeX","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"JuliaLang:master","ref":"master","sha":"ab259c25a79c3389c06a00b974a9c8b712fcff8c","user":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars.githubusercontent.com/u/743164?v=3","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"repo":{"id":2885268,"name":"julialang.github.com","full_name":"JuliaLang/julialang.github.com","owner":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars.githubusercontent.com/u/743164?v=3","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/JuliaLang/julialang.github.com","description":"Julia Project web site.","fork":false,"url":"https://api.github.com/repos/JuliaLang/julialang.github.com","forks_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/forks","keys_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/keys{/key_id}","collaborators_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/teams","hooks_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/hooks","issue_events_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/issues/events{/number}","events_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/events","assignees_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/assignees{/user}","branches_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/branches{/branch}","tags_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/tags","blobs_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/git/refs{/sha}","trees_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/git/trees{/sha}","statuses_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/statuses/{sha}","languages_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/languages","stargazers_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/stargazers","contributors_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/contributors","subscribers_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/subscribers","subscription_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/subscription","commits_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/commits{/sha}","git_commits_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/git/commits{/sha}","comments_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/comments{/number}","issue_comment_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/issues/comments{/number}","contents_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/contents/{+path}","compare_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/compare/{base}...{head}","merges_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/merges","archive_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/downloads","issues_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/issues{/number}","pulls_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls{/number}","milestones_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/milestones{/number}","notifications_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/labels{/name}","releases_url":"https://api.github.com/repos/JuliaLang/julialang.github.com/releases{/id}","created_at":"2011-11-30T18:22:15Z","updated_at":"2015-06-10T11:18:58Z","pushed_at":"2015-09-28T23:33:05Z","git_url":"git://github.com/JuliaLang/julialang.github.com.git","ssh_url":"git@github.com:JuliaLang/julialang.github.com.git","clone_url":"https://github.com/JuliaLang/julialang.github.com.git","svn_url":"https://github.com/JuliaLang/julialang.github.com","homepage":"http://julialang.org/","size":9227,"stargazers_count":59,"watchers_count":59,"language":"TeX","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":91,"mirror_url":null,"open_issues_count":14,"forks":91,"open_issues":14,"watchers":59,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270"},"html":{"href":"https://github.com/JuliaLang/julialang.github.com/pull/270"},"issue":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/issues/270"},"comments":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/issues/270/comments"},"review_comments":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270/comments"},"review_comment":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/pulls/270/commits"},"statuses":{"href":"https://api.github.com/repos/JuliaLang/julialang.github.com/statuses/eba36dee09e05900d005b8e5fedd99ba6dd09776"}}}}