{"action":"created","issue":{"url":"https://api.github.com/repos/Microsoft/ApplicationInsights-dotnet/issues/47","labels_url":"https://api.github.com/repos/Microsoft/ApplicationInsights-dotnet/issues/47/labels{/name}","comments_url":"https://api.github.com/repos/Microsoft/ApplicationInsights-dotnet/issues/47/comments","events_url":"https://api.github.com/repos/Microsoft/ApplicationInsights-dotnet/issues/47/events","html_url":"https://github.com/Microsoft/ApplicationInsights-dotnet/issues/47","id":108713316,"number":47,"title":"Support end-to-end cross-tier correlation","user":{"login":"SergeyKanzhelev","id":9950081,"avatar_url":"https://avatars.githubusercontent.com/u/9950081?v=3","gravatar_id":"","url":"https://api.github.com/users/SergeyKanzhelev","html_url":"https://github.com/SergeyKanzhelev","followers_url":"https://api.github.com/users/SergeyKanzhelev/followers","following_url":"https://api.github.com/users/SergeyKanzhelev/following{/other_user}","gists_url":"https://api.github.com/users/SergeyKanzhelev/gists{/gist_id}","starred_url":"https://api.github.com/users/SergeyKanzhelev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SergeyKanzhelev/subscriptions","organizations_url":"https://api.github.com/users/SergeyKanzhelev/orgs","repos_url":"https://api.github.com/users/SergeyKanzhelev/repos","events_url":"https://api.github.com/users/SergeyKanzhelev/events{/privacy}","received_events_url":"https://api.github.com/users/SergeyKanzhelev/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-09-28T17:59:18Z","updated_at":"2015-11-03T18:43:15Z","closed_at":null,"body":"cc: @abaranch, @briancr-ms, @BogdanBe, @vitalyf007 \r\n\r\nStable version of Application Insights SDK doesn't support hierarchical correlation today. Correlation is implemented using two fields: ```RequestTelemetry.ID``` and ```ITelemetry.Context.Operaiton.Id```. When these fields match - we show them as related in UI.\r\n\r\nCross-tier and hierarchical correlation requires more fields. Typical scenario for this type of correlation that we want to support is for every page view event (collected by JavaScript SDK) we show as \"children\" all ajax calls run on the page. For every ajax call we show corresponding server request collected by .NET server SDK. Every server request may have children operations like \"business logic execution\" or \"rendering\" which in turn may have calls to external dependencies as children activities. So we can troubleshoot page loading sloweness and show the root cause when sloweness was caused by database issue.\r\n\r\nWe propose the following terminology:\r\n\r\n- *Transaction Name* - name of the first/topmost/root operation that initiated entire chain of execution.\r\n- *Transaction ID* - correspondingly unique ID of operation described above.\r\n- *Parent Operation ID\" - unique id of the operation that was active at the moment when telemetry was reported\r\n- *Operation Name* - name of the telemetry item if it is request telemetry or dependency telemetry\r\n- *Operation ID* - unique id of the telemetry item if it is request telemetry or dependency telemetry\r\n\r\n***Note:*** We propose to use term *Transaction* as oppose to *Root Operation* as we believe that transaction is easier to understand and explain.\r\n\r\nLabel for corresponding telemetry items on request details blade will be updated to this:\r\n\"All traces/exceptions/dependencies for this <strike>request</strike> transaction\".\r\n\r\n\r\n\r\nWe can use the following http header names to propagate correlation information:\r\n\r\n- ```x-ms-request-id``` for parent operation ID\r\n- ```x-ms-transaction-id``` for transaction ID\r\n- ```x-ms-transaction-name``` for transaction name\r\n\r\n\r\n\r\nGoing forward In order to support proper end-to-end cross-tier events correlation we want to start using concept called \"correlation vector\". The idea behind correlation vector is very similar to the new EventSource correlation and causation feature in FW 4.6. Here is [Vance Morrison blog post](http://blogs.msdn.com/b/vancem/archive/2015/09/14/exploring-eventsource-activity-correlation-and-causation-features.aspx) explaining this feature.\r\n\r\nCorrelation vector is a string consist of base64-encoded GUID and a dot-delimited vector of integer values: HyCFaiQoBkyEp0L3.1.2 \r\n\r\nIt support two method: ```Increment``` that will increment the last integer in a vector and ```Extend``` that will add ```.0``` to the end of the vector.\r\n\r\nWhenever new layer receives correlation vector it extends it and use it as the current operation identity. Every time new outgoing operation is started - correlation vector of current operation is incremented on one side and extended on other side of operation.\r\n\r\nThe main advantages of correlation vector comparing to traditional ParentOperaitonId, TransactionId approach are:\r\n- Ability to build execution tree even if some events are missing\r\n- Ordering of operation that doesn't rely on event time\r\n\r\nSo the new data field ```CorrelationVector``` will be included into ```Operation``` context. \r\n\r\nHovewer, there are still services using ParentOperationId, TransactionId concept for end-to-end tracking of operations. There are services receiving headers like ```x-ms-request-id``` representing operation ID that should be used as a parent operation of the service's new operation. Thus the proposal is to support fields ```ParentOperationId``` and ```TransactionId``` for these services. \r\n\r\nThe idea is that SDK developer will use ```ParentOperationId``` field for the requests called with this header. However if no operation id header was configured - CorrelationVector-based correlation will be preferred."},"comment":{"url":"https://api.github.com/repos/Microsoft/ApplicationInsights-dotnet/issues/comments/153449984","html_url":"https://github.com/Microsoft/ApplicationInsights-dotnet/issues/47#issuecomment-153449984","issue_url":"https://api.github.com/repos/Microsoft/ApplicationInsights-dotnet/issues/47","id":153449984,"user":{"login":"SergeyKanzhelev","id":9950081,"avatar_url":"https://avatars.githubusercontent.com/u/9950081?v=3","gravatar_id":"","url":"https://api.github.com/users/SergeyKanzhelev","html_url":"https://github.com/SergeyKanzhelev","followers_url":"https://api.github.com/users/SergeyKanzhelev/followers","following_url":"https://api.github.com/users/SergeyKanzhelev/following{/other_user}","gists_url":"https://api.github.com/users/SergeyKanzhelev/gists{/gist_id}","starred_url":"https://api.github.com/users/SergeyKanzhelev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SergeyKanzhelev/subscriptions","organizations_url":"https://api.github.com/users/SergeyKanzhelev/orgs","repos_url":"https://api.github.com/users/SergeyKanzhelev/repos","events_url":"https://api.github.com/users/SergeyKanzhelev/events{/privacy}","received_events_url":"https://api.github.com/users/SergeyKanzhelev/received_events","type":"User","site_admin":false},"created_at":"2015-11-03T18:43:15Z","updated_at":"2015-11-03T18:43:15Z","body":"No, we will have all three identities:\r\n\r\n1.Own (nested operation) id:  ```Request.ID```  &  ```Dependency.ID ```\r\n2.Parent operation id:  ```ITelemetry.Context.Operation.ParentID```\r\n3.Operation (a.k.a. root operation) id:  ```ITelemetry.Context.Operation.ID```\r\n"}}