{"action":"opened","issue":{"url":"https://api.github.com/repos/AutoMapper/AutoMapper/issues/880","labels_url":"https://api.github.com/repos/AutoMapper/AutoMapper/issues/880/labels{/name}","comments_url":"https://api.github.com/repos/AutoMapper/AutoMapper/issues/880/comments","events_url":"https://api.github.com/repos/AutoMapper/AutoMapper/issues/880/events","html_url":"https://github.com/AutoMapper/AutoMapper/issues/880","id":105875467,"number":880,"title":"Materialization","user":{"login":"jasonholloway","id":6573293,"avatar_url":"https://avatars.githubusercontent.com/u/6573293?v=3","gravatar_id":"","url":"https://api.github.com/users/jasonholloway","html_url":"https://github.com/jasonholloway","followers_url":"https://api.github.com/users/jasonholloway/followers","following_url":"https://api.github.com/users/jasonholloway/following{/other_user}","gists_url":"https://api.github.com/users/jasonholloway/gists{/gist_id}","starred_url":"https://api.github.com/users/jasonholloway/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jasonholloway/subscriptions","organizations_url":"https://api.github.com/users/jasonholloway/orgs","repos_url":"https://api.github.com/users/jasonholloway/repos","events_url":"https://api.github.com/users/jasonholloway/events{/privacy}","received_events_url":"https://api.github.com/users/jasonholloway/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-09-10T18:58:25Z","updated_at":"2015-09-10T18:58:25Z","closed_at":null,"body":"Hey, I've been fiddling away at a prototype of something which (hopefully) I can interest you in.\r\n\r\n------------------------------\r\n\r\n#### The Background\r\nProjectUsing and the QueryableExtensions are attractive. They promise the pleasing satisfaction of minimizing EF's data throughput, while also eliminating the various bits of plumbing code needed to push the same data through AutoMapper. They bring closer to reality the happy vision of a slimmed-down, genericized controller that prepares models according to declarative rules.\r\n\r\n#### The Problem\r\nWould-be fans of ProjectUsing are immediately rebuffed by the starkness of its expectations. Or rather, EF's intolerant query-processing regime bangs its hammer and spits out exceptions at all but the most anodyne of projections, thereby scaring the inadvertent user.\r\n\r\nProjections are forbidden from using parameterized constructors; from accessing members unknown to EF's metadata; from referring to constants of a non-primitive type; and from invoking anything but a very small subset of recognised methods. Straight property-mapping - the meat and drink of AutoMapper - works fine, as it happens, because EF has no qualms with simple member-binding on construction. But anything funkier falls foul of its proscriptive tastes.\r\n\r\nThis is sensible behaviour, of course, as by forcibly limiting queries to these hygienic standards, EF makes its own job easier to efficiently perform. \r\n\r\nBut in doing so, it ruins AutoMapper's hopes of being an integrated data-fetching and model-preparation layer. It forms a chasm across which only very few preparatory behaviours can hop.\r\n\r\n#### The Solution\r\nIf only there were some way to efficiently straddle this gap, with subtasks of the overall projection distributed to either the server (EF) or the client (AutoMapper), as circumstances permitted...\r\n\r\nIf only there were some mediating agent, fitted with various cooperative strategies, that could tell the server, when faced with forbidden projections, to return tuples of the requisite data instead, with the client finishing the projection locally...\r\n\r\nWell, would you believe it, I've been working on exactly this for the past couple of weeks. See here: https://github.com/jasonholloway/AutoMapper/tree/materialize2 (there are several new projects in the solution)\r\n\r\n**In particular, look at the Materialize.Demo console project, which runs a full projection and prints out details of each stage (the SqlServer service needs to be running)**\r\n\r\n-------------------------------------------------\r\n### From the Outside In\r\nA new extension method is made available on `IQueryable`: `MaterializeAs<TDest>()` which returns the interface `IMaterializable<TDest>` deriving from `IEnumerable<TDest>` and augmented with various extension methods corresponding to the unary methods of `Queryable`, such as `First<TDest>()`, `Last<TDest>()`, etc.\r\n\r\nThis forms the limited public wrapper of what is internally a brand new `IQueryProvider`, represented by the `Reifiable` class, which serves `ReifyQuery` queryables. As an `IQueryProvider` it is however largely unimplemented beyond simple enumeration and unary methods (and would be for the conceivable future) - hence its being hidden away from misuse.\r\n\r\n`Reifiable<TSource,TDest>.Execute()` therefore oversees the library's core operation: serving incoming queries by rewriting the `Expression` of the given source query, executing this via the given source `IQueryProvider`, and then appropriately transforming the returned objects to yield the requested destination types. It does this consistently by assembling an appropriate stack of `IModifier` classes, each of which performs both source-expression-rewriting and fetched-result-transformation in an ideally-balanced manner.\r\n\r\nThis is all a bit difficult to explain so I've drawn a diagram:\r\n![Modifiers diagram](https://raw.githubusercontent.com/jasonholloway/AutoMapper/materialize2/src/Materialize/modifiers.png)\r\n*(note that I've managed to get the mapper modifiers and other modifiers the wrong way round)*\r\n\r\nMost of the modifier stack is formed of `MapperModifer` instances, provided by an cached structure of `IMapStrategy` classes. In aggregate, these core modifiers almost always append a new `Select` projection to the source query, and then transform whatever that projection fetches client-side.  \r\n\r\nThe structure of these modifers is determined by AutoMapper's `TypeMap` dictionary, via the behaviours of the `PropertyMapRule` and `TranslationRule` classes, which themselves create appropriate `IMapStrategy` instances (which in turn create appropriate `IModifier` instances).\r\n\r\nI'll stop there.\r\n\r\n-------------------------------------------------\r\n### Progress\r\nAs things stand, it's fair to say all the core functionality is in place. There are however important parts still to write:\r\n* Collection-mapping\r\n* EF-specific query proscriptions (for thorough implementation, requires testing against metadata, and a hard dependency against EF) \r\n* Custom constructors\r\n* Before- and after- hooks\r\n* Only properties needed for projections should be fetched (if detectable from expression)\r\n\r\nNevertheless, none of these (or others unmentioned) represent significant leaps from the existing functionality.\r\n\r\n--------------------------------------------------\r\n### Finally\r\nAll of which above leads me to the main question: is this something that interests you, as an idea and/or as an implementation? \r\n\r\nI've written it so far as a dependent library, with references only to AutoMapper's existing public API. But throughout I've been secretly nurturing the hope that it might be considered for the main project."}}