{"action":"created","issue":{"url":"https://api.github.com/repos/halide/Halide/issues/664","labels_url":"https://api.github.com/repos/halide/Halide/issues/664/labels{/name}","comments_url":"https://api.github.com/repos/halide/Halide/issues/664/comments","events_url":"https://api.github.com/repos/halide/Halide/issues/664/events","html_url":"https://github.com/halide/Halide/issues/664","id":57645110,"number":664,"title":"AOT-generated code should include Argument information","user":{"login":"steven-johnson","id":328955,"avatar_url":"https://avatars.githubusercontent.com/u/328955?v=3","gravatar_id":"","url":"https://api.github.com/users/steven-johnson","html_url":"https://github.com/steven-johnson","followers_url":"https://api.github.com/users/steven-johnson/followers","following_url":"https://api.github.com/users/steven-johnson/following{/other_user}","gists_url":"https://api.github.com/users/steven-johnson/gists{/gist_id}","starred_url":"https://api.github.com/users/steven-johnson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/steven-johnson/subscriptions","organizations_url":"https://api.github.com/users/steven-johnson/orgs","repos_url":"https://api.github.com/users/steven-johnson/repos","events_url":"https://api.github.com/users/steven-johnson/events{/privacy}","received_events_url":"https://api.github.com/users/steven-johnson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":17,"created_at":"2015-02-13T19:48:15Z","updated_at":"2015-02-17T21:18:07Z","closed_at":null,"body":"It would be useful if AOT-generated Halide filters included a way to introspect the expected input and output arguments.\r\n\r\nStrawman proposal: Add simple array-of-struct data structures with names that match the filter. e.g.:\r\n\r\n    struct HalideArgumentDescriptor {\r\n      // Halide::Type isn't really available at runtime when running AOT;\r\n      // we'll just sorta replicate it here.\r\n      enum Type { kInt = 0, kUint = 1, kFloat = 2, kHandle = 3 };\r\n      const char* const name;\r\n      const bool is_buffer;\r\n      const Type type_code;\r\n      const uint8_t type_bits;\r\n      const uint8_t buffer_dimensions;\r\n      const double scalar_default;\r\n      const bool has_scalar_minmax;\r\n      const double scalar_min, scalar_max;\r\n    };\r\n    struct HalideArguments {\r\n      const int num_inputs;\r\n      const HalideArgumentDescriptor* inputs;\r\n      const int num_outputs;\r\n      const HalideArgumentDescriptor* outputs;\r\n    };\r\n\r\n    // If the filter we generate is like so:\r\n    // extern \"C\" int my_awesome_filter(buffer_t* in1, buffer_t* in2, int16_t i, float f, buffer_t* out);\r\n\r\n    // We'd generate something like:\r\n    extern \"C\" HalideArguments my_awesome_filter_halide_arguments = {\r\n      { \r\n        /* inputs */ \r\n        4, \r\n        {\r\n          { \"in1\", true, kUInt, 8, 3, 0.0, false, 0.0, 0.0 },\r\n          { \"in2\", true, kUInt, 8, 3, 0.0, false, 0.0, 0.0 },\r\n          { \"i\", false, kInt, 16, 0, 0.0, false, 0.0, 0.0 },\r\n          { \"f\", false, kFloat, 32, 0, 0.0, false, 0.0, 0.0 },\r\n        }\r\n        /* outputs */\r\n        1,\r\n        {\r\n          {\"out\", true, kUInt, 8, 3, 0.0, false, 0.0, 0.0 },\r\n        }\r\n        }\r\n      }\r\n    };\r\n\r\nSince we'd just be adding a new extern name that no one is likely to ever be looking for, existing code should be unaffected. It's a POD of modest size so addition to code size should be unimportant.\r\n\r\nSpecific layout and contents of the descriptor-struct open for discussion, of course. The one given above is similar to one I'm using in a private branch that suits my purposes.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/halide/Halide/issues/comments/74755903","html_url":"https://github.com/halide/Halide/issues/664#issuecomment-74755903","issue_url":"https://api.github.com/repos/halide/Halide/issues/664","id":74755903,"user":{"login":"zvookin","id":1501716,"avatar_url":"https://avatars.githubusercontent.com/u/1501716?v=3","gravatar_id":"","url":"https://api.github.com/users/zvookin","html_url":"https://github.com/zvookin","followers_url":"https://api.github.com/users/zvookin/followers","following_url":"https://api.github.com/users/zvookin/following{/other_user}","gists_url":"https://api.github.com/users/zvookin/gists{/gist_id}","starred_url":"https://api.github.com/users/zvookin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zvookin/subscriptions","organizations_url":"https://api.github.com/users/zvookin/orgs","repos_url":"https://api.github.com/users/zvookin/repos","events_url":"https://api.github.com/users/zvookin/events{/privacy}","received_events_url":"https://api.github.com/users/zvookin/received_events","type":"User","site_admin":false},"created_at":"2015-02-17T21:18:07Z","updated_at":"2015-02-17T21:18:07Z","body":"Andrew, let me clarify the point about dead stripping:\n\nIf one wants the metadata to be there for dlsym on a plug-in shared object,\nthe symbol for the metadata structure cannot be dead stripped. The\nfollowing are ways to avoid dead stripping:\n\n1) The name of the metadata structure must be mentioned in some sort of\nexports flag or file to the linker.\n2) The metadata structure must be accessed by the filter code itself.\n3) The metadata structure must be marked with an export attribute to\ndisable dead stripping.\n\nThe first choice *requires* some extra build logic mentioning the filters\nname, the second two make if very difficult to get rid of the information.\nAnd having the metadata is not just a code size thing as it carries extra\ninformation about the filter that people developing non-open-source code\nmay not wish to place in their binary.\n\nA proposed solution is to put the metadata in a separate .o (or C file\nperhaps) and mark it in such a way that it will not get dead stripped. The\nend user controls whether to include the info or not by adding an extra\nfile to their build.\n\nOn Tue, Feb 17, 2015 at 12:49 PM, Andrew Adams <notifications@github.com>\nwrote:\n\n> I disagree with point two. The thinking behind our runtime is that a small\n> amount of extra data in the .o is better than a more complex build setup.\n> Most of the runtime is unused by most apps (tiff writing, memoization,\n> tracing...), so we're already expecting people to either not care about\n> object size or to strip it.\n>\n> Also, the a library author may not have cared about producing this\n> metadata, but the person trying to use their compiled code dynamically will\n> want it. So on by default seems better than off by default. If someone\n> really objects to having it in their .o, they can strip the symbol.\n>\n> user_context was messy because we didn't want to make it mandatory. I think\n> this should be mandatory.\n>\n> On Tue, Feb 17, 2015 at 12:31 PM, Steven Johnson <notifications@github.com\n> >\n> wrote:\n>\n> > A few folks discussed this more at length locally; here were what I think\n> > the takeaways were:\n> >\n> > -\n> >\n> > It's probably desirable that code could dlsym some symbol to get the\n> > metadata for a loaded file; this argues *against* using std::map as\n> > the intrinsic storage format for the metadata (since there's no guarantee\n> > that std::map will have compatible implementations across the boundary).\n> > Expressing it as a C struct (with careful attention to packing,\n> alignment,\n> > etc) is probably the most portable approach. (Providing utilities to\n> > convert between this and a more C++-friendly std::map structure would be\n> > desirable.)\n> > -\n> >\n> > It's desirable to *not* have the metadata included in a generated .o\n> > by default, since many applications won't want it, and would prefer to\n> > avoid having to rely on dead-stripping to remove it. It's also desirable\n> to\n> > *not* have generation controlled by another build or Target flag\n> > (since that turned out to be a big headache for user-context). Instead,\n> > let's generate the metadata as a separate C source file (either a .h or\n> .c,\n> > details TBD) produced by Generator::emit_filter(); build systems that\n> want\n> > to use the metadata just need to include this new file in the build\n> > manifest, and those that don't, don't.\n> > -\n> >\n> > Don't attempt (yet) to provide overall policy for a registry-based\n> > filter setup (as I mentioned as a thought in an earlier comment); we may\n> > find that desirable at a later date, but that mechanism should be\n> buildable\n> > (and built) on top of this one.\n> >\n> > —\n> > Reply to this email directly or view it on GitHub\n> > <https://github.com/halide/Halide/issues/664#issuecomment-74746956>.\n>\n> >\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/halide/Halide/issues/664#issuecomment-74750228>.\n>\n"}}