{"action":"created","issue":{"url":"https://api.github.com/repos/MatthieuDartiailh/pyclibrary/issues/12","labels_url":"https://api.github.com/repos/MatthieuDartiailh/pyclibrary/issues/12/labels{/name}","comments_url":"https://api.github.com/repos/MatthieuDartiailh/pyclibrary/issues/12/comments","events_url":"https://api.github.com/repos/MatthieuDartiailh/pyclibrary/issues/12/events","html_url":"https://github.com/MatthieuDartiailh/pyclibrary/issues/12","id":87685164,"number":12,"title":"Generalization of backend API","user":{"login":"mrh1997","id":1482077,"avatar_url":"https://avatars.githubusercontent.com/u/1482077?v=3","gravatar_id":"","url":"https://api.github.com/users/mrh1997","html_url":"https://github.com/mrh1997","followers_url":"https://api.github.com/users/mrh1997/followers","following_url":"https://api.github.com/users/mrh1997/following{/other_user}","gists_url":"https://api.github.com/users/mrh1997/gists{/gist_id}","starred_url":"https://api.github.com/users/mrh1997/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrh1997/subscriptions","organizations_url":"https://api.github.com/users/mrh1997/orgs","repos_url":"https://api.github.com/users/mrh1997/repos","events_url":"https://api.github.com/users/mrh1997/events{/privacy}","received_events_url":"https://api.github.com/users/mrh1997/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-06-12T10:33:45Z","updated_at":"2015-06-12T15:46:53Z","closed_at":null,"body":"While reasoning about my RPC library's design which shall be based on the parser of pyclibrary I came to the conclusion that your (really nice) concept of pluggable backends would be a perfect match for my library.\r\n\r\nBut according to my (yet) limited knownledge of the backend it has a drawback: it i too specialized for the current purpose. In fact it does **two** things:\r\n* (dynamicially) generate python ctype objects from a C header file\r\n* provide (really useful) extensions to ctypes (primarly the CallResult class)\r\n\r\nIf these features would be modularized and the python ctype object generation would be generalized to a static interface file generator, the range of applications of the library would be drasticially extended:\r\n* It would be possible to write a backend for **any language** and for **any ffi-library**!!! \r\n* The extensions (CallResult) could be swapped out into a separate module. This means you could use it even if you do not want to use CParser/CLibrary (i.e. if you do not have a C header file, but want to benefit from this Module either)\r\n* projects which need to access a C library could do this easily without the need to add pyclibrary as dependency (pyclibrary would transform the header file once to a python module, which is added the project)\r\n\r\nMy idea of the static interface generator is basicially a template engine which gets the AST from the CParser and outputs a interface definition for a specific language/ffi-library/parametrization. If one still needs dynamic interface object generation (i.e. if the according C header file is not static; a use case which should barely occur), one could still generated the interface definition module as in-memory string object run ```exec()``` on this string.\r\n\r\nTo come to an example: The current backend would be replaced by a template, that converts the following header file:\r\n```C\r\nstruct sample_struct_t {\r\n   int * a;\r\n} ;\r\nvoid sample_func(struct sample_struct_t * x);\r\n```\r\n\r\nto the follong python module:\r\n\r\n```python\r\nimport ctypes\r\n\r\nlib = ctypes.CDLL('...')\r\n\r\nclass sample_struct_t(ctypes.Structure):\r\n    _fields_ = [(\"a\", ctypes.POINTER(ctypes.c_int))]\r\n\r\nlib.sample_func.restype = ctypes.c_void\r\nlib.sample_func.argtypes = [POINTER(sample_struct_t)]\r\n```\r\n(This example generated a python module without using CallResult. But of course it would also be possible to integrate CallResult into the generated module; would depend on the parametrization of the code generator)\r\n\r\n@MatthieuDartiailh: What do you think about it? Regarding all the work to be done: as I need to do it anyway (for my RPC library, which has to compile a C-wrapper from C Headerf files) I would be glad to do it as part of pyclibary instead of my own project."},"comment":{"url":"https://api.github.com/repos/MatthieuDartiailh/pyclibrary/issues/comments/111532515","html_url":"https://github.com/MatthieuDartiailh/pyclibrary/issues/12#issuecomment-111532515","issue_url":"https://api.github.com/repos/MatthieuDartiailh/pyclibrary/issues/12","id":111532515,"user":{"login":"MatthieuDartiailh","id":4481740,"avatar_url":"https://avatars.githubusercontent.com/u/4481740?v=3","gravatar_id":"","url":"https://api.github.com/users/MatthieuDartiailh","html_url":"https://github.com/MatthieuDartiailh","followers_url":"https://api.github.com/users/MatthieuDartiailh/followers","following_url":"https://api.github.com/users/MatthieuDartiailh/following{/other_user}","gists_url":"https://api.github.com/users/MatthieuDartiailh/gists{/gist_id}","starred_url":"https://api.github.com/users/MatthieuDartiailh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MatthieuDartiailh/subscriptions","organizations_url":"https://api.github.com/users/MatthieuDartiailh/orgs","repos_url":"https://api.github.com/users/MatthieuDartiailh/repos","events_url":"https://api.github.com/users/MatthieuDartiailh/events{/privacy}","received_events_url":"https://api.github.com/users/MatthieuDartiailh/received_events","type":"User","site_admin":false},"created_at":"2015-06-12T15:46:53Z","updated_at":"2015-06-12T15:46:53Z","body":"Those ideas are very interesting I think. And I would be very happy to see your work become part of pyclibrary.\r\n\r\nCurrently a backend is meant to back a CLibrary by exposing the specificity of the chosen ffi. It is specific to C but not to ctypes. It makes user life easier by doing several things on top of the ffi : \r\n- it make sure that a single binder per libary is in activity at a time (see metaclass)\r\n- it generate the appropriate classes for any custom types defined in the header.\r\n- it abstracts the way to create pointers and arrays.\r\n- it wrap the functions to CFunctions which add unspecified pointers and always return CallResult (I really like the on the fly creation of pointers because it makes the library much simpler to use).\r\n- it gives access to all the macros and enumeration (which are a pain to copy when creating a ctypes wrapper).\r\n\r\nI think it would make sense to move to a kind of binding generations like the one you suggest. We could actually propose : \r\n- pure ctypes bindings\r\n- smart ctypes bindings (using clibrary and potentially packing it in the module).\r\n- others such cffi bindings, cython or even C bindings (is that what you have in mind for your project ?) in both pure and smart versions (THIS WOULD BE AWESOME)\r\n\r\nCreating an in memory wrapper (or calling a cached one) should remain easy (which would be equivalent to what we are doing now). In some cases you cannot version control the wrapper because it would expose proprietary headers .... (I have cases like that)\r\n\r\nFor the roadmap, I would say we first focus on the parser and the preprocessor (you mentioned somewhere extracting it from the parser which would make sense). Once this is done we can revisit the generation of the bindings."}}