{"action":"created","issue":{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/851","labels_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/851/labels{/name}","comments_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/851/comments","events_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/851/events","html_url":"https://github.com/nodemcu/nodemcu-firmware/issues/851","id":121966486,"number":851,"title":"Lua reentrancy, tasks, callbacks and interrupt callbacks.","user":{"login":"TerryE","id":690173,"avatar_url":"https://avatars.githubusercontent.com/u/690173?v=3","gravatar_id":"","url":"https://api.github.com/users/TerryE","html_url":"https://github.com/TerryE","followers_url":"https://api.github.com/users/TerryE/followers","following_url":"https://api.github.com/users/TerryE/following{/other_user}","gists_url":"https://api.github.com/users/TerryE/gists{/gist_id}","starred_url":"https://api.github.com/users/TerryE/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TerryE/subscriptions","organizations_url":"https://api.github.com/users/TerryE/orgs","repos_url":"https://api.github.com/users/TerryE/repos","events_url":"https://api.github.com/users/TerryE/events{/privacy}","received_events_url":"https://api.github.com/users/TerryE/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-12-14T03:07:58Z","updated_at":"2015-12-16T00:36:51Z","closed_at":null,"body":"This issue was highlighted to me by #795 and #802.  We have Lua instabilities manifesting themselves when the ESP is being hammered by GPIO or OW interrupts.  How it manifests itself is with Lua stack corruptions which cause assert trips if lua_assert diagnostics are available and by a messy death if not.\r\n\r\nThere is a separate issue #846, which might be related but I think not.  Because this stuff is terribly documented within the Espressif documentation, I am largely relying on wider experience and retroengineering of the behaviour of the SDK.  But if anyone else can provide better information, then please add to this discussion.\r\n\r\n*   Lua can either be build to be reentrant or not.  In the case of the eLua implementations it is assumed that the RTS is non -rentrant so the `lua_lock()` and `lua_unlock()` are optimised away.  We don't have the option to enable them because the SDK is essentially non-preemptive and any lock collision would simply result in deadlock.\r\n\r\n*    The SDK uses two primary mechanisms for executing applications code: tasks and callbacks.  Though in many ways tasks are simply a queued callback mechanism.\r\n\r\n*    All task and most callback functions are non-preemptive, that is once a function is initiated, it will run to completion and return control to the SDK -- which in turn can then schedule the next runnable task / cb.  The guideline is that no cb should take longer than 10 mSec, otherwise some functions such as net and wifi will start to fail.\r\n\r\n*   `user_init()` starts the main Lua thread and posts a task to run the  `init.lua`.\r\n\r\n*   Any code chunks input through the UART are also posted as tasks.\r\n\r\n*   Unlike standard Lua, the firmware never closes the main thread, but leaves it open when tasks complete, and hence globals, the thread's registry and dangling upvalues remain in scope.\r\n\r\n*   Callbacks (such as a `tmr.alarm()` callback) are executed using standard \"calling Lua from C\" API features and pick up a Lua thread (not necessarily the main thread -- see #846).\r\n\r\n*    Some callbacks seem to be classed as \"interrupt drivers\", and it isn't clear to me if they are handled within the non-pre-emptive queueing system or are true interrupt routines.  For example I am thinking specifically of cb routines bound by the `ETS_XXXX_INTR_ATTACH()` macros defined in `SDK//include/ets_sys.h` and execute the `ets_isr_attach()` function.  Now these seems to be true interrupt routines.\r\n\r\nThis last category often pick up the Lua_state for the main or other thread and operate on the Lua stack using the stack API calls and even call Lua functions.  _This is a recipe for death_.  It these _are_ true interrupt callbacks then there will be no guarantee that the Lua stack will be in a fit state to be used, and calling any complex processing such as Lua code will break all of the timing guidelines (10µS interrupts disabled) or allow reentrance of non-reentrant code.\r\n\r\nWhat these interrupt cb routines should do is stay firmly outside the lua environment (which really begs the Q of whether we can even use the registry) and if they want to carry out any Lua processing then this should be done by doing a `system_os_post()` to schedule a non-interrupt callback.  This will hit the latency of callbacks such as `gpio.trig()`.  All Lua execution and access to Lua stack and registry should be at the non-preemptive level.\r\n\r\nOn a related note, I see that the SDK also requires the same task approach for the espconn disconnect and abort functions, which we don't do, so we should roll up these changes at the same time.\r\n\r\nDoes anyone agree / disagree with this analysis?"},"comment":{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/comments/164945743","html_url":"https://github.com/nodemcu/nodemcu-firmware/issues/851#issuecomment-164945743","issue_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/851","id":164945743,"user":{"login":"nickandrew","id":7175,"avatar_url":"https://avatars.githubusercontent.com/u/7175?v=3","gravatar_id":"","url":"https://api.github.com/users/nickandrew","html_url":"https://github.com/nickandrew","followers_url":"https://api.github.com/users/nickandrew/followers","following_url":"https://api.github.com/users/nickandrew/following{/other_user}","gists_url":"https://api.github.com/users/nickandrew/gists{/gist_id}","starred_url":"https://api.github.com/users/nickandrew/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nickandrew/subscriptions","organizations_url":"https://api.github.com/users/nickandrew/orgs","repos_url":"https://api.github.com/users/nickandrew/repos","events_url":"https://api.github.com/users/nickandrew/events{/privacy}","received_events_url":"https://api.github.com/users/nickandrew/received_events","type":"User","site_admin":false},"created_at":"2015-12-16T00:36:51Z","updated_at":"2015-12-16T00:36:51Z","body":"What about network callbacks, e.g.\r\n\r\n>  espconn_regist_recvcb(pesp_conn, net_socket_received);\r\n>  espconn_regist_sentcb(pesp_conn, net_socket_sent);\r\n>  espconn_regist_disconcb(pesp_conn, net_server_disconnected);\r\n>  espconn_regist_reconcb(pesp_conn, net_server_reconnected);\r\n\r\nAre these guaranteed to be called only when no Lua task is running?\r\n"}}