{"action":"created","issue":{"url":"https://api.github.com/repos/gobblejs/gobble/issues/26","labels_url":"https://api.github.com/repos/gobblejs/gobble/issues/26/labels{/name}","comments_url":"https://api.github.com/repos/gobblejs/gobble/issues/26/comments","events_url":"https://api.github.com/repos/gobblejs/gobble/issues/26/events","html_url":"https://github.com/gobblejs/gobble/issues/26","id":51453259,"number":26,"title":"Method for nodes to register dependencies programmatically","user":{"login":"Rich-Harris","id":1162160,"avatar_url":"https://avatars.githubusercontent.com/u/1162160?v=3","gravatar_id":"","url":"https://api.github.com/users/Rich-Harris","html_url":"https://github.com/Rich-Harris","followers_url":"https://api.github.com/users/Rich-Harris/followers","following_url":"https://api.github.com/users/Rich-Harris/following{/other_user}","gists_url":"https://api.github.com/users/Rich-Harris/gists{/gist_id}","starred_url":"https://api.github.com/users/Rich-Harris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Rich-Harris/subscriptions","organizations_url":"https://api.github.com/users/Rich-Harris/orgs","repos_url":"https://api.github.com/users/Rich-Harris/repos","events_url":"https://api.github.com/users/Rich-Harris/events{/privacy}","received_events_url":"https://api.github.com/users/Rich-Harris/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2014-12-09T17:04:31Z","updated_at":"2015-01-04T13:29:47Z","closed_at":null,"body":"Gobble build definitions are defined as a sequence of transformations form source to result, rather than (as in JavaScript bundlers like webpack/browserify etc) by taking some final entry point and recursively discovering its dependencies. Generally speaking that's a more appropriate model for the (more general purpose) job Gobble is trying to do.\r\n\r\nThere are some occasions when it would be useful for a transformer to be able to declare its dependencies (that are outside its `inputdir`) programmatically. For example, if you have a bunch of markdown documents that you want to turn into HTML pages, you probably have an HTML template to wrap around the content:\r\n\r\n```js\r\nvar fs = require( 'fs' ),\r\n    marked = require( 'marked' ),\r\n    gobble = require( 'gobble' );\r\n\r\nvar template = fs.readFileSync( 'template.html', 'utf-8' );\r\nvar html = gobble( 'markdown' ).transform( function ( markdown ) {\r\n  return template.replace( '__CONTENT__', marked( markdown ) );\r\n});\r\n```\r\n\r\nIn this example the `html` node would contain the expected result, but if you changed the `template.html` file, Gobble wouldn't know that it needed to re-run the transformation.\r\n\r\nA few quick API ideas:\r\n\r\n```js\r\n// synchronous `gobble.read()` function returns file contents (caches the result\r\n// in memory so it doesn't hit the filesystem each time, obvs) and marks it\r\n// as a dependency\r\nvar html = gobble( 'markdown' ).transform( function ( markdown ) {\r\n  return gobble.read( 'template.html' ).replace( '__CONTENT__', marked( markdown ) );\r\n});\r\n\r\n// `.dependsOn(filename)` method registers dependency and returns `this`. Allows\r\n// explictly declaring dependencies in the build definition in an efficient way, but\r\n// doesn't allow programmatic declaration. Also, no easy caching\r\nvar html = gobble( 'markdown' ).transform( function ( markdown ) {\r\n  var template = fs.readFileSync( 'template.html', 'utf-8' );\r\n  return template.replace( '__CONTENT__', marked( markdown ) );\r\n}).dependsOn( 'template.html' );\r\n\r\n// dependency is explicity declared inside the function, webpack style. Again,\r\n// no help with caching\r\nvar html = gobble( 'markdown' ).transform( function ( markdown ) {\r\n  this.addDependency( 'template.html' );\r\n  var template = fs.readFileSync( 'template.html', 'utf-8' );\r\n  return template.replace( '__CONTENT__', marked( markdown ) );\r\n});\r\n\r\n// file node has a `.contents()` method which returns the contents and\r\n// registers the dependency\r\nvar template = gobble( 'template.html' ); // depends on issue #23\r\nvar html = gobble( 'markdown' ).transform( function ( markdown ) {\r\n  return template.contents().replace( '__CONTENT__', marked( markdown ) );\r\n});\r\n```\r\n\r\n(A synchronous API is preferable I think, since it means it can be used with both directory and file transformers.)\r\n\r\nNone of them leap out as being The Answer - would be interested if anyone out there has any feedback. Figuring this out would mean never again having to use static site generators (which in my experience compensate for their lack of intrinsic flexibility with overwhelmingly complex configuration).\r\n\r\n/cc @OliverJAsh and @theefer, since this is related to the stuff we were talking about a while back. Have you encountered this problem with Plumber?"},"comment":{"url":"https://api.github.com/repos/gobblejs/gobble/issues/comments/68632904","html_url":"https://github.com/gobblejs/gobble/issues/26#issuecomment-68632904","issue_url":"https://api.github.com/repos/gobblejs/gobble/issues/26","id":68632904,"user":{"login":"theefer","id":36964,"avatar_url":"https://avatars.githubusercontent.com/u/36964?v=3","gravatar_id":"","url":"https://api.github.com/users/theefer","html_url":"https://github.com/theefer","followers_url":"https://api.github.com/users/theefer/followers","following_url":"https://api.github.com/users/theefer/following{/other_user}","gists_url":"https://api.github.com/users/theefer/gists{/gist_id}","starred_url":"https://api.github.com/users/theefer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/theefer/subscriptions","organizations_url":"https://api.github.com/users/theefer/orgs","repos_url":"https://api.github.com/users/theefer/repos","events_url":"https://api.github.com/users/theefer/events{/privacy}","received_events_url":"https://api.github.com/users/theefer/received_events","type":"User","site_admin":false},"created_at":"2015-01-04T13:29:47Z","updated_at":"2015-01-04T13:29:47Z","body":"There is currently no operation that does this in Plumber, but it should indeed be possible, since all operations are chunks of a pipeline through which a new execution flows when files change. I've been wanting to do that for `plumber-jshint` (re-run if jshintrc changes) for instance.\r\n\r\nFor your example, it would probably look something like:\r\n\r\n``` javascript\r\nvar template = glob('template.html');\r\npipelines[''] = [\r\n  glob('markdown/**'),\r\n  markdown(template),\r\n  write('out');\r\n];\r\n```\r\n\r\n(Or else `markdown('template.html')` where the markdown operation would be responsible for calling something like `glob` on its argument internally, but that bloats the plugin imo.)\r\n\r\nEither way, I guess the point is that it's nice to be able to rely on the standard API for reading out a file (asynchronously) and being notified when it changes. Option 4 above seems the closest as you're just `gobble()`ing for the file and get a standard node out of it. The other options (esp 2 and 3) are more imperative and as I think you mention it's not clear how you'd do it programmatically (e.g. lookup jshintrc file in parent directories without using blocking fs calls).\r\n\r\nOnce you have both the template and the MD files as top-level objects (node, streams, whatever) that emit files initially and on change, you need a way to combine (\"merge\") them back together into a single node. I'm not familiar enough with Gobble, but it seems like you [already have that ability](https://github.com/gobblejs/gobble/wiki/How-to-write-a-gobblefile#merging-files) (conceptually at least), albeit just writing all the input node contents into the same output dir.\r\n\r\nI'm probably being stupid, but would something like this work technically?\r\n\r\n``` javascript\r\nvar template = gobble( 'template.html' );\r\nvar files = gobble( 'markdown' );\r\nvar html = gobble([template, files]).transform( function ( markdownOrTemplate ) {\r\n  if (markdownOrTemplate.filename !== 'template.html') {\r\n    var template = fs.readFileSync( 'template.html', 'utf-8' );\r\n    return template.replace( '__CONTENT__', marked( markdown ) );\r\n  }\r\n});\r\n```\r\n\r\nNow this is of course terrible and inefficient code, but I'm just trying to grasp the basic building blocks :-)\r\n\r\nIf this is not completely misled, it would seem that the issue is that the only way to merge nodes doesn't allow differentiating which node the files came from. Put otherwise, a transform can only really take a single node as input (source, merger or another transform node), but really here we want two.\r\n\r\nSo `template.contents()` in your option 4 seems to be the closest thing. Would you have a way to cause this code to register `template` as another dependency (or \"input\") to the transform node? I'm curious how that works. More importantly, it looks like you would have to make the `contents()` method return the contents synchronously; how would that work? Would you rely on having registered `template` as an input of the transform and only execute the transform body when both inputs are ready (so the data is available synchronously too)? And re-execute the body if any of the inputs fires?"}}