Python in Knime: Downloading files and dynamically pressing them into workflow
<p>I'm using Knime 3.1.2 on OSX and Linux for OPENMS analysis (Mass Spectrometry).<br><br>
Currently, it uses static filename.mzML files manually put in a directory. It usually has more than one file pressed in at a time ('Input FileS' module not 'Input File' module) using a ZipLoopStart.<br><br>
I want these files to be downloaded dynamically and then pressed into the workflow...but I'm not sure the best way to do that. <br><br>
Currently, I have a Python script that downloads .gz files (from AWS S3) and then unzips them. I already have variations that can unzip the files into memory using StringIO (and maybe pass them into the workflow from there as data??).<br><br> 
It can also download them to a directory...which maybe can them be used as the source? But I don't know how to tell the ZipLoop to wait and check the directory after the python script is run. <br><br>
I also could have the python script run as a separate entity (outside of knime) and then, once the directory is populated, call knime...HOWEVER there will always be a different number of files (maybe 1, maybe three)...and I don't know how to make the 'Input Files' knime node to handle an unknown number of input files. <br><br>
I hope this makes sense. 
Thanks!</p>