{"push_id":850484838,"size":3,"distinct_size":3,"ref":"refs/heads/master","head":"ae1a4f97da757cb26e48dd9bef6b4f4c250841da","before":"19f776dd8480ee37993503673526e107480df45c","commits":[{"sha":"6f44d06d7979c574c07081999b4001dbd52d4241","author":{"email":"d00504da4ef05483190ad031d6e91c1113599f6c@graphlab.com","name":"Yucheng Low"},"message":"Numerous Fixes from internal testing\n\nThis is a long list of fixes made during internal testing.\n\nSome segfaulting Fixes\n----------------------\n    - Eigen apparently wants Eigen::initParallel()\n    - make some environment variable checks not end in null pointer error\n    if environment variable not set.\n\nRandom\n------\n\n - Added another multinomial sampling function\n\nMore SSL fixes\n--------------\n - The certificates are not picked up correctly\n   in the webstor code (which we should deprecate)\n\nFunny Termination Oddity\n------------------------\n- The following code takes a very very long time to terminate (20s-ish) if\n     you just put it in a file and run python on it. (not ipython).\n     ```\n     import sframe as gl\n     gl.SArray([1,2]).apply(lambda x:x)\n     ```\n     and it will print out \"Unable to reach server\" a few times before\n     terminating.\n\n     Can't quite figure out why, but it looks like unity_server is waiting\n     for ping replies from the lambda workers, and the ping replies are not\n     coming. So it has to wait for full 5 seconds for timeout.\n\n     What is doubly fun is that if you do this\n     ```\n     import sframe as gl\n     gl.SArray([1,2]).apply(lambda x:x)\n     raw_input(\"pika\") # and you wait for one second before pressing enter.\n     ```\n     it works great.\n\n     Fix\n      - made the ping wait in 1 second increments rather than 5 seconds at\n      a time. This speeds up the above case to something more like ... 6s\n      for full termination. Which is a little better. But I can't quite\n      figure out why the ping is not responding.\n      - Removed the \"Unable to reach\" message.\n\nLog Message Printing\n--------------------\n- Now that stderr is piped all the way to display, we have some new\n  printing oddities. In a number of places, we use exceptional control\n  flow and that is no good. i.e.\n    ```\n            bool try_to_open_file(const std::string url) {\n              bool success = true;\n              try {\n                general_ifstream fin(url);\n                success = !fin.fail();\n              } catch(...) {\n                success = false;\n              }\n              return success;\n            }\n    ```\n\n    This means that when try_to_open_file(...) returns false, which is\n    actually ok, it emits a bunch of ugly errors to screen. Like:\n    ```\n    1446060349 : ERROR:    union_fstream.cpp(operator():98): Cannot open /tmp/tmpAdJ8Q5/dir_archive.ini for reading\n    1446060349 : ERROR:    general_fstream.cpp(operator():29): Cannot open /tmp/tmpAdJ8Q5/dir_archive.ini for read. Cannot open /tmp/tmpAdJ8Q5/dir_archive.ini for reading\n    1446060349 : ERROR:    unity_global.cpp(operator():121): Unable to load model from /tmp/tmpAdJ8Q5: Cannot open /tmp/tmpAdJ8Q5/dir_archive.ini for read. Cannot open /tmp/tmpAdJ8Q5/dir_archive.ini for reading\n    ```\n    I have an incomplete fix for this case (in that it will still emit ugly\n    messages if the file actually exists, but is unreadable). But this is\n    something we need to be more careful about.\n\n    Since this does appear in a number of places, for now, after offline\n    discussion with Evan, we are bumping LOG_WARNING and LOG_ERROR to\n    stdout, where it is *not* forwarded to the user (it will still show\n    up in the log file).\n\n    More generally, we need some better exception handling. i.e.\n\n    try {\n        log_and_throw(...);  // this does not print to display, but logs it\n                             // The message gets attached to the exception\n    }\n\n    Then the user doesn't actually see LOG_ERROR messages but all of those\n    messages just get associated with the exception.\n\n    Also fixed some color printing issues (don't print the color codes\n    if we are not going to actually print the message)\n\nWindows Linking\n---------------\n\n    There were 2 issues.\n    First issue was with linking. Windows requires the dll (or a .lib) file\n    to be present for linking; even if you are linking a shared library.\n    This means the \"python_header_only\" dependency cannot exactly be \"header\n    only\". But it is still fine.\n\n    Second issue is with termination.\n    Windows appears to kill threads then call DLLUnload (which calls the\n    global object destructors). This is problematic since all of our\n    state is in DLLs. We can do more cleanup here or we can just die.\n    (this is going to be a problem when we eliminate the unity_server\n    subprocess)\n\nRemoving libpython dependency\n-----------------------------\n\n     - Split the pylambda library into two halves (pylambda and\n         pylambda_worker_lib).\n       - pylambda which is the \"server\" only side only and doesn't know how\n       to speak python (this is the commander side which just sends lambdas to be\n       executed) , and pylambda_worker_lib which knows how to speak python (which\n       actually executes the lambdas)\n\n     - Also, everything links in boost python which causes issues.\n       - Split boost into 2 halves, boost and boost_python\n       - Add a python_header_only library\n\n     - This required the addition of an \"override_cmake\" directory which\n     contains rules that overrides the files in cmake/. This can be thought of as a\n     staging area for rules to go before adding them to dato-deps (our dependency\n     building repository)","distinct":true,"url":"https://api.github.com/repos/dato-code/SFrame/commits/6f44d06d7979c574c07081999b4001dbd52d4241"},{"sha":"2a620c78467d16f8b8752b8090abf1024e247ac2","author":{"email":"d00504da4ef05483190ad031d6e91c1113599f6c@graphlab.com","name":"Yucheng Low"},"message":"Missed some files","distinct":true,"url":"https://api.github.com/repos/dato-code/SFrame/commits/2a620c78467d16f8b8752b8090abf1024e247ac2"},{"sha":"ae1a4f97da757cb26e48dd9bef6b4f4c250841da","author":{"email":"178b62e73cc3fab9e30bf27b2372c9ff92401915@dato.com","name":"Evan Samanas"},"message":"Merge pull request #71 from ylow/sync\n\nNumerous Fixes from internal testing","distinct":true,"url":"https://api.github.com/repos/dato-code/SFrame/commits/ae1a4f97da757cb26e48dd9bef6b4f4c250841da"}]}