{"push_id":563331593,"size":1,"distinct_size":1,"ref":"refs/heads/ngas","head":"01f7e36edab27158b17bfdc28ad7b5d74f36ca33","before":"6d9f9736875788d3594c5b54735c175c67c7f78c","commits":[{"sha":"01f7e36edab27158b17bfdc28ad7b5d74f36ca33","author":{"email":"5fb552a76ef3c7ee67681d80e9797e088a6c9859@markwhite.name","name":"Mark White"},"message":"Finished running 4 diff simulations: 2 with 30 degree rotations, with and without integration of training examples across 2nd layer of network.  Signifiicant difference between 30 degree and 0 degrees as to training and test error.  This is understandable: 30 degrees puts most of the differentiating information into one of the 2 input variables.\n\nSeparately, and most important, is the fact that, for both rotations, the test error was significantly better for Proj4.\n\nHave made some minor mods for operation with Docker and AWS execution.  Continuing with experiments using GitHub as code base for simulations on docker.\n\nNow adding simplest logger (StringIO and FileIO), that will feed into Redis, and then possibly mongo.  KEEP IT SIMPLE>>>\n\ngetting urls and $currenthost doing the right thinigs... AGAIN and saving again  REMOTE redis storage working just fine now.\n\nBoth Proj3 and Proj4 working with Redis and environment variables.\n\n*** All std Projects working:  BasicBPDemo, Proj3, Proj4, and Proj3comparedTo4  I'm about to start designing/coding the cat's meow.\n\nJust did a significant amount of reshuffling code across files.  Some renaming of .rb files, also.  All bins work...\n\nWoops, I need to reverse about an hours worth of editing in the Learning Strategy file.!!\n\nAll std Projects working again:  I forgot the \"self.\" in \"self.netInput\"!\n\nspent a little time with docker and redis -- to get persistence between rebuilds and reruns.\n\nNow testing the continuous use of \"alwaysOn\" server for Redis storage of simuluation results........\n\nOk, the server and redis stuff is done for now.  Now I've been adding code for Symmetrical Models for Self-Org and Supervised.  Also considering stochastic switching of neuron outputs according to the probability that the example is from one of the two lobe (Gauss Models).  In other words, if an example is unlikely \"from\" the lobe models, the example should be \"ignored by that neuron\" -- or, the neuron should indicate that it \"has NO OPINION\"!!\n\nMade significant progress on mixture model for the distribuion of a neuron's netInput or outputs.  netInputs or outputs depends on... the learning strategy for effectively and adaptively incorporate sigmoid (and other) nonlinearities in the network's processing.\n\nBeen testing using scratch file.  Debug is going quickly...  Having some numerical (complex number) issues... seems to be related to convergence and using better initial estimates...    Got pdf properly scaled.  Streaming estimate of std and mean seems to be working acceptably.  BUT need to test more.  Also need to make supposedly more stable (but memory hog) batch version for calculating std......\n\nDid quite a bit of renaming of mixture model strategy methods.  Created stable functional testing file for testing 3 of the classes...\n\nCobbling an initial learning strategy Class together for the purpose of simple mixture model fitting. -- not even changing weights, after the input normalization!!\n\nMoved to fast, but got some stuff understood and fixed;  but I now need to do clean-up and perhaps some significant simplification RELATED to OUTPUT NEURON vs OTHER NEURONS.  Perhaps they should/could all be treated the same, or...\n\nRefactoring so that OutputNeuron(s) inherit from Neurons.  This also improves the learning strategies, because a separate version for the  \"output neurons\" does not have to be created.\n\nGot quite a bit working.  Now need to sort-out how/if to use normalization with mixture model self-org.  Also, \"normalization of bias neuron\" is obviously unnecessary.  But what to do with current code...?\n\nnow debuggin lobe mover....\n\nNow finally making some progress.  Making Gauss Models simpler (i.e. re. to calculating std)...\n\nMade some real progress here.  Now just about to fix Normalization and bias weight initialization for current version (i.e., mixture model EM) of sel-organization.\n\nOK, now need to verify things are working properly with new \"bias weight initialization\" scheme.\n\nWorking on moveLobesApart(fractionToMove)\n\nScaling of weights works-- for moving lobes apart and maintaining d'   -- simple linear system before the output non-linearity.  Very simple self-org mechanism, using EM with mixture of 3 Gaussian models, the first 2 having adaptable means and std.  aPriori for each of the 3 models is adapted.  May want to adapt std for 3rd model -- to a limited extent!?\n\nWorking on enabling EASY IMPLEMENTATION of  \"CONTEXT\" as in when a neuron's output can potentially suppress the propagation, activity, or function of another neuron or neurons\n\nmore than midway in improving and integrating learning strategies with context learning control.\n\nSimplifying code for incorporating learning strategies and context.  All bins working normally.  BPDemo, SingleNeuronSelfOrg using mixture model, Proj3 & Proj4 Self-org using prior self-org method.  For Proj3 and Proj4, I'm about to include EM Mixture model self-org.\n\nAdded benchmarks.txt file in nCore \"top directory\"\n\nAppear to have a multilayer mixture model (with potential context).\n\nFirst level of debugging on non-context layer appears to be working!  More progress.  Supervised learning also seems to be working ok.  More and more testing\n\nNeed to take care of very low standard deviations... creating NaN condition...","distinct":true,"url":"https://api.github.com/repos/noisyneurons/nCore/commits/01f7e36edab27158b17bfdc28ad7b5d74f36ca33"}]}