Surprisingly, and confusingly, this bug actually improved the probe RMSE significantly over the correct code.

The code within the training loop was:

movie_features[movieid-1][f] += LRATE2 * (err*(uf_old+alpha_uk_old*dev_u_hat) - LAMBDA2*mf_old);
alpha_u_k[i][f] = LRATE2 * (err*dev_u_hat*mf_old - LAMBDA2*alpha_uk_old);

And within the prediction function:

sum += movie_features[movieid-1][f] * (user_features[u][f] + alpha_u_k[u][f]);

In the training, there was a "+" plus sign absent - it should have been "alpha_u_k[i][f] +=" rather than just "=". And in the prediction function, the "*dev_u_hat" term was missing. It should have been:

sum += movie_features[movieid-1][f] * (user_features[u][f] + alpha_u_k[u][f]*dev_u_hat);

Yet despite these errors, I achieved a probe RMSE of 0.901871 with 100 features. When I actually corrected the code, the RMSE surprisingly increased to 0.907681 (also with 100 features).

I also tried a model where the training step matched the incorrect prediction function. That is, I changed the code to:

movie_features[movieid-1][f] += LRATE2 * (err*(uf_old+alpha_uk_old) - LAMBDA2*mf_old);
alpha_u_k[i][f] += LRATE2 * (err*mf_old - LAMBDA2*alpha_uk_old);
...
sum += movie_features[movieid-1][f] * (user_features[u][f] + alpha_u_k[u][f]);

I tested with 10 features. The RMSEs I got:

Correct model 0.921176
Bad model and bad training 0.913509
Bad model, proper training 0.917931

So the bad model with training that mis-matched the prediction formula actually outperformed the algo where the training step matched the incorrect algorithm (with dev_u_hat absent).

I find this very befuddling, as there is no reason I can think of which the wrong formula with a training step that doesn't even match the formula, would outperform not only the correct model, but also one where the training step was fixed.

The incorrect training step makes the contributions of the term seem pretty random to me - yet if they were random, they shouldn't improve the RMSE. If there is some pattern there, then I can't think of it.

• Qt
• LAPACK
• 64-bit OS (or 32-bit OS with increased RAM limits for processes) and 64-bit compiler

The download includes Average, Globals, KNN, Matrix Factorization, and blending classes. All of these extend the Algorithm class, which contains the Algorithm::predict(movieid, userid, votedate) and Algorithm::runProbe() functions. To create a new algorithm, make a new class (see the included classes for guidance) centered around the predict() function. The setMovie() and determine() functions are just empty functions added for compatibility with the existing code. You can then use that algorithm to predict the data sets via Algorithm::runProbe() and Algorithm::runQualifying() (see algorithm.cpp for details). You can even run an algorithm on the training set using Algorithm::runTaining() if you wish.

Use the Algorithm::buildPreProcessor("fileprefix") function to build pre-processing files, containing residuals that can be loaded and used to generate models of their own. Note that these files are another ~800MB that must be mapped into memory, so you need lots of RAM to do this.

The blending classes are algorithms of their own, and can be used just like any other model. The ::setup(...) functions take in the names of the preprocessor files and perform the blending calculations based on the probe set, with the first argument needing to be the number of models being blended. The code below is an example that will generate residuals for the Average and Matrix Factorization algorithms, then blend then and run the result on the probe set.

Average avg(&db);
avg.buildPreProcessor("somefolder/averagefilename");
 
Matrix_Factorization mf(&db);
mf.calculate_incr();
mf.buildPreProcessor("somefolder/matrixfilename");
 
Blend blend(&db);
blend.setUp(2, "somefolder/averagefilename", "somefolder/matrixfilename");
blend.runProbe;

It implements Icefox's framework, details of which are in this thread.