How it works...

The performance of the model can be assessed using many metrics such as accuracy, Area under curve (AUC), misclassification error (%), misclassification error count, F1-score, precision, recall, specificity, and so on. However, in this chapter, the assessment of model performance is based on AUC.

The following is the training and cross validation accuracy of the trained model:

# Training accuracy (AUC)
> occupancy_train.glm@model$training_metrics@metrics$AUC
[1] 0.994583

# Cross validation accuracy (AUC)
> occupancy_train.glm@model$cross_validation_metrics@metrics$AUC
[1] 0.9945057

Now, let's assess the performance of the model on test data. The following code helps in predicting the outcome of the test data:

# Predict on test data
yhat <- h2o.predict(occupancy_train.glm, occupancy_test.hex)

Then, evaluate the AUC value based on the actual test outcome as follows:

# Test accuracy (AUC)
> yhat$pmax <- pmax(yhat$p0, yhat$p1, na.rm = TRUE) 
> roc_obj <- pROC::roc(c(as.matrix(occupancy_test.hex$Occupancy)),
                       c(as.matrix(yhat$pmax)))
> auc(roc_obj)
Area under the curve: 0.9915

In H2O, one can also compute variable importance from the GLM model, as shown in the figure following this command:

#compute variable importance and performance
h2o.varimp_plot(occupancy_train.glm, num_of_features = 5)

Variable importance using H2O