none
Use of revoScaleR Package RRS feed

  • Question

  • I'm using the rxGlm() function to create GLMs using some quite large data sets - large enough for there to be a big performance advantage compared to R's built in glm() function. However I'm having some problems with things like model diagnostics such as:

    • Chi squared tests (eg on pairs of nested models)
    • F tests (eg on pairs of nested models)
    • Plotting residuals vs fitted values
    • Q-Q plots to assess the suitability of the model

    I had hoped to be able to use the as.glm() to convert model objects creating using rxGlm into "standard" glm model objects, but I'm getting lots of error messages. Is there a way to stop this happening? Or some other way to produce these items?

    Also has the revoScaleR package been removed from the latest release, Microsoft R Open 3.5.3????????? (I'm using 3.4.3)

    Thanks,

    A.



    Tuesday, May 28, 2019 8:23 PM

All replies

  • Hello,

    Their really isn't a need to convert the rxGlm object to a R glm object in order to generate model diagnostic plots and compute summary statistics for the fitted model. You can compute a chisquare test using the rxCrossTabs() function - it takes a model formula as input. You can use the rxStepControl() function for stepwise model selection. 

    It is also possible to plot the residuals vs the fitted values, but rxLinMod() does not by default return fitted values for regression because it is designed for very large datasets. You need to explicitly call the rxPredict() function on the fitted model object to obtain fitted values and residuals from the fit, then you can use the rxLinePlot() function to plot the residuals vs the fitted values. 

    Their are some examples of how to do this on the following page:

    https://docs.microsoft.com/en-us/machine-learning-server/r/how-to-revoscaler-linear-model

    Hope this helps.

    Thursday, May 30, 2019 2:13 PM
  • Thank you...this could be quite helpful. I will look at the link and try a few things out on my data set. I may be back with more questions...

    Very glad to have found someone familiar with the revoScaleR package!

    Thursday, May 30, 2019 2:23 PM