none
Multi-threading using Microsoft R rxXYZ() functions RRS feed

  • Question

  • Hi,

    Does all Microsoft R functions (specifically the functions with prefix rx, e.g., rxImport(), rxDataStep(), rxBTrees(), etc.) use multi-threading by default?

    If yes, how can I monitor how many cores (or threads) in my local computer are being used while executing these functions? I have seen that while I execute rxBTrees(), in Task Manager, 'Microsoft ML Engine' process starts which consumes ~50-60% of CPU. Does this mean parallel computing is happening? How to monitor/control the number of threads?

    If the answer is 'not all functions use multi-threading', then which functions do/do not?

    Thanks.

    Wednesday, July 25, 2018 2:14 PM

Answers

  • Yes, all Microsoft R functions use multi-threading to some degree.

    Their are two different ways to control multi-threading in Microsoft R.

    1).  For all functions that utilize the MKL libraries you can use the function setMKLthreads() to control the number of threads used.

    2). For all RevoScaleR functions you can control the number of threads used via 'rxOptions()' function:

    rxOptions(numCoresToUse=8)  # for example

    To monitor how many cores are used you can do that via the Windows Task Manager.

    Goto the 'Performance' tab, click on 'CPU', then right-click and select 'Change graph' to view the 'Logical Processors' being utilized.

    Friday, August 3, 2018 4:26 PM

All replies

  • Yes, all Microsoft R functions use multi-threading to some degree.

    Their are two different ways to control multi-threading in Microsoft R.

    1).  For all functions that utilize the MKL libraries you can use the function setMKLthreads() to control the number of threads used.

    2). For all RevoScaleR functions you can control the number of threads used via 'rxOptions()' function:

    rxOptions(numCoresToUse=8)  # for example

    To monitor how many cores are used you can do that via the Windows Task Manager.

    Goto the 'Performance' tab, click on 'CPU', then right-click and select 'Change graph' to view the 'Logical Processors' being utilized.

    Friday, August 3, 2018 4:26 PM
  • This works perfectly.

    Thanks @stevefw

    Saturday, August 11, 2018 3:45 AM