none
Multi threading (issues with speed) RRS feed

  • Question

  • Hello,

    I am new to the Open R, having used R for many years. I tried to run a code which I hoped would be considerably faster using multi-threading. However, it turns out it is incredibly slow. I used getMKLthreads() which gets me 6 as expected, but running the code it only seems to be using 25% CPUE. Looking up some other threads, I wanted to set the setMKLthreads() to 1, which resulted in an immediate shut-down of RStudio.

    I also tried to follow the suggestion to use version.compare, but 

    a) don't believe it will give me any additional information as my Open R is running very slow

    b) due to security at work I am unable to use GIT.

    Any ideas? 

    The Code I am running is a loop over 10-20 timesteps, so I understand this will have to be done, however each of the steps does run a boot-strapping procedure calling a package using ADMB.

    If anyone has any suggestions, that would be great.

    Also any possible explanation why each step takes considerably longer with multithreading/open R would be welcome.

    In "normal R one timestep takes approximately 10-15 minutes, in Open R it takes about 1 hour!!

    Thanks already,

    Pia


    Monday, October 21, 2019 8:16 AM

All replies

  • My only thoughts is your RAM is filling up causing delays due to RAM <-> disk storage swapping. While I'm not 100% sure how the multi-threading works - it's possible multi-threading utilizes more RAM as part of parallelization, which could put you in the above situation.
    Tuesday, October 22, 2019 7:03 PM
  • Not all computations in R run faster using the MKL libraries. Mostly matrix computations will be faster when using MKL.

    You can find more information and benchmarks for MKL here:

    https://mran.microsoft.com/documents/rro/multithread

    Monday, November 4, 2019 5:25 PM