none
Using Session Pools in Machine Learning Server 9.3.0 RRS feed

  • Question

  • Hi,  I am currently attempting to use the Core APIs directly to execute R or Python code directly, not as published web services (as we have 100's of R-Scripts/Python Scripts being used) and we don't want to manage generating the API for each of them.  I currently pass the code and inputs/outputs to Machine Learning Server, after Logging in and then creating a session (costly item to do, 2-3 seconds sometimes).  Is there a way to use session pools with the core APIs and to not have to create a session directly?  Or do I have to create a Dedicated Session Pool for my application that is then used by each call to the Core APIs?

    I am trying to reduce or eliminate the creation of sessions if I can.  The documentation here: https://docs.microsoft.com/en-us/machine-learning-server/operationalize/configure-evaluate-capacity#pool even states that there is cost in creating your own session (in time and memory) and recommends using a session pool, but it alludes to the session pool that is already created when the compute node starts, but all methods require an ID for the session.  Is there a way to retrieve the default session pool ids so they can be used?

    Hope this makes sense.

    Thanks in advance for any suggestions you may have.

    -Scott

    Friday, August 31, 2018 6:04 AM

Answers

  • Hi Scott,

    Both the generic and dedicated pools are used only for published web services. 

    If you are using interactive R or Python session you can create the session once and then use remoteScript (https://docs.microsoft.com/en-us/machine-learning-server/r-reference/mrsdeploy/remotescript) multiple times. You can also do that from the core APIs, create a session once and then call execute several times with different scrips (https://microsoft.github.io/deployr-api-docs/#execute-code). There is no need to create a session for every script. However, the session does time out after 3 hours so you might need to create a new one by then. 

    Something else you can consider is to publish a web service that takes a script as a parameter and runs it. In that case you only publish one web service as a REST API endpoint but you can call ot with multiple scripts and use the generic or dedicated pools to improve the performance.

    Hope this helps.

    Efrat

    • Marked as answer by sehunley Wednesday, September 5, 2018 3:21 AM
    Friday, August 31, 2018 11:50 PM

All replies

  • Hi Scott,

    Both the generic and dedicated pools are used only for published web services. 

    If you are using interactive R or Python session you can create the session once and then use remoteScript (https://docs.microsoft.com/en-us/machine-learning-server/r-reference/mrsdeploy/remotescript) multiple times. You can also do that from the core APIs, create a session once and then call execute several times with different scrips (https://microsoft.github.io/deployr-api-docs/#execute-code). There is no need to create a session for every script. However, the session does time out after 3 hours so you might need to create a new one by then. 

    Something else you can consider is to publish a web service that takes a script as a parameter and runs it. In that case you only publish one web service as a REST API endpoint but you can call ot with multiple scripts and use the generic or dedicated pools to improve the performance.

    Hope this helps.

    Efrat

    • Marked as answer by sehunley Wednesday, September 5, 2018 3:21 AM
    Friday, August 31, 2018 11:50 PM
  • Efrat,

    First, thanks for the reply.  Yes, for when I am not using the deployed service (code) model, I have created a session and then reused it for all calculations.  However, I am trying to come up with a more high performance mechanism, and the "Deployed-Code-As-A-Service" with the session pools (Generic or Dedicated) model appears to be the way to go.  However I need this to be done dynamically. I am trying to post my ExecuteRequest data against the following url:  basePath/{service-name}/{service-version}/.

    For Example, let's say that I have deployed 2 services, DeployedCalculationA and DeployedCalculationB to my Machine Learning Server (version 9.3.0).  I would think that I could construct an ExecuteRequest with my inputs and outputs and post to a server address like http://localhost:12800/services/DeployedCalculationA/1.0.0.0 (and likewise, http://localhost:12800/api/DeployedCalculationB/1.0.0.0) supplying the execution request to it, much like the sessionAPI's ExecuteCode already does.  I would still put the bearer token in the header, but I would let the backend take care of getting the session from the session pool so I don't have to take this hit and I also don't have to generate an API client for every service I deploy.  I simply want to RESTfully consume the deployed services, while benefiting from the Session Pool and not having to create a session myself.  Is this possible you think?

    I was hoping there was a generic way to consume deployed services via a RESTful API, in an authenticated manner, without creating the session.

    Again, thanks for your response.

    -Scott



    • Edited by sehunley Tuesday, September 4, 2018 5:03 AM
    Tuesday, September 4, 2018 3:48 AM
  • I was able to get the code working, doing what I wanted.  I can deploy R or Python scripts as Web Services and then use a generic adapter/interface to pull up the inputs and outputs for the deployed web service and map my inputs, execute and then get the outputs and return them to the caller.  I still have to authenticate, but I let the session pool handle sessions verses creating, sharing, recycling and cleaning up the sessions my self.  I also don't have to generate a client via swagger for each service I deploy, but I can still have that option for 3rd parties who wish to consume my service as a "micro-service" of sorts.  I can also take advantage of the batch and dedicated session pools.  So I will close this question. 

    I was curious though, when creating my own session, I couldn't find any information on the lifespan of a session.  I see the information regarding the authorization (bearer token), but couldn't find anything for the session.  Can you tell me how long the session lasts before timing out?

    Thanks,

    -Scott

    Tuesday, September 4, 2018 6:57 PM
  • The session timeout is 3 hours.

    Glad to hear the service solution works for you. 

    Tuesday, September 4, 2018 8:22 PM