none
Remote session execute call timeout 504 error RRS feed

  • Question

  • Upon executing a code on the Remote session, If the execution time is more than 1 hour. The execute POST call fails with 504 error code and the execution stops after 1 hour. Is there some sort of limit for same and if yes how to bypass that?

    Code to reproduce error (To run on Remote session)
    "
    index=1
    while(TRUE) index = index+1
    "

    I am running ML Server 9.2.1 and both the host and client OS is Ubuntu 16.04

    • Edited by Yogesh Bansal Tuesday, December 5, 2017 4:36 PM adeed system info
    Tuesday, December 5, 2017 4:35 PM

All replies

  • Hi Yogesh,

    We currently have a bug around execution longer than 60 minutes that will be fixed in the next version of Machine Learning Server. If you have a customer service account please contact us by opening a ticket.

    Monday, December 11, 2017 8:53 PM
  • Is this bug still exists?

    During remote execution of R script I have status_code: 504, message: a task was canceled, exceptionType: TaskCanceledException.

    It appears with R scripts working longer than 30 minutes.

    Monday, July 30, 2018 12:52 PM
  • I'm also wondering if there is a fix to this bug. I have issues with scripts running more than 1 hour (timeout when this occurs).
    Tuesday, September 11, 2018 10:46 AM
  • I edited the web node's appsettings.json, and set CacheLifeTimeInMinutes to 180 and ConnectionTimeOut to "03:00:00". This allowed me to run a script for longer than an hour.

    ML Server 9.3

    • Proposed as answer by m.ebbesson Tuesday, September 11, 2018 1:06 PM
    Tuesday, September 11, 2018 1:06 PM
  • Hi, i'm currently testing ML Server (R) via REST API and have some ML operations which take longer than an hour to execute, consequently my ExecuteCode() returns with an exception on the hour mark but leaves the R script running, this means that i've lost the atomic blocked operation connection and the OutputParameters that I was waiting on.

    In your post you're able to increase this timeout to 3 hours, could you tell me please exactly what your modified appsettings.json looks like? I tried decreasing the "ConnectionTimeout" in the "ComputeNodesConfiguration" section to 1 minute just to see if it had any effect but it didn't.

    Thanks.

    Thursday, September 27, 2018 7:32 AM
  • Hi, did you get a reply to this? I'm very interested in the solution as well.

    Cheers.

    Tuesday, November 6, 2018 1:21 AM