none
SQL Server Polybase External Table with Hadoop HDFS issue RRS feed

  • Question

  • I am getting the following error accessing the HDFS file, the interesting things is ,PolyBase is connecting to HDFS and trying to access the correct file but somehow resulting into error, can't figure-out why ? Please find the External configuration below

    Error:

    Msg 7320, Level 16, State 110, Line 33

    Cannot execute the query "Remote Query" against OLE DB provider "SQLNCLI11" for linked server "(null)".

     HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743046_2226 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv

    external table configuration


    CREATE DATABASE SCOPED CREDENTIAL Hadoop_Cred WITH IDENTITY = 'raj_ops', Secret = 'raj_ops';
    GO
    CREATE EXTERNAL DATA SOURCE MyHadoopCluster WITH (  
          TYPE = HADOOP,
          LOCATION ='hdfs://192.168.0.107:8020',
          CREDENTIAL = Hadoop_Cred
    );
    CREATE EXTERNAL FILE FORMAT Hadoop_file_format WITH (  
          FORMAT_TYPE = DELIMITEDTEXT,
          FORMAT_OPTIONS (FIELD_TERMINATOR =',',
                USE_TYPE_DEFAULT = TRUE))

                DROP EXTERNAL TABLE HDFS_File

    CREATE EXTERNAL TABLE [dbo].HDFS_File
    (  
    street nvarchar(500),city nvarchar(500),zip nvarchar(500),[state] nvarchar(500),beds nvarchar(500),baths nvarchar(500),sq__ft nvarchar(500),[type] nvarchar(500),sale_date nvarchar(500),
    price nvarchar(500),latitude nvarchar(500),longitude nvarchar(500)
    )  
    WITH (  
          LOCATION = N'/user/raj_ops/TP/Sacramentorealestatetransactions.csv',  
          DATA_SOURCE =MyHadoopCluster,  
          FILE_FORMAT =Hadoop_file_format,  
          REJECT_TYPE = VALUE,  
          REJECT_VALUE = 0  
    );

    SELECT * FROM HDFS_File



    • Edited by SQL_Content Friday, January 17, 2020 2:58 AM
    Thursday, January 16, 2020 3:02 PM

All replies

  • Hi,

    Please refer to PolyBase Setup Errors and Possible Solutions.

    Hope this could help you.

    Best regards,
    Cathy 

    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to  MSDN Support, feel free to contact MSDNFSF@microsoft.com

    Friday, January 17, 2020 8:40 AM
  •  

    I did check the Error and Possible Sollutions documentation , did not help.

    Regards

    sufian

    Friday, January 17, 2020 1:58 PM
  • Here is the error from the polybase error logs

    MSSQLSERVER_WIN-ALO32Q0SFC2_Dms_errors.log

    1/18/2020 12:51:47 PM [Thread:1328] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 0:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[106000:1]: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv ---> Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridgeAccessException[106000]: Java exception raised on call to HdfsBridge_RecordReaderFillBuffer. Java exception message:
    HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethodHelper(IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args, JavaMethodCallAction methodCallAction)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethod[T](IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.HdfsBridgeJniWrapper.RecordReaderFillBuffer(Byte[] byteBufferToFill, Boolean& isDoneReadingInputSplit)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       --- End of inner exception stack trace ---
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalMoveBufferReader.Read()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:1328] [DmsCoreInstrumentation:DmsReaderStepErrorEvent] (Error, Critical):
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[106000:1]: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv ---> Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridgeAccessException[106000]: Java exception raised on call to HdfsBridge_RecordReaderFillBuffer. Java exception message:
    HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethodHelper(IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args, JavaMethodCallAction methodCallAction)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethod[T](IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.HdfsBridgeJniWrapper.RecordReaderFillBuffer(Byte[] byteBufferToFill, Boolean& isDoneReadingInputSplit)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       --- End of inner exception stack trace ---
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalMoveBufferReader.Read()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.Execute(Object status)

    MSSQLSERVER_WIN-ALO32Q0SFC2_DWEngine_errors.log

    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): TRACE FROM DMS:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[106000:1]: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv ---> Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridgeAccessException[106000]: Java exception raised on call to HdfsBridge_RecordReaderFillBuffer. Java exception message:
    HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethodHelper(IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args, JavaMethodCallAction methodCallAction)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethod[T](IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.HdfsBridgeJniWrapper.RecordReaderFillBuffer(Byte[] byteBufferToFill, Boolean& isDoneReadingInputSplit)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       --- End of inner exception stack trace ---
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalMoveBufferReader.Read()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Query:QID4368, Plan:34891161-998e-415e-bdac-ec0d270c4bb0, Step:0
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Command Type:EXTERNAL_ROUNDROBIN
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Number of columns:12
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:0, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:1, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:2, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:3, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:4, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:5, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:6, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:7, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:8, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:9, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:10, SqlDbType:NVarChar
    1/18/2020 12:51:47 PM [Thread:5320] [CommandWorker:ErrorEvent] (Error, High): Target column:11, SqlDbType:NVarChar

    MSSQLSERVER_WIN-ALO32Q0SFC2_Dms_movement.log

    1/18/2020 12:50:12 PM [Thread:1616] [WriterWorker:InformationEvent] (Info, Normal): ==> Consumer Credit 3758af1e-35da-4e2a-8203-e6737d640d65 8: Credit=2 for 8 producers will be sent to node WIN-ALO32Q0SFC2:1433.
    1/18/2020 12:50:12 PM [Thread:1616] [WriterWorker:InformationEvent] (Info, Normal): Issuing Consumer Credit 3758af1e-35da-4e2a-8203-e6737d640d65 8: Credit=2 for 8 producers per node.
    1/18/2020 12:50:12 PM [Thread:1616] [DmsPerformanceMonitor:WarningEvent] (Warning, Normal): Requested Performance counter DmsPendingWriterSteps does not exist!
    1/18/2020 12:50:12 PM [Thread:1616] [DmsPerformanceMonitor:WarningEvent] (Warning, Normal): Requested Performance counter DmsWriterSteps does not exist!
    1/18/2020 12:50:12 PM [Thread:1616] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 15: Executing bulk insert into distribution 8
    1/18/2020 12:51:47 PM [Thread:1328] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 0:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[106000:1]: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv ---> Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridgeAccessException[106000]: Java exception raised on call to HdfsBridge_RecordReaderFillBuffer. Java exception message:
    HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethodHelper(IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args, JavaMethodCallAction methodCallAction)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethod[T](IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.HdfsBridgeJniWrapper.RecordReaderFillBuffer(Byte[] byteBufferToFill, Boolean& isDoneReadingInputSplit)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       --- End of inner exception stack trace ---
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalMoveBufferReader.Read()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:1328] [ExternalMoveBufferReader:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 0: --  Total time waiting for free buffers: 0.0193 msecs
    1/18/2020 12:51:47 PM [Thread:1328] [ExternalMoveBufferReader:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 0: --  Total time in external read: 95302.7391 msecs; Total bytes read: 0
    1/18/2020 12:51:47 PM [Thread:1328] [DmsCoreInstrumentation:DmsReaderStepErrorEvent] (Error, Critical):
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[106000:1]: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv ---> Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridgeAccessException[106000]: Java exception raised on call to HdfsBridge_RecordReaderFillBuffer. Java exception message:
    HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethodHelper(IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args, JavaMethodCallAction methodCallAction)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.JavaBridge.CallAndLogJavaMethod[T](IntPtr javaObject, JavaMethod methodEnum, Boolean ignoreExceptions, Object[] args)
       at Microsoft.SqlServer.DataWarehouse.Hadoop.HadoopBridge.HdfsBridgeJniWrapper.RecordReaderFillBuffer(Byte[] byteBufferToFill, Boolean& isDoneReadingInputSplit)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       --- End of inner exception stack trace ---
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.ExternalAccess.HdfsBridgeReadAccess.Read(MemoryBuffer buffer, Boolean& isDone)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalMoveBufferReader.Read()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ExternalMoveReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:1328] [AbstractReaderWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 0: ExternalMoveWorker Total buffer count: 0, PPax data buffer count 0, Buffer efficiency: N/A.
    1/18/2020 12:51:47 PM [Thread:1328] [CommandManager:InformationEvent] (Info, Normal): Entering Worker Thread for: CommandManager:ProcessCancelMessage: 34891161-998e-415e-bdac-ec0d270c4bb0
    1/18/2020 12:51:47 PM [Thread:5568] [CommandManager:InformationEvent] (Info, Normal): Set up a timeout watchdog thread for: CommandManager:ProcessCancelMessage: 34891161-998e-415e-bdac-ec0d270c4bb0, timeout: 900000 millisecond
    1/18/2020 12:51:47 PM [Thread:5568] [CommandManager:InformationEvent] (Info, Normal): 34891161-998e-415e-bdac-ec0d270c4bb0: Started ProcessCancelMessageAync.
    1/18/2020 12:51:47 PM [Thread:1328] [CommandManager:InformationEvent] (Info, Normal): 34891161-998e-415e-bdac-ec0d270c4bb0: Beginning ProcessCancelMessage.
    1/18/2020 12:51:47 PM [Thread:3580] [AbstractReaderWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 1: RoundRobinBufferReaderWorker -- Rows Read : 0
    1/18/2020 12:51:47 PM [Thread:3580] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 1: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107002:1]: 107002;Blocking queue has been interrupted
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.OneAccessBlockingQueueWrapper`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ConvertingPpaxBlockReader.GetRow()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.ReadFixedWidthFormat(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.IDmsBufferReader.ReadBuffer(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:1988] [AbstractReaderWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 3: RoundRobinBufferReaderWorker -- Rows Read : 0
    1/18/2020 12:51:47 PM [Thread:1988] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 3: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107002:1]: 107002;Blocking queue has been interrupted
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.OneAccessBlockingQueueWrapper`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ConvertingPpaxBlockReader.GetRow()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.ReadFixedWidthFormat(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.IDmsBufferReader.ReadBuffer(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:1480] [AbstractReaderWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 5: RoundRobinBufferReaderWorker -- Rows Read : 0
    1/18/2020 12:51:47 PM [Thread:1480] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 5: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107002:1]: 107002;Blocking queue has been interrupted
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.OneAccessBlockingQueueWrapper`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ConvertingPpaxBlockReader.GetRow()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.ReadFixedWidthFormat(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.IDmsBufferReader.ReadBuffer(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:2848] [AbstractReaderWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 7: RoundRobinBufferReaderWorker -- Rows Read : 0
    1/18/2020 12:51:47 PM [Thread:2848] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 7: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107002:1]: 107002;Blocking queue has been interrupted
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.OneAccessBlockingQueueWrapper`1.Dequeue()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.ConvertingPpaxBlockReader.GetRow()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.ReadFixedWidthFormat(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.ExternalFileConverterBufferReader.Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.IDmsBufferReader.ReadBuffer(MemoryBuffer memBuffer)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:472] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 11: 0 Rows inserted into distribution 3
    1/18/2020 12:51:47 PM [Thread:5724] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 13: 0 Rows inserted into distribution 5
    1/18/2020 12:51:47 PM [Thread:1328] [CommandManager:InformationEvent] (Info, Normal): 34891161-998e-415e-bdac-ec0d270c4bb0: Finished ProcessCancelMessage.
    1/18/2020 12:51:47 PM [Thread:1152] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 2:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 12, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:3896] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 4:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 14, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:5864] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 2: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107001:1]: 107001;Queue has been disposed already
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.ThrowIfDisposed()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinDistributor.SendClose(String planId)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:5052] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 4: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107001:1]: 107001;Queue has been disposed already
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.ThrowIfDisposed()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinDistributor.SendClose(String planId)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:6128] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 6: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107001:1]: 107001;Queue has been disposed already
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.ThrowIfDisposed()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinDistributor.SendClose(String planId)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:564] [AbstractReaderWorker:WarningEvent] (Warning, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 8: Expected exception occurred during plan cancel--
    Microsoft.SqlServer.DataWarehouse.DataMovement.Common.DmsResourceDisposedException[107001:1]: 107001;Queue has been disposed already
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Common.BlockingQueue`1.ThrowIfDisposed()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinDistributor.SendClose(String planId)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.RoundRobinBufferReaderWorker.ReadAndSendData()
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:564] [CommandPlan:InformationEvent] (Info, Normal): Producing a message package with 9 completion messages for planId 34891161-998e-415e-bdac-ec0d270c4bb0 and steps of type Microsoft.SqlServer.DataWarehouse.DataMovement.Common.Plan.ConverterReaderStep
    1/18/2020 12:51:47 PM [Thread:5244] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 9: 0 Rows inserted into distribution 1
    1/18/2020 12:51:47 PM [Thread:4456] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 10: 0 Rows inserted into distribution 2
    1/18/2020 12:51:47 PM [Thread:320] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 12: 0 Rows inserted into distribution 4
    1/18/2020 12:51:47 PM [Thread:2548] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 14: 0 Rows inserted into distribution 6
    1/18/2020 12:51:47 PM [Thread:2184] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 15: 0 Rows inserted into distribution 7
    1/18/2020 12:51:47 PM [Thread:4688] [WriterWorker:InformationEvent] (Info, Normal): QueryId QID4368 PlanId 34891161-998e-415e-bdac-ec0d270c4bb0 StepId 16: 0 Rows inserted into distribution 8
    1/18/2020 12:51:47 PM [Thread:4688] [CommandPlan:InformationEvent] (Info, Normal): Producing a message package with 8 completion messages for planId 34891161-998e-415e-bdac-ec0d270c4bb0 and steps of type Microsoft.SqlServer.DataWarehouse.DataMovement.Common.Plan.SqlTableWriterStep
    1/18/2020 12:51:47 PM [Thread:2800] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 1:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 11, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:2616] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 5:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 15, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:3672] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 3:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 13, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:1436] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 0:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 10, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:4384] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 6:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 16, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:5404] [AbstractReaderWorker:ErrorEvent] (Error, High): QueryId QID4368 PlanId 3758af1e-35da-4e2a-8203-e6737d640d65 StepId 7:
    Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.DmsSqlNativeException: SqlNativeBufferReader.Run, error in OdbcExecuteQuery: SqlState: 42000, NativeError: 8680, 'Error calling: SQLExecDirect(this->GetHstmt(), (SQLWCHAR *)statementText, statementLength), SQL return code: -1 | SQL Error Info: SrvrMsgState: 5, SrvrSeverity: 17,  Error <1>: ErrorMsg: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase. | Error calling: pReadConn->ExecuteQuery(statementText, statementLength, bufferFormat) | state: FFFF, number: 17, active connections: 8', Connection String: Driver={pdwodbc17e};app=RCSmall-DmsNativeReader:WIN-ALO32Q0SFC2\mpdwsvc (5632)-ODBC-PoolId1433;trusted_connection=yes;autotranslate=no;server=\\.\pipe\sql\query
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.ThrowDmsSqlNativeException(String message, NativeOdbcConnection conn)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.DataReader.SqlNativeBufferReader.Run(Object status)
       at Microsoft.SqlServer.DataWarehouse.DataMovement.Workers.AbstractReaderWorker.Execute(Object status)
    1/18/2020 12:51:47 PM [Thread:1152] [DmsCoreInstrumentation:DmsReaderStepErrorEvent] (Error, Critical):

    WIN-ALO32Q0SFC2_Dms_polybase.log

    [2020-01-18 12:50:07,362] [WARN ] [Thread-0] [com.microsoft.polybase.client.HdfsBridge] Log level from core-site.xml configuration file: null
    [2020-01-18 12:51:47,176] [WARN ] [Thread-0] [com.microsoft.polybase.client.HdfsBridgeException] HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
    com.microsoft.polybase.client.HdfsBridgeException: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1494)
    Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743051_2231 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:995)
        at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:638)
        at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:888)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:945)
        at java.io.DataInputStream.read(Unknown Source)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.fillBuffer(UncompressedSplitLineReader.java:62)
        at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.readLine(UncompressedSplitLineReader.java:94)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:144)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:184)
        at com.microsoft.polybase.storage.input.LineRecordReader.nextKeyValue(LineRecordReader.java:144)
        at com.microsoft.polybase.storage.input.LineRecordReaderLineProvider.nextLine(LineRecordReaderLineProvider.java:345)
        at com.microsoft.pp.converter.TextToPaxBlockConverter.readNextTuple(TextToPaxBlockConverter.java:234)
        at com.microsoft.pp.converter.ToPaxBlockConverter.convertNextBlock(ToPaxBlockConverter.java:341)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextKeyValue(ToPaxBlockRecordReader.java:91)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextBlock(ToPaxBlockRecordReader.java:84)
        at com.microsoft.polybase.storage.input.PdwPaxBlockSplitter.fillNextChunk(PdwPaxBlockSplitter.java:129)
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1456)
    [2020-01-18 18:01:08,812] [WARN ] [Thread-8] [com.microsoft.polybase.client.HdfsBridgeException] HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
    com.microsoft.polybase.client.HdfsBridgeException: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1494)
    Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:995)
        at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:638)
        at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:888)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:945)
        at java.io.DataInputStream.read(Unknown Source)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.fillBuffer(UncompressedSplitLineReader.java:62)
        at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.readLine(UncompressedSplitLineReader.java:94)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:144)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:184)
        at com.microsoft.polybase.storage.input.LineRecordReader.nextKeyValue(LineRecordReader.java:144)
        at com.microsoft.polybase.storage.input.LineRecordReaderLineProvider.nextLine(LineRecordReaderLineProvider.java:345)
        at com.microsoft.pp.converter.TextToPaxBlockConverter.readNextTuple(TextToPaxBlockConverter.java:234)
        at com.microsoft.pp.converter.ToPaxBlockConverter.convertNextBlock(ToPaxBlockConverter.java:341)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextKeyValue(ToPaxBlockRecordReader.java:91)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextBlock(ToPaxBlockRecordReader.java:84)
        at com.microsoft.polybase.storage.input.PdwPaxBlockSplitter.fillNextChunk(PdwPaxBlockSplitter.java:129)
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1456)
    [2020-01-18 18:08:04,193] [WARN ] [Thread-13] [com.microsoft.polybase.client.HdfsBridgeException] HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
    com.microsoft.polybase.client.HdfsBridgeException: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1494)
    Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:995)
        at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:638)
        at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:888)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:945)
        at java.io.DataInputStream.read(Unknown Source)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.fillBuffer(UncompressedSplitLineReader.java:62)
        at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.readLine(UncompressedSplitLineReader.java:94)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:144)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:184)
        at com.microsoft.polybase.storage.input.LineRecordReader.nextKeyValue(LineRecordReader.java:144)
        at com.microsoft.polybase.storage.input.LineRecordReaderLineProvider.nextLine(LineRecordReaderLineProvider.java:345)
        at com.microsoft.pp.converter.TextToPaxBlockConverter.readNextTuple(TextToPaxBlockConverter.java:234)
        at com.microsoft.pp.converter.ToPaxBlockConverter.convertNextBlock(ToPaxBlockConverter.java:341)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextKeyValue(ToPaxBlockRecordReader.java:91)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextBlock(ToPaxBlockRecordReader.java:84)
        at com.microsoft.polybase.storage.input.PdwPaxBlockSplitter.fillNextChunk(PdwPaxBlockSplitter.java:129)
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1456)
    [2020-01-19 12:19:05,125] [WARN ] [Thread-0] [com.microsoft.polybase.client.HdfsBridge] Log level from core-site.xml configuration file: null
    [2020-01-19 12:20:54,350] [WARN ] [Thread-0] [com.microsoft.polybase.client.HdfsBridgeException] HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
    com.microsoft.polybase.client.HdfsBridgeException: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1494)
    Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:995)
        at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:638)
        at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:888)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:945)
        at java.io.DataInputStream.read(Unknown Source)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.fillBuffer(UncompressedSplitLineReader.java:62)
        at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.readLine(UncompressedSplitLineReader.java:94)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:144)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:184)
        at com.microsoft.polybase.storage.input.LineRecordReader.nextKeyValue(LineRecordReader.java:144)
        at com.microsoft.polybase.storage.input.LineRecordReaderLineProvider.nextLine(LineRecordReaderLineProvider.java:345)
        at com.microsoft.pp.converter.TextToPaxBlockConverter.readNextTuple(TextToPaxBlockConverter.java:234)
        at com.microsoft.pp.converter.ToPaxBlockConverter.convertNextBlock(ToPaxBlockConverter.java:341)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextKeyValue(ToPaxBlockRecordReader.java:91)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextBlock(ToPaxBlockRecordReader.java:84)
        at com.microsoft.polybase.storage.input.PdwPaxBlockSplitter.fillNextChunk(PdwPaxBlockSplitter.java:129)
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1456)
    [2020-01-19 13:51:24,855] [WARN ] [Thread-8] [com.microsoft.polybase.client.HdfsBridgeException] HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
    com.microsoft.polybase.client.HdfsBridgeException: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1494)
    Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-243674277-172.17.0.2-1529333510191:blk_1073743075_2256 file=/user/raj_ops/TP/Sacramentorealestatetransactions.csv
        at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:995)
        at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:638)
        at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:888)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:945)
        at java.io.DataInputStream.read(Unknown Source)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.fillBuffer(UncompressedSplitLineReader.java:62)
        at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
        at org.apache.hadoop.mapreduce.lib.input.UncompressedSplitLineReader.readLine(UncompressedSplitLineReader.java:94)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:144)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:184)
        at com.microsoft.polybase.storage.input.LineRecordReader.nextKeyValue(LineRecordReader.java:144)
        at com.microsoft.polybase.storage.input.LineRecordReaderLineProvider.nextLine(LineRecordReaderLineProvider.java:345)
        at com.microsoft.pp.converter.TextToPaxBlockConverter.readNextTuple(TextToPaxBlockConverter.java:234)
        at com.microsoft.pp.converter.ToPaxBlockConverter.convertNextBlock(ToPaxBlockConverter.java:341)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextKeyValue(ToPaxBlockRecordReader.java:91)
        at com.microsoft.polybase.storage.input.ToPaxBlockRecordReader.nextBlock(ToPaxBlockRecordReader.java:84)
        at com.microsoft.polybase.storage.input.PdwPaxBlockSplitter.fillNextChunk(PdwPaxBlockSplitter.java:129)
        at com.microsoft.polybase.client.HdfsBridge.recordReaderFillBuffer(HdfsBridge.java:1456)

    Tuesday, January 21, 2020 1:02 PM
  • I am still struggling to get this fixed, appreciate if i get few pointers to look for resolution.
    Friday, January 24, 2020 1:40 AM