HDInsight Hive error: Unable to rename

HDInsight Hive error: Unable to rename

When you use Azure PowerShell to run HiveQL statements against HDInsight or HDInsight emulator, you might get the following error:

Failed with exception Unable to rename: 
wasb://xxxxxxx@xxxxxxxx.blob.core.windows.net/hive/scratch/hive_2014-02-12_14-58-27_594_640725686036024640-1/-ext-10001
to: wasb://xxxxxxxx@xxxxxxxx.blob.core.windows.net/apps/hive/warehouse/delays FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

Creating a /apps/hive/warehouse folder on HDInsight can bypass this error message, but could run into another problem. You might notice a newly created Hive table doesn't contain any records. In the error log, you could find the data file was moved to the /apps/hive/warehouse folder instead of the /hive/warehouse folder.

This problem was fixed in HDInsight clusters version 3.1 and higher, with the exception of HBase deployments (fix in the works).

To automate this fix where needed:
You can use PowerShell to deploy HDInsight following this blog article and this documentation with a custom configuration for the Hadoop Core settings, to include this workaround each time you create a cluster from PowerShell scripts.

These are the two key syntax lines to use with a code sample.

# Customize Core configuration options
$CoreConfigValues = @{ "templeton.hive.properties"="hive.metastore.local=false,hive.metastore.uris=thrift://headnodehost:9083,hive.metastore.warehouse.dir=/hive/warehouse/" }

...
| Add-AzureHDInsightConfigValues -Core $CoreConfigValues
...

This ends up putting the value in this core-site.xml file on the nodes:
"C:\apps\dist\hadoop-2.4.0.2.1.6.0-2103\etc\hadoop\core-site.xml"

  <property>
    <name>templeton.hive.properties</name>
<value>hive.metastore.local=false,hive.metastore.uris=thrift://headnodehost:9083,hive.metastore.warehouse.dir=/hive/warehouse/</value>
  </property>

To fix the problem on HDInsight cluster version 3.0:

  1. Connect to the cluster head node using RDP. For instructions, see Administer HDInsight clusters using Management portal.
  2. Use a text editor to open C:\apps\dist\hcatalog-0.12.0.2.0.7.0-1528\etc\webhcat\webhcat-site.xml.
  3. Find the following tags:
    <property>
        <name>templeton.hive.properties</name>
        <value>hive.metastore.local=false,hive.metastore.uris=thrift://headnodehost:9083,hive.metastore.warehouse.dir=/apps/hive/warehouse</value>
        <description>Properties to set when running hive.</description>
    </property>
  4. Replace /apps/hive/warehouse to /hive/warehouse.
  5. Save the file.
  6. Open Service from Administrative Tools, and restart Apache Hadoop templeton.

To fix the problem on HDInsight cluster version 2.1:

  1. Connect to the cluster head node using RDP. For instructions, see Administer HDInsight clusters using Management portal.
  2. Use a text editor to open C:\apps\dist\hcatalog-0.11.0.1.3.2.0-05\conf\webhcat-site.xml.
  3. Find the following tags:
    <property>
        <name>templeton.hive.properties</name>
        <value>hive.metastore.local=false,hive.metastore.uris=thrift://headnodehost:9083,hive.metastore.warehouse.dir=/apps/hive/warehouse</value>
        <description>Properties to set when running hive.</description>
    </property>
  4. Replace /apps/hive/warehouse to /hive/warehouse.
  5. Save the file.
  6. Open Service from Administrative Tools, and restart Apache Hadoop templeton.

To fix the problem on HDInsight emulator:

  1. Use a text editor to open C:\Hadoop\hive-0.9.0\conf\hive-site.xml.
  2. Find the following tags:
    <property>
      <name>hive.metastore.warehouse.dir</name>
      <value>/apps/hive/warehouse</value>
      <description>location of default database for the warehouse</description>
    </property>
  3. Replace /apps/hive/warehouse to /hive/warehouse.
  4. Save the file.
  5. Open Service from Administrative Tools, and restart Apache Hadoop templeton.
Sort by: Published Date | Most Recent | Most Useful
Comments
  • Why the Templeton service appears disabled? Cannot restart it to make it take the new configuration :(

  • Hi Francisco, There are two head nodes in HDInsight running in Microsoft Azure. Netbios name is Headnode0 and Headnode1. The core hadoop services may be disabled on one of the headnodes, since the services are running on the opposite head node and should only run on one computer at a time.

    You may RDP into the first one headnode0 from the Azure portal website, and then start another RDP from that desktop into a computer named Headnode1 to make the adjustment and restart the Templeton services.

Page 1 of 1 (2 items)