locked
CentOS based linux VM running on Hyper-v : Checking root filesystem fails when kernel switches having old PV(para virtualised driver based on 2.6.32 linux kernel) to new PV(which is equivalent to linux integration component 3.4) RRS feed

  • Question

  • hi all,

    I am running a CentOS base VM on top of Hyper-V server. I upgraded PV drivers of Hyper-V in linux kernel 2.6.32 in order to support Windows Server 2012, then i am hitting below issue on Windows Server 2008 when kernel switches from old PV(which is 2.6.32 based) to new PV(which is equivalent to linux integration component 3.4).i am hitting following filesystem check error messages :
    ******************************************************
    Setting hostname hostname:
    Checking root filesystem
    fsck.ext3/dev/hda2:
    The superblock could not be read or does not describe correct ext2 filesystem. If the device is valid and it really contains an ext2 filesystem(and not swap or ufs or something else),then the superblock is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

    : No such file or directory while trying to open /dev/hda2

    *** An error occurred during the filesystem check.
    *** Dropping you to a shell; the system will reboot
    *** When you leave the shell.

    *****************************************************
    Also, when I go to the repair filesystem mode. I found out the strange behaviour when i ran those command :

    ****************************************
    (Repair filesytem) 1 # mount
    /dev/hda2 on / type ext3 (rw)
    proc on /proc type proc (rw)

    (Repair filesystem) 1# cat /etc/mtab
    /dev/hda2 /ext3 rw 0 0
    proc /proc proc rw 0 0

    (Repair filesystem) 1# df
    Filesystem 1K-blocks used Available Use% Mountedon
    /dev/hda2 4%

    ******************************************
    I think for all above command there should be /dev/sda2 instead of /dev/hda2.

    Also my fstab , and fdisk -l looks like ok for me.
    *******************************************
    (Repair filesystem) 1# cat /etc/fstab
    LABEL=/ / ext3 defaults 1 1
    LABEL=/boot /boot ext3 defaults 1 2
    devpts /dev/pts devpts gid=5,mode=620 0 0
    tmpfs /dev/shm tmpfs defaults 0 0
    proc /proc proc defaults 0 0
    sysfs /sys sysfs defaults 0 0
    LABEL=swap-xvda3 swap swap defults 0 0

    (Repair filesystem) 1# fdisk -l
    Device Boot Start End Block Id System
    /dev/sda1 * 1 49 98535 83 Linux
    Partition 1 does not end with cylinder boundary.
    /dev/sda2 49 19197 39062500 83 Linux
    Partition 2 does not end with cylinder boundary.
    /dev/sda3 ......
    Partition 3 does not ......
    /dev/sda4 ......
    Partition 4 does not end ....

    (Repair filesystem) 1# e2label /dev/sda1
    /boot

    (Repair filesystem) 1# e2label /dev/sda2
    /

    (Repair fielsystem) 1# ls /dev/sd*
    /dev/sda /dev/sda1 /dev/sda2 /dev/sda3 /dev/sda4

    (Repair filesyatem) 1# ls /dev/hd*
    ls: /dev/hd*: No such file or directory


    Kindly suggest any configuration of windows server or kernel configs missing or how to resolve this issues

    Many many thanks for your reply.

    thanks & Regards,
    Ujjwal
    Wednesday, December 26, 2012 9:18 AM

All replies

  • Dubl? ( IMHO, best continue in this thread)

     See another 2-3 msg. and my answer in

    http://social.technet.microsoft.com/Forums/en/winserverhyperv/thread/77f54d37-aed3-4b76-8028-5af4032e8ec9

    ==

     there should be /dev/sda2 instead of /dev/hda2.


    ##

    Try this:

    ===
    Disable legacy ATA driver by adding the following to kernel command line in /boot/grub/menu.lst:
    reserve=0x1f0, 0x8
    . (This option reserves this I/O region and prevents ata_piix from loading).
    ==

    .

    But this disable not only IDE disk[s] , but also CD-ROM

    ##

    I back ported Linux Integration Services Version 3.4 for Hyper-V equivalent code from linux kernel 3.5.7 to 2.6.32 linux kernel.

    ##

     IMHO, You forgot backport patches to ata_piix related Hyper-V

    (  in kernel with patches CD-ROM not disabled )

    ##

    ==



    Thursday, December 27, 2012 9:16 AM
  • Hi Victor,

    ##

    reserve=0x1f0, 0x8

    ##

    I tried your suggestion but it didn't work, issue is same as above.

    ##

    Youforgot backport patches to ata_piix related Hyper-V

    ##I already disabled legacy ATA support from kernel configs.

    Any other suggestions?

    Thanks for your valuable reply,

    -Ujjwal

    Monday, December 31, 2012 4:59 AM
  • ~

    b) Youforgot backport patches to ata_piix related Hyper-V

    I already disabled legacy ATA support from kernel configs.

    ~

    ~

    >already disabled legacy ATA support

     I.e.

    CONFIG_BLK_DEV_PIIX=n

    ?

     This good step, but need also apply some pathes to ata_piix.c

    ~

    Any other suggestions?

    Youre ata_piix.c  contain substring "Hyper-V"? or no?

    ~

    As temporary solution, increase on 1-2 Gb size all .vhd connected to IDE bus

    ( but not increase size of partitions inside disks)

    ~

      Q1:

    Output of blkid contain duplicate UUID ?

      Q2:

    Output of

    dmesg | grep ata

    contain substring "Hyper-V" ?

    ~

      Q2b:

    Output of

    dmesg | grep ata

    is diffirent after boot parameter reserve=0x1f0, 0x8

    ? or not?

    ~




    Thursday, January 3, 2013 8:08 AM
  • ""

    already disabled legacy ATA support

     I.e.

    CONFIG_BLK_DEV_PIIX=n

    ""

    in my kernel config CONFIG_IDE is not set and hence CONFIG_BLK_DEV_PIIX

    ""

    Youre ata_piix.c  contain substring "Hyper-V"? or no?

    ""

    ata_piix.c doesn't contain substring Hyper-V.

    ""

    Q1: Output of blkid contain duplicate UUID ?

    ""

    -> blkid contains duplicate UUID, below are the output.




    ##

    Output of dmesg | grep ata contain substring "Hyper-V"

    ##

    it doesn't contain "Hyper-V" or ata related message and the output doesn't change with boot parameter reserve=0x1f0, 0x8

    thanks,

    -ujjwal

    Friday, January 4, 2013 10:41 AM
  • regarding ata_piix.c patch, is it necessary to apply it because in kernel config CONFIG_ATA_SFF is not set and so CONFIG_ATA_PIIX?

    As far as i understand this patch , it ignore ATA devices on Hyper-V when PV drivers(CONFIG_HYPERV_STORAGE=y) are enabled.

    Also, i am not able to understand duplicate UUID and from where it is picking /dev/hda* ?

    -ujjwal

    Friday, January 4, 2013 11:04 AM
  • i am not able to understand duplicate UUID and from where it is picking /dev/hda* ?

    ~

    VVM:>>

    VVM:>> Output of dmesg | grep ata contain substring "Hyper-V" ?

    VVM:>>

    it doesn't contain "Hyper-V" or ata related message and the output doesn't change with boot parameter reserve=0x1f0, 0x8

    ~~

    ~~~~

    ==

     output of dmesg related "ata" Ubuntu v13.04 mini.iso ( with boot parameter reserve=0x1f0, 0x8)

    ==

     see later ( in "good situation" example  )

    ~~

    {

    ===
    Disable legacy ATA driver by adding the following to kernel command line in /boot/grub/menu.lst:
    reserve=0x1f0, 0x8
    . (This option reserves this I/O region and prevents ata_piix from loading).
    ==

     See output of dmesg related "ata" Ubuntu v13.04 mini.iso ( with boot parameter reserve=0x1f0, 0x8) :

    ~~

    [ 0.176027]  libata version 3.00 loaded.

    [ 0.713319]  ata_piix 0000:00:07.1: version 2.13

    [ 0.713397]  ata_piix 0000:00:07.1: device not available (can't reserve [io 0x0000-0x0007])

    [ 0.713404]  ata_piix: probe of 0000:00:07.1 failed with error -22

    [ 0.713474]  pata_acpi 0000:00:07.1: device not available (can't reserve [io 0x0000-0x0007])

    [ 0.713479]  pata_acpi: probe of 0000:00:07.1 failed with error -22

    ~~

      As result: 1) IDE disk handled by hv_storvsc , but 2) no CD-ROM device

    ==

    ~ # blkid
    /dev/sda1: LABEL="ARCH_BOOT" UUID="009c2043-4bl7-4f95-al4d-fb8951f95b5d" TYPE="ext2"

    ==

    ~~

    VVM>>

    VVM>>Q1: Output of blkid contain duplicate UUID ?

    VVM>>

    -> blkid contains duplicate UUID, below are the output.



    ~~

     This situation is classic problem "

    use hv_storvsc instead of ata_piix to handle the IDE disks devices ( but not for the DVD-ROM / CD-ROM device handling)

    "

    ~~

     For compare, see example "good situation": 

     See output of dmesg related "ata" Ubuntu v13.04 mini.iso ( without boot parameter reserve=0x1f0, 0x8) :

    ~~~~

    ~ # dmesg |grep ata

       . . .

    [ 0.167224] libata version 3.00 loaded.

    [ 0.703109] ata_piix 0000:00:07.1: version 2.13

    [ 0.703267] ata_piix 0000:00:07.1: Hyper-V Virtual Machine detected, ATA device ignore set

    [ 0.703339] ata_piix 0000:00:07.1: setting latency timer to 64

    [ 0.704968] scsi0 : ata_piix

    [ 0.705713] scsi1 : ata_piix

    [ 0.706191] atal: PATA max UDMA/33 cmd 0xlf0 ctl 0x3f6 bmdma 0xffa0 irq 14

    [ 0.706194] ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15

    [ 0.868844] atal.00: host indicates ignore ATA devices, ignored

    [ 0.869142] ata2.00: ATAPI: Virtual CD, , max MWDMA2

    [ 0.871736] ata2.00: configured for MWDMA2

      . . .

    ~~~~

    ===
    ...
    ~ # uname -a
    Linux ubuntu 3.7.0-7-generic #15-Ubuntu SUP Sat Dec 15 14:13:08 UTC 2012 x86_64 GNU/Linux
    ~ # lsmod

      . . .

    hv_netvsc 22769 0
    hv_storvsc 17496 3
    hv_utils 13569 0
    hv_vmbus 34432 3 hv_netvsc,hv_storvsc,hv_utils

      . . .

    ~ # blkid
    /dev/sr0: LABEL=”CDROM" TYPE="iso9660”
    /dev/sda1: LABEL="ARCH_BOOT" UUID="009c2043-4bl7-4f95-al4d-fb8951f95b5d" TYPE="ext2"
      . . .
    ...
    ===

     ( only CD-ROM and 1( one) IDE disk connected to ATA)

    ~~

    regarding ata_piix.c patch . . .

    As far as i understand this patch , it ignore ATA devices on Hyper-V when PV drivers(CONFIG_HYPERV_STORAGE=y) are enabled.

    ~~

     Yes:

    ignore ATA-HDD ( but not ignore ATA CD-ROM )  on Hyper-V when PV drivers(CONFIG_HYPERV_STORAGE=y) are enabled.

    ~

     this patches need be backported:

    {
      cd006086fa5d ata_piix: defer disks to the Hyper-V drivers by default

    and its prerequisite

      db63a4c8115a libata: add a host flag to ignore detected ATA devices
    }

    ~

    ~~

    .

    .

    .

    .

    .

    P.S.

     Are You do this:

    ==

    As temporary solution, increase on 1-2 Gb size all .vhd connected to IDE bus

    ( but not increase size of partitions inside disks)

    ==

    ? fsck write message a-la: "no error in file system" ?

    .

    2013-01-24 Answer by Ujjwal Kumar: As a temporary solution looks ok for me, but [ VVM: need true solution ]

    .

    {

    P.P.S.

    To Ujjwal Kumar :

     My e-mail:

    ZZZZZZZZZZZZZZZ

    please send e-mail to me,  in reply I send to You patches to ata_piix ( and *.c before and after patches) , etc.

    } on 2013-01-14 -- DoNe



    • Edited by Victor Miasnikov Thursday, January 24, 2013 8:01 AM increase on 1-2 Gb size all .vhd connected to IDE bus => Answer by Ujjwal Kumar: As a temporary solution looks ok for me
    Saturday, January 5, 2013 9:34 AM
  • Hi Victor,

    I am running a custom Linux on Hyper-V 2012 and I am also facing an issue in kdump.

    VVM:>>

    VVM:>> Output of dmesg | grep ata contain substring "Hyper-V" ?

    VVM:>>

    it doesn't contain "Hyper-V" or ata related message and the output doesn't change with boot parameter reserve=0x1f0, 0x8

    >>>> It doesn't contain 'Hyper-V' in ata.

    [ 0.167224] libata version 3.00 loaded.

    [ 0.703109] ata_piix 0000:00:07.1: version 2.13

    [ 0.703267] ata_piix 0000:00:07.1: Hyper-V Virtual Machine detected, ATA device ignore set

    [ 0.703339] ata_piix 0000:00:07.1: setting latency timer to 64

    [ 0.704968] scsi0 : ata_piix

    [ 0.705713] scsi1 : ata_piix

    [ 0.706191] atal: PATA max UDMA/33 cmd 0xlf0 ctl 0x3f6 bmdma 0xffa0 irq 14

    [ 0.706194] ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15

    [ 0.868844] atal.00: host indicates ignore ATA devices, ignored

    [ 0.869142] ata2.00: ATAPI: Virtual CD, , max MWDMA2

    [ 0.871736] ata2.00: configured for MWDMA2

    >>>> Thanks for these snapshots:-)

    ===
    ...
    ~ # uname -a
    Linux ubuntu 3.7.0-7-generic #15-Ubuntu SUP Sat Dec 15 14:13:08 UTC 2012 x86_64 GNU/Linux

    please send e-mail to me,  in reply I send to You patches to ata_piix ( and *.c before and after patches) , etc.

    } on 2013-01-14 -- DoNe



    >>>> Can you send me those patches for ata_piix.c code ?

    Thanks in advance,

    Saleem


    • Edited by A Saleem Wednesday, January 29, 2014 12:39 PM
    Wednesday, January 29, 2014 12:37 PM
  • >>>> Can you send me those patches for ata_piix.c code ?

     

     IMHO, since 2013-01-05 problem fixed

    all Linux Kernel 3.4 . .  3.12 . . 3.13 contain all what need for solve this problem


    Wednesday, January 29, 2014 12:53 PM
  •  IMHO, since 2013-01-05 problem fixed

    all Linux Kernel 3.4 . .  3.12 . . 3.13 contain all what need for solve this problem

    Thanks for the response.

    But I am running 2.6.32 kernel. Can I get those patches in 3.4 related to Hyper-V?


    Thanks,

    Saleem


    Wednesday, January 29, 2014 2:00 PM
  • ~

    ~

    VVM>> IMHO, since 2013-01-05 problem fixed

    VVM>>all Linux Kernel 3.4 . .  3.12 . . 3.13 contain all what need for solve this problem

    But I am running 2.6.32 kernel. Can I get those patches in 3.4 related to Hyper-V?


    ~

      First 1 . . 2 :

    libata: add a host flag to ignore detected ATA devices

    ata_piix: defer disks to the Hyper-V drivers by default

    ==

    From cd006086fa5d91414d8ff9ff2b78fbb593878e3c Mon Sep 17 00:00:00 2001
    From: Andy Whitcroft <apw (at) canonical.com>
    Date: Fri, 4 May 2012 22:15:11 +0100
    Subject: [PATCH 1/1] ata_piix: defer disks to the Hyper-V drivers by default

    When we are hosted on a Microsoft Hyper-V hypervisor the guest disks
    are exposed both via the Hyper-V paravirtualised drivers and via an
    emulated SATA disk drive.  In this case we want to use the paravirtualised
    drivers if we can as they are much more efficient.  Note that the Hyper-V
    paravirtualised drivers only expose the virtual hard disk devices, the
    CDROM/DVD devices must still be enumerated.

    Mark the host controller ATA_HOST_IGNORE_ATA to prevent enumeration of
    disk devices.

    BugLink: http://bugs.launchpad.net/bugs/929545
    BugLink: http://bugs.launchpad.net/bugs/942316
    Signed-off-by: Andy Whitcroft

    Signed-off-by: Jeff Garzik

    ---
     drivers/ata/ata_piix.c |   36 ++++++++++++++++++++++++++++++++++++
     1 files changed, 36 insertions(+), 0 deletions(-)

    diff --git a/drivers/ata/ata_piix.c b/drivers/ata/ata_piix.c
    index 7857e8f..3c809bf 100644
    --- a/drivers/ata/ata_piix.c
    +++ b/drivers/ata/ata_piix.c
    @@ -1554,6 +1554,39 @@ static bool piix_broken_system_poweroff(struct pci_dev *pdev)
      return false;
     }
     
    +static int prefer_ms_hyperv = 1;
    +module_param(prefer_ms_hyperv, int, 0);
    +
    +static void piix_ignore_devices_quirk(struct ata_host *host)
    +{
    +#if IS_ENABLED(CONFIG_HYPERV_STORAGE)
    + static const struct dmi_system_id ignore_hyperv[] = {
    +  {
    +   /* On Hyper-V hypervisors the disks are exposed on
    +    * both the emulated SATA controller and on the
    +    * paravirtualised drivers.  The CD/DVD devices
    +    * are only exposed on the emulated controller.
    +    * Request we ignore ATA devices on this host.
    +    */
    +   .ident = "Hyper-V Virtual Machine",
    +   .matches = {
    +    DMI_MATCH(DMI_SYS_VENDOR,
    +      "Microsoft Corporation"),
    +    DMI_MATCH(DMI_PRODUCT_NAME, "Virtual Machine"),
    +   },
    +  },
    +  { } /* terminate list */
    + };
    + const struct dmi_system_id *dmi = dmi_first_match(ignore_hyperv);
    +
    + if (dmi && prefer_ms_hyperv) {
    +  host->flags |= ATA_HOST_IGNORE_ATA;
    +  dev_info(host->dev, "%s detected, ATA device ignore set\n",
    +   dmi->ident);
    + }
    +#endif
    +}
    +
     /**
      * piix_init_one - Register PIIX ATA PCI device with kernel services
      * @pdev: PCI device to register
    @@ -1669,6 +1702,9 @@ static int __devinit piix_init_one(struct pci_dev *pdev,
      }
      host->flags |= ATA_HOST_PARALLEL_SCAN;
     
    + /* Allow hosts to specify device types to ignore when scanning. */
    + piix_ignore_devices_quirk(host);
    +
      pci_set_master(pdev);
      return ata_pci_sff_activate_host(host, ata_bmdma_interrupt, sht);
     }
    --
    1.7.6.5

    ==

    ~

    ~

    ==

    From db63a4c8115a0bb904496e1cdd3e7488e68b0d06 Mon Sep 17 00:00:00 2001
    From: Andy Whitcroft <apw ( at ) canonical.com>
    Date: Fri, 4 May 2012 22:15:10 +0100
    Subject: [PATCH] libata: add a host flag to ignore detected ATA devices

    Where devices are visible via more than one host we sometimes wish to
    indicate that cirtain devices should be ignored on a specific host.  Add a
    host flag indicating that this host wishes to ignore ATA specific devices.

    Signed-off-by: Andy Whitcroft

    Signed-off-by: Jeff Garzik

    ---
     drivers/ata/libata-core.c |    6 ++++++
     include/linux/libata.h    |    1 +
     2 files changed, 7 insertions(+), 0 deletions(-)

    diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
    index 23763a1..d31ee55 100644
    --- a/drivers/ata/libata-core.c
    +++ b/drivers/ata/libata-core.c
    @@ -1973,6 +1973,12 @@ retry:
      if (class == ATA_DEV_ATA) {
       if (!ata_id_is_ata(id) && !ata_id_is_cfa(id))
        goto err_out;
    +  if (ap->host->flags & ATA_HOST_IGNORE_ATA &&
    +       ata_id_is_ata(id)) {
    +   ata_dev_dbg(dev,
    +    "host indicates ignore ATA devices, ignored\n");
    +   return -ENOENT;
    +  }
      } else {
       if (ata_id_is_ata(id))
        goto err_out;
    diff --git a/include/linux/libata.h b/include/linux/libata.h
    index e926df7..6e887c7 100644
    --- a/include/linux/libata.h
    +++ b/include/linux/libata.h
    @@ -247,6 +247,7 @@ enum {
      ATA_HOST_SIMPLEX = (1 << 0), /* Host is simplex, one DMA channel per host only */
      ATA_HOST_STARTED = (1 << 1), /* Host started */
      ATA_HOST_PARALLEL_SCAN = (1 << 2), /* Ports on this host can be scanned in parallel */
    + ATA_HOST_IGNORE_ATA = (1 << 3), /* Ignore ATA devices on this host. */
     
      /* bits 24:31 of host->flags are reserved for LLD specific flags */
     
    --
    1.7.7.6

    ==

    ~

    ~

    ~

    ~

    P.S.

      On 2013-02-27

     File  \linux-2.6.32-358.0.1.el6\drivers\ata\ata_piix.c

    from

    http://vault.centos.org/6.3/cr/Source/SPackages/
    ===
    ...
    kernel-2.6.32-358.0.1.el6.src.rpm 27-Feb-2013 11:10 
    ...
    ===

     contain

    ==

    static void piix_ignore_devices_quirk(struct ata_host *host)
    {
    #if defined(CONFIG_HYPERV_STORAGE) || defined(CONFIG_HYPERV_STORAGE_MODULE)
     static const struct dmi_system_id ignore_hyperv[] = {
      {
       /* On Hyper-V hypervisors the disks are exposed on
        * both the emulated SATA controller and on the
        * paravirtualised drivers.  The CD/DVD devices
        * are only exposed on the emulated controller.
        * Request we ignore ATA devices on this host.
        */
       .ident = "Hyper-V Virtual Machine",
       .matches = {
        DMI_MATCH(DMI_SYS_VENDOR,
          "Microsoft Corporation"),
        DMI_MATCH(DMI_PRODUCT_NAME, "Virtual Machine"),
       },
      },
      { } /* terminate list */
     };
     const struct dmi_system_id *dmi = dmi_first_match(ignore_hyperv);

     if (dmi && prefer_ms_hyperv) {
      host->flags |= ATA_HOST_IGNORE_ATA;
      dev_info(host->dev, "%s detected, ATA device ignore set\n",
       dmi->ident);
     }
    #endif
    }

    ==

    ~

    ~

     I.e. problem solved since RHEL v6.4 and for .el6 kernel 2.6.32

    Wednesday, January 29, 2014 3:38 PM