none
Newbie in PowerShell needs help with a script to copy directories content and ensure nothing is lost. RRS feed

  • Question

  • Hi, I'm new to MS PowerShell but do have a little knowledge with scripting for VBA and R. I have over the years allowed my personal folder with songs get really messy and due to different machines I had similar folder names with different files.

    I have this setup in a NAS which I have 16 TB of space, so I decided to copy all the original folders and files which I mapped as a Z drive in my network, to copy them to this other folder in the NAS called Y.

    When I ran the script I for the 2019 folder which was the smallest of the  3 major ones, everything appeared to copy correctly as seen on the screenshot. 

    But when I tried to copy folders from Z: "iTunes" (285gb - 16,442 files - 12,641 folders ) and "iTunes 2" (606 gb - 57,126 files)

    I got on my Y drive the following: "iTunes" (284gb - 16,309 files - 12,557 folders ) and "iTunes 2" - I manually stopped in the middle and deleted folder.

    I noticed that I was losing files which is the last thing I want since I have been building my music library for 20 years now almost.

    My goal is to successfully copy these folders so that then I can attempt a script that merges the Music folder in each itunes and checks if that folder with same name exists, if it does just merge them, if not copy it with a different name and maybe list out what these are so I can then check these conflicts and help organize even further.


    # I have a NAS with static IP, which I mapped two network drive folders in the NAS e to two letters: 
    # Z (Contains Original Files don't want to lose) & 
    # Y (copy to try and organize with this script)
    
    # Quick look at the z directory I want to copy all folders
    # and subfolders to another already mapped 
    # location Y in the same NAS.
    
    cd z:
    ls
    
        Directory: Z:\
    
    
    Mode                LastWriteTime         Length Name                                                                                                                                                                                
    ----                -------------         ------ ----                                                                                                                                                                                
    d-----        4/26/2020   9:40 PM                2019                                                                                                                                                                                
    d-----       10/31/2019   5:13 PM                iTunes                                                                                                                                                                              
    d-----         3/7/2020   9:51 PM                iTunes 2                                                                                                                                                                            
    d-----         3/7/2020   9:50 PM                Lib. Backup                                                                                                                                                                         
    ------        4/26/2020   9:16 PM             28 Hello.txt      
    
    
    # Begin Copying first folder (285 GB)
    Copy-Item z:\iTunes -Destination: Y: -recurse
    
    # Begin Copying second folder (600+ GB)
    Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    
    # Small folder with backup of itunes files (few MB)
    Copy-Item 'z:\Lib. Backup' -Destination: Y: -recurse
    

    One thing that I noticed was that I was getting these warnings from the console but I have no idea what is actually happening to try and trouble shoot to guarantee that copies are made and transferred.

    # List is huge and has all sort of files, these seem to be just URL's but there are many mp3 on the WriteError:

    as well

    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo : WriteError: (00-safeword-you...d)-web-2011.nfo:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (00-safeword-you...d)-web-2011.sfv:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand Copy-Item : The filename, directory name, or volume label syntax is incorrect. At line:2 char:1 + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : WriteError: (www.minitech.ws...imal & more.url:FileInfo) [Copy-Item], IOException + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand

    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (09 Track_9.mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (1-09 Eyes Wide Open (Original Mix).mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (10 Track_10.mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (17 Frank Sinatr...e Christmas.mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (17 Track_17.mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (18 Track_18.mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (25 Monayears Mi... Ole Biege).mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : WriteError: (A Call From Wit...rich Remix).mp3:FileInfo) [Copy-Item], IOException
        + FullyQualifiedErrorId : CopyDirectoryInfoItemIOError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : The filename, directory name, or volume label syntax is incorrect.
    At line:1 char:1
    + Copy-Item 'z:\iTunes 2' -Destination: Y: -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


    Can anyone try and help with with tackling not only this issue of not having everything copied but also how to have them also merge while checking if the folder or file already exists, merge folders (content) with same name. And some error checking to see file x.mp3 already existed with exact name and wasn't copied while folders with same name just have their content merged and if the content inside them both also match generate notification/ error check to later review after transfer is complete.

    This is entirely for me to learn how to use PowerShell (which seems the tool for this) for a personal project and would greatly appreciate help/ mentoring. :) 

    Thank you for your time & assistance.


    • Edited by technobrat Tuesday, April 28, 2020 9:23 AM
    Tuesday, April 28, 2020 9:13 AM

All replies

  • You are not specifying any actual "Items" in your Copy-Item cmdlet which requires to know which file items you want it to address...i.e...

    Copy-Item z:\iTunes

    Should be...

    Copy-Item z:\iTunes\*

    Your syntax is better suited to ROBOCOPY.

    Also I prefer to work with PSDrives in powershell NOT use windows mapped drives in PS scripting...or better still if the user account has the required permissions to the mapped drive location use the full UNC path instead of mapped drives in the PS script.

    • Edited by durrie Tuesday, April 28, 2020 11:02 AM
    Tuesday, April 28, 2020 10:56 AM
  • You are not specifying any actual "Items" in your Copy-Item cmdlet which requires to know which file items you want it to address...i.e...

    Copy-Item z:\iTunes

    Should be...

    Copy-Item z:\iTunes\*

    Your syntax is better suited to ROBOCOPY.

    Also I prefer to work with PSDrives in powershell NOT use windows mapped drives in PS scripting...or better still if the user account has the required permissions to the mapped drive location use the full UNC path instead of mapped drives in the PS script.

    Thank you very much for the reply!

    I'll be trying the simple copy portion as you described. While I do that (takes hours due to size).

    I chose the mapped drives because it was the way I found how to use PowerShell and access the NAS, I have no idea what UNC path or PSDrives are... I'm not a business IT guy, I'm just trying to find smart solutions to my problems and since I know a bit of coding for Data Science, I'm giving this a try.

    To do what I decrived above, do I have to worry about the UNC path or PSDrives?  I still have the second part which will be merging these files and handle same name folders and merge their contents and FILES if the names match exactly and so does file size, no need to merge but output to a txt file for me to check later.

    Could you please help me a little further or at least point me on the right direction please?

    Moreover if I understood correctly, the "*" is like select all in SQL?

    Thank you for your time and assistance!!

    TB.

    Tuesday, April 28, 2020 11:48 AM
  • You are not specifying any actual "Items" in your Copy-Item cmdlet which requires to know which file items you want it to address...i.e...

    Copy-Item z:\iTunes

    Should be...

    Copy-Item z:\iTunes\*

    Your syntax is better suited to ROBOCOPY.

    Also I prefer to work with PSDrives in powershell NOT use windows mapped drives in PS scripting...or better still if the user account has the required permissions to the mapped drive location use the full UNC path instead of mapped drives in the PS script.

    So I tried your suggestion but to get my effect I had to do the following (it is still running but I had to stop the script delete the files that were being transferred again. 

    Why is it that you believe that doing this way is better:

    
    
    
    # Begin Copying first folder (285 GB)
    # Had to create the same iTunes folder in Y since the below 
    # is copying the content inside iTunes and not the file.
    Copy-Item 'z:\iTunes\*' -Destination: 'Y:\iTunes' -recurse
    
    # Begin Copying second folder (600+ GB)
    # Same as above.
    Copy-Item 'z:\iTunes 2\*' -Destination: 'Y:\iTunes 2' -recurse

    So to ensure copy I need to specify the actual "Items", before I thought the way I had done it I was copying the entire folder and its contents.

    # Begin Copying first folder (285 GB)
    # '\\MYCLOUDEX2ULTRA\Music\iTunes' is the original directory (fixed IP NAS)
    # Network mapped to Z:
    
    # '\\MYCLOUDEX2ULTRA\Entertainment' is the same NAS 
    # Network mapped to Y:
    
    
    # How I thought it would go: 
    Copy-Item z:\iTunes -Destination: Y: -recurse
    
    # You suggested, I modified to place in same folder:
    Copy-Item 'z:\iTunes' -Destination: 'Y:\iTunes' -recurse
    
    # This is what I got from the console, it definidamente did not copy all items and subfolders. It copied some other stuff.
    
    # Code ran for about 5 min.
    
    #Console Output:
    
    
    PS Z:\> # Begin Copying first folder (285 GB)
    Copy-Item 'z:\iTunes\*' -Destination: 'Y:\iTunes' -recurse
    
    # Begin Copying second folder (600+ GB)
    Copy-Item 'z:\iTunes 2\*' -Destination: 'Y:\iTunes 2' -recurse
    Copy-Item : Container cannot be copied onto existing leaf item.
    At line:2 char:1
    + Copy-Item 'z:\iTunes\*' -Destination: 'Y:\iTunes' -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (Z:\iTunes\iTunes Media:String) [Copy-Item], PSArgumentException
        + FullyQualifiedErrorId : CopyContainerItemToLeafError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : Container cannot be copied onto existing leaf item.
    At line:2 char:1
    + Copy-Item 'z:\iTunes\*' -Destination: 'Y:\iTunes' -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (Z:\iTunes\Previous iTunes Libraries:String) [Copy-Item], PSArgumentException
        + FullyQualifiedErrorId : CopyContainerItemToLeafError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : Container cannot be copied onto existing leaf item.
    At line:5 char:1
    + Copy-Item 'z:\iTunes 2\*' -Destination: 'Y:\iTunes 2' -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (Z:\iTunes 2\iTunes Music:String) [Copy-Item], PSArgumentException
        + FullyQualifiedErrorId : CopyContainerItemToLeafError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : Container cannot be copied onto existing leaf item.
    At line:5 char:1
    + Copy-Item 'z:\iTunes 2\*' -Destination: 'Y:\iTunes 2' -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (Z:\iTunes 2\Mobile Applications:String) [Copy-Item], PSArgumentException
        + FullyQualifiedErrorId : CopyContainerItemToLeafError,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Copy-Item : Container cannot be copied onto existing leaf item.
    At line:5 char:1
    + Copy-Item 'z:\iTunes 2\*' -Destination: 'Y:\iTunes 2' -recurse
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (Z:\iTunes 2\Previous iTunes Libraries:String) [Copy-Item], PSArgumentException
        + FullyQualifiedErrorId : CopyContainerItemToLeafError,Microsoft.PowerShell.Commands.CopyItemCommand
    
    
    
    

    Here I try and show that using your method, different things were copied, that weren't in the folder (viewable).

    PS Y:\iTunes> ls
    
    
        Directory: Y:\iTunes
    
    
    Mode                LastWriteTime         Length Name                                                                                                                                                                                
    ----                -------------         ------ ----                                                                                                                                                                                
    d-----        4/28/2020   5:04 AM                Cache                                                                                                                                                                               
    d-----        4/28/2020   5:05 AM                Cloud                                                                                                                                                                               
    d-----        4/28/2020   5:05 AM                Cloud Purchases                                                                                                                                                                     
    d-----        4/28/2020   5:05 AM                Custom                                                                                                                                                                              
    d-----        4/28/2020   5:05 AM                Download                                                                                                                                                                            
    d-----        4/28/2020   5:05 AM                Generated                                                                                                                                                                           
    d-----        4/28/2020   5:05 AM                Remote                                                                                                                                                                              
    d-----        4/28/2020   5:05 AM                Store                                                                                                                                                                               
    ------       10/14/2019   4:54 PM        2140712 iTunes Library (Damaged)                                                                                                                                                            
    ------       10/31/2019   5:03 PM          16384 iTunes Library Extras.itdb                                                                                                                                                          
    ------       10/15/2019   1:04 AM          32768 iTunes Library Genius.itdb                                                                                                                                                          
    ------       10/31/2019   5:13 PM          83251 iTunes Library.itl                                                                                                                                                                  
    ------       10/31/2019   5:16 PM              8 sentinel                                                                                                                                                                            
    
    
    
     
    
    PS Y:\iTunes> cd Z:\iTunes
    
    PS Z:\iTunes> ls
    
    
        Directory: Z:\iTunes
    
    
    Mode                LastWriteTime         Length Name                                                                                                                                                                                
    ----                -------------         ------ ----                                                                                                                                                                                
    d-----         2/4/2018   8:14 AM                Album Artwork                                                                                                                                                                       
    d-----         8/6/2017   9:26 AM                iTunes Media                                                                                                                                                                        
    d-----       12/11/2018   6:55 PM                Previous iTunes Libraries                                                                                                                                                           
    ------       10/14/2019   4:54 PM        2140712 iTunes Library (Damaged)                                                                                                                                                            
    ------       10/31/2019   5:03 PM          16384 iTunes Library Extras.itdb                                                                                                                                                          
    ------       10/15/2019   1:04 AM          32768 iTunes Library Genius.itdb                                                                                                                                                          
    ------       10/31/2019   5:13 PM          83251 iTunes Library.itl                                                                                                                                                                  
    ------       10/31/2019   5:16 PM              8 sentinel                                                                                                                                                                            
    
    
    
    PS Z:\iTunes> 

    They don't match... Any ideas of what I could be doing incorrectly?

    Thank you!

    Tuesday, April 28, 2020 12:28 PM
  • I tried Robocopy:

    # Below allows script execution.
    # set-executionpolicy remotesigned
    $SourcePath = '\\MYCLOUDEX2ULTRA\Music'
    $DestinationPath = '\\MYCLOUDEX2ULTRA\Entertainment'
    $LogFile = "C:\Users\luisf\Downloads\Robocopy_mp3.txt"
    Robocopy $SourcePath $DestinationPath /E /ZB /DCOPY:DAT /COPYALL /R:5 /W:10 /LOG:$LogFile

    Log files said:

    -------------------------------------------------------------------------------
       ROBOCOPY     ::     Robust File Copy for Windows                              
    -------------------------------------------------------------------------------
    
      Started : Tuesday, April 28, 2020 5:56:56 AM
       Source : \\MYCLOUDEX2ULTRA\Music\
         Dest : \\MYCLOUDEX2ULTRA\Entertainment\
    
        Files : *.*
    	    
      Options : *.* /S /E /COPYALL /ZB /R:5 /W:10 
    
    ------------------------------------------------------------------------------
    
    	                   2	\\MYCLOUDEX2ULTRA\Music\
    2020/04/28 05:56:56 ERROR 1314 (0x00000522) Copying NTFS Security to Destination Directory \\MYCLOUDEX2ULTRA\Entertainment\
    A required privilege is not held by the client.
    
    
    
    ------------------------------------------------------------------------------
    
                   Total    Copied   Skipped  Mismatch    FAILED    Extras
        Dirs :         1         0         1         0         0         0
       Files :         0         0         0         0         0         0
       Bytes :         0         0         0         0         0         0
       Times :   0:00:00   0:00:00                       0:00:00   0:00:00
       Ended : Tuesday, April 28, 2020 5:56:56 AM

    Apparently there was a privilege issue, this is a local network, how do change these permissions?

    Tuesday, April 28, 2020 1:03 PM
  • this is a local network, how do change these permissions?

    You don't. Remove the /COPYALL switch. That error is for trying to copy auditing permisissions.  

    In a business environment where you have multiple users accessing the share, you would want to copy the permissions so that the defined security would be retained. But that only really works for Active Directory groups.  For non-AD (local) groups, robocopy will copy the SID, but it won't create the groups or users on the destination system.  

    I save my pictures, music, and videos in a folder named Multimedia. I replicate it between my laptop, my desktop, and an external drive. I use the /mir switch to tell robocopy to mirror the folders. That way when I delete a file/folder or rename something, the "extra" files get deleted.

    I also run it with the /l switch first to see what robocopy is going to do before I have it do the actual copy.   

    robocopy c:\Multimedia\ \\bigblue\multimedia\ /mir /l
     

     



    • Edited by MotoX80 Tuesday, April 28, 2020 4:00 PM
    Tuesday, April 28, 2020 2:12 PM
  • this is a local network, how do change these permissions?

    You don't. Remove the /COPYALL switch. That error is for trying to copy auditing permisissions.  

    In a business environment where you have multiple users accessing the share, you would want to copy the permissions so that the defined security would be retained. But that only really works for Active Directory groups.  For non-AD (local) groups, robocopy will copy the SID, but it won't create the groups or users on the destination system.  

    I save my pictures, music, and videos in a folder named Multimedia. I replicate it between my laptop, my desktop, and an external drive. I use the /mir switch to tell robocopy to mirror the folders. That way when I delete a file/folder or rename something, the "extra" files get deleted.

    I also run it with the /l switch first to see what robocopy is going to do before I have it do the actual copy.   

    robocopy c:\Multimedia\ \\bigblue\multimedia\ /mir /l
     

     



    Thank you for your reply!

    I tried this, removing the /COPYALL :

    # Below allows script execution.
    # set-executionpolicy remotesigned
    $SourcePath = '\\MYCLOUDEX2ULTRA\Music'
    $DestinationPath = '\\MYCLOUDEX2ULTRA\Entertainment'
    $LogFile = "C:\Users\luisf\Downloads\Robocopy_mp3.txt"
    Robocopy $SourcePath $DestinationPath /E /ZB /DCOPY:DAT /R:5 /W:10 /LOG:$LogFile
    And it did not work, still "Access Denied".

    Console output:

    -------------------------------------------------------------------------------
       ROBOCOPY     ::     Robust File Copy for Windows                              
    -------------------------------------------------------------------------------
    
      Started : Tuesday, April 28, 2020 9:37:10 AM
       Source : \\MYCLOUDEX2ULTRA\Music\
         Dest : \\MYCLOUDEX2ULTRA\Entertainment\
    
        Files : *.*
    	    
      Options : *.* /S /E /DCOPY:DAT /COPY:DAT /ZB /R:5 /W:10 
    
    ------------------------------------------------------------------------------
    
    	                   2	\\MYCLOUDEX2ULTRA\Music\
    2020/04/28 09:37:10 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/04/28 09:37:20 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/04/28 09:37:30 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/04/28 09:37:40 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/04/28 09:37:50 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/04/28 09:38:00 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    
    ERROR: RETRY LIMIT EXCEEDED.
    
    
    ------------------------------------------------------------------------------
    
                   Total    Copied   Skipped  Mismatch    FAILED    Extras
        Dirs :         1         0         1         0         0         0
       Files :         0         0         0         0         0         0
       Bytes :         0         0         0         0         0         0
       Times :   0:00:50   0:00:00                       0:00:50   0:00:00
       Ended : Tuesday, April 28, 2020 9:38:00 AM
    
    

    Sorry, this is the first time I'm doing this so any guidance would be much appreciated.

    Ty.

    Tuesday, April 28, 2020 4:42 PM
  • How did you configure MYCLOUDEX2ULTRA? How does it implement file security? Is the file system NTFS, exFAT or FAT32?

    Right click a folder and select properties. Then look at the security tab. What account(s) do you see? Did you have to define an account to that device and then map a drive with it?

    Can you right click a file/folder and change it's read-only and hidden attributes?

    Will robocopy run if you remove the "/DCOPY:DAT /COPY:DAT" switches?

    Tuesday, April 28, 2020 7:13 PM
  • My MYCLOUDEX2ULTRA is a NAS setup with a fixed IP. I also network mapped the folders from the drive that I used such as the ones I want to manipulate. The drives are NTFS formatted.

    Was going to post images but I can't yet due to some verification restriction...

    Both folders Source and Destination both have the below: (note I never changed the default configs of my NAS other than creating 2 users with different accesses to folders which I have access to all, the other person is for my sister when I want to share stuff or she wants to store things there (her own folder in the NAS)

    Under Group or Usernames:

    Everyone

    root(unix user\root)

    root(unix group\root)



    "Did you have to define an account to that device and then map a drive with it?" - I left the "Reconnect at sign-in" option ticked.

    On windows explorer I navigate without having to put the password after a first time.

    MyCloud setup is pretty straight forward, was going to put screenshot but again, I can't...

    Basically the file structure in the NAS, access through fixed ip.

    Two user structures: - admin // -my sister (only access public folders and her own)



    Folder structure in NAS:

    Public - (was public, I changed it to private) - admin access only - Read/Write permissions for admin

    SmartWare - admin access only - Read/Write permissions for admin

    TimeMachineBackup - - admin access only - Read/Write permissions for admin

    Music - admin access only - Read/Write permissions for admin

    mysisterfolder - admin and her user have access - distinct credentials to access



    The greyed out ones are Public access hence the grey.

    Ran this as suggested:

    # Below allows script execution.
    # set-executionpolicy remotesigned
    $SourcePath = '\\MYCLOUDEX2ULTRA\Music'
    $DestinationPath = '\\MYCLOUDEX2ULTRA\Entertainment'
    $LogFile = "C:\Users\luisf\Downloads\Robocopy_mp3.txt"
    Robocopy $SourcePath $DestinationPath /E /ZB /R:5 /W:10 /LOG:$LogFile
    Log output:

    -------------------------------------------------------------------------------
       ROBOCOPY     ::     Robust File Copy for Windows                              
    -------------------------------------------------------------------------------
    
      Started : Tuesday, April 28, 2020 5:03:40 PM
       Source : \\MYCLOUDEX2ULTRA\Music\
         Dest : \\MYCLOUDEX2ULTRA\Entertainment\
    
        Files : *.*
    	    
      Options : *.* /S /E /DCOPY:DA /COPY:DAT /ZB /R:5 /W:10 
    
    ------------------------------------------------------------------------------
    
    ERROR : You do not have the Backup and Restore Files user rights.
    *****  You need these to perform Backup copies (/B or /ZB).
    
    ERROR : Robocopy ran out of memory, exiting.
    ERROR : Invalid Parameter #%d : "%s"
    
    ERROR : Invalid Job File, Line #%d :"%s"
    
    
      Started : %s %s
    
       Source %c 
    
         Dest %c 
           Simple Usage :: ROBOCOPY source destination /MIR
    
                 source :: Source Directory (drive:\path or \\server\share\path).
            destination :: Destination Dir  (drive:\path or \\server\share\path).
                   /MIR :: Mirror a complete directory tree.
    
        For more usage information run ROBOCOPY /?
    
                                                              
    ****  /MIR can DELETE files as well as copy them !
    Seems that I was better off when I was just denied access.

    Any new ideas?

    I wish I could put some screen shots to help... Lets see how long it takes to validate my account...

    Thank you very much for the help.

    Wednesday, April 29, 2020 2:49 AM
  • When you use the windows explorer, are you using a mapped drive, Z:\  (which maps to the Entertainment folder) or the network path \\MYCLOUDEX2ULTRA\Entertainment\? Assuming that you can use the explorer and copy files and create new folders, use whatever you reference in explorer in the robocopy destination. 

    Lets start with the obvious and remove the /ZB switch.

    ERROR : You do not have the Backup and Restore Files user rights.
    *****  You need these to perform Backup copies (/B or /ZB).

    Wednesday, April 29, 2020 12:00 PM
  • I can do it both ways.

    So I have them mapped, but if for what ever reason the network doesn't pick it up, all I do is type this in Windows Explorer path to find the NAS "\\MYCLOUDEX2ULTRA" or just click on the mapped link and they tend to automatically connect.

    I've taken the command you mentioned and ran the below script:

    # Below allows script execution.
    # set-executionpolicy remotesigned
    $SourcePath = '\\MYCLOUDEX2ULTRA\Music'
    $DestinationPath = '\\MYCLOUDEX2ULTRA\Entertainment'
    $LogFile = "C:\Users\luisf\Downloads\Robocopy_mp3.txt"
    Robocopy $SourcePath $DestinationPath /E /R:5 /W:10 /LOG:$LogFile

    The LogFile output:

    -------------------------------------------------------------------------------
       ROBOCOPY     ::     Robust File Copy for Windows                              
    -------------------------------------------------------------------------------
    
      Started : Saturday, May 2, 2020 6:03:47 PM
       Source : \\MYCLOUDEX2ULTRA\Music\
         Dest : \\MYCLOUDEX2ULTRA\Entertainment\
    
        Files : *.*
    	    
      Options : *.* /S /E /DCOPY:DA /COPY:DAT /R:5 /W:10 
    
    ------------------------------------------------------------------------------
    
    	                   2	\\MYCLOUDEX2ULTRA\Music\
    2020/05/02 18:03:47 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/05/02 18:03:57 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/05/02 18:04:07 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds... Retrying...
    2020/05/02 18:04:17 ERROR 5 (0x00000005) Changing File Attributes \\MYCLOUDEX2ULTRA\Entertainment\
    Access is denied.
    
    Waiting 10 seconds...

    Wondering if there is a way to authenticate the user and password for the drive via script?

    I also made sure to run the PowerShell ISE as Admin.

    Thank you very much for your support and patience

    Sunday, May 3, 2020 1:16 AM
  • Windows has a "feature" where a drive that is mapped by a user is not visible in an elevated (run as admin) prompt. It might be that the NAS is seeing the ISE as a different account.

    Don't use the UNC path. Map a drive with an account that has update access and then use the drive letter. 

    If you want to "fix" that feature, create the registry entry described in the "Detail to configure the EnableLinkedConnections" registry entry" section.

    https://support.microsoft.com/en-us/help/3035277/mapped-drives-are-not-available-from-an-elevated-prompt-when-uac-is-co

    If it still doesn't work, try mapping the drive with PS.

    $mypath = '\\MYCLOUDEX2ULTRA\Entertainment\'
    $cred = Get-Credential 
    New-PSDrive W -PSProvider FileSystem -Root $myPath -Credential $cred
    $SourcePath = '\\MYCLOUDEX2ULTRA\Music'
    $DestinationPath = 'W:\'
    $LogFile = "C:\Users\luisf\Downloads\Robocopy_mp3.txt"
    Robocopy $SourcePath $DestinationPath /E /R:5 /W:10 /LOG:$LogFile
    Remove-PSDrive W

      
    Sunday, May 3, 2020 12:21 PM
  • So I've tried using the drive letter as suggested:

    Where;

    '\\MYCLOUDEX2ULTRA\Music' is mapped to 'Z:\' - Source

    '\MYCLOUDEX2ULTRA\MyMusic' is mapped to 'Y:\' - Destination (was Entertainment but decided to just create a new folder in the nas and call it MyMusic where I'll try and sort everything.

    $SourcePath = 'Z:\'
    $DestinationPath = 'Y:\'
    $LogFile = "C:\Users\luisf\Documents\Robocopy_mp3.txt"
    Robocopy $SourcePath $DestinationPath /E /R:5 /W:10 /LOG:$LogFile

    Unfortunately it did not work, this is the log file:

    -------------------------------------------------------------------------------
       ROBOCOPY     ::     Robust File Copy for Windows                              
    -------------------------------------------------------------------------------
    
      Started : Sunday, May 3, 2020 8:22:29 PM
    2020/05/03 20:22:29 ERROR 3 (0x00000003) Getting File System Type of Source Z:\
    The system cannot find the path specified.
    
    
    2020/05/03 20:22:29 ERROR 3 (0x00000003) Getting File System Type of Destination Y:\
    The system cannot find the path specified.
    
    
       Source = Z:\
         Dest = Y:\
    
        Files : *.*
    	    
      Options : *.* /S /E /DCOPY:DA /COPY:DAT /R:5 /W:10 
    
    ------------------------------------------------------------------------------
    
    2020/05/03 20:22:29 ERROR 3 (0x00000003) Accessing Source Directory Z:\
    The system cannot find the path specified.

    If I were to run your suggested script (adjusting the folder to MyMusic) would it un-map the drives I already have? Just wondering because I don't want to lose the working setup? If it's ok, then I'll do it...  Although even if it does mess it up I could always just re-map it right?

    Sorry for the obvious questions but as you can see I haven't done this before :) 

    Monday, May 4, 2020 3:33 AM
  • Something is not right here. Just open a command prompt. Let's do some simple tests and see if the copy command and icacls can see the file system. 

    net use
    dir Z:\
    dir Y:\
    icacls Z:\
    icacls Y:\
    echo test > Z:\test.txt
    copy Z:\test.txt  Y:\
    del Y:\test.txt
    robocopy Z: Y: test.txt


    You're not trying to do this through the task scheduler are you?


    • Edited by MotoX80 Monday, May 4, 2020 1:49 PM
    Monday, May 4, 2020 12:19 PM
  • Pasted your suggestion here is what happened:

    Windows PowerShell
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    PS C:\Users\luisf> net use
    >> dir Z:\
    >> dir Y:\
    >> icacls Z:\
    >> icacls Y:\
    >> echo test > Z:\test.txt
    >> copy Z:\test.txt  Y:\
    >> del Y:\test.txt
    >> robocopy Z: Y: test.txt
    New connections will be remembered.
    
    
    Status       Local     Remote                    Network
    
    -------------------------------------------------------------------------------
    OK           X:        \\MYCLOUDEX2ULTRA\Public  Microsoft Windows Network
    OK           Y:        \\MYCLOUDEX2ULTRA\Entertainment
                                                    Microsoft Windows Network
    OK           Z:        \\MYCLOUDEX2ULTRA\Music   Microsoft Windows Network
    The command completed successfully.
    
    
    
        Directory: Z:\
    
    
    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    d-----         5/3/2020  10:46 PM                2019
    d-----       10/31/2019   5:13 PM                iTunes
    d-----         5/3/2020   4:17 AM                iTunes 2
    d-----         3/7/2020   9:50 PM                Lib. Backup
    ------        4/26/2020   9:16 PM             28 Hello.txt
    ------         5/3/2020   4:07 AM       70674825 Library.xml
    ------         5/3/2020   3:37 AM       57914987 Music.xml
    ------         5/4/2020   7:30 PM             14 test.txt
    
    
        Directory: Y:\
    
    
    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    d-----        4/17/2020   1:02 AM                Dragon Ball GT (1996) Season 1-3 S01-S03 (480p DVD x265 HEVC 10bit
                                                     AC3 5.1 YOGI)
    Z:\ S-1-22-1-0:(F)
        S-1-22-2-0:(F)
        Everyone:(F)
    
    Successfully processed 1 files; Failed processing 0 files
    Y:\ S-1-22-1-0:(F)
        S-1-22-2-0:(F)
        Everyone:(F)
    
    Successfully processed 1 files; Failed processing 0 files
    
    -------------------------------------------------------------------------------
       ROBOCOPY     ::     Robust File Copy for Windows
    -------------------------------------------------------------------------------
    
      Started : Monday, May 4, 2020 7:32:31 PM
       Source : Z:\
         Dest : Y:\
    
        Files : test.txt
    
      Options : /DCOPY:DA /COPY:DAT /R:1000000 /W:30
    
    ------------------------------------------------------------------------------
    
                               1    Z:\
    2020/05/04 19:32:31 ERROR 5 (0x00000005) Changing File Attributes Y:\
    Access is denied.
    Waiting 30 seconds... Retrying...
    2020/05/04 19:33:01 ERROR 5 (0x00000005) Changing File Attributes Y:\
    Access is denied.
    Waiting 30 seconds... Retrying...
    2020/05/04 19:33:42 ERROR 5 (0x00000005) Changing File Attributes Y:\
    Access is denied.
    Waiting 30 seconds... Retrying...
    2020/05/04 19:34:12 ERROR 5 (0x00000005) Changing File Attributes Y:\
    Access is denied.
    Waiting 30 seconds... Retrying...
    2020/05/04 19:34:42 ERROR 5 (0x00000005) Changing File Attributes Y:\
    Access is denied.
    Waiting 30 seconds...
    

    I use powershell command line or use the PowerShell ISE open a script write code, cmd+enter to execute or just press the run button.

    There is a test.txt file in Z:

    So I tried the below which as a test works:

    Windows PowerShell
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    PS C:\Users\luisf> cd z:\
    PS Z:\> ls
    
    
        Directory: Z:\
    
    
    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    d-----         5/3/2020  10:46 PM                2019
    d-----       10/31/2019   5:13 PM                iTunes
    d-----         5/3/2020   4:17 AM                iTunes 2
    d-----         3/7/2020   9:50 PM                Lib. Backup
    ------        4/26/2020   9:16 PM             28 Hello.txt
    ------         5/3/2020   4:07 AM       70674825 Library.xml
    ------         5/3/2020   3:37 AM       57914987 Music.xml
    ------         5/4/2020   7:32 PM             14 test.txt
    
    
    PS Z:\> cp 'test.txt' -Destination:
    At line:1 char:28
    + cp 'test.txt' -Destination:
    +                            ~
    Parameter -Destination: requires an argument.
        + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
        + FullyQualifiedErrorId : ParameterRequiresArgument
    
    PS Z:\> cp 'test.txt' -Destination: 'Y:\'
    PS Z:\> cd y:\
    PS Y:\> ls
    
    
        Directory: Y:\
    
    
    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    d-----        4/17/2020   1:02 AM                Dragon Ball GT (1996) Season 1-3 S01-S03 (480p DVD x265 HEVC 10bit
                                                     AC3 5.1 YOGI)
    ------         5/4/2020   7:32 PM             14 test.txt
    
    
    PS Y:\>

    But this method, if I use it to copy the folder, I run into the issues from the start of this post, not all files are moved etc etc..

    Feeling a little frustrated because this is like the "initial" part of what I'm trying to do and already struggling.🤦‍♂️

    My goal is to successfully copy these folders so that then I can attempt a script that merges the Music folder in each itunes and checks if that folder with same name exists, if it does just merge them, if not copy it with a different name and maybe list out what these are so I can then check these conflicts and help organize even further.

    Anyways, any suggestion why doing "my way" I don't face the access restrictions?

    Thank you once again for your time.

    Tuesday, May 5, 2020 2:53 AM
  • I think that there's something wrong with the file system or one of the settings on your NAS. Try this and see what happens. 

    $source = 'c:\inetpub\'         # source and dest should both have ending \
    $dest = 'c:\test\'
    $skipcount = 0
    $copycount = 0
    $errorcount = 0
    $files = Get-ChildItem -Path $source -Recurse
    foreach ($file in $files) {
        $newfile = $dest + ($file.fullname).Substring($source.length)
        if (test-path $newfile) {                     # does the file already exist? 
            "Skipping {0}" -f $newfile
            $skipcount += 1
        }
        else {
            try { 
                copy-item -Path $file.fullname -Destination $newfile -erroraction stop
                 $copycount += 1
            }
            catch {
                "Error copying {0}" -f $newfile | write-host -ForegroundColor yellow
                $error 
                $errorcount += 1
                $error.Clear()    
            }      
        }
    }
    "We found {0} files to process." -f $files.count
    "Copy count:  {0}" -f $copycount
    "Skip count:  {0}" -f $skipcount
    "Error count: {0}" -f $errorcount
    
    

    Tuesday, May 5, 2020 1:54 PM
  • This is the console output in PowerShell ISE:

    PS C:\WINDOWS\system32> $source = 'c:\inetpub\'         # source and dest should both have ending \
    $dest = 'c:\test\'
    $skipcount = 0
    $copycount = 0
    $errorcount = 0
    $files = Get-ChildItem -Path $source -Recurse
    foreach ($file in $files) {
        $newfile = $dest + ($file.fullname).Substring($source.length)
        if (test-path $newfile) {                     # does the file already exist? 
            "Skipping {0}" -f $newfile
            $skipcount += 1
        }
        else {
            try { 
                copy-item -Path $file.fullname -Destination $newfile -erroraction stop
                 $copycount += 1
            }
            catch {
                "Error copying {0}" -f $newfile | write-host -ForegroundColor yellow
                $error 
                $errorcount += 1
                $error.Clear()    
            }      
        }
    }
    "We found {0} files to process." -f $files.count
    "Copy count:  {0}" -f $copycount
    "Skip count:  {0}" -f $skipcount
    "Error count: {0}" -f $errorcount
    Get-ChildItem : Cannot find path 'C:\i' because it does not exist.
    At line:6 char:10
    + $files = Get-ChildItem -Path $source -Recurse
    +          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : ObjectNotFound: (C:\i:String) [Get-ChildItem], ItemNotFoundException
        + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand
     
    We found 0 files to process.
    Copy count:  0
    Skip count:  0
    Error count: 0
    
    PS C:\WINDOWS\system32> 

    not sure what c:\inetpub\ is...

    Checked if test folder was created:

    PS C:\WINDOWS\system32> cd ..
    
    PS C:\WINDOWS> cd..
    
    PS C:\> ls
    
    
        Directory: C:\
    
    
    Mode                LastWriteTime         Length Name                                                                                                                                                                                
    ----                -------------         ------ ----                                                                                                                                                                                
    d-r---        4/11/2020  10:11 PM                Program Files                                                                                                                                                                       
    d-r---        4/17/2020   7:11 AM                Program Files (x86)                                                                                                                                                                 
    d-r---        2/10/2020   6:38 PM                Users                                                                                                                                                                               
    da----        4/17/2020   1:09 PM                Windows                                                                                                                                                                             
    
    
    
    PS C:\> 


    Wednesday, May 6, 2020 6:11 AM
  • Sorry, those were just the folders I was testing with. Put your X, Y, and Z as source and dest. Or test some subfolders before you try to copy the bulk of the data.

    $source = 'Z:\iTunes 2\Some artist\some album'

    $dest = 'Y:\iTunes 2\Some artist\some album'

    Wednesday, May 6, 2020 12:12 PM
  • Ran the code it worked!

    I'm going to try and use this to move the entire z to w and I'll report back (if it works, it will take hours due to 1 TB of music almost)

    PS C:\Users\luisf> $source = 'Z:\Lib. Backup\'         # z:\Lib. Backup\ = '\\MYCLOUDEX2ULTRA\Music\Lib. Backup' # contains small files 
    $dest = 'W:\'                       # w:\ = '\\MYCLOUDEX2ULTRA\MyMusic'
    $skipcount = 0
    $copycount = 0
    $errorcount = 0
    $files = Get-ChildItem -Path $source -Recurse
    foreach ($file in $files) {
        $newfile = $dest + ($file.fullname).Substring($source.length)
        if (test-path $newfile) {                     # does the file already exist? 
            "Skipping {0}" -f $newfile
            $skipcount += 1
        }
        else {
            try { 
                copy-item -Path $file.fullname -Destination $newfile -erroraction stop
                 $copycount += 1
            }
            catch {
                "Error copying {0}" -f $newfile | write-host -ForegroundColor yellow
                $error 
                $errorcount += 1
                $error.Clear()    
            }      
        }
    }
    "We found {0} files to process." -f $files.count
    "Copy count:  {0}" -f $copycount
    "Skip count:  {0}" -f $skipcount
    "Error count: {0}" -f $errorcount
    We found 7 files to process.
    Copy count:  7
    Skip count:  0
    Error count: 0
    
    PS C:\Users\luisf> cd w:\ 
    
    PS W:\> ls
    
    
        Directory: W:\
    
    
    Mode                LastWriteTime         Length Name                                                                                                                                                                                
    ----                -------------         ------ ----                                                                                                                                                                                
    ------        1/12/2020  10:21 PM        1417216 iTunes Library Extras.itdb                                                                                                                                                          
    ------        12/1/2012   7:27 PM        3641344 iTunes Library Genius.itdb                                                                                                                                                          
    ------        1/12/2020  10:11 PM       19170093 iTunes Library.itl                                                                                                                                                                  
    ------        1/12/2020  10:12 PM       91539686 iTunes Library.xml                                                                                                                                                                  
    ------         3/7/2020   9:50 PM       70497450 Library.xml                                                                                                                                                                         
    ------        1/12/2020  10:21 PM              8 sentinel                                                                                                                                                                            
    ------        3/15/2014   8:01 AM       53726674 Temp File.tmp                                                                                                                                                                       
    
    
    
    PS W:\> 


    Thursday, May 7, 2020 2:09 AM
  • After the success of the individual folder, I tried to just map the drive but PowerShell just said it was running the script but nothing was happening so I stopped the script.

    PS W:\> $source = 'Z:\'                     # z:\ = '\\MYCLOUDEX2ULTRA\Music\' # contains small files 
    $dest = 'W:\'                       # w:\ = '\\MYCLOUDEX2ULTRA\MyMusic'
    $skipcount = 0
    $copycount = 0
    $errorcount = 0
    $files = Get-ChildItem -Path $source -Recurse
    foreach ($file in $files) {
        $newfile = $dest + ($file.fullname).Substring($source.length)
        if (test-path $newfile) {                     # does the file already exist? 
            "Skipping {0}" -f $newfile
            $skipcount += 1
        }
        else {
            try { 
                copy-item -Path $file.fullname -Destination $newfile -erroraction stop
                 $copycount += 1
            }
            catch {
                "Error copying {0}" -f $newfile | write-host -ForegroundColor yellow
                $error 
                $errorcount += 1
                $error.Clear()    
            }      
        }
    }
    "We found {0} files to process." -f $files.count
    "Copy count:  {0}" -f $copycount
    "Skip count:  {0}" -f $skipcount
    "Error count: {0}" -f $errorcount
    
    PS W:\> 

    Would I have to direct every folder?

    Thursday, May 7, 2020 2:19 AM
  •  PowerShell just said it was running the script but nothing was happening so I stopped the script.

    Would I have to direct every folder?

    If you run it again, I would expect that you'll get a bunch of "skipping" messages. If you want to see what it's copying, just add some output like this in the try section. If you don't get ANY messages then we'll need to add some debugging logic to the code. 

            try { 
                copy-item -Path $file.fullname -Destination $newfile -erroraction stop
                 $copycount += 1
                 "Copied {0}" -f $file.fullname      
            }
           

    Note that this code only checks for the presence of a destination file. It will not recopy files that have been changed or delete files from the dest that were deleted from the source (robocopy's /mir function).

    I would also suggest that you use the Windiff utility and compare the source and destination folders to see if they match. It's an old tool but it still works. 

    https://www.microsoft.com/en-us/download/details.aspx?id=7911

    • Edited by MotoX80 Thursday, May 7, 2020 12:25 PM
    Thursday, May 7, 2020 12:16 PM
  • Not sure I understood your answer.

    I was wondering why if I change to just the directory mapped letter not a specific folder it stopped working, or seemed like it wasn't copying anything.

    $source = 'Z:\Lib. Backup\'         # z:\Lib. Backup\ = '\\MYCLOUDEX2ULTRA\Music\Lib. Backup' # contains small files 
    $dest = 'W:\'                       # w:\ = '\\MYCLOUDEX2ULTRA\MyMusic'

    On the above I specified the folder I wanted copied.

    But when I move up one directory to just 'Z:\" it appears to not copy... So if there are 3 folders in 'Z:\' I would have to run it 3x? Why did it not just copy the files in 'Z:\' ?

    $source = 'Z:\'                     # z:\ = '\\MYCLOUDEX2ULTRA\Music\' # contains 3 folders 
    $dest = 'W:\'                       # w:\ = '\\MYCLOUDEX2ULTRA\MyMusic'

    Thursday, May 7, 2020 1:10 PM
  • Not sure I understood your answer.

    I was wondering why if I change to just the directory mapped letter not a specific folder it stopped working, or seemed like it wasn't copying anything.

    What don't you understand? Did you look at the destination directory? Were any files copied? 

    If you look at the original code that I posted, it only output's ("does something") when it encounters an existing file, or if it encounters an error when copying the file. 

    So my reply said to add some output when copying a file. 

    If that's not sufficient, then add code to output both the $file.fulllname and $newfile values before it executes the copy-item statement. 

    Thursday, May 7, 2020 6:25 PM
  • Hi there and once again thanks for all the support.

    I have ran into a few problems, I thought it worked properly but it did not.

    This is the Main folder I want to copy and all it's content (folders and their sub-folders & content). 

    Windows PowerShell
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    PS C:\Users\luisf> cd z:
    PS Z:\> ls
    
    
        Directory: Z:\
    
    
    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    d-----         5/6/2020   1:17 AM                2019
    d-----       10/31/2019   5:13 PM                iTunes
    d-----         5/3/2020   4:17 AM                iTunes 2
    d-----         3/7/2020   9:50 PM                Lib. Backup
    ------         5/3/2020   4:07 AM       70674825 Library.xml
    ------         5/3/2020   3:37 AM       57914987 Music.xml

    As seen above there are folders: 2019, iTunes, iTunes 2, Lib. Backup.

    When the code worked I copied this directory:

    $source = 'Z:\Lib. Backup\'         # z:\Lib. Backup\ = '\\MYCLOUDEX2ULTRA\Music\Lib. Backup' # contains small files 
    $dest = 'W:\'                       # w:\ = '\\MYCLOUDEX2ULTRA\MyMusic'

    What happened which worked was that the contents of folder "Lib. Backup" were copied into "W:\", so almost there, because my intent was to copy the entire folder and content to "W:\".

    So since you've helped so much and I didn't want to keep asking, I decided to just try manually doing each folder into a "ALL" folder in the destination.

    Here is the script:

    $source = 'Z:\2019'                     # z:\ = '\\MYCLOUDEX2ULTRA\Music\' 
    $dest = 'W:\All'                       # w:\ = '\\MYCLOUDEX2ULTRA\MyMusic\All'
    $skipcount = 0
    $copycount = 0
    $errorcount = 0
    $files = Get-ChildItem -Path $source -Recurse
    foreach ($file in $files) {
        $newfile = $dest + ($file.fullname).Substring($source.length)
        if (test-path $newfile) {                     # does the file already exist? 
            "Skipping {0}" -f $newfile
            $skipcount += 1
        }
        else {
            try { 
                copy-item -Path $file.fullname -Destination $newfile -erroraction stop
                 $copycount += 1
            }
            catch {
                "Error copying {0}" -f $newfile | write-host -ForegroundColor yellow
                $error 
                $errorcount += 1
                $error.Clear()    
            }      
        }
    }
    "We found {0} files to process." -f $files.count
    "Copy count:  {0}" -f $copycount
    "Skip count:  {0}" -f $skipcount
    "Error count: {0}" -f $errorcount

    Output: 

    PS C:\Users\luisf> C:\Users\luisf\Documents\PowerShell\CopyItem_post.ps1
    Error copying W:\All\Little Dragon - New Me, Same Us\LPJ1RC~5
    copy-item : The filename, directory name, or volume label syntax is incorrect.
    
    At C:\Users\luisf\Documents\PowerShell\CopyItem_post.ps1:15 char:13
    +             copy-item -Path $file.fullname -Destination $newfile -err ...
    +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [Copy-Item], IOException
        + FullyQualifiedErrorId : System.IO.IOException,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Error copying W:\All\VA - Beatport - DJ MAG ALTERNATIVE TOP100\Disclosure, Khalid - Talk (Original Mix) [Right Hand Music Group, LLC
    Copy-Item : Cannot retrieve the dynamic parameters for the cmdlet. The specified wildcard character pattern is not valid: Disclosure, Khalid - Talk (Original Mix) [Right Hand Music Group, LLC
    At C:\Users\luisf\Documents\PowerShell\CopyItem_post.ps1:15 char:13
    +             copy-item -Path $file.fullname -Destination $newfile -err ...
    +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (:) [Copy-Item], ParameterBindingException
        + FullyQualifiedErrorId : GetDynamicParametersException,Microsoft.PowerShell.Commands.CopyItemCommand
     
    Error copying W:\All\VA - Beatport - DJ MAG ALTERNATIVE TOP100\Disclosure, Khalid - Talk (Original Mix) [Right Hand Music Group, LLC\RCA Records].mp3
    Copy-Item : Cannot retrieve the dynamic parameters for the cmdlet. The specified wildcard character pattern is not valid: Disclosure, Khalid - Talk (Original Mix) [Right Hand Music Group, LLC
    At C:\Users\luisf\Documents\PowerShell\CopyItem_post.ps1:15 char:13
    +             copy-item -Path $file.fullname -Destination $newfile -err ...
    +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (:) [Copy-Item], ParameterBindingException
        + FullyQualifiedErrorId : GetDynamicParametersException,Microsoft.PowerShell.Commands.CopyItemCommand
     
    We found 901 files to process.
    Copy count:  898
    Skip count:  0
    Error count: 3

    So that seemed to work but when I went to check the destination directory "ALL", the folders in 2019 were copied but the contents weren't. Just the "loose" .mp3 files in the 2019 folder and the folders (without contents) were copied.

    I would have expected (with my lack of knowledge) that if I direct my source as Z:\ and destination W:\ the script to copy all folders and subfolders from Z:\ to W:\ . If I do that the script keeps running but nothing appears to be copied so I have to halt the script, and that is what I was referring to before.

    Sorry if I wasn't clear and once again thank you for your assistance I really do appreciate the help.

    Friday, May 8, 2020 2:02 AM
  • Powershell is seeing the special characters in your file names and is trying to process them. Try this version. 

    Set the source and dest variables to the folders that you want to use.  

    $source = 'z:\'         # source and dest should both have ending \
    $dest = 'y:\'
    $skipcount = 0
    $copycount = 0
    $errorcount = 0
    $files = Get-ChildItem -Path $source -Recurse
    "{0} This is PSRobocopy version 2." -f (get-date)| write-host -ForegroundColor yellow
    "We found {0} files and folders to process." -f $files.count
    foreach ($file in $files) {
    	""
    	"Processing:  {0}" -f $file.fullname
    	#$file | format-list -property *                      # uncomment to see all properties for the file. 
            $newfile = $dest + ($file.fullname).Substring($source.length)
    	"Destination: {0}" -f $newfile
    	if ($file.PSIsContainer) {
    		"This is a folder."
    	}
    	else {
    		"This is a file."
    	}
        if (test-path $newfile) {                     # does the file already exist? 
            "Item already exists, skipping {0}" -f $newfile
            $skipcount += 1
        }
        else {
            try { 
                copy-item -LiteralPath $file.fullname -Destination $newfile -erroraction stop
    			if (test-path $newfile) {                     # does the file exist now? 
    				$copycount += 1
    				"Copied {0}" -f $file.fullname	
    			}
    			else {
    				"Error, the destination item does not exist, but Copy-Item did not crash." | write-host -ForegroundColor yellow
    				$errorcount += 1			
    			}  
            }
            catch {
                "Error copying {0}" -f $newfile | write-host -ForegroundColor yellow
                $error 
                $errorcount += 1
                $error.Clear()    
            }      
        }
    }
    ""
    "We found {0} files and folders to process." -f $files.count
    "Copy count:  {0}" -f $copycount
    "Skip count:  {0}" -f $skipcount
    "Error count: {0}" -f $errorcount
    
    

    Friday, May 8, 2020 2:02 PM