none
Using Powershell to Get Directory Sizes RRS feed

  • Question

  • I'm preparing to move production data to new storage, in preparing for the preparation I figure it's great to know just how much data we'll be moving. My current group of users are using a large network share to store a mix of production and archive data due to poor data management in the past. With new storage on the way to our server room I'm tasked with moving only production data to it. To do this I was provided a list of currently active project codes from accounting, I figure I can use this .csv to find the project directories and their sizes since all project directories start with the project codes (ie. project code is xyz0123, directory would be \\server\share\market\xyz0123).

    I began putting together some rough Powershell in addition with a function I found online but quickly ran into issues. Below is the two-step process I walked through. There are definitely more efficient ways to do this I'm aware, but this isn't too time sensitive and resources don't seem to be too taxed with running through the millions of directories we seem to have.

    Step 1, pull the directory paths based on the codes given. Step 2, measure each of the resulting directory paths.

    $path = "C:\users\USER\ActiveProjectCodes.csv" $rootFolder = "\\server\share" $csvExport = "c:\results" $csv = Import-Csv -Path $path

    function Get-FolderSize {
    [CmdletBinding()]
    Param (
    [Parameter(Mandatory=$true,ValueFromPipeline=$true)]
    $Path,
    [ValidateSet("KB","MB","GB")]
    $Units = "MB"
    )
      if ( (Test-Path $Path) -and (Get-Item $Path).PSIsContainer ) {
        $Measure = Get-ChildItem $Path -Recurse -Force -ErrorAction SilentlyContinue | Measure-Object -Property Length -Sum
        $Sum = $Measure.Sum / "1$Units"
        [PSCustomObject]@{
          "Path" = $Path
          "Size($Units)" = $Sum
        }
      }
    } $output = foreach ($project in $csv) {
        $projectname = $project.Job
        Get-ChildItem $rootFolder -Recurse -Depth 2 | where-Object {$_.PSIsContainer -eq $True -and $_.FullName -like "*$projectname*"} | Select-Object -Property fullname
        }

    $output2 = foreach ($fullname in $output) {
        $filepath = $fullname.FullName
        Get-FolderSize -Path $filepath -Units GB
        }

    $output2 | Export-Csv $csvExport

    The issue seems to be with pulling the initial $output. I have to use wildcards for the project codes due to the unfortunate renaming done by project leaders who add on to the project codes with names of the project. When running this alone I get a resulting csv with over one million paths, essentially it's pulling all directory paths. What am I missing with this step?

    • Edited by nistan Thursday, October 3, 2019 2:00 PM
    Thursday, October 3, 2019 1:58 PM

All replies

  • Rather than iterating containers using PowerShell, I would recommend the SysInternals du/du64 command, which should be accurate and faster:

    https://docs.microsoft.com/en-us/sysinternals/downloads/du

    Drop the du.exe (or du64.exe, if you have 64-bit OS) into a directory in your Path, and you're good to go.

    If you want a PowerShell wrapper for the du/du64 command (so you can sort/filter/calculate/etc.), you can use Get-DirStats.ps1:

    https://gist.github.com/Bill-Stewart/cca4032f8d04b7388b5fc9a0d5b8806d


    -- Bill Stewart [Bill_Stewart]

    Thursday, October 3, 2019 2:47 PM
    Moderator
  • $path = 'C:\users\USER\ActiveProjectCodes.csv'
    $rootFolder = '\\server\share'
    $csvExport = 'c:\results'
    
    Import-Csv -Path $path |
    	ForEach-Object{
        	    Get-ChildItem "$rootFolder\*$($_.Job)*" -Recurse -Depth 2 -Directory
    	} |
    	Select-Object fullname |
    	Export-Csv $csvExport



    \_(ツ)_/




    • Edited by jrv Thursday, October 3, 2019 2:55 PM
    Thursday, October 3, 2019 2:53 PM
  • As an update, as inefficient as my Powershell is I think the issue was with the csv header, the code is looking for $project.Job however my csv was set up to use $project.Project. D'oh! I'm trying this again this evening and will report back with whether that was it or not.
    Thursday, October 3, 2019 6:16 PM