{"id":22,"date":"2010-12-21T16:44:11","date_gmt":"2010-12-21T23:44:11","guid":{"rendered":"http:\/\/ryan.witschger.net\/?p=22"},"modified":"2014-06-24T15:17:20","modified_gmt":"2014-06-24T21:17:20","slug":"multi-threading-in-powershell-v2","status":"publish","type":"post","link":"http:\/\/www.Get-Blog.com\/?p=22","title":{"rendered":"Multithreading with Jobs in PowerShell"},"content":{"rendered":"

NOTE: I have written a better script for generic multithreading which I have covered in my post HERE<\/a>. If you are looking for a script to cover your every day needs, please read that article instead as I believe it is a better script. This script, however, is easier to understand if you are looking to learn this for yourself!<\/p>\n

This post is actually the reason that I created the blog. When I first started looking into multithreading in PowerShell V2 I really didn\u2019t find anyone on the web that really had a good explanation or how-to. So, why would you want to multithread your scripts? Well, if you have ever tried to run a certain script against every server or even workstation in your org you know that it can take a very long time to run because it is hitting each server, one at a time, in sequence. Wouldn\u2019t it be great if you could run your script against 20 servers at a time? As it turns out, you can, and it\u2019s easier than you think.
\nAll of the work comes in understanding the 4 main cmdlets surrounding multithreading in PowerShell V2: Start-Job, Wait-Job, Get-Job and Receive-Job. The basic flow states that you start all jobs, then wait for all jobs to finish, then see what jobs you have (get) and receive all the output. <\/p>\n

Start-Job {Get-Process}\r\nGet-Job | Wait-Job\r\nGet-Job | Receive-Job<\/pre>\n
So, the above basic script starts a job,then waits for all jobs to finish and then receives all the data that all jobs (only one in our case) contain.  Now this basic construct is limited,because if you try to pass a variable via the code block it won\u2019t work.  That is because whatever code block you pass is a bit like opening a new PowerShell session, pasting it in, and hitting enter.  Any variables that you have aren\u2019t in that new pristine environment.  The guys over at Microsoft gave us a way to pass info in though with the cmdlets argument \u201cArgumentList\u201d where you can use a variable from the host session to be passed to the new session.  This, however, only works when calling a script, not a code block, so we have to also use the argument \u201cFilePath\u201d and provide a second PowerShell script.  This is actually a good thing, because it means we can multithread any script we write as long as it has a consistent output and takes something as an argument.  Take the following short script:<\/p>\n
### Get-OperatingSystem.ps1 ###\r\nParam($ComputerName = \"LocalHost\")\r\nGet-WmiObject -ComputerName $ComputerName -Class Win32_OperatingSystem<\/pre>\n
Now, this script is perfect because it is going to return an object and take a computer name as an argument when executed.  Presumably, we would normally multi-thread something that has a much longer execution time, but keep in mind that if the host is offline this script could take a long time to run.<\/p>\n
### Start-MultiThread.ps1 ###\r\n$Computers = @(\"Computer1\",\"Computer2\",\"Computer3\") \r\n\r\n#Start all jobs\r\nForEach($Computer in $Computers){\r\n    Start-Job -FilePath c:ScriptGet-OperatingSystem.ps1 -ArgumentList $Computer\r\n}\r\n\r\n#Wait for all jobs\r\nGet-Job | Wait-Job\r\n\r\n#Get all job results\r\nGet-Job | Receive-Job | Out-GridView<\/pre>\n
So there it is.  It\u2019s that easy to multithread any script that you\u2019ve written, but this is the most basic construct.  What happens if you want to control how many threads are open?  How about letting the user know where the script is in execution and how things are going?  Let\u2019s start by adding a param block to the front of the script to get a bunch of information.<\/p>\n
Param($ScriptFile = $(Read-Host \"Enter the script file\"), \r\n    $ComputerList = $(Read-Host \"Enter the Location of the computerlist\"),\r\n    $MaxThreads = 20,\r\n    $SleepTimer = 500)<\/pre>\n
Now that we have a way to get some basic setting from the user let\u2019s read in out computer list from the file provided and kill any currently running jobs.<\/p>\n
$Computers = Get-Content $ComputerList\r\n\r\n\"Killing existing jobs . . .\"\r\nGet-Job | Remove-Job -Force\r\n\"Done.\"<\/pre>\n
Now let\u2019s get our loop control going and start making some threads<\/p>\n
$i = 0\r\n\r\nForEach ($Computer in $Computers){\r\n    # Check to see if there are too many open threads\r\n    # If there are too many threads then wait here until some close\r\n    While ($(Get-Job -state running).count -ge $MaxThreads){\r\n        Write-Progress  -Activity \"Creating Server List\" \r\n                        -Status \"Waiting for threads to close\" \r\n                        -CurrentOperation \"$i threads created - $($(Get-Job -state running).count) threads open\" \r\n                        -PercentComplete ($i \/ $Computers.count * 100)\r\n        Start-Sleep -Milliseconds $SleepTimer\r\n    }\r\n\r\n    #\"Starting job - $Computer\"\r\n    $i++\r\n    Start-Job -FilePath $ScriptFile -ArgumentList $Computer -Name $Computer | Out-Null\r\n    Write-Progress  -Activity \"Creating Server List\" \r\n                -Status \"Starting Threads\" \r\n                -CurrentOperation \"$i threads created - $($(Get-Job -state running).count) threads open\" \r\n                -PercentComplete ($i \/ $Computers.count * 100)\r\n    \r\n}\r\n<\/pre>\n
So this section of code is pretty simple if you just break it down.  First we use a while loop to hold there until the current number of running jobs is lower than the number we have declared as our maximum.  The Write-Progress command is simply letting the user know what\u2019s going on.  Once we clear the while loop we are ready to add jobs to the list of running jobs.  So we start another job with the Start-Job command and then write our progress out to the user.   Once this block of code is done we want to wait for all the jobs to close.  In my short example I used Get-Job | Wait-Job which does the job, but it hides the progress from our user, so instead I developed this little tidbit.<\/p>\n
While ($(Get-Job -State Running).count -gt 0){\r\n    $ComputersStillRunning = \"\"\r\n    ForEach ($System  in $(Get-Job -state running)){$ComputersStillRunning += \", $($System.name)\"}\r\n    $ComputersStillRunning = $ComputersStillRunning.Substring(2)\r\n    Write-Progress  -Activity \"Creating Server List\" \r\n                    -Status \"$($(Get-Job -State Running).count) threads remaining\" \r\n                    -CurrentOperation \"$ComputersStillRunning\" \r\n                    -PercentComplete ($(Get-Job -State Completed).count \/ $(Get-Job).count * 100)\r\n    Start-Sleep -Milliseconds $SleepTimer\r\n}<\/pre>\n
So this block of code is going to read in the computer names that we are still waiting on and show them in the write-progress command.  Again I\u2019ve used a while loop which would run unchecked if not for the start-sleep that I\u2019ve placed in there, which is like saying \u201cHey, only check our progress every half second or so\u201d.  If I didn\u2019t have the start-sleep it would simply max the processor.<\/p>\n
Once all that is done, it is simply a matter of getting the output from all our workers.  You can just use Get-Job | Receive-Job to spit is all out to the console or you can push any object out to PowerShell V2\u2019s grid view which I love oh so much.<\/p>\n
Get-Job | Receive-Job | out-gridview<\/pre>\n
So there is a script which allows you to multithread any script in your arsenal.  Enjoy!<\/p>\n
The full text for the script I use is as follows:<\/p>\n
Param($ScriptFile = $(Read-Host \"Enter the script file\"), \r\n    $ComputerList = $(Read-Host \"Enter the Location of the computerlist\"),\r\n    $MaxThreads = 20,\r\n    $SleepTimer = 500,\r\n    $MaxWaitAtEnd = 600,\r\n    $OutputType = \"Text\")\r\n    \r\n$Computers = Get-Content $ComputerList\r\n\r\n\r\n\"Killing existing jobs . . .\"\r\nGet-Job | Remove-Job -Force\r\n\"Done.\"\r\n\r\n$i = 0\r\n\r\nForEach ($Computer in $Computers){\r\n    While ($(Get-Job -state running).count -ge $MaxThreads){\r\n        Write-Progress  -Activity \"Creating Server List\" \r\n                        -Status \"Waiting for threads to close\" \r\n                        -CurrentOperation \"$i threads created - $($(Get-Job -state running).count) threads open\" \r\n                        -PercentComplete ($i \/ $Computers.count * 100)\r\n        Start-Sleep -Milliseconds $SleepTimer\r\n    }\r\n\r\n    #\"Starting job - $Computer\"\r\n    $i++\r\n    Start-Job -FilePath $ScriptFile -ArgumentList $Computer -Name $Computer | Out-Null\r\n    Write-Progress  -Activity \"Creating Server List\" \r\n                -Status \"Starting Threads\" \r\n                -CurrentOperation \"$i threads created - $($(Get-Job -state running).count) threads open\" \r\n                -PercentComplete ($i \/ $Computers.count * 100)\r\n    \r\n}\r\n\r\n$Complete = Get-date\r\n\r\nWhile ($(Get-Job -State Running).count -gt 0){\r\n    $ComputersStillRunning = \"\"\r\n    ForEach ($System  in $(Get-Job -state running)){$ComputersStillRunning += \", $($System.name)\"}\r\n    $ComputersStillRunning = $ComputersStillRunning.Substring(2)\r\n    Write-Progress  -Activity \"Creating Server List\" \r\n                    -Status \"$($(Get-Job -State Running).count) threads remaining\" \r\n                    -CurrentOperation \"$ComputersStillRunning\" \r\n                    -PercentComplete ($(Get-Job -State Completed).count \/ $(Get-Job).count * 100)\r\n    If ($(New-TimeSpan $Complete $(Get-Date)).totalseconds -ge $MaxWaitAtEnd){\"Killing all jobs still running . . .\";Get-Job -State Running | Remove-Job -Force}\r\n    Start-Sleep -Milliseconds $SleepTimer\r\n}\r\n\r\n\"Reading all jobs\"\r\n\r\nIf ($OutputType -eq \"Text\"){\r\n    ForEach($Job in Get-Job){\r\n        \"$($Job.Name)\"\r\n        \"****************************************\"\r\n        Receive-Job $Job\r\n        \" \"\r\n    }\r\n}\r\nElseIf($OutputType -eq \"GridView\"){\r\n    Get-Job | Receive-Job | Select-Object * -ExcludeProperty RunspaceId | out-gridview\r\n    \r\n}<\/pre>\n","protected":false},"excerpt":{"rendered":"
Do your long complicated scripts run too slowly against many targets?  It’s easier than you think to to start multi-threading in PowerShell V2!<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[4],"tags":[],"_links":{"self":[{"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=\/wp\/v2\/posts\/22"}],"collection":[{"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=22"}],"version-history":[{"count":2,"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=\/wp\/v2\/posts\/22\/revisions"}],"predecessor-version":[{"id":196,"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=\/wp\/v2\/posts\/22\/revisions\/196"}],"wp:attachment":[{"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=22"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=22"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.Get-Blog.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=22"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}