Dec 212010
 

NOTE: I have written a better script for generic multithreading which I have covered in my post HERE. If you are looking for a script to cover your every day needs, please read that article instead as I believe it is a better script. This script, however, is easier to understand if you are looking to learn this for yourself!

This post is actually the reason that I created the blog. When I first started looking into multithreading in PowerShell V2 I really didn’t find anyone on the web that really had a good explanation or how-to. So, why would you want to multithread your scripts? Well, if you have ever tried to run a certain script against every server or even workstation in your org you know that it can take a very long time to run because it is hitting each server, one at a time, in sequence. Wouldn’t it be great if you could run your script against 20 servers at a time? As it turns out, you can, and it’s easier than you think.
All of the work comes in understanding the 4 main cmdlets surrounding multithreading in PowerShell V2: Start-Job, Wait-Job, Get-Job and Receive-Job. The basic flow states that you start all jobs, then wait for all jobs to finish, then see what jobs you have (get) and receive all the output.

So, the above basic script starts a job,then waits for all jobs to finish and then receives all the data that all jobs (only one in our case) contain. Now this basic construct is limited,because if you try to pass a variable via the code block it won’t work. That is because whatever code block you pass is a bit like opening a new PowerShell session, pasting it in, and hitting enter. Any variables that you have aren’t in that new pristine environment. The guys over at Microsoft gave us a way to pass info in though with the cmdlets argument “ArgumentList” where you can use a variable from the host session to be passed to the new session. This, however, only works when calling a script, not a code block, so we have to also use the argument “FilePath” and provide a second PowerShell script. This is actually a good thing, because it means we can multithread any script we write as long as it has a consistent output and takes something as an argument. Take the following short script:

Now, this script is perfect because it is going to return an object and take a computer name as an argument when executed. Presumably, we would normally multi-thread something that has a much longer execution time, but keep in mind that if the host is offline this script could take a long time to run.

So there it is. It’s that easy to multithread any script that you’ve written, but this is the most basic construct. What happens if you want to control how many threads are open? How about letting the user know where the script is in execution and how things are going? Let’s start by adding a param block to the front of the script to get a bunch of information.

Now that we have a way to get some basic setting from the user let’s read in out computer list from the file provided and kill any currently running jobs.

Now let’s get our loop control going and start making some threads

So this section of code is pretty simple if you just break it down. First we use a while loop to hold there until the current number of running jobs is lower than the number we have declared as our maximum. The Write-Progress command is simply letting the user know what’s going on. Once we clear the while loop we are ready to add jobs to the list of running jobs. So we start another job with the Start-Job command and then write our progress out to the user. Once this block of code is done we want to wait for all the jobs to close. In my short example I used Get-Job | Wait-Job which does the job, but it hides the progress from our user, so instead I developed this little tidbit.

So this block of code is going to read in the computer names that we are still waiting on and show them in the write-progress command. Again I’ve used a while loop which would run unchecked if not for the start-sleep that I’ve placed in there, which is like saying “Hey, only check our progress every half second or so”. If I didn’t have the start-sleep it would simply max the processor.

Once all that is done, it is simply a matter of getting the output from all our workers. You can just use Get-Job | Receive-Job to spit is all out to the console or you can push any object out to PowerShell V2’s grid view which I love oh so much.

So there is a script which allows you to multithread any script in your arsenal. Enjoy!

The full text for the script I use is as follows:

  58 Responses to “Multithreading with Jobs in PowerShell”

  1. Big thanks !!

  2. The above script is great and I will defniitely use it for my needs.
    However, I think something is missing there in order to make it bulletproof:
    * In 1st part, you start the jobs with threads limits (20 in the example). If limit is reached, script is paused.
    * In 2nd part, scripts that run for too long time (MaxWaitAtEnd) are killed.

    So… I guess this is the problem: what if you have more than 20 jobs, hanging/freezing ? The timeout section of part 2 will not be reached, and no job at all will be killed…
    I guess timeout management should be managed WITHIN first part instead of “after”
    Of course, we can increase max number of threads to bypass the problem, but you will be agree, I guess, this is not a ‘nice’ solution.

    Thanks anyway, that script will definitely help me !

    • I guess my only general comment here is that you probably have some work to do on the child script if that many are failing. I would troubleshoot the child to get it working well before running on a large number of threads.

  3. I managed to get the script to pass the parameter, included a

    $computer=$args[0]

    at the top of my script file!

  4. Hi,

    I am trying to get this multi-threading to work for around 2000 client machines, I want to query computer information on all clients and report back to a database. I have the script that collates computer info and writes to a db but I need to be able to multithread it to get quicker results.

    I have tried using this example but I am getting the error:

    Cannot validate argument on parameter ‘ComputerName’. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

    Basically I think the multithreading script is not passing the $computer variable onto my script file

    Start-Job -FilePath $ScriptFile -ArgumentList $Computer -Name $Computer | Out-Null

    so my $scriptFile also uses the $computer variable to get computer info for that remote computer but liker I said the multi thread script is not passing the variable through.

    Any ideas what I am doing wrong?

  5. Hi Ryan,

    Did you ever get a problem where $(Get-Job -state running).count gives an intermittent error of “the property ‘count’ cannot be found on this object”?

    • This will happen when Get-Job returns only one job. The reason is that “count” is a property of an array or collection, but when there is only one job, the returned value is not in fact array or collection. You can force it into an array by using @().

      • Sorry, should I clarified that. I know about the array versus single object and tested wrapping it in @() but still got the error (plus there was multiple jobs there anyway). I’ll carry on testing anyway…thanks Ryan.

  6. I would like to thank you for your blog and explanation on the code for multi-threading scripts. I found this very useful and it allowed me to learn how to multi-thread scripts and understand how Jobs work in powershell. This changed the run time of a script that will touch over 400 servers across 3 domains grabbing wmi and dell warranty information. In the original script it would take about 3 to 4 hours to pull the information now I get it done in about 30 minutes depending on the response time from the dell web site. Great help thank you.

  7. Thanks for the blog post, this was really helpful! Do you know if it’s possible to get more detailed “state” information from a running job? I’m working on a script to migrate a whole mess of VMs from one environment to another and then do a bunch of processing/cleanup on each VM. I’d like to move onto the next VM on my list once the migration portion of the script has completed for the prior VM (but allow the processing/cleanup to just continue in the background). So, rather than reporting state as “running” is there a way to put flags in my script to make the state more specific (eg. “migrating” vs. “cleaning” vs. “done”)?

  8. Hello Ryan,

    Thank you for the great work. I wrote a script using your logic to hit each server in a list and check if it is online or not.My script works fine. But I want to split my output good/bad servers into two different text files, I cannot hard code it in my script as obviously resource might be used by another thread. If I write it in thread logic my output (both good and bad) will be in the same file. Any idea how can i nicely format only desired output.

  9. Hello Ryan,

    The script is great and thank you for the detailed explanation. I am wondering if it is possible to split the output good/bad to different files. i.e If I run my script across 1000 servers to know if they are online or not, Is there a way to nicely format and direct offline servers alone to OfflineServers.txt. I am not able to do it from my script itself as obviously the resource would be in use by another thread. Once again thank you for the great work you did!!

    Regards,
    Pradeep JC

  10. Thanks for some of the info Ryan.

    Your While loops moderating the open threads is pretty cool.

    I stole them from you!! 🙂 thanks

  11. Ryan, I too am greatly benefiting from your code. Thank you!

    I have a problem, though, and I’m wondering whether you have insight into possible causes. I don’t know if the culprit is PowerShell, Windows, SQL Server, or what. We’re logging everything, but there aren’t any clues.

    I am looping through a dictionary of database names and kicking off a job for each one that sets the database in single-user mode, restores it from a backup on the network, and returns it to multi-user mode. I am incrementally testing it against two development servers, and each one restores 10 of 11 databases and brings them back online. The 11th is left in a restoring state.

    The $WaitAtEnd isn’t exceeded nor even nearly reached. I wait for one sever to complete and Receive-Job(s) before staring the next. I thought possibly that the job ends before the restore does. But again, all the restores complete, and we execute post-restore SQL operations on them, except the last one.

    Any ideas?

  12. Any chance you could post a tutorial on doing the same thing but with the -scriptblock command. I am trying to create a framework that will be the basis for all the scripts we use in our office and want to implement start-job, but having considerable amount of issues. Been messing with my script for a few days now, using your tutorial and can”t get anywhere really.

    • If your running a ForEach loop that contains a script appending -asjob effectively puts your -ScriptBlock in its out Runspacepool.

      foreach($computer in $computers){

      Copy-Item “c:yourlocalscript.ps1” -Destination “\$Computerc$tempc”

      Invoke-Command -ScriptBlock {powershell.exe c:tempcscript.ps1} -ComputerName $computer -AsJob

      }

  13. Thanks, dude. This script works a treat!

  14. […] There are times where running parallel tasks in PowerShell would be handy.  Unfortunately, while Workflows can run foreach –parallel, there’s no built in function to handle this in PowerShell.  I came across a few tasks where parallelization would save time, and figured it was time to dive into the subject. I ended up with three functions that heavily borrow from the work of Tome Tanasovski, Boe Prox, and Ryan Witschger. […]

  15. I used a different $ScriptFile which took less time to execute than a GWMI, I was using a ping script. I found the parent script was finishing before the last item returned it”s results. If I ran a get-job I”d see HasMoreData = True on the last job. Adding this check into the while statement fixed the issue:

    While ( ($(Get-Job -State Running).count -gt 0) -or ($(Get-Job -State Running).HasMoreData -eq $True) ) {

  16. The idea of using Start-Job for multi-threading seems to be clean and easy, but I am not successful in passing arguments. I have a move-files.ps1 that moves files accepting source and target locations as arguments. When I try to multi-thread like this:

    $locations = @(“c:temp* j:temp”,”c:temp_2* j:temp”)
    ForEach($location in $locations){
    Start-Job -FilePath c:move-files.ps1 -ArgumentList $location
    }

    the Get-Item in move-files.ps1 generates errors:

    The filename, directory name, or volume label syntax is incorrect.
    + CategoryInfo : ReadError: (C:temp_2:String) [Get-Item], IOException
    + FullyQualifiedErrorId : DirIOError,Microsoft.PowerShell.Commands.GetItemCommand
    The filename, directory name, or volume label syntax is incorrect.
    + CategoryInfo : ReadError: (C:temp:String) [Get-Item], IOException
    + FullyQualifiedErrorId : DirIOError,Microsoft.PowerShell.Commands.GetItemCommand

    How should I provide the source and target locations via Start-Job to make it work?

    • Never mind.
      This was taken care of by using two arguments:

      $locations = @(“c:temp*”,”c:temp_pyx*”)
      $target = “j:temp”

      ForEach($location in $locations){
      Start-Job -FilePath c:move-files.ps1 -ArgumentList $location, $target
      }

  17. Was so happy to find this. Hoping this will greatly reduce some processing time on a couple tasks that are eating up huge chunks of my day to run right now. Great tutorial Ryan! Thanks for writing this!

  18. […] in PowerShell V2 | Ryans PowerShell Blog Posted on 2012/06/19 by mschinz Multi-Threading in PowerShell V2 | Ryans PowerShell Blog. This entry was posted in Development, Powershell, Scripting by mschinz. Bookmark the […]

  19. Hi Ryan, thanks for the info! This works for some of my smaller projects, however, I have a very very large dataset that needs to be passed. Some 10,000+ computers that I need to run reports against and run a few other administrative tasks. The script runs perfectly but has a fatal flaw. the $max_threads doesnt throttle the creation of threads, only how much can be thrown into processing at once. At about 2500 threads powershell is overloaded and just closes; no warning no errors.

    The way I thought I”d work around this was by breaking the jobs down into smaller chunks to the job. However, this seems problematic. Doing some testing it appears that the script is running (i.e. job is running) however, it”s been next to impossible to have the results sent back from that job.

    Essentially I start with a large list; break that down into only what I want to work on (sometimes as small as only a few hundred computers or so) then break these out into “chunks” output the chunks into a csv, then pass the csv as a parameter to the job. This I can tell is working only because if I screw up the CSV, the script complains about parameters not being found or that -ComputerName could not be validated. However, I”m at a loss of actually getting my array of custom objects back from the job to be combined in the main script and ultimately be output to csv or excel. the output of the script is an array of objects.

    • Akk! NM

      There”s something wrong. I cant get your script to work at all. The jobs hang. using your script and modifying what the scriptfile does, I can get very simplistic things to work; but using [adsi] or anything involving connecting to a remote computer seems to fail. Which is frustrating because it”s not seeming very consistent. Test-Connection -computer $computer works and gives me the results of the ping test, however, Get-Process -computer $computer fails.

      Are there limits to what I''m "allowed" to do in a job?

      • It sounds to me like you are running under a non-privileged account. Test-Connection will work, but Get-Process will not. Both of these items are fine to use within a job.

    • Instead of breaking up the work to smaller chunks, try removing the completed jobs as new jobs are created. The only issue would be if you need assess to the job results, as that requires more logic…

      #”Starting job – $Computer”
      Remove-Job -State Completed

  20. Great Tutorial Ryan,

    What License is this script released under?
    I want to incorporate it into an openSource Script I am writing for log parsing. https://github.com/lhaig/pslogreader.

    • My scripts are available to everyone to use. I would ask for a little credit if re-posting, but it certainly isn”t required.

  21. Hi Ryan,

    It only works for me when my script file dose simple Ping. When it query WMI object for example

    Get-WmiObject
    -query “SELECT *
    FROM win32_service
    WHERE ( name LIKE ”%SQL%”
    OR name LIKE ”%MSOLAP%”
    OR name LIKE ”%ReportServer%”
    OR name LIKE ”%MSDtsServer%”)
    AND State = ”Running”
    AND StartMode = ”Auto””
    -computername $Computer

    All thread hang.

    I think it is becaue the query result can”t return in a proper time, which causes the hang.

    Have you ever seen this? And what will be the solution?

    Thanks

    • Get-WMIObject can be a picky command. It will hang forever if the WMI on the client is having issues. Does the thread hang if you run it aside from the multi-threading script?

  22. Hey Ryan,

    Excellent post. I, too, have been trying to get my head around PS multi-threading; and failing! But your guide has made it clear to me.

    Thanks!

  23. Very good post ….. Just want to point it out two corrections

    line 52 …. If ($OutputType = “Text”){ should be If ($OutputType -eq “Text”){
    and
    line 60 ….ElseIf($OutputType = “GridView”){ should be ElseIf($OutputType -eq “GridView”){

    Raheel

    • You are absolutely correct. I had fixed my own script some time ago, but I”ve updated my post to reflect your finding as well.

      Thanks!

  24. Ryan – well done. Thanks for taking the time to put this down.
    (I”m calling 15+ long-running web services)

    Would be happy to see a revision with Stanley”s inclusions – then you”ve arrived 🙂

    Mark

  25. Great script Ryan, I”m running a health script on over 1000 servers and outputting to excel. Multithreading would be the answer to get the runtime down from 8 hours to 2 hours but cant seem to get my head around outputting the multithreaded script to excel. Any ideas?

    • Outputting to Excel can be a bit complicated. It essentially means making an object for Excel, then the workbook, then the sheet, then each cell, filling out the info, then moving to the next cell, filling out info, etc, etc then using commands to show the Excel window and save the file. It”s possible if you are really trying to nicely polish your script. I typically do this for scripts that send emails to my team; however, if you are just trying to do something quick in a table format then simply use PowerShell V2”s Out-GridView Cmdlet. It creates a table which can be sorted, filtered and searched, and if you want to move it into Excel, then you can copy and paste (not via a right click, but Ctrl-C works).

      To get you started:
      Get-Process | Out-GridView

      • Try using aspose.com. Great component for any Excel manipulation you may need to do on the server. They offer the component for free with a disclaimer page if you need it to just test. Try it with LINQPad. It”s great.

        A thank to Ryan for this multi-threaded example. I”m using it for Essbase calc scripts and it is just so nice to take exports that were running for 8 hours and dump them in 4 minutes with scripting the dimensions of the dump. Very helpful.

    • I realize this is a little late, but piping your output to export-csv also works well to quickly get output into excel. Most people with MS Office have .csv files default to open with Excel.

      $myoutput | export-csv -notypeinfo myfilename.csv

  26. Hrm the $(Get-Job -state running).count doesn”t seem to be actually providing an integer… or anything

    • I would say that this likely means that you are currently not running any jobs. If the return from Get-Job is not some type of array then it won”t have a property called “.count” which means it will be nothing. You can force it to be an array by “typecasting” the variable like this:

      @(Get-Job).count

      Notice that the $ became an @ to force the information into an array.

  27. awesome, thanks a lot for putting this together!

  28. This is by far the best multi-threaded script yet. I looked forever for something that would allow me to loop through 1000+ PCs to refresh their machine policy for ConfigMgr and this does the trick nicely. Thank you for sharing.

  29. This, however, only works when calling a script, not a code block, so we have to also use the argument “FilePath” and provide a second PowerShell script.

    I”m not sure the above is 100% accurate. I used your code sample and was able to to write a ScriptBlock passing the same parameters to and also using the -ArgumentList to pass the pertinent variables to my scriptblock and it worked just fine.

    • [cci_powershell]start-job {get-process} -ArgumentList $ProcessName[/cci_powershell] will work, yes, because you are passing a argument to a command; however, you can””t pass a variable to your codeblock to get something like [cci_powershell]start-job {get-process -ComputerName $Computer} -ArgumentList $Computer[/cci_powershell] so I find that this method is VERY limited.

      • It is possible to pass variables into unnamed code blocks by using the param function. The example you gave would look like start-job {param($Computer); get-process -ComputerName $Computer} -ArgumentList $Computer

        If you create an empty array like $jobs = @() and then go $jobs += start-job ... you can use the $jobs array instead of get-jobs so that the script will only wait for the jobs created by this script. And you can put $jobs | remove-job to remove the jobs created.

        Thanks for the great article

        • Stanley,

          Thank you so much for clearing up the issues I”ve had with passing variables into a script block. I think you have shown a very easy solution that makes me kind of hit myself upside the head! Also a great hint with tracking your jobs in an array.

        • Great post I learned a lot…

          Before I hit this comment I was wondering why code is assuming u have only one powershell job running one same job….

          Thus I assign job name $Compname + DBNetworkBackup also to ensure jobs can be uniquely identified

          I created function fnGetDBNetworkBackup{Get-Job -Name *DBNetworkBackup} to filter out a particular job threads… and replaced Get-Job with this function everywhere in code

          code is working great….

  30. Multi-Thread within Multi-Thread ?

    Is there a way you can multi-thread within a multithread? I have three scripts 1 that collects the reports I want to run, second that collects the servers I want to run them on, and the third actually runs the report against the server. I have to do it this way as any server could hang the entire process..

    I have your multithreading set up (Your PROCESS IS AWESOME BY THE WAY) which runs fine except it doesn”t wait for the second script to finish before moving on to the next thread (it just rips through and runs all the threads). The second script however does wait for the threads to finish.

    So how do I get the 1st script to wait on the 2nd script before continuing onto the next thread?

    Thanks in advance.

    • Hi Ralph, thanks for the comment, I appreciate it.

      I am not sure I fully understand what you are asking. If you are waiting for one script to finish before starting the next, then you aren”t really mulithreading, right? If that is what you are looking for then a simple for loop should suffice. The point of starting all the threads is so that several running at once will make things go faster.

      To answer your question directly, though, I think the script will wait for the single child thread to finish if you set my $maxthreads variable to 1.

  31. Wow ! This is really amazing work and well documented. Thank you for taking the time to write this out.

  32. Hi Ryan

    Nice Job this Code works well. I use it to send distributed commands over several unix servers! Good work!

    Kindly regards,
    Flo

  33. hi guy”s very nice job

    i use you”re script to launch multi command launch in vmware station but there is some probleme
    the first :
    when we have less than 2 station there is a 0 division error
    next one :
    vmware snapin its very heavy and start-job create a new runspace each time … and its hard to debug when a job crash … or a freezing job

    after there is another solution throttlelimit i think …

    C:PS>$jobWRM = invoke-command -computerName (get-content servers.txt) -scr
    iptblock {get-service winrm} -jobname WinRM -throttlelimit 16 -AsJob

    thanks

  34. I am using your code for multithreading, but I get the following error:

    U:>powershell -c “c:/wamp/www/serverstatus/MultiThread.ps1”

    WARNING: column “Command” does not fit into the display and was removed.

    Id Name State HasMoreData Location
    — —- —– ———– ——–
    1 Job1 Running True localhost
    ccwelton-mercy

    Unhandled Exception: System.ObjectDisposedException: Cannot access a closed file
    .
    at System.IO.__Error.FileNotOpen()
    at System.IO.FileStream.Write(Byte[] array, Int32 offset, Int32 count)
    at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
    at System.IO.StreamWriter.Write(Char[] buffer, Int32 index, Int32 count)
    at System.IO.TextWriter.WriteLine(String value)
    at System.Management.Automation.Remoting.OutOfProcessTextWriter.WriteLine(Str
    ing data)
    at System.Management.Automation.Remoting.Client.OutOfProcessClientSessionTran
    sportManager.CloseAsync()
    at System.Management.Automation.Remoting.Client.BaseClientTransportManager.Fi
    nalize()

    Any ideas?

    • It looks like the multithreading portion of the script is working and that the error that you are getting is actually with the rest of your script. It is difficult for me to tell you any more without seeing the code, but based of the modules faulting and the error received, it looks like you are trying to open a file while remoting and then it is being closed before you can actually use it.

      I believe the threading it working because it is listing your job, named job1. I believe the rest of the output there is the contents received from a get-job command.

Leave a Reply to Dave Cancel reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">

(required)

(required)

This site uses Akismet to reduce spam. Learn how your comment data is processed.