There are two main options to execute parallel copy of files, one is to user JFlower project (https://sourceforge.net/projects/jflower) to create multithread in NAnt, the other option is to copy the files using PowerShell. The JFlower option we will discard because it creates a third part dependency with no support and with no updates warranty. The PowerShell option to execute parallel copy can be implemented in different ways, here some examples:
- Use Copy-Item with the Invoke-Command.
$servers = Get-Content 'C:\bpi_servers.txt'
Invoke-Command -ComputerName $servers -ScriptBlock
{
Copy-Item -Path '\\machine01\_Projectos\137\Mainline\Artifacts\Latest\XML' -Destination "E:\NetDados\teste" -Verbose
}
Requirements: WinRM, PowerShell >=3, .Net Framework 4.0 in client and server
Note:
“By default, PowerShell will talk to up to 32 computers at once; if you specified more than that, it will queue them up, so that, as one computer completes, the next one in line will begin. If you have a awesome network and powerful computers, you could raise that number by specifying the -throttleLimit parameter of Invoke-Command—read the command’s help for more information.”
- User Workflow, foreach –parallel and Copy-Item together
$computers = ‘W16TP5TGT01’, ‘W16TP5TGT02’
workflow parallelcopy
{
param ([string[]]$computername)
foreach -parallel ($computer in $computername)
{
InlineScript
{
$s = New-PSSession -ComputerName $using:computer
Copy-Item -Path \\machine01\_Projectos\137\Mainline\Artifacts\Latest\XML -Destination E:\Netdados\teste -ToSession $s
Remove-PSSession -Session $s
}
}
}
Requirements: WinRM, PowerShell >=3, .Net Framework 4.0 in client and server
Note: Good for cenários were existes the need to execute a workflow that survives reboots or when execution order matters. Implemented with WWF 4.X.
- Use Start-Job together with robocopy
$startDate = (Get-Date)
Write-Host "Start Date:" + $startDate
$serverlist = @('SSMWFVNPBCOM01','SDMWFVNPBCOM01','SSMWFVINTCOM01','SDMWFVINTCOM01','SSMWFVNPBCOM01','SDMWFVNPBCOM01','SSMWFVINTCOM01','SDMWFVINTCOM01')
$jobs = foreach ($serverName in $serverlist)
{
Write-Host "Starting copy job for $serverName"
Start-Job -Name "Copyjob $serverName" -ArgumentList $serverName -ScriptBlock
{ param($serverName) if (Test-Connection -ComputerName $serverName -Count 1 -Quiet)
{ robocopy "c:\temp" "\\$serverName\netdados" teste.zip /R:0 }}
}
write-host "Waiting for job(s)"
$jobs | Wait-Job
#write-host "Job(s) output(s)"
#$jobs | Receive-Job
$endDate = (Get-Date)
Write-Host "End Date:" + $endDate
$timeSpan = NEW-TIMESPAN –Start $startDate –End $endDate
Write-Host "This script took $($timeSpan.Milliseconds) milliseconds to run."
Requirements: PowerShell >=2 no cliente e SMB protocol between cliente and server
Note: Did not find a throttle specification for start-job, we need to control throttling with code.
Discussion:
There are different ways to implement parallel copy of files in Nant.
In this specific scenario, we will choose to user Start-Job together with robocopy because our infrastructure already has PowerShell 2 or higher in every machine and SMB protocol is already configured in firewall and routing rules.