Even faster PowerShell module loading

Considering dbatools now has over 500 commands, we are always eager to keep an eye on the speed of our import process.

As outlined in the blog post decreasing module import times, we’ve taken multiple approaches to reduce the import times, including combining all commands into one large allcommands.ps1 file.

Last week, I took it a step further.

Binaries?

I don’t know the in-depth internals of Import-Module, but I know that importing a DLL filled with C# cmdlets is extremely fast. For instance, Microsoft’s SqlServer module imports 100 commands in less than a second. Sometimes it’s closer to half a second!

I wondered if we could somehow compile our commands into a C# binary but that seemed far-fetched. One thing we could do, though, is use compression! It works for SQL Server backups, could it work for PowerShell?

Yes 😁

Turns out that the approach worked! I believe this is due to the performance benefits of streaming and reduced I/O. Note this technique is part of a multi-pronged approach which includes runspace and not using Get-ChildItem.

Here’s how I did it. First, each time I publish the module, I rebuild allcommands.ps1 then zip it. This reduces the size of our module on disk a bit as well, too, since the uncompressed ps1 is over 5MB and the zip is less than 1MB πŸ‘

Next, I added the following to our module file. This code is run each time the module imports. You’ll notice that it opens the zip and streams it right in as a script block.

Before

After

Trade off

What’s the trade off? More CPU usage on your part for the moment it takes to stream the file (though you’ll save on I/O) and for me, it’s an extra (automated) step.

Ultimately, this approach shaved off about a third of our import time. If you’re looking to squeeze as much speed out of your import as possible, compression can help. And don’t forget, if you have super slow imports, it may be your Execution Policy.

Chrissy is a Cloud and Datacenter Management & Data Platform MVP who has worked in IT for over 20 years. She is the creator of the popular SQL PowerShell module dbatools, and holds a number of certifications, including those relating to SQL Server, Linux, SharePoint and network security. You can follow her on Twitter at @cl.

Posted in PowerShell

Leave a Reply

Your email address will not be published. Required fields are marked *

*