Update: This method is faster but some anti-virus software hate it. I’ve since reverted our method of import to an all-in-one ps1 👎
Considering dbatools now has over 500 commands, we are always eager to keep an eye on the speed of our import process.
As outlined in the blog post decreasing module import times, we’ve taken multiple approaches to reduce the import times, including combining all commands into one large allcommands.ps1 file.
Last week, I took it a step further.
I don’t know the in-depth internals of Import-Module, but I know that importing a DLL filled with C# cmdlets is extremely fast. For instance, Microsoft’s SqlServer module imports 100 commands in less than a second. Sometimes it’s closer to half a second!
I wondered if we could somehow compile our commands into a C# binary but that seemed far-fetched. One thing we could do, though, is use compression! It works for SQL Server backups, could it work for PowerShell?
Turns out that the approach worked! I believe this is due to the performance benefits of streaming and reduced I/O. Note this technique is part of a multi-pronged approach which includes runspace and not using Get-ChildItem.
Here’s how I did it. First, each time I publish the module, I rebuild allcommands.ps1 then zip it. This reduces the size of our module on disk a bit as well, too, since the uncompressed ps1 is over 5MB and the zip is less than 1MB 👍
Set-Content-EncodingUTF8-PathC:\github\dbatools\allcommands.ps1-Value"### DO NOT EDIT THIS FILE DIRECTLY ###"
What’s the trade off? More CPU usage on your part for the moment it takes to stream the file (though you’ll save on I/O) and for me, it’s an extra (automated) step.
Ultimately, this approach shaved off about a third of our import time. If you’re looking to squeeze as much speed out of your import as possible, compression can help. And don’t forget, if you have super slow imports, it may be your Execution Policy.
Recently, I had a couple colleagues tell me that their PowerShell code sucks.
One of them is a beginner who manages a large SharePoint farm and the other is more advanced and wrote a few commands that saves me hours of work each week.
See the disconnect there? Their PowerShell code, which helps manage Enterprise platforms and saves hours of work a week, not only doesn’t suck, it’s actually awesome.
your ps is actually awesome
Their PowerShell is awesome because it works. Their PowerShell code is awesome because they’re using it. It’s also awesome because the time they invested in creating code that:
Will save people time in the future
Helped them practice a valuable skill set
Keeps them relevant
Will look fabulous on their resume
Nobody goes from being a beginner to an expert overnight, and it’s important to allow ourselves the time to learn.
my own story
I feel like I’ve got code that sucks, too. In particular, I’m tortured by one of the commands I used to be super proud of, Import-DbaCsvToSql. I die inside when people talk to me about it, but I leave it in dbatools because people love it. It’s useful and, enough of the time, it works 😉
I learned a ton when I was creating that command, and even did a presentation about it.
The adventure of creating that command (and Bruce Payette attending my session!) was so exciting, it prompted me to dive even deeper into PowerShell.
Yet I fight back against my impulse to remove Import-DbaCsvToSql from dbatools. I literally have to resist the urge, but I do it by reassuring myself that:
One day I’ll rewrite it
Until then, some people really find it useful
Import-DbaCsvToSql is embraced because it’s fast. But it’s fast because it has no error handling. No error handling can be okay if you’ve got some perfect input, but input is rarely perfect.
The command simply doesn’t work enough for me, yet at the same time it tries to do too much. It totally stresses me out.
And, y’all, I showed this code, along with Start-SqlMigration.ps1 to Lee Holmes of the PowerShell team, all proud. He was so kind in response. I didn’t follow best practices, but he didn’t chide me. He said I was doing great things, and should keep on writing code and continue to improve along the way.
it’s all about adoption
Ultimately, I believe that Lee cared most that PowerShell had an enthusiastic adopter, even if I still had a lot to learn.
And I now realize that adoption is what’s actually important. Adoption will lead to that extreme excitement that I feel when I execute something and announce to my coworkers that I’m chillin because PowerShell is doing all my heavy lifting. From there, I’ll use that momentum to make my code even better.
Then I’ll invite my coworkers over to see my improved code, and we’ll celebrate. Then they’ll invite me over to see their improved code, and we’ll celebrate.
And then we’ll all live happily ever after, with our awesome code.
I am beyond honored that the community and the PowerShell team felt my work deserved to be acknowledged among the other highly decorated community contributors. And there’s something extra magical about being acknowledged not just by the PowerShell Team, but also by my peers. It ranks in the top achievements of my life.
I keep my PowerShell Community award right next to my MVP award, and I smile every time I hold it. Thanks to all involved for the huge honor 💞.
Today’s blog post is part of T-SQL Tuesday. T-SQL Tuesday is the brainchild of Adam Machanic. It is a monthly blog party on the second Tuesday of each month. Everyone is welcome to participate.
This month’s T-SQL Tuesday, hosted by my buddy and YouTube star Bert Wagner (blog | twitter), is all about the SQL Server tools I use and love. This post is about PowerShell, of course.
dbatools is such a fun toolset to work on, but specifically, I can no longer live without Start-DbaMigration. Even in smaller shops, migrations are often required and they are always a lot of work.
At least they used to be, before I built the command that started it all: Start-DbaMigration. Start-DbaMigration is an instance to instance migration command that migrates just about everything. It’s really a wrapper that simplifies nearly 30 other copy commands, including Copy-DbaDatabase, Copy-DbaLogin, and Copy-DbaSqlServerAgent.
Here is what it looks like in action:
So simple. To migrate an entire instance, all I had to do was execute the following:
How cool is that?! It is such a relief that I no longer have to dread migrations. I don’t have to go find sp_help_revlogin and execute it then copy the stuff over. I don’t have to figure out the default paths of whatever. I don’t have to worry that I’ll copy over a job but not its associated schedule.
Everything is just done for me, and I can sleep better at night 😴
Ever since I was a baby DBA working for a company with a small budget, I’ve always wanted something like dbachecks. dbatchecks helps automate the checklists that we gotta go through every day, week, month, quarter, year and so on.
This used to be a manual pain, but now it’s automated, open-source, free and gorgeous 💅.
That whole module is some awesome code. And you can read about it here.
Thanks for joining me for this month’s T-SQL Tuesday. Sorry I never write about things in-depth.
I was first introduced to open-source on IRC back in 1996. I joined the #toriamos channel on undernet, which was managed by an eggdrop bot named [iCKy].
[iCKy] was an open-source TCL and C-based program that did cool stuff for us, like auto-ban users if they were flooding the channel. There was even a party line that sort of worked like a Linux shell.
This all looked like hacker stuff to me, so I wanted to know how it worked. This meant I had to learn Linux, which I did. Then I had to learn about eggdrops, which I did. Then I setup a few bots on an always-online Linux server. All for $0, too awesome.
By July 1998, I knew TCL well enough to hack an IRC script together and release it. But I gave the project an awful name that would trigger spam filters so I’ll skip it and move on to my second, more mainstream open source project ;)
There used to be this open source directory called freshmeat.net, which eventually became freecode. freshmeat was leet as hell and I dreamt of being part of it.
Finally, I had a project I thought was decent enough to add and released 1.0 on September 5, 1999. SPAST, which stood for Simple Procmail Anti-spam Template, was a procmail template that I setup to filter mail being delivered to my postfix email server.
As the name suggests, SPAST was simple, which made it easy to implement and adopt. I laughed recently when I read the archived review at porkmail.org which said
Not the best possible approach, but it makes it easy to get started.
Totally valid 😂
I found it pretty cool that a couple other projects were inspired by or forked from SPAST, like SPASTIC. SPASTIC, which still exists at sourceforge, was maintained by a guy named Keith Winston and the project eventually grew to have 11 other developers!
Keith even wrote an article titled SpamAssassin vs. Spastic, which compared the two projects. Ultimately, he stopped developing SPASTIC and adopted SpamAssassin. By that time, I’d since migrated to Exchange in an effort to replicate my work environment.
Fighting spam was always a fun way to pass time, and years later in 2007, I wrote a tutorial on how to use Gmail to filter spam in Exchange instead of Postini (which was eventually purchased by Google anyway).
Considering that I’ve dreamt of having a cool open source project since The Verve’s Bitter Sweet Symphony was a chart topper, you can imagine how rewarding it’s been that dbatools has gotten so popular! And it’s a project that revolves around my two favorite things: SQL Server and PowerShell.
We now have over 110 contributors to our GitHub repository and nearly 1500 people in the Slack channel. The experience as a maintainer for a relatively successful project has been everything I’ve dreamed and more 😊
Well, if it wasn’t my very first, it is close enough. Back in 1997, I was taking a computer class at a non-progressive college in Northwest Louisiana. Most of the tech classes still teaching Fortran and the faculty pushed back hard when I asked if we could work with more modern technology.
I was disappointed with my experience there overall, but did enjoy this one class by Dr. John Barber called “Computers and Composition”. Dr. Barber was cool and the first person to introduce me to cyberpunk culture. And that’s about all I remember about the class, except for a website that I worked on as part of an assignment.
The site was called “The Q Zone” and it was built with NetObjects Fusion (and optimized for NetScape, y’all). In my mind this site was HUGE, but looking at it now, it’s only five pages. Five awesome pages of sci-fi fun.
Word and paper do not suffice anymore as hypertext and electronic writing infuses with the various powerful technologies that heighten the communication process. These technologies include the use of Java, Dynamic Hyper Text Markup Language (DHTML) and all types of animated graphics and embellishments.
So good. So 90’s. So Shoney’s.
This animated embellishment brought to you by giphy.
Although I’ve been using SSMS 2017 on my demo machine for a while, I kept SSMS 2016 on my primary workstation until today. I’ve been unsure of the new look of SSMS 2017, but now that I’ve fallen in love with the way it handles XEvents, I decided to make the switch.
Whaatup! That’s super handy. Unfortunately, you can’t run SSMS 2016 and 2017 at the same time, and figuring out how to uninstall SQL Server Management Studio 2016 has been a pain.
Sometimes, SQL Server Management Studio 2016 won’t show up in Add/Remove programs. So I tried to uninstall every bit of SQL Server 2016 since my local db engine was hosed anyway; that didn’t work. As you may recall, in 2016, Microsoft decoupled SSMS from the Engine installer so that wasn’t a big shock.
Ultimately, with the help of StackOverflow, I learned that you can uninstall SSMS 2016 by re-running SSMS-Setup-ENU.exe. If you didn’t keep this file, you can download it again here.