Desired State Configuration - Part 1

I have been diving into Powershell Desired State Configuration lately, as the technology is now a few years old and a little more mature, especially with the coming of Powershell 5. Many in the industry named this technology the technology that you should learn last year or the year before that already and not to mention that Microsoft keeps hammering on it that you should learn about it.

So what is Powershell DSC? In its core it is a configuration management platform. It allows for systems and applications to be installed and configured by means of a configuration document, and can manage configuration drift. Configuration management tools are not new, and have been around for a while with other tools like Puppet, Chef, Ansible, Salt and others.They serve the same purpose with varying features, but are mostly catered to Linux platforms. The same way that Powershell was an answer to having a proper commandline automation tool for Windows that supported its .Net and object oriented nature, DSC is Microsoft's answer to Configuration Management. But Microsoft went further by making DSC a platform that other tools use. So instead of competing against companies like Pupped and Chef, Microsoft has actively engaged with these companies so they could use the technology to better automatically configure Windows systems and applications.


Powershell and DSC inventor Jeffrey Snover describes DSC as captain Picard on the Starship Enterprise ordering Riker to go to that distant planet by saying "Make it so number one", and Riker executes that order: Picard desires something, and Riker "makes it so". DSC is basically the same thing inside your computer: With DSC, you desire something to be in a certain state through a configuration file and the local agent (called the Local Configuration Manager a.k.a. LCM) makes it so. This LCM is not all knowing in itself, but it uses things called DSC Resources to do the tasks you want it to do. These resources are now rapidly being developed by both Microsoft as well as the community around it, but if you can't find the thing you want to have configured, you can create them yourself. The main source for resources is the Powershell Gallery, and Powershell 5 has got the PowerShellGet module which now lets you install modules straight from this repository, just like you would use apt-get from a linux repository.

What makes DSC so cool?

• You can deploy a system complete with applications and settings, nuke it and deploy the exact same machine (which makes devs happy because your dev matches your prod environment exactly)
• Your OS/application is as you documented it in your configuration file, and configuration drift can be corrected by DSC
• The DSC configuration is a Powershell based script, so you can use Powershell for variables and such, and do troubleshooting with it too
• DSC can be used not only for Windows systems, but is slowly making its way to hardware devices too (e.g. Cisco Nexus)
• There's linux support too, but as far as I've seen it, that's a bit of a work in progress. Cool nonetheless. 
• Puppet/Chef/Salt/Ansible support the use of the DSC resources!
• Windows Nano Server doesn't support Group Policies, but it supports DSC instead

There are more cool things about it, and DSC resources are added daily to make it even cooler. In another post I will go over the more technical details.

Surpress SSH and shell warnings on ESXi

Small oneliner to surpress the warning you get when you enable SSH:



Like so:

Get-VMHost | Set-VmHostAdvancedConfiguration -Name UserVars.SuppressShellWarning -Value 1

Wait-Job, how nice

A few posts ago I had a bit of a tacky way to wait until a job is done:

While (Get-Job -State "Running") {

    Write-Host -ForegroundColor Yellow "Running..."

    Start-Sleep 1

}

Turns out, there is a much more elegant function to do this:

Wait-Job -State Running

or as the built in example shows:

Get-Job | Wait-Job

This command waits for all of the background jobs running in the session to complete. Nice!

Beautify HTML output from ConvertTo-HTML

Quick one to write down before I forget it again, and have to search the interwebs again:

Whenever you use ConvertTo-Html, the output is usually.. meh. However, you can add some CSS styling and include it in the converting process:

$Header = @"

<style>

Body{background-color:white;font-family:Arial;font-size:10pt;}

Table{border-width: 1px; border-style: solid; border-color: black; border-collapse: collapse;}

TH{border-width: 1px; padding: 2px; border-style: solid; border-color: black; background-color: #cccccc;}

TD{border-width: 1px; padding: 5px; border-style: solid; border-color: black; background-color: white;}

</style>

"@

Get-Process | ConvertTo-Html -Head $Header | Out-File C:\temp\process.htm

Which gives a little more oomph to your HTML output:



You can also put the CSS code in a separate file (e,g, D:\table.css), and include the CSS with

ConvertTo-Html -CssUri "D:\table.css"

Getting an overview of reboots revisited

In my previous post I mentioned invoke-command using the -AsJob option. But then I saw a post from Tommy Maynard that made things even quicker and certainly a bit easier.

Instead of doing a "foreach server in serverlist" loop to run the invoke-command scriptblock, you can do an invoke-command on the entire serverlist in one go!

$ServerList = Get-Content C:\Scripts\ServerList.txt.txt

$block = {Get-EventLog System | Where-Object {$_.EventID -eq "1074" -or $_.EventID -eq "6008" -or $_.EventID -eq "1076"} | Select Machinename, TimeWritten, UserName, EventID, Message}

Invoke-Command -ComputerName $ServerList -ScriptBlock $block | ft -AutoSize -Wrap |Out-File C:\scripts\ServerRestart.txt

This certainly cuts down on scriptsize :-)

How much time you save depends on the amount of servers, but I tested it with 5 VM's, and saw some surprising things in my test: Invoke-command feeding the serverlist as described above actually took longer than invoke-command using the -Asjob option.

Usually I get 14 seconds for the old fashioned "foreach" test, then about 6 seconds for the invoke-command feeding the serverlist, and about 2 seconds for the invoke-command with -Asjob.

Here's the test I ran a few times after another:

$ServerList = Get-Content C:\Scripts\ServerList.txt.txt

# Test 1: Old fashioned way

Write-Host -ForegroundColor green "Foreach server in serverlist"

Measure-Command {foreach ($server in $ServerList){Get-EventLog System -ComputerName "$server" | Where-Object {$_.EventID -eq "1074" -or $_.EventID -eq "6008" -or $_.EventID -eq "1076"} | ft Machinename, TimeWritten, UserName, EventID, Message -AutoSize}}|select Seconds|ft -AutoSize

# Test 2: Doing the test with invoke-command, feeding the serverlist in one go

$block = {Get-EventLog System | Where-Object {$_.EventID -eq "1074" -or $_.EventID -eq "6008" -or $_.EventID -eq "1076"}}

Write-Host -ForegroundColor green "invoke with serverlist"

Measure-Command {Invoke-Command -ComputerName $ServerList -ScriptBlock $block}|select Seconds|ft -AutoSize

# Test 3: Doing the test with "Job" functionality

Write-Host -ForegroundColor green "Job Functionality"

Measure-Command {

foreach ($Server in $ServerList){

Invoke-Command -ComputerName $Server -ScriptBlock $block -AsJob

}

While (Get-Job -State "Running") {

}    

Get-Job| Receive-Job |ft -AutoSize

Get-Job | Remove-Job

Write-Host -ForegroundColor Yellow "Done!"

}|select Seconds|ft -AutoSize

The weird thing however, is if I swap test 2 and 3 around, invoke-command using -Asjob consistently works slower than the invoke-command with serverlist.  The old fashioned test still took  14 seconds, but the invoke-command with -Asjob took 2 to 6 seconds, and invoke-command feeding the serverlist consistently took 1 second.

Last try:

So I wasn't sure why the last piece of the script went faster than the middle part, but swapping it consistently made the latter one faster. So I ran each of the two tests in a loop for about 10 times in a row, and lo and behold: The invoke-command using -Asjob was running at 1-2 seconds, and feeding the serverlist in one go ran consistently for 1 second. For both tests, the first run took about 6 seconds, and this could be the same delay that is apparent in the test I listed above.

I assume that somehow a connection is made using Invoke-Command, and this session somehow stays alive.

In the end: Feed invoke-command the serverlist as a whole. It's easier to manage. and a little bit faster consistently.

Getting an overview of server reboots

A quick one that I wanted to keep for reference, after a forum post on powershell.org. Someone wanted to have a list of reboot times from the eventlog and put it in a text file, so I commented:

$ServerList = Get-Content "C:\Scripts\ServerList.txt"

foreach ($i in $ServerList){

    Write-Output $i "`n"+"==========================" | Out-File -FilePath c:\scripts\ServerRestart.txt -Append

    $Output = Get-EventLog System -ComputerName "$i" | Where-Object {$_.EventID -eq "1074" -or $_.EventID -eq "6008" -or $_.EventID -eq "1076"} | ft Machinename, TimeWritten, UserName, EventID, Message -AutoSize -Wrap

    $Output | Out-File -FilePath c:\scripts\ServerRestart.txt -Append

}

This obviously only works if you have enough credentials to access the remote server. This in itself should do the job for not too many servers, or lots in due time, because it runs on one server at a time.

If you want to kick things up a notch, and parallelize things, you can run it as a job with Invoke-Command with "-Asjob":

$ServerList = Get-Content "C:\Scripts\ServerList.txt"

$block = {Get-EventLog System | Where-Object {$_.EventID -eq "1074" -or $_.EventID -eq "6008" -or $_.EventID -eq "1076"} | ft Machinename, TimeWritten, UserName, EventID, Message -AutoSize -Wrap }

foreach ($Server in $ServerList){

Invoke-Command -ComputerName $Server -ScriptBlock $block -AsJob

}

While (Get-Job -State "Running") {    

    Write-Host -ForegroundColor Yellow "Running..."

    Start-Sleep 1        

}    

Get-Job| Receive-Job |Out-File C:\scripts\ServerRestart.txt

Write-Host -ForegroundColor Yellow "Done!"

Which will give the following output:

Get-Job| Receive-Job |Out-File C:\scripts\ServerRestart.txt

Id     Name            PSJobTypeName   State         HasMoreData     Location             Command                  
--     ----            -------------   -----         -----------     --------             -------                  
35     Job35           RemoteJob       Running       True            localhost            Get-EventLog System | ...
37     Job37           RemoteJob       Running       True            127.0.0.1            Get-EventLog System | ...
Running...
Running...
Running...
Running...
Running...
Done!

This should speed things up a bit, as now all machines run it simultaneously. The only thing that doesn't get done that way is the divider-line at the top.

To be totally neat, you should clean up the jobs too:

Write-Host -ForegroundColor Yellow "Clearing Up completed jobs"

Get-Job -State "Completed" |Remove-Job

Now you should be able to do a Get-Job and see if there are any failed jobs. Yes yes, that could be automated too..

SSH via Powershell

In hosting environments, you rarely see only one flavor of operating system, so to be able to manage linux systems with Powershell via SSH is a big help. With Windows Server 2016, SSH connectivity will be built in, but until that time you can quite easily get it right now.

Step 1:
Go to https://sshnet.codeplex.com and download the .Net 4.0 binary from the downloads section and place the "Renci.SshNet.dll" file somewhere where you can access it. In my example I put it in C:\Scripts

Most likely Chrome will ask you whether it is safe to download, obviously say "Keep":







Step 2:
Go to the file, rightclick and select Properties, then unblock the file.



Step 3:
Here comes the example code. Quick explanation: First you load the DLL, then you create an object with the DLL file (I'm sure there's a technical programmy way of saying this, but I have no clue), then you suddenly have all these cool features like "Connect" and "RunCommand". It is not an interactive thing (as far as I know), and you only have a single command you can run, so you can't create a scriptblock and run it all at once.

#Load the SSHNet Library

[void][reflection.assembly]::LoadFrom( (Resolve-Path "C:\Scripts\Renci.SshNet.dll") )

#The variables you need to log on

$server = "Server01"

$login = "someuser"

$password = "somepassword"

#Call the SSHNet Library as an object

$SshClient = New-Object Renci.SshNet.SshClient($server, 22, $login, $password)

#Connect to client using SSH

  $SshClient.Connect()

    

   if ($SshClient.IsConnected) {

    Write-Host "Connected to " $server

    #Run the command on the linux host

    $cmd = $SshClient.RunCommand('ls -l')

    

    if($cmd.Result -ne '')

    {

        Write-Host Output from $server

        Write-Host $cmd.Result

    }

    

  }

$SshClient.Disconnect()

$SshClient.Dispose()

Which gives the following output:

Connected to  Server01
Output from Server01
total 12
-rw-r--r-- 1 Akos users 136 Feb 21  2008 local.cshrc
-rw-r--r-- 1 Akos users 157 Feb 21  2008 local.login
-rw-r--r-- 1 Akos users 174 Feb 21  2008 local.profile

Of course this is a simple example, but you can load in a CSV file with server, login and password info, and then have a command run on however many servers as you want. It's always fun being able to tell your boss that you're doing that thing on 50 servers, and then getting a cup of tea while your script is doing your work for you. This SSHNet client can also upload files using sftp but I haven't tried that yet.