SubSonic and PowerShell

Jan01

I have been a fan of Rob Connery's work on SubSonic for a while now.  As readers of my blog know, I have also been exploring PowerShell quite a bit lately as well.  Recently, I had a need to update a DotNetNuke module to PowerShell 2.0.3.  Unfortunately, some of the APIs had changed between SubSonic 1.x and 2.x.  Because of the desire to keep the module Medium Trust compatible I don't use the build-provider and instead pre-generate my DAL.

In the past I had used the sample web app to generate the classes for just those tables needed for my module.  In this iteration, I decided to go the SubCommander route and generate the needed classes from the command-line.  The SubSonic web-site has a batch file to handle this task, but I decided to use a command line that didn't date back to my college days.  While the batch file is a workable solution, it really is showing its age, especially when compared with PowerShell.

SubCommander is an extremely powerful tool and has dozens of options and command-line parameters.  Using it can be a daunting task.  Fortunately,  using a command-line script can significantly simplify the API, and when combined with the tab expansion features in PowerShell Plus makes it trivial to use.

So I fired up PowerShell Plus and created the New-DAL function.

function global:get-sqluser($username="", $password="", [switch]$forcesecure) {
    # We are creating an object to which we'll add custom properties
    $user = New-Object object | select-object UserName, Password
    
    if ($username.length -eq 0) {
        # No username was specified, so we should use Get-Credential to prompt for a user
        # We also define a default username in order to suppress console output
        # The results are added as synthetic properties to the PSObject we created above
        $cred = Get-Credential "SqlUser"
        $user.UserName = $cred.GetNetworkCredential().Username
        if ($forcesecure) 
        {
            $user.Password = $Cred.Password
        }
        else
        {
            $user.Password = $cred.GetNetworkCredential().Password
        }
    } else {
        $newpassword = $password
        
        # If we are using secure passwords, then we need to convert our string to a securestring
        if ($forcesecure) 
        {
            $newpassword = New-Object System.Security.SecureString
            [char[]]$password | for-each {$newpassword.AppendChar($_)}
        }
        
        # In this case we can just create synthetic properties using the values passed to the function
        $user.UserName = $username
        $user.Password = $newpassword
    }
 
    # Return our synthetic object
    $user
}
 
# This function uses SubSonic to generate the Data Access Layer.
function global:New-Dal(
    $server = "localhost",
    $db,
    $username = "",
    $password = "",
    $namespace = "Test" ,
    $lang = "vb",
    $BuildPath = "D:\Batch\DAL",
    $SonicPath = "D:\Program Files\SubSonic\SubSonic 2.0.3\SubCommander\",
    $configfile = "",
    [switch]$excludetables,
    [switch]$excludeods,
    [switch]$excludeviews) { 
 
    $sonic = "sonic.exe"
    
    if ($configfile.length -gt 0)
    {
        if (Test-Path $configfile)
        {
            $generateconfig = "generate /config '$configfile'"
 
            # Due to some difficulties with Invoke-Expression we'll change directories
            # to excecute the command and then reset the directory when we are through.
            $savepath = $pwd
            Set-Location $SonicPath
 
            if ($excludetables -eq $FALSE) {Invoke-Expression(".\$sonic $generateconfig")}
 
            Set-Location $savepath
        }
        else 
        {
            "Please provide a valid configuration file name when using the configfile parameter."
            exit
        }
    }
    else
    {
        # Use a database helper function that allows us to get a password in a secure manner.
        # If the username is not an empty string then the function just returns the original username/password
        $cred = get-sqluser $username $password
    
        # Create some standard argument strings for SubCommander
        $generatetables= "generatetables /override /out '$BuildPath' /lang $Lang"
        $generateods= "generateODS /override /out '$BuildPath' /lang $Lang"
        $generateviews = "generateviews /override /out '$BuildPath' /lang $Lang /viewStartsWith View"
    
        
        "Removing and recreating the target directory: $BuildPath ..."
        del $BuildPath -recurse -force
new-item $BuildPath -itemtype Directory -force | Out-Null 
    
        $username = $cred.username
        $password = $cred.password 
    
        # Building our provider string.  Everything is parameterized.
        $Provider = "/server $server /db $db /userid $userName /password $password /generatedNamespace $namespace"
    
    
        "Generate using connection string: "
        "`tserver             = $server"
        "`tdb                 = $db"
        "`tuserid             = $userName"
        "`tpassword           = $password"
        "`tgeneratedNamespace = $namespace"
        "`n"
        
        # Due to some difficulties with Invoke-Expression we'll change directories
        # to excecute the command and then reset the directory when we are through.
        $savepath = $pwd
        Set-Location $SonicPath
    
        if ($excludetables -eq $FALSE) {Invoke-Expression(".\$sonic $generatetables $provider")}
        if ($excludeods -eq $FALSE) {Invoke-Expression( ".\$sonic $generateods $provider" )}
        if ($excludeviews -eq $FALSE) {Invoke-Expression( ".\$sonic $generateviews $provider" )}
        
        Set-Location $savepath
    }
}
 

I have created a whole library of database related functions like Get-SqlUser, which I was able to re-use in the New-Dal function.  To really see why PowerShell is such a powerful solution, lets look at a quick video where we see the new function in action.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5


Creating a management dashboard with PowerGadgets and PowerShell

Dec16

DashboardSeveral months ago I stumbled upon PowerGadgets and have been finding more and more uses for it ever since.  I was initially intrigued by the ability to run powershell scripts but didn't have any immediate needs that required that much power.  One of the strengths of PowerGadgets is the ability to run database queries and then to present the data using very slick graphs, guages and maps.  On Vista, these visualizations are displayed using the Windows Sidebar, however you can also use floating windows which allows PowerGadgets to be equally at home on Windows XP.

My sidebar is now littered with numerous gadgets that I use to show key application metrics, from DotNetNuke Marketplace sales figures to helpdesk work queues.  I have found that picking a few key queries allows me to monitor multiple web applications without requiring me to constantly login to a bunch of different apps.  The nice thing is that as your needs change, you can quickly add new gadgets and remove older gadgets which don't have as much meaning.  Since it takes less than 5 minutes to configure most gadgets, you feel free to create them as you need them, and get rid of them when they are no longer needed.

Recently, I found a need that could not be solved with a simple database query.  The DotNetNuke website was experiencing serious performance issues.  Team members were IM'ing me a couple of times a day to tell me that the website wasn't responding again.  There are hundreds of applications available to perform website monitoring, but I was really interested in finding a lightweight solution that I could get running quickly and modify as I needed to.  Otto Helweg from Microsoft had a quick and dirty PowerShell script for doing web site monitoring that provided a good starting point.  So I fired up PowerShell Plus and modified his script to query the DotNetNuke website and report how long it took to return the results.  His script was still too heavy and did not provide the quick visual impact that a tool like PowerGadget makes possible.  With some heavy trimming I was able to get to the essence of the script.  Since I was also interested in seeing the site performance as a graph, I added some additional code to persist the timing data to a simple text file which is then used to drive the graph.   You can see the script below.  Notice that the persistence code is actually the most complicated part of the script and that monitoring the site is actually a very trivial part of the script.

#Standard Settings
$source = "http://www.dotnetnuke.com/default.aspx"
$timingsfile = "D:\Users\Joe\Documents\PowerShell Utility Scripts\PsObject\Web\monitor.txt"
$retaincount = 90
#Check time to download page
$wc = New-Object System.Net.WebClient
$curtime = (measure-command { $content = $wc.DownloadString($source) }).TotalSeconds
#Update array values
$timings = Get-Content $timingsfile
$webtimes = New-Object System.Collections.ArrayList
switch (($timings | measure-object).Count ) 
{    
$null    {"We don't have to do anything." | Out-Null } 
#single line comment hack         
#Get-Content returns an array of string unless it is a single line    
#so when one line is present the variable is a single string    1         
{[void] $webtimes.add($timings)}        
#when multiple lines are present the variable is an array of strings    
default {[void] $webtimes.AddRange($timings)} 
}
[void] $webtimes.add($curtime)
if ($webtimes.count -gt $retaincount) { $webtimes.RemoveAt(0) }
$webtimes | Out-File 
$timingsfile
#Output array of values
$webtimes

One of the great features of PowerGadgets is the ability to schedule a script or a query to run on a set schedule.  With one setting I can have my script run every minute.  With a script in hand, I created the following gadget in less than 5 minutes, and most of that time was spent just configuring the graph features.  Now I know at a glance if our site is having serious problems.  As long as I don't have too many spikes, or (heaven forbid) a maxed out graph, then everything is just fine.  If I start to see a bunch of spikes or I see the chart stay above 30 second response times, then I know it is time for me to act.

WebMonitor

Future revisions to this script will include additional logic to send SMS messages to predefined phones whenever my site hits certain performance criteria (like 3 consecutive timeouts or 10 timeouts in a 90 minute period).  This will allow me to ensure that we can be responsive to site outages whether we are sitting in front of the computer or not.

I have asked the PowerGadget team for a couple of enhancements that I think would really enhance the usability of PowerGadgets:

1.  Allow me to define parameters for scripts in the PowerGadgets Creator.  At run-time a gadget user could open a settings page that allowed them to edit these values.  This would not require the user to know any PowerShell.  So in my example above, I would like to package this gadget (which PowerGadget already does) and allow my gadget users to customize the site they are monitoring, the file where they are storing results, or the number of results to retain without needing to edit the script.

2. Allow gadgets to have an “alarm” function – if a gauge reaches a certain level, or data on a chart meets certain criteria, then take some action.  This feature should include both simple alarms and scripted alarms.  A simple alarm might include predefined triggers like simple value comparisons, and predefined actions like sending email, or flashing the gadget window.  A scripted alarm would allow the developer to define a PowerShell script which evaluates the gadget data/context and returns a boolean.  The developer would also define a PowerShell script to be executed if an alarm event is triggered.

I urge you, if you haven't checked it out already, to give PowerGadgets a try.  You just might make your job of monitoring servers and applications much easier.

UPDATE:  Updated the script to resolve a bug when your timings file is empty or only contains one line.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5


Why not just use Add-Member

Oct17

In my last post Getting a Username/Password in PowerShell (cross posted to PowerShellCommunity.org), I was asked why not just use the Add-Member cmdlet.  Having been doing software development for my entire adult life, this is not the first time this question has come up.  Ok.  Maybe not those exact words, but something very similar - why use code X when code Y does the same thing.  Where I come from there is really only one response to this - look at both code alternatives and determine which one is the most efficient at doing the job without also becoming a maintenance problem.  So lets take a peek under the hood a bit so we can see why I chose PSObject.Members.Add over using the Add-Member cmdlet.

Scripting is code - except on the command line

When working in PowerShell I try to keep a clear understanding of my usage scenario.  If I am writing something intended to be typed on the command line then I tend to favor a more script oriented approach where I make heavy use of piping one command into another and where cmdlets provide all of the heavy lifting.  However, when I sit down to write a function, I am coding and I tend to favor a much more expanded and explicit coding style using variables and objects.  This makes it easier to maintain the code and to understand what exactly I am trying to accomplish.

Know what is happening under the cover

Once I know what coding style I will be using then I need to evaluate the effectiveness and efficiency of the coding alternatives.  Often this means you will need to understand what is happening inside the framework.  One of the reasons that I am excited about the announcement that .Net is going to provide Source code is that I will finally be able to look at the source code with full commenting.  Until that happens though, I still have Reflector and I use it very heavily.

In this case the question is:  Why use Members.Add instead of using Add-Member?  Lets look at what is going on in Add-Member and you'll see why.

Microsoft.PowerShell.Commands.AddMemberCommand.ProcessRecord() is the relevant method that gets executed when we run the Add-Member cmdlet.  There is some basic error checking to make sure that certain cmdlet parameters were set - like membertype.

Next we have a giant case statement: 

Select Case Me.memberType
    Case PSMemberTypes.AliasProperty
        member = Me.GetAliasProperty
    Case PSMemberTypes.CodeProperty
        member = Me.GetCodeProperty
    Case PSMemberTypes.NoteProperty
        member = Me.GetNoteProperty
    Case PSMemberTypes.ScriptProperty
        member = Me.GetScriptProperty
    Case PSMemberTypes.ScriptMethod
        member = Me.GetScriptMethod
    Case PSMemberTypes.MemberSet
        member = Me.GetMemberSet
    Case PSMemberTypes.PropertySet
        member = Me.GetPropertySet
    Case PSMemberTypes.CodeMethod
        member = Me.GetCodeMethod
    Case Else
        'Deal with error...
End Select


This case statement doesn't provide any real value to our function since I already know what type of members I wish to add, I don't need the cmdlet to make a conversion from my "string" to a root class type.  Even once the membertype is determined, the cmdlet calls another function (GetNoteProperty in our case) which we see below: 

Private Function GetNoteProperty() As PSMemberInfo
    Me.EnsureValue1HasBeenSpecified
    Me.EnsureValue2HasNotBeenSpecified
    Return New PSNoteProperty(Me.memberName, Me.value1)
End Function
 

So in essence, this big case statement is doing exactly the same thing as my code, except with a lot more overhead.  It is creating a new PSNoteProperty.  Now lets look at the next part of the ProcessRecord method.

Dim info2 As PSMemberInfo = Me.inputObject.Members.Item(member.Name)
If (Not info2 Is Nothing) Then
    If Not Me.force Then
        MyBase.WriteError("...")
    Else
        If info2.IsInstance Then
            Me.inputObject.Members.Remove(member.Name)
            goto Label_0201
        End If
        MyBase.WriteError("...")
    End If
    Return
End If
Me.inputObject.Members.Add(member)

So we see that beyond a bunch of extra error handling to deal with members that already exist, this code just calls Members.Add().

At this point I think it is safe to say that my original code and the Add-Member cmdlet are performing exactly the same function.  However, with the cmdlet there is a performance penalty to be paid for parsing all of the strings passed to the cmdlet and converting them to types.  Determining how to act based on some of the parameters (like -force).  Also, I think that my code is just as readable as using the cmdlet.  Ultimately though, Add-Member is just sugar coating for adding a PSMemberInfo object to a PSObject.Members collection.

So, the next time you have a question about whether to use a cmdlet or whether to use "code", pull out reflector and see what is really happening under the hood.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5


Getting a username/password in PowerShell

Oct16

As part of getting ready for my PowerShell session at OpenForce '07, I am creating a set of helper functions for working with SMO to manipulate and query the database server.  A common need when working with the database is to pass the username and password to various SMO methods.  When I first started coding my examples, I just passed a username and password as parameters into my functions.  This works, but does not exactly look professional when you are sitting in a presentation and typing out passwords in plaintext.

I decided that it would be better to use Get-Credential in this case since it would provide a professional dialog and keep the password hidden throughout the process.  Since I wanted to support both scenarios, I came up with the below function. 

function global:get-sqluser($username="", $password="") {
    # We are creating an object to which we'll add custom properties
    $user = New-Object Management.Automation.PSObject
    
    if ($username -eq "") {
        # No username was specified, so we should use Get-Credential to prompt for a user
        # We also define a default username in order to suppress console output
        # The results are added as synthetic properties to the PSObject we created above
        $cred = Get-Credential "SqlUser"
        $user.psobject.members.add( (New-Object System.Management.Automation.PsNoteProperty UserName, $cred.username.split("\")[1]) )
        $user.psobject.members.add( (New-Object System.Management.Automation.PsNoteProperty Password, $Cred.Password) )
    } Else {
        # In this case we can just create synthetic properties using the values passed to the function
        $user.psobject.members.add( (New-Object System.Management.Automation.PsNoteProperty UserName, $username) )
        $user.psobject.members.add( (New-Object System.Management.Automation.PsNoteProperty Password, $password) )
    }
 
    # Return our synthetic object
    $user
}

It is not perfect as the two cases return the password as either a cleartext string or a securestring.   Also, I don't like the hack with the Get-Credential since I don't really need a domain qualified name (notice the ugly split("\")[1] that is needed to get just the username without a domain qualifier).  My next version will resolve both of these issues, but for now this will have to suffice.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5


PowerShell for DotNetNuke

Oct14

For the past couple of months I have been playing with PowerShell and it has been a real pleasure to dig-in and get my hands dirty in a command-line interface again.  Not since my early days with the Amiga and ARexx have I enjoyed learning about a new CLI and an associated scripting language.  The PowerShell team has done a great job of bringing windows scripting and the CLI into the 21st century.  Not only has Microsoft been putting in a lot of effort into PowerShell, but you are also seeing an explosion of third-parties who have also jumped onboard to push the new language and hosting environment.  Companies like Sapien, Quest, /n Software, and ShellTools are creating great tools and sponsoring community sites like PowerShellCommunity.org.

With all of this excitement and passion in the PowerShell community, how could I not look to see how I could use PowerShell with DotNetNuke.  Thus was born my session idea:  PowerShell for DotNetNuke which I will be presenting in Las Vegas at OpenForce '07.  During my session I will be showing how you can use PowerShell to enhance managing DotNetNuke, both for installing new sites and for sites that are already up and running. 

One of the early ideas I had was that I might even be able to create a DotNetNuke module that would let a user execute PowerShell commands and scripts on the server.  Since DotNetNuke allows you to lock down access to any module it is fairly secure.  The initial code for this module is based off earlier examples by Scott Hanselman and Dominick Baier.  Here is a sample video of my work so far.

 

I still have some enhancements that I will be making before the conference, but you can download the latest code here.

 

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5


PowerShell and DotNetNuke

Jul08

PowerShell Having worked with DotNetNuke for more than 4 years now, I have performed literally hundreds of installations.  While the installation process has gotten easier over the years as we have beefed up the installation routines, it is still not automatic. Even though I can install DNN in under a couple minutes, I often cut corners.  I usually just give the Asp.Net process account full control of the web application folder and use an sa account in the DB.  Neither of these are great practices, but setting up the correct permissions is a little more tedious and after you perform this task dozens of times you find that you soon start taking shortcuts.  The downside to these shortcuts is that I am not really testing out my site in a configuration that mirrors a typical production environment.  This can lead to subtle bugs that will be harder to track down later.

OpenForce400x131_thumb This is where PowerShell can really make a difference.  In my upcoming session at DotNetNuke OpenForce '07 I will be discussing how you can use PowerShell with DotNetNuke to perform a lot of management and administrative functions.  This post is the first in a series of that will show various ways we can use PowerShell to enhance and simplify our DotNetNuke experience.  These blog posts will cover many of the topics I plan to cover in my session.  I am not an expert in PowerShell yet, but my skills are quickly improving.  This series will follow my learning experience as I tackle various tasks for which PowerShell is well-suited.

The first set of posts will focus on simplifying the installation experience.  My goal is not to create "The Ultimate DotNetNuke Installer", but rather to show how you can use PowerShell to easily script a set of very repetitive tasks thereby ensuring consistency and making the overall process easier.  In some cases I may use hard-coded values and business rules, but hopefully it will not be too difficult for you to take my scripts and adapt them to your situation.  So let's begin.

The first step is to define the process we wish to automate.  We'll break our process down and develop the script over a couple of posts.

My installation process looks like this:

  1. Unzip the DNN package to my website directory.
  2. Set permissions for the "Network Service" Account
  3. Create the Web Application in IIS
  4. Create a blank database
  5. Create a user account in the database
  6. Run the DotNetNuke web-based installer

Step 1:  Unzipping Files

This step actually turned out to be much easier than I anticipated.  DotNetNuke uses SharpZipLib in the module installer for extracting files and folders from an embedded resources file.  Since PowerShell is right at home in .Net I thought that I would just end up re-purposing the original installer code I wrote for DotNetNuke.  Using SharpZipLib is a relatively low-level abstraction for handling zip-files.  You have complete control, but it also means that there is a lot of work needed to extract the files while maintaining the folder structure.  In DotNetNuke we use approximately 50 lines of code to unzip a file.

Thinking that there might be an easier way in PowerShell, I did what all good programmers do - I Googled it to see if someone else had already solved my problem. Bingo.  David Aiken had my answer.   The answer turned out to be very simple, and it highlighted one of the great aspects of PowerShell.  Even though PowerShell is based on .Net and has no problem working in the .Net world;  it is equally at home working with COM and WMI.  A little more research showed that the "shell.application" COM object was commonly used in VBScripts and WSH for unzipping files.

function Extract-Zip
{
    param(
        [string] $zipfilename = $(throw "Please enter a zipfilename"),
        [string] $destination = $(throw "Please enter a destination directory"),
        [switch] $showdialogs
    )

    if(test-path($zipfilename))
    {    
        "Unzipping $zipfilename ..."

        $shellApplication = new-object -com shell.application
        $zipPackage = $shellApplication.NameSpace($zipfilename)
        $destinationFolder = $shellApplication.NameSpace($destination)
        # NOTE: This scripts uses the PowerShell Community Extensions for the ternary operator

        $vOptions = (?: {$showdialogs} {0} {20})

        # CopyHere vOptions Flag
        #  4 - Do not display a progress dialog box.
        # 16 - Respond with "Yes to All" for any dialog box that is displayed.

        $destinationFolder.CopyHere($zipPackage.Items(), $vOptions)
        
        "Completed unzipping file..."
    }
}

I have made a few changes from David's original script to enforce the parameters and to suppress the dialogs that popup when extracting the files.  This is a good example of how to make required parameters and how to use switches to power up your scripts.  We'll add this with some ACL magic in our next post.

 

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5


PowerShell to the rescue

Jun28

PowerShell Recently, while working on a new module installer I needed to be able to roll back my test environment as I went through the various iterations.  This was not a trivial installation and therefore I knew that I would likely have to take several stabs at this before I got it right.

Normally, I would turn to either Virtual PC or VMWare in this case.  Unfortunately Vista and Virtual PC were not cooperating today and it was taking forever just to do simple file copies between the VPC and host machine.  Installing the module which clocked in at just over 4Mb took forever.  All of my time savings was being eaten up by a finicky Vista install.  Since I re-paved my machine a few months back I did not currently have a VMWare installation running on this machine so that was not really an option.

 Lucky for me, my friendly neighborhood superhero just happened to be in the area and heard my calls for help.  Whipping out my trusty copy of Windows PowerShell in Action I was able to quickly craft a function to handle this daunting task.  Since I am a relative newbie at PowerShell it was a good learning excercise.  Once I got past dot sourcing my script file, I was off and running.

function RestoreDNN ($user, $password) {
    "Restoring website..."
    $dnnDir = 'D:\Websites\DNNTest'
    Copy-Item "$dnnDir\config\web.Config" $dnnDir
    Copy-Item "$dnnDir\config\siteurls.Config" $dnnDir
    Remove-Item "$dnnDir\app_code\MyModule" -recurse -force
    Remove-Item "$dnnDir\desktopmodules\MyModule_*" -recurse -force

    "Stopping web service..."
    Get-Process w3wp |Stop-Process

    "Restoring database..."
    osql -U $user-P $password -Q "RESTORE DATABASE DNNTest FROM DISK = 
            'c:\MSSQL\Backup\dnntest.bak'"

    "Done"
}

With the script written and run in my console, I am now able to restore my environment by a simple command "RestoreDNN myUser myPassword", 10-15 seconds later my DNN environment is back to where it was before I installed the module, and ready for another round of testing.

This script is a sample for illustrative purposes and is not intended to be production code.  I have cut out some repetitive code and removed specific project references.  I am sure that over time I will make it a little more generic and more robust, but for now it does what I needed, and took me less time to write than it would have taken to install and setup a VMWare environment.

If you are interested in PowerShell, then I highly recommend you read the blogs by the PowerShell team or ThePowerShellGuy as they contain a lot of valuable information for both newbie's and experience PowerShell gurus.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5