It came about because some of the SQL MVPs were trying to figure out the best way to download a VM that had been split up into 36 different equal-sized files.  I’m sure there’s a better way than this but at the same time it took me about 4 minutes to write.  Write is the wrong term, more like copy/paste/change a few things.

Give it a whirl and let me know what you think! 

I showed how you could pull size and free space information from all of your data files, then used the same technique to pull size information from all of your tables across every database on an instance.  Later we created a database and some tables, and then showed how to do some basic login comparisons, all with SQLPSX.

In addition, we stored the results of the frees pace and table size information in a database using the Out-DataTable and Write-DataTable functions.  Finally we wrapped up with a quick demo on how the new SQL Server cmdlets inside of the SQLPS Module in Denali combined with some new functionality in PowerShell v3 CTP1 can really make backing up a database with PowerShell a lot easier.

Here’s what you’ll need to recreate today’s demos;

• Download and install SQLPSX
• The Out-DataTable function from the TechNet script repository
• The Write-DataTable function from the TechNet script repository
• and these scripts
• Oh, and here are the handful of slides that I had

I hope I see you next week at the PASS Summit or next month during another webcast, this time on customizing your own functions to work with the SMO.

I woke up at 5 am this morning to kick off my download.  I’ve finished the install and I’m happy that I can finally report that we have new cmdlets for SQL Server. 

First off, after running Get-Module -ListAvailable I saw:

Manifest   SQLASCMDLETS
Manifest   SQLPS

This means that SQLPS inside of SQL Agent should finally allow you to import a module during your job step. 

There is now a Function called SQLServer that, from what I can tell, is the SQLServer Provider.  When under PS SQLSERVER:\> if you do a dir | Select pschildname you will find the following list:

PSChildName
———–
SQL
SQLPolicy
SQLRegistration
DataCollection
XEvent
Utility
DAC
IntegrationServices
SQLAS

From my notes it looks like XEvent, IntegrationServices, and SQLAS are the directories that have been added here.  Can’t wait to try out the IntegrationServices portion!  [Update]: Apparently I may need a defaul instance install for this to work.

I also noticed that Tab-Completion appears to be fixed!!    Also, SQL Registrations > ‘Database Engine Server Group’ are split up into instances like CMS was.  Not sure what that’s all about just yet.  In fact when I try to use it, it doesn’t seem to work.  Maybe I should read the help files.  But I think they may have just gotten CMS & RS reversed.  CMS isn’t working for me either yet.

When you drill down under XEvents and go to your Denali instaince you will see: Packages, & Sessions.  I guess this means that I finally need to read Jonathat’s blog series on X-Events.

When I imported the SQLPS module I received a warning about unapproved verbs.  Oh well, hopefully they’ll work that out soon.  Next I ran Get-Command -Module “SQLPS” and found the following list of cmdlets.

Analysis Services got some love too in the form of 11 cmdlets!  After inporting the SQLASCMDLETS module I ran this Get-Command -Module “SQLASCMDLETS” and found the following.

More news to follow soon!

To answer Paul Randal’s question of why I should get to win a week of training from SQLskills this is what I have to say:

Plain and Simple.  I’m a Second Generation DBA with over a decade of experience with SQL Server.  I want some deep level ideas for some new PowerShell scripts to simply tasks for DBAs everywhere.  I’ll also spend countless hours telling all my fellow DBAs here in Atlanta what all I learned until they can take it no more and demand this level of top-notch training for themselves! 

I’ll get started with the PowerShell scripts right now even.  This little script will search twitter for a giver hashtag, in this case: #SQLskills.  It will then save everything it finds off to an XML file and then query it:

$SearchHashtag = "sqlskills"            

$wc = new-object net.webclient
$url = "http://search.twitter.com/search.atom?q=%23$SearchHashtag"
$OutFile = "c:\temp\Download_$($SearchHashtag)_$([DateTime]::Now.ToString('yyyyMMddHHmmss')).xml"
$wc.DownloadString($url) | add-content $OutFile;            

[xml]$tweetxml = get-content $OutFile            

$tweetxml.feed.entry | Format-List            

($tweetxml.feed.entry).count            

So I just wanted to encourage everyone who makes their living off of SQL Server to take a minute to add these two RSS feeds to your favorite feed-reader. 

https://connect.microsoft.com/rss/68/RecentFeedbackForConnection.xml

https://connect.microsoft.com/rss/68/RecentlyModifiedFeedbackForConnection.xml

If you haven’t been on Connect for a while and wonder what’s out there, well I’ve got a PowerShell script for you  

Invoke-Sqlcmd -ServerInstance denali.db.5026258.hostedresource.com `
-Database denali -Username denaliconnect -Password Wide0pen -Query "
SELECT ID
        , Title
        , ItemStatus
        , UpVoteCount
        , DownVoteCount
        , ItemDescription
        , Author
        , Modified
        , ValidationCount
        , WorkAroundCount
        , URL
        , OpenedDate
  FROM denali.dbo.ConnectItems" | out-gridview

OK, you can totally take that login info and use your SQL Management Studio to connect to this database too but I wanted to show you a REALLY cool use for the Out-GridView command.  When run the PowerShell script and the window pops up Just Click on + Add Criteria > ItemDescription > Add.

Like this:

Then just type in a search term like “SSIS” and see what you get back…

Rob has also put together a way to search the data that is much better and where we hope to ultimately go with it.  Again, that’s all for another post.

It’s a huge honor to get the chance to be a guest blogger on Hey Scripting Guy! and I hope these posts were able to make some people’s (o.k. A Lot of People’s) lives easier. Given the number of people checking out PowerShell for the first time based on the comments to my MCM Videos download script I figured I’d call these out today:

Use PowerShell to Obtain SQL Server Database Sizes

Use PowerShell to Change SQL Server Service Accounts

Use PowerShell to Script SQL Database Objects

I know that the Hey Scripting Guy! blog is the #1 blog on TechNet but I’m not allowed to know any specifics beyond that. I did however catch this in yesterday’s post: “In general, the articles we published on SQL Server and on SharePoint have been really popular.” So hopefully that means we’ll get to hear some more SQL voices there in the near future. Maybe even in time for SQL Rally, who knows?

To get a very fast answer to this answer I opened up PowerShell.exe because while I normally demo in PowerShell_ISE.exe, I don’t use a profile in PowerShell.exe (more on that in a later post).  I ran this command to get the count of just the cmdlets.

(get-command -CommandType Cmdlet).count

Then I fired up SQLPS.exe (typically found in C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn) and ran the same command (after switching out of SQL Server and over to the C drive; no idea why that was necessary).

The only problem here is that it’s not really 236 – 130 = 106 because SQLPS.exe adds 5 cmdlets that I made sure to exclude (along with everything else I normally use) from PowerShell.exe. 

To get a more accurate list of the cmdlets and which one has what I decided to write a script that everyone con run on their own machine.  One of the many cool features of PowerShell is that you call call it’s executable and pass it a -command or even a –file.  I went ahead and threw together this simple command below so that you can see the differences yourself.

Note: I assume you are running Windows 7 which comes with PowerShell 2.0

$SQLPS=SQLPS.exe -command "CD C:\; get-command -CommandType Cmdlet | select CommandType, Name; exit";
$PSTwo=powershell.exe -command "get-command -CommandType Cmdlet | select CommandType, Name; exit";

Compare-Object -ReferenceObject $SQLPS -DifferenceObject $PSTwo|
Export-CSV C:\temp\SQLPSCommandDifferences.csv -NoTypeInformation

Now all that you have to do is open up that csv file that was just created (C:\temp\SQLPSCommandDifferences.csv) and have a look. 

Note: I could have just displayed the info inside of my PowerShell window by leaving off the last line and the pipe but then I would have needed to do something to format the output so I went with the CSV file instead.

For more information on this topic be sure to check out Chad Miller’s post on it.

While using the product I ran into a single small problem. When I check a database into Team Foundation Server via SQL Source Control it doesn’t include the drop statements. The deployment procedures at my company just go smoother with DROP statements in each of the stored procedures. Similarly, for databases that haven’t been deployed outside of the Development environment it was better for us to drop and re-create the tables from scratch. It isn’t the kind of thing you’d ever do in production, but given the length of some development projects it is very useful if you can do this when needed.

The version of SQL Source Control I used doesn’t script drops or permissions by default. If there is an option to change this I can’t find it. Rather than switch tools I used this as an excuse to learn how to work with the different scripting options available inside the SMO exposed by PowerShell. Find your options this way:

$dbname="AdventureWorks"
$server="WIN7NetBook"

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | out-null
$SMOserver = New-Object (’Microsoft.SqlServer.Management.Smo.Server’) -argumentlist $server
$db = $SMOserver.databases[$dbname]

$Scriptr = new-object ('Microsoft.SqlServer.Management.Smo.Scripter') ($SMOserver)
$Scriptr.Options | Get-Member

To work around this problem I linked my database to my TFS server (from SQL Source Control inside of SSMS). I committed all the changes to TFS so that everything was checked in. Then I opened Visual Studio, checked out out all the objects on to my local machine, generated the scripts with PowerShell just the way I wanted them, overwrote the local TFS code with the code I had just generated, and finally, I checked everything back in to TFS. Doing all this was pretty quick, worked great, and SQL Source Control didn’t care that the scripts weren’t the ones that it had generated.

Be forewarned, when you include the ScriptDrops option it only generates the drop statement. To work around this see my tips here. Now all that is left is to add the Permissions option.

The resulting code also does some cool things like divide objects into folders by type; put everything under a folder structure that includes the name of the database; and another folder that is simply datatime so that you can run it multiple times as you tweak the code to fit your environment, or, just keep old versions of the code laying around in case someone dropped something that hadn’t been checked into TFS yet. I’ve also included the portion of code that only adds the drop statements to objects like stored procedures but not tables.

You can download the script here. Or just take a look at it here. When/if you run the script, it creates a PowerShell function. To call it all you have to do is something like this:

Script-DBObjectsIntoFolders "AdventureWorks" "WIN7NetBook"

And you could even call it in a loop to script out all of your user databases:

foreach ($dbn in invoke-sqlcmd -query "SELECT name
  FROM sys.databases WHERE owner_sid !=0x01"`
 -database master -serverinstance WIN7NetBook )

{ #Begin Loop
$dbn;
Script-DBObjectsIntoFolders  $($dbn.name) "WIN7NetBook"
} #End Loop

Script-DBObjectsIntoFolders

function global:Script-DBObjectsIntoFolders([string]$dbname, [string]$server){
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | out-null
$SMOserver = New-Object (’Microsoft.SqlServer.Management.Smo.Server’) -argumentlist $server
$db = $SMOserver.databases[$dbname]

$Objects = $db.Tables
$Objects += $db.Views
$Objects += $db.StoredProcedures
$Objects += $db.UserDefinedFunctions

#Build this portion of the directory structure out here in case scripting takes more than one minute.
$SavePath = "C:\TEMP\Databases\" + $($dbname)
$DateFolder = get-date -format yyyyMMddHHmm
new-item -type directory -name "$DateFolder"-path "$SavePath"

foreach ($ScriptThis in $Objects | where {!($_.IsSystemObject)}) {
#Need to Add Some mkDirs for the different $Fldr=$ScriptThis.GetType().Name 
$scriptr = new-object ('Microsoft.SqlServer.Management.Smo.Scripter') ($SMOserver)
$scriptr.Options.AppendToFile = $True
$scriptr.Options.AllowSystemObjects = $False
$scriptr.Options.ClusteredIndexes = $True
$scriptr.Options.DriAll = $True
$scriptr.Options.ScriptDrops = $False
$scriptr.Options.IncludeHeaders = $True
$scriptr.Options.ToFileOnly = $True
$scriptr.Options.Indexes = $True
$scriptr.Options.Permissions = $True
$scriptr.Options.WithDependencies = $False
<#Script the Drop too#>
$ScriptDrop = new-object ('Microsoft.SqlServer.Management.Smo.Scripter') ($SMOserver)
$ScriptDrop.Options.AppendToFile = $True
$ScriptDrop.Options.AllowSystemObjects = $False
$ScriptDrop.Options.ClusteredIndexes = $True
$ScriptDrop.Options.DriAll = $True
$ScriptDrop.Options.ScriptDrops = $True
$ScriptDrop.Options.IncludeHeaders = $True
$ScriptDrop.Options.ToFileOnly = $True
$ScriptDrop.Options.Indexes = $True
$ScriptDrop.Options.WithDependencies = $False

<#This section builds folder structures.  Remove the date folder if you want to overwrite#>
$TypeFolder=$ScriptThis.GetType().Name
if ((Test-Path -Path "$SavePath\$DateFolder\$TypeFolder") -eq "true") `
        {"Scripting Out $TypeFolder $ScriptThis"} `
    else {new-item -type directory -name "$TypeFolder"-path "$SavePath\$DateFolder"}
$ScriptFile = $ScriptThis -replace "\[|\]"
$ScriptDrop.Options.FileName = "" + $($SavePath) + "\" + $($DateFolder) + "\" + $($TypeFolder) + "\" + $($ScriptFile) + ".SQL"
$scriptr.Options.FileName = "$SavePath\$DateFolder\$TypeFolder\$ScriptFile.SQL"

#This is where each object actually gets scripted one at a time.
IF ($ScriptThis.GetType().Name -NE "Table") { $ScriptDrop.Script($ScriptThis) }
$scriptr.Script($ScriptThis)
} #This ends the loop
} #This completes the function

Last week Paul Randal (twitter) blogged and asked people to take part in a survey asking what the top Wait Type on their systems are.  Since I have 100s of SQL Servers and I wanted to give him an answer for each one. And because I’m lazy…  I modified Paul’s (or Glen’s [ blog | twitter ] ) query to include a ServerName column then Select’ed the results of the query on my local machine into a table. Once I had the table set up I fired up PowerShell and used the same type of script that I showed off in today’s Hey Scripting Guy post. The only difference being that I swapped out the query and table name of course.

CREATE TABLE [dbo].[TopWaitTypes](
   [InstanceName] [nvarchar](128) NOT NULL,
   [WaitType] [nvarchar](60) NOT NULL,
   [Wait_S] [decimal](14, 2) NULL,
   [Resource_S] [decimal](14, 2) NULL,
   [Signal_S] [decimal](14, 2) NULL,
   [WaitCount] [bigint] NOT NULL,
   [Percentage] [decimal](4, 2) NULL
) ON [PRIMARY]

Make sure to read to the bottom for the significantly less code version!

foreach ($RegisteredSQLs in dir -recurse SQLSERVER:\SQLRegistration\'Database Engine Server Group'\Development | where {$_.Mode -ne "d"} )
{
$dt=invoke-sqlcmd -query "WITH Waits AS
    (SELECT
        wait_type,
        wait_time_ms / 1000.0 AS WaitS,
        (wait_time_ms - signal_wait_time_ms) / 1000.0 AS ResourceS,
        signal_wait_time_ms / 1000.0 AS SignalS,
        waiting_tasks_count AS WaitCount,
        100.0 * wait_time_ms / SUM (wait_time_ms) OVER() AS Percentage,
        ROW_NUMBER() OVER(ORDER BY wait_time_ms DESC) AS RowNum
    FROM sys.dm_os_wait_stats
    WHERE wait_type NOT IN (
        'CLR_SEMAPHORE', 'LAZYWRITER_SLEEP', 'RESOURCE_QUEUE', 'SLEEP_TASK',
        'SLEEP_SYSTEMTASK', 'SQLTRACE_BUFFER_FLUSH', 'WAITFOR', 'LOGMGR_QUEUE',
        'CHECKPOINT_QUEUE', 'REQUEST_FOR_DEADLOCK_SEARCH', 'XE_TIMER_EVENT', 'BROKER_TO_FLUSH',
        'BROKER_TASK_STOP', 'CLR_MANUAL_EVENT', 'CLR_AUTO_EVENT', 'DISPATCHER_QUEUE_SEMAPHORE',
        'FT_IFTS_SCHEDULER_IDLE_WAIT', 'XE_DISPATCHER_WAIT', 'XE_DISPATCHER_JOIN', 'BROKER_EVENTHANDLER',
        'TRACEWRITE', 'FT_IFTSHC_MUTEX', 'SQLTRACE_INCREMENTAL_FLUSH_SLEEP')
     )
SELECT @@ServerName AS 'ServerName',
     W1.wait_type AS WaitType,
     CAST (W1.WaitS AS DECIMAL(14, 2)) AS Wait_S,
     CAST (W1.ResourceS AS DECIMAL(14, 2)) AS Resource_S,
     CAST (W1.SignalS AS DECIMAL(14, 2)) AS Signal_S,
     W1.WaitCount AS WaitCount,
     CAST (W1.Percentage AS DECIMAL(4, 2)) AS Percentage
FROM Waits AS W1
INNER JOIN Waits AS W2
     ON W2.RowNum <= W1.RowNum
GROUP BY W1.RowNum, W1.wait_type, W1.WaitS, W1.ResourceS, W1.SignalS, W1.WaitCount, W1.Percentage
HAVING SUM (W2.Percentage) - W1.Percentage < 95; -- percentage threshold" -ServerInstance $RegisteredSQLs.ServerName -database master
Write-DataTable -ServerInstance "Win7NetBook" -Database CentralInfo -TableName TopWaitTypes -Data $dt
}

After reading Chad Miller’s (Blog|Twitter) excellent post yesterday though, I decided to combine our approaches.  If you haven’t done so already, you’ll need to download Chad’s invoke-sqlcmd2 and Write-DataTable functions and load them up in your environment.  You’ll also need to store the edit the query from Paul’s blog to add a ServerName column and then store it in a .sql file in the working directory of your PowerShell session.  Once that’s all done if you haven’t given up and closed your browser you can use a piece of code like this to gather up all of this information from all of the server you have in your Registered Servers list. 

For safety’s sake when I put out scripts like this I always make sure to dive down into the \Development branch of my Registered Servers list so that you will always be able to start somewhere ‘safe’ when you run this for the first time.  Or just get a big red nasty error message if you don’t have a “Development” Registered Servers Group

Here’s the code that I came up with for me:

foreach ($RegisteredSQLs in dir -recurse SQLSERVER:\SQLRegistration\'Database Engine Server Group'\Development | where {$_.Mode -ne "d"} )
{
$dt=invoke-sqlcmd2 -ServerInstance $RegisteredSQLs.ServerName -database master -InputFile ./PaulAndGlensWaitQuery.sql -As 'DataTable'
Write-DataTable -ServerInstance "Win7NetBook" -Database CentralInfo -TableName TopWaitTypes -Data $dt
}

Paul had mentioned that “The free survey system only allows a single vote per IP address – if you have any other results, send them in email (mailto:paul@SQLskills.com?Subject=Wait stats) or attach a comment below.” so I plan to send him an email with the top wait from each of my servers, minus the server name of course.

I Hope this helps start some ideas in your head about how you can leverage PowerShell in your environment!

By the way…  I got the idea for leveraging the Registered Servers piece from an old Buck Woody ( blog | twitter ) post so pretty much none of this is my code, I just put it all together.  And THAT is why I love twitter!

I’ll be speaking at SQL Saturday #48 this weekend and I am happy to announce I will be unveiling my more advanced PowerShell session to help people customize scripts for their own environment.  I will be showing off several new scripts that I built for my session at the PASS Summit this year.  Please be ready to give plenty of brutal feedback on these scripts so that I can incorporate that feedback in the final versions that I take to the Summit.

Besides my two sessions I will be joined by Eric Humphrey ( blog | twitter ) and Microsoft Scripting Guy Ed Wilson ( blog | twitter ) who will also be speaking about PowerShell.  Eric totally stole the idea for his session from me but hey I guess ‘Great Minds Think Alike’   Just a quick glance at the schedule shows experts descending on Columbia from Richmond, Tampa, Orlando & Alabama just to name a few so don’t even think of coming up with some lame excuse that it’s too far to drive.  We’ll also be joined by a bunch of SQL Experts from Microsoft’s Charlotte campus so it’s definitely work the trip and I hope to hear you heckle me there!

Take a look at the schedule and you’ll see that it’s packed with SQL Server experts so come on out and join
 us, the football hasn’t gotten all that good yet so you’ve got no reason not to.  Also, I have a new demo that Lee Holmes helped me get working that if you love to hate extracting data from Excel, this will make your life much simpler. 

A whole post would take more time than I have. Instead I’m going to mention something that Rob Farley (of LobsterPot Solutions in Australia) said in a SARGability talk for the AppDev Virtual Chapter of PASS a few months back.

Rob talked about the way that we find entries in an index using a Seek operation, and compared to using a phonebook. In particular, he described the bit at the top corner of the phonebook, which we use to find the right page before looking at each record. This is very much how an index works. An index is stored in a b-tree, with the levels at the top being like the corner sections in the phonebook, and the leaf level of the tree being like all the records there. 

A quick bit of searching flickr found an image at http://www.flickr.com/photos/blinky5/376596220/ which describes this perfectly. If you’re looking for Wilma Todd, you can use the top corner to figure out which page she’s on and then find her record on the page very easily.  So what’s my point?: The next time that you’re picking a data type for a column think of this.  If you use a data type that’s twice as large as what you really needed (int vs. smallint, nvarchar vs varchar) and then you realize you need to put an index on it, it would be be like doubling the font in this phone book thereby requiring twice as many pages to print the book.  May not seem like much but when you have to scan for data you’re now going to have to physically touch twice as many pages.  And hey, now the books twice as heavy as it needed to be just like your backups are going to be! 

Anyway, I don’t have any more time to spend on this post, gotta get my code ready in case I need to fill in tomorrow for someone during the 24 Hours of PASS, so I’ll just publish it and wish you all a Happy T-SQL Tuesday.

I’ve recently learned that if you only have PowerPoint 2007 you’ll need to download this viewer because I built the slides in PowerPoint 2010. Here are the scripts for Columbus and Edmonton.

I think both events went really well and got a lot of people saying: ‘Hey, this PowerShell stuff could work in my environment’. I really hope they catch the fever! I’d love to speak to each of these groups next year and see what questions they come up with and see how much they’re using it. I’ve got a more advanced session that I will be debuting a little later this year so hopefully I’ll get to share it with them too.

Huge thanks to Jonathan Boulineau ( twitter ) and Colin Stasiuk ( blog | twitter ) for all they do for their groups all year long.

As always, there will be something new that I’ve never showed off before, most likely just a failed script from my Atlanta session but hey, they didn’t get to see it yet.    I’ve also been asked to include some sys-admin type stuff so I will do a walk through of Implicit Remoting and probably some important tips on working with Services.  There will also be an XML demo if there are any developers that want to check it out.

Details:

• The meeting will be held at the Columbus Library
• Date: 8/24/2010
• Time: 6:00 PM – 7:30 PM
• 3000 Macon Rd, Columbus, GA 31906

Yesterday we laid the ground work for PowerShell Week.  Today we will learn about Variables in PowerShell. You will notice that PowerShell variables work very differently from SQL Server variables.

The evolution of variables in SQL Server is pretty straight forward. In SQL Server 2000 you had to DECLARE a variable as a specific type in one statement and then SET the value of the variable in another statement, like this: DECLARE @SQLvariable VARCHAR(12) SET @SQLvariable = ‘OldFeature’. By the time SQL 2008 hit the streets we could DECLARE and SET a variable in one statement like this: DECLARE @SQLvariable VARCHAR(12) = ‘NewFeature’ . PowerShell evolves variables to the next logical step.  You no longer need to declare or type your variable. Instead PowerShell uses the value of the variable to infer it’s type. If a variable does not alreayd exist PowerShell simply creates it for you.  In PowerShell our statement looks like this: $SQLvariable = ‘NewFeature’. (Beginer tip: $ identifies variables in PowerShell just like @ identifies them in SQL Server.)

Since it doesn’t work in SQL let’s try it in PowerShell: $SQLvariable = ‘NewFeature’.  OK so we loaded data into a variable, ‘big deal’ right?  How do we know that we actually loaded it and more importantly what datatype is it?  Well in PowerShell you can simply call the variable to get its value. (Another beginer tip: you don’t need to use a keyword like SELECT for this, just the variable name: $SQLvariable.)  What if you just want to determine the datatype?  All you have to do is tack on the .GetType() Method: $SQLvariable.GetType().

$SQLvariable = 'NewFeature'
$SQLvariable
$SQLvariable.GetType()

When you run this here’s what your result will be:

As I alluded to earlier, another feature of PowerShell is that it remembers your variables for you, even between executions.  Basically as long as you don’t end your session the variable, it’s datatype, and its value will persist.  You will see later than you can also set a variable in one script and consume it in another.

Let’s try this again with a new variable and set it to a different value:

$PoShvariable = 123
$PoShvariable
$PoShvariable.GetType()

What happens if once the datatype is set we try to set it to a different, incompatible datatype?  It’s going to explode right?  Nope, you’ll have to throw something more difficult than that at it.  Give this a try:

$PoShvariable.GetType()
$PoShvariable = 'NewFeature'
$PoShvariable
$PoShvariable.GetType()

Since PowerShell relies on .NET a variable could be any datatype in .NET so if you’re sitting inside the AdventureWorks database you can see for yourself, just run this:

$MyTable = get-item TABLESProduction.TransactionHistory
$MyTable.GetType()            

$MyStoredProc = get-item StoredProceduresHumanResources.uspUpdateEmployeePersonalInfo
$MyStoredProc.GetType()

Now earlier I mentioned that PowerShell keeps track of and remembers the variables that you previously declared.  So how do you see what variables are already there?  Just run this command and you can see every variable you have running in the session, even the internal ones.

Variable

In the list of items that get returned you’ll see one called Error that holds the last error message that occurred in your session.  This handy feature can help you when you have to debug your own work.

Is that all there is to know about variables in PowerShell?  Not hardly, but I think that’s a good stopping point for this post.  More on variables later.

https://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032449694&EventCategory=4&culture=en-US&CountryCode=US

If you haven’t caught this presentation at a SQL Saturday yet, or if you want to see it again, then this is a chance to watch from your desk and follow along in Powershell for yourself.

The show is at 3pm Eastern/Noon Pacific.  You can see previous geekSpeak shows and subscribe to their feed here.  Please stop by and heckle me check it out.

https://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032449694&EventCategory=4&culture=en-US&CountryCode=US

If you haven’t caught this presentation at a SQL Saturday yet, or if you want to see it again, then this is a chance to watch from your desk and follow along in Powershell for yourself.

The show is at 3pm Eastern/Noon Pacific.  You can see previous geekSpeak shows and subscribe to their feed here.  Please stop by and heckle me check it out.

“This year we are hoping to aid in raising awareness for women in technology.  The US Department of Labor reported in 2009 women made up only 22.9% of computer programmers, and just 20.9% of computer software engineers.  For comparison, women chief executives counted for 23.4% – more women are CEOs than programmers!  (As an aside, 79.2% of meeting and convention planners are women, I’m a rarity it seems.)

This problem isn’t a simple one, and it not going to be fixed by a clever logo, but we can help by raising awareness.”

Here’s some more info on the event:

The theme for CodeStock 2010 is about connections.  Connecting with peers and building a professional network.  Connecting with IT roles outside of your own.  Connecting with the people who can help make your ideas reality.

• Keynote by Microsoft Developer Evangelist Rachel Appel
• Sessions for Developers, IT Professionals, and Entrepreneurs
• 100+ break out sessions + Open Spaces (self-organizing sessions)
• Panel discussions and live podcasts
• CodeStock Social Event to meet and make connections

Space is limited so register today at CodeStock.org

If you read through all of these posts and aren’t inspired to go create some report that you’ve been meaning to get around to I don’t know what will ever inspire you.  I’m going to summarize them in the order they were received.

Rob Farley shows us some new features that involve easy mapping in SQL Server 2008 R2 with Report Builder 3.0.  VERY COOL!

Michael Coles Was part of the reason this was taking so long.  STOP what you are doing and DO THIS!!

Brad Schulz Well I’m not sure who Ella Vader is but I sure agree with the sentiment about the airlines.  Continuing with the ‘Holy Crap I’ve got to try that!’ theme that we’ve got going is a way to build org-charts or any other type of hierarchy you need.

David Gardiner points us to some useful posts on Microsoft Report Viewer 2010.  Damn handy of him.

Pinal Dave steps us through with screenshots how to configure a Management Data Warehouse. 

Stef Bauer shows us an easy way to setup an HTML table for presenting a cleaner list of events in an email.

Barnaby Self doesn’t want to go off on a rant or anything but would like to show us how to build a report using an MDX query (which I didn’t know is something that you can’t do straight out of the box).

Gethyn Ellis Goes into a more advanced version of something that I touched on in my post.  It’s important to off load your reporting to a different server.  If you don’t the next thing that you will hear is that your organization wants to store everything in XML. 

Jes Borland gives us a method to use when you need to find out what your business needs when building a new report.  If you don’t already have one yourself, print this out and pin it to your cube wall. (Oh and use it!)

Jason Strate steps us through how to create a Wait Stat report and even gives us the code!!

Dave Levy Shows us how to take the dashboard inside of SSMS and give it to anyone that doesn’t have SSMS installed but wants to know why the server is slow.  I will be getting up from my desk now to go show a couple of sysadmins how to do this!

Nicolas Cain gives us an explanation on row versioning.  I will be pointing a few of the developers I work with at this post.  If that doesn’t get them to stop putting WITH (NOLOCK) in their queries I’m just going to find myself a lead pipe (Just kidding, I do not condone work place violence unless it involves the physical safety of the data; and even then I don’t have to worry about it, that’s why our security guards carry guns.)

Stephen Spanos shows us how to create an updateable report.

Jack Corbett Keeps us from looking like fools by showing us how to prevent emailing a report that is blank.  Why has no one ever told me how to do this before?!  Thanks Jack!!

Pat Wright Shows us how to work with vertical tables.  You need to take a look at this.  I don’t know if this is allowed but I asked for a follow-up post because this one hits really close to home on something that I’m working on right now.

Seth Phelabaum Saw into the future came through with something that we’re all going to need after we get inspired and setup a bunch of the reports that these people have showed us are so easy to do.    A way to report on your reports.

Chad Miller shows us how to use PowerShell to report on the connection strings inside of your SSIS packages.

Robert Davis Points out a feature bug of running SSRS 2008 with smart card login on Windows 2008*.  This is not the first time I have seen a bug reported about SSRS causing problems with things that seem completely unrelated so I guess I will just take comfort that the SSRS really like the concept of securing every angle.

Sankar Reddy Answers a freaking excellent question: Who put my database offline?  He tells us about an undocumented feature fn_dblog that lets us know changes that have been going on with our db.  I’m definitely going to spend some time

Michael Swart takes us back to that subject brought up by Jes Borland about gathering requirements and encourages us to be the user.

Paul Randal shows us that we can query internal objects in one database that doesn’t have the functionality we need from another database that does have the functionality that we do need. 

Mike Walsh who hosted last month (and I wish would have warned me just how much reading I was in for!) shows us a feature that I had no idea was in SQL Sentry.  It actually reports database growth for you now.  I use SQL Sentry about 3 times a week and man I’m starting to feel like I need a full day class on all of the new features they’ve packed into this new version.

Kalen Delaney tells us some history info on the COMPUTE BY clause which I didn’t know was being removed.  I have some ways I wish they would improve it instead of removing it and Adam Machanic tells me ‘COMPUTE BY has been replaced by a feature called Grouping Sets – Grouping Sets produces a single result, with NULLs in place of the values for columns that are being grouped’… but I digress back to Kalen’s terrific post.  If you’ve ever tried to explain how a cube works to someone that understands relational data but doesn’t understand dimensional data you’re in luck because she has included a diagram that can really help

Mark Blakey takes us back to the topic of MDX in SSRS reports and shares a rule with us “Rule 1 – Never trust the automatically generated hidden datasets”. 

Adam Machanic the founder of this little shindig answers the question “Why is Adam wasting my time with this garbage?”  He also walks us through some common problems that arise with purely technical reports.  In many of the places that I have worked no one has taken the time to solve these problems.  Well at least now I have somewhere to point them to.

Jorge Segarra shows us how to gets some nice reports out of a SQL Server 2008 feature called Policy Based Management.

Jason Brimhall who technically did two posts on this topic – shows us how he answered the question “why my primary file group was still so large after moving all of the User objects out of that filegroup and into new filegroups?”  I actually posed a very similar question to Paul Randal during the BOF lunch at the 2009 PASS Summit.  Maybe I should finally get around to blogging everything I wrote down from his answer.

Andre Kamman steps us through how to create an index like you find in the back of a book.  Then he take it further and teaches us how to make it look good.

Gabriel Villa Shows us how to set parameters in SSRS.

Adam Haines shows us how to create and email HTML reports.

Stacia Misner shares with us some observation about new properties and globals that you can find in the upcoming release of Reporting Services including PageName, ResetPageNumber, Globals!PageName & Globals!PageNumber, Globals!OverallPageNumber and Globals!OverallTotalPages.

Steve Jones walks us through his solution to one of the most common problems I run into when automating reports.  Two people running the same report at different times and getting different answers.

Aaron Nelson and finally I give some insight into a better way to allow desktop gadgets (and other reports) to report off of your production sales data.  I then walk you through how to create a desktop gadget using a third party tool.

Since this topic was Reporting I thought I’d point out that a free e-book has been release for SQL Server 2008 R2 and Chapter 9 is on Reporting Services.  You’ll probably recognize that one of the names on that book is one of the bloggers that joined us in this round of T-SQL Tuesday.  You can read more about the book here.

That’ wraps up the wrap-up.  If there’s something that I missed or a better summary that I could use please let me know.

Oh, and some advice to future hosts.  Consider taking a half-day off.

Spring Break or not, time marches on and so does T-SQL Tuesday.  I’ll be your host this month and we are going with “Reporting”. 

The Post must go live between 00:00:00 GMT on Tuesday the 13th of April and 00:00:00 GMT on Wednesday the 14th (also known as 8pm EDT, 5pm PDT).

You Don’t Know What T-SQL Tuesday Is Yet?

Adam Machanic (Also on twitter as @AdamMachanic) had a great idea 5 months back – Invite new and existing SQL Server bloggers to post about the same topic on the same day. The results have been excellent – diverse skill sets and data related job roles all posting from differing perspectives on the same issue.

Since this is still early in the game, a quick list of the post roundups from the first 4 topics:

• #001 Adam Machanic started this off with a topic of Date/Time tricks
• #002 Adam then wanted to hear about your Puzzling Situations
• #003 Rob Farley got romantic on us and asked about Relationships
• #004 Mike Walsh wanted to glean the ins and outs of IO

Reporting

Reporting is a potentially huge topic so here are just a handful of ideas that might get you started:

• Server Performance Reporting
• Technologies to help offload reporting
• Reporting services tricks
• T-SQL tricks for presentation and formatting
• Database design considerations for reporting
• Self-service reports
• Reports for executives
• Reporting from Oracle in SSRS

Don’t forget that the DMVs are pretty much qualified as ‘Reports’ so those count too!

Pesky little rules…

Once again, please note the time is in GMT (occasionally called UTC). I also echo earlier encouragements to feel free to write your post ahead of time and schedule it. The rules are around when the post publishes, not when it is written.These rules are the same as the previous couple of T-SQL Tuesday’s but I’ll recap:

1. The Post must go live between 00:00:00 GMT on Tuesday the 13th of April and 00:00:00 GMT on Wednesday the 14th (also known as 8pm EDT, 5pm PDT). If it isn’t, it can’t be included in the round up post.
2. Your post must link back to this post (Trackback or Comment).
3. “T-SQL Tuesday #005” MUST be in the title of the post.
4. It is your responsibility to verify the trackback or comment appears here on this post.  If you don’t see your trackback add your own comment with a link back to your T-SQL Tuesday post and it will be included in the roundup.

Twitter

Not a rule but a great idea. A lot of the folks who read and participate are on twitter. Follow the hashtag #TSQL2sDay and when your post goes live, tweet a link to it with that tag.

Want To Host?

All you have to do is participate in at least two events and let Adam Machanic know. You can tweet him or leave a comment on his blog, all described in his first T-SQL Tuesday Invitation.

If you need any more clarification, leave a comment. I will respond in the comments or update this post if necessary. Have fun thinking of a topic and I look forward to reading the posts!

So today Shyam and I are holding a little contest, We’re each going to put together some code to detach all of the databases on one server, copy them to a new server, and then attach them all.  Shyam is using an approach where he uses a lot of SMO calls, I am using an approach where I do a bunch of SQL queries.  I’ll keep all of you posted on who wins, why, and give you the code once we’re done (minus any edits for security but it will be working code). 

It’s really great to have a coworker like Shyam to hold a friendly little competition like this with.  I can’t wait to see how he ends up solving a couple of the problems that we’ve already come across.  Here’s some of the P. of C. code that we’re both using to get us started; it’s basic ‘code that writes code’ but what we’re going after is ‘code that writes AND executes code’.

This generates the list of databases and files including file# to use with the sp_attach_db command.  I know that in 2005+ you should use ALTER DATABASE … WITH ATTACH but we have a mix of SQL 2000 and 2005 in this project.  (which means I’m actually running this command through a linked server connection on the 2000 boxes.)

SELECT CASE WHEN ROW_NUMBER() OVER(PARTITION BY db.name ORDER BY sf.filename) = 1 THEN ‘GO
sp_attach_DB @dbname=”’+SUBSTRING(db.NAME, 1, LEN(db.NAME))+”’,’ ELSE ‘, ‘ END+‘
 @filename’+CONVERT(VARCHAR, ROW_NUMBER() OVER(PARTITION BY db.name ORDER BY sf.filename))+‘=”’+SUBSTRING(sf.filename, 1, LEN(sf.filename))+””
  FROM master..sysaltfiles sf
  JOIN master..sysdatabases db
      ON sf.dbid = db.dbid
 WHERE db.sid !=0×01
 ORDER BY db.name, sf.filename

This just generates the copy file statements in PowerShell that we will need to move the database files.

SELECT ‘
copy-item “‘+SUBSTRING(
saf.filename, 1, LEN(saf.filename))+‘” -destination “\\DestinationServerName\Drive$\’+SUBSTRING(SUBSTRING(saf.filename, 1, LEN(saf.filename)), 4, LEN(saf.filename))+‘”‘
 FROM master.dbo.sysaltfiles saf
 JOIN master.dbo.sysdatabases db
ON saf.dbid = db.dbid
 WHERE db.sid !=0×01
ORDER BY db.name, saf.filename

I’ve posted a copy of my slide deck that I used for my PowerShell sessions at SQL Saturday #33 in Charlotte.  Anyone is welcome to use all or part of it if they would like to do they’re own PowerShell session.  I’d love to hear any suggestions for improving it.  I’m working on getting all of the scripts I used together so that I can post them up on the Scripting Guys repository.

I’m still not satisfied with the Virtualization slide deck so I’m going to continue to tweak that one and hopefully post that after SQL Saturday #29 in Birmingham.

Because I have too few hours in each day and I still suck at this blogging thing I am combining my MacGyver Moment and TSQL2sDay posts into this one so here we go.

The Beginning:  Once upon a time in a conference room on a very high floor sat over a dozen technical people trying to quickly fix a server that had begun crashing the day before.  This company was pretty big.  Their data, it was massive.  We only worked with summary rows because of this and yet it was still in numbers that make you stop and shake your head every once in a while.  For instance, one time the daily import process failed just a few percent in and it already had over 400,000 rows cleansed and loaded. 

To set the scene a little: We had developers, data folks, server experts, network experts, a SAN expert, the SAN expert’s boss, application SMEs, business liaisons, directors, and the company’s TAM from Microsoft.  Oh and another SAN expert who also doubled as a Microsoft SQL Server MVP who just happened to specialize in High Availability (specifically clustering).  This conference room had been turned into a War Room and was going to stay that way for as long as it took until the problem was fixed. 

The Problem:  A problem child of an application that already had a reputation for crashing every once in a while had now started crashing every couple of hours.  I don’t know the exact specs of the hardware anymore but it was pretty close to the top of the line.  It had something like 16 LPs and 16 Gigs of RAM backed by nearly 2 dozen disks on a SAN that held over 200 disks.  This server had a twin and based on the error logs that was part of the problem.  It was in a single node failover cluster and all of a sudden it had started logging 15,000 millisecond timeout errors while trying to check on it’s disk resources.  This timeout in turn was causing whichever node was active to freak out and think that it had lost contact with the SAN altogether. 

The Situation: While the overall SAN supporting this server is large by a lot of companies’ standards it was actually just a run of the mill small SAN at this place.  In fact we later ordered up another one that looked just like it (only 2 years faster in every aspect) but at the time we kept on being told how solid this particular SAN was.  There were a lot of moving parts and we were a fresh set of eyes being brought in.  One of the moving parts that we had taken note of was that another server on different disks on the same SAN had recently been rebuilt as a 64 bit server.  We took a look and said I’m sorry but your SAN just can’t support the needs of this SQL Server.  You would have thought I had just told the SAN guys that I had caught them cross-dressing or something like that.  They assured me that “No [little] Windows box” could ever take down their mighty SAN.  Regardless of that we were stuck with that particular SAN for at least the next couple of weeks and needed to come up with something to help.

The Other Problem:  We asked for all the IO stats they could give us but unfortunately there was something about a new version that needed to be installed and needing to wait for an outage window and all that.  Bottom line we were stuck with some total IO numbers over a wide enough time frame that they weren’t all that useful however; being a glorified mathematician I took all the numbers I had and tried to figure out the size of the ‘unknown’ portion by subtracting out the three servers I could get stats for myself.  As it turned out in one particular time interval that was just enough information for me to notice a trend.  It wasn’t much but it was more solid than anything else being offered up (or at least I felt so) and the situation was to say the least a little tense.

The Lame-Brained Idea:  One of the sever experts mentioned that he could “borrow” some RAM from a server that hadn’t gone into production yet and we could stick that in the servers (plural because they are a FC) that were crashing to see if that would fix it.  Given the error message mentioned above I just didn’t think this would work.  I had come up with something that just didn’t sound right even saying it but par for the course I said it anyways.  I asked them to humor me and stick the “borrowed RAM” into a box that wasn’t crashing; that other box in the situation that had recently been rebuilt as 64 bit.  They laughed at my idea and I don’t blame them.  Unless you stepped through the situation in the the exact same order as I did this was like saying let’s just reboot and see if that fixes it (oh wait it does reboot every time it crashes ).  It’s easy to decide to stick the only spare RAM you have into a server that’s crashing; it’s a little bit tougher to make the call to bring down a completely different system and have them stick the RAM in that one.  If you’re wrong you’ve just caused a second outage for no reason. 

Stupid Windows Guys:  That’s what they must have thought when they agreed to give my idea a try.  We had all noticed the crashes had been happening almost like clockwork.  Almost but no one could tie them back to anything else happening on that machine.  As we all know, almost only counts in horse shoes and hand grenades.  And occasionally, wild ass guesses.  Funny thing happened though.  We were able to guesstimate the next crash within 5 to 10 minutes at this point only it didn’t happen this time.

The Hidden Pattern:  Now I’ve always enjoyed learning about the storage engine internals but I sure ain’t no Paul Randal.  I am a quick study and love a good disaster though.  What I had found was that there was cube that was being built every couple of hours on the newly minted 64 bit server and while the system did a good job of still letting you query the old cube while the new cube was being built there would inevitably come a point where something was happening that was causing the SSAS engine to flush ooh-gobs of data back down to the disk.  More data than the total size of the cubes even.  The SAN ended up getting saturated with gigs and gigs of non-stop writes (and I hear with RAID 5 that’s like a bad thing but maybe it’s just a rumor).

The Takeaway:  Make sure you know what tools your SAN admin has available and more specifically find out what they can collect on a continuous basis for weeks on end, you might be surprised by the answer.  The disks your server relies on may be fine but the controller cards accessing it might be getting saturated from something else.  You may need to purchase some additional software or something.  Look for solutions in places they might not make sense unless you hold your head just right.  The error log is your friend.  And above all, just because someone says it’s not possible, don’t completely believe them until they prove it to you.

Tag-You’re-it:  To keep the fun going I’m tagging Peter Schott and Matt Schultz (Matt blame Jorge) to find out what their MacGyver moments have been.  Thank you Geoff Hiten for tagging me.

After you download the files and RTFM you can then you can follow along on this next part.  Oh wait, first a little warning straight from Buck Woody:

Script Disclaimer, for people who need to be told this sort of thing:  

Never trust any script, including those that you find here, until you understand exactly what it does and how it will act on your systems. Always check the script on a test system or Virtual Machine, not a production system. All scripts on this site are performed by a professional stunt driver on a closed course. Your mileage may vary. Void where prohibited. Offer good for a limited time only. Keep out of reach of small children. Do not operate heavy machinery while using this script. If you experience blurry vision, indigestion or diarrhea during the operation of this script, see a physician immediately.

get-module -listAvailable

import-module SQLServer

Invoke-SqlBackup -sqlserver "WINX64ULT7\Kilimanjaro" -dbname "AdventureWorks" `

-filepath "C:\Temp\AdventureWorks_db_$(((Get-Date).ToString("yyyyMMddHHmm"))).bak"

foreach ($dbn in invoke-sqlcmd -query "SELECT name  FROM sys.databases WHERE owner_sid !=0x01" `

-database master -serverinstance WIN7\Kilimanjaro )

{

$k="C:\Temp\" + $($dbn.name) + "_db_$(((Get-Date).ToString("yyyyMMddHHmm"))).bak"WIN7\Kilimanjaro" -dbname $($dbn.name)  -filepath $k

$dbn; Invoke-SQLBackup -sqlserver "

}

Background:  We’re doing a Data Center move and they wanted to move a copy of all of the dbs over to the new DC to test the applications.  In some shops this might be easy because they just delete backups more than a day old.  For us, we are blessed with enough room to keep several days worth of backups for some of our critical systems.  This is further complicated by the fact that we have over 100 databases to move and they have different retention policies based on importance. 

I know that the databases take up 1.8 TB thanks to another PowerShell script I wrote but since we use backup compression our data is about a fifth of that size.  But what is the exact number?!  I’m glad you asked.

I threw together a script I had used previously and modified it so that would recurse down the directory structure of our db archive server and pickup on .bak files that were made in the last day.  Now, because nothing is simple where I work, there is a software product – which will remain nameless for now – that also backs up transaction log files with the .bak extension; but luckily also throw in _TLOG_ so I was able to filter them out pretty easily.

   1: $c=0

   2: $l=0

   3: foreach ($f in dir -include *.bak -exclude *TLOG* -recurse |

   4: where{ $_.LastWriteTime -gt [datetime]::Now.AddDays(-1) } )

   5: {$c += 1; $l += $f.length}

   6: "Total File Count: $c and total size of .bak files in Bytes: $l"

   7: >> c:\Temp\MyBackupFileSizes.txt

I went and checked a bunch of these file sizes manually and they came out exactly correct

If you’re new to PowerShell I hope you give this a try and come up with a way that it could be useful to you.

$conn = New-Object System.Data.SqlClient.SqlConnection(“Data Source=YourMachineName\YourInstanceName; InitialCatalog=master; Integrated Security=SSPI”)
$conn.Open()
$cmd1 = $conn.CreateCommand()
$cmd1.CommandType = [System.Data.CommandType]::StoredProcedure
$cmd1.CommandText =“sp_databases”
$data = $cmd1.ExecuteReader()
$dt = new-object “System.Data.DataTable”
$dt.Load($data)
$dt | format-table #PipeLining is Awesome!
$conn.Close()

But this code is so much Nicer!

#Go ahead and add the SQL Snapins
add-pssnapin SqlServerCmdletSnapin100

Conclusion:

Unless you really like writing 6 or 7 extra lines of code every time you query your database, load up the Snapin and use invoke-sqlcmd.

Once I’m done with all this (around May) I will come back and thread these posts together into a bigger story and show you how much time we saved from end to end.  So far I already have about 3 of the bite-sized posts that I need to write and we’ve saved at least 4 hours on tasks that previously would have required loads of manually clicking around.

I hope this format helps people easily take advantage of what I’ve found useful between leveraging SQL and PowerShell.

btw…  If you’re looking to learn more about PowerShell during your daily commute I HIGHLY recommend that you subscribe to the PowerScripting Podcast.  My coworker Shyam and I are catching up on back episodes and listening to it nearly every day on the way into work.  They don’t talk about SQL Server too much but what they do talk about is still very useful to us.