SQLvariations: SQL Server, a little PowerShell, maybe some Hyper-V Rotating Header Image

SQL Server

Get Answers for Paul Randal’s Survey FAST!

You may call it ‘lazy’ but I call it ‘efficiently gathering quality results’. Smile

Last week Paul Randal (twitter) blogged and asked people to take part in a survey asking what the top Wait Type on their systems are.  Since I have 100s of SQL Servers and I wanted to give him an answer for each one. And because I’m lazy…  I modified Paul’s (or Glen’s [ blog | twitter ] ) query to include a ServerName column then Select’ed the results of the query on my local machine into a table. Once I had the table set up I fired up PowerShell and used the same type of script that I showed off in today’s Hey Scripting Guy post. The only difference being that I swapped out the query and table name of course.

CREATE TABLE [dbo].[TopWaitTypes](
  
[InstanceName] [nvarchar](128) NOT NULL,
  
[WaitType] [nvarchar](60) NOT NULL,
  
[Wait_S] [decimal](14, 2) NULL,
  
[Resource_S] [decimal](14, 2) NULL,
  
[Signal_S] [decimal](14, 2) NULL,
  
[WaitCount] [bigint] NOT NULL,
  
[Percentage] [decimal](4, 2) NULL
)
ON [PRIMARY]

Make sure to read to the bottom for the significantly less code version!

foreach ($RegisteredSQLs in dir -recurse SQLSERVER:\SQLRegistration\'Database Engine Server Group'\Development | where {$_.Mode -ne "d"} )
{
$dt=invoke-sqlcmd -query "WITH Waits AS
    (SELECT
        wait_type,
        wait_time_ms / 1000.0 AS WaitS,
        (wait_time_ms - signal_wait_time_ms) / 1000.0 AS ResourceS,
        signal_wait_time_ms / 1000.0 AS SignalS,
        waiting_tasks_count AS WaitCount,
        100.0 * wait_time_ms / SUM (wait_time_ms) OVER() AS Percentage,
        ROW_NUMBER() OVER(ORDER BY wait_time_ms DESC) AS RowNum
    FROM sys.dm_os_wait_stats
    WHERE wait_type NOT IN (
        'CLR_SEMAPHORE', 'LAZYWRITER_SLEEP', 'RESOURCE_QUEUE', 'SLEEP_TASK',
        'SLEEP_SYSTEMTASK', 'SQLTRACE_BUFFER_FLUSH', 'WAITFOR', 'LOGMGR_QUEUE',
        'CHECKPOINT_QUEUE', 'REQUEST_FOR_DEADLOCK_SEARCH', 'XE_TIMER_EVENT', 'BROKER_TO_FLUSH',
        'BROKER_TASK_STOP', 'CLR_MANUAL_EVENT', 'CLR_AUTO_EVENT', 'DISPATCHER_QUEUE_SEMAPHORE',
        'FT_IFTS_SCHEDULER_IDLE_WAIT', 'XE_DISPATCHER_WAIT', 'XE_DISPATCHER_JOIN', 'BROKER_EVENTHANDLER',
        'TRACEWRITE', 'FT_IFTSHC_MUTEX', 'SQLTRACE_INCREMENTAL_FLUSH_SLEEP')
     )
SELECT @@ServerName AS 'ServerName',
     W1.wait_type AS WaitType,
     CAST (W1.WaitS AS DECIMAL(14, 2)) AS Wait_S,
     CAST (W1.ResourceS AS DECIMAL(14, 2)) AS Resource_S,
     CAST (W1.SignalS AS DECIMAL(14, 2)) AS Signal_S,
     W1.WaitCount AS WaitCount,
     CAST (W1.Percentage AS DECIMAL(4, 2)) AS Percentage
FROM Waits AS W1
INNER JOIN Waits AS W2
     ON W2.RowNum <= W1.RowNum
GROUP BY W1.RowNum, W1.wait_type, W1.WaitS, W1.ResourceS, W1.SignalS, W1.WaitCount, W1.Percentage
HAVING SUM (W2.Percentage) - W1.Percentage < 95; -- percentage threshold" -ServerInstance $RegisteredSQLs.ServerName -database master
Write-DataTable -ServerInstance "Win7NetBook" -Database CentralInfo -TableName TopWaitTypes -Data $dt
}

After reading Chad Miller’s (Blog|Twitter) excellent post yesterday though, I decided to combine our approaches.  If you haven’t done so already, you’ll need to download Chad’s invoke-sqlcmd2 and Write-DataTable functions and load them up in your environment.  You’ll also need to store the edit the query from Paul’s blog to add a ServerName column and then store it in a .sql file in the working directory of your PowerShell session.  Once that’s all done if you haven’t given up and closed your browser you can use a piece of code like this to gather up all of this information from all of the server you have in your Registered Servers list. 

For safety’s sake when I put out scripts like this I always make sure to dive down into the \Development branch of my Registered Servers list so that you will always be able to start somewhere ‘safe’ when you run this for the first time.  Or just get a big red nasty error message if you don’t have a “Development” Registered Servers Group Open-mouthed smile

Here’s the code that I came up with for me:

foreach ($RegisteredSQLs in dir -recurse SQLSERVER:\SQLRegistration\'Database Engine Server Group'\Development | where {$_.Mode -ne "d"} )
{
$dt=invoke-sqlcmd2 -ServerInstance $RegisteredSQLs.ServerName -database master -InputFile ./PaulAndGlensWaitQuery.sql -As 'DataTable'
Write-DataTable -ServerInstance "Win7NetBook" -Database CentralInfo -TableName TopWaitTypes -Data $dt
}

Paul had mentioned that The free survey system only allows a single vote per IP address – if you have any other results, send them in email (mailto:paul@SQLskills.com?Subject=Wait stats) or attach a comment below.” so I plan to send him an email with the top wait from each of my servers, minus the server name of course.

I Hope this helps start some ideas in your head about how you can leverage PowerShell in your environment!

By the way…  I got the idea for leveraging the Registered Servers piece from an old Buck Woody ( blog | twitter ) post so pretty much none of this is my code, I just put it all together.  And THAT is why I love twitter!

I’ll be Speaking at SQL Saturday #48 in Columbia, SC

image I’ll be speaking at SQL Saturday #48 this weekend and I am happy to announce I will be unveiling my more advanced PowerShell session to help people customize scripts for their own environment.  I will be showing off several new scripts that I built for my session at the PASS Summit this year.  Please be ready to give plenty of brutal feedback on these scripts so that I can incorporate that feedback in the final versions that I take to the Summit.

Map picture

Besides my two sessions I will be joined by Eric Humphrey ( blog | twitter ) and Microsoft Scripting Guy Ed Wilson ( blog | twitter ) who will also be speaking about PowerShell.  Eric totally stole the idea for his session from me but hey I guess ‘Great Minds Think Alike’ ;-)   Just a quick glance at the schedule shows experts descending on Columbia from Richmond, Tampa, Orlando & Alabama just to name a few so don’t even think of coming up with some lame excuse that it’s too far to drive.  We’ll also be joined by a bunch of SQL Experts from Microsoft’s Charlotte campus so it’s definitely work the trip and I hope to hear you heckle me there!

Start Time Large Session BI 1 BI 2 DB Admin 1 DB Admin 2 DB Dev 1 DB Dev 2 Misc
08:15 AM SQLSat Staff
Opening Remarks
             
8:30 AM Alejandro Mesa
Parameter Sniffing
Jessica Moss
Make Reporting Services Work For You
Stuart Ainsworth
Confessions of a Data Integrator: Bad Designs
Andy Warren
DBA 101: The Basics
Sergey Pustovit
SQL Server Performance Related DMVs
Alex Tocitu
PowerSQL(CLR)
Eric Humphrey
Things To Do With PowerShell & SMO
William Pearson
Attribute Discretization in Analysis Services
9:45 AM Andy Leonard
Database Design for Developers
Wayne Snyder
Information Visualization – Making great Charts
John Welch
Creating Custom Components for SSIS
Janis Griffin
SQL Server Service Broker – An Overview
Aaron Nelson
The Dirty Dozen: PowerShell Scripts for Busy DBAs
Matthew Campbell
Going Spatial
Andrew Kelly
Maximizing Plan Re-use in SQL 2008
Jose Chinchilla
Get Cert! Get Cred!
11:00 AM Sergey Pustovit
SQL Server Diagnostics Tools Unleashed
Evan Basalik
Troubleshooting SSRS Performance
William Pearson
Getting Started with MDX
Sandra Mueller
Data Files and Transaction Logs — beyond the GUIs
Geoff Hiten
Clustering for Mere Mortals
Tim Chapman
How, where, why, and when to use Dynamic SQL
Rafael Salas
Managing Database Schemas With VS201 DB projects
Andy Warren
Building a Professional Development Plan
12:00 PM Stuart Ainsworth
Lunch & Red Gate Software Demo
             
1:15 PM Andrew Kelly
Storage and I/O Best Practices for SQL Server
Andy Leonard
SSIS Design Patterns
Mark Tabladillo
Data Mining with PowerPivot 2010
David Taylor
To click or to type, that is the question.
Aaron Nelson
PowerShell 2.0 Beyond the Dirty Dozen
Brett Tomson
T-SQL Enhancements in SQL Server 2008
Alex Tocitu
CLR 101
Chris Skorlinski
Top 5 fastest ways as DBA to get fired
2:30 PM Ed Wilson
Windows PowerShell Best Practices for SQL DBA’s
Julie Smith
Cool Tricks to Pull from your SSIS Hat
Sandra Mueller
OLTP (yes!) Databases and Cube Design
Ben DeBow
Consolidated SQL Server Architectures
Evan Basalik
Diagnosing connectivity issues with SQL Server
Stuart Ainsworth
You Got XML In My Database? What’s Up With That?
Bob Langley
Introduction To Column Level Encryption
Eric Humphrey
Object Relational Mappers for the DBA
3:45 PM Geoff Hiten
Bad SQL
Jessica Moss
Who Needs a Data Warehouse?
Jose Chinchilla
Business Intelligence: Decaffeinated Please!
Chris Skorlinski
Introduction to Transactional Replication
Janis Griffin
Tuna Helper – Proven Process for Tuning SQL
Brett Tomson
What’s New In SSRS 2008 (With Added R2 Flair)
John Welch
Processing Flat Files with SSIS
Rafael Salas
Planning your ETL architecture with SSIS
4:45 PM SQLSat Staff
Closing Ceremonies & Raffle
             

Speaking About PowerShell at SQL Saturday in Raleigh

image I’ll be speaking at SQL Saturday #46 in Raleigh this weekend!  I’m really excited to be speaking at this event because I talked Microsoft Scripting Guy Ed Wilson ( blog | twitter ) into being a presenter too :-)   and take a wild guess what we’ll both be speaking about!  It wasn’t really hard to get Ed to join us actually; I sent him a tweet saying something like ‘Hey Ed can you come out and speak to us SQL folks’ and his reply was ‘I’ll submit something tonight’.  Might want to keep that in mind the next time that you’re looking for a speaker. 

Map picture

Take a look at the schedule and you’ll see that it’s packed with SQL Server experts so come on out and join
 us, the football hasn’t gotten all that good yet so you’ve got no reason not to.  Also, I have a new demo that Lee Holmes helped me get working that if you love to hate extracting data from Excel, this will make your life much simpler. 

Start Time Track 1 Track 2 Track 3 Track 4 Track 5 Track 6
8:30 AM Andy Leonard
Build Your First SSIS Package
Rafael Salas
Dimensional Modeling: Why Should You Care?
Sergey Pustovit
SQL Server Performance Related DMVs
Kevin Boles
Common TSQL Programming Mistakes*
Flavio Almeida
Self-Service BI with PowerPivot
Geoff Hiten
Clustering for Mere Mortals
9:45 AM John Welch
Processing Flat Files with SSIS
Mark Tabladillo
Data Mining with PowerPivot 2010
Jana Sattainathan
Partitioning in SQL Server 2005/2008
Tim Chapman
How, where, why, and when to use Dynamic SQL
Jessica Moss
Who Needs a Data Warehouse?
Aaron Nelson
The Dirty Dozen: PowerShell Scripts for Busy DBAs
11:00 AM Chris Skorlinski
Introduction to Change Data Capture
Mark Tabladillo
Document Classification using DMX in SSAS
Andrew Kelly
Maximizing Plan Re-use in SQL 2008
Kevin Goode
Embracing the CLR
Mike Davis
Reporting Services 2008 R2, the New Stuff
Brian Kelley
Fortress SQL Server
12:00 PM         SQL Saturday
Red Gate Software
 
01:00 PM John Welch
Patterns for SSIS Configuration and Deployment
William Pearson
Attribute Discretization in Analysis Services
Grant Fritchey
Identifying and Fixing Performance Problems using
Andy Leonard
Database Design for Developers
Jason Hall
Recipe for a Happy DBA – A Guide for SQL Server De
Ed Wilson
Windows PowerShell Best Practices for SQL DBA’s
02:15 PM Andy Leonard
SSIS Design Patterns
Steve Wright
The Shade Tree Mechanic’s Guide to SSAS
Thomas LaRock
Performance Tuning Made Easy
Allen White
XQuery Basics
Geoff Hiten
Bad SQL
Brian Kelley
The Dirty Business of Auditing
3:30 PM Chris Skorlinski
Integrating CDC and SSIS for Incremental Data Load
Mike Davis
Using Parameters in SQL Server Reporting Services
Kevin Goode
Statistics, how to prove everything but the truth.
Kevin Boles
Advanced TSQL Solutions
SQL Saturday
Q & A — Stump the Speakers
Amy Styers
Virtualizing SQL Best Practices
4:30 PM         SQL Saturday
SWAG Give Away
 

Picture the Index: T-SQL Tuesday #10 – Indexes

TSQL2sDay150x150I don’t really have the time to write this month, but indexes are such a key feature of databases that I thought I should.

A whole post would take more time than I have. Instead I’m going to mention something that Rob Farley (of LobsterPot Solutions in Australia) said in a SARGability talk for the AppDev Virtual Chapter of PASS a few months back.

Rob talked about the way that we find entries in an index using a Seek operation, and compared to using a phonebook. In particular, he described the bit at the top corner of the phonebook, which we use to find the right page before looking at each record. This is very much how an index works. An index is stored in a b-tree, with the levels at the top being like the corner sections in the phonebook, and the leaf level of the tree being like all the records there. 

A quick bit of searching flickr found an image at http://www.flickr.com/photos/blinky5/376596220/ which describes this perfectly. If you’re looking for Wilma Todd, you can use the top corner to figure out which page she’s on and then find her record on the page very easily.  So what’s my point?: The next time that you’re picking a data type for a column think of this.  If you use a data type that’s twice as large as what you really needed (int vs. smallint, nvarchar vs varchar) and then you realize you need to put an index on it, it would be be like doubling the font in this phone book thereby requiring twice as many pages to print the book.  May not seem like much but when you have to scan for data you’re now going to have to physically touch twice as many pages.  And hey, now the books twice as heavy as it needed to be just like your backups are going to be! 

Phonebook

Anyway, I don’t have any more time to spend on this post, gotta get my code ready in case I need to fill in tomorrow for someone during the 24 Hours of PASS, so I’ll just publish it and wish you all a Happy T-SQL Tuesday.

Columbus and Edmonton Code and Slides

I spoke in Columbus, GA and Edmonton, um…, somewhere in Canada on back to back nights this Tuesday and Wednesday. (I’m kidding, I know it’s in Alberta, and you guys like the Jets right? ;-) ) I used the same slide deck for both sessions; well, for Edmonton I didn’t even use a slide deck, but the scripts I covered were a little different.

image I’ve recently learned that if you only have PowerPoint 2007 you’ll need to download this viewer because I built the slides in PowerPoint 2010. Here are the scripts for Columbus and Edmonton.

I think both events went really well and got a lot of people saying: ‘Hey, this PowerShell stuff could work in my environment’. I really hope they catch the fever! I’d love to speak to each of these groups next year and see what questions they come up with and see how much they’re using it. I’ve got a more advanced session that I will be debuting a little later this year so hopefully I’ll get to share it with them too.

Huge thanks to Jonathan Boulineau ( twitter ) and Colin Stasiuk ( blog | twitter ) for all they do for their groups all year long.

I'll be speaking at the Columbus, GA PASS Chapter

Next Tuesday I’ll be speaking at the Columbus, GA PASS Chapter.  This was actually supposed to be the first time for me to speak at a User Group meeting (as opposed to a SQL Saturday) but duty called in Atlanta this week. 

As always, there will be something new that I’ve never showed off before, most likely just a failed script from my Atlanta session but hey, they didn’t get to see it yet.  :-)   I’ve also been asked to include some sys-admin type stuff so I will do a walk through of Implicit Remoting and probably some important tips on working with Services.  There will also be an XML demo if there are any developers that want to check it out.

Map picture

Details:

PowerShell Week at SQL University – Post 1

Getting Started With PowerShell Variables

Yesterday we laid the ground work for PowerShell Week.  Today we will learn about Variables in PowerShell. You will notice that PowerShell variables work very differently from SQL Server variables.

The evolution of variables in SQL Server is pretty straight forward. In SQL Server 2000 you had to DECLARE a variable as a specific type in one statement and then SET the value of the variable in another statement, like this: DECLARE @SQLvariable VARCHAR(12) SET @SQLvariable = ‘OldFeature’. By the time SQL 2008 hit the streets we could DECLARE and SET a variable in one statement like this: DECLARE @SQLvariable VARCHAR(12) = ‘NewFeature’ . PowerShell evolves variables to the next logical step.  You no longer need to declare or type your variable. Instead PowerShell uses the value of the variable to infer it’s type. If a variable does not alreayd exist PowerShell simply creates it for you.  In PowerShell our statement looks like this: $SQLvariable = ‘NewFeature’. (Beginer tip: $ identifies variables in PowerShell just like @ identifies them in SQL Server.)

Since it doesn’t work in SQL let’s try it in PowerShell: $SQLvariable = ‘NewFeature’.  OK so we loaded data into a variable, ‘big deal’ right?  How do we know that we actually loaded it and more importantly what datatype is it?  Well in PowerShell you can simply call the variable to get its value. (Another beginer tip: you don’t need to use a keyword like SELECT for this, just the variable name: $SQLvariable.)  What if you just want to determine the datatype?  All you have to do is tack on the .GetType() Method: $SQLvariable.GetType().

$SQLvariable = 'NewFeature'
$SQLvariable
$SQLvariable.GetType()

When you run this here’s what your result will be:

image

As I alluded to earlier, another feature of PowerShell is that it remembers your variables for you, even between executions.  Basically as long as you don’t end your session the variable, it’s datatype, and its value will persist.  You will see later than you can also set a variable in one script and consume it in another.

Let’s try this again with a new variable and set it to a different value:

$PoShvariable = 123
$PoShvariable
$PoShvariable.GetType()

image

What happens if once the datatype is set we try to set it to a different, incompatible datatype?  It’s going to explode right?  Nope, you’ll have to throw something more difficult than that at it.  Give this a try:

$PoShvariable.GetType()
$PoShvariable = 'NewFeature'
$PoShvariable
$PoShvariable.GetType()

 

 image 

Since PowerShell relies on .NET a variable could be any datatype in .NET so if you’re sitting inside the AdventureWorks database you can see for yourself, just run this:

$MyTable = get-item TABLESProduction.TransactionHistory
$MyTable.GetType()            

$MyStoredProc = get-item StoredProceduresHumanResources.uspUpdateEmployeePersonalInfo
$MyStoredProc.GetType()
image

Now earlier I mentioned that PowerShell keeps track of and remembers the variables that you previously declared.  So how do you see what variables are already there?  Just run this command and you can see every variable you have running in the session, even the internal ones.

Variable

In the list of items that get returned you’ll see one called Error that holds the last error message that occurred in your session.  This handy feature can help you when you have to debug your own work.

Is that all there is to know about variables in PowerShell?  Not hardly, but I think that’s a good stopping point for this post.  More on variables later.

I’ll Be Speaking at geekSpeak

Next week I am giving a PowerShell for Data Professionals session on MSDN’s geekSpeak.  If you haven’t heard of  geekSpeak all you really need to know is they have one primary rule for presenters:Your webcast is limited to only 1 Power Point slide. geekSpeak presentations focus on practical live examples and interactive question and answer sessions that allow you to learn from a presentation in a way that you can immediately start applying to your job.

https://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032449694&EventCategory=4&culture=en-US&CountryCode=US

geekSpeak

If you haven’t caught this presentation at a SQL Saturday yet, or if you want to see it again, then this is a chance to watch from your desk and follow along in Powershell for yourself.

The show is at 3pm Eastern/Noon Pacific.  You can see previous geekSpeak shows and subscribe to their feed here.  Please stop by and heckle me check it out. :-)