I have a session on the ballot that’s called Using Columnstore Indexes in SQL Server 2012 and below is the session abstract for you to review.  You might not be sure whether to vote for my session or you don’t know whether you need to attend session on columnstore indexes, so I’ll give you one really good reason.

The Reason for Learning about Columnstores

Columnstore Indexes are really important because in SQL Server 2012 they will have to be seriously considered by DBAs and BI Architects as a replacement for cubes (or as an alternative) and as the primary indexing scheme for star schemas, report subsytems, data warehouses and any other reporting and analysis scenario.  Making the right decision will require you to have in-depth knowledge of how these indexes work.  That’s what you’ll get if you attend my session.

Session Abstract: Using Columnstore Indexes in SQL Server 2012

Columnstore Indexes in SQL Server 2012 will allow you to significantly improve the processing time of common data warehousing queries without creating cubes, aggregated tables, or other techniques normally used to improve performance. This session will show how to implement this new type of index in SQL Server and demonstrate their advantages compared to traditional solutions. Carlos will also discuss the scenarios for which columnstore indexes should be implemented to provide powerful but flexible BI solutions.

More Columnstore

Go to this blog post I wrote several weeks ago for more info on columnstore indexes.

The Future

Instead of going to the past, I decided to go to the future and write about the new features in SQL Server 2012 that will help us with aggregation, a subject that should be close on any BI developer’s mind.  I chose the topic because of T-SQL #016 by Jes Borland on Aggregations since there are going to be some interesting new functions to help us aggregate better.

Windows in T-SQL

The Windowing functions (which have nothing to do with Windows) in the upcoming version of SQL Server let us do in one query Analytic calculations that include sliding aggregations, standard deviations, and more.  This way we can compare averages, sums, and other numbers using preceding and current rows in a much cleaner, simpler way than we can today.

SELECT OrderDateKey, DailySales,
WeekAvgCurrent = AVG(DailySales)
OVER (ORDER BY OrderDateKey ROWS BETWEEN 6 PRECEDING AND CURRENT ROW),
WeekAvg1WeekAgo = AVG(DailySales)
OVER (ORDER BY OrderDateKey ROWS BETWEEN 13 PRECEDING AND 7 PRECEDING),
WeekDiff = AVG(DailySales)
OVER (ORDER BY OrderDateKey ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) – AVG(DailySales)    OVER (ORDER BY OrderDateKey ROWS BETWEEN 13 PRECEDING AND 7 PRECEDING),
WeekStdDev = STDEV(DailySales)
OVER (ORDER BY OrderDateKey ROWS BETWEEN 6 PRECEDING AND CURRENT ROW)
FROM (SELECT f.OrderDateKey, SUM(SalesAmount) as DailySales
                  FROM FactInternetSales f
                  inner join DimDate dd on dd.DateKey = f.OrderDateKey
                  where dd.CalendarYear = 2003
                  group by f.OrderDateKey) as DailyTable
order by OrderDateKey

The query produces this result from AdventureWorksDW. Only the first 12 rows are shown and I modified the column titles slightly for readability:

The results show you that we can get the average sales for the last 7 days, the sales for the 7 days before that, compare the two to get the difference between the last two weeks, and the standard deviation for the week.

All in one fairly simple straightforward query. Very powerful.

Many thanks have to go to Chris Shaw (T | B) and Jeremy Lowell (B) who put in countless hours to make the event so successful.  They brought in excellent sponsors, put together an all-star lineup of speakers, had good prizes, good food, and numerous activities including a ski trip.  Other tireless volunteers included Gabriel Villa (B|T), Andrew Dykstra (T), Rebecca Mitchell (B|T) and several others.

My Session

My session on Columnstore Indexes was well attended, had good audience participation and great questions.  Thanks to everyone who came.  I hope I was able to help you take a small step in getting ready for the coming of SQL Server 2012.

The People

My favorite part?  Talking to people.  There’s nothing better to get your juices flowing and head spinning than having spirited discussions on tech and non-tech subjects with Bill Pearson (T), Jim Murphy (B|T), Jeff Renz (T), Jason Horner (B|T) and so many others.

Using Columnstore Indexes in SQL Server 2012

 
Columnstore Indexes in SQL Server 2012 will allow you to significantly improve the processing time of common data warehousing queries without creating cubes, aggregated tables, or other techniques normally used to improve performance. This session will show how to implement this new type of index in SQL Server and demonstrate their advantages compared to traditional solutions. Carlos will also discuss the scenarios for which columnstore indexes should be implemented to provide powerful but flexible BI solutions.

Click here for a blog I wrote last week with more details about my session. I’ll see you in the Springs!

If you’re interested in Business Intelligence like I am, there are two BI tracks for you to follow.  I’m presenting a session titled Using Columnstore Indexes in SQL Server2012 at 11am in one of the BI tracks that will give you a good overview of one of the new features coming this year in SQL Server.  It’s especially useful if you’re looking to get faster query response times in your data warehouse, reports, or ad-hoc querys.

More and more, events such as SQL Saturday have become what database professionals rely on for their training.  It brings together a boat load of SQL Server knowledge together all at once in your own city and makes it extremely accessible.  There’s no better place to be if you need to know more about database performance, business intelligence, the cloud, or database administration.

My presentation, Using Columnstore Indexes in SQL Server 2012, will include demos that cover the important things to know about Columnstore indexing. If you haven’t heard much about this type of index structure, here are a few things to know:

-          Intended for Data Warehousing, Reporting and Ad-hoc Queries
-          Does not use B-Tree structure like row indexing
-          Can only have 1 per table
-          Does not allow table to be updated
-          Best built with columns that have non-unique, repetitive data
-          More advantage is gained from bigger data
-          Column order doesn’t matter
-          Maximum benefit gained with queries that aggregate data
-          Don’t use for single-row type queries
-          Memory-intensive to build and execute

It’s Fast

Why build a Columnstore index? It can be very fast to query, much faster than the typical row-based indexing we are used to. They’ll make reporting faster, and they can be part of the solution for the elusive ad-hoc query challenge that power business users want.

Creating a Columnstore

A Columnstore can and probably should contain every column in the table with a few exceptions.  Unique data columns in some cases should be avoided.  Also, creating a Columnstore index requires a lot of memory, so you can reduce the stress on your system by not including every column.

Using the Index

The SQL Server optimizer will decide if it can benefit from using the index just like it does with other indexes. You’ll be able to check the execution plan to see if SQL Server decided to use it.

Here’s a preview of the type of comparisons, analysis and best practices I’ll be presenting at SQL Saturday #104. I’ll focus on comparing this new type of index with row-based indexes and I’ll help you determine when the right time to use it is.

This query uses the AdventureWorksDW and is a common type of query when using a dimensional model. It aggregates data from a fact table while joining to dimension tables:

SELECT dp.EnglishProductName, dp.ProductLine, dd.CalendarQuarter,
sum(SalesAmount) as TotalSales
FROM FactInternetSales f
inner join DimProduct dp on dp.ProductKey = f.ProductKey
inner join DimDate dd on dd.DateKey = f.OrderDateKey
where dd.CalendarYear = 2004
group by dp.EnglishProductName, dp.ProductLine, dd.CalendarQuarter
order by dp.EnglishProductName, dp.ProductLine, dd.CalendarQuarter

This is the I/O and CPU usage:

CPU time = 200 ms,  elapsed time = 493 ms.
Table ‘DimProduct’. Scan count 0, logical reads 516, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table ‘Worktable’. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table ‘FactInternetSales’. Scan count 1, logical reads 2062, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table ‘DimDate’. Scan count 1, logical reads 66, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

And the execution plan which is showing us that SQL Server is doing a clustered index scan on the fact table:

Columnstore Query

I’ll run the same query after adding a Columnstore index that includes every column in the FactInternetSales table.  Below is an abbreviated version of the DDL:

CREATE NONCLUSTERED COLUMNSTORE INDEX CX_FACT_INTERNET_SALES ON dbo.FactInternetSales
(
ProductKey,
OrderDateKey,
DueDateKey,
…
)

Executing the same query produces the following results:

CPU time = 80 ms,  elapsed time = 343 ms.
Table ‘DimProduct’. Scan count 0, logical reads 516, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table ‘Worktable’. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table ‘FactInternetSales’. Scan count 1, logical reads 64, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table ‘DimDate’. Scan count 1, logical reads 66, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

As you can see from the execution plan, the query optimizer used the Columnstore index this time.  The CPU utilization was reduced from 200ms to 80ms and the logical reads went from 2062 to 64 on the FactInternetSales table. The disclaimer is that this is not a scientific test, but it’s a quick preview of what Columnstore indexes can do.

SQL Saturday #104

This is a snippet of what you’ll see if you attend my presentation in Colorado Springs. I’ll use larger tables to demonstrate how to get the most out of Columnstore indexes, how to use them with a partial set of columns and with partitioning, and I’ll compare them to their equivalent row-based queries.

With a good data model we can develop a fair number of reports with queries like this, but there are more complex queries that we need to write.  SCD type 2 dimensions ensure that fact data is related to dimension data in the state the dimension was in when the fact was created.  However, to report accurate sales results requires us to stray from the typical star schema query.

A requirement comes to you asking you to compare 1^st quarter sales with 2^nd quarter for each business unit in your organization, but to only include business units that have greater than $100K in sales year to date.

The first thing we do is get a list of business units with YTD sales greater than $100K.

Select  bud.bus_unit_id from Sales_Fact sf
Join Business_Unit_Dim bud on bud. business_unit_dim_id = sf. business_unit_dim_id
Join Date_Dim dd on dd.date_dim_id = sf.date_dim_id
Where dd.Year_YYYY = 2012
Group by bud.bus_unit_id
Having sum(sales_amt) > 100000

The Business_Unit_Dim table has SCD type 2 changes but the query doesn’t care.  It will return the right results even if a particular business unit has been changed 100 times this year.  The key is getting the right list of business unit ids and then using them as a subquery in the outer query.

The outer query looks like this:

Select bud.business_unit_name, dd.quarter_name, sum(sales_amt) from Sales_Fact sf
Join Business_Unit_Dim bud on bud. business_unit_dim_id = sf. business_unit_dim_id
Join Date_Dim dd on dd.date_dim_id = sf.date_dim_id
Where dd.Year = 2012 and dd.Quarter in (1,2) and
Bud.business_unit_id in
(Select  bud.bus_unit_id from Sales_Fact sf
Join Business_Unit_Dim bud on bud. business_unit_dim_id = sf. business_unit_dim_id
Join Date_Dim dd on dd.date_dim_id = sf.date_dim_id
Where dd.Year_YYYY = 2012
Group by bud.bus_unit_id
Having sum(sales_amt) > 100000)
Group by bud.business_unit_name, dd.quarter_name
Order by bud.business_unit_name, dd.quarter_name

This example shows us how to trend data over time using a star schema and that it’s the date dimension that give us the ability to do that.  It doesn’t depend on SCD changes and it shows we often need to query a star schema to get the right subset of data we want first, and then we query the star again to get the ‘report’ data.  It also shows us that a healthy library of queries that we can use as subqueries can greatly improve our query development process.

The myth:  On the surface you might think the Summit is a boring trip back to your school days, when you had to sit and listen for days on end to that economics professor talking about macroeconomic theory.  Could there be anything drier than meeting with a bunch of DBAs and attending sessions on clustering, indexing and reporting?

The truth:  The Summit is a vibrant event full of positive atmosphere and lively personal interaction.  There are people all over who want to help you, meet you, and talk to you.  Learning, teaching and discussing with others, that’s what the Summit is all about.

Outside of the sessions I attended, I spent 90% of my time at the Summit meeting people, shaking hands, introducing myself, hearing about their backgrounds, handing out my contact info, and reconnecting with SQL friends.  Even with all that networking, I still didn’t get to meet everyone I would have liked to meet.  I attend a conference like this with several goals in mind and one of those is to introduce myself to people who have sparked my interest because of the work they do or something they’ve published.

There’s so much going on that I couldn’t fit everything in that I wanted to do.  I just waived hello in passing to several familiar people but didn’t get a chance to talk to them.  I completely missed a few people that I really wanted to meet.  One was Mark Ginnebaugh (B|T), who seems to run everything in the Bay Area.  I wanted to get together with Mark because I have a reason to be in SF at times and he seems like the guy to know.  I also wanted to talk to Audrey Hammonds (B|T), who had written a blog post that stuck with me urging women to speak at technical conferences.

I even got to meet Jes Schultz Borland (B|T)….even though it was brief, Jes is just like I thought she would be

That’s why I’m going back next year, to talk to the people I missed, and to meet the people I didn’t.

I hope to see you there and please say hello!  I welcome any questions and conversation you might have.

If you’re from out of town and new to Portland, take a stroll downtown and along the Willamette river.  Pop in to a local coffee shop and drink the city’s own Stumptown coffee.  Go to Powell’s bookstore for one of the best book browsing experiences you can have.  Make sure you take in the view of Mt. Hood if it’s a clear day.  You can always go to the nationally famous Voodoo Donuts where I find the standing in line and meeting people experience more interesting than the donuts themselves.  And if you want good food and drink you can’t miss if you partake at Pine State Biscuits, A Pizza Scholls, Deschutes Brewery, Moroccan cuisine at Dar Essalam in Wilsonville, or Petite Provence, probably the best french boulangerie/patisserie in the world (sorry France!)  I don’t think I’ve had a disappointing meal yet in Portland.   To top it off, the SQL Saturday event itself is on the waterfront.

See you there and don’t forget to come to my session Real-time Data Warehouse and Reporting at 10:15 in the morning.  I’m going to cover numerous topics in a little over an hour; SQL Server Replication, CDC, Star Schemas, SSIS, and Reporting, all in real-time.