Excel on the Left.  Other Data Tools on the Right.

Today I’m going to “get my nerd on” in a big way.  Buckle up.

The genesis of this post is an email I’ve been meaning to send to my contacts at Microsoft – one I’ve been thinking about writing for at least a year.  But I also figure it’s the sort of thing you folks might find interesting, and I really don’t have time these days to write the same “opus” twice, so here goes – two birds with one stone.  And it’s a friendly stone.

Has there ever been a tool as flexible as Excel?

Let’s take a moment and just marvel at Excel’s “range.”  (VBA macro programmers – yes the joke is intentional).

You can do damn near anything in Excel.  Calendar chart visualizations.  Music videos.  Beautiful art.  More music videos.  Respiration wavelengths.  Chess games.  Word clouds.

But those are just the outliers really – the novelties.  The truly valuable examples are much less dramatic and happen hundreds of thousands of times every day.  I’ll give some examples in the next section.

Feature A Was NEVER “Intended” to Be Used With Feature B!

“Hey You Got Your Slicer in My Conditional Formatting!”
(And then the whole jar fell into a bucket of DAX)

We don’t have to look far for examples:  just this past Thursday I showed how Excel’s “Show Values As” feature can be used to quickly generate a Pareto-style analysis on top of a PowerPivot model.

Quiz:  did anyone at Microsoft ever think about this?  Probably not!  It’s just… something that’s possible, waiting for us to discover it.

How about combining pivot sorting, slicers, DAX, and a hidden column to enable sorting by slicer?  I’m pretty comfortable guessing that I was the first person on the planet to try that, and did so long after leaving Microsoft.

Not enough?  Staying right here in just this tiny little corner of the Internet, how about conditional formatting controlled via slicer?  Or using wingdings fonts to turn slicer tiles into icons?  Using the hyperlink function to create “drill down into new report” functionality in reports?

None of these things was ever intended or planned by Microsoft.

But that is not a “slight” to Microsoft – in fact it’s a testament to something truly beautiful.

The Network Effect:  How 20 Becomes 11 billion.

Let’s be really conservative and say that Excel has 500 functions and features (it’s actually many more than that).

And then let’s observe that we commonly see 4 such features “strung together” to create a Nifty Application.  Roughly speaking then, there are 62 billion potential Nifty Applications out there in the universe.

So, no one is going be running out of clever new tricks to discover any time soon

But let’s say Microsoft gets lazy and only adds 20 new features to the next version of Excel.  How many combinations do we have now?  73 billion Nifty Applications – an increase of 11 billion.

Stated differently, a 4% increase in functionality led to an 18% increase in utility.  Spend $4, get $18.

In Excel, there really IS a free lunch.  (Well, “buy one get three and a half free” anyway).

Other Tools (BI Tools in Particular) Have No Network Effect!

I’ve been looking at a lot of BI tools lately.  And while they often impress me in certain ways, they all lack a network effect.  They all pretty much do exactly what they were designed to do and nothing else.

Why is that?  Well for one thing they lack a grid.

In Excel, all of the Happy Little Features Communicate and Cooperate via the Grid
(More Accurately, via the Calculation Engine Underneath the Grid)

Islands in the Stream! 

Without the grid, and the calculation engine underneath, all of Excel’s features would just be islands.

Oh I’m sure they’d be connected in small groups.  Slicers would of course be connected to pivots for instance, and charts would be connected to their data.

But those would be isolated groups with no broader network effect!  The grid and calc engine in Excel form the hub through which all of its features cooperate.  Data in pivots can be referenced via formulas, chart titles can reference into the grid, conditional formatting thresholds can reference into the grid, etc.  Excel’s features respect the calc.  They respect the grid.

BI tools lack flexibility, and a network effect, because they lack that underlying hub -  a highway that all features connect to in a read/write manner.

And this is one big reason why “Export to Excel” is the third most common button in BI tools – after OK and Cancel (one of my favorite all-time posts BTW).

DAX Engine:  MASSIVE Expansion of the Network Effect!

PowerPivot’s DAX Engine is “Plugged In” To the Hub
(Benefit Not Drawn to Scale)

I think that picture pretty much sums it up.  In some sense it’s fair to call the DAX engine “Excel’s second hub!”  I like the sound of that.

Sure, the DAX engine doesn’t “listen” to the Excel grid, but it does listen to other features like Slicers, and the grid listens to it (via pivots and cube formulas, and in 2013, also via Tables).  And I guess in some sense, the DAX engine does respond to the calc engine when you’re using cube formulas.

But for my money, the DAX engine is very much part of the network.  It falls in the Major Clapping of Hands Department.

What Keeps Me Up At Night

Keeping with the trend of illustrations…

Many of the Recent Additions to the Excel Family Do NOT Yet Participate in the Network Effect
(In fairness, Data Explorer is pretty close to earning its “Network Badge.”)

Let me tell you a secret.  Adding new features to Excel is HARD work.  Shockingly difficult work.  I used to joke that changing the font on the bold button would take six weeks.  (I exaggerate – in reality it only would take four weeks).

You can’t just drop a new batch of program code into Excel and expect it to work.  Adding features to Excel is a lot like implanting a new organ into a transplant patient – you must carefully open the patient up, make room for the new functionality, painstakingly stitch it into the circulatory system, carefully close the patient back up, and then monitor the patient for several months afterwards.

For instance, the Analysis Services team loaned some engineers to the Excel team in order to implement Slicers.  (That’s right, we would NOT have slicers today if not for those brave souls who volunteered to uproot and move across campus).  Those engineers were supposed to be on loan for about 9 months.  They were gone for two years, and more engineers got sent over than originally planned.

I think, in hindsight, this was judged to have been a poor investment of resources, because everything done since then has a distinctly “bolt on” feel.

Save 75%, Lose 75%.  Then Lose Some More.

Listen, I can absolutely sympathize with that stance.  Who wants to spend 4 engineer-months on a feature when they can get it done in 1?  It’s crazy – Excel features come at a staggering engineering cost.

But the losses pile up too when you bolt things on rather than deeply integrating them.  You don’t gain that network effect advantage – an advantage that multiplies the value of your investment.

So you save 75% of the implementation cost.  But you lose 75% of the benefit you would have gained from integration.

But only longtime Excel-heads realize that there WAS a 75% functionality loss!  From the outside, you built a feature and it does what you expected!  You perceive no loss!  I worked on Excel for years without understanding that there was a massive loss.  We should not expect the SQL teams at MS to understand it either.  But we sure can hope that they try.

Furthermore, the bolt-on approach yields a feature that feels like an alien to Excel folks.  It’s hard to learn to use.  It’s hard to “dip your toe in” by incorporating it into a broader, more traditional spreadsheet.  And then it’s supremely frustrating for the user when they discover that it’s not compatible with other Excel features like they expect.

So they walk away.  I hate to say it, but in its current “island” form, Power View does not get me terribly excited.  On the other hand, if suddenly I could have Card Views or animated charts as part of a normal Excel spreadsheet, with some degree of slicer and calc chain integration…  I’d go from Mildly Interested to Full High Priest Mode.

“Composeable” is the 1-word summary

Another way to have worded this post would have been to say that Excel is strong because its component features are “composeable” with each other.  (Yes I am making up a word).

Composeable – please think about it, dear friends in Redmond.  It’s crucially important, and it’s a big difference between Excel and traditional BI tools.  It’s taken me most of my career to figure that out.

“Computer?  Tell me how many products account for the top half of all sales?”

The scene in Star Trek IV where Scotty picks up the mouse and tries to instruct the computer by speaking into it was, in my opinion, the lone bright spot in an otherwise lackluster movie.  We all feel the same way sometimes of course.  We just want to say “hey computer can you just do what I want without making me trick you into doing it?” 

(Related:  I had a professor once who liked to say “as computer scientists we are trained to communicate with the dumbest things in the world – computers – so you’d think we’d be able to communicate quite well with people.”  He also writes a science and politics blog and was one of my favorite professors.)

OK, so how do we build something like this in PowerPivot:

The Handful of Products Shown Account for 50% of All Sales – But How Did I Build This?

First, Don’t Make it Harder Than It Is!

A large corporation recently hired me to introduce PowerPivot into their reporting and analysis teams.  And on day two, they asked me how to do what is sometimes called Pareto Analysis – if I rank products (or customers) from 1 to N in terms of their value, how far down that list do I need to go before I hit 50% of all sales, or 80% of all sales, etc.?

Well my first instinct was to do something fancy in DAX.  Something “clever,” as Tyler Durden would call it.  And I think I’ll share that approach on Tuesday.  So I launched into a DAX approach on the whiteboard.

But then someone asked “hey can’t we just use Show Values As?”  I paused and scratched my head.

Um, yes.  Show Values As – an Underrated Tool

Check this out.  DAX-free solution.  You don’t even need PowerPivot for this – it works in a regular pivot too.

Just add Product and Sales to your pivot like you always would:

Product on Rows, Sales Measure on Values – Nothing Fancy

Then Right Click One of the Sales Cells, Sort Largest to Smallest

So now you have a sorted pivot:

But now for the magic trick.  Right click again, choose Show Values As, and Running Total In:

That pops a dialog.  Choose the field that is on Rows – Product Name in this case:

Now you get this:

This is Now a Running Total!  No DAX!

Go back to the Show Values As flyout and this time choose % Running Total In:

Switch from Running Total to % Running Total

And then we get:

Here’s Where We See Just These First 13 Products Account for 50% of All Sales

Tradeoffs

OK, longtime Excel pivotheads are yawning right now.  “Come one Rob we’ve known about this forever.”  I have a confession to make:  I’ve been aware of it too, but I hadn’t really paid too much attention to all of the options on that flyout.  Percent of total was the one option I’d used a lot of and the others just seemed to disappear into the background for me.

So thank you Lisa for bringing this back into focus for me

For other folks who’ve been up to their eyeballs in DAX for awhile, take a moment and review the options on this sucker:

So Many Options!  Take a Tour, Try Them Out.

But there’s a catch.  PowerPivot has no idea that the numbers have been changed to percent running totals.

For instance, let’s write a new measure:

[My New Measure]=
[Total Sales] * 2

and see what we get:

My New Measure Returns 2x the Original Dollar Sales Value, Not 2x the % Running Total Value

So if you want to use your fancy % Running Total value in another measure calculation, this Show Values As technique is no help.

This is because even in the % running total column, PowerPivot sent only the original sales numbers to Excel.  At the last minute, before displaying those numbers, Excel itself runs some calculations of its own (think of it as “post-processing” of the numbers given to it by PowerPivot) and then displays those results.  The PowerPivot calculation engine never sees those % running total numbers, in other words.

On Tuesday I plan to show a DAX solution to this problem for contrast.

There Are More than 18 Copies Left at the Warehouse.  But Not THAT Many More

When I was down at MrExcel HQ With Chandoo this weekend, Bill and I decided to check how many copies of the book were left in inventory at the distributor.

We printed 3,000 copies back in November.  The distributor only had about 350 left at the warehouse as of Friday.

Neither of us expected the inventory to be that low.  You see, book sales data isn’t exactly a timely business – Bill and I just got paid in April for the books that sold in November.  And while Nielsen does provide a window into how many books sold this past week, they only capture part of the market.  The proportion out there “in the shadows” is unknown, and it’s significant.

So much like what we see in the night sky is light that left distant stars many years ago, our only reliable sales data lags reality by 4-5 months.

Good News Bad News

Hey good news – the book has been selling even better than we knew!  Bill tells me that most books never sell their entire first print run, so to be running out so quickly is a very good sign.

Hey bad news – we gotta scramble to print a second run before the supply runs out

Fixes!

This presents an opportunity to fix a few things in the book:

1. The Title.  Reviews of the book repeatedly stress how human, informal, and understandable it has been to read.  I find this particularly gratifying.  But it highlights what a terrible job I did picking a title.  By far it has the most intimidating and non-human title of all the PowerPivot books out there.  So I think we’re going to be bold and rename it.
2. Referencing the supporting files early.  Many people have made it all the way to the end of the book before discovering the hyperlink to the supporting files.  That also seems pretty boneheaded and will be corrected.
3. Ken Puls.  Rounding out the list of boneheaded mistakes, Ken was the guy who convinced me to write the book, and I didn’t thank him in the acknowledgments!  This mistake will be erased from history, and the first edition will come to be regarded much in the same way as mis-struck coins.
4. Typos and other mistakes.  There are a number of smaller little gotchas that people have pointed out to me that will also be fixed.  I have a list of these (in Excel of course!).  If you’ve got something you think I should look at, let me know and I’ll send you my list – it maybe be that I’ve already got it covered.

Two Other Positive Angles

One – this is a good sign for PowerPivot, too.  You can’t sell thousands of books in a short time unless adoption is growing.

Two – publishing through Bill’s company is just a phenomenal way to get a book to market.  As far as I can tell, I got all of the same benefits I would have received if I had gone through one of the big labels, but I retained creative control (got to be myself!), share copyright to my own work, and get about twice as much per book sold as I would through the larger houses.

If you are thinking of writing a book – whether a “tech” book or otherwise – I highly recommend you consider going through Bill.  Drop me a note if you’d like to hear more about my experiences here.

Chandoo:  “It’s 46 miles to Uniontown, we’ve got a full tank of gas, half a stack of Excel books, it’s overcast, and we’re wearing sunglasses.”

Rob:  “Hit it.”

Hitting the Road.  Holding a Mini-Summit.

Chandoo and I are jumping in the car this afternoon for a short journey down I-77 to a place MrExcel calls home.  Well, at least, that’s where Bill lives and works today.  He is, sadly, moving to Florida this summer.  (The second friend of mine to leave Ohio for Florida since I moved here – traitors! )

This is going to be fun, having the three of us in one place like this.  But I’m a little bummed that we couldn’t get a good picture of us sitting IN the car.  There was too much glare on the windshield.

Otherwise, we were prepared to re-enact this classic scene:

Me, Chandoo, and Bill in One Place?  Serious Stuff?  Maybe.

When a bunch of Excel “heavies” get together like this for a day, you might think the gathering looks like this:

Sometimes, that’s true.  But they just as often turn out more like the barbecue scene from Mystery Men:

I’m interested to see which way today goes

Very busy week for me.  Fortunately Miguel is at the ready with another guest post.

It occurs to me that we’ve now had guest posts from the US, the UK, Holland, Canada, and Panama.  That’s pretty cool. 

Miguel told me that I was going to LOVE the first picture in this post.  And I do.  But I must say that, once I saw the title, I expected something along the lines of the picture at the right.  The picture below is better.

OK, over to Miguel…

Figure shows the population distribution for Panama in 2010
Made Entirely in Excel!

If you read my latest guest post at Powerpivotpro’s blog, you’ll know that I’m working on a personal project trying to get a more visual aproach of the latest census Data for the country of Panama (where I’m from and I currently live in).

So far, I’ve managed to map more than 600 points on my map with their respective coordinates and I was able to use the heat map but then I found out that we couldn’t use the Choropleth map style (the ones that color the entire region) but instead we could play with the point’s radius and the intensity of the colors in the heatmap.

What’s GeoFlow anyway?

It’s a tool that complements what Power View hasn’t been able to bring to the table which is:

1. 3D Mapping
2. Heatmaps
3. Predefined Scenes and story-telling like scenarios (record cinematics)

which translates into enriching the current experience of the user by bringing a new way of interacting with your data (data exploration and more story-telling).

Moreover, what I find extremely useful is that if I wasn’t aware of the territory, I’d be able to go deeper into the map and really dig into the areas surrounding the most heated locations so I could see if there’s any correlation between the point being mapped and perhaps another destination.

Perhaps there are blank spots because these are ‘protected forest areas’?

The Chart types that you get are:

1. Bubbles
2. Columns
3. HeatMap

Click here to download the add-in if you haven’t done it yet.  (NOTE that GeoFlow is 2013 only!)

Interactivity and Sharepoint Integration? No? Okay…

One of the most ambitious efforts put into the latest tools of Excel 2013 like Power View and Powerpivot, it’s the interaction that the end-user has with the tools themselves. With Power View you get incredible dashboards like this one:

Where the end-user could slice and dice the data either on Excel 2013 or even on the web with the Sharepoint Integration but, on the other hand, there’s no corporate story behind GeoFlow as it’s only an end-user tool for now with Excel 2013.

What about the filtering inside GeoFlow? well, I haven’t been able to see this feature yet so in order to filter data you’ll need to do it from the source before using GeoFlow

Understanding how BING works to map your location points

Now, here’s where the tricky part comes. In a nutshell, your dataset is not likely to get a 100% accuracy when your points get mapped. Bing will take your data and try to find an exact match or an approximate match.

Why? Because it depends on how unique the name of the Place that you’re trying to map is. For example, I tried mapping a location within Panama by the name of Paris and BING ended up mapping Paris, France. On a second attempt I tried using a concatenation of that place and the name of my Country but we ran into a new issue.

Thankfully, GeoFlow gives us some Geocoding alerts in case there were some mismatch or close matches when you click in the link that has the % of results mapped:

This is the window that you get:

= there was a close match and it will be mapped using the location shown in the Result column

= there was no match at all and it will not be mapped

Fundamentally, in order to get better mapping results you’ll need to learn how BING works and how you can try to make things easier for BING to map your data. (if you have coordinates, use them)

This will be a completely new post on better practices when trying to use maps with Power View and GeoFlow!

Using Layers

Whenever you use GeoFlow, you’ll notice that you’re basically creating:

1. Scenes
2. Each one with 1 or more layers

and the layers can be mixed, let’s say that I have a heatmap and on top of it I want to have some 3D Columns based on something else.

Practical Case

Let’s say that I have the population distribution for Panama and I want to use it as my base-layer and then on top of it I’d like to analyze the sales of my product just to see if there’s any correlation on amount of population and the amount of sales. I could change the measure used from population to let’s say, amount of resellers or stores within a particular area and so on. You get the idea

More Population = more sales? or perhaps I’m selling more in an specific place for specific customers?

Power View or GeoFlow ? Why not blend them together as Power View 2.0 ? (a switch to show your map in 2D or 3D)

I’d love to have the 3D Mapping and heatmaps as part of Power View in order to provide an extremely immersive yet dynamic way of data discovery but it seems that the immersive experience in a 3D Map it’s what Microsoft is heading towards to and as a completely separated tool from Power View. But, wouldn’t be cool if Power View could support heatmaps and 3D Maps?

Either way, GeoFlow tries to give you the most immersive experience that you can possibly find to interact with your geographical data and it’s something that is worth trying.

Also, I’ll be posting a more in-depth analysis about the Panama Census data on my blog using GeoFlow and Power View in case you’re interested in that.

Guest Post From Miguel Escobar!

Today we’re blessed with another guest post from Miguel Escobar.  I love the style of this one, and the movie industry tie ins of course.  I also really like that Miguel “detected” the similarity between Hans Rosling’s TED talk and the Power View demos that first emerged in 2011 – the first time I saw Power View, my immediate thought was “Amir is riffing on Hans Rosling” and I wondered how many other people were thinking the same thing.

But above all I love to hear people’s stories.  How they came to “get involved” in this stuff.  I would say Miguel falls somewhere on the more advanced end of the spectrum – even using the term “Self-Service BI” kinda gives that away – but the fact is that the worlds of BI and Excel are indeed converging.  So let’s hear Miguel’s story shall we?

-Rob

What has been the impact of Self-Service BI in our modern world?

Its self-service BI a good thing? heck yeah it is. You can bet on that BUT…is that such a good thing that would drive engagement of the users into actually transforming data into information and information into insights? and even further…decisions based on those insights?

3 years ago I couldn’t even think of having a great reporting and visualization tool at my reach because I couldn’t afford that and college tuition at the same time and then Powerpivot became available…how did it impact me and some of my friends?

Some Background History

When I first saw Hans Rosling Ted Talk when I was still in high-school, I couldn’t stop asking myself how he actually managed to create such a great software that obliterates BI solutions that were available during that time. Few years later, with more knowledge about reports, data, statistics and software’s available in the market, gapminder.org released a version of their software publicly so it can be consumed by anyone with an internet access. Sadly, if you wanted to change the dataset you would be facing a tremendous challenge since the software wasn’t designed to import new datasets but to consume the one that was already loaded in it.

That was back in 2010, and back then there were no easy to use or accessible (Free) tools that could use any particular dataset and bring the interactivity that was shown during the Ted Talks of Hans Rosling…at least not until 2011.

By 2011, Powerpivot was out and the Tabular Model was something that was gaining audience and then… it came the Power View presentation by Amir Netz using the data from Boxofficemojo.com at the PASS Summit Conference. When I saw that presentation, it felt like he was selling that tool directly to me as I was at that time working in the Theatrical Film Distribution industry (specifically 20Th Century Fox) and it was probably one of the first times that I could see a relevant comparison against Hans Rosling software…but at that time, it was only available through Sharepoint 2010 using SQL Server 2012 Reporting Services from either a Powerpivot Model or Tabular.

It wasn’t until 2012 when we first had the chance to get our hands into the Excel 2013 Preview where now the xVelocity was integrated with Excel and not something solely in Powerpivot and also the fact that Power View had a client-side inside Excel 2013. Now we had the chance to get that Hans Rosling feel with any dataset that we wanted right from the most familiar tool of any office worker (and anyone with Excel 2013 installed).

Microsoft did not stop there….they went ahead and created things such as:

Geoflow (3D Map Visualization and more)

Data Explorer (Extract, Transform and Publish any data from anywhere)

We could say that the tool that Hans shows us wins but Power View gets pretty close to the same experience. How close you  may ask?

There are still some functionalities that are not available in Power View like logarithmic scales, the way to defragment the bubbles into the subgroups and more but, think about this tool as the starter of a series of enhancements that will sure hit the market soon. Similar to what happened with Powerpivot v1 and how it looks and runs now (Comparing Powerpivot from 2009 against the one in Excel 2013 its like night & day)

In Conclusion

“The seemingly impossible is possible”

Yes, what I thought that could only be available through thousands of dollars worth of investment in corporate BI  in order to get the Hans Rosling reporting experience..its now incorrect thanks to Excel 2013.

We as users, the ones that didn’t have access before to BI but that are engaged in this fascinating world of BI, are now taking a much more active role in the discovery of insights around the world. We can now take the data and be able to create meaningful insights and create visually stunning-TEDtalk like reports and share them. We could even go further as to using these new tools for our Countries and help to reach clear insights that could potentially reshape our world by starting from our counties or states.

Self-Service BI will prove to be one of the most meaningful progress in mankind since its taking the reporting process that regular excel geeks like myself know (Which is playing with VLOOKUP and Pivot tables) into a whole new level of richness that could only benefit ourselves in terms of knowledge and understanding in order to reach the most meaningful insight.

The whole reason behind this posts is to give my opinion and point of view on how things are going right now in terms of self-service BI, the impact of self-service BI in the current and future era and what our duty is as citizens of the world.

I’m starting my journey in using the knowledge that I have of Powerpivot and use it for the analysis of the census and other data of my hometown from

http://www.contraloria.gob.pa/inec/Redatam/index_censospma.htm

but I’m stuck with the R+SP software since I need to create the query from there in order to consume the data in Excel so if any of you know how to query that data let me know!

Image of Amir Netz was taken from: http://michaeljswart.com/tag/amir-netz/ (awesome blog btw)

My Blog:  http://thepoweruser.wordpress.com/

The Youtube Channel: http://www.youtube.com/user/PowerUserXL

Twitter: @migueldotxlsx

This Highly Scientific Chart Reflects My Experiences of the Past Four Years

So Many Things to Write About!

There are many things competing for my digital ink today.  Microsoft has released a workbook size optimizer and I owe someone named “Doody” a post on that.  I have more to add to the last two posts.  I have a “tournament rules versus prison rules” post that is dying to get onto the page, inspired by Jeremy Bartz.  My wife and I spent yesterday with Chandoo and his family.

But something in the news recently brought another long-simmering topic to the foreground…

“We Steal Millions From Our Customers, and They Have No Idea”

Pilot Flying J Allegedly Underpaid Fuel Rebates to Multiple Trucking Firms –
a Practice That Only One Such Firm Detected!

An absolutely fascinating story is taking place in Tennessee.  The FBI has raided the headquarters of Pilot Flying J (a huge chain of gas stations and truck stops across the US), and even seized the computers of top execs, in connection with a scam.

The alleged scam goes like this:

1. If you’re a trucking company and operate a fleet of 100 trucks, you buy a lot of fuel.
2. That makes you a very valuable customer for someone like Pilot – they want your business.
3. So Pilot offers you a rebate – if you buy X amount of fuel per month, they will refund you N percent at the end of the month.
4. But certain people at Pilot figure out that smaller trucking companies have NO IDEA how much they are owed in rebates each month.
5. So those people start shortchanging those smaller customers – sending them smaller rebates then they are owed – and pocketing the difference!
6. This practice was referred to as “managing the discount,” or more colorfully, “jacking the discount,” or my favorite, “f—ing the customer.”

One Customer DID Discover the Scam – From the ESPN Article (Click for the article)

3.5 Cents Per Gallon in Rebates Were Withheld???

OK, diesel fuel is selling for less than $4 a gallon these days, so to shortchange someone 3.5 cents in rebates is a full 1% of the full purchase price. 

Not 1% of the rebate.  1% of the purchase price.  Rebates were typically around 4% of the purchase price, from my reading of the affidavit.

So they were shaving about 25% off of the discounts owed!  Wow, that’s a lot of shaving!

How Did They Get Away With This?

Simple.  They just needed to know which of their customers were sophisticated enough to catch it.  And if you weren’t sophisticated, they screwed you:

From Conversations Taped by FBI Informants

So Much Lurking in the Dark!  So Much Opportunity!

The lesson of this story, for me, is NOT that there is fraud in the world.  I expect this sort of thing happens all over the place.  In some sense the folks at Pilot are just unlucky that someone went to the FBI.  I’m sure many companies run similar scams and are currently sweating whether this Pilot thing will unmask their own fraud.

The bigger lesson is just how many people are blind to their own data, and how expensive that is.  Being defrauded is just ONE way you can lose money.  Bad, uninformed decisions take many forms.

Recently Said to Me by a Top Exec at One of the World’s 25 Biggest Companies

I talk (and write) a lot about traditional BI vs. Excel, how the latter dwarfs the former, and how PowerPivot blends the strengths between the two.  But make no mistake – that tension is NOT the primary change going on in the world.  Nor does it even represent the bulk of the opportunity.

The primary opportunity, in terms of data, is “lighting up the darkness,” as Bob Marley would say.

We have a lot of darkness to light up.  Which is a good thing for people like us.

Oh, and God Hates Cleveland!

Funny side angle:  the only reason I am even aware of this story at Pilot is because the CEO of Pilot recently bought the Cleveland Browns, so the story surfaced in my daily reading of sports news.

Bill Simmons loves to joke about how “God Hates Cleveland” and this is a new angle to that story.

Promising new owner pledges to turn the Browns franchise around.  Everyone gets excited.  Oh wait, promising new owner may be going to jail?  (The FBI alleges he was well aware of the scam over the years).  Wouldn’t that be funny?

Thursday’s Post “Fixed” The Number of Negative Stores for a Month at 8.
Now We Vary That Threshold That With a Slicer.

Let’s take Thursday’s post and extend it a bit.

In the picture above you’ll see that I have 5 selected as my threshold on the new slicer, and 48 months “qualify” for that threshold – there are 48 months where at least 5 stores were negative.

Now let me select 9 on the threshold slicer:

Raising the Threshold to 9 Weeds Out 10 More Months, Only 38 Months Exhibited 9+ Negative Stores

How’d I Do This?

With one of my favorite “go to” techniques:  the disconnected slicer.

I created a table in Excel with the numbers 1-10 and then pasted that into PowerPivot:

The Stores Negative Table:  Pasted From Excel,
and NOT Related to Any Other Tables in my Workbook

Then I create a “harvester” measure off of it:

(By the way, this simple and crazy-powerful technique is explained from scratch in the book, and also used many times on the blog – search for “disconnected slicers” and you’ll find multiple posts.)

Then I modify my original [Negative Growth Stores Measure] by simply subtracting the new [min Negative Stores] measure from the original formula:

[Negative Growth Stores Measure] =
   COUNTX(VALUES(Stores[StoreID]),
          IF([SameStoreSales]<0,1,BLANK())
         ) - [Min Negative Stores]

This formula, then, will return 0 when I have exactly the number of negative stores that my threshold slicer specifies, and greater than zero when there are more negative stores than the threshold.

So now I just modify my Values Filter to be Greater than or Equal to 0 (rather than 8, as it was before):

Change the Values Filter to Be >= 0

And that’s it!

The Readout

I also added two “readout” formulas:

Here’s that second formula as text in case you’d like to copy it:

=”Months have at least ” & CUBEVALUE(“PowerPivot Data”,”[Measures].[Min Negative Stores]“,Slicer_Stores_Negative) & ” stores that went negative.”

Download the Workbook!

Click here to download this workbook

Do You Get This Error From Time to Time in Excel 2010?  I Do.  And it’s Quite Fixable.

This morning, when I loaded up my workbook from last Thursday to start today’s blog post, I got this error.

For awhile now I’ve been meaning to post that this is very “fixable.”

When I opened the workbook, I had a cell in the pivot selected:

To make the error go away:

1. Click OK.
2. Select a cell outside of the pivot
3. Now select a cell inside the pivot again
4. Usually the error goes away

Just click OK, Select a Cell Outside the Pivot, Then Select Back Inside the Pivot

It’s an exciting day at the Collie household.  The one and only Chandoo is on a plane with his family right now, and I’m picking them up at the airport tonight!

For a huge Excel nerd like me, the chance to hang out with this guy over the next couple months is a massive gift.

Multi-Month Skills Swap!

For a long time, I had unconsciously assumed that Excel was a very structured and rigid thing – that all “experts” used in in precisely the same ways.  (At Microsoft, we were exposed disproportionately to the Wall Street crowd, and those folks tended to be rather homogeneous in their discipline).

But when I moved away from Seattle, the first time I sat down with Mr. Excel for lunch here in Ohio, we were immediately “swapping” techniques.  Sentences like “oh wow, I never thought of doing it THAT way!” were common from both of us. 

In recent years I’ve come to expect that as the norm.  Yes, we all do some things similarly.  But we all bring something unique to the mix.  Seriously, everyone reading this has some special angle on things that hasn’t occurred to others.

So the fact that one of the “magicians” of the Excel world is going to be living down the street from me this summer is Fan.  Freaking.  Tastic.  I’m giddy.

A Few Masterclass Openings Remain

I think there are still a few slots open for Chandoo’s Excel classes in Chicago, Columbus, Cleveland, and Washington DC, and the PowerPivot course he and I are jointly teaching in Columbus.  Check out the registration page here for details.

“Real” Post Still Coming

I still want to follow through on last Thursday’s post, so stay tuned.