(I struggled mightily to decide what to write about this week.  In my frustration, I sat down to write about how difficult it is to choose a topic these days, and I subsequently realized, “hey, that IS the topic!”  Read on and see why.)

The “School of Fish” Problem

In our team meetings behind the scenes here at PowerPivotPro, I often like to talk about the “School of Fish Problem.”  In nature, it’s always fascinated me that prey fish concentrate themselves into huge, dense schools – as a defensive tactic.  This has always struck me as counterintuitive.  Doesn’t it seem like that would be more convenient for the predators, and therefore a bad strategy for the prey?

I mean seriously, if you were a shark, wouldn’t you PREFER your food to gather all together like this?  Just swim right through with your mouth open, right?

Paradoxically, the shark can go hungry when confronted with a school, whereas a lone fish is an easy meal.

Paradoxically, the shark can go hungry when confronted with a school,
whereas a lone fish is an easy meal.

Well, it turns out that this disrupts the shark’s ability to focus in on a single target.  Charge in blindly, and the school just “flows” around the shark.  Try to focus on a single fish, and the outline of said fish is constantly being blurred by the movement of the others around it.  It’s kinda like jamming radar with lots of false signals.

So yeah – paradoxically, the shark can go hungry when confronted with a school, whereas a lone fish is an easy meal.

What’s the Point? (Part One)

Two points really.  One, I suffer a bit from the “school of fish” problem when trying to decide what to write about.  Here’s my OneNote list of potential blog topics:

image

When You Have a “School” of Potential Topics,
It’s Hard to Choose One to Eat…  I Mean, Write About.

That my list overfloweth should not surprise us, however, because…

…Working in Data Today, There is “Too Much” Opportunity!

Opportunities Abound in Data

There’s a reason that TSOFP (The School of Fish Problem) comes up so much in our team conversations.  With today’s landscape, to say “we work in Data” is really no different than saying “we work in Business.”  Every organization, regardless of size, has data, and every organization is driven to improve.  Data, in turn, is at the heart of improvement.

Furthermore, now that robust tools are affordable (and learnable by normal human beings), the WHOLE WORLD IS NOW IN PLAY.

Ten years ago, professional data specialists, in a way, had it easy.  Since the tools and services were exorbitantly expensive, the universe of possible customers was small:

  1. You had to focus on large organizations only, because only they possessed the budgets required.
  2. Furthermore, even WITHIN those orgs, there were only 1-2 people with power to authorize the level of spending required, so you “lasered in” on those individuals specifically.  You didn’t waste your time with departments, you went to the top of the Enterprise itself.
  3. Finally, the client generally had to be local, given the amount of communication and sweet-talking required in order to get a client to spend such massive sums, and then the intensive amount of grinding communication require to implement the project.

 

Put all of that together, and it was simple – a couple of dozen human beings in a given region had the ability to green light “serious” data projects. So, as a data pro (or a data pro firm), you knew where to focus your attention.

But now?  That world has gone bye bye.  We have another saying at PowerPivotPro:  every week, we discover an opportunity that could be the basis for an entire new company.  There aren’t single fish out there, it’s all schools everywhere.

Every week at PowerPivotPro, we discover an opportunity that could be the basis for an entire new company.

What to DO with all that opportunity?

By far our favorite solution to the “too much opportunity” problem is to help others with the opportunities THEY have spotted.  The domain expertise, and even more importantly, the relationships required to succeed in a given industry are not something that we can replicate, nor do we really want to.  We’d rather be “data IQ on loan” and help jumpstart dozens of other successful efforts – whether that be a new initiative within an existing firm or a brand-new company.  I don’t see us becoming super-vertical – we are going to remain broadly horizontal.

That said, it’s still a daily challenge for us, even in the broad “horizontal” sense.  So many promising avenues – too many.  Should we be “all in” on riding the wave of Power BI marketing since that’s got such momentum already?  How about continuing to lead the charge on the Modern Excel / Power Pivot awareness front, since that audience outnumbers the BI world 100 to 1?  Accounting and finance, should we make that a focus?  Sometimes the answers are all yes.  File this under “good problems to have,” but it’s still a challenge.  We’re enjoying it.  Stay tuned.

Rob Collie

One of the founding engineers behind Power Pivot during his 14-year career at Microsoft, and creator of the world’s first cloud Power Pivot service, Rob is one of the foremost authorities on self-service business intelligence and next-generation spreadsheet technology.

This Post Has 13 Comments

  1. Put all of that together, and it was simple – a couple of dozen human beings in a given region had the ability to green light “serious” data projects. So, as a data pro (or a data pro firm), you knew where to focus your attention.

    But now? That world has gone bye bye. We have another saying at PowerPivotPro: every week, we discover an opportunity that could be the basis for an entire new company. There aren’t single fish out there, it’s all schools everywhere.

    By far our favorite solution to the “too much opportunity” problem is to help others with the opportunities THEY have spotted. The domain expertise, and even more importantly, the relationships required to succeed in a given industry are not something that we can replicate, nor do we really want to. We’d rather be “data IQ on loan” and help jumpstart dozens of other successful efforts – whether that be a new initiative within an existing firm or a brand-new company. I don’t see us becoming super-vertical – we are going to remain broadly horizontal.

    – Fantastic point!!

    Keep up the great work. Every day, there are more departments in more companies buying into the power that these tools bring!

  2. Reminds me of an experience.

    When I was a kid, my Mom took us to an Easter Egg Hunt. I remember looking out at, what seemed at the time, to be an ocean of Easter Eggs as far as the eye could see in every direction. This was going to be an amazing event. What was hard to understand as a 5 year old kid was that there were a lot of much older kids. This story ends in tears and and no eggs.

    Just Sayin

      1. I personally didn’t read Greg’s comment that way, even if that’s how he intended it. I read it merely as “don’t take it for granted, press the gas pedal urgently.” Which is good advice, and pretty close to our existing mindset.

        Things change quickly in other words. The previous BI “regime” got upended virtually overnight. When is the next big disruption?

        That said, the existing (new) landscape doesn’t seem to be populated with mature skillsets. As companies go, we flatter ourselves to think we are on the forefront of the self-service revolution, and yet even we learn something important and new every week. Internally we like to say we are halfway inventing and halfway discovering a new industry.

        So far, the “mature” skill sets are playing the old game, and/or using the new game as a sales funnel for the old game. Neither of which is a “gobble all the easter eggs” threat. Someone big and established would need to reinvent themselves to pose that particular kind of threat.

        Complacency has no place at this table, but I’d bet on “the unexpected” as the biggest egg-gobbling threat, as opposed to something we already know about.

  3. To be innovative is look where everyone has looked but see what no one else has seen…..yet. As they say a plan without execution is a delusion as well as an illusion.

  4. Someone has an hypothesis of whats going on with the last o’reillys report?

    “… the Microsoft BI stack tools (Excel, PowerPivot and PowerBI) contribute the least to a Data Scientist’s ability to increase their earnings potential. ”

    https://lnkd.in/ezuGSGj

    1. well… these are tools that are designed for the mass market, not for highly-skilled data scientists so it makes sense.

  5. Javier is right about the mass market. It would have been interesting to see the sample size and standard deviation for the salary numbers. Dollars to donuts that the tools like Jaspersoft have far fewer practitioners and are thus more spiky in their distribution. Everybody knows the market for Excel users, you have to guess for the others.

    Another good comparison would have been to see similar job titles and salaries without BI skills. Every starting analyst claims Excel (and probably pivot tables) as a core skill these days – how much does a particular BI skill increase your earnings from a set baseline? An analyst who has the time, opportunity, and motivation to learn Jaspersoft or Spotfire is probably starting from a higher baseline – access to those tools doesn’t come free from your dept IT the way Excel does. In other words, if Excel and Spotfire both boost your salary by $40K, which tool would you choose to learn? We just don’t know the marginal value from the report.

    I too would like to see Rob’s potential blog list. Depth vs. breadth… the topics probably reflect that disparity. As in the survey Javier noted I’d also be curious as to where Rob’s breakdown of business lies in terms of corporate areas: finance, operations, HR, etc… which dept is drinking the Power Kool-Aid the fastest?

  6. Accounting and Finance is a huge opportunity for P3. My board packet comes from power pivot.

    My goal, VBA & Power Pivot skills. I think these skills could lead to fractional CFO role.

Leave a Comment or Question