1: The Potential Of Small-Scale Research

@NOTE: Get the examples I have in front of readers early on in the book :) Maybe, to make it easier to not-really-integrate these, they are shaded text box-style callouts, mini case studies of a sort?

This was merely the first example I could find of a particular genre that is well-represented on the Internet:

You see what’s going on here, right? @deisidiamonia is mocking those who confidently espouse opinions on Twitter without institutional backing for those opinions.

This is a nicely simplistic example of a more complex problem that is larger than just the incentives of social media:

1: Genuine experts really don’t want their expertise undermined by folks who are merely farming likes and shares on social media platforms.

2: It is actually possible to mis-use tools we don’t have the proper training to master, and that mis-use can lead to harm.

So @deisidiamonia is – in their own cartoonish way – acting in defense of the honor and social position of real expertise. This is a big part of the cultural backdrop we consider when we think about small-scale research.

If I buy a \(5000 vintage Martin guitar to play 3-chord songs on the weekend, nobody's really been harmed by my usage of this tool, it's just that the tool's potential is utterly wasted on my lack of ability to play it well. On the other hand, if my wife complains of abdominal pain and I try to remove her appendix with a \)5 X-acto knife, I’m mis-using a tool in a way that almost certainly will harm her.

But what if I create a survey instrument, field it and get a few hundred data points, and start advising clients on an important issue based on what I’ve learned from that survey? Am I mis-using that tool in a way that can lead to harm?

When we consider doing research, too many of us fear something like the tweet I included above being aimed at us. Either a well-trained master of the tool spots an error that was invisible to us and calls us out, a loudmouth social media user who is chasing likes and retweets takes a pot shot at us, or we cause actual harm to another despite our intentions to help. These fears are not entirely baseless, but they are based on a misapprehension of the world of research that I hope to correct in this guide.

I hope to illuminate a small corner of the much larger world of research: the small-scale research corner. The value of using small-scale research to help your clients make better decisions is high, and – if you’re careful with your usage of the tools – the risk of causing harm is low.

Data Is Powerful

Our culture worships data. And rightly so. Data combined with human ingenuity and sweat is a godlike tool that’s pulled us out of a nasty, brutish, and short existence into across-the-board increases in comfort, wealth, health, and technological & human potential. But cults form around deities, and so data is more than just a powerful tool.

Data can also be a way to justify taking a quick shortcut from an inner emotional sense of what’s true to an external posture of haughty certainty. We can go further and use data as a social cudgel to attack enemies. Or we can assemble enough data to feel that we walk about in priestly vestments, closer to the divine than the unwashed masses we look down on.

Data is powerful. But data is not an unalloyed good, nor is it always the best tool to guide decisions. Data can only be as good as the people who produce and consume it. But data can be an instrument for improving decision making and wellbeing, and an ability to produce and consume it should be accessible to us, not just large well-funded institutions and companies. For us to do that, we should start with understanding the broader landscape of research.

The 5,000-Foot View

We’ll roughly divide the world of research into 3 not-equally-sized sectors:

• Academic/Scientific Research

• Small-Scale Research

• Business Research

@TODO: illustrative sketch

Academic/Scientific research is what we are most familiar with. Anyone who cites numbers about COVID-19 death rates, case counts, transmissibility, and the like is making use of the output of the academic/scientific research world. If there’s one thing that outsiders might know about this world’s methods, it’s the idea of statistical validity. Most of us don’t really understand statistical validity, but we know it’s important, difficult to achieve, and tempting to cheat at, and if we don’t like what a given research product seems to say, the easiest way to discredit it is to find some flaw related to statistical validity.

Small-Scale Research is something you’ll come to understand via this guide. Small-Scale Research (SSR) can enable better decision-making within businesses using methods that untrained researchers can use without getting wacky results. SSR keeps the cost reasonable by keeping the scope very narrow and using methods that generate insight and contextual richness rather than definitive declarations about cause-effect or extremely precise measurements.

Business Research is a superset of SSR. There are 3 approaches to business research:

• Seek to understand cause-effect in the context of a business decision.

• Measure the under-measured in order to help manage risk.

• Use inexpensive research methods to earn visibility and trust through social signaling but without supporting (or being on the hook for) any specific decision.

The next chapter of this guide will much more fully explain academic/scientific and business research so we can clearly see where SSR fits in between these two much larger worlds.

Before we get there, it’s worth thinking about why we should invest in SSR.

SSR Forces A Literature Review

After you roughly define your SSR question, you will do a brief literature review. If this sounds intimidating or technical, it’s actually not. A SSR literature review is like Googling around for stuff, except using specialized search engines (more and more useful options are entering this market all the time). It would be unwise to start a SSR project without doing a literature review, because you don’t want or need to duplicate prior efforts. So we could say that SSR forces a literature review.

This is a very good thing. Through the lit review, we’ll get a crash course in the relevant prior art. If our SSR question is roughly aimed at understanding the value of branding, we’ll find that there have been serious academic inquiries into this question. [@TODO: link to a few scite and others with pre-populated queries for this] Perhaps this will cause us to refine, narrow, adjust, or abandon our SSR project. This is good!

In fact, if all that a SSR project did was motivate a few hours of literature review, most of us would dramatically reduce our ignorance about the prior contributions of academic/scientific research to our area of expertise. This may or may not change how we work with clients, but it can’t possibly hurt, and it’s likely that our expertise will be enriched.

SSR Forces Us To Seriously Consider Context

Once you commit to a SSR question, any latent anxiety that your mind contains will gather itself and start saying, “but what about this? What if this is connected somehow to the question I’m investigating?” This is good, because this forces your thinking outward from the SSR question to the context surrounding it.

If your question is “does better branding increase sales?”, that’s an excellent starting point question. Good! Whatever anxiety resides in your mind will quickly marshal its forces to ask: “what other stuff could increase sales? Or decrease sales even if the branding is helping? Or cause branding to increase sales in some situations but decrease sales in others? Or…. or…. or…?” What’s happening here is that you are trying to locate your SSR question within the larger context of anything and everything that could be connected to it. This is a VERY GOOD THING, because almost no effect anywhere in the world has a single sole cause. The context will always be relevant, and sometimes dominant in understanding the research question you are pursuing.

Eventually your investigation of the surrounding context needs to resolve into a refinement of your initial SSR question so that you can settle down into research, but this preceding “anxious phase” is good because it forces you to seriously consider context, and if there’s one thing that can make you a better consultant, it’s a better grasp of your client’s context.

@TODO: Good diagram opp of 2 timelines. 1) Anxious phase extends into infinity with sad face, anxious phase is time-limited and yields good byproducts and transitions into research implementation phase with smiley face.

SSR Can Create Intellectual Property

Intellectual property (IP) is your expertise packaged and made usable without your direct involvement. For us indie experts, IP is generally not something we invest much effort in protecting in a legal sense or worry about being stolen. Most clients would rather pay us to help apply our IP, and most competitors are too proud or incompetent to bother with stealing or borrowing it. Sure, there are exceptions, but spending money to protect our IP would be like buying meteorite strike insurance for a car.

SSR can create or enrich IP. “I want to create IP” is not the best motivation for investing in SSR. “I want to understand X better so I can help my clients make better decisions” is a much better motivation, but the SSR that fulfills your desire to more deeply understand X can end up being, or contributing to, valuable IP. (If the speculative nature of all of this puts you off, that may be a sign that your business or thinking is not in a place that could leverage SSR.)

SSR Can Contribute To Your Point Of View

Your point of view (POV) is like an “intellectual fingerprint” – a way you have of seeing things leading to an argument that is distinctive in the market. Every person has a fingerprint, and every person has a point of view. But clients do not find every POV interesting or relevant. Points of view with content that clients find useful, challenging, intriguing, or suggestive of a better path forward are the ones that are most interesting or relevant to them.

Your way of seeing the world is informed by where you stand. Do you stand firmly rooted in your own belief or experience, or do you stand more rooted in what data tells you? The content of your point of view will be influenced by the context of where you stand.

I periodically run a workshop that helps consultants clarify and sharpen their point of view. With very few exceptions, participants stand rooted in their own experience, but they want their POV to come more from data. Earlier I said our culture worships data. I wasn’t exaggerating, and this explains why most of us want our POV to have the power that data can confer.

SSR can enrich your POV with unique data that you have assembled and interpreted, which can combine powerfully with the output of other, complementary, research.

Ultimately SSR Helps Our Clients Make Better Decisions

This is really the bottom line here. It’s the ultimate reason to invest in SSR. Well-designed SSR can help our clients make better decisions, which – bit by bit – enhances the health of the market we serve, which creates more and better opportunity for us. If you’re seeing something like a spirit of service (or long-term investment) being the most powerful motivator for SSR, then you’re seeing this thing clearly.

SSR Calls For Strength And Humility

Data is powerful. It can be a powerful way to understand truth and reality. It is more often a fairy tale we tell ourselves about why we decided a certain way.

This is an argument both for getting more fluent at creating and using data, and an argument for humility around the whole idea of data’s value. Some suggested reading for you:

• “Alchemy” by Rory Sutherland is a fun, worthwhile read here.

• “How to Measure Anything” by Douglas Hubbard is a much less fun, but equally worthwhile counterbalancing read.

If you’re up for it, read these two books back to back. You’ll find yourself suspended in a sort of “intellectual hammock”, pulled in two opposing directions with respect to the value of data. This is the right place from which to think about this stuff.

Why Don’t We Do More Small-Scale Research?

Let me be clear: well-executed small-scale research is very rare. There are good reasons why.

We mis-aprehend research generally, and business research specifically. We hear the word “research” and tend to assume that means expensive, complex, technical, inaccessible stuff. Most of us don’t know about this little niche of accessible, useful methods that untrained but motivated people like us can use to create unique value, and so we hear the word “research” and quickly decide “that’s not for me”.

We lack formal training in SSR methods. In college, I did one small-scale research project involving surveys and SPSS as part of a senior Political Science thesis project. That was the extent of my schooling’s contribution to my understanding of SSR. Maaaybe if we have done UX or product validation work we’ve had some exposure to SSR-friendly research methods, but the majority of us lack even semi-formal training in SSR methods. I hope this guide helps, but this lack of training partially explains why SSR is rare.

We are intimidated by the idea of research. The tweet I referenced at the start of this chapter lives in our head, or at least the social threat it represent lives in our mind as a fear of “getting in over our heads”. And so many of us are intimidated by the idea of SSR, because we fixate on the research part (and our associated fears) and undervalue how the small-scale part can make SSR usable and valuable for us.

We see few examples of our peers doing SSR. This reinforces the notion that it’s difficult, complex, and risky. We have some examples of SSR used for earning visibility and trust – for marketing, but that’s just one of several ways research can be leveraged, and examples where it’s used for decision support are less visible to us and therefore more mysterious.

We practice an unlicensed profession, and so there’s little incentive for us to raise our game beyond what improvisation, gut feel, past experience, “best practices”, and a dash of confidence can achieve. Said more cynically, our clients are surprisingly tolerant of really mediocre consulting services, which reduces the incentive for us to level up the quality of our advice using data. Said more positively, often the status quo at a client is so bad that improvisation, gut feel, past experience, “best practices”, and a dash of confidence can create a miraculous amount of relative improvement!

All together, these factors cause us to under-utilize research. Again, many of these are good reasons to not invest in SSR. Doing competent but – let’s be honest – utterly ordinary work can be monetized in totally adequate ways. Building a small team, leveraging a bit of luck, and avoiding making any terrible decisions for 10 or 20 years can buy you two really nice houses, a few college educations for kids, a funded retirement account, and quite a few nice vacations and meals in restaurants. All without touching SSR with a ten foot pole. Not bad!

So after accounting for all the reasons to invest in SSR and considering all the reasons we don’t, I think your decision will come down to an emotion rather than an objective decision process. That emotion is dissatisfaction. The folks who are willing to invest in SSR tend to be dissatisfied with the status quo. They have a hunger to advance the state of the art. A hunger that, frankly, I have been unable to fully explain. I have this hunger. Some folks I know who “should” be earning more money have this hunger and invest in SSR anyway, despite the “illogical” nature of the investment. And I know others who are earning way more money than they need to live well, can’t explain how exactly the SSR will contribute more revenue, and also have this hunger.

The best I’ve got for an explanation is this: it’s a hunger to understand more deeply combined with a concern that our own personal experience to date is limiting our potential to understand. We just have to gain this deeper understanding. Maybe this is really driven by even deeper, more primal motivations for status, power, etc. I don’t really know.

I do know that you can stop reading this guide if you’re sure you don’t have this hunger. There are easier, less risky ways to optimize your business to make more money and serve your clients better.

But if you do have this hunger, or if doing SSR is part of your job, or if you’re merely curious, then read on. I won’t waste your time with anything other than the essential concepts, details, and examples you need to understand and execute small-scale research.