How crowdsourcing helps some –but not all research activities

Left Image: Here’s some sample data that is derived from NACTAS 2007 North American Social Technographic Online Survey.

Although I primarily rely on the proven Forrester Research methodology, when it makes sense, I tap into my network to discuss concepts and get examples. While only a part of methodology is evident to the public (what you see on this blog, or on twitter) this post will make it crystal clear what works and what doesn’t.

Forrester methodology
For all of my reports, I follow the research methodology that many other analysts have used before me for the past 25 years. This tested process may involve analyzing Forrester’s large technographic and consumer data banks, work with our data research team, conduct interviews, surveys, talk with clients (buyers), talk with vendors (sellers) discuss with fellow analysts, venture capitalists, and research online it’s a thorough way to see the market holistically.

A social computing analyst uses social technologies
As an analyst that covers Social Computing for the Interactive Marketer, it’s natural to use the very tools that I’m researching in fact, it was one of the factors in how I earned the role. Using the tools is a great way to learn what works and what doesn’t, a key step before I make recommendations to clients. Also, this blog serves as a supplamant to clients as well as myself, for example I’m known for building many “industry index” lists in the space, it helps me to keep track of who’s doing what, and since I screen each submission, the signal to noise ratio is higher than a wiki.

In tune with my market: the Interactive Marketer
At Forrester, we’re client centric, and as such, I serve the Interactive Marketer. Recently, we ran a survey of my blog readers, and found out that the largest group of my blog readers (14k subscribed) are interactive marketers or work at an interactive agency. Also having a twitter network of 10k followers is an amazing resource. Fortunately, this audience is a match to the role that I serve, making the following two uses cases possible:

When crowdsourcing makes sense
These tools make it easy to flesh out concepts and find examples but deeper research is always required, here’s some examples where it does work:

Use Case 1: Fleshing out concepts
I’ve found that asking questions from my blog and from my Twitter account are helpful for certain information gathering, but not for other needs. For example, I used a discussion on my blog and twitter from social media experts to vet a definition for the term “Online Community”, given this is the crowd of experts on this topic (and they are very willing to collaborate), this was an appropriate way to use this. I then scrubbed the answers, and was able to extrapolate the general key ideas (crowd sourcing is often rough on the edges) and come to agreement with my editor. In the endnotes of the report Online Community Best Practices we referenced the crowdsourcing effort. Please note in that report Online Community Best Practices, I interviewed 17 companies, and relied ton Technographic data, a standard procedure in report methodology less than 5% of the report was crowdsourced

Use Case 2: Finding case studies and examples
Also, Forrester reports are chockfull of examples of companies doing it right and wrong. Given my network is tuned into what’s happening with brands and social media, I can easily get an answer if I don’t know it. It’s a direct example of using a social computer (and since my coverage is on social computing) this is a good fit. Of course, finding the examples is just the start, often, I need to understand the case study, and that often involves a typical research phone interview.

For both of the above use cases. additional follow-ups and analysis are required.

Traditional methodologies are core
Asking my blog readers and twitter followers to provide broad responses that represent mainstream North America, Europe, or Asia isn’t effective. Fortunately, Forrester has a great deal of Technographic data (we’ve large data teams) at my disposal around user behaviors and consumer preferences I don’t need to crowd source crowd source to get this, and rely on this for my data collection. I’m currently focused on a Forrester Wave of the online community platforms, this particular methodology does not call for any social media use I’m using a tried and true methodology, learn more about it on the Forrester site.

Sometimes, I engage in discussion on twitter that appeals to me personally: movies, music, events, art, sports, and occasionally politics. Being the curious individual I am, I’ve been known (before Forrester) to ask questions of my network and get responses if’s a fascinating use of social media. Although some may not see the separation, it’s important to note that these discussions often have nothing to do with my role as an analyst.

Summary:
I primarily rely on Forrester’s data, process, and methodologies that have been tested and refined over the last 25 years. This includes input from our large survey data, interviews, client meetings, vendor briefings, and talking with fellow analysis. When it makes sense, I can rely on my large network of Interactive Marketers and Social Computing experts to flesh out concepts and find examples although additional analysis is almost always required.

As I discover new ways to use these tools, I’ll update this post, I’d love to hear your responses