Research is always challenging, I’m required to stay objective, follow a consistent methodology, and inform the market who is strong and who’s not. Believe me, I’ve gotten my share of angry calls, emails, blogposts, comments, and conversations with brands and vendors as a result of my reports. I’ve found that one thing that really helps everyone to understand is to be open and transparent about the process.
In every Wave, like the Community Platform Wave I recently published, there are vendors that are elated and those that are disappointed, this is a normal outcome of reports that make the tough decisions to help brands make decisions. I’m empathetic to some of the vendors who were not in the Wave and want to make some clear explinations as they have to answer their colleagues, customers, and investors.
To start with, in a market of now 100 vendors, it’s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog (free after registration). We used this information to filter out which vendors would be appropriate for the Wave report. I made a few public calls on my blog, twitter, indicating my intent as a result over 50 vendors submitted. We generally do not follow up unless we’ve heard client demand from inquiry calls, and I turned to 3 analyst peers for their inclusion of any additional vendors.
You can understand the challenges in filtering a large market to just a few vendors, given the amount of hours required to spend on each vendor, Forrester limits the number of candidates involved in the Wave, for this one, nine was a manageable size and we capped it at that number.
Next, we filtered vendors to meet the needs of external communities used for customers (I serve the Interactive Marketer) as well as vendors were primarily serving enterprise class companies.
Some vendors chose not to participate as being involved in a research report takes time of the vendors and perhaps precious time away from clients. Vendors have to participate in meetings, plan travel to meet me in lab, and be involved in several communications throughout the process. In fact, a multi billion dollar company who owns a community platform chose not to participate despite my offer.
I’m required to be objective, without this, I lose credibility and then everything else will fall apart. Several vendors who are in the community space who are Forrester clients were not included in the Wave, in fact, some were livid and gave me an loud earful and I actually suspect they may leave us as a client. Despite this, I’m supported by my management, and our CEO to stay objective.
Now for the good news, while there is no immediate plans, Forrester is known for updating Wave’s in the space after a few years. Secondly, if you’re in this space, you know I publish a weekly digest giving updates to buyers about what’s happening in the market. I know for a fact the digest is read by senior decision makers at large corporations. If you are a vendor in this space, you can schedule a briefing with me and we can discuss your situation, I speak to clients, press, and media frequently and will always match market needs with vendors as deemed appropriate.
I believe in the open conversation, and will support the right for every vendor and customer to voice an opinion. I hope this helps provide insight to why some vendors did and did not make the Wave, we tried every effort to be fair in our first report in a very large market.
.
Jeremiah,
I applaud you for your integrity. I worked for Meta Group awhile back before they were acquired by Gartner and we took the same stance of objectivity in our research.
The calls you get from irate vendors in the market indicate a failure on their part to properly communicate their value.
I’ve always found it fascinating that companies will call up demanding to know why they were not included or why they were placed at a certain point in the report or study.
The questions they should be asking is of themselves – What value does our product have to the market and how can we effectively communicate that to everyone on a broad a stage as possible.
Again – I admire your integrity and your objectivity. To the vendors – quit your cryin’ and start communicating way more effectively.
Hi Jeremy,
I don’t quite understand how you can give a balanced report if you exclude your own clients (I do understand that you are a stakeholder, but you are supposed to evaluate the BEST vendors, and you show a bias against your own). Earlier in the article you also mention that in a market of now 100 vendors, it™s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog. Does that mean you are biased towards those people that don’t hear your call, don’t follow you or don’t find your methodology the most ideal as it takes time away from their clients? It would appear that in essence it does.
In the end, in order to evaluate a product, you analyze the most prevalent needs of the market space, segment them into categories of needs, and test products in the marketplace in relation to those needs, then rank them per category. As long as you are transparent which clients are on your book, I have no problem with them being included in a study (in fact, they are right to be angry because they suffer an obvious disadvantage from the relationship).
Sorry, I misunderstood the following paragraph: Several vendors who are in the community space who are Forrester clients were not included in the Wave, in fact, some were livid and gave me an loud earful and I actually suspect they may leave us as a client.
I guess the confusing came from the understanding of the word ‘Wave’. I now understand that ‘Wave’ is an end result.
The other paragraph:
‘To start with, in a market of now 100 vendors, it™s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog (free after registration). We used this information to filter out which vendors would be appropriate for the Wave report. I made a few public calls on my blog, twitter, indicating my intent as a result over 50 vendors submitted.’
lead me to assume that vendors needed to actively submit to the Vendor Product Catalog before they get included.
I am not an analyst and have only been following your blog for a short while. I will have to get used to some of the terminology and read the blog posts you reference in each post to fully understand the article in question.
Hi Jeremy,
I don't quite understand how you can give a balanced report if you exclude your own clients (I do understand that you are a stakeholder, but you are supposed to evaluate the BEST vendors, and you show a bias against your own). Earlier in the article you also mention that in a market of now 100 vendors, it™s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog. Does that mean you are biased towards those people that don't hear your call, don't follow you or don't find your methodology the most ideal as it takes time away from their clients? It would appear that in essence it does.
In the end, in order to evaluate a product, you analyze the most prevalent needs of the market space, segment them into categories of needs, and test products in the marketplace in relation to those needs, then rank them per category. As long as you are transparent which clients are on your book, I have no problem with them being included in a study (in fact, they are right to be angry because they suffer an obvious disadvantage from the relationship).
sports shoes
Nike Sport Shoes
Women's Nike Sport Shoes
Men's Nike Sport Shoes