Methodology
This is a short guide to our methodology, showing
you why industry leaders trust our data.
Leading Tech companies trust us -and you should too- not only because we have been researching competitive markets for 20+ years , but because we have been producing accurate insights based on our findings and data.
Let us walk you through our methodology: how we design our research and surveys, and how we collect, cleanse and analyse the data.
Ask the right questions to the right audience.
The right way.
First things first: we focus on asking the right questions. At SlashData, we may have data in our DNA, but we know that plunging headfirst into data is not where your quest for answers should begin.
Instead, we want to support you in your thought process and to ensure that you are asking the right questions for your business problem at hand. Knowing exactly which questions you need answered will help us specify not only what data you need, but also who we should target, and how big your sample should be.
We then translate your business questions into research questions and a survey. Your quest for answers is turning into a research plan.
Finding the Source
Most of our research at SlashData is tailored to the needs of our customers: we design an outreach plan and target the specific audience required in their research. Our expertise allows us to provide you with the market research you need and assist you in numerous ways: whether that be non-developers using ecommerce tools,users of project management tools or software developers. We are able to target any audience in the technology space.
We partner with multiple panel providers for our custom projects, thus eliminating bias by design and we can support outreach to multiple countries, including difficult-to-reach countries, e.g. China. Our surveys are offered in multiple languages. We manage the translations.
For projects focused on software developer audiences, we can use our own panel as well as the Developer Nation community, which encapsulates many developers who use SlashData resources: forum users, newsletter recipients, podcast listeners, etc.
Our Developer Nation surveys, run twice per year and bring a minimum sample of 12,500 and are helpful as they provide rich profiling data (100+ questions) on software developers as well as larger samples. We also offer the option to add custom questions to these surveys, to address specific client needs.
The surveyed developers are not the same in each wave, instead, we reach out to the global developer population afresh each time.
This way, we ensure that we accurately represent the mosaic of all different communities, backgrounds, goals and software projects out there. Depending on project needs, we can also provide a premium sample, which means that each respondent is verified through GitHub and LinkedIn.
Sophisticated data capture
Our proprietary survey tool is unsurpassed in the research space, generating more data than any other survey tool. Its “question-by-question” view means that we capture exactly the timings associated with each question, rather than the survey as a whole, while capturing key information like click positions (in randomised questions), the extent of the viewport examined in each question, as well as the setting and unsetting of options.
This respondent data allows SlashData to create a best-in-industry data cleansing methodology.
Cleansing the data
Our cleansing methodology is both extensive and thorough to ensure only high-quality data.
We include sophisticated implementations of speeding, straightlining as well as other industry standard checks, but we far surpass this with our many advanced features.
Our implementations lead the industry standard. For instance, when considering the speed at which questions are answered, we do so on a grouped level: taking into account the language (because certain languages are faster to read than others) as well as other respondent characteristics like experience (if appropriate). Grouping in this way ensures more accurate speeding checks. Further, we also check a respondent’s speed against the minimum read time for that given question. Due to our unique question-by-question timing, we are able to carefully monitor speeding (and pick out responses that are not usable). We also compare a respondent’s answer time against the minimum read time for that question so as to check if the respondent spent enough time to properly understand the question.
As part of our suite of more advanced checks, we have advanced bot detection, including VPN detection, repeated VPN cycling or other unusual network features. Such advanced techniques protect your data against bots and/or click-farm activities.
Around most questions we build extensive consistency checks.
An example of this would be if you claim to use React (a front-end JavaScript based framework), but also say that you cannot code nor know any JavaScript. By checking for these logical inconsistencies we are able to detect boredom, carelessness or deception and adjust accordingly in each case.
Weighting the data
No matter how hard we try, we’re always bound to get some sample bias. Beware of anyone who says they don’t. In certain projects, we deem it appropriate to weight the data. This is done by creating respondent weights using the ranking technique, where we weigh data to adjust for and minimise sample bias that occurred during the fielding/outreach phase of a project.
This may occur, for instance, if a particular region (e.g. South Asia) brought a disproportionate proportion of the sample. In such a case, we will reduce the weight of South Asian respondents. In practice, though, we consider many attributes (not just region) and weight in a multivariate way.
Advanced statistical reporting
We have advanced statistical reporting available.
This includes:
Hypothesis testing - a very useful feature to see if there is a real vs only random fluctuation between two groups
Confidence intervals - which would give high and low estimates of usage.
Data you can trust
From collection to analysis and presentation, we ensure that our data and insights provide the most possibly accurate representation of the audience we are surveying and the market as a whole.
This way, you may make decisions and build a strategy that drives innovation in Tech and in your product. With confidence.