Many modern competitive battles have been won by attracting developers – whether it’s Apple in the case of mobile platforms, Amazon in cloud, or Salesforce in CRM platforms.
Companies are now investing 10s of millions of dollars in attracting and engaging developers to build extensions, products and partnerships around their platform. As these developer platforms and programs are becoming commonplace, we wanted to investigate their best practices; what makes them tick.
Scott Apeland oversees developer relations at Intel and has recently launched a whole new developer program aimed at encouraging developers to build new products with their artificial intelligence and machine learning tools. We spoke to Scott about how he identified which developers to target and how he is tracking the effectiveness of their developer program.
Intel hosts one of the largest developer programs in the world, with over 20 million active developers visiting their platform and making use of their APIs in projects and products.
With a central focus on developers as customers, Intel has been able to introduce more commonly understood metrics to measure customer satisfaction. One of these metrics is the Net Promoter Score (NPS), which asks: “How likely is it that you would recommend Intel Developer Zone to a friend or colleague?”
Intel survey their developers every three months and aggregate the responses. Developers rate their likelihood to recommend out of 10 Intel conducts the NPS surveys via monthly emails, anonymous feedback, through their Innovators and Managers Programs, and at roadshows and workshops.
In the first half of 2016, Intel received over 2,500 responses and 2,600 comments on developers experience with Intel. Measuring their results by business unit and overall, Intel has recorded a jump from an average NPS score of 19.5 in 2015 to 27 in 2016. The Internet of Things and Robotics developer tools jumped the greatest in terms of improvements in developer satisfaction during this time, while other developer programs such as mobile app development and 3D cameras also increased levels of developer satisfaction.
VisionMobile: How do you first identify which developers need to be targeted in a developer program?
Scott Apeland: Because Intel’s developer program is one of the most diverse and broadest, it is growing quickly. We had 16 million developers in 2015, and now we have reached 20 million in the Intel Developer Zone. This means we have to be really good at understanding different types of developer needs.
So we profile the developers in each audience segment and then create developer personas. We also use VisionMobile’s developer segmentation model to understand the different motivations of each segment and that’s very helpful for us too.
For example, in our data center developer program, we are working with HPC (High Performance Computing) developers with advanced C++ skills. They are really proficient at optimizing software to get the most out of the hardware. These developers are trying to get the most performance they can through techniques such as parallelization and vectorization. We help them by providing have the tools, training and support at the right level for these advanced developers who have years of experience.
But also in the data center program, we have developers who are trying to take advantage of brand new tech like network function virtualization. These developers need a different set of tools, and have different partners in the ecosystem and skillsets to deal with. That’s just in the data center world.
In the laptop space, we have a game developer program where we are helping devs take advantage of the latest platforms and create new experiences with 3D graphics, virtual reality, and 3D video. This is a whole different type of developer. And in the IoT space, you are often dealing with a “maker” looking at what problems they can solve using a combination of h/w and software together in a new innovative way. They are more likely to be individuals innovating by themselves but value sharing their knowledge and expertise in a community.
VM: How do you align building a developer program with overall business goals?
SA: Each business unit comes to us with their objective. For example, in the data center business unit, they have an objective to accelerate the growth of artificial intelligence (AI) and machine learning technology. To achieve this we must have the developer community on board and partner with them on delivering open source frameworks and tools.
So we just rolled out our AI developer program on November 17. The program was front and center in Intel’s announcement and is focused on upstreaming tools, technology and training, making them available to all types of developers.
We’ve been working with the developer community for many years but each time we put together a new initiative we start by interviewing real developers and software experts both inside and outside of Intel. Internally we spoke with developers optimizing the AI frameworks and creating the software tools. Externally we spoke with the top developers in our black belt program. We have a close relationship with them and can discuss what’s important to them and how to make them successful. They provided us with valuable insight to refine the program and focus it squarely on real developer needs. At AI day we rolled out a new AI Developer Zone with frameworks, tools, tutorials, videos, sample code, a partnership with Coursera, a contest with Kaggle to solve early detection of cancer, and a brand new student developer program. Academia plays an important role in advancing AI, so we put a strong focus on that as well. All of those elements came together and culminated in our entrance to the market.
VM: How do you measure the success of your developer program?
SA: We use a number of metrics, including the number of developers using our technology and the impact their applications have in the market. But one of the most important success metrics is developer satisfaction because without that we can not maintain developer mindshare and support long term. To measure satisfaction we use the Net Promoter Score (NPS). It gives us a top level benchmark: would developers recommend us to their colleagues? It is an indicator that management can use to assess how we are doing in general.
In addition to this score, what is even more valuable is the comments we get back from the survey. We get thousands of comments back (2,656 last quarter) around how we can improve things and that’s where we hear about the need for more additional tutorials and code samples, for example, or how the forums are tough to use online. We get continued feedback so we can really make this work for the developer side.
What I do is I make the NPS a top level goal in our division. The goal is to improve our score year over year, and every quarter, my staff will review our NPS results. Everyone will read all the comments and then we get together and decide on common themes that are emerging. For example, we may notice the need for more tutorials in Chinese or that a particular developer tool is difficult to use, and we take that feedback and we form an action team. That team is empowered to dive in and address the issue. The team includes a dev evangelist, a content author, a community manager, a web experience expert and a geo-rep for global input. They go and look at each comment in greater detail and put together a plan to improve that area. Then the team reports back every couple of weeks on their progress. Since we’ve started this approach we’ve been able to move the needle on NPS with a jump from 1.6 two year ago when we started to 27 today – you don’t see that kind of jump in the NPS world. NPS has really helped us in driving a customer-centric culture in our organization..
For any business opening up APIs to external developers, it is possible to measure the number of visits to the developer portal page or the number of developers requesting an API key. But in the same way that ‘likes’ in social media are often just a vanity metric, developer portal visits and API signups tell you very little about whether a developer program is successful or not.
Intel have found that by using the Net Promoter Score, they can engage regularly with their developers as first-class customers and support them to become champions of Intel’s products. It is not just a satisfaction metric that Intel have introduced, it is a key customer engagement strategy. They use the NPS and associated feedback as a continuous improvement mechanism to identify what developer resources they can next create and how to respond to new developer needs. The proof is in the results: each year, more and more developers are comfortable with recommending Intel to their colleagues.
Do you have best practices you ‘d like to share from your developer program? Drop us a line here: hello@visionmobile.com.
Disclaimer: Intel is a customer of VisionMobile, but there was no financial motivation behind this article.