Research Methodology

Nakono has developed a robust research methodology that is used for all the markets we cover. Our approach involves 7 key phases:

  1. Define market
  2. Analyse key products, services and companies: 20-25 company profiles
  3. Analyse the status of the market around the globe: 15 country profiles
  4. Data validation and fact-checking
  5. Market analysis
  6. Develop market foreast model
  7. Conslusion

Our methodology is summarised in the following diagram:


Information Gathering

Especially at the beginning, but also on an on-going basis during the whole project we carry out an in-depth information gathering activity which collects the facts and data that comprise the foundation for the project.

We use a combination of six techniques to gather information about the markets we cover:

  • Highly-structured, desk-based web research.
  • Proprietary information retrieval tools.
  • Industry analyst relations programs.
  • Premium online research resources.
  • Data validation and fact-checking.
  • Primary data: executive interviews.

Highly Structured, Desk-based Web Research

We use advanced web search techniques to locate reliable data for use in our reports and to support the production of our own proprietary analysis and market forecasts. We have also developed our own software to monitor and cache news as it is generated by press release wires, companies and online news outlets. As of August 2012 our search corpus included over 7 million documents. Our analysts also use Twitter to receive the latest information from over 500 leading journalists and executives.

We use targeted research to source a wide range of relevant facts, data points and other information from specific types of public domain source that include, but are not limited to, the following:

  • Financial Data: Company Annual Reports and SEC Filings.
  • Company Publications: Presentations made by listed companies to investors (e.g. “Investor Day”).
  • Conference Calls: Conference calls set up by listed companies to discuss quarterly and annual results.
  • Industry Associations: These organisations regularly publish annual reports on behalf of their members as well as the results of occasional research projects. This category of resource includes over 1,500 industry associations.
  • Government Statistical Functions, Regulatory Bodies and NGOs: These organisations publish a large amount of data, some of which is in the form of dynamic content (e.g. online databases) that are not accessible via standard web searches.
  • Branded News Publications and Research Providers: We have developed a list of over 2,000 news trade publications, blogs, newspaper & magazine sites and analyst firms spanning the 24 markets that we cover in detail. We use language translation tools to allow us to track and search these sources in order to access the latest information.
  • Company Press Releases: We monitor press releases that are published to press release services (e.g. PRNewswire, BusinessWire plus around 10 others) or which we receive directly from the company itself. This category of resources encompasses over 33,000 companies.

Proprietary Information Retrieval Tools

In order to extract relevant information from the huge volume of content available via the web, and especially our own database, we have developed our own software that allows us to quickly extract relevant information from over 4 million tech documents. Our searchable data set is growing daily with the market at the rate of over 50,000 articles per day.

Industry Analyst Relations Programs

We obtain advance information through special programs that have been set up by many listed technology and media companies especially for industry analysts (e.g. Nokia Industry Analyst Relations Program).

Premium Online Research Resources

Additional data points are sourced from premium online databases and research resources to which we have access (e.g. Factiva, Hoovers, DialogPro).

Data Validation and Fact-checking

We sometimes need to clarify statements that have been made by a company, fact-check information that we find surprising or better understand data that is unclear or conflicts with other information we have found. We are especially careful to be clear on how others define markets. In these cases we will call the company concerned and ask them for comment.

In some cases we will call the company’s analyst relations department but often we will reach into the company itself by targeting key individuals who we think will be willing to speak with us.

Primary Data: Executive Interviews

We draw heavily on the results of a number of in-depth executive interviews with people working in the relevant sector.


Research Process

Define market & use cases

Many modern technology markets – like online video or mobile advertising – at first seem quite simple but they often turn out to be less so. For instance:

  • Should Internet television be included as part of online video? If we say that Internet television should not be included then what about the case where someone uses a connected TV set to view YouTube via the YouTube app?
  • Is the situation where someone uses their Smartphone to access an Internet website that carries normal display ads an example of mobile advertising? Or is this online advertising?
  • What about the situation where someone receives a song recommendation from a friend on a social networking site and then decides to purchase a subscription for the underlying music service? Should this sale be included as part of online music sales, social networking revenues or both? And what might be the difference isfthe music service was ad-supported? Or what would the difference be if the sale of the subscrtiption occurred on the social networking site, versus on the music subscription service site?

It is therefore important to be clear about how a market is defined at the outset, not only in order to be able to quickly answers questions such as these, but also to be clear about what is included in the forecast and what is not and to avoid double counting.

This part of the process will usually involve identifying a number of use cases that will paint a clear picture about the market from the user’s perspective.

Historic market data

Although no two markets are exactly the same, it is possible to use the rates at which representative historic technology markets have evolved to set an envelope for the development trajectory for a new market.

By looking at actual data for markets that are already established on a stable growth path (e.g. HD television) or markets that are already mature (e.g. mobile telephony) it is possible to gain an understanding about the upper and lower bounds on:

  • The rate of growth for the market in question in its “mid-term” absolute steady growth phase (defined in terms of units sold or customers added per month – not merely in terms of percentages).
  • The saturation level (in terms of unit sales or user penetration).
  • The time the market took to go from a given level penetration (e.g. 1%, 5%) to the saturation level (e.g. 90% of the total addressable market).
This provides a good basis for sanity-checking the detailed forecasts that will be prepared later. 

Clearly, if the analyst forecasts a rate of growth that is much higher than any other comparable technology market then the analyst will need a detailed rationale that can explain why this could happen.

Company and country profiles

We will analyse the market concerned (e.g. tablets, eBooks, Over the Top (OTT) Internet television) along two dimensions:

  • Company Profiles: We are looking to identify around 20 companies who are centrally involved in the market and which:
    • Operate internationally: We would not restrict ourselves to only analysing companies based in the United States for example. Instead, we would want to analyse companies that are based in other countries as well, especially the leading emerging countries such as BRIC (Brazil, Russia, India and China).
    • Operate at different points in the value chain: If we were analysing the mobile advertising market then we would not just look at ad platform providers. We would look at advertisers and publishers as well.
  • Country Profiles: We will then analyse the market concerned separately for between 15 and 24 of the world’s leading countries. Each country profile involves looking into the market in question, say Germany, in order to answer questions such as:
    • Who are the main players active in the market?
    • What are the leading products/services and their features?
    • Are there any unique regulatory, structural or competitive factors that might affect the rate of growth of the market or its ultimate size?
    • What is the current stage of the market defined in terms of penetration levels or unit shipments?
    • Does the market have any special requirements? For example, dual SIM is an important factor in the Indian mobile phone market while a mobile-based payment sales approach will be vital in some African markets.
    • Are there any infrastructure dependencies that will affect the rate or manner of development of the market? For instance, a broadband Internet connection is required to view Internet television while the eBooks market requires an installed base of eReading devices.

This 2-dimensional approach provides an excellent foundation for building a detailed understanding of the market.

Analyse the market

Having defined the market, identified a number of end-user use cases, completed the company and country profiles we will then have somewhere in the region of 150 pages of raw analysis.

At this stage we want to extract the key themes, identifying the key factors driving the market (e.g. the product might be free, there might be low entry barriers for service providers) and those inhibiting its growth (e.g. poor user interface, censorship issues).

This is the time when the real drivers of the market become apparent and we are looking to develop a narrative that allows the analyst to ‘tell the story’ of the market.

For each country we will identify the long-term saturation level for the product/service and also decide where the market is on the adoption curve, based on reviewing actual market data and comparing that with the equivalent point on a curve that applied for a market that already exists.

Develop market forecasting model

By the time the analyst team comes to building a spreadsheet model – after hundreds of person-hours of research – team members will have developed a sound understanding of the market.

The final stage is to transfer the understanding of the market that has already been developed into a spreadsheet-based market forecast model.

By definition, a market forecast is the result of making a large set of assumptions about the market which themselves cannot be made without having already developed a deep understanding of what makes the market tick.

The main focus at this stage is on defining the rate at which the market will develop which is in our analysis mainly due to the interplay between two sets of market forces, one set at work on the demand-side and the other on the supply-side.

We have defined a list of total of 22 market forces (some on the demand side with most on the supply side – technology markets are primarily driven by the supply side) and we then carry out an analysis of each one to set a variable in our market forecast model. Each variable is allocated a score from -5 to 5 where “-5” means that this force is “a strong inhibitor” and a “5” means this force is a “strong driver”.

For example:

  • Examples of supply-side forces: We will look at R&D investment to gain an understanding of the rate at which the feature sets are going to evolve and how the relative utility of the product will be improved. We will also look at the maturity of current products in the market and assess how far away they are  from something that will stimulate mass demand. Further, we look at the level of marketing and promotional activity that is being undertaken by the leading vendors and take a close look at pricing and therefore how affordable the product is. We will also look at how easy it is for users to find out about the product and buy it. Clearly, if the distribution infrastructure is still being built out then this will limit sales. Whereas if the product is already “everywhere” this might indicate that the rate of sales (e.g. number of products sold per day) will have already reached the maximum level possible.
     
  • Examples of demand-side forces: We will look at the underlying value proposition to assess its overall utility. Is this a novelty that will wear off? Or is this something that delivers lasting benefits – something that everyone will eventually need?. Another important aspect will be the extent to which adopters of the product can recommend it to peers, something that is very important when it comes to the adoption of social media.

The result of this exercise will be a model that projects the number of unit sales (or the rate of adoption for a free service) as well as the penetration level (e.g. the percentage of the total addressable market that has the product or service).

Benchmarking and sanity-checking

In the final stage, we engage on an iterative process where we benchmark and sanity-check the results. This nearly always results in some changes to input variables, which involves us going back and thinking again about the rationale that was used to set that variable.

We use a list of 31 metrics that we run through at the end to make sure that our numbers are sensible. This involves checking a number of key ratios and checking the implications of our model in terms of, for example, consumer expenditure, supplier gross margins, long-term rates of growth, percentage of GDP etc.

What we are working towards is a firm understanding of the whole market, an appreciation of the factors driving and inhibiting the market’s growth, a set of numbers that are sensible all rounded off with a rationale or “storyline” that makes sense.

Conclusion

The preparation of a good conclusion is the hardest part of the whole research process: the challenge is to sum up what might be 250 pages of research in a few pages.

The conclusion must capture the main findings and convey to the reader the essense of the market, for instance; what makes the market tick? how will its development will affect other markets? what are the main opportunities? which players are best positioned and which are under pressure?


Market Forecasting

Curve-fitting Techniques & Mathematical Models

Our experience is that curve-fitting techniques are often not applicable for the markets we cover.

We would therefore not advocate the use of techniques that use, for example, Fisher-Pry and Gompertz, which are based on calibrating standard functions that resemble an ‘S’ curve.

Our research interest is in forecasting the development of disruptive technology markets which are characterised by a lack of historic data, an unstable market environment, unproven value propositions and rapidly evolving product features. Calibrating a standard mathematical model in such cases is often little more than guesswork.

We would also say that curve-fitting forecasting techniques can be a lazy way of conducting market forecasts as the analyst can fall into the trap of believing that he does not need to do get stuck into the detail in order to understand what makes the market tick, naively thinking the can instead just punch a few numbers into an apparently sophisticated mathematical function which then does all the hard work.

Besides, a classic ‘S’ curve approach can be too simplistic: although all technology markets initially follow an ‘S’ curve, the market’s development might depart from the ‘S’ curve, especially in the long term after the ”saturation” level is attained when the market might:

  • Start growing again (e.g. where VoIP languished for many years without growing much at all but has recently entered a period of renewed growth which has been enabled by major technological improvements).
  • Enter a contraction phase which will result in a new saturation level being reached that is below the initial saturation level (e.g. cable and satellite Pay-TV which will lose some, but not all, subscribers to Over the Top (OTT) Internet television services).
  • Enter a period of terminal decline (e.g. the photographic film and music retail markets are both experiencing terminal decline).

Curve-fitting models like Gompertz and Fisher-Pry cannot handle market situations such as these.

Nakono’s Market Forecasting Approach

Nakono’s approach to forecasting disruptive technology markets is based on a six-step processs:

  • Define market & use cases: Cleary define what is included in the market and what is not. Look at the market through the eyes of the end user to identify diverse examples of how the associated products/services are actually use.
  • Review historic market data: We have developed a large data set spanning a wide range of tech markets which we use to identify relevant data and to frame our own proprietary analysis and forecasts.
  • Company and country profiles: Develop a deep understanding of the market.
  • Analyse market: Identify forces both driving and inhibiting the market’s growth. Decide on saturation levels and stage of the market on a country-by-country basis.
  • Develop market forecasting model: This is based on quantifying supply-side / demand-side drivers that define the rate at which the market will grow.
  • Benchmarking and sanity-checking: We then look at our numbers using a range of criteria to make sure that the forecasts are sensible.

Market Forecasting Accuracy

Nakono’s specific expertise involves analysing early-stage, disruptive technology markets. This is a very challenging field where many of the standard research techniques and methodologies that have been well-proven in other research fields (e.g. consumer goods) simply do not work.

For many of the markets that we analyse there is frequently no historic data, the consumer proposition is unproven, the industry structure is not stable and the product or service itself is subject to rapid, major change.

When we are preparing 5-year forecasts for such markets, which by definition involves assessing long-term saturation levels (which might not be reached for 15 or more years) then one should not be looking for precision.

Instead, our focus in on producing forecasts that are “directionally correct” and where the underlying analysis and rationale will stand the test of time. A market forecast is a lot more than a set of numbers. Equally important as the actual numbers is the associated analysis and insight which should paint a clear picture of how the market will evolve, and why. In the end the forecast is just the conclusion of the assumptions and rationale that the forecast is based on.

It is important to appreciate that a market forecast such as, for instance, “1,845.2 million subscribers in 2015”, does not mean that we have developed a forecasting methodology that can forecast subscriber numbers to an accuracy of +/- 0.005%,  even though that is what this particular forecast technically implies.

What it means is that we have a developed a spreadsheet model which might involve hundreds of separate assumptions, where one of the many outputs is this particular number – but the confidence level on this particular number might be +/- 20% or more.

If we are talking about a forecast for the number of PC users then the confidence level is going to be perhaps +/- 2%. But if we are talking about the installed base of 8k ultra-high definition TV sets in China in 2016 then the confidence level could be +/- 50%.

With such a large uncertainty level it is meaningless to attempt to use a statistical analysis (e.g. Monte Carlo simulation) where each of the model’s input variables are individually assigned a  profile for their value (e.g. normal distribution, uniform distribution etc). Such an approach would convey a level of accuracy that is completely out of whack with the overall uncertainty of the model’s underlying assumptions. For one thing, many emerging tech markets are frequently impacted by unforeseen events that can set the market off in unforseen direction; for instance a regulatory development, or the decision by one market in include a hefty tax on the sale of a given category of product.

It is a different matter if one is analysing a mature market such as, for example, soap powder, where:

  • The product’s value proposition is well-defined, proven and stable
  • The industry structure is stable
  • The product price is stable
  • The competitive landscape is stable
  • There is a lot of historic data
In such situations it is possible to use quite sophisticated statistical forecasting techniques to produce very accurate forecasts. 
 

But when analysing disruptive markets, we primarily focus on understanding of the factors that are driving the market and aim for the forecast numbers themselves to be “directionally correct”, which is what we mostly achieve.


Last Updated: 12 November 2015