• Increase font size
  • Default font size
  • Decrease font size

HAL2

Member Area
Technology Impact Model

Quantifies the value of public support for R&D and technology commercialization activities. It has been used, for example, to measure the impact of the Canadian Space Station Program and the RADARSAT program.

Model Overview

Economic returns from technology result from its commercialization and application, rather than from its development. The immediate return from a million dollars worth of technology development is essentially the same as from a million dollars worth of road building (or any other activity). However, technology development creates greater returns than other activities in the long-term because of its commercialization and application potential. When assessing the potential payoff from technology development, it is these downstream benefits that must be identified and estimated. The HAL Technology Impact Model is concerned with the quantification of these downstream benefits from the public support of R&D and the development of technology.

Sponsored Activity, Spin-off Activity, and Diffused Activity

In the HAL Technology Impact Model, we define public support of R&D and technology development as Sponsored Activity. Such Sponsored Activity leads to Spin-Off Activity within the firms or organizations under contract to the sponsoring agency. This Spin-Off Activity results from the expertise and capabilities attained through their involvement in the Sponsored Activity. Possession of a skilled and experienced labour force, production facilities, technological and project management expertise, and so on, will make a significant contribution to the competitiveness of the industrial team in both Canadian and world markets.

Spin-Offs are considered to occur only within a firm that has performed Sponsored Activity. Technology that is developed during this activity, and related technology that is developed by research laboratories, will be diffused throughout the industrial sectors in Canada. Thus, Diffused Technology Activity results in technology-producing firms other than firms who have undertaken Sponsored Activity.

The HAL Technology Impact Model incorporates the following features:

  • Stages of development of the R&D process from basic research through market development to commercialization;
  • Assessments of increased R&D activity induced by the sponsored activity (the budget enhancement effect);
  • Assessments of incrementality whereby what did happen is compared to what would have happened in the absence of the sponsored activity;
  • Assessments of attribution whereby other contributions to the success of R&D are explicitly recognized so that the sponsoring program is not attributed all the benefits;
  • Spin-off activity (implementation of R&D results) in organizations that engage in sponsored R&D activity;
  • Diffusion of technology, innovations and information that extends the impacts of the results of R&D beyond the program participating organizations; and
  • Time response of implementation or commercial sales from different stages of R&D and engineering activity.

These features and other aspects of a methodology for assessing economic impacts of sponsored R&D are presented below.

Inputs to the HAL Technology Impact Model include:

  • Expenditures: Total expenditures of the activity that generate spin-off benefits. This includes both public and private sector investments.
  • Type of activity: The nature of the activity funded. Speculative activity with a large research component will have larger spin-offs than a contract for a deliverable (eg. hardware) which requires little or no new technology development.
  • Incrementality and attribution: Incrementality refers to the difference in impacts and effects in Canada between what would have happened without the sponsored activity and what did (or will) happen with the activity. An attribution factor determines what portion of the total economic activity is justifiably attributable to the sponsored activity (ie. accounts for the other contributing factors). Only using the incremental and attributable activity to estimate the economic impacts of the sponsored activity results in lower, but more realistic and credible, impact estimates than in many other approaches to economic analysis.
  • Other model parameters: Include import content, number and size of firms in the sector, and sponsorship.

The Model does not account for:

  • Consumer surplus benefits;
  • Benefits to users of new technologies (spin-offs);
  • Induced economic impacts; or
  • Qualitative benefits.

Consumer surplus benefits result from valuing goods and services (usually those resulting from the investment under review) on a 'value to the user' or 'willingness-to-pay' basis rather than the market price. These benefits have not been included for reasons of lack of data and information.

Benefits from the use and application of spin-off technologies in Canada, other than for the firms that produce and sell those technologies, are very difficult to estimate. The benefits that accrue are often in the form of improved product quality or lower cost production in the field of application. Such improvements improve the competitiveness of Canadian firms and result in economic development benefits. It is recognized, however, that many such new technologies would likely be developed even without public support within some timeframe and in some country. The only benefits that can legitimately be counted here are those resulting from the advanced timeliness of such developments for Canadian firms because of the development in Canada of the spin-off technologies.

Including induced economic impacts will increase the overall economic benefits from the sponsoring program but will not affect the relative assessment when comparing programs. Induced economic impacts can be calculated separately using tools such as Statistics Canada's Input/Output model of the Canadian economy.

Qualitative benefits should be taken into account separately.

Innovation Adoption Process

From past work, HAL has developed a description of the innovation development process from basic research to commercialization. We apply this description to define the stage of development of activity associated with Spin-Offs and Diffused Technology. This stage of development is then used to define the probability of achieving commercial success for a Spin-Off or Diffused Technology Activity, and to determine 'downstream' or future expected costs of commercialization. This description is also applied to define the delay and time response of expected activity commercial resulting from R&D activity.

The innovation development process can be characterized as comprising four stages of activity (Research, Development, Engineering, Market) with three major decision points (a Go/No Go decision after each of the first three stages). This four-stage process is illustrated in the figure below.

As shown, the typical R&D and innovation development process begins with many 'ideas', only a few of which survive the inevitable assessments during investigation, research, and development. The typical process is characterized by 20 'projects' at the output of the research stage for every project that is fully successful through market development to commercialization.

The typical number of projects at each stage for each successful project can be interpreted in probabilistic terms to define probabilities of success. Using the average numbers of projects at each stage, the probability of a given idea reaching successful commercialization is one in a 100, or one percent. After successful completion of the research stage, the probability of commercial success is increased to one in 20, or five percent.

Assuming that a successful project would generate one million dollars of sales, the expected sales from any given project at any stage in the process would be the product of the cumulative probability and the sales of $1,000,000. From the research stage, the expected sales would be 0.01 times $1,000,000, or $10,000 per project.

Simplified Four-Stage Innovation Development Process Model

Costs of research, development, engineering and market development vary widely depending on the product, the innovation, the technology, the firm, the sector, etc. Surveys and analyses have been undertaken and aggregate average values exist in the literature . For the high-technology sector, typical values for expenditures by stage of development (as a percentage of revenue or sales) are ten percent for R&D, ten percent for Engineering and 15 percent for marketing. These typical values have been calculated to include average allocations for G&A and profit - that is, the typical cost estimates for G&A and for profit have been allocated proportionally to engineering, marketing and costs of production.

These average cost estimates have been applied in the four-stage innovation development process model shown in table below. The appropriate probabilities of success from each stage, the probabilities of a project surviving the decision process, and the cumulative probabilities of success from each stage through all future stages to successful market development are also illustrated.

As illustrated below, the ten percent cost allocation for R&D has been divided equally between the research stage and the development stage. The average cost per project in each stage, and the ratio of sales to this average project cost, are shown in the figure. This ratio of sales to the cost of the specific successful project at a given stage can be very large; i.e., the sales payoff for that one successful project can be as high as 2000 or 400 to one in the research stage. (The problem, of course, is to identify which project at these early stages will be successful.) These large ratios of sales to development costs are typical, and are sometimes quoted in a misleading way.

Four-Stage Process Model with Probabilities and Costs

 
Research
Development
Engineering
Market
Total Cost of Stage
$50,000
$50,000
$100,000
$150,000
 
In
Out
In
Out
In
Out
In
Out
Number of Projects
100
20
10
6
4
2.5
1.5
1
Cumulative Probability of Sales
0.01
0.05
0.1
0.167
0.25
0.4
0.667
1.0
Cost per Project
$500
$2,500
$5,000
$8,333
$25,000
$40,000
$100,000
$150,000
Sales Project Cost
2,000
400
200
120
40
25
10
6.7
Prob. x Sales Project Cost
20
20
20
20
10
10
6.7
6.7
Stage of Development Factor
2.9
3.3
4.0
6.7

A more meaningful ratio is the expected sales (sales multiplied by the probability of successfully achieving the sales) divided by the project cost at each stage. These ratios are also shown in the above table, and are the more typical 20 to one for the R&D stages, ten to one for the engineering stage, and 6.7 to one for market development.

A more useful ratio of sales per unit cost takes into account the costs of the "downstream" stages of development. This ratio is defined by apportioning expected sales between the current stage and all downstream stages in proportion to the expected costs of each stage. These apportioned expected sales are then divided by the current stage average project cost to yield a ratio of apportioned sales per unit project cost. Such a ratio recognizes that to achieve successful sales, there are future project costs to be incurred, and such future stages need to share the expected successful sales to justify their costs.

We define this ratio as the Standard Spin-Off Ratio. It is essentially a "Spin-Off" ratio - that is, a measure of the expected sales resulting from an R&D activity. The lower values at the earlier stages are consistent with the realization that government support is appropriate at these stages.

Generators and Receptors

Generators and Receptors in Spin-Offs and Diffusion

 

An illustration of the process of how sponsored activity leads to spin-offs, and how these two activities lead to diffused technology, is shown above. In this figure, the initial Generator is identified as the Sponsored Activity that is closely linked to Receptor 1 where spin-offs are produced. Receptor 1 is the collection of firms (and other organizations, if appropriate) that are involved in the Sponsored Activity. By our definition, Spin-Off Activity takes place only in these organizations.

Once Sponsored Activity and Spin-Off Activity is underway, the diffusion process transmits certain ideas, information, licences, etc. to a broader set of firms and organizations which are illustrated as Receptor 2 in the figure. These organizations are still producers of technology, but they are once removed from the organizations that are involved in the sponsored activity. This group (Receptor 2) receives stimuli from both the Sponsored Activity and the Spin-Off Activity.

The linkage between the (Sponsored Activity) Generator and Receptor 1 is very close ? there is usually a contractual linkage. The linkages between these activities and Receptor 2 is much more tenuous than that between the (Sponsored Activity) Generator and Receptor 1.

There is a set of organizations called Receptor 3 that represents the Users of New Technology in the Canadian Economy. It is within these firms that the new technologies will be put to use to reduce costs of production, to improve product quality or to develop new products. These impacts are analyzed separately.

Factors Affecting Spin-Offs and Diffusion

Estimating spin-offs and the extent of technology diffusion is not straightforward. Identifying the direct linkage between sponsored activity and spin-offs is often problematical, even for known spin-offs over historical time periods.

Factors that characterize the relationship between sponsored activity and spin-offs or the adoption of innovations include:

  • The capability of firms that undertake the sponsored activity to pursue spin-offs and innovations at the same time as performing the sponsored activity;
  • The capability and desire of private business to investigate and evaluate new technologies;
  • The number of firms or entrepreneurs that have access to a specific new technology;
  • The degree of linkage between firms that have the interest and desire and the firms that have direct access to the new technologies (usually via the sponsored activity); and
  • A long list of product and market development factors (time lags due to certification, registration, patenting, etc., competition from foreign sources, market conditions, business conditions, etc.).

Time Response Component

The Spin-off component of the HAL Technology Impact Model calculates the total spin-off and diffused sales resulting from the total sponsored expenditures. It makes no distinction about when either the expenditures or sales occur. The distribution of benefits over time, however, is important since a benefit is preferred sooner to later. The Time Response component of the HAL Model translates the stream of sponsored expenditures into a stream of spin-off and diffused sales so that later benefits can be appropriately discounted in value.

The essential feature of the Time Response component is the concept of a chain of stages of development activities that an innovation process passes through from the early stages of R&D to commercialization. The average time required to proceed from stage to stage, or from any stage to commercialization, has been studied and quantified. As well, the technical risk for the successful development of innovations to commercialization has been defined (see the section titled 'Innovation Adoption Process').

Distribution of Spin-off and Diffusion Sales

Spin-off sales are attributed to a particular year in the same proportion as the sponsored expenditures that occur in that year. For example, if 10% of sponsored expenditures are to be spent in 1999, then 10% of the spin-off sales are attributed to 1999. These sales are then distributed over time from that year forward.

When the sales will occur depends on the stage of development of the program. The HAL Model considers four stages of development: Research, Development, Engineering, and Market. Typically, a sponsored program will have technologies in a number of these stages. The time response of each stage is modelled separately and the proportion of the program at each stage is used as an input.

Sales can only occur once a technology is brought to market. Each stage of development will have a delay, during which there will be no sales, and then sales will occur over time. The delay is greatest at the research stage and least at the market stage. The delay for each stage is an input to the HAL Model.

Once sales begin, they are assumed to increase quickly, reach a maximum, and then decline slowly. This behaviour is modelled using a Weibull distribution. The shape of the distribution is defined by two parameters that are inputs to the HAL Model. They are determined empirically and are the same for all stages of development.

The total distribution over time of spin-off sales attributed to a particular year's sponsored expenditures is calculated as the sum of these four delayed and scaled distributions, one for each stage of development. The Time Response component of the HAL Model is illustrated below. In this example, the Base Year is considered to be year 0. Spin-off Sales of $1M are attributed to year 10.

The portfolio of projects responsible for these sales is distributed among the stages of development as follows: Research 0.3, Development 0.4, Engineering 0.2, Market 0.1. Sales from projects in the Market Stage begin immediately. Sales from Engineering Stage projects are delayed three years, Development Stage projects are delayed five years and Research projects are delayed 12 years. In each case, sales are distributed over time according to a Weibull distribution. The Total Sales over time are shown by the dotted line.

The distribution of diffused sales is calculated in a manner similar to the calculation of the spin-off sales. The difference is that Diffused Sales are considered to be driven by Spinoff Sales, which themselves have a complex distribution over time.

Spinoff Sales Distribution

 

Analysis of Uncertainty

An important aspect of any analysis is the requirement to specify the degree of confidence in the results. It is our experience from reviewing many studies that too seldom is the level of confidence or the range of results specified in the documentation.

The HAL approach not only specifies the level of confidence of the results, but incorporates the analysis of uncertainty of key factors. This approach comprises:

  • Identification of the key factors which are uncertain;
  • Quantification of this uncertainty using expert input; and
  • Combining these assessments of uncertainty within a rigorous framework of applied probability theory.

For more information on the HAL uncertainty analysis process, click here.