Broschuere Guide 7 Measuring the Impacts of Quality Infrastructure e

7/28/2019 Broschuere Guide 7 Measuring the Impacts of Quality Infrastructure e

1/44

MEASURING THE IMPACTS OFQUALITY INFRASTRUCTUREImpact Theory, Empirics and Study Design

Jorge Gonalves, Jan Peuckert

PhysikalischTechnischeBundesanstaltBraunschweig und Berlin i li


2/44

2

Imprint

Published by: Physikalisch-Technische Bundesanstalt

Bundesallee 100

38116 Braunschweig, Germany

Responsible: Dieter Schwohnke

Phone: +49 531 592-82 00Fax: +49 531 592-82 25

E-Mail: [email protected]

www.ptb.de/q5

Text: Jorge Gonalves

Jan Peuckert, Technische Universitt Berlin

Layout: Jenko Sternberg Design GmbH

(www.jenko-sternberg.de),

Physikalisch-Technische Bundesanstalt

Photos: Physikalisch-Technische Bundesanstalt,Hamburger Haen und Logistik AG

As o: April 2011


3/44

3

Jorge Gonalves holds a degree in economics rom the Universidade Nova de Lisboa (The New University oLisbon), as well as a Master o Science in International and Development Economics rom the Hochschule rTechnik und Wirtschat Berlin (University o Applied Sciences Berlin). Since March 2010 he is a research assistant atthe Chair o Innovation Economics at the Technische Universitt Berlin (Berlin Institute o Technology). He conductsresearch on the role o quality inrastructure or innovation systems and its implications or development policy.

Jan Peuckert has been employed at the Chair o Innovation Economics at the Technische Universitt Berlin

(Berlin Institute o Technology) as a research assistant since October 2006. He successully completed his study obusiness administration at the Humboldt University o Berlin and at the Instituto Superior de Cincias do Trabalhoe da Empresa (Higher Institute o Business and Labour Sciences) in Lisbon. The main ocus o his research wasparticularly the comparison o national innovation systems as well as the advancement o sustainability innovationsin developing and newly industrializing countries. Thereby, he has worked so ar on the development o compositeindicators or the international comparison o the general ramework conditions or innovation and environmentalregulatory regimes and the analysis o the international diusion o environmental management standards.

This study resulted rom a research project commissioned by the German Federal Ministry or EconomicCooperation and Development (BMZ). The views expressed in this publication do not necessarily reect those othe BMZ.

Bundesministeriumfr wirtschaftliche Zusammenarbeit

und Entwicklung

The authors


4/44

4

Content

Introduction 5

The unctioning o quality inrastructure 7

Empirics on the impacts o dierent quality inrastructure elements 22

Designing impact assessment studies on quality inrastructure 27

Conclusions 38

Reerences 39

Annex: Summary Matrix 42

A theory o impacts concerning quality inrastructure 13

Quality inrastructure as part o innovation systems 7

Standardization 22

1st Stage: constructing an impact theory 28

Positive impacts 13

Particularities o quality inrastructure 10

Metrology 25

2nd Stage: ormulating a method 29

Negative impacts 15

Complementarities 11

Conormity assessment 26

3rd Stage: interpreting results 34

The contributions o individual quality services 16

Public Intervention 11

Accreditation 26


5/44

5

Introduction

Quality inrastructure reers to all aspects o metrology, standardization, testing, quality management, certifcationand accreditation that have a bearing on conormity assessment. () This includes both public and privateinstitutions and the regulatory ramework within which they operate. (BMZ, 2004)

As it shall be argued, this institutional complex plays a vital role within national innovation systems, especially in acontext o rapid growth o global trade and value chains, and higher concerns with consumers and environmentalprotection. Underpinned by this understanding, a theory o impacts regarding quality services is depicted. Framing

quality inrastructure within the concept o innovation systems provides tools to better understand which kind oimpacts are to be expected rom the existence o quality services, and thereore to select the most appropriatemethodologies to assess them.

Accurately assessing the impacts o quality services contributes decisively to improve the understanding at thepolitical, entrepreneurial and academic levels about the relevance o quality inrastructures to the unctioningo innovation systems, how they enhance the perormance o economic agents and improve the outcomes osocial, economic and environmental policies. The key objectives to be achieved through the realization o impactassessment studies on quality inrastructure are:

the identifcation o the range o positive and negative eects that they have in societies;

to advise policy makers on how the accomplishment o their strategic goals may be acilitated through thedevelopment o quality services;

the raise o awareness o consumers, frms and industries about the existence o such an institutional complex,i.e. how it shall be used as a platorm where various economic agents are to participate and interact in order toaddress to a multitude o social, economic and environmental challenges;

to learn ways o improving the unctioning o the existing quality inrastructure.

Estimating the impacts o quality inrastructure services requires the adjustment o the general methodologiesapplied in impact studies to the particularities o quality systems. For the assessment o the impacts, the researcherhas to bear the strong complementarities between services in mind, as they pose an obstacle when trying to isolatethe impacts o individual quality services. The researcher has to question whether each impact is a consequence

o the unctioning o an individual service or rom the integration o various services. I the assessment studydemonstrates that a quality service is unsuccessul to bring out the expected impacts, an enquiry is to be made tounderstand whether the complementarities between services were properly considered.

The approach to impact studies on quality services outlined in this article states that they have to pass throughthree stages: establishment o an impact theory, verifcation o the theory, interpretation o the results. The impacttheory defnes the range o possible positive and negative impacts resulting rom the unctioning o a qualityservice or o a group o quality services, identifes the stakeholders and the impacts channels. To veriy the impacts,a methodology has to be selected rom a variety o possible approaches according to the inormation required,the inormation available, and the kind o impacts that are at stake. Interpreting the results requires situatingthe impacts in a context, in a strategy, listing the limitations o the study and proposing ways o optimizing theoutcomes.


6/44

6

The purpose o this report is to oer a better understanding about the kind o methodologies that researchersshould employ when assessing the impacts o quality services. We start by presenting the concept oquality inrastructure as constitutive o innovation systems, and as having certain characteristics (i.e. strongcomplementarities between the various services and the need or public intervention) that need to be addressedwhen developing impact studies. In the second chapter, a theory o impacts concerning quality inrastructureis delineated, the outcomes that its individual elements are expected to produce are summarized. The outlinedtheory o impacts is conronted with the existing impact assessment studies on quality services in the third

chapter. In the ourth chapter, based both on the presented theory o impacts and on the methods employed inexisting empirical studies, the methodological approach to impact assessment studies applied to quality services issystematized and a guideline proposed.


7/44

7

The unctioning o quality inrastructure

Quality inrastructure is a system o institutions which jointly ensure that products and processes meet predefnedspecifcations. Most notably, it aims at providing technical support to companies so they can improve theirproduction processes and at ensuring compliance with regulations or international requirements.

Quality, the congruence o the actual properties with the required characteristics, is the result o the integrationand coordination o a series o activities in several interrelated subjects, which are the main services o qualityinrastructure: standardization, metrology, conormity assessment, and accreditation. In the ollowing, the

importance o this institutional complex shall be discussed as a consequence to a context that demands a set onew capacities rom the innovation system.

Quality inrastructure as part o innovation systems

According to an evolutionary interpretation, innovation is the societal process o problem-solving by introducingsomething new (e.g. a procedure, a device, an organizational structure). Innovation capability is the competenceo social systems to fnd the adequate solutions or emerging challenges. Innovation system is a concept thatdescribes the network o actors and institutions that interact within this process o adaptation as a reaction to thereceived signals o misft. As defned by Lundvall et al (2009):

[A] national innovation system is an open, evolving and complex system that encompasses relationships withinand between organizations, institutions and socio-economic structures which determine the rate and direction oinnovation and competence-building emanating rom processes o science based and experience based learning.

The better the unctioning o innovation systems: the more eective the perormance o societies while creatingnew processes, products, and organizational mechanisms; the aster the response to social or environmentaltransormations; the more successul adaptation o productive systems to new strategic economic objectives.

In what concerns to economics, innovation is concretized when, or instance, production systems engage athigher-value activities, develop goods and management practices that respond to rising needs, better realize thebenefts associated with the participation in global markets and value chains.

A context o rapid growth o global trade, integration o production systems in global value chains, and o greater

social concerns regarding consumers and environmental protection is demanding a set o new competenciesrom innovation systems. Eectively coping with such pressing concerns requires the development o qualityinrastructures.

According to OECD (1999), 80% o the total global trade is already aected by standards and regulation. The costsassociated with technical regulations, standards, compliance certifcation and testing range rom 2 to 10% o totalproduction costs (idem, 1996).

The production system needs constantly to adapt processes, inputs, technologies and the design o productsto meet oreign / international standards and regulations in order to have access to external markets. Thisrequires the traceability o measurement units to a common reerence so that goods meet the characteristicsrequired by such standards and regulations can be developed.

Firms need to be confdent that their products comply with all the regulations imposed by the targetedmarkets as the costs and time associated with disputation may be prohibitive. The system has to ensure theadequate calibration o measurement instruments and a structure that evaluates whether the products conormto the norms.


8/44

8

I the utilization o national quality services is relatively expensive or i they are not internationally recognized,local frms lose international competitiveness when compared with countries with better unctioning qualityinrastructures.

Consumers are willing to pay a price premium or products and services addressing their concerns (e.g. airtrade, residues management, social protection o workers) or that oer new utilities. In order to reap thebenefts associated with producing innovative goods and services, frms need to be able to make their superior

characteristics visible to consumers. Credible certifcation plays such a role, hence increasing competitiveness olocal companies in global markets and creating incentives or more innovations.

Presently, intermediate goods account or 54% o the total manuactured imports (OECD, 2007). This is aconsequence o a more proound integration o economies and opens an opportunity or countries to join largerand ever more complex production systems.

Joining international value chains implies an eort to adapt the productive processes to: new measurementmethods and units, characteristics demanded by the buyer in the next stage o the production chain,characteristics o the goods that proceed rom the previous production stage. The integration in such acomplex network o conversions requires metrological progresses and integration o the national system omeasurement units in the global system.

The more complex the exchanged goods along the value chain, the more needed is the codifcation o theproduct characteristics. This makes the ormulation o contracts and the compliance to specifcations moredifcult and costly, hence raising a barrier to entering global chains. Integrating the national quality servicesin regional bodies acilitates the writing o contracts, the development o metrological tools ensures thecompliance with specifcations, and thereore promotes the participation o local companies in chains thatproduce high value added goods, the continuous technological upgrading and higher levels o productivity.

The increase in the number o partners involved in value chains implies higher coordination costs and higherlevels o trust, i.e. the outcomes o ones activities correspond to the expectation o the productive agent inthe next stage o the productive chain. The stability o the outcomes o the productive activities, resulting romaccurate measurements, improves the trust between the various frms, increasing the likelihood o emergingnew partnerships, products and processes.

R&D is more eective i it takes place at all stages o the value chain - i all the agents are developing betterprocesses and i these eorts are oriented in the same direction (e.g. adapted to consumers needs, availabilityo inputs, and so on). Guaranteeing the compatibility o methods, instruments, or properties o productsemployed in research activities promotes the cooperation and coordination between frms during theseprocesses. It acilitates the inclusion o the national innovation system in the global innovation system.

The raise o awareness about the need to protect consumers (especially in the felds o health and ood saety) andenvironment has been substantiated in an increasing number o regulations that set limits to the characteristicso the products that are traded, demanding rom the producers the development o products and processes thatrespond to such concerns.

In order to defne a norm concerning a certain property that is considered harmul to consumers or to theenvironment, instruments are required to measure it.


9/44

9

Guaranteeing the enorcement o the norms at a national level requires a system o institutions that evaluatesthe conormity o goods to the norms. I there are regional imbalances in the system to evaluate theconormity o goods, the consumers and environment rom the region with a less demanding system willprobably end up getting the goods that would not be accepted in the stricter areas.

I the compliance with a norm is expensive or technically difcult, companies may be orced out o the market.The system has to acilitate the compliance with the norm, saeguard competition, and protect small and

medium frms that are more likely o having fnancial constraints or the adaptation to the new rules.

The ollowing table summarizes contributions o quality inrastructure to innovation systems given such pressingdemands.

Table 1: Contributions o quality inrastructures to innovation systems

Rising concerns Importance o innovation system Contributions o quality inrastructure

Reaching global

markets

Capacity to constantly adapt tooreign requirements

Ensure compliance Guarantee competitiveness Production o goods with higher

value

Traceability o measurement unitsand procedures, resulting in the

comparability o the results.

Development and calibration omeasurement instruments

Cost-eective adaptation processes Making innovations visible to

consumers

Integrating global

value chains

Integration in more complex

production systems

New orms o integrating dierent

productive sectors

Increase number o partnerships

Integration in global innovation

systems

Harmonization o processes,

materials, measurement units and

instruments

Comparability allows writing o

contracts Decrease o uncertainty and leverage

trust between worldwide productive

agents

Harmonizing tools and methods used

in R&D

Protection o

consumers and

environment

Ability to measure certain properties

Guarantee quality o product

Discovering cost-eective procedures

Protect small and mediums

enterprises

Development o tools to measure

such properties

Setting technical regulations and

guaranteeing conormity

Avoidance o regional imbalances

Minimization o market power

creation


10/44

10

Innovation systems consist o a network o ormal and inormal entities where procedures and relations depend ona variety o actors, e.g. cultural values, historical practices, trust between economic agents, etc. which dier romsociety to society. Analogously, the architecture o a quality inrastructure depends on such cultural phenomenaand on the strategic goals they serve or, thus varying across countries.

There are, nevertheless, some general conditions that quality systems are to assure in order to optimize theircontributions to national innovation systems:

Confdence. Products and services that are traded conorm to the characteristics settled in contracts.

Reliability. In a value chain where dierent companies are producing at dierent stages, each enterprise hasto trust on its counterparts regarding, or instance, the measurements units and procedures that are used, thematerials or methods that are employed.

Comparability. The possibility to compare dierent products and services in terms o their properties acrosscountries or regions.

Traceability. An uninterrupted chain o comparison measurements with increasingly higher accuracyinstruments (smaller measurement uncertainty), starting at the instrument used in industry up to the national

measurement standard. (Sanetra and Marbn, 2007, p. 63)

Competence. The dierent institutions within the quality system are technically able to perorm the tasks thatare assigned to them.

Conormity. Products and processes meet the requirements that were established through a standard.

Transparency. Practices and procedures o the involved institutions are accessible and auditable by the civilsociety.

Impartiality. Institutions need to be protected rom political inuence and regulatory capture in the ulfllmento their technical contribution.

The impacts o quality systems go beyond the unctions above described. The direction o innovation dependson the context because innovation is not an end in itsel but a means o reaching a variety o goals. Thus, whennew processes are ound or new products are developed they are contributing to various spheres within thesocial, economical and environmental realms, solving concrete problems, helping to achieve specifc objectives.Consequently, looking or the impacts o a quality system implies fnding a broad range o eects that are situatedat various levels.

Particularities o quality inrastructure

In order to assess the impacts o quality inrastructure services one has to take the particularities o a qualitysystem into consideration and adapt the general methodologies employed in impact assessment studies to these

specifcities. There are two key characteristics o quality inrastructures that need to be addressed when designingan impact study in this feld. On the one side, the eective perormance o an individual quality service dependson the unctioning o the quality system as a whole. This poses a challenge when establishing the causal relationbetween intervention and impact: which service is responsible or each impact?


11/44

11

On the other side, quality services, according to their nature and the idiosyncrasies o each society, are perormednot only by private institutions, but also by public bodies. For this reason, the impacts have to be assessed takingin consideration the reasoning o public intervention, recognizing the various perspectives associated with thequality service and the medium-long run objectives associated with services as such, hereby creating an additionalchallenge: how to analyze the public interest o a service?

ComplementaritiesThe eectiveness o a quality service depends on the ways that the dierent services are interrelated, how theymutually support each other. To serve as an illustration, it is only possible to ormulate a standard regarding theproperties o a product, i the measurement units associated to those properties are defned and measurementinstruments exist and are calibrated. This standard is made eective when producers comply with it and this ismore likely i there are incentives to do so, or instance, when compliance is signalized to consumers throughcertifcation. The impacts o certifcation are more signifcant the greater the trust that consumers have about thecertifcation institute, which is higher i the competences o the institute are evaluated by an accreditation institute.

The high interdependence between quality services advice us to look at these activities as integrative o a system,not as independent activities that can be tackled individually irrespectively rom the unctioning o the whole. This

is an important consideration both when defning a strategy or a quality system and to measure the impacts oquality services (i.e. oten impacts have to be attributed to various services and not to only one intervention). Itrequires the raming o each service within the unctioning o the whole inrastructure (how does it support andis supported by other elements?) and the embedding o this inrastructure as a pillar o the national innovationsystem (which strategic role is it perorming?).

Public Intervention

The design o the strategy or quality systems and coordination o the various quality services require the directexecution o certain activities by public bodies:

necessary or the development o regulatory rameworks which typically can only be enorced by institutions

with legislative power;

as providers o services because certain activities (namely metrology) within this inrastructure are otenperceived as public goods, meaning that private agents when let alone will not provide the most desirabledesign o the quality service rom a welare perspective;

required in order to integrate the national quality inrastructure in the international quality system - a vitalstep or the reduction o technical barriers to trade, and hereby to improve the access o local companies toglobal markets and value chains. National metrology institutes may integrate regional metrology organizationswhich ensure the accuracy o measurements within the region and promote the regional use o nationalmeasurement and calibration capabilities (Sanetra and Marbn, 2007, p. 43). Analogously, accreditationinstitutes may join regional accreditation organizations or agreements.


12/44

12

However, depending on the capacities and limitations o the national production system and the strategicapproach to capacity building, the pace o the implementation o new norms and o the international integration isto be defned and will aect dierently each stakeholder.

For example, i the productive abric is not prepared yet to create or assimilate certain advanced technologiesrequired to conorm to an international agreement and i it is, nevertheless, enorced in a short period o time,small frms may be driven out o the markets, while larger frms are more capable to fnance their adaptation to

these new demands. Thereore, assessing the impacts requires the analysis o the how dierent stakeholders areaected by quality services.

Apart rom trying to analyze how dierent players win or lose rom certain interventions and how it matters orthe design o the services in order to optimize the public interest (minimize the social losses, maximize the socialbenefts), this has to be ramed given the medium-long run political and economic strategy associated with thequality service at stake. One has to enquire about the meaning o the analyzed intervention in terms o the uturewelare or interests o the overall population or certain economic sectors, which imply a urther orward-lookinganalysis.

Finding the appropriate methods to estimate the impacts o quality services requires the understanding o thesystemic role o each quality service, the public interest associated to the various services, and also o the kinds o

social, economical and environmental impacts that are expected as a consequence o the unctioning o qualityinrastructures.

In the ollowing chapter, a theory o impacts concerning quality inrastructures is outlined. It aims at providing asummary o the possible goals that can be reached through the means o a quality system.


13/44

13

Quality inrastructures may enhance the perormance o economic systems in various ways. Depending on theirdesign, their impacts range rom the stimulation o network eects and knowledge transer, to the increase ineconomic efciency and markets competition. But quality inrastructure may also have counteracting eects. In thischapter, we start by summarizing a spectrum o possible positive and negative social, economic and environmentalimpacts rom the unctioning o a quality system as a whole.

Positive impactsNetwork eects through enhancing interoperability

Quality inrastructures increase the value o goods with network externalities and promote the developmento networks as long as they allow products to work as a part o a larger number o productive systems. Byensuring compatibility between products, their integration in a broader number o value chains is acilitated,hence increasing the number o economic agents that derive benefts rom using them. The linkages between

frms and industries are multiplied, more inormation is exchanged and the cooperation in research increased.

Better management procedures

Quality inrastructures aim at improving not only production related procedures, but also managementpractices which may generate improvements in all aspects o a company: better quality, new schemes ointernal organization and o service delivery, and also acilitate the creation o new products and services.(European Commission, 2007, p. 7)

Knowledge transer

Through metrology, standardization or accreditation, a quality system is providing inormation about the stateo the art o a certain technology. As this inormation is reely available to everyone (both to market agents andnon-market agents such as research institutes), they are able to internalize it and use to generate new productsor processes.

Competition

Opening markets. The development o a quality inrastructure may reduce the dependency o businesses to asmall number o suppliers, hence opening the markets. By guaranteeing comparability, traceability, conormity,a quality system broadens the buying possibilities o companies, increases the competition among suppliersand the confdence o buyers on the quality and reliability o those suppliers that conorm to the norms (DIN,2000).

Reducing entry barriers. Quality inrastructures enhance transparency and help frms reaching new markets.According to European Commission (2007), companies are able, through the inormation contained instandards, to appropriate the preerences o consumers in oreign markets and the required technicalspecifcations. Furthermore, through certifcation and accreditation, local frms obtain more easily the trust

rom the oreign consumers as it guarantees that their products are in conormity to the specifed norms.

Economic efciency

Economies o scale (the cost reduction associated with a larger scale o production). A quality inrastructure,

A theory o impacts concerning quality inrastructure


14/44

14

namely through standardization processes, reduces variety in the sense that it sets limits to a certain number ocharacteristics o products. This enables lower per-unit costs or suppliers since they do not need to produce aspecifc good or each buyer. Instead, they produce a homogenous good that serves all their clients.

Economies o learning. The mentioned variety reduction also allows economies o learning, which is theincrease o efciency associated with the skills and experience gained by being ocused on products with ewertechnical variations.

Reduction o transaction costs. As long as the supplier complies with the standard, the product is certifed, andthere is trust regarding the certifcation, the consumer does not have to incur in costs associated, or instance,with double testing to check whether the product conorms to the requirements.

Reduction o search costs. The eort o fnding a product that has some specifc characteristics is substantiallyreduced given that there are standards that speciy those properties and compliance is assured throughcertifcation.

Reduction o adverse selection. A well unctioning quality inrastructure enables producers to dierentiatetheir products in terms o quality (the producers that comply and those that do not comply with certainnorms). Through the conormity assessment ramework, innovators are allowed to dierentiate their product/

service oer or to improve their internal processes (European Commission, 2007). That is a major incentiveas through this dierentiation innovators are able to charge a price premium (a higher price than theircompetitors) or their innovative products, hence compensating the investment made with higher revenues.

Reducing inormation asymmetry. The eective existence o a quality inrastructure may reduce the dierencesin the inormation that the various economic agents (consumers, producers, governments) possess regardingthe product that is traded. By doing so, allocation o resources is more accurate and market efciencyincreased.

Box 1: Asymmetric inormation

In Akerlos (1970) amous paper about ailure in the market or second hand cars (lemons), the cause

o the market ailure is in asymmetric inormation, and as Barzel (1982) argues, the problem in the lemoncase is that measurement is really rather expensive. I it were not, then buyers could cheaply measure anyo the product characteristics they want to, and they would have perect (or perectly adequate) knowledgeabout the quality o the product. It is the act that measurement is not cheap that means the buyer bearsthis risk rom asymmetric inormation. The growth o trade requires the reduction o transaction costs, andan essential part o that is the emergence o common standards and measurements. The comparabilityand traceability o measurements reduces some o the risks in trading, reduces transaction costs and hence

acilitates trade. Markets or innovative new products require a high degree o inormation and trust, andimprovements in the quality and comparability o measurement are important or such products. Theaccurate measurement o product characteristics makes it easier to demonstrate quality and saety, and henceto sustain a price premium or superior products (Swann, 2009, p. 65).


15/44

15

Consumers and environmental protection

Concerns o consumers and environmental protection are increasing and quality inrastructure is a means tosaeguard such interests. For example, i a certain chemical is regarded as harmul or the health o consumersor or the environment, metrology instruments can be developed to measure the levels that products containo that chemical; a technical regulation (a mandatory standard) can be established orbidding the existenceo this chemical beyond a certain level; conormity assessment guarantees that products eectively ollow the

norm.

Negative impacts

Constraints on technological innovation

Variety reduction. This may be perceived as a contradiction as it was mentioned as a positive impact o aquality inrastructure. However, namely in the process o standardization the positive eects on technicalchange are counterbalanced by their negative eects on product variety. By speciying product characteristics,such as orm, perormance, or interace, standards limit innovation. Obsolete, inappropriate standards mayhamper technical change by preventing the adoption o superior technology through the lock-in eect.

(Guash et al., 2007, p.24).

Box 2: Lock-in eects

Constraints on Competition

Potential barriers to trade. A quality inrastructure may create impediments to trade: i measurement procedures require complex practices and technologies which oreign companies have

difculties to internalize; i compliance with dierent national standards requires high adaptation costs or i testing is expensive; i the existing certifcation agencies are difcult to reach or corrupt; i the created standards are not well documented, difcult to fnd, poorly understood, or protected by

intellectual property rights.

Creation o market power. Quality inrastructures may have anticompetitive impacts when only one or a ewproducers are able to internalize the benefts or control the content o a product or a measurement procedure,namely in technological felds where only a ew players have property rights, exclusive knowledge, or the

exclusive resources needed to use a technology (Guash et al., 2007, p. 25).

The QWERTY computer keyboard is a well-known case o technological lock-in. Although many alternativekeyboard layouts have been designed to increase typing speed and comort, the nineteenth-century QWERTYlayout is still used because the costs o retraining workers to use a dierent keyboard would be prohibitivelyhigh. Furthermore, new users will still preer to learn to type on a QWERTY keyboard as long as this layoutdominates the market. (Guash et al., 2007, p.25)


16/44

16

In this subsection, we look in detail into the expected impacts o the dierent quality services to innovationsystems, how their specifcities have implications on the unctioning and impacts o the system as a whole.

Standardization

Standardization may work as an instrument that: osters international trade because it normalizes the characteristicso products and processes across countries; promotes technological transers by acilitating the integration o value

chains; creates incentives or the investment on R&D by reducing the risks associated with it as long as companiesare able to inuence the content o standards (DIN, 2000, p. 16).

As summarized by Araby (no date), it also increases the economic surplus o both consumers and producers:

For consumers: transmits inormation about the products, allows a more eective comparison between goods, increases competition among producers leading to lower prices, creates the possibility or the consumer to mix and match components within the same system.

For producers:

reduces production costs (economies o scale and learning); through the inormation contained in the standards knowledge and technologies are diused, hence

productivity and efciency increased; permit repetitive production, reduction o inventories the combination o parts and components.

However, the misuse o such an instrument can also bring adverse eects. Standardization can, or instance, worktowards the protection o certain companies or national industries. This discussion is particularly complex giventhe multitude o impacts that standardization may have considering the perspectives o each individual frm, thesociety or an industry as a whole.

Box 3: Controversy about innovation eects o standards

The contributions o individual quality services

Meeting standards and regulations can also be perceived by business as a constraint on their ability toundertake certain orms o innovation. There is though scope or debate about whether such constraintson individual businesses reduce competition and innovation at the level o the market or the economyas a whole. The argument is essentially that by setting some limits on novelty, the trajectory o productdevelopment is moved closer to the preerred path or customers, who value backward compatibility andsome stability in market oerings. Many irms report inding meeting standards and regulations to be

actors hampering individual innovation projects. () There are two reasons why this is an apparent andnot a real paradox. One is that what the individual irms perceives as a problem may be o beneit to thenational innovation system and thereby to overall economic perormance. Standards enable larger marketsand promote competition, so individual irms see a smaller market share and more competitors than theywould like. But this diversity aces customers with more compatible choices and lower prices, promoting their

welare. (NSSF, no date, p. 3)


17/44

17

While some positive eects o standardization on economic efciency are widely undisputed, the aboveargumentation (Box 3) shows that the impacts o standards on competition and innovation are somehowambivalent and subject to debate.

Furthermore, once the technology is available, an additional problem requently arises o reaching agreementamong buyers and sellers on the content, orm, and timing o the standard. Although common to most standards-setting processes, this latter problem is particularly severe in markets based on advanced technologies because o

the typically intense levels o competition and the dynamics o the technological change process (Tassey, 1982,p.311).

In standardization many conicting interests are at stake. Through the establishment o standards some players winand some lose as frms try to control their content. When analyzing the impacts, it is important to rame the servicewithin the strategic objectives to be reached through the means o quality inrastructure and the specifc rolesthat standardization is to undertake. This is a way o evaluating whether the interests o societies are saeguardedor i the creation o a standard is serving vested interests. It promotes the transparency and accountability ostandardization institutes to civil society.

Metrology

Quality inrastructure services such as testing, inspection, certifcation, and accreditation rely on accuratemeasurements. The metrological and calibration activities are typically perormed by the National MetrologyInstitute (NMI). According to Sanetra and Marbn (2007, p. 66) the role o a NMI is to:

() obtain, conserve, develop and disseminate the basic measurement units and the highest level o calibrationstandards. It provides traceability to the national system and it ensures that international technical guidelines areollowed or the metrological perormance and testing procedures o measuring instruments subject to legal controls, androm the point o view o manuacturers it ensures that their products meet international specifcations or metrologicalperormance and testing.

Once a well unctioning NMI is established, it is easier or companies, research institutes, testing labs, universities,to interact and collaborate, fnd more efcient production processes and new products or the markets. The qualityo the goods produced will be more consistent, hence acilitating commercial transactions and allowing their

regulation the creation o new standards. The integration o the NMI in regional metrology institutes strengthenstrust in the national production system, acilitates the access o local companies to new markets or their exportsand osters the integration o frms in global value chains.

As asserted by Swann (1999), metrological and calibration activities are oten regarded as having a public goodcharacter. There are reasons that sustain the idea that this sector should be supported by public policy, that justiypublic intervention and unding:

High fxed costs associated with the research and small marginal costs or the spreading o the developedknowledge;

High externalities, since the investment o one metrology agency in this feld will create benefts or other

players;

Strong network eects derived rom metrological activities;


18/44

18

Need to ensure that knowledge is not monopolized;

Need or impartiality and integrity o measurements.

For these reasons, Swann deends that the production o metrology is more efcient when centralized, coordinatedat one place, and when the fndings are provided to all the players. The contributions o metrological activities

to economic systems are also relevant or enabling technology-based growth as they aect all stages o thetechnological change process rom research and development (R&D) to market transactions involving the fnaluser (Tassey, 1982, p. 311).

The eective establishment o NMIs enhances the ollowing impacts:

On market efciency: Reduction o inormation asymmetry. More accurate measurements reduce the inormation gap between

what buyers and sellers know about a product, hereby reducing inormation asymmetries, which will inturn reduce raud.

Reduction o transaction costs. Once it ensures smaller measurements errors, the disputation within themarket will be reduced, namely the costs associated with double testing, the time spend with discussions

or the ormulation o contracts regarding the properties o the traded good, converting units. Improve allocation o resources. I transaction costs and inormation asymmetry are reduced, economic

efciency is increased in the sense that decisions are better made by economic players according to theirneeds and budget constraints.

On governance: Government revenue. A signifcant part o the fscal revenues comes through taxing transactions, namely

exports. The more accurate the commodities are evaluated, less arbitrariness will underlie the tax systemand the better the estimations that government makes about its own revenues. This is relevant or politicalstability and also or the better intervention o governments in the markets, namely by using fscal tools(taxes and subsidies) in response to structural and conjectural changes.

Allowing regulation. Legal metrology allows the credible enorce o regulations associated with, orexample, issues as public health, public saety, or environmental issues (e.g. the CFCs and ozone

depletion). This is only possible once there is a measurement ramework that allows evaluating whether aproduct is in conormity with the regulation itsel.

On eectiveness o R&D Provision o inormation. To measure is to know and i you cannot measure it, you cannot improve

it, are words attributed to Lord Kelvin, 19th-century mathematical physicist: The provision o accurateinormation about physical product characteristics is an important input to R&D and part o theknowledge production and innovation process, supporting the improvement o process technologies.


19/44

19

Conormity assessment

The conormity assessment has the role within a quality system o dierentiating those goods and services thatconorm to a standard and those that do not. As argued by Guash et al. (2007), without such dierentiation,standards are o limited use and the economic benefts associated are not reaped. Thereore, the importance othe conormity assessment is directly related with the impacts that metrology and standardization are supposed tobring by increasing their magnitude.

The conormity assessment ensures that product conorm to specifed characteristics, increases the inormation thatconsumers have regarding the products, creates incentives or producers to upgrade their production processes- it establishes a visible link between standards and the market (DTI, 2007, p.2). As elements o the conormityassessment ramework testing, inspection and certifcation services can be distinguished.

Testing and inspection activities directly beneft some o the stakeholders: Decrease the gap between expectations or the outcome o a certain production process and actual

results. I returns become more predictable, the risks o investment are reduced and investment levels areincreased.

Improve the accuracy o measuring equipment means that the research or new products or processes usesbetter tools and thereore the results o such research are likely to be better.

The benefts o certifcation are to be directly elt by producers and consumers: For producers who aim at providing goods that comply with certain standard, they proft rom certifcation

because they are able to dierentiate their products rom those that do not comply with standards. Thepossibility o having the certifcation o a product or process creates incentives or producers to investin their quality, as it will be compensated with a dierentiated demand curve or their products and thepossibility o charging a higher price or their products. This brings more dynamics to the innovationsystem as it publicly recognizes that innovations add value to commodities.

For consumers, it is benefcial because it increases the inormation available about the dierent productsin the markets. They are able to compare products regarding a certain number o characteristics, thereorebeing able o making better judgments about the quality o the product that they are buying. Theirdecisions are more rational in the sense that they can fnd product that are more suitable given their needsand budget constraints.

Accreditation

The main unction o accreditation is to assess the competence o the remaining quality institutions, i.e. to createtrust o the economic agents in the quality inrastructure. For this reason, accreditations most salient role is toampliy the impacts o each individual quality service and o the system as a whole. For instance, as accreditationensures the correctness and independence o the certifcation process, it guarantees the quality o the certifedcommodities to the consumer. As the credibility o certifcates increases, producers reap more easily the benefts otheir innovations, which create incentives or investment in R&D.

Accreditation may indirectly also contribute to increase the quality o quality services. For instance, the evaluationo laboratories contains, implicitly or explicitly, suggestions to improve the perormance o these laboratories. This

represents a transer o knowledge rom the accreditation institutions to the assessed laboratories (Gilmour et al.,2003).


20/44

20

This element o quality inrastructure is especially important in the context o international trade and globalizationo value chains. On one side, as long as the buyer trusts in the certifed good, he does not need to do new testsin order to assess the quality o the good (avoid double testing), and on the other side, contracts are more easilysettled, the penetration in oreign markets ostered.

Optimally, accreditation bodies are independent, impartial and recognized internationally. When that is so, theyensure competence, confdence, reliability, transparency and political independence. This is particularly more

difcult in small countries, where accreditation institutions have insufcient income to ace the costs and, thereoreare more dependent on subsidies rom the state (jeopardizing the objective o impartiality). In addition, the lacko practice o the sta keeps low their expertise when compared to their counterparts in bigger countries (Sanetraand Marbn, 2007),


21/44

21

Conormity

Assessment

Check whether

management

procedures,products or services

conorm with

established

standard

Conormity

Conidence

Reliability

Firms Consumers

Reduction oinormationasymmetry

Innovationpremium

Accreditation Formal

recognition that

an organization

or person is

competent to carry

out speciic tasks

Competence

Traceability

Transparency

Political

independence

Qualityinrastructureas a whole

Economicintegration ininternationalmarkets and valuechains

Provideinormation toquality servicesabout betterpractices

QI Activity Main unctions Main beneiciaries Main Impacts

Standardiza-

tion

Formulation o

standards (optional

compliance)

and technical

regulations

(compulsory

compliance)

Knowledgeexchange

Coordination Harmonization

o products and

procedures

Firms Consumers

Economies o scale Economies o

learning

Innovation Diusion o

technology Competition Lower market prices Consumers and

environmentprotection

Metrology Establish

measurement

procedures and

ensure calibration

o measurement

instruments

Traceability

Comparability

Uncertainty

reduction

Firms Industry Government

Consumers

Eiciency o R&D Access to oreign

markets

Integration in globalvalue chains

Stability ogovernmentrevenues

Consumerprotection against

raud

Table 2: Impact expected rom quality inrastructure services


22/44

22

Empirics on the impacts o dierent quality inrastructure

elementsStandardization

Most o the current empirical studies concerning the role o standardization are at a macroeconomic level and areocused on its impacts on economic growth, trade, innovation, and development. In this subsection some o theoutcomes o these studies shall be summarized.

Impacts on growth

Standardization has dierential impacts across sectors and countries: For instance, a more signifcant impact on

productivity has been observed in mature industries than in R&D-intensive industries. An overview o the empiricalliterature is given by AFNOR (2009).

Table 3: Impacts o standardization on growth

(Source: AFNOR, 2009)

Blind, Grupp and Jungmittag (1999). In this study the German economy was analyzed during the period o

1960 1996. It identifed a statistically signifcant positive impact o standards on productivity and in terms othe contribution o standards to growth. It was estimated that the impact o standards accounted or about1% point o the average growth rate.

AFNOR (2009). This study observes the impact o standardization in two dimensions. From a macroeconomicstandpoint, standardization directly contributes to the growth in the French economy by an average o0.89% per year, or almost 25% o GDP growth. The study also confrms the microeconomic beneftso standardization acknowledged by companies o all sizes and rom all sectors o activity: productinteroperability, increased productivity, market share gains, and ease o cooperation with public R&Dinstitutions.

DTI (2005). This study analyzed the impacts o standardization in the UKs GDP growth in the period 1948

2002. It concluded that standards contributed with 0.25 percentage points o the total GDP growth in UKduring this period.

Country Analysed Germany UK Canada Australia France

Impact in % points ostandards on GDP growth

0,9 0,3 0,2 0,8 0,8

Contribution o standards togrowth o GDP (%)

27,3 11,0 9,0 21,8 23,8

Contribution to theproductivity o work (%)

30,1 13,0 17,0 Not estimated 27,1

Organization DIN DTICanadianCouncil oStandards

StandardsAustralia

AFNOR


23/44

23

Impacts on trade

Empirical studies generally hint at a trade enhancing role o harmonized standards, but there is no consensusconcerning the role o country-specifc standards. A review o the empirical literature on standards and trade isgiven by the Guash et al. (2007) and Swann (2009).

Blind and Jungmittag (2001). In this paper, two models are estimated. One takes in consideration the trade

relations o Germany with the rest o the world and the other one, the bilateral trade relations betweenGermany with UK. It was ound that: regarding the adoption o international standards, standardization hasa positive and weak but signifcant eect on trade (increase o imports and exports); when Germany adoptednational standards, that the eect was negative and weak but signifcant.

Moenius (1999, 2004). These studies reer to a trade-o associated with the implementation o standards at anational level. From one side it demands adaptation costs or exporters, thereore hindering trade; on the otherside, they provide relevant inormation or new entrants in the market regarding the preerences o consumerswhich otherwise would be very difcult and costly to collect. Furthermore, it is argued that the latter eectprevails; meaning that the reduction o transaction costs compensates the increase o adaptation costs, andthereore trade is promoted. In these studies 12 OECD countries are analysed.

Swann, Temple, and Shurmer (1996). In this study, an econometric analysis is made in order to evaluate theimpacts o standards on UKs trade perormance between 1985 and 1991 regarding the manuacturing sector.The authors argue that, according to the results, it improved the trade balance perormance o UK but thatit also helped making the market more open. It is urther asserted that national standards are by and largeirrelevant or the trade perormance o UK.

Raballand and Aldaz-Carroll (2005). In this paper, it is assessed the impacts o existing multiple standardson trade costs applied to the case o pallets. The main fnding is that most o the burden o such an excesso variety is on low income countries. The increase o costs is associated with necessity o manually load andunload the goods that are in the pallets at borders given the specifc standards o the destination country,increasing delivery time and costs. As argued, this will be particularly more difcult to bear or low incomecountries given the unavailability o outsourcing, renting or exchanging pallets. Taking the case o the exportso bananas rom Ecuador to Europe, the lack o standardized pallets will increase by 21% the transportation

costs. How Do Diering Standards Increase Trade Costs? The Case o Pallets.

Chen et al. (2006). This study tries to quantiy the impact o standards and technical regulations on the exportperormance namely in low income countries. Their results indicate that when standards are not harmonized, itcreate diseconomies o scale, prevents the entry o exporters in new markets. As argued, testing and inspectionprocedures reduce trade by 9% and 3% respectively. Multiple standards reduce the probability o an exportingcompany to penetrate in more than 3 markets by 7%.

Impacts on technological innovation

The empirical relationship between innovation and standardization shows an inverted U nonlinear pattern. Forexample, standards promote the adoption o new technologies as sources o inormation on innovation, but also

introduce economic constraints that impede the innovation process.


24/44

24

DIN summary report (2000). This study takes in consideration our main stakeholders in the standardizationprocess: businesses, private households, the state and the standards body. In order to collect inormation, itwas conducted a survey to companies in dierent industrial sectors, interviews to dierent partners, regressionswere estimated in order to establish a relation between innovation and standards. According to this study,which also reers to the difculties o separating the eects o standards on innovation rom other actors, thereare positive eects o standardization on innovation. As argued in this paper, standards lead to innovationby a reduction o the risk associated with R&D and by directly reducing its costs. Among other things,

harmonization o technical rules (implied in standards) helps businesses and institutions to communicate,cooperate and establish alliances, and thereore reduces costs. However, as argued, these alliances can alsolead to a higher concentration o market power, and thereore to less competition.

Temple, Spencer & Witt (2005). These authors estimated the elasticity o labor productivity with respect tostandards to be around 0.05. This means that a 10% increase in the standards catalogue is associated witha 0.5% increase in labour productivity and, given that this study takes into consideration only UK during theperiod 1948 to 2002, it means that during this period standards contributed or 13% o labor productivitygrowth. They urther add that standardization processes are very connected with other actors such asinnovation and that it would be difcult to separate the specifc eects o trade separated rom other actors.

Impacts on low and middle income countries

Empirical fndings on the impact o standardization on low and middle income countries draw a very mixedpicture. On the one hand, some case studies reveal the potential o standards as a technical barrier to trade, whichexcludes developing countries rom international markets. On the other hand, there are positive cases illustratingstandardization induced technological upgrading and transition to higher-value-added activities o frms indeveloping countries. Buyer-driven value chains seem to be o particular relevance to the integration o small lowand middle income countries exporters to the global economy.

Wilson and Otsuki (2004). This study was based on a survey that enquired 689 companies, in variousindustries, spread in 17 low and middle income countries. The majority o the frms surveyed acknowledgedstandards as a barrier to their exports. Nevertheless, a signifcant percentage o Eastern European and Latin

American companies stated that the existence o standards were important or the success o their exports(namely, quality standards and testing/certifcation requirements). By some companies it was also recognized

that standards reduce production costs. Trade barriers posed by mandatory standards: technical regulations,additional compliance costs or each export market.

Czubala, Shepherd, and Wilson (2007). In this paper which examines the impacts o standards in 47Sub-Saharan countries regarding the textiles and clothing industries, it was ound that EU standards that dier

rom ISO standards restrict the level o exports rom those countries to EU, but harmonized with ISO standardshave no signifcant negative impact on exports.

Maertens and Swinnen (2006). This study analyses the impacts o EU standards on Senegals exports o oodand vegetables. The main conclusions were that, in spite o the stricter demands o EU in terms o standards,the exports rom this country increased signifcantly; and that the existence o such standards led to structuralchanges in the supply chain, namely rom production structure mostly based in small arms to large-scale

arms, yet, as argued, with positive eects in terms o the welare o rural households. The householdsbenefted rom the development o labor markets that substituted the product markets.


25/44

25

Naik (2006). This study analyzes the impacts o EurepGAP certifcation requirements on the grape sectorin India. The major fndings were that such requirements increase production costs by 40%; imply theemployment o techniques harmul or the soil and the usage o residues that may be toxic; have redistributiveconsequences, namely by impeding small armers to enter in the market. On the other side, Indias marketshare is growing, domestic markets are developing, and an innovation system has been developed made opartnerships between public and private entities.

Metrology King et al. (2006) tested whether good measurements would support innovation because the incentives to

innovate depend on the capacity o the frm and consumer to measure and veriy whether the product hascertain special characteristics. He collected data rom the Community Innovation survey and regarding theuse o the National Measurement System (NMS) by dierent industrial sectors. The fndings were dierent

or product innovations and process innovations. While the impacts o good measurements were signifcantand positive or the ormer, the eects on the latter were not signifcant. It was also shown that the directionand magnitude o the impacts also depend on other actors that range rom good testing systems to the levelcompetition on goods and services markets.

NIST (2006). The American institute or standards and technology conducted a survey which enquired or

measurements requirements on dierent sectors asking how innovation was rerained by the lack o goodmeasurements. Two o the most notable outcomes o this study were: lacking accuracy o measurement tool isthe most common problem to innovation, namely in those sectors whose dynamics lead to more technologicalchanges (e. g. IT, telecommunications, health care, nanotechnology); the inexistence o standards, metrics andso on, that enable the assessment o new technologies hinders signifcantly innovation.

Birch (2003). The Canadian institution or legal metrology assessed the impacts o their measurement activitieson trade. In this study it was argued that or each dollar spent by this organization with periodic inspectionin order to evaluate the quality o measurement tools would save in average $11.40 in measurement errors.This value would vary rom sector to sector. In those where the measurement accuracy was more relevant, thisvalue would go up to $28.7.

National Measurement System o UK (1999). In this paper several case studies made by the Department o

Trade and Industry o UK (DTI) are analysed. Among others, the case o Warm Petrol is a good illustrationo the importance o legal metrology. This case reers to the losses associated with the temperature o oil,higher than the ambient, incurred by retailers because suppliers were ignoring that petrol contracts when itcools to the ambient temperature and that there are vapour losses associated with the temperature at whichit is delivered. The dispute between retailers and suppliers went on or more than a decade and the lossesallied with the high competition on the retail market started putting many frms out o the business. Thisissue caught the attention o DTI who made an assessment o the benefts i legislation would be made bythe National Measurement System in order to orce oil suppliers to keep the temperature o the oil below theambient temperature. In order to do so, the costs that producers would have to incur to comply with the newlegislation were calculated, and amounted to 15 millions, and the costs associated with to implement thislegislation was o 75 millions. On the other side, externalities are expected rom this legislation: retailerswould ace lower losses and less retailers would go out o business hence increasing competition. For retailers,

losses would be reduced by 80 million. Regarding the retailers that would not go out o business, thereduction o employment that is avoided amounts to a beneft o 13 million. As mentioned in the paper,most o the benefts would be reaped every year while the costs would mostly be only incurred once.


26/44

26

Lima et al. (2008). The Brazilian Agricultural and Forest Management and Certifcation Institute made animpact assessment study to measure the consequences o socioenvironmental certifcation on community

orest management in the State o Acre or wood production. This study shows that the impacts o thiscertifcation were relatively small, because there are already several public institutions involved withCommunity Forest Management. Still, some changes were noted, namely regarding the disposal o residues,awareness about the use o fre, measures to protect wildlie and degree o involvement in environmentalcomplaints. In order to sanction the impacts o this certifcation, it was selected a group o community

producers with similar economic and geographic characteristics to those o the certifed community producersand against whom a group o certifed producers were compared regarding: environmental preservation,quality o the administration o the association, use o Personal Protection Equipment (PPI), and income romwood sales.

Imho and Lee (2007). This study aimed at assessing the impacts o certifed air-trade plantations in Bolivia.Some o the major fndings were that: air trade has impacts on poverty reduction and thereore contribute orthe reduction o political related conicts; it increased the competition among intermediaries as a result o theemergence o air trade cooperatives which led to an increase o the price paid to the producers; it enabledcapacity building because certifed air-trade implies constant training on organic production, managementissues and so on. In order to conduct this study, 160 interviews were made using a semi-directly administeredquestionnaire, which permitted the gathering o data regarding prices, yields, processing stages, and average

costs.

Martin et al. (2000). This is an impact assessment study on NISTs certifed reerence material or the sulurcontent in ossil uels on a range o economic and environmental issues, namely, on the increase o productionefciency, changes o transaction costs, reduction o sulur entering the environment, reduction o penaltiesimposed by regulatory agencies. In order to do so, it was evaluated how the reerence material led to a changein the accuracy o the measurement o sulur in ossil uels and in the cost o these tests, how these changesaected the behavior o stakeholders, and the quantifcation o the economic benefts or losses associated withthese changes. In order to compare the situation beore and ater the creation o standard reerence materials,it was assumed that the level o uncertainty associated with the measurement o sulur in ossil uels wouldbe today similar to what it was prior to the introduction o the reerence material in the middle 1980s. Theestimated present value o benefts was US$409 million against costs o US$3 million.

Accreditation

One o the main unctions o the accreditation element o the quality inrastructure is to strengthen theinternational recognition o the national quality system. However, it was not possible to fnd impact studiesspecifcally related with this quality inrastructure service. A probable reason or that concerns to the difculty toisolate the impacts resulting rom a given accreditation rom other quality inrastructure activities.

Conormity assessment


27/44

27

The two main tasks o impact studies are to establish causal relations between an intervention or group ointerventions and changes in the social, economical and/or environmental domains and to estimate the magnitudeo such relations.

Box 4: Case studies (CEPAL, 2011)

In order to eectively do so, the design o an impact study needs to be adapted to the specifcities o theinterventions that are analyzed. In the case o quality inrastructure services, on the one side, impact studies haveto be aware o the existence o complementarities between services, meaning that certain impacts can only beunderstood as a consequence o the activities o various quality services. LATUs study is illustrative o an impactanalysis that takes the eects o various quality services into consideration. On the other side, impact studies haveto take also into account that quality services oten receive public intervention and thereore they shall be able toanswer the question o whether the overall public interests are saeguard, i.e. stating how the various stakeholdersbeneft or lose rom a certain quality service while bearing in mind the middle- and long-term eects o theseservices (situating them in the strategic goals o policy makers). INTIs study in Argentina oers a detailed pictureo how the dierent stakeholders were aected by the intervention, i.e. economic agents in the private sector, the

State, and with an emerging economic sector that ollows INTIs intervention.

On the strictly technical side, the most prominent challenge associated with doing an assessment study on qualityservices has to do with the kind o impacts that one is looking or. These services perorm systemic unctions withininnovation systems and or this reason many o their eects on social, economical and environmental realms areindirect. This does not mean that it is not possible to estimate these impacts. Instead, that it is more delicate toestablish a causal relation (i.e. to isolate impacts o the quality service) as this requires the kind o data that usuallyis not collected through common means.

To overcome this obstacle one has to establish a system o data collection since the beginning o theimplementation o the quality service. A list o indicators required or the utures assessment study is to bedefned. Those indicators that are not already collected by any institution or whose quality is questionable have to

be identifed and a mechanism to collect them established. These systems support insightul analyses, which arenot only important or the estimation o the impacts but also to provide suggestions or the improvement o theservices. INDECOPIs assessment o the impacts o installing water micro-meters in Lima is exemplifcative o theneed to establish a data collection system beore the intervention. This case concerns to the water consumptionlevels o the households and how it changed ater installing the water meters. This baseline data would not be

Designing impact assessment studies on quality

inrastructure

In order to systematize the methods employed in impact assessment studies on quality inrastructure,literature on the topic was reviewed and lessons were derived rom the ollowing impact assessment studies,which had our methodological support: A case study conducted by CENAMEP (Panama) about the eects o large scale calibration, A case study conducted by GRADE (Peru) about the eects o the introduction o Pisco certiication on the

Pisco industry, A case study conducted by INDECOPI (Peru) about the eects o installing water meters on private

households in the city o Lima, A case study conducted by INMETRO (Brazil) about the eects o the development o certiied reerence

materials or bioethanol on producers and laboratories, A case study conducted by INTI (Argentina) about the eects o legal metrology, A case study conducted by LATU (Uruguay) about the eects o various quality inrastructure services on

the milk industry.


28/44

28

collected and thereore it would be impossible to estimate the change in the consumption patterns i a datacollection system as such would not be put in place beore the actual intervention.

The basic structure o an impact assessment study consists o three stages: construction o a theory on theimpacts o a specifc project; ormulation o the appropriate methodologies to test each impact; establishment oconclusions that can be derived rom the results (whether and how the theory o impacts was verifed) along withthe provision o policy recommendations.

1st Stage: constructing an impact theory

In this stage, the researcher has to stipulate the links between a quality service and its expected main social,economic and environmental consequences, as well as to identiy the stakeholders most probably aected by suchimpacts. The selection shall be broad enough to capture all the important impacts; but not too broad because itwould be too costly to evaluate all the imaginable impacts.

The eectiveness o an impact study relies on its capacity to convince the reader that the claimed changes are dueto the existence o certain quality service and not to other actors. Studies in this feld have to address the idea thatsome impacts are the consequence o the activities o more than one quality service and that some players maylose rom such activities. Thereore, the mapping o impacts has to state those that the service is producing alone,

the ones that are resultant rom the combination o various quality services, and stipulate a number o possibleagents that may be benefted or hindered by the existence o certain activities. By doing so, the systemic nature oquality services is addressed and the assessment study becomes better prepared to contribute or the ameliorationo the service itsel.

In addition, a theory o impacts shall question the existence o actors that may condition the perormance oindicators and which are external to the intervention.

At last, an inquiry shall be made concerning the location, sector o activity or other socio-economic characteristicso the stakeholders and that may aect the magnitude o the impacts o the quality service. This contributes or theunderstanding o how the impacts shall be maximized given specifc socio-economic eatures o the stakeholdersand may oer important policy recommendations.

Structuring a detailed theory o impacts supports an investigation able to capture: a wide variety o impacts, issuesinuencing the eectiveness o the quality service, external actors that need to be isolated in order to avoid biasedestimations.

Apart rom contributing or the development o a consistent impact assessment study, such an approach improvesthe cost-eectiveness o the study, especially as through a single survey all the required inormation can begathered.

To sum up, an impact theory states: what aects whom in which ways and through which channels. For each oneo the ormulated impacts, the researcher must be clear about such a narrative and may sum it up entire theoryusing the Summary Matrix (Annex 1). For each stated impact, an appropriate method and indicators are to beselected and this is the second stage o an impact assessment study.


29/44

29

There are no methods that can be said perect or all the situations. The selection o a method always depends onwhat is tried to be proven (which impact?), the inormation that is required in order to assess it (which indicators?),the inormation that is possible to collect with quality. Accordingly, a study may utilize dierent methods toestimate each impact. There are our main approaches that are oten used to estimate the impacts o interventionsin the feld o quality inrastructure. In the ollowing, a brie description o the reasoning underlying these methodsis presented.

Experimental method

This method requires the selection o the social group that receives the intervention o a quality inrastructureservice (i.e. treatment group) and a comparison group that does not (i.e. control group) prior to the beginningo the intervention. Such selection process has to guarantee that both the treatment and the control group arerandomly selected and share the same relevant characteristics. Both groups have to be comparable: the dierencesrealized ater the intervention should be attributed to the intervention itsel, and not to other socio-economic

eatures, nor to the inuence o other interventions. In order to estimate the impacts, the dierence between thetwo groups sanctioned by the indicators beore and ater the intervention would be considered the impacts o theservice. INDECOPIs study uses this method. The capacity o establishing a causal relation between the interventionand the impacts and o estimating their magnitude is very high. However, the process o implementing a studyas such requires more time, and more costs with human and technical resources. Thereore, this method is

especially suitable or pilot projects, i.e. when the implementation o a certain quality service would require veryhigh investment levels, thus being important to know in advance whether the intervention leads to results thatcompensate the investment.

Quasi-experimental method

The quasi-experimental method is very similar to the experimental method. The key dierence in this empiricalapproach is the lack o random assignment. This may be the case when a priori selection o the treatmentand control groups is not possible, or instance within an impact assessment study o a services that is alreadyunderway. The INDECOPI case illustrates the difculty in realizing random assignment, as there are technicalas well as moral restrictions to the purely random implementation o water meters. Since a defciency inrandomization makes it harder to rule out conounding variables and introduces new threats to internal validity,INDECOPI has still made substantial eorts to make the household selection as random as possible.

In quasi-experimental designs, where random assignment cannot be assured, the membership assignment to thetreatment or control group may be subject to a selection bias. One way to overcome this problem is to match theparticipants o the two groups aterwards by propensity scores according to their socio-economic characteristics,

or instance, age, education, income, sectors o activity, religious belies, etc.

GRADEs study gives a good example or a quasi-experimental design. In this case, there exists a data selectionproblem, as those companies that comply with the pisco certifcation have very dierent characteristics whencompared with those that do not, since the certifcation is de acto obligatory to producers that sell in Lima andexport, but unnecessary or those producers that serve other markets in Peru. The study describes how these biasesmay be corrected or by propensity score matching procedures. In many cases this posterior matching may beeasier applicable than the rigorous experimental method, but its power to establish a causal relation between an

intervention and the impacts is, naturally, not so strong.

2nd Stage: ormulating a method


30/44

30

Both the experimental and the quasi-experimental methods are eective when the expected impacts o anintervention can be directly sanctioned by socio-economic indicators. However, quality services oten have impactsnot so easily sanctioned through such indicators. Both the contingency valuation and the hypothetical deductivemethods may be solutions to overcome this drawback.

Contingent valuation method

INMETROs study on the impacts o the development o a certifed reerence material or bioethanol uses thecontingent valuation method. This method was initially created to estimate the value o public goods or services

or which no market exists since users cannot be excluded rom its utilization, as or instance environmental qualityor public security.

INMETRO argues that there was no market established or the kind o certifed reerence material when the studywas put in place, as it was not yet widely used by laboratories and producers. Thereore the contingency valuationmethod was applied to estimate whether the benefts rom the introduction o the certifed reerence materialwould at least cover the costs associated with its development and production. A notable advantage o thismethod is that it is less costly then the implementation o an experimental approach (as was used by INDECOPI),however the quality o the results may be questioned.

Similar to the two previous approaches, the contingent valuation method is survey based. This technique consistso inquiring stakeholders regarding their willingness to pay or a certain public good or service or which no marketexists. The most salient challenge associated with this kind o approach is the need to avoid strategic responsesby the participants. This precondition may be seriously violated i respondents expect to inuence the price o thegood or service by their answers. For example, i a good provided by public authorities or a certain ee bringsbenefts to a company, this frm may preer to understate the true value o the good, earing that i it states the realvalue the ee will be higher in the uture. In other situations, respondents may be induced to exaggerate the value.

Since INMETRO can be expected to sell their certifed reerence materials in the uture, the occurrence o strategicresponses cannot be ruled out. Since potential buyers are asked to value the product by a uture potential seller,the situation comes very close to a negotiation. Consequently, the respondents are most likely to understate theirtrue valuation o the product. In this case, the method thus leads most likely to an underestimation o the truebenefts o the product. However, since the objective o the INMETRO study was to show that benefts at least

oset the costs o development, a conservative estimation o the benefts supports the positive result.

Hypothetico-deductive method

The hypothetico-deductive approach is among the mentioned approaches the one that requires the most abstractconstruction. It relies on reasoning when empirical measurement is not possible. Optimally, deduction begins withtheory (laws or law-like premises), it moves through the use o logical rules and reasoning to hypothesis derived

rom the theory, and then tests hypotheses via prediction and observation (Ezemeri et al., 1999, p. 30).

This method is useul whenever it is difcult to fnd socio-economic indicators representative o the impacts thata quality service is having. For example, i an intervention increases the accuracy o an instrument to measurethe content o sulphur in oil by x%, one hypothesized impact could be: How much lower is the probability o a

company to ail on complying with the regulation that states the maximum level o sulphur allowed in a givenquantity o oil? In this case, the changes o accuracy are estimated and using this number as a premise, theconsequences o such a change are hypothesized and its magnitude predicted.


31/44

31

With this method the impacts are not empirically verifed - it would be difcult to isolate eects o a bettermeasurement instrument with the actual augment on the probability o complying with a norm because there aremany other variables inuencing it. The lacking o empirical verifcation particularly acilitates the overestimationo the impacts o a quality service and that may seriously damage the credibility o the study. Though it is a veryimportant methodological approach given the kind o impacts resulting rom quality services, it needs to becareully handled, particularly when it comes to the design o the counteractual.

The experimental and quasi-experimental methods are suitable when the indicators can be sanctioned by thestakeholders. For example, i the hypothesized impact concerns to the reduction o the health problems ater thecreation o a certain technical regulation, the stakeholders may be asked to state the number o days per monththat they have to miss work due to health problems.

On the other side, i the creation o a certifed reerence material or o a standard, as hypothesized in the impactstheory, leads to a reduction o production costs which are difcult to isolate given the existence o many other

actors contributing to changes in the production costs, the hypothetico-deductive approaches may be the mostappropriate methods.

Among the case studies reviewed, both INTI and CENAMEP developed their impact assessment studies usinga hypothetico-deductive approach. Both o them deal with the impacts o a better calibration o scales. They

started by empirically measuring the change in the precision o the scales ater the intervention, and developed areasoning concerning the impacts on costs and government revenues that ollows such a hypothetico-deductiveapproach.

The counteractual

These methods are trying to answer to the question: how would things dier without the quality service? Thehypothesized situation is called counteractual. This is one o the most crucial elements o an assessment study. Apoor design o the counteractual damages the credibility o the studys results. Thereore, it is very important toproperly establish the counteractual and to elaborate on the reasons or such a selection i the reader does notagree with the realism o the counteractual, he will hardly believe in the outcomes o the study.

I a quality inrastructure service concerns to a small neighborhood in the outskirts o a given city, the perect

counteractual would be: the same small neighborhood in the same period o time without receiving thementioned service. Since it is not possible to have the two situations simultaneously, there is no perectcounteractual, only approximations.

There are two ways o constructing the counteractual:

By comparison with a similar society that did not receive the intervention. This is the control group approachused in the experimental and quasi-experimental approaches


32/44

32

Figure 1: The control group approach

By comparison with what would be the situation o the society that was aected by the quality servicei there had been no intervention. This is the non-experimental approach used in contingent evaluationand hypothetico-deductive methods. Especially the study by INTI is very clear about the construction o acounteractual as such. Similarly, LATUs study analyzes the trends concerning the quality o milk in Uruguayas a consequence o the various interventions. In this case it is more difcult to defne a beore and aterintervention because

Documents

Broschuere Guide 7 Measuring the Impacts of Quality Infrastructure e