Zusammenfassung:  Dieser Artikel präsentiert ein neues Datenset zu Messung von bür-gerlicher Freiheit und diskutiert dieses in methodologischer Hinsicht bezüglich Reliabi-lität, Validität und Dimensionalität. Der Datensatz umfasst jährliche Angaben von Ende der 1970er bis 2003 für 28 (post-)kommunistische und 20 lateinamerikanische Län-der. Theoretische Grundlage bildet die liberale Theorie. Zur Messung von bürgerlicher Freiheit werden fünf Indikatoren bestimmt: (1) Unabhängigkeit der Gerichte; (2) freie Meinungsäußerung; (3) Versammlungs- und Organisationsfreiheit; (4) Gedanken-, Ge-wissens- und Religionsfreiheit; (5) Bewegungs- und Niederlassungsfreiheit. Mittels sta-tistischer Tests wird gezeigt, dass die Daten eine hohe Intercodiererreliabilität besitzen und eine gemeinsame latente Dimension aufweisen. Die Studie stellt nicht nur ein neues Messverfahren und einen neuen Index für bürgerliche Freiheit vor, sie bietet auch einen genauen Leitfaden für die Erstellung subjektiver Messungen, der Fragen des Fokus, des Umfangs, der Konzeptualisierung, der Messung und der Aggregation abdeckt.

Schlüsselwörter:  Bürgerliche Freiheit · Messung · Datensatz

Abstract:  This article presents the construction of a new dataset on respect for civil liberty and provides in-depth discussions of the methodological issues related to the reliability, validity, and dimensionality of the scores. The dataset covers 28 (post-) com-munist and 20 Latin American countries on an annual basis from the end of the 1970s till 2003. It is theoretically well-grounded in liberal theory and consists of five indi-cators: 1) independence of courts; 2) freedom of opinion and expression; 3) freedom of assembly and association; 4) freedom of thought, conscience, and religion; and 5) freedom of movement and residence. Statistical tests show that the data is character-ized by a fairly high degree of inter-coder reliability and that the indicators reflect a common latent dimension. Apart from proposing a new dataset and an index on civil liberty, this study offers a meticulous guideline for the creation of subjective measures that addresses the choices concerning focus, scope, conceptualization, measurement, and aggregation.

The Civil Liberty Dataset: Conceptualization and Measurement

Svend-Erik Skaaning

AUFSäTZE

Svend-Erik Skaaning PhD (*)Department of Political Science, University of Aarhus, Bartholins Allé, bygning 1331,DK-8000 Århus C, DenmarkE-Mail: skaaning@ps.au.dk

ZFVP XX (2008) XX:1–23DOI 10.1007/s12286-008-0003-4

2 S.-E. Skaaning

Keywords:  Civil Liberty · Measurement · Dataset

Introduction�

The concept of civil liberty, which pervades internationally defined human rights stan-dards, calls for comparison and measurement. Political theoretical/philosophical re-flections as well as UN treatises and other international conventions, however, do not themselves identify each right in sufficient detail to provide a basis for cross-national measurement and comparison; despite their great significance as a foundation for world-wide agreement on the importance of specific rights. To collect and analyze quantitative information on a particular human right thus requires a lengthy process of developing and testing suitable measurement instruments and data processing procedures (Claude and Jabine 1992: 12). Yet, in the words of Gerardo Munck and Jay Verkuilen (2002: 6, 31):

To a large extent, problems of causal inference have overshadowed the equally important problems of conceptualization and measurement … [but] the careful development of measures constitutes the foundation for efforts at drawing causal inferences and is a critical task in itself.

Because the measurement of core concepts is crucial for the description and explanation of political phenomena, likely effects of inadequate measurement are illusionary in-sights instead of the coveted objective: cumulative knowledge gain. The easiest way to handle the measurement task would be to follow the standard practice, which is to draw upon readily available datasets. However, the existing datasets and affiliated measures on the respect for civil liberties are flawed by significant problems (Skaaning 2006a).

Illustrating this point, the disaggregated scores of the widely used Civil Liberty Ra-ting generated by Freedom House are not publicly available and the underlying coding scheme is merely a checklist that has undergone several changes. Furthermore, Freedom House neither carries out dimensionality tests nor interrater reliability tests, and its measure builds upon a very maximalist definition of civil liberty including questionable components such as personal social freedoms, absence of economic exploitation, and the right to own property and establish private business (cf. Freedom House 2004). By most methodological standards, the best alternative to the Freedom House scores has hitherto been the impressive CIRI Human Rights Dataset, but this dataset is characterized by a number of shortcomings as well. For instance, the measurement levels of the items are not justified and rather rough. Freedom of religion and freedom of movement, two fundamental civil liberties known to face very different degrees of restrictions across time and space, are even coded in a dichotomous way (presence/absence). To specify this problem, the lacking ability to distinguish more adequately between different levels of government repression is one of the main reasons why Germany was assigned the

1 The author would like to thank Jørgen Elklit, Hans-Joachim Lauth, Gerardo Munck, Jørgen Møller, Kim Sønderskov, and the anonymous reviewers for their valuable comments and sug-gestions.

The Civil Liberty Dataset: Conceptualization and Measurement 3

same score on the Empowerment Rights Index2 as Georgia, Haiti, and Kyrgyzstan in 2004. Besides, the aggregation rules behind this indexation are hardly supported by any argumentation (cf. Cingranelli and Richards 2004).

A blatant research challenge thus emerges. The rather insufficient state of the art has motivated the creation of new data gathered in the Civil Liberty Dataset (CLD). Any such effort to create a new indicator of civil liberty should be guided by the goals of ma-ximizing validity and minimizing systematic and random measurement errors (Bollen 1993: 1224). An additional criterion is to present the procedures of the data generation to facilitate quality assessment and ongoing learning processes (King et al. 1994: 8). Yet too few scholars fulfil these criteria and explain and justify the decisions shaping the construction of datasets and measures. The generation of data is a very complex under-taking indeed, which involves an iterative process of theorizing and testing, alongside a wide range of consequential choices (Munck and Verkuilen 2003: 2-3). Consequently, to make decisions more open to scrutiny, this article offers detailed descriptions and discussions of the methodological choices connected to the construction of the Civil Liberty Dataset.

In short, this article presents the construction of an original data compilation covering Latin American and (post-) communist countries3 and proposes the associated Civil Li-berty Index (CLI). The methodological considerations naturally related to the task, such as examinations and discussions of reliability, validity, and dimensionality, follow along this presentation. The article is organized in the four orders of decisions coped with in the process of data and index construction, that is, the determination of the dataset’s focus and scope, conceptualization, measurement, and aggregation procedures.

Focus and Scope

The initial task in constructing a dataset is to specify the principal focus of the assess-ment. To determine the core meaning of civil liberty (cf. Skaaning 2006b), a useful star-ting point is liberal political thinking in general and Isaiah Berlin’s concept of negative liberty in particular. He used this concept to settle the area within which a person or a group is or should be left to do what he or she is able to do or be, without other peo-ple interfering (1997: 200-201). David Miller has nuanced Berlin’s distinction between negative and positive liberty by suggesting an overall tripartition of the main traditions of views on liberty into republican (self-government), idealist (self-realization), and

2 The values of the Empowerment Rights Index reported in the CIRI dataset are based on a sim-ple addition of the scores for five items (political participation, freedom of speech, freedom of religion, freedom of movement, and workers’ rights).

3 Latin American countries: Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, the Dominican Republic, Ecuador, El Salvador, Guatemala, Haiti, Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru, Uruguay, and Venezuela. Post-communist countries: Albania, Arme-nia, Azerbaijan, Belarus, Bosnia-Herzegovina, Bulgaria, Croatia, Czech Republic, Estonia, Georgia, Hungary, Kazakhstan, Kyrgyzstan, Latvia, Lithuania, Macedonia, Moldova, Mon-golia, Poland, Romania, Russia, Serbia-Montenegro, Slovakia, Slovenia, Tajikistan, Turkme-nistan, Ukraine, and Uzbekistan (Czechoslovakia, the Soviet Union, and Yugoslavia).

4 S.-E. Skaaning

liberal (self-protection) freedom. Of these, liberal freedom understands freedom as a property of individuals and it consists of the absence of constraint or interference by public authorities and/or other persons in general (1991: 2).

The latter part of this definition hints to the next step in the focus clarification, na-mely that any given civil liberty measure can be designed from two perspectives. One concerns government compliance – as a means of measuring whether a government respects a particular right – and the other concerns individual enjoyment, that is, as a means of measuring whether each person fully enjoys the right. Although compliance and enjoyment are two sides of the same coin, it makes sense to approach indicators for human rights from both angles (Green 2001: 1085-1086) because the enjoyment of rights can be infringed upon by other parties than the government.

An overview of previous conceptualizations of civil liberty demonstrates that the constitutive attributes of the basic concept show much convergence. However, flagrant divergences also exist (Skaaning 2006a). One of the primary differences mirrors a wi-dely acknowledged, although disputed, theoretical (and empirical) division between personal integrity rights and what I call personal exertion rights, such as freedom of speech, assembly, and association.4 The group of personal exertion rights has elsewhere been labelled basic freedoms (Saward 1994: 16) or – sometimes with a slightly different meaning – political and/or civil rights/liberties (Shue 1980: 19; Milner et al. 1999: 405; Landman 2004: 927; Diamond and Morlino 2005: xxv). The Civil Liberty Dataset is meant to cover the latter type of rights, that is, the extent to which the state restricts cer-tain freedoms understood as actions that individuals or groups might wish to perform. Thus, civil liberty is henceforward primarily understood as something different from, although associated with, personal integrity rights and other alternative conceptualiza-tions of civil liberty/civil rights (cf. Waldron 2003: 195). After the introduction of this restriction, the core concept in question is seen to encompass only one principal dimen-sion – in contrast to most definitions of political democracy (e.g., Dahl 1971: 4-8).

When constructing the CLD, neither the general conditions of the respective ci-vil liberties nor the existing legal protections affect the coding. In other words, the population’s inability to utilize the liberties due to lack of initiative, commitment, finan-cial means, or the like does not influence the score. Similarly, the presence or absence of formal guarantees of civil liberties in the constitution or formal legislation is only taken into consideration when observed by the public authorities. Only the actual practices of governments/states (public authorities in the broadest sense) and their agents, meaning their concrete performance regarding respect for and guarantee of civil liberty, are as-sessed. Furthermore, the geographic distribution of civil liberty violations in a country does not affect the assessment but, on the other hand, only violations that occur within the internationally recognized borders of a country are considered. In addition, it is considered a civil liberty violation if large (mainly ethnically defined) groups without citizenship in the country where they have resided for many years do not hold the same rights as the citizens of the particular country.

4 This distinction is also reflected in the disposition of international and regional human rights conventions.

The Civil Liberty Dataset: Conceptualization and Measurement 5

As for the scope of the dataset, the immature state of knowledge about the develop-ment in and explanation of civil liberty makes it vital to facilitate both intra- and inter-regional comparisons. Therefore, the obvious choice is to focus on countries from two distinctive and broad regions that have been swept by the recent and significant regime changes, namely Latin America and post-communist Eastern Europe. In this way, the dataset can contribute to the debate on the status of the ‘transition paradigm’ as well as the fruitfulness of intra- and interregional comparisons as represented by the positions of Phillippe Schmitter and Terry Karl (1994; Karl and Schmitter 1995) on the one side and Valerie Bunce (1995a; 1995b) on the other.

Inspired by their dispute, the dataset (tentatively) covers 20 Latin American and 28 post-communist countries and the construction is based on the premise that it is impor-tant to study the intra- and interregional differences and similarities in the levels of and developments in civil liberty. Furthermore, the CLD is prioritized to cover the regime changes linked to the third wave of democratization in Latin America as from the mid-1970s and the widespread regime changes in the communist countries behind the Iron Curtain initiated in 1989. Even though it is inconvenient, the completion of a research process demands that a (tentative) final year is selected, which in this case is the year the project was launched, namely 2003.

Conceptualization

The task of conceptualization – specifying the meaning of the core concept by iden-tifying constitutive attributes – affects the entire process of data generation, given that it provides the anchor for all subsequent decisions (Munck and Verkuilen 2002: 7). There are apparently two main ways to approach a definition of civil liberties: either to derive a descriptive definition from the characteristic features of existing regimes or to base a conceptual definition on the history and philosophy of liberal thought. The latter approach is preferred because it rests on abstract values rather than depending on value judgments about present regimes (cf. Forewaker and Krznaric 2001: 3). Then again, the outcomes from either approach are not expected to diverge much.

A detailed exploration into liberal political theory/philosophy, general global and re-gional human rights conventions, and key historical events in the agenda setting of civil liberties (Skaaning 2006b) identified five key civil liberties that the government/state is not allowed to restrict its citizens from performing. Consequently, these aspects are selected to constitute the items of the CLD. The assessed rights expected to reflect a political regime’s liberality are: 1) independence of courts; 2) freedom of opinion and expression; 3) freedom of assembly and association; 4) freedom of thought, conscience and religion; 5) freedom of movement and residence (see figure 1).

These fundamental civil liberties have, more or less without exception, been em-phasized by liberal theorists from the Enlightenment and onwards as well as by the authors of the predominant international human rights conventions. For instance, Ben-jamin Constant associated (civil) liberty with the right to be subjected only to laws, to express one’s opinion, to come and go without permission, to associate with other individuals, and to practice one’s religion of choice (1988: 311). John Rawls’s battery of

6 S.-E. Skaaning

basic civil liberties includes moral liberty and freedom of thought, belief, and religious practice; freedom of speech and assembly; liberty to form associations; and the rule of law understood as regular and impartial administration of law (1971: 211-212, 222-224, 235-243), while the liberties shown in figure 1 are virtually identical to the list of basic freedoms accentuated by Michael Saward (1994: 16). Both the abstract meaning and the specific identification of civil liberties thus rest on a solid foundation building on liberal political theory.

Any definition of civil liberty should address its internal relationship to the concepts of human rights and democracy. As it is beyond the scope of this article to go into detail with this issue, a condensed illustration of my point is shown in figure 2. On the highest level of abstraction we find six human rights categories, written in upper-case letters. Elsewhere they have been divided into three generations of human rights according to their historical sequence (Umozurika 1998: 539).5

5 The first refers to civil liberty and democracy; the second to economic rights, social rights, and cultural rights; and the third to solidarity rights such as peace and environmental protec-tion. The placement of the latter four types of human rights in the different corners does not indicate theoretical or empirical ‘proximity’ to the other concepts.

Civil Liberty

Independenceof courts

Freedom ofopinion andexpression

Freedom ofassembly andassociation

Freedom ofthought,conscience, andreligion

Freedom ofmovement andresidence

The extent to which citizens have the right to settle and travelwithin their country as well as to leave and return to theircountry of own choice without being subject to actuallimitations or restrictions

The extent to which citizens and groups have the right to beunder the jurisdictionof, and to seek redress in, courts that canreview and interpret the law and pass judgments without beingsubject to actual limitations or restrictions

The extent to which citizens, groups, and members of the presshave the right to hold views freely and to seek, obtain, and passon information on political issues broadly understood withoutbeing subject to actual limitations or restrictions

The extent to which citizens have the right to have and changereligion or belief of their own choice and alone or in a communitymanifest their religion or belief in practice, worship, observance,and teaching in private or public as well as proselytizepeacefullywithout being subject to actual limitations or restrictions

The extent to which citizens have the right to gather freely andcarry out peaceful demonstrations as well as to join, form, andparticipate with other persons in political parties, culturalorganizations, trade unions, or the like of their choice withoutbeing subject to actual limitations or restrictions

Figure �   Component Items and their Definitions

The Civil Liberty Dataset: Conceptualization and Measurement 7

On the next level, written in italics, personal integrity rights and personal exertion rights are subsets of civil liberties, while – inspired by Bowman et al. (2005) – access to political power, accountability of government, and political liberties are emphasized as different features of democracy. The concrete rights, written in lower-case letters, make up the lowest level of abstraction. We see that freedom of expression and freedom of association and assembly are both considered to be civil liberties (personal exertion rights) and democratic rights (political liberties), while freedom of movement and free-dom of religion are only civil liberties in the form of personal exertion rights. Finally, judicial independence, integrity, and impartiality, here called the right to independent courts, is a civil liberty linked to personal integrity as well as personal exertion.

The relation between civil liberties and democracy can be summarized in different ways (Bobbio 1990: 48-49). Some see civil liberties and democracy as distinct pheno-mena; that is, democracy is only associated with political self-governance (access to political power and sometimes accountability of government), which can exist, but does not necessitate, simultaneous respect for civil liberties. Berlin (1997: 201-203) argues that liberty in the negative sense is compatible with some kinds of autocracy or, at any rate, with the absence of self-government because civil liberty concerns the area of control, whereas self-government concerns its source. In short, there is a logical distinc-tion between answering the questions about who governs and the extent of government interference in the life of the citizens, respectively (Böckenförde 1991: 365).

Yet, Jürgen Habermas (1992: 610-616; 1996: 294-301), among others, has criticized this schism between civil liberties and popular sovereignty as being flawed becau-se certain rights are constitutive conditions for free political opinion formation and decision-making and thus immanent prerequisites for the meaningful functioning of

DEMOCRACY

Personal Integrity Rights

ECONOMIC RIGHTSCULTURAL RIGHTS

Personal Exertion Rights

freedom ofreligion

Access toPolitical Power

competition

inclusiveparticipation

AccountableGovernment

civilian supremacy

national sovereignty

freedom of movement

independent courtsfreedom ofexpression

freedom ofassembly andassociation

SOLIDARITY RIGHTS SOCIAL RIGHTS

CIVIL LIBERTIES

PoliticalLiberties

dignity

security

Figure 2   Civil Liberties and Democracy in the Human Rights Landscape

8 S.-E. Skaaning

self-government. In line with this argument, Robert A. Dahl (1971: 3) incorporates freedom of speech and press as well as freedom of association and assembly in his po-lyarchy criteria, while – inspired by Habermas – Hans-Joachim Lauth (2004) includes the rule of law (independent courts) as well. Some even consider most, if not all, human rights as democratic rights (Beetham et al. 2001). No matter which relation is judged to be more reasonable, however, it is plausible to treat the features as analytically distinct in theoretical and empirical studies. In addition, all conceptulalizations involve trade-offs, meaning that a choice between maximalist and minimalist strategies – linked to different advantages and weaknesses (Munck and Verkuilen 2002: 9) – must first and foremost depend on the research question at hand.

Measurement

Even though the conceptual challenge has been tackled, it is far from evident how to assess whether countries can be characterized by the state’s observance of civil liberties. Before the actual assignment of scores, a number of decisions concerning measurement level, coding guidelines, and sources must be taken in order to facilitate the coding process and to improve the consistency and utility of the dataset.

As regards the first issue, the measurement level, all items are coded on the basis of a four-point scale with two endpoints. The four points theoretically refer to an overall distinction between ideal typical characteristics of liberal, semi-liberal, illiberal, and anti-liberal regimes (see figure 3). This design is more or less parallel to the division of political regimes into democratic, semi-democratic, authoritarian, and totalitarian types (cf. Merkel and Crossaint 2000; Lauth 2004: 321). Whereas the classical tripartition of political regimes into democratic, authoritarian, and totalitarian regimes used to play – and, indeed, still plays – a dominant role in many classification attempts (Linz 2000; Neumann 1957: ch. 9; Merkel 1999: ch. 1), a slight revision has lately been suggested by numerous scholars.

Based on research on contemporary regimes, the need for an intermediate category between democracies and authoritarian regimes has been emphasized because many present-day countries are only inadequately covered by any of these terms (Schedler 2002; Diamond 2002; Case 1996; Karl 1995; Zakaria 1997; Ottaway 2003; Levitsky and Way 2002; Merkel et al. 2003). There is no reason to believe that this development is not present when it comes to civil liberty since many of the adjectives attached to the democracy concept, such as illiberal, hard, and electoral, refer to inadequate protection of one or more civil liberties (cf. Collier and Levitsky 1997: 438-441). Thus, it seems reasonable to include the semi-liberal category and, accordingly, introduce a four-point

32 41

Anti-liberal(severely restricted)

Illiberal(fairly restricted)

Semi-liberal(modestly restricted)

Liberal(unrestricted)

Figure 3   Measurement Scale

The Civil Liberty Dataset: Conceptualization and Measurement 9

assessment scale. The term anti-liberal, which corresponds to the widely disputed term totalitarian, is used for two main reasons. First, it seems to make up a more proper endpoint of the scale, and second, some of the non-democratic regimes in Latin Ame-rica and many, if not all, of the communist regimes and a few of the post-communist regimes have been deemed totalitarian instead of authoritarian by many scholars.

A more detailed fixed set of coding standards was elaborated that makes the respec-tive scales correspond more straightforward to concrete circumstances. The outcome is outlined in a coding manual inspired by the comprehensive codebook offered by Cingranelli and Richards (2004) in connection with their Human Rights Data Set. At first, several exploratory uses and discussions of this codebook’s guidelines in teaching sessions uncovered some of its advantages and shortcomings. This procedure then led to the drafting of a codebook used to guide the coding process (see extract in the ap-pendix). The codebook was finally subjected to minor adjustments in an iterated fitting process (cf. Bowman et al. 2005: 957), in which the description of the individual scale points for each item was refined to fit specific contexts and the way information was presented in the sources.

The availability of such annual sources on civil liberty observance in the countries of interest is rather poor and the existing sources diverge very much as to the amount and quality of the information provided. However, the U.S. State Department’s annual publication, the Country Reports on Human Rights Practices, is rather consistent across nations, over time, and vis-à-vis the different aspects covered, including the issues co-vered by the component items. The reports are based on information available from a wide variety of sources, among them U.S. and foreign government officials, victims of human rights abuse, academic and congressional studies, and reports from the press, international organizations, and nongovernmental organizations concerned with human rights. In an initial phase, U.S. embassies gather such information throughout the year and prepare the initial drafts of the reports. The reports are then reviewed by the Bureau of Democracy, Human Rights, and Labor, in cooperation with other State Department offices, which draw on their own sources of information such as reports provided by U.S. and other human rights groups, foreign government officials, representatives of the United Nations and other international and regional organizations and institutions, and experts from academia and the media.

The final reports assess the degree to which human rights standards are respected in countries around the world and have, unsurprisingly, been a continuing source of con-troversy since they were first issued in the mid-1970s. The first two reports, those issued in 1975 and 1976, were severely criticized by Congress, which officially turned to the Library of Congress for alternate information. Consequently, substantial changes were made in the preparation of the reports, which significantly improved the scope, quality, and independence of the subsequent reports (McNitt 1988: 97-98). In any case, critics such as the Lawyers Committee for International Human Rights have frequently char-ged the State Department with biased reporting. The prominent accusation has been that the level of repression in countries ideologically opposed to the United States is overe-stimated, whereas countries in which the U.S. has compelling interests are favored.

Yet careful and critical examinations tend to agree that the annual State Department reports are an invaluable source accurately reporting on the conditions in most of the

10 S.-E. Skaaning

countries most of the time (cf. Poe et al. 2001: 650-651). A qualitative comparison with two other prominent sources of comprehensive cross-national information on a broad spectrum of human rights practices, namely the annual reports by Amnesty Inter-national and Human Rights Watch, shows that the Country Reports on Human Rights Practices are by far the most detailed and complete. They are not only more extensive and systematic6 than those of the other monitoring organizations, but they also cover more countries and human rights issues. Actually, the alternatives hardly ever provide usable information on the civil liberties examined in this study, so their inclusion as extra sources would yield little.7

Furthermore, a quantitative evaluation of the difference between scores obtained by coding the annual reports provided by the State Department and Amnesty International with regard to personal integrity rights showed that the Country Reports on Human Rights Practices have at times been politically biased but, that being said, there is no reason to believe that State Department biases have affected the description of the vast majority of cases in any systematic way (Poe et al. 2001: 661, 670). Besides, the bias that appeared in some of the initial State Department reports in the 1970s and early 1980s has tended to disappear over time (Poe et al. 2001: 677). In keeping with this finding, Judith Innes (1992) has shown that accurate and unbiased reporting on human rights issues has become an intrinsically important goal for many key actors within the State Department. Public debate over the information among experts and human rights organizations has increased consistency in definition, accuracy in measurement, and comprehensiveness of coverage over the years. Thus, the reports have come to be generally independent of the administration’s political stance.

Even though it is preferable to base the dataset on more than one source in order to increase the amount of information and diminish the influence of potential bias (cf. Bowman et al. 2005), resource matters (time, etc.) meant that the State Department’s country reports were selected as the only source. One implication of this choice was that the dataset does not go further back in time than 1977 for most Latin American countries8 and 1979 for all communist countries except Yugoslavia (1977). The possi-bility for other scholars to scrutinize the coding through replication, though, is rather high due to the easy access to the main source, of which the relevant parts make up their own subsections.

In transforming the information into the scores constituting the dataset, at least two independent coders assigned scores to all the country-years. At the time of the coding process, one of the coders, the present author, was a Ph.D. scholar in political science

6 The amount of information included in the reports has significantly increased since the be-ginning, so they now range from approximately 10 to 100 pages for each country with a ten-dency to be more comprehensive for large and/or trouble-ridden countries. Also, the number of issues assessed has increased during the years, but the five topics of interest have always been covered by the reports.

7 This is probably the main reason why Cingranelli and Richards (2004) only use the Amnesty International reports as sources for the coding of personal integrity rights and not of personal exertion (empowerment) rights included in their CIRI Human Rights Dataset.

8 The first year of assessment for Argentina, Brazil, Chile, and Cuba is 1979.

The Civil Liberty Dataset: Conceptualization and Measurement 11

specialized in comparative methodology and regime development in Latin American and post-communist countries. I assessed all the country-years in question, whereas the second assignment was split equally between two trained master’s students (political science) with a broad knowledge of methodological issues and the countries considered. Deviations were settled through subsequent discussions. In the event that this procedu-re did not result in agreement, the plan was to authorize the third coder with the final judgment. This procedure, however, was not applied because consensus was eventually reached with regard to all country-years and items, that is, 4,930 scores.

Inter-coder reliability, meaning the extent to which independent coders agree in their evaluation, is a necessary although not sufficient criterion for validity, and without it, the uncertainty of the results obtained using the dataset increases tremendously. The most simple and widespread measure of inter-coder reliability is to calculate the per-centage of agreement. However, in the methodological literature it is seen as mislea-ding and inappropriately liberal, so a measure known to be rather conservative, namely Cohen’s Kappa, is also employed. As the principal source for some fairly similar CIRI indicators9 is the same (the Country Report on Human Rights Practices), the correlation (Gamma) between these and the civil liberty indicators is another means of assessing the reliability. The results of these tests, separate for each variable and region, are shown in table 1.

The values indicate that the inter-coder reliability of the Civil Liberty Dataset is acceptable. The Cohen’s Kappa test coefficients almost all exceed 0.80, while the coeffi-cients for the Latin American data are all lower than their post-communist counterparts. A likely explanation for this difference is that the scores referring to Latin American countries to a larger degree reflect the coding of less informative older reports, while it has simply been easier to evaluate the almost unexceptionally harsh repression level of

9 The indicators are: Independent Courts, Freedom of Speech and Press, Freedom of Assembly and Association, Freedom of Religion, and Freedom of Movement.

Table �:   Inter-Rater Reliability StatisticsLatin American Post-Communist

Cohen’s Kappa

Percentage Agreement

Correlation with CIRI Indicator

Cohen’s Kappa

Percentage Agreement

Correlation with CIRI Indicator

Independence of Courts 0.803 0.876 0.807 0.845 0.892 0.860Freedom of Opinion and Expression

0.738 0.855 0.891 0.823 0.877 0.924

Freedom of Assembly and Association

0.773 0.850 0.933 0.861 0.903 0.949

Freedom of Thought, Conscience, and Religion

0.811 0.914 0.921 0.817 0.881 0.894

Freedom of Movement and Residence

0.807 0.887 0.925 0.856 0.901 0.906

12 S.-E. Skaaning

communist regimes.10 Additionally, in the Latin American context it has frequently been uncertain whether violations have been committed by government agents or others and whether the actions taken against citizens have been based on legitimate reasons. The values linked to two items, freedom of opinion and expression and freedom of assembly and association, even fall below the suggested benchmark. Yet, as the levels are still rather high and considering the well-known conservativeness of the measure, the inter-rater reliability is found to be appropriate. This judgment is also supported by the high correlations with the CIRI indicators and the fact that none of the differences among coding scores was more than one point. The somewhat lower correlation between the indicators reflecting the independence of courts is probably due to a higher degree of discrepancy in the respective definitions.

Aggregation

The disaggregated data are, together with the codebook, available on the project web-site (www.democracy-assessment.dk), allowing researchers and others interested in civil liberty issues to scrutinize the assessments and to use and them as they please. Instead of concluding the presentation of the new dataset now, I will continue with the construc-tion of an index. The reason is that, despite the loss of information, we often use composite measures to climb up the ladder of abstraction and/or to minimize problems with multicollinearity, degrees of freedom, etc. Before combining the indicators and constructing an overall index, however, the dimensionality of the dataset is examined in order to uncover whether the indicators not only conceptually but also empirically reflect a common latent dimension, so an aggregation does not force a multidimensi-onal phenomenon into one composite measure. The coefficients in table 2 show the correlation (Gamma) between the component items, where the coefficients related to Latin American and post-communist countries are found above and below the diagonal, respectively.

With regard to the post-communist data, all coefficients exceed 0.84, indicating strong unidimensionality. Actually, the level of some of the correlations, for example between the indicators for the independence of courts and freedom of expression, are so high that their scores are virtually equivalent. The results for the Latin American data are more mixed. The coefficients differ more – ranging from 0.467 to 0.871 – and are generally lower, but then again, on a relatively high level. The covariation between independent courts and freedom of movement to some extent constitutes an exception.

The items appear to tap into the same dimension and to support this claim further. Examinations of the dataset were carried out by calculating the item-total11 correlations (Gamma) and Cronbach’s alpha in addition to running a factor analysis (principal com-ponent) (see table 3). With regard to the post-communist countries, the items tend to be extremely unidimensional. The scope of the item-total correlations is 0.825-0.955;

10 The latter reason finds more support in year by year calculations of the inter-rater reliability (Kappa) than the former.

11 A tentative index constructed through simple addition of the items except the item in question.

The Civil Liberty Dataset: Conceptualization and Measurement 13

Cronbach’s alpha value is 0.937 and the removal of any item would reduce it. Further-more, the factor analysis only extracts one dimension12 and the factor loadings of the items are very high.

Analyses of the Latin American data also support that the items reflect the same underlying phenomenon in that the factor analysis only extracts one component, and Cronbach’s alpha is as high as 0.859. The overall picture for the Latin American data, though, is a bit more blurred as illustrated by the generally lower item-total correla-tions and factor loadings. Two items, namely independence of courts and freedom of thought, conscience, and religion, generally appear less associated with the latent di-mension. Nevertheless, they are used in the index construction because the items ensure analytically crucial and interesting variation (especially in the Latin American context) and because the conceptual analysis underscored their coherence with the other items, which underlined the implausibility of any exclusion. With an alpha value of 0.915 and high correlations and factor loadings, similar tests including all countries confirm the region-specific findings.

In order to explore whether the high correlations between the items are artificially boosted by a high degree of stability across the years and countries in question, as was

12 Post-communist: Principal component. The eigenvalue of the first component is 4.011, ex-plaining 80.2% of the variation, whereas the eigenvalue of the second component is only 0.343, explaining less than 7% of the variation. Latin American: Principal component. The eigenvalue of the first component is 3.232, explaining 64.6% of the variation, whereas the eigenvalue of the second component is only 0.666, not even explaining 14% of the variation. All: Principal component. The eigenvalue of the first component is 3.745, explaining 75% of the variation, whereas the eigenvalue of the second component is 0.487, explaining less than 10% of the variation.

Table 2:   Dimensionality Statistics – Item-Item CorrelationsIndependence 

of CourtsFreedom of Opinion and Expression

Freedom of Assembly and 

Association

Freedom of Thought, 

Conscience, and Religion

Freedom of Movement 

and Residence

Independence of Courts

0.802 0.656 0.584 0.467

Freedom of Opinion and Expression

0.974 0.871 0.659 0.722

Freedom of Assembly and Association

0.927 0.939 0.553 0.741

Freedom of Thought, Conscience, and Religion

0.884 0.916 0.910 0.610

Freedom of Movement and Residence

0.847 0.904 0.851 0.848

14 S.-E. Skaaning

seemingly the case under communism, the factor loadings (principal component) for each year since 1979 have been calculated. The results for 2003 and every fifth year back to 1983 shown in table 4, however, indicate that this is not a critical issue.

The dimensionality statistics also show that freedom of opinion and expression and, modestly less, freedom of assembly and association seem to be the best individual predictors of the level of civil liberty. This finding is interesting because, besides being defining attributes of the civil liberty concept, the freedoms are also constitutive parts of many democracy definitions (cf. figure 2 above); a high covariation between the respect for democratic and civil liberties (and, to some extent, similar causes and effects), at least in the countries and the period under consideration, is therefore to be expected.

Concerning the actual aggregation rule, simple addition (equals arithmetic mean) is preferred because the component items are considered to have the same weight. Depen-ding on the context, some people find some of the civil liberties in focus more important, while others would come to a different order of priority. In political theory/philosophy,

Table 4:   Factor Loadings in Selected YearsLatin American Post-Communist

Year Inde-pen-

dence of

Courts

Freedom of Opi-

nion and Expres-

sion

Free-dom of Assem-bly and Associa-

tion

Free-dom of

Thought, Consci-

ence, and Religion

Free-dom of Move-ment and

Resi-dence

Inde-pen-

dence of

Courts

Free-dom of Opinion

and Expres-

sion

Free-dom of Assem-bly and Associa-

tion

Free-dom of

Thought, Consci-

ence, and Religion

Free-dom of Move-ment and

Resi-dence

1983 0.867 0.942 0.906 0.703 0.830 0.830 0.937 0.681 0.551 0.7551988 0.762 0.909 0.913 0.775 0.815 0.975 0.975 0.856 0.637 0.9451993 0.713 0.902 0.833 0.653 0.698 0.870 0.929 0.859 0.651 0.8131998 0.614 0.803 0.792 0.810 0.852 0.908 0.859 0.866 0.837 0.7692003 0.661 0.858 0.900 0.770 0.773 0.915 0.901 0.838 0.887 0.809

Table 3:   Dimensionality Statistics ContinuedIndepen-dence of Courts

Freedom of Opini-on and 

Expression

Freedom of Assem-bly and 

Association

Freedom of Thought, Consci-

ence, and Religion

Freedom of Move-ment and Residence

Latin American

Item-Total 0.584 0.824 0.737 0.561 0.617Cronbach’s Alpha 0.858 0.804 0.808 0.845 0.827Factor Loadings 0.705 0.875 0.863 0.752 0.812

Post- Communist

Item-Total 0.916 0.955 0.904 0.875 0.825Cronbach’s Alpha 0.921 0.912 0.918 0.928 0.930Factor Loadings 0.896 0.930 0.912 0.875 0.863

AllItem-Total 0.756 0.913 0.819 0.753 0.761Cronbach’s Alpha 0.911 0.880 0.887 0.901 0.897Factor Loadings 0.806 0.918 0.896 0.844 0.859

Note: Cronbach’s Alpha values if item deleted.

The Civil Liberty Dataset: Conceptualization and Measurement 15

the prevailing tradition is also not to weight or prioritize among them. Furthermore, the liberties are not assumed to be mutually constitutive. This means that if one of them is given a low value, it does not necessarily affect the interpretation of another item. In order to obtain full membership in the group of liberal regimes, however, all civil liberties must be fully observed. Moreover, all the items have end points, thus limiting the possibility of compensation/substitution in significant and reasonable ways.

The robustness of the aggregation rule is examined by comparing (correlating) the Civil Liberty Index (CLI) with six different ways of combining the five items. Among the alternatives are the geometric mean, which is relevant when several quantities mul-tiply together to produce a product, and the factor scores (principal components) lin-ked to the dimensionality assessment. Yet another procedure, multiplication of the item values, is suitable if each component item must have a certain minimum to make the other affiliated aspects meaningful.13 A related manner of reasoning applies to the use of the minimum value as index score. In this case, the logic of necessity is used on all four item levels (scale points), whereas – in contrast – use of the maximum value corresponds to identifying all items as individually sufficient. The final option is a weighted simple average, where the scores for freedom of opinion and expression and freedom of assembly and association are given double weight because they may be jud-ged more important due to their twofold function as both civil and democratic liberties. The correlation coefficients (Pearson) are shown in table 5.

The results indicate that it can make a considerable difference which aggregation rule is chosen. The three alternatives connected to necessity or sufficiency argumenta-tion lead to coefficients below 0.9 but not lower than 0.7. As it is rather obvious that none of the individual items are sufficient for civil liberty, the aggregation procedure of equalizing the index score with the maximum score is an implausible route to follow. If the main concept was constituted by two or more mutually constitutive dimensions, it would (more likely) be proper – indeed, maybe even indispensable – to regard different elements as necessary and thus try to adjust the composite measure by using some sort of multiplication or minimum procedure in the aggregation process. According to Gary Goertz (2006), the minimum procedure is probably the proper way to operationalize our concepts in most cases.

However, although his arguments are certainly analytically appealing, I do not regard the civil liberty aspects to be necessary in the sense of being fully mutually constitutive. In my view, Goertz’s point is more relevant in relation to participation and competition

13 The items were rescaled (0-3) in order to support this logic.

Table 5:   Correlations between CLI and Indices Based on other Aggregation Methods or Data

Geo-metric Mean

Multi-plication

Factor Scores

Mini-mum

Maxi-mum

Weighted CIRI FH

Latin American 0.995 0.727 0.999 0.760 0.807 0.992 0.841 -0.835Post-Communist 0.999 0.750 1.000 0.912 0.885 0.997 0.838 -0.924All 0.997 0.734 1.000 0.835 0.874 0.995 0.856 -0.887

16 S.-E. Skaaning

in the case of democracy, for example (cf. Bowman et al. 2005). Whereas it does not make much sense in terms of democraticness to have inclusive participation without contestation, as was to some extent the case under communist regimes, citizens’ free-dom of movement still makes sense without freedom of expression. Furthermore, when measuring the overall respect for civil liberty, we would lose more information through the application of the minimum procedure as only four scores would be viable. It also seems rather implausible to equalize the level of civil liberty in a country assigned only one low score (e.g., 1, 4, 4, 4, 4) to the level in a country assigned low scores on all items (e.g., 1, 1, 1, 1, 1), and even concerning the measurement of democracy, applica-tion of the minimum rule has forcefully been questioned (Marsteintredet 2007). Finally, if the main concept only covers one dimension, as theoretically and empirically justified with regard to civil liberty, it makes less sense to apply alternative procedures. As it makes virtually no difference if the arithmetic mean is replaced by the geometric mean, factor scores, or the weighted index, and because it is easy to interpret and handle, I thus prefer simple addition as the principal rule of aggregation.

Claims for the validity of measures are commonly based on their high statistical correlations with other measures of the same phenomenon. Yet highly correlated mea-sures may all contain the same errors, share similar biases, or be determined by outside influences that may render their close associations spurious (cf. Forewaker and Krznaric 2001: 5; Munck and Verkuilen 2002: 29). The same points apply to uncritical use of cor-relations between different indicators expected to reflect the concept in question. Simple correlations are not able to capture all kinds of (theoretical/empirical) relationships and high correlations can be the result of a tendency of clustering, when cases tend to be assigned very high and very low scores and while crucial disagreements exist for cases in the ‘gray zone’ of intermediate scores (Goertz 2006). Basically, strong theoretical justifications in every step of construction as well as reliable empirical information are obviously the most essential parts of sound datasets and measures, but in many cases different factors force us into trade-off situations distancing the results from the ideal. Thus, researchers should refrain from interpreting such correlations as much more than indications of reliability – to the extent that these measures build on the same sources – and as indications of whether the results would be significantly different if the mea-sures were interchanged in statistical models.

It is interesting, though, that the CLI, in spite of the differences, is highly correla-ted (Pearson) with rival measures such as Freedom House’s (FH) (2006) Civil Liberty Rating14 and Cingranelli and Richards’s (CIRI) (2006) Empowerment Rights Index15, also shown in table 5. The strong correlations in the range of 0.83 and 0.9216 indicate

14 A composite index based on assessments of freedom of expression and belief, associational and organizational rights, rule of law, and personal autonomy and individual rights.

15 A composite index based on assessments of freedom of speech, freedom of religion, freedom of movement, freedom of political participation, and workers’ rights.

16 The yearly correlations with the Freedom House index is fairly stable in both regions, while the correlations with the CIRI index are stable across the assessed years for the Latin Ameri-can countries but comparatively lower in the early 1980s and the early 1990s for the (post-) communist countries.

The Civil Liberty Dataset: Conceptualization and Measurement 17

that the measures very much tap into the same latent dimension. They are not fully interchangeable, though, so the application of the CLI in empirical analyses as a depen-dent and explanatory variable would lead to somewhat different results, just as it has been shown to be the case with democracy indices (Hadenius and Teorell 2004; Bollen and Paxton 2000; Casper and Tufis 2003). The notably lower correlation between the measures concerning Latin American countries is probably caused by differences in constituent parts of the indices/scales. In other words, these differences have a greater effect in this setting due to the lower extent of unidimensionality compared with the post-communist countries, which are very much characterized by a tendency that all good (and, inversely, bad) features coincide.

Limitations on the agenda and scope of this article means that an inquiry into the explanatory consequences of the new dataset are addressed in detail elsewhere, where the CLD and the CLI – as well as some of their alternatives – are employed to track and explain the development in civil liberties (Skaaning 2006c). Nonetheless, to give an impression of whether it would make a difference if the minimum scores, the CIRI Empowerment Rights Index, or the Freedom House Civil Liberties Rating were used in the place of the CLI, the respective standardized scores (0-100) for 2003 of all the assessed countries are shown in table 6.

Without going into a detailed discussion of the scores, it is obvious from the table that the employment of the minimum aggregation rule implies a loss of information and leads to a quite different ranking and score of the countries than the CLI. The procedure means that the cases can only achieve the scores 0, 33, 66, and 100 and tend to cluster in the two middle categories. Although the resulting groups are somewhat reasonable, the set of countries with the lowest score (i.e., Cuba, Haiti, Belarus, Kazakhstan, Kyr-gyzstan, Tajikistan, Turkmenistan, and Uzbekistan) tends to be more homogeneous than the rest.

As for the other measures, they tend to agree more. As we would anticipate, the Central Asian and Caucasian countries show a poor record, and the new EU members outperform the others. Also not very surprising, Costa Rica and the Southern Cone countries (Chile, Argentina, and Uruguay) are among the best, while Cuba and Haiti have much to improve. Noteworthy differences do exist, though. For example, Free-dom House assesses the respect for civil liberty in Haiti much lower than the CLI and the CIRI measure. This dissimilarity could, however, be the result of Freedom House focusing on all civil liberty violations and not only the ones committed by government agents. Therefore, it is more disturbing that in four cases (Chile, Peru, Armenia, and the Ukraine) the CIRI index score is more than 30 points lower than the corresponding va-lues of the Freedom House measure and the CLI. It is also puzzling that Turkmenistan, with an exceptionally repressive regime, achieves the same score in the CIRI index as Russia and even better than Armenia, Azerbaijan, and the Ukraine. Furthermore, the case of Belarus illustrates the problems connected to a dichotomous coding of freedom of religion and freedom of movement, since it is assigned the lowest possible combined score even though these civil liberties are not severely repressed.

To illustrate, in brief, that the CLI not only outperforms the alternatives on the de-scriptive level but also provides higher criterion validity, I examine the relationship between the measures and an operationalization of the most persuasive theoretical

18 S.-E. Skaaning

account for civil liberty in the post-communist countries, namely the communist legacy (Kitschelt 2003; cf. Skaaning 2006c). Following Herbert Kitschelt (2001), I distinguish between different kinds of legacies, ranging from bureaucratic-authoritarian to highly patrimonial types, differentiated by the Weberian ‘bureaucraticness’ of the state as well as the power balance between communists and the (liberal, oppositional) civil society. These features are traceable to the pre-communist era and were jointly expected to sha-pe the regime trajectories after the communist breakdowns in 1989-1991. Supporting the construction of a new measure on civil liberty, Kitschelt’s assessment of the vital (pre-)communist legacies shows a significantly higher correlation with the CLI than the indices from CIRI and Freedom House – both in the short term (1993: 0.68, 0.60, and 0.61, respectively) and intermediate term (2003: 0.77, 0.69, and 0.70, respectively).

Table 6:   Comparison of Index Scores (2003)CLI Mini-

mumCIRI FH CLI Mini-

mumCIRI FH

Argentina 80 33 100 83 Bosnia-Her. 53 33 40 50Bolivia 73 33 70 67 Bulgaria 73 67 60 83Brazil 73 33 90 67 Croatia 80 67 80 83Chile 93 67 60 100 Czech Rep. 100 100 80 83Colombia 60 33 50 50 Estonia 93 67 100 83Costa Rica 93 67 80 83 Georgia 53 33 50 50Cuba 7 0 0 0 Hungary 80 67 90 83Dom. Rep. 80 67 60 83 Kazakhstan 40 0 30 33Ecuador 67 33 80 67 Kyrgyzstan 40 0 50 33El Salvador 73 33 80 67 Latvia 73 67 80 83Guatemala 67 33 50 50 Lithuania 80 67 90 83Haiti 53 0 50 17 Macedonia 67 67 80 67Honduras 67 33 70 67 Moldova 53 33 50 67Mexico 73 33 50 83 Mongolia 73 67 60 83Nicaragua 80 33 80 67 Poland 87 67 70 83Panama 80 33 70 83 Romania 67 33 50 83Paraguay 67 33 80 67 Russia 40 33 20 33Peru 60 33 30 67 Serbia-Mon. 60 33 80 83Uruguay 93 67 80 100 Slovakia 73 67 100 83Venezuela 60 33 60 50 Slovenia 87 67 90 100Albania 60 33 80 67 Tajikistan 33 0 30 33Armenia 60 33 10 50 Turkmenistan 13 0 20 0Azerbaijan 47 33 10 33 Ukraine 53 33 10 50Belarus 20 0 0 17 Uzbekistan 27 0 0 17Note: All index scores are rescaled to a range of 0-100, 100 designating the highest level of civil liberty.

The Civil Liberty Dataset: Conceptualization and Measurement 19

Conclusion

Recapitulating the limitations of alternative measures already addressed in the intro-duction, these concerns support the construction and use of the Civil Liberty Dataset and the Civil Liberty Index presented in this article, which demonstrated the decisions made in order to provide for improved measurement of civil liberty. The prime goal was to develop a dataset that, on the one hand, is able to identify the level of respect for civil liberties and, on the other hand, will be as reliable and valid as possible given the available resources. In this way, it can serve as an effective tool in the comparison of individual countries across time and in the comparison of several countries across space and/or time, either on an intraregional or interregional basis. Whether this goal has been achieved is – and will remain – an open question, but the decisions have been followed by explicit justifications. The cards are placed on the table, and other scholars now have a chance to scrutinize the generative procedures.

Once again, it is worth stressing that the index is the first measure which exclusi-vely focuses on civil liberties understood as personal exertion freedoms. It is different from, and thus supplements, previous attempts to measure respect for democracy and personal integrity, respectively. It is an easily reproducible measure of a political regime aspect: specifically, the upside-down relationship between the rulers and the ruled, and it builds upon widely shared agreement in liberal theory about fundamental liberties that people should have the freedom to perform without facing obstructions from the public authorities.

The design of this dataset has made it possible for scholars to use the scores on the disaggregate items as they see fit and/or to apply the proposed index. Covering 20 La-tin American and 28 post-communist countries on an annual basis from the end 1970s to 2003, the scope of the dataset is relatively comprehensive. It fits the study it was initially designed to support, that is, an inquiry into the (dis)respect for civil liberty in Latin American and post-communist countries after the third wave of democratization. On the other hand, this agenda also involves a limitation of the number of countries and years covered, so the utility of the dataset for studies on civil liberty dealing with other research questions may be restricted – at least until a possible extension is embarked upon. As the data relate to a rather high level of abstraction, it bears mentioning that extra contextual and specific information is needed if variations of sub-national level or detailed accounts for cases are in focus. Nevertheless, the CLD is found to have several advantages over existing datasets and it is the latest – in some regards even the first, but hopefully not the last – to address a crucial topic, which has not received much independent attention in political science.

References

Beetham, David, Sarah Bracking, Iain Kearton and Sturt Weir. 2001. International IDEA Demo-cracy Assessment Handbook. The Hague: Kluwer.

Berlin, Isaiah. 1997. Two Concepts of Liberty. In: Isaiah Berlin. The Proper Study of Mankind: An Anthology of Essays. London: Chatto & Windus. 191-242.

20 S.-E. Skaaning

Bobbio, Noberto. 1990. Liberalism and Democracy. London: Verso.Böckenförde, Ernst-Wolfgang. 1991. Staat, Verfassung, Demokratie. Frankfurt a.M.: Suhrkamp.Bollen, Kenneth. 1993. Liberal Democracy: Validity and Method Factors in Cross-National Mea-

sures. In: American Journal of Political Science (37). 1207-1230.Bollen, Kenneth and Pamela Paxton. 2000. Subjective Measures of Political Democracy. Compa-

rative Political Studies (33). 58-86.Bowman, Kirk, Fabrice Lehoucq and James Mahoney. 2005. Measuring Political Democracy:

Case Expertise, Data Adequacy, and Central America. Comparative Political Studies (38). 939-970.

Bunce, Valerie. 1995a. Should Transitologists be Grounded? Slavic Review (54). 111-127. Bunce, Valerie. 1995b. Paper Curtains and Paper Tigers. Slavic Review (54). 979-987. Case, William. 1996. Can the ‘Halfway House’ Stand? Semidemocracy and Elite Theory in Three

Southeast Asian Countries. Comparative Politics (28). 437-464.Casper, Gretchen and Claudio Tufis. 2003. Correlation versus Interchangeability: The Limited

Robustness of Empirical Findings on Democracy using Highly Correlated Datasets. Political Analysis (11). 196-203.

Claude, Richard P. and Thomas B. Jabine. 1992. Exploring Human Rights Issues with Statistics. In: Thomas B. Jabine and Richard P. Claude (eds.). Human Rights and Statistics: Getting the Record Straight, Philadelphia: University of Pennsylvania Press. 5-34.

Cingranelli, David L. and David L. Richards. 2004. Cingranelli-Richards (CIRI) Human Rights Database Coder Manual, http://ciri.binghamton.edu/documentation/web_version_7_31_04_ciri_coding_guide.pdf (15.02.2006).

Cingranelli, David L. and David L. Richards. 2006. Cingranelli-Richards (CIRI) Human Rights Database, http://ciri.binghamton.edu/ (04.02.2006).

Collier, David and Steven Levitsky. 1997. Democracy with Adjectives: Conceptual Innovation in Comparative Research. World Politics (49). 430-451.

Constant, Benjamin. 1988. The Liberty of the Ancients Compared with that of the Moderns. In: Benjamin Constant. Political Writings. Cambridge: Cambridge University Press. 309-328.

Dahl, Robert. 1971. Polyarchy: Partizipation and Opposition. New Haven: Yale University Press.

Diamond, Larry. 2002. Thinking About Hybrid Regimes. Journal of Democracy (13). 21-35.Foweraker, Joe and Roman Krznaric. 2001. How to Construct a Database of Liberal Democratic

Performance. Democratization (8). 1-25.Freedom House. 2004. Methodology, http://www.freedomhouse.org/template.cfm?page=35

&year=2004 (12.12.2004).Freedom House. 2006. Freedom in the World, http://www.freedomhouse.org (04.04.2006).Goertz, Gary. 2006. Social Science Concepts: A User’s Guide. Princeton: Princeton University

Press.Green, Maria. 2001. What We Talk About When We Talk About Indicators: Current Approaches

to Human Rights Measurement. Human Rights Quarterly (23). 1062-1097.Habermas, Jürgen. 1992. Faktizität und Geltung. Frankfurt a.M.: Suhrkamp.Habermas, Jürgen. 1996. Die Einbeziehung des Anderen. Frankfurt a.M.: Suhrkamp.Hadenius, Axel and Jan Teorell. 2004. Same, Same – But Different: Assessing Alternative Indi-

ces of Democracy. Paper presented at the APSA Annual Meeting, Chicago, September 2-5, 2004.

Innes, Judith. 1992. Human Rights Reporting as a Policy Tool: An Examination of the State De-partment Country Reports. In: T. B. Jabine and R. P. Claude (eds.). Human Rights and Stati-stics: Getting the Record Straight. Philadelphia: University of Pennsylvania Press. 235-257.

Karl, Terry. 1995. The Hybrid Regimes of Central America. Journal of Democracy (6). 72-86.Karl, Terry L. and Philippe C. Schmitter. 1995. From an Iron Curtain to a Paper Curtain: Groun-

ding Transitologists or Students of Postcommunism? Slavic Review (54). 965-978.

The Civil Liberty Dataset: Conceptualization and Measurement 21

King, Gary, Robert Keohane and Sidney Verba. 1994. Designing Social Inquiry: Scientific In-ference in Qualitative Research. Princeton: Princeton University Press.

Kitschelt, Herbert. 2001. Post-Communist Economic Reform: Causal Mechanisms and Conco-mitant Properties. Paper presented at the APSA Annual Meeting, San Francisco, August 29 – September 2, 2001.

Kitschelt, Herbert. 2003. Accounting for Postcommunist Regime Diversity: What Counts as a Good Cause? In: Grzegorz Ekiert and Stephen E. Hanson (eds.). Capitalism and Democracy in Central and Eastern Europe: Assessing the Legacy of Communist Rule. Cambridge: Cam-bridge University Press. 49-86.

Lauth, Hans-Joachim. 2004. Demokratie und Demokratiemessung: Eine konzeptionelle Grundle-gung für den interkulturellen Vergleich. Wiesbaden: Verlag für Sozialwissenschaften.

Levitsky, Steven and Lucan Way. 2002. Competitive Authoritarianism. Journal of Democracy (13). 51-65.

Linz, Juan. 2000. Totalitarian and Authoritarian Regimes. Boulder: Lynne Rienner Publisher.Marsteintredet, Leiv (2007). Graded Conceptual Membership: Linking Radial Concepts and Fuz-

zy Sets in the Study of Democracy. The Committee on Concepts and Methods: Political Concepts Working Paper (19).

McNitt, Andrew. 1988. Some Thoughts on the Systematic Measurement of the Abuse of Human Rights. In: D. L. Cingranelli (ed.). Human Rights: Theory and Measurement. Houndmills: Macmillan. 89-103.

Merkel, Wolfgang. 1999. Systemtransformation: Eine Einführung in die Theorie und Empirie der Transformationsforschung. Opladen: Leske + Budrich.

Merkel, Wolfgang and Aurel Crossaint. 2000. Formale und Informale Institutionen in defekten Demokratien. Politische Vierteljahresschrift (41). 3-31.

Merkel, Wolfgang, Hans-Jürgen Puhle, Aurel Crossiant, Claudia Eicher and Peter Thiery. 2003. Defekte Demokratie. Band 1: Theorie. Opladen: Leske + Budrich.

Miller, David. 1991. Introduction. In: David Miller (ed.). Liberty. Oxford: Oxford University Press. 1-20.

Munck, Gerardo and Jay Verkuilen. 2002. Conceptualizing and Measuring Democracy: Evalua-ting Alternative Indices. Comparative Political Studies (35). 5-34.

Munck, Gerardo and Jay Verkuilen. 2003. Bringing Measurement Back In: Methodological Foun-dations of the Electoral Democracy Index. Paper Presented at the APSA Annual Meeting, Philadelphia, August 28-31, 2003.

Neumann, Franz. 1957. The Democratic and Authoritarian State: Essays in Political and Legal Theory. Glencoe: Free Press.

Ottaway, Marina. 2003. Democracy Challenged: The Rise of Semi-Authoritarianism. Washington: Carnegie Endowment for International Peace.

Poe, Steven, Sabine Carey and Tanya Vazquez. 2001. How are These Pictures Different? A Quan-titative Comparison of the US State Department and Amnesty International Human Rights Reports, 1976-1995. Human Rights Quarterly (23). 650-677.

Rawls, John. 1971. A Theory of Justice. Oxford: Oxford University Press.Saward, Michael. 1994. Democratic Theory and Indices of Democratization. In: D. Beetham

(ed.). Defining and Measuring Democracy. London: Sage. 6-24.Schedler, Andreas. 2002. The Menu of Manipulation. Journal of Democracy (13). 36-50.Schmitter Philippe C. and Terry L. Karl. 1994. The Conceptual Travels of Transitologists and

Consodologists: How Far to the East Should They Attempt to Go? Slavic Review (53). 173-185.

Skaaning, Svend-Erik. 2006a. Measuring Civil Liberty. The Committee on Concept and Methods: Political Concepts Working Paper 8, CIDE.

Skaaning, Svend-Erik. 2006b. Defining and Founding Civil Liberty. CDDRL Working Paper 56, Stanford University.

22 S.-E. Skaaning

Skaaning, Svend-Erik. 2006c. Democracy besides Elections: An Inquiry into the (Dis)Respect for Civil Liberty in Latin American and Post-Communist Countries after the Third Wave. Aarhus: Department of Political Science, University of Aarhus.

Umozurike, U. O. 1998. Human Rights and Development. International Social Science Journal (158). 535-543.

Waldron, Jeremy. 2003. Security and Liberty: The Image of Balance. Journal of Political Philo-sophy (11). 191-210.

Zakaria, Fareed. 1997. The Rise of Illiberal Democracy. Foreign Affairs (76). 22-43.

Appendix: Extract from the CLD Codebook

Freedom of Thought, Conscience, and Religion (frerel)

Definition

The component specifies the extent to which individuals and groups have freedom of thought, conscience, and religion, that is, the right of the citizens to have and change religion or belief of own choice and alone or in community manifest their religion or belief in practice, worship, observance, and teaching in private or public as well as proselytize peacefully without being subject to actual limitations or restrictions.

Component Scale

(1) Severely restricted. Hardly any freedom of religion exists. As a rule, any kind of religious practice is controlled by the government and harshly suppressed.

(2) Fairly restricted. Some elements of autonomous organized religious practices exist and are officially recognized, but major religious directions are repressed, prohibi-ted, or systematically disabled.

(3) Modestly restricted. There are minor restraints on the freedom of religion, predo-minantly limited to a few isolated cases, but as a rule, there are no interventions or prohibitions on communities or individual worshippers.

(4) Unrestricted. Unhampered freedom of religion exists.

Qualifying Violations

Ad 1) Religious activity is directly prohibited by government policy or is severely re-stricted for all major religions. In some instances, penalties on persons who engage in religious practices exist as well as educational campaigns against religion and obliging citizens to expose believers. Religious leaders are appointed by and subjected to public authorities, who control the activities of any religious direction in detail. Open expres-sion of certain religious belief or any religious belief in general is incompatible with membership in the ruling party or attainment of influential public positions.

Ad 2) The government discourages religious beliefs in general or harasses some of the major religious directions in the country, but traditionally established religions are

The Civil Liberty Dataset: Conceptualization and Measurement 23

recognized and religious practice is generally tolerated, although kept under surveil-lance by the public authorities. People openly expressing traditional religious beliefs of certain sorts are discriminated and/or intimidated by public authorities. Sometimes members of several minor religious groups are not allowed to teach or practice their re-ligion or face severe restrictions concerning registration, places of worship, restrictions on voluntary conversion, etc.

Ad 3) The government places minor or few restrictions on the rights of religious groups such as discrimination against a few minority religions in terms of denial of registration, hindrance of foreign missionaries to enter the country, not allowing citi-zens or foreigners to proselyte or hindering the access to or construction of places of worship. Few instances of discrimination and/or intimidation of individuals or groups due to their (often non-traditional) religion are carried out by public officials.

Ad 4) Citizens enjoy the right to practice any religious belief they choose. Religious groups may organize, select, and train personnel; solicit and receive contributions; pub-lish works; and engage in consultations without government interference. There are no government restrictions on establishing and maintaining places of worship. All religious groups may worship freely and uphold contact with their coreligionists abroad. Active missionary presence is not restrained. The redistribution of former places of worship after earlier confiscation is not considered in the assessment. Whether the clergy is able to freely advocate partisan political views, oppose government laws, support political candidates, and otherwise freely participate in politics is only taken into consideration as far as restrictions and repression by public authorities is attributable to religious rather than direct political discrimination. It does not count as a restriction if religious communities must register, if the public authorities routinely grant registration and do not abuse the process to discriminate against a religion, and if the government does not constrain the right to worship before registration.