[author]Yao Jie，Sun Tao

[content]

Replication of Conjoint Experiments on Legal Deterrence: Using the Drunk Driving Legislation in China as an Example

*Author

Yao Jie：Associate Professor, School of Humanities and Social Sciences, Harbin Institute of Technology (Shenzhen)

Sun Tao: Professor, School of Humanities and Social Sciences, Harbin Institute of Technology (Shenzhen)

Abstract: Effective legislation and enforcement in the area of road traffic safety has proved to reduce traffic injuries and fatalities in developed countries, yet such research has been limited in China, using driving-under-the-influence(DUl) as an example. Due to lack of high-quality data and primary research on driver decisions, the deterrent effects of DUl laws in China remain unclear, despite the criminalization of drunk driving since May 2011 and associated enforcement efforts by the government. Based on the classical theory of deterrence and recent advances in differential deterrence, this study as the third one in a series on legal deterrence, replicated a scenario-based conjoint experiment and established Hierarchical Bayes models for quantifying the unique effects of perceived certainty, swiftness, and severity of DUl punishment, as well as detecting variations in risk perceptions among general Chinese drivers. This replication study supported the usefulness of conjoint experiments for examining DUl decisions among drivers from different countries and groups, and helped for expanding the external validity of differential deterrence perspectives. It also provided conceptual and methodological insights into conducting experimental research and replication for the advancements of general public policy research in China.

Key words: Conjoint Experiment ; Experiment Replication ; Hierarchical Bayes ; Differential Deterrence

1. Introduction

With the rapid growth of global motorization and urbanization, the importance of road traffic safety has become increasingly prominent. Each year, road traffic accidents lead to over 1.35 million deaths globally, with nearly 50 million people injured or disabled, and traffic incidents have become the leading cause of death among individuals aged 5–29. Compared to developed countries, the disparity between the surge in motor vehicles and the lagging road safety measures is more evident in emerging economies, resulting in 90% of global road traffic fatalities occurring in low- and middle-income countries. Faced with the heavy burden road traffic injuries impose on national economies and families, the World Health Organization launched two consecutive “Decade of Action for Road Safety” global initiatives (2011-2020, 2021-2030), aiming to promote measures and related research to effectively reduce road traffic injuries worldwide. Among these, implementing and enforcing effective road traffic safety laws to deter dangerous driving behaviors (such as speeding and driving under the influence) has been shown by numerous studies to be a critical approach in reducing road traffic casualties.

Using driving under the influence (DUI) as an example, several relevant laws were enacted in the United States during the 1980s and 1990s, with numerous empirical studies validating their effectiveness in significantly reducing the proportion of intoxicated drivers involved in fatal traffic accidents. These studies largely rely on the classical deterrence theory framework to explain the influence of legislation on unlawful behavior, wherein laws, during their creation and enforcement, are expected to produce a deterrent effect, which is influenced by the certainty, severity, and swiftness of the punishment. In recent years, senior Western scholars such as Piquero have proposed the concept of “differential deterrence,” which suggests that the deterrent effects of law vary distinctly among different individuals. Although deterrence theory has seen extensive application in DUI law research abroad, most existing studies adopt program evaluation research methods, which focus on the direct outcomes of law enforcement (e.g., reduction in alcohol-related traffic accidents) but cannot examine the micro-level deterrent processes that influence individual behavior—such as which factors affect personal decision-making and how to adjust legislative and enforcement details to improve the effectiveness of the law.

Additionally, as an important branch in the field of public administration, behavioral public administration has developed rapidly in recent years, accompanied by the rise of experimental methods. The number of experimental studies published in international journals has increased significantly over the past five years. Compared to traditional research methods in public administration, the primary advantage of experimental methods lies in their ability to perform causal inference to ensure the internal validity of research, while also aiding in micro-level analysis of government behavior processes and citizen experiences, thereby improving the accuracy and effectiveness of public decision-making. At the same time, the reproducibility of experiments and the extent to which findings can be generalized beyond the sample (i.e., external validity) serve to assess the reliability of research conclusions, making replication—a key method for validating and expanding theory—increasingly important. Overall, experimental research is still in its early stages in the Asia-Pacific region, with substantial room for theoretical exploration; conducting high-quality experimental studies and replications in this region, especially in China, is of great significance for guiding public administration research within the region and fostering comparative studies with other regions. This advancement supports the localization and Sinicization of international public administration theory.

Legislation on road traffic safety related to driving under the influence (DUI) has been a priority for the Chinese government over the past decade, with the criminalization of drunk driving in May 2011 receiving broad societal attention, and considerable government investment in enforcement has been widely recognized. This law has high public awareness and timely enforcement, having accumulated ten years of implementation, making it suitable for evaluating long-term effects and providing timely reference for future policy optimization. Unfortunately, due to the lack of publicly available primary traffic accident data in China, it is challenging to evaluate changes before and after the law’s implementation using standard statistical methods (e.g., interrupted time series analysis). Although official reports from the Ministry of Public Security indicated a significant reduction in alcohol-related traffic accidents and fatalities during the first three years following criminalization, causal inference cannot be drawn from this data alone, and results from surveys based on alternative data sources have been inconsistent. Moreover, few studies have examined how specific factors within the law deter drivers’ decision-making and risk perceptions; in the long term, further verification is needed to determine whether the effectiveness of this legislation can be confirmed and maintained. Therefore, taking China’s DUI legislation as an example, applying experimental methods to the field of legal deterrence represents an important supplement to existing research.

This paper will first provide an overview of experimental research methods and previous experimental studies based on legal deterrence theory, followed by an introduction to the original conjoint experiment and key replication points of this study (Section 2). Next, the paper will present the experimental design, data collection, and analytical methods used in this study (Section 3). Subsequently, the analysis results will be discussed, including sample characteristics, the conjoint analysis model, sensitivity analysis, and subgroup comparisons (Section 4). Finally, the paper will summarize the similarities and differences among general Chinese drivers, college students, and U.S. drivers, demonstrate the existence of differential deterrence, and elaborate on the applicability and importance of replicating and extending this conjoint experimental method in the different contexts of China and the U.S. (Section 5).

2. Literature Review

2.1 Experimental Research Methods

2.1.1Types of Experiments

Based on different research designs, the main types of experiments in public administration include lab experiments, survey experiments, and field experiments. Some scholars have also highlighted the existence of natural experiments and quasi-experiments, although these last two differ significantly from commonly used experimental methods. Among experimental studies published in international journals over the past five years and those published in the Asia-Pacific region in the past decade, survey experiments are the most widely used. This is due to their practical design advantages and the broader reach of online surveys, which also improves sample representativeness.

2.1.2 Conjoint Experiment Method

As a method rooted in conjoint analysis, the conjoint experiment can be viewed as a specific type of survey experiment that collects large-scale respondent data through embedded questionnaires. Conjoint analysis systematically combines multiple elements to simulate real-world scenarios, using statistical modeling to quantify the impact of each element; it is widely applied in marketing research. Its distinctive feature lies in using experimental design to scientifically construct a series of virtual products with different attributes, assessing respondents’ purchase intentions to predict market response for specific products. In the field of public policy research, conjoint experiments accurately estimate the unique effects of each policy element on public behavior by evaluating respondents’ decision-making responses in various hypothetical policy scenarios. Compared to traditional survey experiments in public administration, conjoint experiments allow for the simultaneous examination of more factors, providing a unique advantage in simulating complete scenarios (including competitive situations). This enables respondents to judge combinations of multiple factors rather than evaluating each factor in isolation, making the decision-making process closer to real life. The experimental data obtained thus reveal clearer distinctions in the effects of each factor and illuminate their interactions more transparently.

Due to its immense potential in analyzing public behavior decision-making and the effectiveness and specificity of public policies, conjoint experiments have become increasingly popular in public administration in recent years. In situations where policy effects cannot be assessed through natural or quasi-experiments (e.g., due to a lack of publicly available original official data or absence of pre- and post-policy implementation data, making it impossible to use interrupted time series analysis), conjoint experiments provide causal evidence from another perspective. Moreover, as a type of survey experiment, conjoint experiments reach a broader range of respondents than field and laboratory surveys. It should be noted that most existing conjoint experiment studies in public administration apply traditional statistical models to analyze results overall (i.e., based on the average of all subjects), with limited methods for addressing individual differences. In areas where individual characteristics significantly impact outcomes (such as differential deterrence in law), integrating the widely used Hierarchical Bayes (HB) method from marketing research could facilitate more precise statistical modeling of individual differences.

2.2 Experimental Research Based on Legal Deterrence Theory

2.2.1 Legal Deterrence and Differential Deterrence

As one of the most representative research frameworks in criminal justice and criminology, classical deterrence theory is widely applied to studies on how laws influence the psychological behavior associated with unlawful actions. The three core elements of this theory are the certainty, severity, and swiftness of legal punishment, which together shape the actual deterrent effect of laws: certainty relates to probability, meaning the risk of unlawful actions being detected and punished; severity refers to the intensity of the punishment; and swiftness pertains to the speed at which unlawful actions are penalized. In other words, if laws are not consistently enforced, punishment is not severe, and retribution is not timely, merely enacting a law is insufficient to prevent related behaviors. It should be noted that since deterrence is a perception-based phenomenon, individuals must perceive the threat of legal punishment for it to impact their behavior, according to classical social psychology theory.

Over the past decades, classical deterrence theory has been applied in numerous empirical studies, with one of the key findings in recent years being that the effectiveness of legal punishment depends on specific characteristics of the target population, which led to the concept of differential deterrence. Future research should shift focus from whether legal punishment can deter unlawful behavior to how deterrence can be more effective—specifically, under what conditions and for which individuals punishment is effective. In the context of DUI legislation, although DUI laws abroad are generally effective, differential deterrence exists across different countries and regions. One source of this variation is differences in the legal environment (i.e., various combinations of enforcement intensity, forms of punishment, and execution processes). Additionally, key individual characteristics of drivers (e.g., social background, moral awareness, self-control) influence their perception of legal risks, and these differences in perception further affect the deterrent impact of punishment risks across different groups.

2.2.2 Experimental Research on DUI Legal Deterrence

Conducting experimental research facilitates the study of environmental and individual difference factors related to DUI legal deterrence. Most existing studies on the effectiveness of DUI laws employ program evaluation methods, typically focusing on a single legal factor and only able to assess limited current schemes. This approach cannot accurately quantify the unique contributions of various elements of legal deterrence when other factors are present. Experimental research, however, can expose target populations to different hypothetical legal environments, allowing for observation and recording of participants’ decisions in these environments to answer these questions. Although very limited in number, laboratory experiments on DUI decision-making have been conducted. For example, Nagin and Pogarsky tested the effects of different license penalties, while Jacobs and Piquero combined penalty avoidance (low vs. high certainty) with severity (small fines vs. one-year imprisonment) to examine the impact of four hypothetical scenarios.

Inspired by existing experimental research, Yao and colleagues were the first to construct a more comprehensive hypothetical DUI legal environment using conjoint experiment methodology. By systematically varying multiple factors (e.g., forms of DUI penalties, enforcement intensity, and speed) and analyzing changes in participant decisions, they quantified the impact of each factor on DUI likelihood and predicted DUI incidence under different legal environments. This experiment selected seven main factors associated with U.S. DUI laws (including the legal blood alcohol concentration, enforcement intensity, detention, fines, license suspension, speed of punishment, and alternative transportation options). Using a blocked fractional factorial design, 36 hypothetical legal scenarios were created, and the DUI likelihood expressed by 121 American college student drivers in each scenario was analyzed with hierarchical Bayesian statistics, yielding unique deterrent effects for each experimental factor at the individual level (rather than average-based modeling). Similar to the findings of prior program evaluation studies, this experiment also found that enforcement intensity (reflecting the certainty of legal deterrence) had the greatest impact on potential DUI behavior. It also demonstrated, for the first time, the feasibility of the conjoint experiment method in the field of legal deterrence and the advantage of using hierarchical Bayesian analysis to model individual differences in DUI decision-making.

Based on the above findings and within the context of China’s criminalization of drunk driving in May 2011, which introduced a 2-3 month imprisonment penalty, Yao and colleagues replicated the above conjoint experiment in China, exploring DUI decision-making among 109 Chinese college students. Compared to the original experiment, the overall design and analytical methods remained largely the same, but certain experimental elements and content were adjusted according to China’s context (e.g., modifying the length of detention penalties, adding license point deductions, and removing the legal blood alcohol concentration limit). Additionally, personal characteristic factors closely related to differential deterrence were included as control variables in the statistical model. The study identified both similarities and differences in risk perception and DUI decision-making between Chinese and American college student drivers, confirming the applicability of this conjoint experiment in the Chinese context and laying the groundwork for studies on the general Chinese driving population.

2.3 Key Points for Replicating Existing Conjoint Experiments

Given the difficulty in accessing comprehensive traffic accident data domestically, this paper aims to further replicate the aforementioned series of conjoint experiments conducted by Yao and colleagues. This approach scientifically assesses the general deterrent effect of China’s “criminalization of drunk driving” policy over the past decade from another perspective, thereby expanding the external validity of legal deterrence theory and the concept of differential deterrence. Prior to conducting any experimental replication, it is necessary to determine the theoretical and practical significance of the research question within the Chinese context and assess whether the experimental method can effectively address it by referring to the critical replication decisions summarized by Walker and the special considerations pointed out by Chinese scholars for experiments replicated in China. This involves evaluating whether the context and design of the original experiment suit China and, ultimately, determining the type of replication to be conducted.

Firstly, the two conjoint experiments by Yao and colleagues targeted an influential policy in developed countries—DUI laws—that has extensive social impact. The findings hold substantial scientific significance, and the experimental method adopted serves as a methodological complement to existing studies primarily based on program evaluation, making it suitable for further replication.

Secondly, the original experiment in the aforementioned series was conducted in the United States, while the U.S. and China differ greatly in historical contexts, political cultures, and internal mechanisms. For example, the Asia-Pacific culture emphasizes collectivism and a tendency to defer to government authority, and crime tends to carry more severe social stigma. Therefore, the second experiment initially selected a sample of Chinese college student drivers, preliminarily demonstrating the applicability of the original experiment’s context and design in China. It is noteworthy that due to historical and economic development factors, the proportion of Chinese college students with driving licenses is far lower than that of their American peers, and many do not own private vehicles (either personal or provided by parents) during their college years, resulting in less driving experience and fewer opportunities. In contrast, U.S. college students, who generally acquire driving licenses earlier and gain more driving experience, more closely represent the general driving population in the U.S. (excluding age-based limitations). Therefore, as the third study in this series of conjoint experiments, this paper will replicate the experimental design and procedures for Chinese college student drivers with the general Chinese driving population to better compare the effects of legal deterrence in the contexts of China and the U.S.

Finally, regarding the type of experimental replication for this paper, as shown in Table 1 on the following page, replication can be categorized into four types based on the target group and measurement analysis methods in the original and new experiments. Schmidt argues that from basic direct replication to more complex generalization extensions, replication research progressively contributes to promoting generalizable concepts and mid-level theoretical construction. This paper aims to conduct an empirical generalization type of experimental replication, applying the measurement and analysis of the original experiment to a new subject group (i.e., general drivers). The selection of the new research group combines both consistency and diversity. Choosing the general driving population in China as the replication sample is motivated by their closer resemblance to the U.S. college student sample in terms of driving experience and their geographic proximity to the Chinese college student sample, facilitating a comparative analysis of risk perception and driving decision-making differences both between Chinese and American drivers and within the Chinese driver population.

2.4 Research Framework of the Study

Based on the above replication key points, this paper summarizes the feasibility, expandability, and comparability of applying the conjoint experiment method in the field of legal deterrence, as well as the corresponding research content and methods, as shown in Figure 1. Feasibility refers to the fact that the original experiment used widely applied legal deterrence and differential deterrence as its theoretical framework, demonstrating that conjoint experiments can effectively simulate legal environments and quantify elements of legal deterrence. Expandability indicates that the original experiment is not limited by the type of law or geographic location, highlighting the method’s flexible application across DUI laws in different countries. Comparability suggests that the results of this series of experiments facilitate the comparison of legal contexts across countries and different types of driving groups, thereby providing more empirical evidence for differential deterrence.