[author]Liu Zhuang

[content]

Illusion and Reality: Legal Artificial Intelligence and Beyond

Liu Zhuang

Associate Professor, School of Law, University of Hong Kong

Abstract: This article clarifies two misunderstandings in the Chinese legal academia regarding the interdisciplinary field of law and data science. First, it addresses the misconception that computational legal studies, law and big data, and law and artificial intelligence are entirely new new disciplines and withe new methods and paradigms. This misconception stems from a lack of understanding the relation between statistics, data science, and computer science,as well as their penetration in the social sciences over the past four decades. Second, it discusses the unrealistic and science-fiction-like expectations surrounding legal artificial intelligence, emphasizing the limited scope and capabilities of artificial intelligence in legal applications. I also discuss the potential and more realistic contributions of data science and artificial intelligence to legal research and practice, particularly in measuring and understanding the effect of law and policy.

Keywords: Artificial Intelligence in Law; Empirical Legal Studies; Data Science; Interdisciplinary Studies; Applications and Limitations;

Introduction

Artificial Intelligence has once again attracted keen public attention and debate in 2023 with the introduction of large language models such as ChatGPT. Text is the expression of law, and generating text ("writing documents") is a core task in the legal field. Many people feel that generative AI such as ChatGPT will have a broad application prospect in the legal field. In fact, long before ChatGPT was launched, the concepts of law and big data, computational legal studies, and artificial intelligence legal studies have become very popular in China, and the industry is looking forward to the fundamental changes that big data and artificial intelligence will bring to the practice of law, and the legal community hopes to take advantage of the new research topics and research methods to "overtake" and make world-leading academic research. The world's leading academic research.

However, under these visions, the legal profession seems to have no understanding of data science and AI itself, and has little interest in the specific knowledge of data and algorithms that are the basis of AI. In this regard, it is difficult to tell how much of our expectations of legal big data and AI are based on illusion and how much on truth.

This paper first uses actual research cases to illustrate that computational legal studies, big data legal studies, legal data science, and legal artificial intelligence are just different names for the same field of study and the same research method. Without data, algorithms and artificial intelligence can not be established; the so-called artificial intelligence is only a series of algorithms used to process data. Regardless of what we call the above fields, their core is only the use of data and algorithms to study law-related issues.

Further, I will endeavour to dispel two misconceptions about the relevant disciplines in our legal academy.

Firstly, it is believed that computational legal studies, law and big data, and legal artificial intelligence is a discipline that has emerged in recent years, or that it is a new discipline with a new methodology and paradigm - thus, we don't need to accumulate too many disciplines and read too much literature in detail, and we can even "overtake" or even establish a school of thought as long as we "do it big and fast". We just need to "work hard and fast", and then we can "overtake" or even establish a school of thought. This misunderstanding is probably due to a lack of familiarity with the academic lineage of empirical legal research, especially quantitative research. Fundamentally, this in turn is due to a lack of familiarity with the relationship between statistics, data science and computer science, and the penetration and application of these disciplines in the social sciences in the last forty years. In this regard, this paper will sort out the development of the disciplines and clarify the origins of the relevant disciplines and research methods.

Second, overestimating the possible impact of AI on law, or even having fantasised and sci-fi expectations of legal AI. This is still largely due to the refusal to put in the hard work to understand the basic knowledge and principles of AI, and being too easily seduced by new vocabulary and concepts rather than new ideas and methods. In this regard, this article will explore the possible scope of the application of big data and AI in law and its limitations, pointing out that it is difficult for AI to deal with both the generalised application of law and, even more so, the ubiquitous problem of factual judgement in practice.

On the basis of dispelling the illusion, the paper will explore the practical possible contributions of data science and AI to legal practice and legal research. Unlike most practitioners of legal AI, this paper argues that the greatest role of data science and AI for law is in legislation. The analysis and study of data can help us to better measure and understand the effects of the operation of the law, which in turn will enable us to legislate more scientifically. In addition, I will use real-world examples to illustrate the role of legal data science and AI in providing tools to aid and automate legal decision-making.

In summary, in the face of the data and intelligence age we are in, it will be difficult for law to solve problems using traditional methods such as systemic and doctrinal analyses, "eclecticism", and comparative law; or rather, these methods will become less persuasive and have a smaller audience. Jurisprudence should face up to the real problems, break down the narrow disciplinary barriers and gateway views, study and develop cross-disciplines, and make painstaking efforts to understand the basic knowledge and principles of other disciplines. Only by "grasping the fundamentals of things" can research be thorough and truly persuasive and grasp the masses.

1. The same field under multiple names

In 2018, The Quarterly Journal of Economics, a leading economics journal, published the article Human Decisions and Machine Predictions. The article focuses on a very specific area - the bail decisions of US judges. In the US, after the police have detained a suspect, a bail judge has a very short time to decide whether to release the suspect on bail, or to take him or her straight to jail, pending a formal court hearing. Human Decision-Making and Machine Prediction demonstrates in a rigorous way that machines are capable of making better judgements than judges when it comes to bail. Specifically, the authors used data on bail cases in one US state and used a random forest algorithm to predict the recidivism as well as the flight risk of each suspect while on bail, and in the end found that the algorithm's predictions were more accurate than the judge's judgement.

The contribution of the research may be better understood by reading from the conclusion - "Our findings imply that (replacing judges with machines for decision-making) would reduce crime by up to 20%, while keeping the size of the prison population in custody constant". "This means that if our algorithm were to be rolled out nationally, it would be equivalent to approximately 20,000 additional police officers for the country". To put it in more blunt terms, with this algorithm, the United States could fire all of its bail judges, and society could become a much better place.

Legal decisions have always been considered among the most complex human decisions. Although Human Decision-Making and Machine Prediction limits its study to only a small area of legal decision-making, the fact that a machine can outperform a judge has shocked the academic community anyway.

As is customary in jurisprudential thinking, we may need to label this research and place it in a sectoral area of jurisprudence. Just how do we classify this research as a field? As you can see, this research certainly requires data and data science - the basis for making predictions lies in the availability of a large amount of bail data. Thus, we could call it data law and big data or legal data science. Prediction, of course, also requires computation and algorithms, so it seems that it could also be classified as computational legal studies.

The article uses machine learning, which is at the heart of today's AI technology, or rather, the main function of AI is to make predictions (even generative modelling, which is based on the principle that it is only making predictions about language). So, it would be appropriate to categorise the research as legal AI. It is just that, in this way, it seems tricky for law to define the specific discipline of this research.

Imagine how the above discussion is different from the academic debate on whether securities regulation belongs to the realm of commercial law or economic law.

Leaving aside these jurisprudential discussions, we can all feel that this research uses rigorous methodology and addresses real issues, and I am afraid that it does not matter which discipline it is categorised under. If sharp enough, we can also feel that computational legal studies, data jurisprudence, legal data science and legal artificial intelligence are just different names for the same field. In fact, algorithms and AI can't be established without data - all machine learning is based on data; so-called AI is just a collective term for a series of algorithms used to process data - and those human-like machines are not the the essence of AI. Regardless of what we call the above fields, at their core they are simply the use of data and algorithms to study law-related issues.

In this field, we clearly need cross-disciplinarity, which in turn requires us in particular to break down narrow disciplinary barriers thinking, and in turn to focus in particular on studying practical problems and avoiding conceptual entanglements. Human Decision Making and Machine Prediction is a collaboration of five authors. Three of them are computer scientists from Cornell University and Stanford University, and two are economists from the University of Chicago and Harvard University. Judging from the composition of the authors, one can't help but wonder: why does an AI study need the participation of economists? And economics as a representative of the social sciences, is not only about the social phenomena of the discipline, and even in many people's view is relatively "soft" discipline? Does artificial intelligence seem to be more "hard" and contain more technological components? At the same time, a major research breakthrough related to the law, but no legal scholars involved, which inevitably makes people reflect on the embarrassing situation of law in today's intelligent era.

In fact, the core difficulties of the above study (Type II statistical fallacy and causality inference) were solved by economists rather than computer scientists, which fully reflects the high degree of cross-fertilisation between contemporary social sciences and natural sciences.

Today, a number of disciplines, including economics, political science and psychology, define their field of study as an inquiry into human decision-making behaviour; in terms of methodology, most of them use physics as a benchmark, mimicking the success of physics over the past few centuries - the use of mathematical models for theoretical construction, the use of statistical methods for theoretical empirical analyses and tests of propositions. In this respect, how judges make decisions (an empirical, or contingent, question), and how judges should make decisions (a contingent question), have been subsumed within the study of the social sciences in general.

The whole of modern social science is riddled with the expansion of similar methods, which social science researchers have taken to calling "economics imperialism" - many of the data-science, or quantitative, tools we now use to study social phenomena were developed by economists. But in fact, this is essentially the expansion of mathematics and statistics, the "imperialism" of the natural science method. Thus, a complete understanding of the cutting-edge applications of AI and data science in the law requires not only an understanding of computer science, but also an understanding of the various methods that are used to study social life and human decision-making, including machine learning methods for predictive purposes and social science methods for causal inference.

It is a disturbing fact that legal people have fallen behind, both in comparison to the natural sciences and in comparison to other areas of the social sciences. The fact that machines make more accurate decisions than judges is certainly an important technological breakthrough, but legal people have not been involved in this endeavour. In fact, few lawyers have made significant contributions to the important field of legal technology, and even fewer lawyers actually understand legal technology.

If the future is an era dominated by intelligent technology, and if machines and artificial intelligence can really replace judges gradually, then how should the future legal practitioners conduct themselves? I am afraid that legal practitioners cannot just bury themselves in their own familiar fields and use doctrines, "eclecticism" and comparative law to cope with the challenges of the future era, but should be more problem-oriented and learn the perspectives and methods of different disciplines. The conceptual analysis and systematic analysis that law and big data and artificial intelligence focus on are not likely to be too persuasive in the era of big data and artificial intelligence.

2.Academic origins of legal data science and legal AI

Looking at the current state of research in China, we have several major misconceptions about the relevant disciplines. One of them is that computational legal studies, legal data science, legal artificial intelligence are just one or some disciplines that have emerged in recent years. In fact, these disciplines have been born for a long time and have their own academic origins and development logic.

Quantitative research in the social sciences has been increasingly developed since the 1970s. In terms of the methods used, these studies are broadly divided into three categories.

The first category is research that explores the correlation between social phenomena using basic statistical methods, such as correlation analysis, logistic regression, linear regression, and so on. What is correlation? That is, the tendency of two variables to move together. For example, there is a positive correlation between height and weight, and a positive correlation between sentencing and the severity of the offence. Quantitative research in the field of law has evolved along with the quantification of the social sciences as a whole.

In the 1980s, researchers datamined U.S. Supreme Court decisions and found strong correlations between judges' decisions and their political party affiliation-Democratic justices were more likely to rule in favour of legalising abortion, racial affirmative action, restricting the freedom to bear arms, and tightening economic regulation; Republican justices were the just the opposite. Again, early scholars studying why people obey the law found a high correlation between parties' agreement with procedural fairness and their agreement with the outcome of the litigation; that is, parties who believed the process was fair also agreed more with the outcome. In the last decade or so, Posner's main research effort in his later years has been on such quantitative empirical studies of judges and the judicial system.

Of course, going back three hundred years, statisticians in London found as early as 1665 that there was a strong positive correlation between the number of cases of the Black Death and the number of cats in London neighbourhoods. This discovery inspired the City of London to order the culling of a number of cats, but it also caused the plague to grow more and more rampant - in the end, it turned out that the Black Death was spread by rats. Yes, correlation is not the same as causation. Inferring causality incorrectly can have very serious consequences. This is a problem that is emphasised in almost all first lessons in statistics.

This is why the second type of quantitative social science, which began to emerge in the 1990s, has focused its attention on discovering causality. This round of quantitative social science has been dominated by economists, and is known in economics as the "credibility revolution" in empirical research. The so-called credibility revolution means that data analyses are not only satisfied with finding correlations between phenomena, but also with determining causality. The purpose of the revolution is also clear: "Don't kill the innocent cats" - to avoid wrong laws and public policies that harm society.

So what kind of statistics and data analysis methods can infer causality from correlation? The answer is surprisingly simple: there is one and only one method of thought by which humans corroborate the causality of things from an empirical (rather than theoretical) point of view, and that is experimentation.

Today, experimental methods are widely used in the natural sciences, but humans do not naturally experiment. The history of science is long, and experimentation has only become a method consciously used by scientists in the last four or five hundred years. It was only in the time of Bacon and Mill Jr. that there were systematic summaries and reflections on this method. The development in the social sciences is much more recent, the experimental method being first applied to social psychological research. Such experiments were largely conducted in laboratories, with artificially set and therefore well-defined treatment and control groups. Obviously, many aspects of social life cannot be reproduced in the laboratory and are therefore difficult to study in the laboratory. For example, how can population growth, crime control, and the effects of law enforcement be studied in the laboratory? The inability to conduct experimental research in real-life scenarios has thus become a major obstacle to the development of empirical social science.

In this regard, economics pioneered a series of new ideas in the 1990s to apply algorithms similar to experimental methods to the analysis of real-world data, which in turn enabled quasi-experimental studies of real societies. These methods included matching, double differencing, instrumental variables, and breakpoint regression. Today, these methods are standard in quantitative social science research, and researchers in economics, political science, sociology, and other fields are familiar with them. A number of economists who have developed these methods have won the Nobel Prize in Economics, exemplified by Joshua D. Angrist and Guido W. Imbens, the 2021 Nobel Prize winners.

Quasi-experimental methods have also been widely used in legal research since the 1990s. Readers of Devil's Economics are often amazed by Steven D. Levitt's finding that the legalisation of abortion in the United States in the 1970s led to a decline in (juvenile) crime in the 1990s. The central idea of this study was quasi-experimental, specifically the way the data were analysed was a non-standard double-difference method. In other studies, he used instrumental variable methods to measure the elasticity coefficient of the number of police officers on the crime rate (how many points of crime reduction does hiring one more police officer?) ; using releases of incarcerated prisoners triggered by prison overcrowding litigation to measure the relationship between incarceration rates and crime rates (how does the random release of an inmate from custody lead to a change in the crime rate?) .

Not only in criminology and criminal law, but in fact, over the last thirty years, these quasi-experimental methods have gradually been applied to almost all areas of law, including constitutional law, contract law, property law, corporate law, procedural law, international law, and so on.

Since the 21st century, there have been further developments in quantitative social science.

On the one hand, due to the development of the Internet and the popularity of methods such as data crawling and natural language processing, the scale, data diversity, and data granularity of the data used for research have been greatly enhanced, providing the raw material for large-scale data analyses - for example, textual data, social network data, image-audio-video data, dynamic real-time high-frequency financial and economic data, have been collected and applied on a large scale; on the other hand, computer storage capacity and computing power have grown exponentially in more than a decade ("Moore's Law"), providing the basis for the application of more complex algorithms ("artificial intelligence"). foundation - a wide variety of non-linear algorithms, especially deep learning methods such as neural networks, are widely used. As a result of these two points, significant advances have been made in the discipline of data science and, in particular, artificial intelligence. At the same time, a third category of quantitative social research, known in recent years as computational social science, has begun to emerge as a result of the application of these data and intelligence methods.

In essence, the research goals of the third type of quantitative social science are very close to those of the first two types of quantitative social science. For example, like the first type of quantitative social science, it focuses on discovering the correlation between things; like the second type of quantitative social science, when it is able to discover causality, it also tries to answer the question of "why" and infer causality as credibly as possible. Unlike the first two, the third type of quantitative social science places more emphasis on "making predictions" - it has prediction accuracy as a central goal. This is a goal that has not been favoured by quantitative social science in the past.

As the quantitative social sciences as a whole have grown, so has the use of large-scale data to make predictions in legal research; machine learning methods have been applied exploratively in many areas of sectoral law. Against this background, we have the important result of Human Decision Making and Machine Prediction, introduced at the beginning of Part I of the article.

To summarise, the three categories of quantitative empirical legal research broadly favour relevance, causality, and predictive power, respectively. We can call the last category of studies by different names, such as computational legal studies, legal data science, and legal artificial intelligence, but the lineage of research behind them is clear and progressive; this field of research is not new, and it did not just "fall out of the sky".

The researchers of the above three types of research also have a great deal of overlap. The first scholars to use correlation to do research later began to use quasi-experimental methods; with the development of machine learning and other algorithms, they also began to try to use larger-scale data and newer algorithms. This overlap is understandable - a researcher who has specialised in data analysis since the 1990s will naturally move with the times and adopt the latest research methods.

There is also a clear kinship between these methods: correlation analysis is the basis for causal inference and quasi-experimental methods; regression analysis, invented by statisticians a long time ago, is to this day one of the most important methods for machine learning ("artificial intelligence"). For learners, the first two types of research are also the basis for the third, and need to be mastered in a sequential manner. This also means that there is a lot of catching up to do in this area: not only learning about the trendy "artificial intelligence", but also familiarising ourselves with the main methods and results of all previous quantitative empirical research.

It should also be clarified that, despite all the methodological developments, causality remains the jewel in the crown of the quantitative social sciences, and the knowledge that all quantitative research aspires to acquire. This is understandable - the ultimate goal of scientific research is to understand the causal relationships between things, to answer the question of 'why' in order to improve human knowledge - not just to explore correlations or to make conclusions. It's not just about exploring correlations or making accurate predictions. After all, human curiosity is the driving force behind scientific development.

At the same time, computational social science, which uses complex algorithms, has its own fatal weakness: the application of complex algorithms, especially deep learning methods such as neural networks, improves prediction ability while reducing the interpretability of the algorithms, i.e., reducing our understanding of the relationship between the independent variable and the outcome variable; very often, we only know that the prediction accuracy is improved, but we don't know what factors are responsible for the accuracy. know what factors led to the accurate prediction and become more confused. This obviously deviates from the essence of scientific exploration.

3.Could machines replace judges?

In addition to overestimating the "novelty" of law and big data and computational legal studies, we are also prone to overestimate the possible impact of artificial intelligence on the law, and even have fantastical and sci-fi expectations of legal artificial intelligence. This is largely due to the refusal to work hard to understand the basic knowledge and principles of artificial intelligence, and too easily seduced by new terms and concepts rather than new ideas and methods.

Using machines to make legal decisions has been the dream of many great thinkers. Leibniz, for example, tried to reduce the law to a set of algorithms that could be executed automatically on a machine, which, after being informed of the facts of the case, would give a legal conclusion. In the view of many people, the machine is more impartial and selfless, and the judgement will be made by the machine rather than the judge, which will completely eliminate human discretion in the law enforcement process as well as the resulting abuses, fundamentally removing the extra-legal factors in the administration of justice, safeguarding the fairness of adjudication, and realising social justice. Today, thanks to the rapid development of artificial intelligence technology, the realisation of this dream does not seem far away.

Using technologies based on data science and artificial intelligence, machines can indeed learn and simulate judges' legal judgements. For example, by learning data from past judgements, an algorithm might find that suspects who steal $100,000 are typically sentenced to three years in prison; those who steal $150,000 are typically sentenced to four years. By finding these patterns, the algorithm would be able to simulate a judge for sentencing. This is the idea of machine learning - to summarise the laws by summing up the experience, and then apply the laws to similar scenarios. Another way to think about it is as an expert system: we tell the machine directly about the provisions of the criminal law, which state that theft of 100,000 yuan is punishable by three years' imprisonment, and 150,000 yuan is punishable by four years' imprisonment. By programming a complex system of rules into the machine, the machine can then use logical reasoning (deduction) to determine the sentence.

Similarly, in civil and administrative cases, machine learning and expert systems can help machines make judge-like legal judgements. For example, in the case of a sale contract, by studying data from previous judgements, the algorithm found that the judge generally ruled that the defaulting party should return the price of the contract and compensate the other party in full for the losses caused by the default. In similar cases, the algorithm can then apply the same logic to judge the case. Similarly, we can edit the terms of contract law into computer instructions and order the machine to apply logical reasoning directly to the judgement.

Theoretically, as long as the amount of data is large enough (enough cases that can be learnt by the machine), or as long as the input system of rules is comprehensive enough, the machine will be able to make judge-like legal judgements and complete the work of applying the law. Only, both conditions are difficult to fully realise under the current level of technology.

The real world is so rich, and the nuances of the law so varied, that it is difficult to exhaust the logical propositions and replicate and automate all the logical systems in the law by means of an expert system. The machine learning approach seems to be more feasible, but it also faces the problems of data and cost - real legal problems are numerous and nuanced, which puts high demands on the machine's ability to understand the law; at the same time, a lot of legal problems appear in only a few cases, which are highly idiosyncratic and thus difficult to provide a sufficiently large training sample. .

Insufficient training samples make it difficult for the machine to find patterns in them. What's more, a case often involves multiple legal issues, and the combination of different legal issues makes the task even more complex for the machine. In essence, the machine is faced with a huge variety of legal problem types and nearly endless permutations and combinations, but the amount of data (judgements) for training is always limited. This imposes fundamental limitations on machine learning. These problems are easier to overcome in criminal law, because the types of offences and penalties stipulated in the criminal law are inherently more limited, and it is easier for the machine to learn the rules and laws therein; whereas the types of civil and commercial transactions are complex, and the cases involve a wide range of legal issues, which are much more difficult to be dealt with by the machine.

The above problems are only problems in the application of the law, that is, there are already clear legal provisions in reality, the machine only needs to find these laws and apply them to make judgements and draw conclusions. In legal practice, there are two other problems which are more difficult to be handled by the machine than the application of laws: one is the judgement in difficult cases, and the other is the judgement on the facts of the case.

In the absence of clear legal provisions or clear precedents, machines are powerless. This means that it is difficult for machines to solve really difficult cases (this is what Dworkin defines as a difficult case - a case where there is no law or precedent that can give a clear conclusion). In difficult cases, judges generally use their discretion to make judgements, and to a large extent, they are doing the work of creating law, i.e., legislating. Legislation is based on the judge's social experience and his or her knowledge of the value judgements behind the law (what Dworkin calls "principles" to disguise the inevitably anti-democratic elements of justice).

These are tasks that, frankly, machines are simply incapable of performing-imagine how machines could accumulate vast amounts of social experience and use it to fill in legal gaps and create new laws? This is completely beyond the scope of machine learning objects. Unfortunately, large language models also struggle with this problem. Linguistic models learn from established human knowledge (language), which means that the results of their learning do not go beyond established norms.

It is also difficult for machines to make effective judgements on factual issues in cases. Difficult cases are in the minority (although they may well be a "critical minority" for legal research), but factual issues are present in every case. Returning to the previous example, is a machine capable of determining what is "theft" and what is "breach of contract"? To determine whether an act is theft, it is necessary to consider the specific method of the act, the environment at the time of the act, the intention of the suspect and other elements, and in the judicial process, these elements are supported by witness testimony, video, statements and other materials to establish.

In the face of this process, it is difficult for a machine to extract key information from these modal and numerous basic materials (text, audio, video, images), to understand the significance of each basic material for judging the nature of the act, to identify the truth or falsity (especially in the case of testimony and confessions), and even more difficult to synthesise these materials into a holistic understanding of the facts.

I am afraid that "theft" is only the simplest example, and many judgements in the law require a more complex understanding of the facts. For example, how to judge whether the tortfeasor has "fault" - to judge fault, you need to know the duty of care of the average person in society, that is, you need to perceive how the average person in a similar situation will behave; how to judge whether the suspect has "intent" - to judge intentionally, you need to infer their behaviour at the time of the act. "Intentionality" - to judge intentionality, it is necessary to infer the subjective state of mind at the time of the act; how to judge the "foreseeability" of the loss of the other party in the event of a contractual breach of contract --To judge foreseeability, it is necessary to know how much information the breaching party had and should have had; how to judge whether the goods sold were "defective" and whether the seller had fulfilled its duty to inform the other party. -To judge defects, you need to know what the general quality and condition of similar goods are. The list goes on.

As can be seen, the judgement of facts requires a great deal of social experience and simple practical reason. For humans, acquiring both social experience and practical reason is not difficult - we know roughly what is at fault, what is intentional, what is foreseeable, and what passes for commodity quality. But for machines, this means that the algorithm has no explicit learning task (no defined outcome variable) and no training set (training data) to learn from. Or rather, the training set is the entire social life, without boundaries.

There is a more difficult problem than just plain old single facts: how to make machines understand complex "stories" as well as people. Storytelling - narrating and understanding narratives - is a core human ability, and a core task of legal work. After all, the legal process is about reconstructing a past event, a "story". The story is based on facts, and the facts are based on evidence. However, everything - the authenticity of the evidence, the reliability of the facts - needs to be judged on the basis of one's own social experience, to form a "testimony of the heart". To a large extent, judging the truth of a story is the most difficult point in legal decision-making.

Imagine when Wu Xieyu told the motive for his crime and how perfect his mother's character was, would a machine have the ability to judge the truth of this statement? (What about humans?) When Rao Rongzhi stresses that she was a coerced helper in all the cases, does the machine have the ability to judge her role in the crime based on all the evidence in the chain of evidence? Even, it doesn't need to be a complex case, the conundrum exists in everyday disputes as well. In a lending dispute, if both parties only have a verbal agreement, how can the machine judge whether the loan really exists? Divorce cases, the machine how to judge both sides of the relationship has indeed broken?

Aristotle said that literature is more serious than history. The perception of the beauty of literature involves the judgement and understanding of the authenticity of the story; insight into human nature, through the discourse and pretense, to discern the facts that occurred in the past, precisely is also the difficulty of the law, but also the difficulty of artificial intelligence. In this sense, law, together with literature, constitutes the human intelligence that artificial intelligence can hardly break through.

4.Possible contributions of data science and artificial intelligence to law and big data

So far, we have explored the difficulties that machines would have to face to replace judges. Indeed, to have machines replace judges is essentially equivalent to having to create a strong AI (which is hardly likely to be realised for decades). This is also quite understandable - legal decision-making involves complex factual, rule-based and value-based judgements, and also requires from time to time the creation of rules and the filling of loopholes on the basis of social experience and practical rationality, which invokes virtually all human intelligence at the highest level, and is thus only possible to be dealt with in an integrated manner by a machine with human-like intelligence.

In retrospect, it seems that one has to ask oneself: do we really need machines to do all the judges' tasks in a big way? In fact, the current mainstream of intelligent technology development is not so much generalised AI, but rather domain-based AI, which is used to solve a specific and small problem. This also applies to the legal field - after the disillusionment with artificial intelligence, we can be more down-to-earth, more practical considerations: what machines can actually do for the law people.

Firstly, unlike most practitioners of legal AI, I believe that the biggest role that data science and AI can play for the law is in legislation. The analysis and study of data can help us to better measure and understand the effects of the operation of the law, which in turn allows us to legislate more scientifically.

In China, many important discussions in jurisprudence lack research on basic factual issues, especially those supported by rigorous scientific evidence. For example, the legal profession has repeatedly discussed whether the death penalty should be abolished, but we have almost no evidence beyond simple perceptions of the attitudes of the general public towards the death penalty (whether they support abolition or not), and the extent to which abolition would weaken the deterrent effect of the penalty and affect social governance. Similarly, criminal law academics have explored whether the penalties for the offence of buying trafficked women and children should be increased to curb buying behaviour by cracking down on the buyer's market, but our lack of systematic understanding of the volume and structure of the trafficking market, the usual identity of the sellers (whether they are relatives or acquaintances), whether the purchase of women to get married and give birth to a child is a new demand, and the problems in law enforcement at the grassroots level in rural areas has led to a lack of certainty that Whether the increase in penalties is really beneficial to the protection of women and children.

In setting a lower age limit for civil capacity, we were unable to find evidence to show which was more beneficial to children and adolescents, ten or six years of age - a "compromise" of eight years of age was adopted, which is laughable. Similar examples abound, and it can even be said that the lack of scientific evidence is a problem that permeates almost every aspect of legal research and discussion. The lack of scientific evidence also means the lack of the most intuitive and powerful argumentation tools, which makes the legal research in the legislative issues lack of good, jurists in the public discussion of important issues appear to be footing on the ground, the foundation is not stable. In fact, the essence of most legislative issues is a matter of public policy, which has long since developed into a discipline that speaks with data analysis.

Data analysis and data science certainly cannot solve all the problems encountered in legislation, but they can provide scientific evidence on many issues and help us deepen our understanding of a large number of problems in the legislative arena. For example, by analysing data from a survey of more than 30,000 people, researchers had found that only about 68% of Chinese people support the death penalty - far lower than many jurists had previously thought, and even lower than the support rates in Japan and Taiwan, China - and that the country's The claim that ordinary people generally support the death penalty is not particularly reliable. At the same time, the data also show that the support for the death penalty among people with a university education or higher is about 10% higher than that of the general public, which suggests that support for the death penalty comes more from the social elite, and that public resistance to abolishing the death penalty, I'm afraid, also comes mainly from the people who have had some education and who are worried about public problems in society.

Moreover, even after controlling for the education factor, those who frequently express their opinions online are more supportive of the death penalty than the general public (8% higher). This shows that public opinion on the internet does not necessarily represent the real public opinion. Those who cry foul on the Internet are not necessarily typical Chinese people.

Another example is that many Chinese and foreign scholars have concerns about whether the live broadcast of court hearings will affect the fairness of trials. Former U.S. Supreme Court Justice David Souter is vehement: "The day you see a camera coming into our courtroom, it's going to roll over my dead body". To address this issue, the researchers carried out an experimental study of live trial broadcasting in China, using natural language processing methods to analyse a large number of courtroom voice data. The researchers found that during the live broadcast of court hearings, only the speech speed of the parties slowed down significantly, while the speech speed of the judge and the litigation agent did not change significantly, and the range of the fundamental frequency (reflecting the speaker's pitch) of all the subjects was significantly reduced. At the same time, the judges' use of legalese increased significantly and appeared to be more solemn and dignified.

These findings suggest that the live streaming of court hearings encourages the parties to be more cautious and reduces the extreme emotions and behaviours of all the subjects during the trial, while judges and litigators with more experience in live streaming are not unduly affected by the live streaming. This all suggests that the live streaming of court hearings has not interfered with the fairness of the trial.

As can be seen, for almost every issue of legislation and legal policy, we need to provide, and can provide, a lot of scientific evidence using data science methods. These studies are specific and nuanced, with varying research designs and analytical approaches. The field is wide open for a myriad of new and interesting questions to be explored in depth in this direction.

Second, another area of application of legal data science and artificial intelligence is legal decision-making assistance. Algorithms and machines are hardly a substitute for judges to do all of their work, but they are sufficient to help or assist lawmakers and parties to make better decisions in some specific areas.

In the early 2000s, it had already been shown that decision tree models could use data to predict US Supreme Court decisions with a predictive accuracy that exceeded that of professionals such as lawyers and big data professors. From the perspective of the parties, this algorithm can be a good decision-making aid - when the algorithm is able to accurately predict the outcome of a judgement, the parties can rely on the algorithm's predictions to make more rational decisions in litigation (e.g., whether to sue or not, whether to settle, etc.).

In fact, when the data is fine enough, the algorithm can also provide more guidance to the parties, such as which lawyers have a higher probability of winning in similar cases, which courts are more willing to support the parties' claims and process them faster, which courts are more efficient in enforcement, and so on. In a study, researchers used data from Chinese publicly available adjudication documents, from which they obtained information on Chinese lawyers in litigation, which made it possible to calculate the probability of success for each lawyer, each law firm, in each court, and in each type of case. The researchers also analysed the length and efficiency of judgments in all courts across the country. Based on these analyses, clients can make better choices of lawyers, of causes of action, and even of courts.

A wide range of decision-making aids can also assist courts and other law enforcement agencies, not just the parties involved. For example, a number of states in the United States have long used a recidivism risk prediction system (COMPAS). Through data analysis, the system can predict the recidivism risk (probability) of each offender, and judges can adjust sentencing based on these risk predictions to achieve the social effect of deterring and curbing crime. As another example, scholars in China have analysed the big data of referee documents to study the sentencing differences between different judges, and then identify abnormal behaviours in the sentencing of judges, a method that can help the courts to better manage trials, promote equal sentencing in the same case, and reduce the abuse of discretionary power in trials.

Over the years, many legal technology enterprises at home and abroad have invested a lot of resources in research and development of class case search and class case push tools. Regardless of China and foreign countries, class cases are important references for judges' judgement, as well as an important basis for lawyers and parties to make decisions. Class case retrieval and analysis is the basic work of every legal practitioner. If the algorithm can automatically perform class case retrieval and class case push by mining text data, it will provide great convenience for legal practitioners in various fields. Of course, judging from the existing development, the class case recognition technology at home and abroad is still not mature, and the push of class cases is not accurate. However, with the development of intelligent technology, some of the technical "bottlenecks" of class case recognition will certainly be broken, and class case push will be deeply applied in at least some legal fields.

The above are examples of data science and artificial intelligence to provide decision-making aids for lawyers. We can foresee that in the future, a variety of intelligent tools will be more prevalent. This will undoubtedly reduce the cost of legal work and dramatically improve the accuracy and science of legal decision-making.

Third, legal AI can also provide numerous automated tools to enhance the efficiency of legal work. For example, we can use statistical analysis and image recognition tools to automatically identify and detect abnormal behaviours in administrative enforcement or live trial broadcasts. We can also use translation tools to translate legal texts in different languages. We can also use big language models to summarise cases, contracts, legal opinions and other documents to improve the speed of legal reading. Even big language models can help us automatically generate various legal texts - of course, under the existing technical conditions, the generated texts are not yet accurate and perfect.

Among a variety of intelligent technologies, big language models give people the most room for reverie. Text is the expression of law, and generating text ("writing documents") is the core work of the legal field. Whether it is judges, prosecutors, lawyers, corporate legal and other legal workers, or ordinary people who sign contracts and participate in litigation, they all use text as a medium to deal with legal issues. Therefore, many people have felt early on that big language models such as ChatGPT will have a broad application prospect in the legal field. For example, big language models can answer legal questions, help draft contracts and instruments, assist in writing judgements, and so on. As of this writing, a number of generative AI products in the legal field have been released, and the transformation of legal work by big language models is happening.

However, general-purpose biglanguage models such as ChatGPT are not optimised for the legal domain and are therefore difficult to perform more specialised legal tasks. Typically, if ChatGPT is consulted about a legal problem, it will only give a logical but generalised answer and conclude with the recommendation that "you should consult a professional lawyer and understand the relevant legal requirements". In order for a big language model to be able to solve legal problems, it needs to be infused with legal knowledge. There are two ways to make the model learn legal knowledge: pre-training and fine-tuning.

At present, no matter how to train the legal big language model, it is more difficult to solve the model's "hallucination" problem, that is, the model generates the content on the surface appears to be serious and professional, but the substance of the content is made up - - it is "serious nonsense". For example, the model will "make up" non-existent laws and cases. A lawyer in the United States used ChatGPT to prepare a legal document that cited four non-existent false cases, resulting in the lawyer being seriously punished by the court.

Under the current technological conditions, it is difficult to completely eliminate the "illusion" error, because it is rooted in the training principle of the large language model. Big Language Model is a generative model based on statistical learning, through the learning of a large amount of text data, to predict the next possible words or sentences, so as to complete the dialogue and text generation. In this process, the model will select the next most suitable word from the training set according to the pre-trained statistical model and probability distribution, and continuously generate new dialogue content. A model trained in this way will appear fluent in form, but may be factually incorrect in content.

Of course, in time, we believe that some of the problems here will be effectively solved, for example, the industry has already proposed the combination of a large language model and a knowledge base (knowledge graph), the combination of a large language model and a retrieval algorithm, and other ideas, which are expected to alleviate the problem of the model's "illusions" and provide more accurate domain knowledge. At the same time, it is also possible to first use the big language model to carry out some simple tasks, for example, to help government departments, especially law enforcement departments, to generate formatted administrative (law enforcement) documents, to help the court filing division to summarise the prosecution materials, and so on, in which we can effectively limit the information sources of the linguistic model to avoid the "illusion" problem.

Conclusion

Computational legal studies, data law, and big data AI have made big breakthroughs in recent years, but their application scenarios remain specific and therefore limited. For example, using machine learning, we can predict bail decisions, we can predict sentences, we can predict US Supreme Court judgements; using big language models (such as ChatGPT), we can automatically generate legal documents, and we can conduct interactive legal Q&A.

However, most of these existing applications are built on the basis that humans have already structured a large amount of unstructured data (video, voice, text). At the same time, for each application scenario, researchers need to find the right research questions, collect large-scale data, and repeatedly tune the model - that is, they all need a lot of human intervention; even the "artificial" component is much larger than the "intelligent" component. Even the "artificial" component is much larger than the "intelligent" component. In the case of general AI in the legal field, the science fiction component is much higher than the science component. Artificial intelligence is condensed more data scientists hard sweat; expect the machine once and for all to liberate the legal people, it is still too early.

What can we expect from legal AI (legal data science, computational legal studies, data law and big data ......)? The research in this paper points out that we should first of all not stick to the concepts and definitions of jurisprudence, but pay more attention to practical issues and embrace the contribution of different ideas and approaches to jurisprudential knowledge with a more open mind and vision. We should neither close our eyes and ears to the exciting and cutting-edge developments in the field, nor be overly optimistic and blindly believe in its future. We need to understand the field at a technical level and use these methods to "get to the root of the matter".