The Researcher of Law Institute of China Law Society

Abstract: The advancement of large model technology has greatly driven the development of generative AI, but its data training and deployment have also generated new risks and challenges, including harmful contents, sensitive data leakage, misinformation, misuse for illegal activities, possible environmental and economic harms, risk transmission to downstream, etc. The EU recently intends to set specific obligations for providers of foundation models and generative foundation models, but it deviates from the original legislative intention of regulating based on risk classification. Our country has issued special measures to regulate deployers of large models, but they only have a limited control of the risks of large models. To regulate the risks of large models, it is necessary to follow the data utilization security paradigm, regulate based on risk categorization and classification, and achieve the co-governance of upstream and downstream participants. The key is to construct a regulatory framework for new risks, which mainly includes setting up a special agency to guide development, assessing and responding to risks, regulating data training to avoid data leakage and illegitimate output, building a control system based on risk classification by the risks of specific uses, establishing a transparency system throughout the entire life cycle of large models, and improving the co-governance mechanism of upstream and downstream to prevent the generation of illegal contents.

At the end of 2022, ChatGPT emerged with astonishing "humanoid" capabilities in language comprehension, text generation, and knowledge reasoning. Within two months of its launch, ChatGPT had over 100 million daily active users, prompting domestic and foreign technology giants to lay out their large model technologies and trigger a "hundred model war". While big model technology drives the rapid advancement of generative artificial intelligence, its automated content generation based on big data training has also raised new risk challenges, such as generating harmful content and data leakage. How to prevent regulatory risks, balance the development of artificial intelligence with security, and guide the healthy development of generative artificial intelligence have become common challenges faced by human society.

In response to this, major powers are actively researching countermeasures, while China and the European Union are at the forefront of exploring specialized legislation. In May 2023, China included the draft of the Artificial Intelligence Law in the annual legislative work plan of the State Council. The Interim Measures for the Management of Generative Artificial Intelligence Services, formulated in July 2023, mainly regulate providers of generative artificial intelligence services and do not regulate pure providers of large model technology. The European Union proposed the Artificial Intelligence Law in April 2021, and in June 2023, the European Parliament passed an amendment to the proposal, which stipulated the obligations of basic model providers, but basically regulated large models in comparison to high-risk systems. At present, the existing legislative exploration is not sufficient to fully regulate the new risks of the large model.

This article focuses on the new risks brought by generative artificial intelligence big model technology in this context, combined with the legislative experience of the European Union and the legislative exploration of China, proposing the basic ideas and institutional framework for regulating big model risks, in order to provide useful reference for artificial intelligence legislation in the era of big models.

1. The new breakthroughs in artificial intelligence big model technology have brought new challenges

The new breakthrough in big model technology has sparked a new wave of development in generative artificial intelligence, opening up a new path towards universal artificial intelligence. However, the new characteristics of big model technology also bring new risks and challenges.

1.1 New Development of Generative Artificial Intelligence Driven by Large Model Technology

The so-called generative artificial intelligence refers to models and related technologies that have the ability to generate content such as text, images, audio, and video. Unlike traditional rule-based or template based simple content generation, the rapidly developing generative artificial intelligence in recent years is mainly driven by "deep learning" technology, which can automatically learn structures and patterns from data through training and generate new high-quality content based on these patterns. Famous deep learning models include generative adversarial networks, diffusion models, and transformer models.

Among them, Transformer is a deep learning model based on self-attention mechanism, which can efficiently and parallelly process sequence data, making it possible to train large-scale data. Based on this model, OpenAI proposed the first generation of generative pre training model GPT-1 in 2018, which achieved unsupervised model "pre training" with large-scale unlabeled data, and then supervised model "fine-tuning" with labeled data to better adapt to downstream tasks, bringing generative artificial intelligence into the era of "pre training models". Later, BERT, LaMDA, T5, and others were pre trained models based on Transformers, and the large-scale data pre training of these models required strong computing power support.

This type of generative pre training model, also known as a universal model or basic model, is referred to as a generative artificial intelligence big model (hereinafter referred to as a "big model") in this article. It refers to a model trained on large-scale data with massive model parameters that can adapt to a wide range of downstream tasks. It can be seen that the lifecycle of large models is divided into a model training phase mainly based on data training and a model deployment phase mainly based on model adaptation. Large model training has technical characteristics based on big data, relying on strong algorithms, and requiring large computing power. The technical characteristics of the trained large model can be summarized as follows:

One is the large parameter scale: the parameter scale of large models is usually over one million, even exceeding one trillion. For example, the parameters of GPT-3 reach 175 billion, and the parameters of Beijing Zhiyuan's "Wudao 2.0" reach 1.75 trillion. It should be pointed out that these parameters only reflect the data learned by the model and do not contain or store the data learned by the model.

The second is to generate new content: based on patterns learned from training data, large models can generate new content. Taking ChatGPT as an example, it learns how words appear together with other words in context from a large amount of existing text and responds to user requests by predicting the next most likely word and each subsequent word.

The third is the emergence of new abilities: as the parameter size increases, larger models exhibit "emergence abilities" that smaller models do not have, including small sample prompt learning ability, thinking chain reasoning ability, etc. For example, GPT-3 can operate multiplication of two digits without specialized training, which exacerbates the inexplicability of artificial intelligence.

The fourth is to present universality: compared to specialized models trained on annotated data to perform classification, translation, and other tasks, large models train massive parameters and powerful emergence capabilities on a large amount of unlabeled data, allowing them to handle various different tasks through fine-tuning and other adaptation methods, and even handle unseen tasks, greatly improving the universality of large models.

The emergence of generative artificial intelligence models marks the shift of the research paradigm in artificial intelligence from training specific task models to training general task models, demonstrating a feasible path towards general artificial intelligence. In March 2023, GPT-4 was released, which not only demonstrates more general intelligence than previous large models, but also can accept input from images and text, achieving multimodal data processing. Microsoft Research believes that the performance of the GPT-4 is astonishingly close to human level, and there is reason to consider it an early version of the General Artificial Intelligence (AGI) system, which can be called the "spark of General Artificial Intelligence". The development of generative artificial intelligence models has opened the prelude to moving towards universal artificial intelligence.

1.2 New Risks Caused by Generative Artificial Intelligence Large Models

As can be seen from the above, big models are the result of training big data with high computing power, and their ability comes from deep learning of abstract co-occurrence patterns in a large amount of unlabeled data. Essentially, they are driven by big data. From the perspective of finding patterns and unleashing data value in big data, unlike traditional data mining and analysis that mainly rely on high-cost inputs such as expert annotation of data and design features, big models mainly perform unsupervised learning on a large amount of unlabeled data, automatically and efficiently extracting patterns and patterns from the data, which ultimately manifest as large-scale parameters in big models. The large model obtained through training and refining parameters through big data has strong capabilities and universality, and is itself a concentrated reflection of the value of training data. Therefore, the author believes that training and calling big models is a new way to utilize big data, and big models are an efficient way to realize the value of big data. Compared to the traditional "security issues of artificial intelligence systems themselves" and human generated content, the new way of utilizing big data, big models, has raised new risk challenges in terms of automated content generation through data training and model calls, mainly including the following aspects:

1.2.1 The risk of generating harmful content such as prejudice and discrimination

The large amount of data used for training large models is mostly unlabeled data, which is prone to bias, discrimination, and even harmful content called "toxicity" in the technical community, such as insult, hatred, violence, and pornography. Large models generate content based on the patterns learned from this data, and the generated content inevitably reflects the same problem. The most concerning issues are prejudice and discrimination. Prejudice can be understood as a subjective understanding and attitude, which often leads to objective differential treatment of specific groups. Unfair differential treatment can lead to discrimination, such as gender discrimination caused by gender bias. The National Institute of Standards and Technology in the United States categorizes artificial intelligence biases into three categories: systemic bias, which refers to biases caused by institutional norms, practices, and processes in culture and society; Statistical and computational bias refers to bias caused by insufficient representativeness of training samples; Human prejudice refers to systematic errors in human thinking. A study has tested text generation image models such as DALL-E2 and Stable Diffusion, and found that when prompted for "CEO", the generated images are all male images in suits and shoes. The reason for this result is that the training data itself has systematic bias and statistical bias, and does not have fair representativeness. From the perspective of system bias, if the training data mainly comes from a certain language or country, the large model will inevitably be branded with the cultural traditions, mainstream values, and ideologies of that language or country; We should be vigilant about the cultural and value conflicts that may arise from the application of large models, and guard against the ideological security risks they may bring.

1.2.2 The risk of disclosing personal information and sensitive data

This risk mainly stems from two aspects: first, the large model leaks personal information and sensitive data in the training data. Large scale model training often uses publicly available network data captured on a large scale, which may include personal information such as names and phone numbers and may even include sensitive personal information such as biometrics and travel trajectories, as well as high-risk data. Moreover, many large models default to user input prompts as training data, which may also contain personal information and sensitive data. Research has found that large models may "remember" and leak personal information, sensitive data, including copyrighted materials in these training data under specific input induction. In March 2023, Samsung was exposed to have experienced three sensitive data breaches within less than 20 days of being allowed to use ChatGPT, resulting in the leakage of sensitive confidential information such as semiconductor device measurement data, product yield, and internal meeting content. The second is to infer personal information and sensitive data through large models. Large models have emerged with powerful reasoning abilities, which may infer sensitive personal information such as religious beliefs and economic conditions of specific individuals, and may even analyze sensitive data related to national and public security. A study has found that if one claims to be engaged in research to prevent nuclear terrorism in prompt instructions, they can bypass ChatGPT's refusal to respond to nuclear weapon manufacturing prompts and persuade them to provide detailed instructions on how to manufacture nuclear bombs. Although the prompt instruction no longer worked shortly after the discovery was published, it did demonstrate the powerful ability of large models to extract sensitive data.

1.2.3 The risk of generating incorrect or misleading information

The generation of new content for large models is based on the intrinsic correlation and co-occurrence probability of training data. For example, if the high-frequency co-occurrence words for "not moving forward" in training data are "right turn", "left turn", etc., then after the user inputs "not moving forward", the large model may randomly output "right turn" according to its parameters. However, the training data may not have authenticity, timeliness, or correlation, so the model output results may sometimes be inaccurate, untrue, and may even generate erroneous or misleading information. OpenAI has pointed out that the output of ChatGPT can sometimes be inaccurate, untrue, and misleading, occasionally producing incorrect answers, and even fabricating facts or producing "hallucinations" output. In professional fields that require high accuracy of information, such as the legal and medical industries, relying solely on information generated by large models without verification may cause significant harm. For example, if one blindly believes in the treatment suggestions provided by the big model for certain physical discomfort symptoms, does not seek medical attention, or takes medication doses incorrectly, it may delay treatment or cause harm to the body. For example, recently, two lawyers from New York State in the United States cited six cases collected by ChatGPT in their legal documents submitted to the court. However, the court found that these cases were all fabricated by ChatGPT, and ultimately fined the lawyer and his law firm $5000 each.

1.2.4 The risk of abuse in illegal activities such as deception and manipulation

The above three types of risks mostly come from big data training of big models, and usually belong to risks caused by non-human intentional factors. The trained large model has strong universality and carries the risk of intentional misuse for committing illegal crimes such as deception and manipulation. Compared to the unintentional error information mentioned above, large models may be intentionally abused to create false information. The super strong generation ability of large models, as well as their "humanoid" output and interaction ability trained on a large amount of human data, make it possible to produce more realistic and deceptive false information in a low-cost manner on a large scale, such as producing more convincing phishing emails in large quantities. The more deceptive false information generated by these large models, if pushed through personalized recommendation systems supported by the large models, due to the "filtering bubble" and "information cocoon" effects, is likely to cause audience polarization, and even carry out targeted manipulation of audience concepts and behaviors. This may not only infringe on the rights and interests of private entities, but also pose a serious threat to a country's national security, especially political and cultural security. For example, in the early days of the Russia-Ukraine conflict, in March 2022, videos of Ukrainian President Zelensky and Russian President Putin announcing their surrender appeared on mainstream social platforms, which were later confirmed to be deeply forged. In addition, large models may also be abused for committing other illegal crimes, such as generating malicious software code to carry out network attacks.

1.2.5 Risks that may endanger the environment and socio-economic development

Even if the large model is not abused, its normal use may still pose certain risks to the environment and socio-economic development. But currently, these risks do not seem to be as urgent and precise as the aforementioned types of risks. However, from the development of artificial intelligence, these risks are likely to become major challenges in the near future. We should pay attention to and monitor the growth and evolution of these risks, and be prepared for any potential risks. For example, the high computational power requirements of large models can consume a large amount of energy and resources, which may cause certain environmental hazards. A study has found that training the GPT-3 large model generates 552 tons of carbon dioxide and consumes 1287 megawatt hours of electricity. However, it is also believed that the generalization ability of GPT-3 eliminates the need to retrain the model for each task and has potential energy advantages. For example, for a long time, there have been many views that artificial intelligence will eliminate a large number of jobs. But studies have pointed out that artificial intelligence tools are endowing rather than replacing human factors, and if developed and deployed ethically, artificial intelligence can empower people to do more. In addition, research has also focused on the risks that the application of large models may bring, such as increased inequality, decreased work quality, and damage to the creative economy.

1.2.6 Universality causes risk transmission to downstream applications

The large model exhibits strong universality and can be used to solve a wide range of downstream tasks. But this universality also means that the defects of the large model itself will be inherited by all downstream models, and the risks caused by the defects of the large model itself will be transmitted to downstream applications. The inherent flaws of large models mainly stem from the flaws in their training data. Therefore, the risks that large models can transmit to downstream applications are mainly the risks caused by their big data training, including the generation of harmful content, leakage of sensitive data, generation of incorrect information, and other risks. The risk transmission of large models to downstream applications means that the risk control of large models must rely on the joint efforts of upstream and downstream participants in the value chain of the large model. Among them, the most important ones are the main body for training and developing large models, and the main body for adapting large models to solve downstream tasks. In this article, the former is referred to as a large model provider, and the latter is referred to as a large model deployer. Unless the provider of a large model is also a deployer, in general, due to the inexplicability of deep learning algorithms and emergence capabilities of large models, deployers of large models face significant difficulties in understanding and responding to the risks of large model transmission. To cope with the risks of large model transmission, it is necessary for the providers of large models to share necessary technical documents and related information.

In summary, the new risks caused by big models as a new way of utilizing big data can be divided into two categories: one is the risks caused by model data training, mainly manifested as the generation of harmful content, leakage of sensitive data, generation of incorrect information, etc; Another type is the risks caused by model deployment applications, mainly manifested as misuse of illegal activities, potential harm to the environment and economy, and transmission of risks to downstream applications. As mentioned earlier, the first type of risk comes from the model training stage, rooted in the quality and sensitivity of the training data, such as insufficient representativeness of the training dataset, the presence of harmful content and sensitive data. The latter type of risk occurs during the model deployment phase, rooted in the misuse of the model, negative externalities, and universality of the model. In the face of these new risks, the technology community is working hard to research effective mitigation measures, promoting the alignment of large models with human values and intentions through reinforcement learning based on human feedback (RLHF), and has achieved certain results. For example, compared to GPT-3.5, GPT-4 has a 40% higher authenticity evaluation score for generated content, a 29% higher probability of complying with its policy response for sensitive requests (such as medical advice), and an 82% lower tendency to respond to requests that do not allow content.

Although these technological research efforts have significantly improved the security of large models, the risk challenges caused by large models remain prominent. Dealing with the new risks brought by large models, relying solely on technical mitigation measures and alignment measures is far from enough. It is also necessary to explore legal regulatory measures that adapt to the technical characteristics and development needs of large models.

2. The EU Exploration and Mirror of Legislation for Generative Artificial Intelligence Large Models

The EU, which was the first to explore comprehensive legislation on artificial intelligence, has responded promptly to the risks and challenges brought by large models. Although the proposal for the Artificial Intelligence Law proposed by the European Commission in April 2021 mainly aimed to classify artificial intelligence systems into four risk levels based on specific uses: unacceptable, high, limited, and minimum. Initially, it did not involve artificial intelligence systems without specific uses. However, with the development of large model technology, how to regulate large models and general artificial intelligence systems has become an unavoidable issue for EU legislation on the Artificial Intelligence Law. In December 2022, the European Council passed a common position on the proposal of the Artificial Intelligence Law, specifically adding a chapter on "General Artificial Intelligence Systems". However, this chapter mainly authorizes the European Commission to legislate on this in the future without making targeted provisions.

The European Parliament has had a more comprehensive discussion on these issues, and in June 2023, a proposed amendment to the Artificial Intelligence Act (hereinafter referred to as the "Proposed Parliamentary Version") was passed, proposing a relatively complete general artificial intelligence risk regulation scheme centered on large models. It can be foreseen that in the upcoming tripartite negotiations between the European Parliament, the European Council, and the European Commission on the final text of the Artificial Intelligence Law, the regulation of large models will be a key issue. Below is a brief summary of the institutional exploration of the EU's regulatory generative artificial intelligence model, based on the latest text of the proposed parliamentary version.

2.1 Specific obligations have been set for the basic model and generative basic model

The proposed parliamentary version refers to the so-called large model in this article as the "basic model" and stipulates the obligations of its providers.

2.1.1 Obligations of basic model providers

The proposed parliamentary version adds Article 28b "Obligations of Basic Model Providers", which clarifies in paragraph 2 the seven obligations that basic model providers should comply with, including:

One is the obligation of risk management. Require appropriate design, testing, and analysis to demonstrate the identification, reduction, and mitigation of reasonable and foreseeable risks to health, safety, basic rights, the environment, democracy, and the rule of law in an appropriate manner before and throughout the development process, and to document any remaining unmitigated risks after development.

The second is the obligation of data governance. Require only processing and inclusion of datasets constrained by appropriate data governance measures based on the underlying model, particularly reviewing the appropriateness of data sources, potential biases, and appropriate mitigation measures.

The third is the obligation of technical reliability. Require achieving appropriate levels of performance, predictability, interpretability, correctability, security, and cybersecurity throughout the lifecycle, and conducting evaluations using appropriate methods, such as those involving independent experts.

The fourth is the obligation to protect the environment. Require the application of relevant standards to reduce energy use, resource utilization, and waste, and improve energy efficiency and overall system efficiency; As long as technology is feasible, energy resource consumption and other environmental impacts should be measured and recorded.

The fifth obligation is to provide information. Require the development of extensive technical documents and easy to understand usage instructions to enable downstream providers to comply with the obligations of high-risk artificial intelligence system providers.

The sixth one is the obligation of quality management. Require the establishment of a quality management system to ensure and record compliance with this regulation, and may conduct tests to meet this requirement.

The seventh is the obligation of model registration. Require registration in a publicly accessible EU high-risk artificial intelligence system database.

2.1.2 Obligations of Generative Basic Model Providers

The fourth paragraph of this article further stipulates the obligations of providers of generative basic models. Whether it is the provider of the basic model used in generative artificial intelligence systems or the provider of specialized generative artificial intelligence systems, in addition to the seven obligations mentioned above, the following three obligations should also be followed:

One is the obligation of transparency. Require compliance with the transparency obligation of Article 52 (1), which means that the artificial intelligence system, provider itself, or deployer shall inform natural persons who come into contact with the artificial intelligence system in a timely, clear, and easily understandable manner that they are interacting with the system, unless this is evident from the usage situation and scenario.

The second is to prevent the generation of illegal content obligations. The training, design, and development of basic models are required to ensure that they are trained, designed, and developed in accordance with recognized existing technological levels, with sufficient safeguards to prevent the generation of content that violates EU laws, and without compromising fundamental rights, including freedom of speech.

The third obligation is to disclose training data that is protected by copyright. Require a sufficiently detailed summary of the use of training data protected by copyright law to be recorded and publicly available without compromising the copyright legislation of the European Union or member states.

2.2 Risk classification regulation of artificial intelligence systems based on basic models

The proposed parliamentary version continues and improves the risk-based regulatory approach of the proposal. In addition to complying with the specific obligations mentioned above, basic model providers will also be subject to the risk classification regulations of artificial intelligence systems once the basic model is deployed and integrated into them. Except for the minimum risk scenario, artificial intelligence systems based on basic models are subject to varying degrees of regulation when their specific use causes unacceptable, high, and limited risks.

2.2.1 Prohibit artificial intelligence systems that pose unacceptable risks

Article 5 of the proposed parliamentary version clearly lists the artificial intelligence systems that should be prohibited, because EU legislators believe that these systems pose an "unacceptable risk" to human security. In the initial proposal, these systems included the use of subconscious technology, exploitation of people's weaknesses, and systems for social scoring. On this basis, the proposed parliamentary version has significantly supplemented and improved the list of prohibited artificial intelligence practices and systems to prohibit the manipulative, invasive, and discriminatory use of artificial intelligence systems, mainly including systems that use purposeful manipulation or deception techniques; Emphasize systems that exploit people's weaknesses, including those that utilize known or predicted personality traits or socio-economic conditions; Real time remote biometric recognition system in public places; "After the fact" remote biometric recognition system, unless authorized by the judiciary and necessary for law enforcement to prosecute serious crimes; A biometric classification system that uses sensitive features such as gender, race, religion, political orientation, etc; Predictive policing system (based on profiling, location, or past criminal behavior); Emotional recognition systems in law enforcement, border management, workplaces, and educational institutions; A system that indiscriminately captures facial images from social media or closed-circuit television recordings to create a facial recognition database.

2.2.2 Full lifecycle obligations of high-risk artificial intelligence systems

EU legislators believe that artificial intelligence systems that pose a high risk can be launched in the European market, but must comply with certain mandatory requirements and undergo prior compliance assessments. Article 6 of the proposal clarifies two major categories of high-risk artificial intelligence systems: first, systems used as product safety components or applicable to Annex 2 EU health and safety coordination legislation (such as systems in the automotive and medical device fields); The second is the system deployed in the eight specific areas identified in Annex 3, which the European Commission can update as necessary through authorized legislation. Eight specific areas are: biometric recognition and classification of natural persons; Management and operation of critical infrastructure; Education and vocational training; Employment, worker management, and self employment; Access and enjoyment of basic private and public services and benefits; Enforcement; Immigration, asylum, and border control management; Judicial administration and democratic procedures.

The proposed parliamentary version continues this classification rule, completely retaining Annex 2 of the proposal, but significant modifications have been made to the identification of high-risk systems in the second category and the content of Annex 3. The European Parliament believes that systems belonging to the eight specific areas of Annex 3 are not automatically classified as high-risk systems, but must meet additional limiting conditions, namely "posing a significant risk of harm to the health, safety, or basic rights of natural persons", in order to be considered high-risk systems. Furthermore, the expressions in various fields of Annex 3 have been further supplemented and improved. The first field has been modified to "biometrics and biometric based systems", and some new high-risk systems have been added under each field, including emotion recognition systems beyond the provisions of Article 5, systems for evaluating personal education and vocational training levels, systems for determining personal health and life insurance qualifications, etc. Especially, systems that affect voter voting in political elections and systems recommended by large-scale social media platforms have been included.

The third part of the proposal sets out obligations and requirements for high-risk artificial intelligence system providers throughout the entire system lifecycle, including risk management, data and data governance, technical documents, record keeping, transparency and information provision, human supervision, technical reliability (Articles 9-15), as well as quality management, compliance assessment, stakeholder obligations, etc. Other parts also stipulate obligations such as registration in the EU high-risk system database, post market monitoring, reporting of serious accidents, etc. The parliamentary version of the proposal continues these provisions and further improves relevant expressions.

2.2.3 Limited risk artificial intelligence systems have transparency obligations

EU legislators believe that artificial intelligence systems that pose limited risks mainly involve specific manipulation risks. In order to prevent people from being manipulated and to make informed choices or retreat decisions, these systems should have transparency obligations. Article 52 of the proposal lists three types of such systems and corresponding transparency obligations: firstly, systems that interact with humans should be informed that they are interacting with artificial intelligence systems; The second is a system for emotion recognition or social classification based on biometric data, which should inform the operation of the system; The third is a system that generates or manipulates content such as images, audio, or video ("deep forgery"), and should disclose that the content is artificially generated or manipulated, except for legitimate purposes such as legal authorization. The proposed parliamentary version continues this provision and improves the corresponding notification content based on the characteristics of different systems.

2.3 Experience and Mirror of EU Legislative Exploration

In summary, EU legislators have timely paid attention to and responded to the risk challenges in the current wave of general artificial intelligence development driven by large models: based on the risk grading regulation method, with the large model as the center, different obligation requirements have been set for basic models, generative basic models, and artificial intelligence systems based on basic models, and a hierarchical supervision scheme for general artificial intelligence has been proposed. However, from the latest parliamentary version of the proposal, it can be seen that the obligations undertaken by basic model providers, in addition to environmental protection obligations and the obligation to prevent the generation of illegal content and disclose copyrighted training data in generative situations, are basically consistent with those undertaken by high-risk artificial intelligence systems in terms of risk management, data governance, technical reliability, information provision, quality management, model registration, and other aspects. This indicates that EU legislators actually classify the basic model itself as high-risk, and regulate the basic model similar to high-risk artificial intelligence systems.

However, from the requirements of risk management and technical reliability, it is not feasible to regulate the basic model with this approach. Considering the universality of the basic model, in terms of risk management, it is required to identify, reduce, and mitigate all reasonably foreseeable risks, at least by considering and analyzing possible risks in all high-risk uses listed in Annex 3 of the Artificial Intelligence Act, and then developing and implementing mitigation measures for all these risks based on this; In terms of technical reliability, it is required to achieve and evaluate appropriate levels of performance, safety, etc., which requires reliability testing and evaluation for all high-risk applications in these technical aspects. Meeting such regulatory requirements requires incalculable costs and costs, which is a task that is basically impossible to accomplish. Moreover, it is unnecessary to have large model providers implement risk mitigation and reliability assurance for all hypothetical possible uses, many of which may not ultimately be realized.

Upon closer examination, the risk classification regulation method is determined based on the specific use of artificial intelligence systems. EU legislators directly regulate large models with universality according to high-risk artificial intelligence systems, without considering the different risks of the actual specific use of large models, nor seeing that it is often the deployers of large models rather than the providers who determine their actual use, let alone the two different types of risks caused by the training and deployment of large model data. This kind of one size fits all legislative plan actually deviates from the original intention of risk grading regulation. We should start with the characteristics of big model technology and new challenges, and explore practical and effective risk regulation solutions for big models that meet the needs of the development of general artificial intelligence.

3. Regulatory Framework for New Risks in Generative Artificial Intelligence Large Models

In July 2023, the National Cyberspace Administration and seven other departments formulated the "Interim Measures for the Management of Generative Artificial Intelligence Services" (hereinafter referred to as the "Interim Measures"), which focuses on providers of generative artificial intelligence services to the public within China using generative artificial intelligence technology, and excludes enterprises and research institutions that only "research and apply generative artificial intelligence technology". This actually distinguishes between service providers and technology providers of generative artificial intelligence. If the latter does not provide generative services to the domestic public, the Provisional Measures are not applicable. So for generative artificial intelligence services based on big model technology, the Provisional Measures regulate only big model deployers and do not regulate pure big model providers. This regulatory approach, which focuses on the deployment and application of large models and encourages training and development of large models, adheres to the principle of balancing development and security, which is conducive to the innovative development of China's large model and general artificial intelligence industries and deserves high recognition.

However, from the previous analysis, it can be seen that the new risks caused by large models are rooted in the model training stage, and the universality of large models will transmit these risks to downstream applications. Dealing with these risks cannot be separated from the communication and cooperation of upstream and downstream participants in the value chain of large models. From this perspective, apart from the situation where the provider of the large model is itself the deployer, the current provisions of Article 7 of the Interim Measures, such as the legality requirements for training data, and Article 14 of the Measures for "Model Optimization Training" rectification measures, only apply to downstream large model deployers. They only regulate small-scale data training when deployers adapt to large models. In reality, they cannot control the upstream large model base and its risks, and cannot fully solve the risks and challenges brought by large models. Therefore, from the perspective of risk prevention and development needs of generative artificial intelligence models, there is still significant room for improvement in the Interim Measures.

Based on the technical characteristics of the large model, and in response to the new risks it causes, combined with the legislative progress at home and abroad, this article proposes the following suggestions on the basic ideas and institutional framework for regulating the risks of the large model.

3.1 Basic ideas for risk regulation in large-scale models

The new risks caused by the regulation of big data models, a new way of utilizing big data, should follow the paradigm of data utilization security, adopt a risk based classification and grading regulation method, and achieve cooperation and governance among upstream and downstream participants.

3.1.1 Follow the paradigm of data utilization security

Big models are trained using big data to refine parameters, and their training, development, and deployment are a new way of utilizing big data, which poses new risks and challenges. The new risks of regulating large models should follow the paradigm of data utilization security. The Data Security Law of the People's Republic of China has established a new paradigm for data security, which requires "ensuring that data is in an effective protection and lawful utilization state", ensuring both the "self-security" of traditional data and the "utilization security" of large-scale data flow and mining. The key to the data utilization security paradigm, in my opinion, is to ensure the controllability and legitimacy of large-scale data flow and utilization. For large models, their data utilization includes two aspects: first, during the model training phase, data extraction of patterns and patterns from big data is used for training, and the training results are presented as large models, especially the algorithms and parameters within them; The second stage is the model deployment phase, which generates new content based on the data training results, i.e. the large model, in response to data input. In short, the data utilization of large models includes using data to train the large model and reusing the training results of the data to generate content. As can be seen from the previous text, the two new types of risks of large models, namely the risks caused by model data training and the risks caused by model deployment and application, are rooted in the utilization of data in these two aspects.

Following the paradigm of data utilization security to regulate the new risks of large models requires ensuring both the controllability and legitimacy of data training for large models, as well as the controllability and legitimacy of the deployment and application of large models. Specifically, during the model training phase, on the one hand, the collection and aggregation of training data should be controllable, and sensitive high-risk data such as important data and personal identification information should not be included in the training data; On the other hand, considering that the results of data training will determine the content of model generation, the source and content of training data should be legitimate, such as excluding biased and discriminatory data as much as possible. For the model deployment phase, on the one hand, the specific deployment of the model should comply with controllability. For example, the model should not be used in areas with unacceptable risks, and corresponding risk control obligations should be stipulated when using the model for high-risk purposes; On the other hand, the purpose and generated content of the model should be legitimate, and the model should not be used for illegal purposes. It should be prevented from generating illegal content.

3.1.2 Based on risk classification and grading regulations

The Provisional Measures emphasize the need to implement inclusive, prudent, and classified supervision. The author believes that the key to implementing this principle is to implement risk based classification and grading regulations for large models. The risk classification of large models should be mainly distinguished based on the root causes and risk areas of the large model. It can be divided into traditional risks inherent in the large model system and new risks caused by the utilization of large model data. The latter, as mentioned earlier, includes various risks caused by model data training and various risks caused by model deployment and application. For the regulation of the former, there are currently specialized provisions in relevant laws and regulations such as the Cybersecurity Law of the People's Republic of China (hereinafter referred to as the Cybersecurity Law); For the regulation of the latter, it is necessary to follow the above data utilization security paradigm and explore institutional design to ensure the controllability and legitimacy of big model data training and deployment applications. The risk classification of large models should be mainly determined based on the actual risk level of specific uses of the large model. Drawing on the proposal of the European Union's Artificial Intelligence Law, risks can be divided into four levels: unacceptable risk, high risk, medium risk, and low risk.

Risk based hierarchical regulation needs to focus on the practical use of large models and match different regulatory methods based on risk levels. Risk management generally includes four strategies: accepting risks, avoiding risks, controlling risks, and transferring risks. For the use of large models with unacceptable risks, efforts should be made to avoid risks and strictly prohibit them in principle; For high-risk and medium risk applications of large models, emphasis should be placed on controlling risks and defining risk control obligations that are appropriate to the risk level. Due to the impossibility of achieving absolute security, for low-risk applications of large models and residual risks that still exist after taking measures to avoid and control risks, an appropriate strategy is to accept the existence of risks, which is also the principle of "emphasizing both development and security". From this perspective, whether it is the EU's proposal for the parliamentary version of the Artificial Intelligence Law to regulate large models as high-risk systems in a one size fits all manner, or the draft of the Interim Measures that required the generation of content to be "truthful and accurate", there is no hierarchical regulation of risks, and there is a suspicion of pursuing absolute safety, which does to some extent overlook the balance between risk prevention and industrial development.

3.1.3 Realize cooperation and co governance between upstream and downstream

Proper response and regulation of risk transmission from large models to downstream applications cannot be achieved without the joint efforts of upstream and downstream participants in the value chain of large models. To determine the risk control obligations that upstream and downstream participants should bear, it is necessary to clarify the different roles of relevant entities and their control levels over the large model when it is used for downstream tasks. At present, there are two main ways for downstream large model deployers to call large models: open source access and API (application programming interface) access. In the case of open source access, the provider will publicly disclose the parameters and source code of the model, and the deployer can directly check the source code and parameters and make modifications and adaptations according to the open source license. In the case of API access, the provider only provides the deployer with the API call interface for the large model. The deployer can use some training data to fine tune the model to adapt to downstream tasks, but has no authority to modify the model's source code and parameters. But whether it's open source access or API calls, it's the deployer who decides the actual use of the big model.

It can be seen that for large model deployers, in API mode, the source code and parameters of the large model are still completely controlled by the provider, and they cannot know the underlying technical details of the large model, nor can they cope with risks by modifying the large model. Even if the model source code and parameters can be modified in open source mode, considering the inexplicability of the large model algorithm and emergence ability, they cannot fully understand and control the risks brought by upstream data training of the large model. For large model providers, they cannot intervene in the data training of the deployer's adaptation model, nor can they intervene in the deployer's decision on the actual use of the large model, nor can they control the risks caused by model adaptation data training and model deployment applications. Therefore, it is not feasible to comprehensively control the risks of large model applications solely by deployers or providers, and it is necessary to achieve cooperation and co governance between the two. From this perspective, whether the Provisional Measures only focus on risk prevention from the perspective of large model deployers, or the European Union's proposed parliamentary version only assigns the responsibility of risk management to large model providers, they are all allowing them to complete impossible tasks and cannot actually achieve the goal of comprehensive control of large model risks. In addition, the end users of content generated by artificial intelligence systems based on large models determine the specific content and content audience generated, and users and content audiences are also important entities involved in managing the risks of content generated by large models. In order to achieve collaborative governance of large model risks, these upstream and downstream participants need to engage in sufficient risk communication and information sharing, as well as collaborate to take necessary risk response measures.

3.2 A New Risk Regulatory Framework for Large Models

Based on the above basic ideas, regulating the new risks caused by big models focuses on ensuring the controllability and legitimacy of big model data training and deployment applications. Based on the progress of domestic and foreign legislation, this article proposes the following regulatory framework.

3.2.1 Establish specialized agencies to guide development, evaluate, and respond to risks

It is recommended to establish a specialized artificial intelligence regulatory agency, named the "Artificial Intelligence Development and Security Committee", based on foreign experiences, to comprehensively monitor and respond to the risks and challenges brought by technological paths such as large models, and guide and promote the development of artificial intelligence security. At the end of 2018, the United States established the National Artificial Intelligence Security Committee under the 2019 Defense Authorization Act, responsible for reviewing the development of artificial intelligence, machine learning, and related technologies to comprehensively address the national security needs of the United States. The European Parliament proposes the establishment of a European Artificial Intelligence Office to ensure the effective and coordinated implementation of the law. Its responsibilities clearly include the regulation of large models: providing special supervision and monitoring, establishing a regular dialogue system with basic model providers on whether the basic models and artificial intelligence systems utilizing these models are compliant, and best practices in industry self-governance; Record and monitor the operation of known large-scale training of large models, as well as publish annual reports on the development, diffusion, and use of basic models, and attach policy choices to address unique risks and opportunities of basic models.

The author believes that specialized agencies can focus on two aspects of risk supervision for large models, in addition to supervising the implementation of legal obligations: first, organizing mandatory risk assessments for high-performance large models. The current EU proposal for large-scale model risk assessment mainly involves self-assessment by providers and deployers. The security assessment of generative artificial intelligence services with public opinion attributes or social mobilization capabilities mentioned in Article 17 of the Provisional Measures also belongs to self-assessment, focusing only on information content security risks. Considering the potential risks of large models that may have a significant impact, to avoid unbearable consequences, it is recommended that specialized agencies organize relevant experts in the field to conduct mandatory third-party risk assessments on certain high-performance large models before their launch. The scope of this mandatory evaluation should be limited to large models with strong capabilities, which can be defined from the performance indicators of large models, such as those with parameters exceeding 100 million or computational complexity exceeding a certain threshold. Given that the evaluation of large model technology is just beginning to explore, specialized agencies should be responsible for organizing research and development of evaluation methods and standards for large models. Based on the results of risk assessment, specialized agencies can organize the development of risk response measures for large models. The second is to strengthen the monitoring, research, and response to long-term risks in large models. As mentioned earlier, the normal use of large models may also pose certain risks to the environment and socio-economic development, such as their impact on the environment and employment. However, current research shows that these risks have not yet posed an urgent threat and can be referred to as a long-term risk. For such long-term risks, the current appropriate response strategy is to accept the risk, but it is also necessary to strengthen monitoring and research. Once it is discovered that it has transformed into a real threat, timely adjustment of response measures should be made. For monitoring purposes, large model providers should have an obligation to regularly report on resource consumption and other situations.

3.2.2 Standardize data training to avoid data leakage and improper output

As can be seen from the previous text, the new risks caused by regulating big model data training are to ensure the controllability and legitimacy of data training, minimize data leakage and the generation of harmful or even illegal content caused by data training, and improve the accuracy and reliability of generated content. The data training of big model providers and deployers should meet the following requirements:

One is to ensure that the collection and aggregation of training data are controllable. Due to the fact that large models may "memorize" some of the training data and their powerful reasoning ability can analyze sensitive information, it is advisable to avoid using sensitive high-risk data as training data to minimize sensitive data leakage. The author believes that important data related to national security and public interests, privacy and personally identifiable information related to personal rights, and trade secrets related to business interests of enterprises should not be used as training data for public large models in principle.

The second is to clearly define high-risk and medium risk uses and establish corresponding risk control obligations. The author believes that high-risk uses mainly refer to uses that may endanger national security, economic operation, social stability, public health and safety, etc. The parliamentary version of the EU proposal defines high-risk uses using the method of "specific domain listing+abstract element recognition", which lists systems that may have high-risk uses in eight areas such as key infrastructure management and operation. Then, based on the limiting conditions of "posing a significant risk of harm to the health, safety or basic rights of natural persons", it specifically determines whether it constitutes high-risk uses. This definition method can be used as a reference for China. For deployers who decide on high-risk uses of large models, drawing on EU proposals, their risk control obligations can be defined from aspects such as risk management, transparency, record keeping, and technical reliability. In terms of risk management, it is recommended to refer to Article 29a of the European Union's parliamentary version of the proposal regarding the deployment of high-risk systems, which requires deployers to establish a risk assessment mechanism involving multiple stakeholders, including those affected, in order to assess the basic rights and impacts of high-risk system deployers. As for medium risk applications, drawing on the understanding of "limited risk" in the European Union, the author believes that they mainly refer to applications where people may be misled and manipulated by automation systems due to opaque operation, which may endanger human autonomy. To avoid such risks, it should be stipulated that the deployer has a certain transparency obligation to inform the user of the existence and operation of the artificial intelligence system, ensuring that people have the right to choose whether to use the system after being informed.

3.2.3 Building a risk classification and control system based on specific use risks

As mentioned earlier, the new risks arising from regulating the deployment and application of large models are to ensure the controllability and legitimacy of the deployment and application of large models. The main target of regulation is the deployers of large models. To ensure controllability, it is necessary to classify and control the risks of specific uses of large models. According to the above suggestion, different degrees of regulation should be applied to unacceptable risks, high risks, medium risks, and low risks. From this perspective, it is reasonable for the Provisional Measures to focus on regulating deployers, but they only propose the principle of graded supervision and do not specify risk graded regulation. There is still significant room for improvement in risk control. According to the aforementioned risk control strategy, low-risk uses should be allowed, unacceptable risk uses should be prohibited, and appropriate risk control obligations should be set for high-risk and medium risk uses. The latter two aspects are the focus of the hierarchical control system. Specifically:

One is to clarify the "prohibited list" for deploying large model applications. The prohibition list mainly aims to clarify the use of unacceptable risks, and in my opinion, it mainly refers to the use that may seriously endanger national security, public safety, and major public interests, causing consequences that are difficult to bear. This acceptability often depends on a country's core values, national interests, cultural traditions, etc., and different countries may have different definitions. From the definition of the EU proposal, it is based on the EU's values of safeguarding basic rights, with a focus on prohibiting manipulative, invasive, and discriminatory uses. The use of subconscious or deceptive techniques, exploiting the weaknesses of vulnerable groups and other manipulative behaviors mentioned in the EU definition can be used as a reference for China to define such prohibited uses; However, the predictive policing and other purposes that the EU intends to prohibit have already caused great controversy in the EU legislative process, and whether they should be included in China's prohibited list needs further argumentation. It should also be pointed out that the EU proposal excludes military use from the scope of application due to legislative power limitations. The author believes that China's future legislation should clearly prohibit the use of large models for military purposes such as autonomous weapon systems and nuclear deterrence.

The second is to clearly define high-risk and medium risk uses and establish corresponding risk control obligations. The author believes that high-risk uses mainly refer to uses that may endanger national security, economic operation, social stability, public health and safety, etc. The parliamentary version of the EU proposal defines high-risk uses using the method of "specific domain listing+abstract element recognition", which lists systems that may have high-risk uses in eight areas such as key infrastructure management and operation. Then, based on the limiting conditions of "posing a significant risk of harm to the health, safety or basic rights of natural persons", it specifically determines whether it constitutes high-risk uses. This definition method can be used as a reference for China. For deployers who decide on high-risk uses of large models, drawing on EU proposals, their risk control obligations can be defined from aspects such as risk management, transparency, record keeping, and technical reliability. In terms of risk management, it is recommended to refer to Article 29a of the European Union's parliamentary version of the proposal regarding the deployment of high-risk systems, which requires deployers to establish a risk assessment mechanism involving multiple stakeholders, including those affected, in order to assess the basic rights and impacts of high-risk system deployers. As for medium risk applications, drawing on the understanding of "limited risk" in the European Union, the author believes that they mainly refer to applications where people may be misled and manipulated by automation systems due to opaque operation, which may endanger human autonomy. To avoid such risks, it should be stipulated that the deployer has a certain transparency obligation to inform the user of the existence and operation of the artificial intelligence system, ensuring that people have the right to choose whether to use the system after being informed.

3.2.4 Establish a transparency system that runs through the entire lifecycle of the large model

To effectively address the new risks of large models, upstream and downstream participants need to face the inexplicability of large models, strive to understand the principles of training, deployment, and operation output of large models, as well as engage in sufficient risk communication and information sharing to facilitate collaborative risk management. The key to achieving these two aspects is to establish a transparency system that runs through the entire lifecycle of the large model. Specifically, it includes three aspects:

One is the obligation of information disclosure for large model providers and high-risk deployers. From the perspective of EU legislation, the proposal initially only stipulated that high-risk artificial intelligence systems have transparency obligations. The parliamentary version of the proposal clearly requires that the basic model should be registered in the EU's high-risk artificial intelligence system database and relevant information should be disclosed in accordance with Annex 8 requirements. The author agrees with this approach, and it is necessary to maintain a certain level of transparency regardless of whether the large model is used for high-risk purposes. Drawing on the regulations of the European Union and taking into account the previous discussion, the author believes that large model providers should disclose the following information and keep it updated: basic information such as provider names; The source of training data for large models; The ability, limitations, and reasonable and predictable risk mitigation measures of the large model; The computational power required for large model training and the potential impact on the environment; The performance of large models based on public or industry benchmarks; Explanation of internal and external testing and optimization of the large model. Deployers of high-risk models should also refer to these contents to disclose the deployment and application of the model, and emphasize the expected use, limitations, potential risks, and mitigation measures of high-risk systems. In terms of information disclosure, China can learn from the European Union to establish publicly accessible databases.

The second is the transparency obligation of risk use deployers and users in the big model. In addition to the transparency obligation of risk deployers in the aforementioned large model, users of medium risk systems based on the large model should also have a certain degree of transparency obligation towards those affected by the system. For example, Article 52 of the parliamentary version of the EU proposal adds a new provision: when users of systems interacting with humans make decisions using the system, they should inform the person in contact with the system, who is responsible for the decision-making process, as well as existing rights and procedures. These rights and procedures allow opposition to the application of the system and seeking judicial remedies for decisions made or damages caused by the system, including the right to seek explanation; Users of emotion recognition systems or biometric classification systems that are not prohibited should obtain the consent of those who have access to the system before processing biometric data and other personal data; Users of the "deep forgery" system should disclose in an appropriate, timely, clear, and visible manner that the content is artificially generated or manipulated.

The third obligation is for upstream participants in the value chain of the large model to provide necessary information to downstream participants. Large model providers should provide necessary technical documents and usage instructions to deployers and high-risk users to support the normal operation and legal use of downstream artificial intelligence systems, especially in compliance with regulatory requirements for high-risk systems. Annex 4 of the EU proposal stipulates that this information includes a general description of artificial intelligence systems, detailed descriptions of elements and development processes, detailed information on operation and control, and detailed descriptions of risk management. The proposed parliamentary version believes that it should also include: the main objectives of the system, output quality, and output interpretability; The structure, design specifications, algorithms, and data structures of the system, as well as their interrelationships and overall logic; Appropriateness of specific system performance indicators; The energy consumption of system development and the expected energy consumption used. The author believes that the provision of necessary information should consider the agreed upon utilization of large models among upstream and downstream participants and strike an appropriate balance between technical information sharing and trade secret protection.

3.2.5 Establish a sound upstream and downstream co governance mechanism to prevent the generation of illegal content

The risks arising from the deployment and application of large models should not only ensure the controllability of specific deployments, but also ensure the legitimacy of specific uses. This requires both prohibiting the use of large models for illegal purposes and preventing the generation of illegal content by large models. The latter involves the new challenge of machine generated illegal content, which is the key to regulating the new risks of large models. In response to the information content security risks associated with large model generated content, both the European Union's proposed parliamentary version and the Provisional Measures explicitly require the prevention of illegal content generation. The reason why large models generate illegal content may be due to harmful outputs of models with training data defects, or it may be intentional deployment and misuse of large models to generate illegal content. Therefore, preventing large models from generating illegal content requires cooperation and joint governance between upstream and downstream. The data standardization training obligations of the aforementioned large model providers and the risk control obligations of the large model deployers are powerful measures to minimize the generation of illegal content by large models.

But if an artificial intelligence system based on a large model still generates illegal content, how should it be detected and dealt with in a timely manner? In response to this, the Provisional Measures require that generative service providers, namely large model deployers, should bear the responsibility of network information content producers, take timely disposal measures such as stopping generation and model optimization training when illegal content is found, and report to relevant regulatory authorities. However, upon careful consideration, although the deployer provides a generative service, it is based on the technology of a large model provider. The user who issues instructions to determine the specific generated content and audience is the service's user, who is the most important content producer among them. Only identifying the deployer of the large model as a content producer is actually making the deployer directly responsible for the user's behavior of generating content. Considering that users can generate massive amounts of content using generative services, this requirement seems too strict; Allowing deployers to take corrective measures for model optimization training, but apart from open-source models and other situations, deployers often cannot optimize and modify upstream large models, and in reality, they are unable to achieve the rectification goals.

This indicates that relying solely on deployers is difficult to accomplish the important task of discovering and disposing of illegal content, and a joint governance mechanism for discovering and disposing of illegal content should be further improved. One is to improve the joint governance mechanism for discovering illegal content. At present, the Provisional Measures emphasize that deployers have the obligation to "discover" and dispose of illegal content. However, considering that users will generate massive amounts of content, it is almost impossible to require manual review one by one. Therefore, referring to the provisions and understanding of Article 47 of the Cybersecurity Law, the author believes that the obligation of deployers to discover illegal content cannot be understood as a universal review obligation for all generated content. Discovering illegal content requires the joint participation of deployers, users, and regulatory authorities. In addition to the deployers having a certain "proactive" discovery obligation, that is, they should adopt manual review and supervision, identification and filtering measures based on existing technological levels, actively search for illegal content, and should also facilitate the reporting mechanism of illegal content, improve the reporting and inspection mechanism of regulatory authorities, Mobilize users to actively report illegal content; Deployers should promptly dispose of illegal content that is passively obtained through user reports, notification from regulatory authorities, and other means. The second is to improve the joint governance mechanism for the disposal of illegal content. When the deployer finds that they are insufficient to prevent the generation of illegal content, they should promptly inform the large model provider of the relevant situation, and the provider should take measures such as modifying model parameters, optimizing model training, etc. for rectification, and report to the relevant regulatory authorities. Of course, if it is found that the generation of illegal content is caused by the deployer adapting the model, it should be rectified by the deployer through measures such as model adaptation optimization training, and the responsibility for rectification should not be imposed on upstream providers.

The original text was published in Administrative Law Research, 2024, Issue 2. Thanks for the authorization of the WeChat official account "Editorial Department of Administrative Law Research".