This book is the result of a one-day seminar on 'Safety and Risks in the Use of Expert Systems' which was held in Copenhagen on May 19th, 1988. The idea to have a seminar about this topic evolved from the discussions in the 'Sub-committee for Computer Systems' working under the auspices of the 'Committee on Risk Assessment'. The 'Committee on Risk Assessment' was appointed by a resolution in the Council of the Danish Academy of Technical Sciences in February 1985 and started its work in June the same year - so far for a period of three plus one years - with the following terms of reference:

• The object of the Committee is to promote risk assessment research and the application of its results in Denmark.
• During fulfilment of this object, the Committee must initiate and co-ordinate research projects and co-ordinate the exchange of information between creators and users of the results of risk assessment research within all professional disciplines, including ensuring improved information about international risk assessment research in Denmark.
• The Committee is primarily meant to act as promoter and co-ordinator of activities carried out under other auspices.

During the discussions in the sub-committee about the risk involved in the use of computer systems, it became clear that expert systems as a new type of computer systems might also represent a new, and potentially very large, source of risks. The background for the seminar was a discussion about how expert systems have changed from being a subject for research to the position where they are in practical use as advice and support systems in industry and commerce. The essence of an expert system is the ability to give advice and support in cases where the problems are incompletely understood and very unstructured. This naturally raises some concerns about the safety and reliability of their functioning. At the time there seemed to be little in the way of an established position on how to answer these questions, and the seminar was therefore organised to provide an overview of the main issues and current trends.
The seminar was organised into two sessions which focused on the theoretical and practical aspects of safety and risk in the use of expert systems. Each session had two invited speakers, and the book presents the extended version of these four prepared contributions, as well as an introduction to the subject and a summary based on the discussions that took place during the seminar.
Acknowledgement is due to a number of people. First of all, the four invited authors who accepted the challenge of presenting their view on Safety and Risk in the Use of Expert Systems and who afterwards took the trouble together with their co-authors to revise, and considerably extend, their presentations. One of them, Robert J. Taylor, also served as a member of the organising committee - in addition to being the driving force in the sub-committee on computer systems. The other members of the organising committee were Professor Palle Thoft-Christensen, Aalborg University Center, and the scientific secretary of the 'Committee on Risk Assessment', Ms. Birthe Schouby, who also took care of the practical arrangements of the seminar. Thanks are due to Dr. Jens Langeland-Knudsen who together with Professor Thoft-Christensen chaired the sessions, and to Mr. Pierre Laraignou, who made sure that the presentations and question & answer sessions were properly tape recorded, and who later did a good job in transcribing parts of the recordings. And I would finally like to thank the 'Committee on Risk Assessment' with its chairman, Mr. Niels Hjort, who from the beginning fully supported the idea of organising a seminar on this topic.

Table of Contents

• Erik Hollnagel: Preface
• Jens Langeland-Knudsen: Introduction
• Erik Hollnagel: The Reliability Of Expert Systems - An Inquiry Of The Background (1. Introduction; 2. The Development in the Use of Expert Systems; 3. Reliability and Risk; 4. Safety and risks in the Use of Expert Systems.)
• J. Robert Taylor: Control Of Software Reliability (1. Introduction; 2. Expert Systems and Process Control; 3. Software Reliability; 4. The Proper Use of Expert Systems.)
• Didier Dubois & Henri Prade: Handling Uncertainty In Expert Systems - Pitfalls, Difficulties, Remedies (1. Introduction; 2. Certainty Factors in Pattern Directed Inference Systems; 3. Semantics of Uncertain or Vague Facts; 4. Semantics of Weighted Production Rules; 5. Limitations of Inference Engines; 6. Current Trends in Approximate Reasoning; 7. Conclusion.)
• Poul Raaholt Olsen: Safety And Risks In The Use Of Expert Systems (1. Introduction; 2. Benefits of the Knowledge-Based Approach; 3. An Example; 4. Risks of the Knowledge-Based Approach; 5. Steps Towards Minimising Risk.)
• Giovanni Guida & Luca Spampinato: Assuring Adequacy Of Expert Systems In Critical Domains: A Constructive Approach (1. Introduction; 2. The Concept of Adequacy: The Key Issue for Expert System Application in Critical Domains; 3. Assuring Adequacy through an Appropriate Development Methodology; 4. Knowledge Monitor: A Development Support Tool for Assuring Adequacy; 5. Conclusion; 6. Notes.)
• Erik Hollnagel: Issues In The Reliability Of Expert Systems (1. Introduction; 2. Evaluation of Expert Systems; 3. Knowledge Representation and Reasoning Mechanisms; 4. On The Use of Expert Systems.)