Date First Published: September 24, 2019
Date Last Revised: July 12, 2020

Audit in General

The term audit, as used traditionally, refers to the examination of records or financial accounts to check their accuracy. Currently, ‘audit’ has a more general meaning i.e. the examination or evaluation of the status of adherence to prescribed methods and/or the achievement of desired standards in certain aspects of a facility, organisation, business or endeavor.

In healthcare services, audit as an assessment of quality consisted of many activities. Traditionally, the audit method has been applied primarily to the appraisal of outcome. However, because the quality of outcome is dependent on many factors, the quality level of these factors and their effect on the service delivery system has to be considered and therefore audited together or as separate activities.

Assessment of the quality of outcome of a particular service starts from the measurement of the quality of outcome of an individual case and then combined to reflect the quality of outcome of the entire service.

Meaning of Medical Audit

Medical audit is a distinct form of audit used by health care providers. It’s method consists of the analysis of retrospective data obtained from records. Quality assessments by planned collection of data are not audits but should be called prospective quality studies.
In an audit, assessment is done on recorded data originating from the entire patient population or a sub-population made up of those belonging to a defined category. However, the whole exercise still starts with studying the quality achieved in individual cases. As such it is a variant of statistical quality control as used in other industries. It can be defined as below:
“Medical audit is a systematic, critical analysis and quantified comparison against explicit standards, of the quality of current medical practice, including the procedures used for diagnosis and treatment, the use of resources, and the resulting outcome and quality of life for the patient in order to improve the quality of care to patients.” (definition modified from Department of Health(1989) Working for patients, Working paper 6, command 555, London: HMSO.

Objectives and Benefits of Medical Audit

The motivation to carry out audits arise from the desire of providers of care to know the level of quality of the services they provide and to improve it to a higher level. 

The main objectives of carrying out an audit is to know whether the provider of care has achieved the desired quality characteristics of its service. The results are used to:

• provide comparison with previous and future performance:
• allow comparison with other care providers
• know the degree of inadequacy or shortcoming
• provide feedback for quality improvement

Hence, medical audit may be performed by various auditors depending on who s being audited as shown below:

UNIT BEING AUDITED	AUDITOR
Work (cases) of a care provider	the provider him/herself
Services provided by a Unit or Team	– the team on their own – voluntary review by colleagues (Peer review) – unit manager or supervisor
Services provided by a facility	– Facility Manager – Enterprise manager – External audit team

Role of Medical Audit in Quality Management

Audit is a form of quality assessment that gives an indication on the level of quality achieved for a given service for a defined group of patients in a given period. It is an assessment of past events. The main value is in using lessons learned from it to plan the improvement of service delivery in the future. It is not a very useful as a tool for Quality Control because it cannot prevent poor quality service at the time of its delivery. Indeed, the realization of occurrence of poor quality may be made too late to allow for corrections to be made. If it to be taken as Quality Control it can be equated with after-sales assessment just like customer satisfaction questionnaires. In the manufacturing industry, the goods can be recalled and then repaired or replaced. With patient care, unlike in other services, the same cannot be done because the unwanted outcome cannot be undone. The correction is limited only to damage control e.g. managing the complications that arise.

Preferably, quality control of services is done before or while the the processes are being performed. For example, the care provider may be given guidance on how to proceed or be forewarned on dangers or risks involved in carrying out a process or using a drug or instrument. Another important method of control much used in patient care is supervision.

Factors That Affect Quality

The system that describes the factors involved in manufacturing or service delivery is often called the Systems Theory of Operations Management. It is as depicted below:

A good outcome is the result of the transformation of inputs by various processes. In patient care, it is dependent on

• the initial condition of the patient,
• adequacy and condition of machines, instruments and material used
• how well the process is carried out
• knowledge and skills of care providers
• presence of a conducive environment
• how well patients comply with instructions and advice.

Criteria for Assessment in Audit

In an audit, the quality level of the outcome as well as the factors that affect on the service delivery becomes the criteria by which judgments are made. Each item together with the characteristics being looked at is a criterion. Hence, for each item the characteristics to be appraised is determined by the Audit team. Then for each characteristic or set of characteristics, the actual conformance or achievement will be compared with a desired value called standards. Standards may be set by the Audit team itself or universally accepted standards can be used.
The relevant characteristics to look at are different for each input and process. If a standard operating procedure (SOP) and/or a business plan is available then the criteria, characteristics and standards can be derived from them. The standard for the quality of output or outcome are set as objectives or targets in the SOP. However, the Audit team may set their own standards of outcome but these should be stipulated clearly.

CRITERIA	CHARACTERISTICS
Input – Facility – Equipment – Human resource – Raw material – Information	Adequacy, Availability, Functional (in working order)
Processes – Technique – Communication – Decision making – Information management – Documentation	Efficiency, Conformance (Performed correctly) Flexibility/Responsiveness Safety
Output / Outcome	Desired level achieved Targets met

An auditor must have in hand a set of written criteria i.e. the items to look at in the form of a questionnaire or other documentation. Each criterion and its characteristics can be set in an explicit or implicit way.

Explicit Criteria

If the characteristics and standards of a criterion can be expressed accurately it is said to be explicit. Usually these can be counted or measured or otherwise specified in a precise manner. For example, items such as facility, equipment, machines and instruments can be assessed objectively as to their characteristics of existence, adequacy and being in working order. For a process, the main characteristic would be whether it is performed in the manner prescribed in the SOP. The finding could be degrees of conformance (full, partial or non). The standard aimed at is full conformance

In the care of a patient suffering from an illness the outcome is defined as the resultant effect on the patient and the disease which can be cure, restoration of function or provision of comfort. These are described as one or more objectives, targets or end-points which depends not only on the type of illness but also also its complexity or severity. Therefore, it is important that they are defined in clear terms in the SOP so that auditors can make use of them to make accurate judgment. If auditors want to set their own standards these must be spelled out explicitly.

Implicit Criteria

Many characteristics cannot be measured or described precisely. Even for such a criterion, at least the characteristics to be looked at should be listed. In the absence of specifics, the auditor have to make value judgment based on his/her own standards termed as implicit criteria. It must be realized that this is subject to bias and is not accurate and. Credibility of the assessment is dependent on the wisdom of the auditor. Therefore, auditors chosen for assessment of such areas should be of some standing i.e. are themselves knowledgeable, experienced and reliable.

For example in the appraisal of human resource (operators, workers), characteristics of knowledge, skill and attitude are looked at. Knowledge can be appraised through an interview or question and answer session. Skills are observed by one or more auditors and their opinion is taken as the result. The assessment of attitude is quite subjective. During an audit, however structured the above characteristics are described and observed, the result is considered only a generalization, lacks accuracy and credibility. Yet, they are still useful for instances when stringent methods are not available.

Methods of Quality Assessment in Healthcare

Different methods can be applied to assess quality depending on whether the the aspect is

• outcome or
• factors that affect the quality of a service.

Many methods are available but the most common are discussed below:

Inspection

The person performing an inspection uses his/her senses (sight, hearing, smell, touch) to evaluate things in their static state. Assessment of facilities, equipment and resources is often done through inspection. Measurements and counts can also be made.

Observation

To observe is to determine what is happening by looking and/or hearing for a certain duration. Observation is made on a dynamic state for a duration that is sufficient to for the happening to be completed. Procedures and processes are assessed through observation. The assessor/auditor has to be present at the site where they are being carried out or may review footage of closed circuit video recordings. He/she must be aware of the correct way they are to be done (based on for example the Standard Operating Procedure – SOP). The performance of procedures or processes is operator dependent. To look for consistency the observation of how an operator performs, the observation can be repeated on other similar cases performed by the same person, if necessary. Also performance by different operators can be compared.

Interview

Information can be gathered through a face to face interview or through a self administered questionnaire. Their use is mainly to gauge the knowledge, understanding, awareness, opinions and attitude of service providers.

Audit through the Study of Recorded Data

Recorded data e.g. medical records and charts can be used for the following:

• existence of documentation of data as required
• determining outcome

When quality is assessed through recorded data, the exercise is called a retrospective study, This method requires certain knowledge and skills in data collection, analysis and interpretation. It can be done with data from hand-written or electronic records but there are differences in the methodology i.e. the data management processes used. Because the study is done on data recorded prior to the study the study is said to be retrospective.

Refer to https://drdollah.com/hospital-information-system-his/data-extraction-analysis-and-presentation-of-clinical-data/

Types of Studies

Reviews of Individual Cases

Often, when there an unwanted incident or when a serious complaint is received regarding the care of a patient, a review of the care is done as an inquiry. This consists of going through the medical record combined with interviewing the persons involved in the care of the patient. The aim is to establish facts and to look for adherence to policies and procedures or to discover deviations from it.

Review of the records of one patient (chart review) is not an audit. The methodology is qualitative i.e. the record of a single patient is reviewed by a person or group of persons (e.g. clinical departments or hospitals). The conclusions reached depends on some set criteria. Often, the judgement is made by experts who depend on their knowledge and experience. The criteria used in these instances is implicit criteria i.e. without detailed written specifications. This is is the main method  used in Mortality and Morbidity reviews.

Semi-Quantitative Studies

If reviews are made on a considerable number of similar individual cases, (as described above), certain conclusions can be generalized based on the frequency of occurrences of certain events among the cases. It is not considered as a full quantitative study because only a limited number of cases are studied. Instead the assessment can be said to be a semi-quantitative study . When the study is done on a considerable number of cases, rates can be calculated and be used to compare with results of past or future reviews. When done for a clinical unit/department, hospital, a district, state or nation, the activity can be considered as an audit. Examples include National Maternal Mortality Audit and Peri-operative Morality Audit and so on

Quantitative Studies

In an audit based on a study of recorded data, the aim is to count the number of occurrences of a characteristic. When this is done on a sufficiently large number of records, it can be considered as a quantitative study. The study can be done on all records or a sub-set of them termed as a sample. The sample must consist of a significant proportion of the entire records (termed as the population) so that the findings from it can be said to be representative of the population.

Audit of Outcome

Purpose of Audit of Outcome

Medical Audit is done primarily to determine the quality of outcome. For this purpose, the designation of the outcome as desirable / acceptable or otherwise need to be clarified. The criteria depends on the objectives of the type of service or care to be given.

Service/Care Objectives

For the service, the intended objectives must be defined in the SOP or Care Plan designed for it. These vary with the type and stage of the disease, It can be some or any of these:

• cure,
• reduction of effects of the disease,
• removal or relief of symptoms,
• prevention of complications
• restoration of function,
• reduction of disability.

Desired Characteristics of Outcome

In quality management, the desired outcome is defined as:

1. Satisfying customer needs,
2. Fit for use,
3. Compliance with outcome standards

The result at the end of a service that is given must satisfy the needs of the customer (in the case of clinical care: the patient). Each patient may have certain expectations but the care provider after considering the diagnosis and relevant factors should define the needs based on what is achievable. The patient should be made aware of this.

Whatever service that is given should be useful i.e. offer some benefit. It is not sufficient to provide all kinds of services when they do not lead to achieving the objectives of care mentioned above

At the beginning of care (in the Care Plan or SOP), the characteristics of desired end points/targets need to be specified as standards. In this way, the outcome characteristics can be measured and compared with it.

Criteria of Determining Quality of Outcome

The measurement of quality of outcome in clinical services, though difficult, must follow the principles of measuring quality i.e.

• defining the criteria for concluding that a good outcome has been achieved,
•  use valid parameters, scales and units,
• use a valid measuring tool,
• define and set tolerance levels or control limits,
• compare actual measurements with acceptable limits or standards.

Methodology for Audit of Outcome

Lately, auditing has become formalized and more organized. Hospitals or health authorities often appoint audit committees to identify clinical or other problems and audit them. Therefore, audit can be internal or external.
The person in charge of the service will have to decide on which areas of patient management he would like to examine. Different clinical problems require emphasis on different issues. Then, the auditor must have a clear idea on which approach is the best or most appropriate for each aspect of patient care.
For this purpose there must be a model of good practice or benchmark. This could take the form of an expert opinion or the accepted practice level as expected by professional bodies or in comparison with those achieved in other facilities. However, these may not be uniform. Therefore the concept of consensus development has emerged, where groups of people come together and agree on the most appropriate ways to manage various clinical problems based on good evidence. These often take the form of policies, clinical practice guidelines and standard operating procedures.

Method of the Audit of Outcome

To define outcome as a defined quality feature cab be challenging. The outcome may be defined by one or more quality attributes. Survival or death are easily counted. Cure or improvement is less easy. If these are difficult to measure exactly, an alternative measurable characteristic or a combination of characteristics can be used as surrogate indicators. Indicators are often indirect measures of the quality feature, i.e. they may be just a reflection or proxy of the actual measure of outcome.
In certain situations, problems are more easily detected. Therefore lack of or low frequency of problems are often used as an indicator of quality.

Audit of a Certain Service

Audit for a particular service aims at determining the incidence of certain measure(s) or quality or alternatively the incidences of problems within all or a section of the patients receiving the service. This group of patients being assessed is termed as the population under study. If it is difficult to study the entire population a sample of it is studied. The sample must be representative of the population. This is achieved by taking a adequate number of samples and by defining the characteristics of the patients to be included clearly.
One way is to define the period when the service is being studied. For example, only the cohort of patients served between certain dates are studied. For example, instead of continuously auditing it is done only at intervals for a few selected months in a year. monitoring,
If easily detectable problems are to be counted, then incidence notification and detection in records or registers can be used. The incidence is calculated as the frequency of occurrence within a population by time period, location, case type etc. If the proportion is to be expressed in percentage then:
Incidence = (number of occurrences divided by population) X 100.

Extent of Audit

From the perspective of quality control, the quality of all service products of an organization or facility must be measured. Each service has many quality features therefore one outcome indicator for a service is not enough. To make it comprehensive, healthcare must have data management services that are comprehensive and practical. Data must be should be built in as part of work processes i.e. as and when tasks are performed. Quality management should not depend on separate data collection efforts just for the sake of audit. Indeed there should be regular reports on quality provided through Healthcare Information Systems.

The Need for Dividing Cases into Categories

Response of individuals to intervention vary due to
Natural variation
Severity of illness
Co-morbid factors
Compliance
Therefore standards often determined for a population rather than individuals (Medical Audit)

Grading, Scoring, Staging

Can be done only if data for criteria and the value for that criteria is available e.g. if O2 Sat on admission is a criteria for Severity Score for Bronchial Asthma then O2 Sat must be measured for all cases
Grading, Scoring and Staging must be done routinely (not at time of performing Audit)

Data collection as part of work processes

While Medical Audit is usually done as a Retrospective Study
Data Capture in Medical Records of data to be used for the Audit must be planned
These data include:
variables to be used as criteria for categorisation
Variables used to measure quality of outcome (therapeutic end points, measures of success, complications, side effects, disability, quality of life, period away from work / school etc)

Audit of Processes

Purpose of Audit of Processes

There are two main purposes of the The audit on  procedures or processes i.e.

• extension of the audit of outcome as the means to know the factors lead to that shortfall in quality.
• audit processes on their own in instances

However :

1. When a particular procedure or process affects the outcome to an extent that can be taken as  a surrogate / proxy indicator of outcome
2. To determine the adherence to the prescribed ways of performing the processes (e.g. as described in the SOP).

Desired Characteristics for Processes

Summary: Steps in Medical Audit

• Define product or service
• Define population / patient profile (categorization, severity, stage, co-morbidity)
• Ensure data for scoring / staging are captured
• Define quality features of product for each category (standards, benchmarks)
• Determine how these features are to be measured (outcome standards)
• Ensure that data for above measurements are recorded

Audit as a Measurement of Quality

As a Quality Assessment and Control tool, measurements has to be made. Measurements is necessary in order to identify  existence or otherwise of a the quality feature judged by comparison with  a  standard.  To make a valid measurement the following is necessary:

• features to be measured must be known
• requires a measuring tool
• requires a measuring scale
• control limits or tolerance levels need to be defined
• Collect data continuously for both numerators (occurrences) and denominators (populations) in registries, census etc.
• Calculate incidences, performance for entire population or categories at intervals
• Compare with relevant standards & determine shortfall
• Study causes
• Improve through improving conformance, remedial measures, re-design or re-engineering

Audits  are used to identify the existence of problems and their frequency of occurrence. Therefore, the main measurement method is counting  the occurrence of success or failure as defined by conformance to certain specifications.  Results of audits are presented as rates (proportions, percentages). It is desirable that the matter being audited is the outcome. However, where  one or more processes affects the outcome critically, they be taken as a surrogate measure of outcome.

Purpose of Audit in the Quality Improvement Cycle

Basically, audits are used to identify the existence of problems and their frequency of occurrence. Efforts to identify causes of the problems and what needs to be done to improve outcome  is not really part of the audit activity, even though in some countries (e.g. the UK) the whole quality improvement through problem solving is considered as an audit.

Measuring tool

Some quality characteristics including treatment end points may be defined quite clearly by anatomical, physiological or biochemical parameters. However, in many instances a good outcome requires the summation of many variables. In medical audit often an “assessment” of quality rather than precise measurement is used. The assessor is required to give a value judgement based on applying his / her own implicit criteria. The results are therefore subjective.

Data Management Requirements for a Good Audit

In most instances medical audit is performed as a retrospective appraisal of existing data. Often the data required is missing, incomplete or of poor quality because of poor documentation

Audit of outcome is made easier if as a policy of documenting the outcome at intervals during the care of a patient and at the end of the care episode.

The status of the outcome of care should be assessed and documented at

• critical phases of a visit
• at the end of each visit
• at the end of the episode (i.e. at the final visit)

Setting Acceptable Standards or Limits

The difficulty in setting standards of quality in Clinical Patient Care stems from several factors that include:
– Non-comparable heterogeneous population
– Variation in accepted care practicesvbn mbbbb
– Context issues (variations due to resources available)

Standards For a Population

For a homogenous population: valid to use one standard
For heterogeneous population: variations in outcome invariably occur
Therefore: different standards apply to different sub-categories (product lines, different models of same product)

Comparison of Outcome

Causes of Variation

In patient care variation is contributed by

1. the severity of the illness
2. the ability of the patient to respond to the illness
3. age
4. pre-morbid health e.g. nutritional status, physical fitness
5. pre-existing illness
6. concurrent illness.

Categorisation of Patients into Homogenous Groups

Standardised methods and outcome standards can only be applied to patients with similar characteristics. Therefore the patient population needs to be sub-divided into sub-populations, stratified or categorized based on degrees of severity, stage of illness, or pre-morbid status or a combination of these. Scoring systems have been devised for various illnesses based on the summation of some valid criteria for which purpose data may have to be collected.
It would be an advantage if this categorization is done and documented as care is given. Otherwise the data required for this purpose is made available. This calls for a planned system of documentation.

Good Documentation – the key success factor for performing audits

In the earliest and simplest form medical audit takes the form of going through the patients’ medical records and looking at the various steps in the processes of care and the outcome achieved. Often the data is missing, inadequate and of poor quality. Standardization of practices should include standardization of mandatory documentation. The most common missing data is that of / for measurement of outcome of care.

Documentation of Outcome

The assessment of outcome should be part of the activity of progress review and recorded as part of the progress notes in the medical records. This is true for intermediate outcome i.e. results of various interventions performed.
The final outcome, in traditional practice, is derived retrospectively rather than explicitly stated at the time care is given. A change in approach needs to be advocated. The recording of outcome or, otherwise, the parameters from which outcome can be computed indirectly, should be mandated at certain points in the care of the patient. Appropriate occasions may be:

1. Immediately after an intervention (immediate effect of therapy),
2. At predetermined intervals in progress review depending on nature
3. Illness (intervals of minutes, hours or days)
4. At the end of an outpatient encounter (especially follow up visit)
5. On discharge for an in-patient encounter
6. At the end of a care episode
7. Ad hoc as and when events that reflect on outcome (indicators) occur

Maintaining a Disease and Procedure Registry

Quality measurement for a population of patients is often based on the incidence of an event in a known population. This population can be easily identified if a registry of all patients and procedures performed are available through disciplined record keeping.

After the Audit

Stop Gap Measures

However, currently carrying audit is time consuming and laborious. In the meantime, to overcome the lack of a proper data management system the following approach can be undertaken:

• Audit is performed at intervals for key services and limited to key features of them
• Use of incident or event reporting mechanisms for easily recognizable failures or unwanted events
• Investigation of Customer complaints regarding poor standards of service.

Assessing quality of outcome through event reporting and through customer complaints is only a small part of medical audit.

Root Cause Analysis of Failures

Failure to conform to specifications
Poor design
Design cannot be implemented
Design not capable of producing desired results
Circumstances beyond control

Comparison

Often doctors show their results to their colleagues in the same hospital during clinical meetings or to colleagues from other hospitals in joint meetings. The results of their audit may also be presented at conferences or published in journals for critical appraisal.

This brings about the concept of peer review, which means assessment by other members of the profession of the work of a particular doctor / unit or hospital.

Types of Audit

The activity of Accreditation is an example of healthcare services audit. It relies mainly on inspection and observation.

Supervision vs Audit