The withdrawal of the Draft National AI Policy Framework shortly after its release, while unfortunate, can be viewed as among the best motivations to engage with the myriad AI risks and the need for such a framework.
The Minister of Communications, Solly Malatsi, reported that the draft cited non-existent academic and legal sources, a failure known as “AI hallucination.” He observed that this error, ironically, demonstrated the necessity of human oversight, which was the intended subject of the policy.
Despite this hiccup, the draft is an important step. All parts of the healthcare value chain will be looking forward to engaging with a verified version of the draft AI framework.
We believe government remains committed to regulating use of AI in a principled and structured way. Setting the parameters for the use of AI is a matter of growing concern particularly in the fields of healthcare and financial services where the risks related to mistakes, hallucinations and manipulated data are especially great with potential consequences not yet fully understood.
AI has already moved into South Africa’s healthcare system from claims decisions to care‑pathway recommendations. In certain clinics, nurses use AI tools to categorise patients. For example, a clerk enters a patient’s identity number into a database to retrieve their medical history. A triage application then assigns the patient a colour code: green for routine, orange for high risk, and red for urgent. If the system flags a patient as “red,” the nurse moves them to the front of the queue.
The question is whether people can trust it. Two pillars should guide policy and practice: operational standards that make AI safe and transparent and clear liability rules for when harm occurs.
Patients and providers will distrust these systems unless a transparency framework makes use of them subject to accountability. A transparency framework also requires that AI is understandable: any opacity will frustrate patients and providers.
Because healthcare decisions affect physical safety, transparency requires a rigorous assessment of potential harm. Any AI tool supporting clinical treatment must be tested for unintended effects and regulations should reflect that need.
Automated decisions do not absolve a provider of the duty of care. Models must be tested for bias across demographic groups and corrected. A national policy must ensure clinical accountability, require human intervention in design and establish an authority, such as an Ombudsperson, for AI-related complaints.
Further, use of AI in management and administration of medical schemes, hospitals and healthcare practices must be equally transparent and open to challenge or dispute.
Many will point out that South African law already requires transparency. The Protection of Personal Information Act (POPIA) mandates the lawful processing of data, and the National Health Act prioritises patient welfare. In addition, the FSCA’s principles require that customers understand decisions affecting their benefits. AI must not diminish these duties.
But the current laws provide only general guidance. POPIA and the National Health Act were not designed for agentic AI or adaptive AI. While common-law negligence and medical device rules still apply, they do not address AI-specific problems. This regulatory vacuum creates uncertainty.
If an algorithm determines the urgency of care or the payment of a medical claim, the institution must be able to explain the decision. “The system flagged it” is an insufficient explanation. Patients must know which factors determined the outcome and how to contest it.
Clear standards for use of AI in clinical settings are essential. At a minimum, these should include clinical validation before use, bias testing for different groups, human-in-the-loop and override options, audit trails for model changes, ongoing performance checks and clear patient disclosures with ways to challenge decisions.
More explanation is needed for use of clinical AI than for tools such as credit-scoring. Cancer diagnosis, for example, demands a higher standard. Without clear rules for developers, hospitals and clinicians, practitioners won’t feel confident using these tools. These standards should act as safe harbours: following them clarifies and reduces liability for unexpected failures, while breaking them increases fault.
Liability rules should make clear who is responsible when harm occurs. Regulators need to specify when developers are responsible for faulty or misleading models, when institutions are liable for poor governance or deployment, and when clinicians fail in their duty of care by ignoring or using AI without the doctor’s verification.
The rules should also include contractual indemnities, a way for complaints to be heard (an Ombudsperson), and follow existing laws on negligence and product liability.
Private healthcare companies can help regulators draft these standards and rules by working with all stakeholders, which makes it essential to engage constructively with the draft AI framework when it is reissued.
This way everyone, developers, institutions, and clinicians, knows their roles. Providers must also clearly tell patients, in plain language, ideally preferably in their mother tongue, whenever AI is used to support a diagnosis or a treatment decision.
Private providers are ready to test these disclosure frameworks. We can contribute our clinical experience and operational data to help the government draft a comprehensive “AI in Healthcare” study that provides guidelines for a practical, enforceable system. The nature of medical consultation has changed and the private healthcare sector must help write the new, hallucination-free rules of practice.
- Satish Antony is Director: Chief Analytics and Strategy Officer, AfroCentric
