Ethical Reasoning over Moral Alignment: A Case and Framework for
  In-Context Ethical Policies in LLMs
Abhinav Rao Aditi Khandelwal Kumar Tanmay Utkarsh Agarwal Monojit Choudhury
In this position paper, we argue that instead of morally aligning LLMs to
specific set of ethical principles, we should infuse generic ethical reasoning
capabilities into them so that they can handle value pluralism at a global
scale. When provided with an ethical policy, an LLM should be capable of making
decisions that are ethically consistent to the policy. We develop a framework
that integrates moral dilemmas with moral principles pertaining to different
foramlisms of normative ethics, and at different levels of abstractions.
Initial experiments with GPT-x models shows that while GPT-4 is a nearly
perfect ethical reasoner, the models still have bias towards the moral values
of Western and English speaking societies.