They benefit from an implicit guarantee, which becomes explicit in times of crisis. This changes incentives: if risks are ultimately borne by the taxpayer, funding costs may not fully reflect risks, thus incentivising excessive risk-taking, balance sheet growth, and management compensation.
While encouraging the presence of different kinds of banks, the regulator has to ensure that risk management is robust as niche banks are susceptible to risks such as concentration risk which needs to be mitigated through adequate capital requirements and other suitable regulatory measures such as appropriate exposure norms and limits on leverage.