issued by the firm's affiliate based on other

information the firm knows about the investor because the use of such
technology does not, and is not intended to, affect an investment-
related behavior or outcome. For the same reason, the use of a firm's
chatbot that employs PDA-like technology to assist investors with basic
customer service support (e.g., password resets or disputing fraudulent
account activity) would not qualify as covered technology under the
proposed definition.
    We request comment on all aspects of the definition of covered
technology, including the following items:
    1. Is the scope of the proposed definition of a covered technology
sufficiently clear? We intend for the proposed definition to cover PDA-
like technologies; are there ways we could revise the proposed
definition in order to better accomplish this? Are there any
technologies covered by the proposed definition that go beyond PDA-like
technologies and should be excluded? For instance, should the proposed
definition distinguish between different categories of machine learning
algorithms, such as deep learning, supervised learning, unsupervised
learning, and reinforcement learning processes? Do one or more of these
categories present more investor protection concerns related to
conflicts of interest relative to other categories? Would firms be able
to identify what would and would not be a covered technology for
purposes of the proposed rules? If not, what additional clarity would
be beneficial? We have described examples of technologies to which the
definition would or would not apply. Should the definition be revised
to include or specifically exclude such examples?
    2. Would the definition adequately include the technology used by
firms that would present the conflicts of interest and resulting risks
to investors that these proposed rules are designed to address? If not,
how should this definition be changed to further the objective of the
proposed conflicts rules? Please explain your answer, including the
extent to which these technologies do or do not present conflicts of
interest risks to investors. Alternatively, do the technologies
included in the proposed definition include technology that does not
typically result in risks to investors that these proposed rules are
designed to address?
    3. Is the proposed definition of covered technology appropriately
calibrated to allow for future technological developments? What
adjustments, if any, should the Commission make to help ensure that the
definition of covered technology will remain evergreen despite future
technological advancements? Conversely, what adjustments to the
definition of covered technology, if any, are necessary to avoid
covering those future technological advancements that do not possess
characteristics that the proposed rules are intended to address?
    4. The proposed definition of covered technology only applies to
technologies that are used to optimize for, predict, guide, forecast,
or direct investment-related behaviors or outcomes. Do the terms
``optimize for,'' ``predict,'' ``guide,'' ``forecast,'' and ``direct''
appropriately scope the definition? Is it clear what these terms are
intended to capture or would further explanation be helpful? Are there
certain technologies that would fit within one or more of those terms
but which should be outside the scope of the proposed definition?
Alternatively, are there certain technologies that would fall outside
those terms but which should be within the scope of the proposed
definition? If so, should we use additional or different words to
clarify the meaning? For instance, should we include the term
``influence'' in the definition? If so, how would ``influence'' differ
from the terms ``guide'' or ``direct'' in the definition? Should we use
``nudge'' or ``prompt'' in the definition? Alternatively, should we
remove any of the terms in the proposed definition? For instance, are
the terms ``guide'' and ``direct'' redundant or do they express
distinct meanings within the context of the definition? Does ``guide''
capture broader activity than ``direct'' and cause the rule to capture
technologies that should not be in scope? Should the definition be
limited to technologies that direct or influence an investor?
    5. Should the proposed definition of covered technology apply to
technologies that are used to optimize for, predict, guide, forecast,
or direct investment-related behaviors or outcomes, directly or
indirectly? Are there certain PDA-like technologies that optimize for,
predict, guide, forecast, or direct investment-related behaviors or
outcomes indirectly that should be covered by this definition? If so,
what are they and why? If the definition did include the term
``indirectly,'' would it include technologies that should not be
covered by the proposed conflicts rules?
    6. Should the definition of covered technology not include
technology that is solely meant to inform investors, as proposed?
    7. Does the term ``covered technology'' adequately reflect the
definition? Should some other defined term be used, such as ``covered
processes'' or ``covered methods''? Are there any other terms that
should be used?
    8. Does the phrase ``investment-related behaviors or outcomes''
sufficiently clarify the intended scope of the rule and which
technologies would not be within the definition? Is it clear what the
phrase ``investment-related behaviors or outcomes'' would capture or
would further explanation be helpful? Are there certain behaviors or
outcomes that may not be ``investment related'' but should nonetheless
be covered by the proposed definition? For instance, should PDA-like
technologies used for back office or administrative functions, such as
trade settlement, the routing of customers' orders, accounting, or
document review and processing, be included in the covered technology
definition? Are commenters aware of any PDA-like technology that is
used for back office functions, such as the routing of customer orders,
that is also used to engage or communicate with investors (i.e., that
involve an investor interaction)? Are there certain investment-related
activities that may not be ``behaviors or outcomes'' that should be
covered by the definition? Is either ``behavior'' or ``outcome''
overbroad, capturing activities beyond those intended by the
definition? Should a different term, such as ``investment-related
covered technology'' be used?
    9. Are there aspects of this definition that should be broadened,
narrowed, revised, removed, or added? For instance, should the
definition be limited to the use of predictive data analytics and/or
artificial intelligence that optimizes for, predicts, guides,
forecasts, or directs investment-related behaviors or outcomes?
Alternatively, should we limit the scope of the definition to
technologies that are used to provide investment advice or
recommendations? Should we otherwise limit the scope to technologies
that are used directly by investors? Should we expressly exclude
technologies that are not used by investors but instead are used by
individuals who are associated with a firm and use the technologies in
communicating with investors?
b. Investor Interaction
    The proposed conflicts rules include definitions for both
``investor'' and ``investor interaction.'' \126\ For brokers or
dealers, the definition of investor would include a natural person, or
the legal representative of such natural person, who seeks to receive
or receives services primarily for personal, family or

household purposes. The definition is designed to capture both
prospective and current retail investors.\127\ For investment advisers,
the definition of investor would include a client or prospective
client, and any current or prospective investor in a pooled investment
vehicle advised by the investment adviser.\128\ The use of PDA-like
technology by investment advisers of pooled investment vehicles, such
as algorithmically targeted advertisements that are designed to solicit
investors in a pooled investment vehicle or algorithmically designed
investment strategies in pooled investment vehicles, present the same
investor protection concerns as advisers that use the same or similar
technology to target or advise their advisory clients. Accordingly, we
are proposing to define ``investor'' so that the proposed conflicts
rules would broadly apply both to clients that receive investment
advisory services from an investment adviser and to investors in a
pooled investment vehicle advised by the investment adviser.\129\
---------------------------------------------------------------------------

    \126\ See proposed conflict rules at (a).
    \127\ See supra note 6. Broker-dealers are subject to regulation
under the Exchange Act and SRO rules, including a number of
obligations that attach when a broker-dealer offers services to a
retail customer, including making recommendations, as well as
general and specific requirements aimed at addressing certain
conflicts of interest. The application of these obligations can vary
depending on a broker-dealer's business lines and activities, as
well as the level of customer sophistication. See Regulation Best
Interest, Exchange Act Release No. 83062 (May 9, 2018) [83 FR 21574
(May 9, 2018)], at 21575 (``Reg BI Proposing Release''); see, e.g.,
FINRA Rule 2210 (applying broker-dealer obligations related to
communications with the public differently to communications
directed to retail versus institutional investors). Here, the focus
of the proposed rules for broker-dealers is on retail investors.
    \128\ See proposed rule 211(h)(2)-4(a) (specifying that ``pooled
investment vehicle'' has the same meaning as in 17 CFR 275.206(4)-8,
meaning any investment company as defined in section 3(a) of the
Investment Company Act of 1940 or any company that would be an
investment company under section 3(a) of that Investment Company Act
but for the exclusion provided from that definition by either
section 3(c)(1) or section 3(c)(7) of the Investment Company Act).
    \129\ See proposed conflict rules at (a) (defining
``Investor'').
---------------------------------------------------------------------------

    The proposed conflicts rules would generally define investor
interaction as engaging or communicating with an investor, including by
exercising discretion with respect to an investor's account; providing
information to an investor; or soliciting an investor.\130\ This
definition would capture a firm's correspondence, dissemination, or
conveyance of information to or solicitation of investors, in any form,
including communications that take place in-person, on websites; via
smartphones, computer applications, chatbots, email messages, and text
messages; and other online or digital tools or platforms. This
definition would include engagement between a firm and an investor's
account, on a discretionary or non-discretionary basis. This definition
would also capture any advertisements, disseminated by or on behalf of
a firm, that offer or promote services or that seek to obtain or retain
one or more investors. The proposed definition is intended to be
sufficiently broad to encompass the wide variety of methods, using
current and future technologies, that firms could use to interact with
investors.\131\
---------------------------------------------------------------------------

    \130\ See proposed conflict rules at (a).
    \131\ See generally Investment Adviser Marketing Release, supra
note 19 (a recent Commission rule designed to accommodate the
continual evolution of the use of technology in the investment
adviser industry as it relates to advisers marketing their services
to clients and investors).
---------------------------------------------------------------------------

    The proposed definition is generally designed to limit the proposed
conflicts rules' scope to a firm's use of covered technology in
interactions with investors. This aspect of the proposed conflicts
rules recognizes that the conflicts associated with the use of covered
technology in investor interactions present a higher risk of harm to
investors than conflicts associated with technologies that are not used
in such interactions. For instance, a firm could utilize covered
technology to analyze historical data and current market data to
identify trends and make predictions related to the firm's intra-day
liquidity needs, peak liquidity demands, and working capital
requirements. A firm could likewise use covered technology to make
investment decisions about its own assets. Similarly, a firm could
implement covered technology for automation of, for example, ``back
office'' processes like the routing of customers' orders \132\ and
accounting and trade settlement. In each of these examples, the use of
covered technology for these processes does not involve an investor
interaction, and therefore would not be subject to the proposed
conflicts rules.
---------------------------------------------------------------------------

    \132\ Although routing of customers' orders is not covered by
this proposal, broker-dealers owe their customers a duty of ``best
execution.'' Best execution requires that a broker-dealer seek to
obtain for its customer orders the most favorable terms reasonably
available in the market under the circumstances. See, e.g., Newton
v. Merrill, Lynch, Pierce, Fenner & Smith, 135 F.3d 266, 270 (3d
Cir. 1998). See also Kurz v. Fidelity Management & Research Co., 556
F.3d 639, 640 (7th Cir. 2009); Geman v. SEC, 334 F.3d 1183, 1186
(10th Cir. 2003); see also FINRA Rule 5310 (Best Execution and
Interpositioning). The Commission recently proposed a rule that, if
adopted, would establish through Commission rule a best execution
standard for broker-dealers. See Regulation Best Execution, Exchange
Act Release No. 96496 (Dec. 14, 2022) [88 FR 5440 (Jan. 27, 2023)].
---------------------------------------------------------------------------

    In contrast, when a firm's use or potential use of a covered
technology in any investor interaction could involve a conflict of
interest, a firm would be subject to the framework of the proposed
conflicts rules. The proposed definition of investor interaction does
not make any distinctions based on the manner in which an investor or
the investor's account interacts with the covered technology or on the
manner in which the firm uses the technology in the interaction.
Meaning, ``use'' of covered technology in an investor interaction can
occur directly through the use of a covered technology itself (e.g., a
behavioral feature on an online or digital platform that is meant to
prompt, or has the effect of prompting, investors' investment-related
behaviors) or indirectly by firm personnel using the covered technology
and communicating the resulting information gleaned to an investor
(e.g., an email from a broker recommending an investment product when
the broker used PDA-like technology to generate the
recommendation).\133\
---------------------------------------------------------------------------

    \133\ To the extent a broker-dealer uses PDA-like technology to
make a recommendation to a retail customer, the broker-dealer would
also be subject to Reg BI and its attendant obligations, including
the Conflict of Interest Obligation, as to the recommendation.
Similarly, an investment adviser making a recommendation to its
client would also be subject to fiduciary obligations that include a
duty of loyalty under which an adviser must eliminate or make full
and fair disclosure of all conflicts of interest. See Fiduciary
Interpretation, supra note 8.
---------------------------------------------------------------------------

    Unlike a purely ministerial or back office function, these examples
involve an investment-related communication with an investor and would
be considered an investor interaction under the proposed definition.
Similarly, a firm may use covered technology to provide individual
brokers or advisers with customized insights into an investor's needs
and interests and the broker or adviser may use this information to
supplement their existing knowledge and expertise when making a
suggestion to the investor during an in-person meeting. Such a scenario
would result in the firm using a covered technology in an investor
interaction under the proposed rules. An investor interaction would
also include firms' use of game-like prompts or marketing that
``nudge'' investors to take particular investment-related actions on
digital platforms. In addition, the investor interaction definition
covers solicitations, for example, a firm utilizing covered technology
that scrapes public data, which the firm in turn uses to solicit
clients through broadcast emails.\134\
---------------------------------------------------------------------------

    \134\ See infra section II.A.2.e (acknowledging that although a
firm's use of covered technology to solicit investors to open an
account falls under the definition of an investor interaction, it
may not involve a conflict of interest that would require
elimination or neutralization under the proposed conflicts rules).
On the other hand, a conflict of interest may appear if a firm's
chatbot is programmed to solicit only investors that scraped data
show are heavy gamblers, and thus perceived as being more profitable
to the firm as investors that might invest in risky, high-profit
investments that earn the firm more money relative to other
investments.

---------------------------------------------------------------------------

    The proposed definition of investor interaction would include
interactions that have generally been viewed as outside the scope of
``recommendations'' for broker-dealers.\135\ For example, under the
proposed definition, an investor interaction could include: firms' use
of research pages or ``electronic libraries'' to provide investors with
the ability to obtain or request research reports, news, quotes, and
charts from a firm-created website; or firm's use of technologies to
generate emails to investors as part of a firm-run email communication
subscription that investors can sign up for and customize, and which
alerts investors to items such as news affecting the securities in the
investor's portfolio or on the investor's ``watch list.'' \136\
Accordingly, the proposed definition would capture firm communications
that may not rise to the level of a recommendation, yet are nonetheless
designed to, or have the effect of, guiding or directing investors to
take an investment-related action.
---------------------------------------------------------------------------

    \135\ See NASD Notice to Members 01-23 (Apr. 2001) (Online
Suitability--Suitability Rules and Online Communications)
(discussing the types of online communications may constitute
``recommendations'' under the NASD suitability rule); Reg BI
Adopting Release, supra note 8, at section II.B.2 (discussing
factors to consider when determining whether a ``recommendation''
has been made by a broker-dealer).
    \136\ See NASD Notice to Members 01-23, id.
---------------------------------------------------------------------------

    The proposed definition would exclude from the investor interaction
definition interactions solely for purposes of meeting legal or
regulatory obligations.\137\ These interactions are subject to existing
regulatory oversight and/or do not involve the type of conflicts the
proposed rules seek to address. This exclusion would apply to
interactions with an investor for purposes of obligations under any
statute or regulation under Federal or State law, including rules
promulgated by regulatory agencies. For example, the proposed
definition would exclude interactions with investors solely for anti-
money laundering purposes, such as using PDA-like technologies to
identify and track investor activity for the purposes of flagging
suspected fraudulent transactions and requesting identification and
verification of the transaction from an investor (e.g., sending two-
factor authentication messages).\138\ If a firm, however, includes as
part of such an interaction actions that are not reasonably designed to
satisfy its obligations under applicable law (e.g., circulating a link
to a digital platform that includes features designed to prompt
investors to trade along with the annual delivery of Form ADV), and
such additional actions are otherwise within the definition of an
investor interaction, then such action would be considered an investor
interaction for purposes of the proposed conflicts rules.
---------------------------------------------------------------------------

    \137\ See proposed conflicts rules at (a).
    \138\ The activities covered under this legal and regulatory
obligation exception would qualify as an investor interaction that
uses covered technology absent this exception. However, as a
practical matter, many of these activities would not involve a
firm's use of covered technology under the proposed definition,
because such activities would not involve an analytical,
technological, or computation function, algorithm, model,
correlation matrix, or similar method or process (e.g., delivery of
Form ADV or summary prospectus pursuant to legal obligations).
---------------------------------------------------------------------------

    In addition, the proposed definition would also exclude
interactions solely for purposes of providing clerical, ministerial, or
general administrative support. For example, the proposed definition
would exclude basic chatbots or phone trees that firms use to direct
customers to the appropriate customer service representative. This
aspect of the exclusion is only intended to cover basic or first-level
customer support designed to efficiently answer simple questions like
providing the business hours of a branch office or the balance in the
investor's account, or to guide the investor to a human representative
in the appropriate department of the firm who is trained to address the
investor's question. On the other hand, if a firm sought to employ a
more advanced chatbot designed to answer complex investment-related
questions, such as when or whether to invest in a particular investment
product or security, this would no longer fit within the exclusion for
clerical, ministerial, or general administrative support, and would
constitute an investor interaction under the proposed definition.
    In either case, the exclusions would be limited to interactions
that are ``solely for the purpose'' of the relevant category (or
categories) of conduct in order to help ensure that interactions that
serve several purposes, including purposes that are not excluded, will
be within the scope of the definition of investor interaction.\139\ The
``solely for the purpose'' language is designed to help ensure that all
the functions of a dual-use technology like a chatbot would be
considered when evaluating conflicts of interest associated with use of
the chatbot.
---------------------------------------------------------------------------

    \139\ Interactions that are for the purpose of both categories
of conduct would also fit within the exclusion. For example, an
algorithm whose purpose was both to comply with legal or regulatory
obligations and to conduct other clerical, ministerial, or general
administrative support functions would fit within the exclusion so
long as the algorithm did not also have a third purpose that was not
excluded from the definition.
---------------------------------------------------------------------------

    We request comment on all aspects of the proposed definitions of
investor interaction and investor, including the following items:
    10. For broker-dealers, the proposed definition of investor means a
natural person, or the legal representative of such natural person, who
seeks to receive or receives services from the broker-dealer primarily
for personal, family or household purposes. Should we narrow the
definition of investor as applied to broker-dealers to only cover
retail customers, as defined under Reg BI? Should we expand the
definition of investor for brokers or dealers to cover all current and
prospective investors and not just retail investors? We have stated
that investors may not be able to understand the complexities of
covered technologies and any conflicts associated with their use.
Should we expand the definition of investor for broker-dealers to cover
a certain subset of non-retail investors? The proposed definition of
investor for investment advisers is not limited to services ``primarily
for personal, family or household purposes.'' Should we add such
limitation in the investment adviser conflicts rule?
    11. Should we narrow the definition of investor for investment
advisers? For example, should we only apply it to retail investors, as
defined in Form CRS? If so, please explain why in comparison to other
rules under the Advisers Act.
    12. For investment advisers, the proposed definition of investor
also includes investors or prospective investors in a pooled investment
vehicle that is a client or prospective client of the investment
adviser; should we retain this in the final rules? Are there special
considerations for investors in a pooled investment vehicle that cause
them to need less protection from conflicts of interest associated with
a firm's use of covered technology? If the definition of ``investor''
continues to include investors in pooled investment vehicles, as
proposed, are there certain structures or types of pooled investment
vehicles that should not be included? For example, should investors in
collateralized loan obligation vehicles be excluded? Are there unique
characteristics of such vehicles,

investors, or investors in other pooled investment vehicles, which make
the additional protections that would be provided by the proposed
conflicts rules unnecessary? The proposed definition of ``investor''
would incorporate the definition of ``pooled investment vehicle'' in
rule 206(4)-8. Should we define the term ``pooled investment vehicle''
(or use another term)? Should we define the term more broadly for
purposes of this rule to include other vehicles to which an investment
adviser may provide investment advice that rely on other exclusions
from the definition of investment company, such as companies primarily
engaged in holding mortgages that are excluded pursuant to section
3(c)(5)(C) of the Investment Company Act, or collective investment
trust funds or separate accounts excluded under section 3(c)(11) of the
Investment Company Act?
    13. Will the proposed definition of investors present challenges
for firms that are dually registered as investment advisers and broker
dealers?
    14. Should we define ``prospective investor'' in the proposed
rules? If so, how should we define this term and why? For example,
should we define ``prospective investor'' as any person or entity that
engages in some way with a firm's services (e.g., downloads the firm's
mobile app, visits the firm's website, or creates a log-in)? If not,
should we provide guidance regarding how firms can identify prospective
investors?
    15. Is the proposed definition of investor interaction sufficiently
clear? Would firms be able to identify what would be an investor
interaction for purposes of the proposed conflicts rules? Are there
activities that are not covered by the proposed definition of investor
interaction that should be? Are there activities that are covered by
the proposed definition that should not be? For instance, should a firm
soliciting prospective investors be included within the definition?
Should the proposed definition be limited to interactions in which
investors directly interact with, or otherwise directly use, covered
technology? Do situations in which investors do not directly interact
with covered technology raise the same concerns of scalability as those
in which investors do interact directly?
    16. Do commenters agree that investor interactions, as proposed,
may entail conflicts of interest that are particularly likely to result
in investor harm or to take additional effort to discern? Are there
types of activities we should specifically include or exclude within
the definition?
    17. Do commenters agree that the definition of investor interaction
should exclude interactions solely for purposes of meeting legal or
regulatory obligations or providing clerical, ministerial, or general
administrative support? Should we remove any or all aspects of these
exclusions from the definition in the final conflicts rules? In the
case of interactions solely for the purpose of meeting legal or
regulatory obligations, should we broaden or narrow the exclusion? For
example, should we take into account legal or regulatory obligations as
a result of compliance with foreign law, or with policies, rules, or
directives of SROs (including securities exchanges) or other bodies?
Generally, would investor interactions that fall under the proposed
exclusions employ covered technology (e.g., technologies that optimize
for, predict, guide, forecast, or direct investment-related behaviors
or outcomes)? If so, how? If not, is the exception for legal or
regulatory obligations additive? Is the exclusion for providing
clerical, ministerial, or general administrative support sufficiently
clear? For instance, is it clear this phrasing would capture trade
settlement and the routing of customers' orders or would further
explanation be helpful?
    18. Do the proposed conflicts rules adequately address how a firm
would treat a single covered technology that features functions that
are both included and excluded from the investor interaction
definition? For instance, a chatbot that is used for both general
customer support help (e.g., password resets) and to provide more
advanced functions, such as guiding an investor as to when and whether
to invest in a particular investment product. Should the proposed
conflicts rules treat these dual-purpose covered technologies
differently than covered technology used solely for purposes of meeting
legal or regulatory obligations or providing clerical, ministerial, or
general administrative support?
    19. To the extent we retain or expand the exclusions, are there any
conditions we should add in order for a firm to be able to rely on
particular exclusions? For example, should we require that a firm
create and maintain a written record if it relies on an exclusion? Are
there other activities that should be excluded? For example, should we
provide a more principles-based exclusion for certain activities that
the firm affirmatively identifies in writing as low-risk and that are
already part of existing compliance programs or subject to other laws,
rules, regulations, or policies?
    20. As specified in the proposed definition of investor
interaction, the definition would include discretionary management of
accounts where the engagement is with the investor's account, even if
there is no communication or other interaction with investors
themselves at the time of trades in their accounts. Should the
discretionary management of accounts be included within the definition
of investor interaction? Should it be excluded? Do commenters agree
that a firm's discretionary management of accounts using covered
technologies may entail conflicts of interest that are particularly
likely to result in investor harm and are not sufficiently addressed
under the current applicable legal framework? Why or why not?
2. Identification, Determination, and Elimination, or Neutralization of
the Effect of, a Conflict of Interest
    The proposed conflicts rules would require a firm to eliminate, or
neutralize the effect of, certain conflicts of interest associated with
the use of a covered technology in investor interactions.\140\ The
proposed conflicts rules would also require firms to take affirmative
steps as a precursor to eliminating or neutralizing the effect of these
conflicts. First, a firm would be required to evaluate any use or
reasonably foreseeable potential use of a covered technology in any
investor interaction to identify whether it involves a conflict of
interest, including through testing the technology. Second, a firm
would be required to determine if any such conflict of interest results
in an investor interaction that places the interest of the firm or an
associated person ahead of investors' interests. Third, the proposed
conflicts rules would require a firm to take a particular action--
elimination or neutralization--to address any conflict of interest the
firm determines in step two results in an investor interaction that
places its or an associated person's interest ahead of investors'
interests.\141\ The proposed conflicts rules thus supplement, rather
than supplant, existing regulatory obligations related to conflicts of
interest, laying out particular steps a firm must take to address
conflicts of interest arising specifically from the use of covered
technologies in investor interactions.\142\

This is because the nature of these technologies (for example due to
their inherent complexity and ability to rapidly scale transmission of
conflicted actions across a firm's investor base) requires additional
steps to address conflicts associated with their use in investor
interactions, compared to conflicts of interest more generally.
---------------------------------------------------------------------------

    \140\ See infra section II.A.2.e.
    \141\ On the application to interests of associated persons, see
infra sections II.A.2.c, II.A.2.d, and II.A.2.e, and proposed
conflicts rules at (b)(2) and (3).
    \142\ The elimination or neutralization requirement of the
proposed rules applies only to a narrower, defined subset of the
broader universe of conflicts--those conflicts that a firm
determines actually place the interests of the firm or certain
associated persons ahead of the interests of investors. This is in
contrast to, for example, an investment adviser's fiduciary duty,
which encompasses any interest that might incline the adviser,
consciously or subconsciously, to provide advice that is not
disinterested., or similarly in contrast to the broader universe of
conflicts covered by Reg BI. Other conflicts of interest that only
might affect the firm's investor interactions would continue to be
subject to these other obligations, as applicable.
---------------------------------------------------------------------------

a. Evaluation and Identification
    The proposed conflicts rules would require a firm to evaluate any
use or reasonably foreseeable potential use by the firm or its
associated persons of a covered technology in any investor interaction
to identify any conflict of interest associated with that use or
potential use.\143\ This requirement of the proposal, in connection
with the requirement to test and periodically retest any covered
technology, is designed to help ensure that a firm has a reasonable
understanding of whether its use or reasonably foreseeable potential
use of the covered technology in investor interactions would be
associated with a conflict of interest.
---------------------------------------------------------------------------

    \143\ See proposed conflicts rules at (b)(1).
---------------------------------------------------------------------------

    The proposed conflicts rules do not mandate a particular means by
which a firm is required to evaluate its particular use or potential
use of a covered technology or identify a conflict of interest
associated with that use or potential use. Instead, the firm may adopt
an approach that is appropriate for its particular use of covered
technology, provided that its evaluation approach is sufficient for the
firm to identify the conflicts of interest that are associated with how
the technology has operated in the past (for example, based on the
firm's experience in testing or based on research the firm conducts
into other firms' experience deploying the technology) and how it could
operate once deployed by the firm. If a technology could be used in a
variety of different scenarios, the firm should consider those
scenarios in which it intends that the technology be used (and for
which it is conducting the identification and evaluation process). It
should also consider other scenarios that are reasonably foreseeable
unless the firm has taken reasonable steps to prevent use of the
technology in scenarios it has not approved (for example, by limiting
the personnel who are able to access the technology).
    A firm could adopt different approaches for different covered
technologies.\144\ Such approaches could vary depending on the nature
of the covered technologies employed by the firm at the time they are
implemented, how the technologies are used, and the firm's plans for
future use of those technologies. For example, a firm that only uses
simpler covered technologies in investor interactions, such as basic
financial models contained in spreadsheets or simple investment
algorithms, could take simpler steps to evaluate the technology and
identify any conflicts of interest, such as requiring a review of the
covered technology to confirm whether it weights outcomes based on
factors that are favorable for the adviser or broker-dealer, such as
the revenue generated by a particular course of action.\145\ Even when
a firm identifies a conflict of interest associated with a simple
covered technology, depending on the facts and circumstances, it may
determine that such conflict of interest does not actually result in
the firm's or an associated person's interests being placed ahead of
those of investors, and that the conflict of interest does not need to
be eliminated or its effects to be neutralized.
---------------------------------------------------------------------------

    \144\ Cf. U.S Chamber of Commerce Technology Engagement Center,
Report of the Commission on Artificial Intelligence Competitiveness,
Inclusion, and Innovation (Mar. 9, 2023), at 82 (``Chamber of
Commerce AI Report''), https://www.uschamber.com/assets/documents/CTEC_AICommission2023_Report_v6.pdf (calling for ``impact
assessments'' to help categorize potentially harmful uses of certain
technologies in a risk-based framework).
    \145\ See infra section II.A.2.d, discussing financial models.
---------------------------------------------------------------------------

    Firms that use more advanced covered technologies may need to take
additional steps to evaluate technology adequately and identify
associated conflicts adequately.\146\ For example, a firm might
instruct firm personnel with sufficient knowledge of both the
applicable programming language and the firm's regulatory obligations
to review the source code of the technology, review documentation
regarding how the technology works, and review the data considered by
the covered technology (as well as how it is weighted).\147\ A firm
seeking to evaluate an especially complex covered technology and
identify conflicts of interest associated with its use may consider
other methods as well. For example, if a firm is concerned that it may
not be possible to determine the specific data points that a covered
technology relied on when it reached a particular conclusion, and how
it weighted the information, the firm could build ``explainability''
features into the technology in order to give the model the capacity to
explain why it reached a particular outcome, recommendation, or
prediction.\148\ By reviewing the output of the explainability
features, the firm may be able to identify whether use of the covered
technology is associated with a conflict of interest.\149\ Developing
this capability would require an understanding of how the model
operates and the types of data used to train it.
---------------------------------------------------------------------------

    \146\ These steps could be included in the policies that the
firm would be required to adopt under the proposed conflicts rules,
and may also be necessary to satisfy the proposed recordkeeping
amendments. See infra section II.A.3 and II.B. A written description
of a covered technology prepared in accordance with policies and
procedures that are reasonably designed to prevent violation by the
firm of the proposed conflicts rules generally should include a
written evaluation of the technology and identify any conflicts of
interest presented by the technology. This would also assist the
firm in preparing records that would comply with the proposed
recordkeeping amendments. See infra section II.B.
    \147\ When evaluating the data considered by a covered
technology used by a firm, both the data itself and the weighting of
the data may inform a firm's determination of whether or not any
conflict of interest it identifies and evaluates would result in an
investor interaction that places the interest of the firm ahead of
the interests of investors. See infra section II.A.2.d.
    \148\ See supra section I.B.4 (describing complex or opaque
technologies, sometimes referred to as ``black boxes'').
    \149\ Testing (such as A/B testing) that is designed to
determine the influence of a particular factor may also be helpful
and is discussed infra. If the output of the explainability features
is not sufficient for the firm to identify whether a conflict of
interest exists at all, the firm may still be able to use the output
to determine that any conflict of interest that may exist still does
not result in its interests being placed ahead of investors'
interests, or alternatively that any conflicts of interest that may
exist have been eliminated or their effect has been neutralized due
to controls the firm placed on its use of the technology. See infra
section II.A.2.d (discussing using explainability features for
determination) and infra section II.A.2.e (discussing using
explainability features for elimination or neutralization).
---------------------------------------------------------------------------

    Not all of these steps would be necessary (or possible) in all
circumstances. So long as the firm has taken steps that are sufficient
under the circumstances to evaluate its use or reasonably foreseeable
potential use of the covered technology in investor interactions and
identify any conflicts of interest associated with that use or
potential use, this aspect of the proposed conflicts rules would be
satisfied. To the extent a technology is customizable, we anticipate a
firm will be able to evaluate the technology and identify the conflicts
associated with its use through the choices it makes when customizing
the technology. For

technologies that are not customizable, we anticipate a firm will be
able to evaluate the technology and identify conflicts via other means.
    For example, a firm that licenses a covered technology from a third
party may have no access, or limited access, to the underlying source
code of the technology. In such circumstances, provided that the other
documentation regarding how the technology functions is sufficiently
detailed as to how the technology works, the identification and
evaluation could be satisfied through review of such documentation.
Firms without access to the underlying source code could review, for
example, documentation about how the technology can be tailored to its
investors' requirements (such as how to tailor it to eliminate, or
neutralize the effect of, conflicts of interest). In circumstances
where the firm is relying only on the technology's documentation, its
testing methodology would be of special importance to help the firm
discover whether there is any undocumented functionality that could be
associated with a conflict of interest.
    When evaluating a covered technology and identifying conflicts of
interest, a firm should consider the circumstances in which a covered
technology would be deployed in investor interactions. Firms that use a
covered technology in investor interactions that operates autonomously
or with limited involvement by firm personnel should consider
subjecting it to more scrutiny because the firm's personnel may not
immediately notice if the conflicts become apparent once the technology
is deployed, or if its outputs change over time.\150\ On the other
hand, if a covered technology is only used to provide first drafts of
marketing materials, or is only used to provide investment ideas that
will be more fully considered by firm personnel who are trained on the
firm's compliance policies, and the drafts or ideas are subjected to
scrutiny throughout the review process before the output is ultimately
used in an investor interaction, the covered technology generally may
need comparatively less scrutiny.
---------------------------------------------------------------------------

    \150\ This tendency would also mean that the technology would
need to be tested on a more frequent basis. See infra section
II.A.2.b (discussing proposed testing requirement as it would apply
to technologies that ``drift'' or that operate autonomously).
---------------------------------------------------------------------------

    In certain cases, it may be difficult or impossible to evaluate a
particular covered technology or identify any conflict of interest
associated with its use or potential use within the meaning of the
proposed rules. For example, many large language models may consider
millions of different data points, which could make it difficult for a
firm to determine whether certain of those data points implicate the
firm's interest. In some cases, it may be difficult for the firm to
understand exactly what is in the data set that the model is
considering, for example, if it was trained on a data set from the
entire internet. Likewise, there may be situations where a firm does
not have full visibility into all aspects of how a covered technology
functions, such as if the firm licensed it from a third party.\151\
However, a firm's lack of visibility would not absolve it of the
responsibility to use a covered technology in investor interactions in
compliance with the proposed conflicts rules.
---------------------------------------------------------------------------

    \151\ FINRA has stated that outsourcing an activity or function
to a third-party vendor does not relieve broker-dealers of their
supervisory obligations, which must be reasonably designed to
achieve compliance with Federal securities laws and regulations, as
well as FINRA rules. See Vendor Management and Outsourcing, FINRA
Regulatory Notice 21-29 (Aug. 13, 2021), https://www.finra.org/sites/default/files/2021-08/Regulatory-Notice-21-29.pdf. We also
recently proposed a rule that, if adopted, would govern outsourcing
by investment advisers of certain covered functions, and would in
certain cases require investment advisers to obtain reasonable
assurances that third parties could meet certain standards required
by the Advisers Act and the rules thereunder. See Proposed
Outsourcing Rule, supra note 124.
---------------------------------------------------------------------------

    The Commission is aware that some more complex covered technologies
lack explainability as to how the technology functions in practice, and
how it reaches its conclusions (e.g., a ``black box'' algorithm where
it is unclear exactly what inputs the technology is relying on and how
it weights them). The proposed conflicts rules would apply to these
covered technologies, and firms would only be able to continue using
them where all requirements of the proposed conflicts rules are met,
including the requirements of the evaluation, identification, testing,
determination, and elimination or neutralization sections. For example,
as a practical matter, firms that use such covered technologies likely
may not meet the requirements of paragraph (b) of the proposed
conflicts rules where they are unable to identify all conflicts of
interest associated with the use of such covered technology. However,
in such cases, firms may be able to modify these technologies, for
example by embedding explainability features into their models and
adopting back-end controls (such as limiting the personnel who can use
a technology or the use cases in which it could be employed) in a
manner that will enable firms to satisfy these requirements.
    We request comment on all aspects of the proposed conflict rules'
identification and evaluation requirement, including the following
items:
    21. Do the proposed conflicts rules' identification and evaluation
requirements complement, overlap with, or duplicate the existing
regulatory framework for broker-dealers and investment advisers? If so,
in what ways? Specifically, would firms' compliance with those other
regulatory requirements contribute to compliance with the proposed
conflicts rules, and vice versa?
    22. Is the proposed requirement that a firm evaluate any use or
reasonably foreseeable potential use of a covered technology to
identify any conflict of interest associated with that use or potential
use sufficient for a firm to understand how it should comply with the
proposed conflicts rules? Should firms only be required to evaluate a
technology used in investor interactions and identify associated
conflicts of interest if they reasonably believe their use (or
potential use) of the technology could be associated with a conflict of
interest that results in their interest being placed ahead of
investors' interests? Absent the evaluation and identification required
under the proposed rule, how would firms form such a reasonable belief?
Should we use some other standard, such as a good faith, recklessness,
or actual knowledge standard, or some other option? Would such a
standard be sufficient to protect investors from the potential harmful
impact of conflicts of interest? Is the requirement sufficiently
general that it would continue to apply to future technologies with
features we may not currently anticipate? If we were to provide
additional clarity (whether through guidance or by changing the
regulatory text), how should we ensure that the rule's requirement to
identify and evaluate these conflicts is sufficiently general that it
would continue to apply to future technologies with features or
functionality that we may not currently anticipate? Should we define
the terms ``identify'' or ``evaluate'' in the regulatory text and, if
so, how should they be defined? Should we use different terms to
address this concept and, if so, which terms and how should they be
defined?
    23. The identification and evaluation requirement would also
require firms to identify and evaluate conflicts of interest associated
with use or potential use of a covered technology by an associated
person; what challenges, if

any, would firms face due to this aspect of the proposed conflicts
rules? Should we make any changes as a result? For example, should we
limit the scope of the requirement to conflicts of interest of which
the firm is aware or reasonably should be aware or should we limit the
scope to any conflict that is reasonably foreseeable? Instead of or in
addition to covering conflicts of interest associated with firms'
associated persons' use of covered technologies, should we prescribe
any additional requirements, such as additional diligence or policies
and procedures, relating to conflicts of interest associated with
firms' associated persons' use of covered technologies? The proposed
conflicts rules would consider conflicts of associated persons only for
associated persons that are individuals, and not of entities that
control, are controlled by, or are under common control with a firm,
but many of the Commission's enforcement actions relating to
undisclosed conflicts have involved conflicts of firms' affiliated
entities, and not of individuals.\152\ In addition to natural persons,
should we broaden the requirement to cover entities controlling,
controlled by, or under common control with firms?
---------------------------------------------------------------------------

    \152\ See, e.g., In re. Charles Schwab & Co, supra note 89.
---------------------------------------------------------------------------

    24. Do the proposed conflicts rules provide appropriate clarity
around when a firm uses covered technology in an investor interaction?
For instance, is the guidance included in this release clear that the
proposed conflicts rules would not distinguish between a firm directly
using a covered technology in an investor interaction, such as when an
investor interfaces with the covered technology without an intermediary
of the firm, and when a firm uses covered technology indirectly in an
investor interaction, such as where staff of the firm receives the
output and communicates it to the investor? Do commenters agree with
this scope? Should we instead exclude ``indirect'' use in investor
interactions? Alternatively, should we include indirect uses in
investor interactions but apply the rule differently? If so, what
safeguards, if any, would be necessary or appropriate for indirect uses
in investor interactions? As an example, should the rule make a
distinction between an investor interaction using a covered technology
itself (e.g., a behavioral feature on a digital platform) and an
investor interaction in which the firm uses covered technology
indirectly (e.g., a broker emailing a recommendation that it generated
using AI-tools)? Should we revise the rule text to explicitly include
``indirect'' investor interactions, for example by adding the phrase
``directly or indirectly''? Alternatively, should the rule text include
a definition of ``use'' within the context of a firm's use of a covered
technology in an investor interaction?
    25. How can scalability rapidly exacerbate the magnitude and
potential effect of the conflict in a way that could make full and fair
disclosure and informed consent unachievable or more difficult? Does
this depend on who the investors are (e.g., individuals versus
entities)? Is it possible to disclose conflicts that are associated
with the use of certain covered technologies in a manner that would
enable investors to understand and provide consent? What are the
characteristics of such technologies, and how do they differ from PDA-
like technologies? How should the final conflicts rules account for
such technologies? For instance, should certain uses of covered
technologies by firms not be subject to the identification,
determination, and elimination or neutralization requirements in the
proposed conflicts rules? Should we permit firms to provide disclosure
regarding their use of such technologies as an alternative method of
complying with the proposed conflicts rules? If so, should the final
rules contain principles pursuant to which firms would decide whether
and how they are able to disclose the conflicts? Should the Commission
instead adopt disclosure standards or criteria? What would those
disclosure standards or criteria entail? For example, should one such
standard be that the technology is easily understandable to laypersons?
What would constitute ``easily understandable to laypersons''?
Alternatively, should the Commission set out different classes of
conflicts of interest or different classes of covered technologies and
prescribe different ways to address each such conflicts or
technologies?
    26. Are there particular methods that firms use to identify and
evaluate conflicts of interest that we should discuss in the proposed
conflicts rules? Should we describe particular methods of
identification and evaluation that would comply with the rules? If we
were to address such methods specifically, how would we ensure that the
rule continues to apply to new technologies and new types of investor
interactions as they develop?
    27. How widespread is the use of ``black box''-type models
currently? Under existing law, do firms believe that it is possible to
use black box technologies in compliance with the applicable standard
of conduct and, if so, what steps do they take to comply with the
applicable standard of conduct? How will firms using black box
technologies meet the requirements of the proposed conflicts rules?
Will this require significant changes in firms' practices? What
challenges would firms face when identifying and evaluating conflicts
of interest associated with black box technologies, where the outputs
do not always make clear which inputs were relied on, and how those
inputs were weighted? Are there situations where firms are not able
conclusively to identify and evaluate all potential conflicts of
interest associated with a covered technology, including because it is
a black box? How prevalent are these situations? Will they be able to
identify and evaluate whether a firm interest is being considered, or
to determine whether such interest is being placed ahead of the
interests of investors? Instead of or in addition to the proposed
requirements, should we explicitly require that any technologies used
by firms be explainable? Is our understanding correct that firms could
build ``explainability'' features into the technology in order to give
the model the capacity to explain why it reached a particular outcome,
recommendation, or prediction?
    28. How will firms conduct conflict of interest identification and
evaluation using personnel who are well-trained on both the inner
workings of covered technologies used in investor interactions and how
to identify common conflicts of interest under the applicable standard
of conduct? Are there other methods firms may use, such as third-party
consultants and, if so, should we explicitly address these other
methods? For example, should we explicitly permit or require a firm to
rely on an analysis prepared by a third party identifying and
evaluating the conflicts of interest that could be associated with a
particular covered technology? If we were to explicitly address third-
party analyses, are there particular situations we should address? For
example, should we permit firms to rely on analyses by developers of
covered technologies that are licensed to firms? What standards would
be necessary in order for a firm to reasonably rely on a third-party
analysis? For example, should a third-party analyst be required to
demonstrate a particular level of expertise, possess a particular
credential, certification, or license, or be independent from the
developer of the technology or the firm relying on the analysis? How
should firms address situations where the underlying source code is not
available

or is incomplete, or where it is very complex?
    29. When firms license covered technologies used in investor
interactions, is the available documentation sufficient for them to
determine whether such technologies present conflicts of interest? Is
review of such documentation sufficient for a firm to identify and
evaluate conflicts of interest?
b. Testing
    As part of the identification and evaluation requirement, the
proposed conflicts rules would include a requirement to test each
covered technology prior to its implementation or material
modification, and periodically thereafter, to determine whether the use
of such covered technology is associated with a conflict of
interest.\153\ This obligation would help ensure that conflicts of
interest that may harm investors are identified in light of how the
covered technology actually operates. For example, such testing may
surface additional information that would not be apparent simply from
reviewing the source code or documentation for the covered technology
or the underlying data it uses. It may also surface pre-existing
business practices of a firm where the firm considers firm-favorable
information in its interactions with investors, and the firm's use of
covered technology that replicates such business practices is
associated with a conflict of interest by causing the technology to
consider such firm-favorable information.
---------------------------------------------------------------------------

    \153\ Proposed conflicts rules at (b)(1). Testing would only be
required by the proposed conflicts rules as part of the
identification and evaluation prong of the rules. As a practical
matter, some firms that believe they have eliminated, or neutralized
the effect of, conflicts of interest associated with their use of a
covered technology may wish to confirm this through testing. See
infra section II.A.2.e (describing elimination and neutralization).
---------------------------------------------------------------------------

    Although the proposed rules would not specify any particular method
of testing or frequency of retesting that the firm must conduct, there
are two specific times testing is required. A firm would be required to
conduct testing prior to the covered technology being implemented.\154\
A firm also would be required to conduct testing before deploying any
``material modification'' of the technology, such as a modification to
add new functionality like expanding the asset classes covered by the
technology. We would not generally view minor modifications, such as
standard software updates, security or other patches, bug fixes, or
minor performance improvements to be a ``material modification.''
During the time that the material modifications are being tested, a
firm could continue to use an older version of the covered technology
if the firm's use of such previous version of the technology complies
with the proposed conflicts rules.
---------------------------------------------------------------------------

    \154\ See infra section II.A.2.e for additional information
regarding drift.
---------------------------------------------------------------------------

    The proposed requirement to retest a covered technology
periodically does not specify how often retesting would be required. As
a result, a firm also would need to determine how often, and the manner
in which, to retest covered technologies used in investor
interactions.\155\ As with the proposed identification and evaluation
requirement, a firm's testing methodologies and frequencies may vary
depending on the nature and complexity of the covered technologies it
deploys. Relatively simple or easy-to-understand covered technologies
where the risk of a conflict of interest is low could be subject to
similarly simple testing protocols, and such testing could even take
place concurrently with the firm's efforts to identify and evaluate any
conflicts of interest associated with the covered technology. For
example, firms that use relatively straightforward technology may
determine that it is appropriate to expend the majority of their
testing efforts when technology is first implemented (i.e., first
deployed) or when it is substantially modified, and any periodic
testing may focus only on a sampling of the firm's covered
technologies.
---------------------------------------------------------------------------

    \155\ Though the policies and procedures requirement of the
proposed conflicts rules would not explicitly require a firm to
specify how often it would retest its covered technologies, as a
practical matter, many firms may find it easier to comply with the
requirement to retest their covered technologies periodically by
implementing a policy to guide firm personnel.
---------------------------------------------------------------------------

    On the other hand, firms that use complex covered technologies
generally should use testing methodologies and frequencies that are
tailored to this complexity and that are based on a review of the
particular features that make the technologies more or less likely to
involve a conflict of interest. For example, a firm may determine that
it is necessary to use specific testing methodologies for certain
complex covered technologies. Some covered technologies may need to be
tested using A/B testing to determine what factors are being optimized,
to determine whether any of those factors are the firm's interests (or
act as proxies for the firm's interests), or to estimate the effect of
the methodology with and without the factors that involve the firm's
interests.\156\ Firms may also choose to review data about a
technology's historical performance to monitor signs that it may be
optimizing for firm-favorable factors.
---------------------------------------------------------------------------

    \156\ See Seldon, supra note 74. Though the testing requirement
is contained in section (b)(1) of the proposed conflicts rules,
testing could also be used to aid compliance with other aspects of
the proposed conflicts rules. For example, as discussed infra,
testing may assist a firm in the determination process in section
(b)(2) of the proposed conflicts rules or the elimination and
neutralization process in section (b)(3) of the proposed conflicts
rules.
---------------------------------------------------------------------------

    Likewise, certain learning models are prone to ``drift'' or
``decay,'' which can occur when the data the models were trained on
differs from the data that they encounter once deployed, and their
outputs differ from what would be expected because the training data
did not account for such difference. When models are constantly
optimized, this can result in a feedback loop that, over time,
magnifies small biases and causes the outputs to differ from what would
be expected.\157\ If a model has experienced drift, the drift, on its
own, would not constitute a material modification. But if a firm is
aware that a model is prone to drift (e.g., due to information
developed during the evaluation and identification stage, or through
review of the technology's documentation), the firm would need to take
this into account as it complied with other aspects of the proposed
conflicts rules in order to help ensure that the steps it took to
comply with the proposed rules were effective. A firm that uses covered
technologies that exhibit this phenomenon may determine that it is
necessary to test the technology more frequently to determine if it
continues to function in accordance with the proposed conflict rules,
even if the covered technology has not been modified by the firm. The
same may be true for covered technologies that function with limited
involvement from firm personnel, since otherwise firm personnel may not
immediately notice any changes in how the technology functions.
---------------------------------------------------------------------------

    \157\ See AI Infrastructure Alliance, Everything You Need to
Know about Drift in Machine Learning (May 25, 2022), https://ai-infrastructure.org/everything-you-need-to-know-about-drift-in-machine-learning/.
---------------------------------------------------------------------------

    As firms consider appropriate timing and manner of retesting, they
should consider the nature and complexity of the technology. For
example, a firm may determine to test relatively uncomplicated
technology or technology used only for interactions that are subject to
numerous other compliance controls less frequently than it would test a
very complex technology that interacts directly with investors

without any other human interaction. A firm should also consider
whether covered technology continues to be used as intended and as
originally tested. For example, if a firm originally develops a
technology only for a limited purpose, but then begins to use the
technology in additional investor interactions that differ
substantially from the original use case, the firm may determine it is
necessary to retest the technology with respect to this new use case in
order to determine whether any unforeseen conflicts arise as a result.
    We request comment on all aspects of the proposed conflicts rules'
testing requirement, including the following items:
    30. Is the proposed requirement to test covered technologies used
in investor interactions prior to implementation sufficiently clear?
For example, are there circumstances where it would not be apparent
when a technology has been ``implemented'' for purposes of the proposed
conflicts rules? Should we specifically define the term
``implementation,'' for example by defining it to mean the first time
the technology is used in investor interactions? If a firm deploys a
covered technology on a ``pilot'' basis to a limited group of users,
should this not be considered to be an ``implementation'' for purposes
of the proposed conflicts rules, even if the technology is used in
investor interactions? If we were to provide such an exclusion, what
additional safeguards should be required? For example, should firms
seeking to rely on this exclusion be required to subject the covered
technology to enhanced oversight, such as requiring regular reports on
how the technology is being used, requiring members of the pilot group
to determine independently whether their use of the technology is
resulting in interactions that place the firm's interests ahead of
investors' interests, or only permitting certain firm personnel to use
the technology? Should the exclusion be time-limited, such as a
limitation of 30, 60, or 90 days? Who would be eligible to be in the
pilot group? Should investors be required to be notified, or to
affirmatively consent before interactions with such investors are made
part of such a pilot program? Would such a limitation create incentives
not to test covered technologies thoroughly enough?
    31. Is the proposed requirement to test covered technologies prior
to material modification sufficiently clear? For example, are there
circumstances where it would not be apparent when a technology has been
``materially modified'' for purposes of the proposed conflicts rules?
We expressed our view that normal-course software updates, bug fixes,
and security and other patches are not ``material modifications''
triggering retesting. Should we require testing of such updates, fixes,
and patches? Should we modify the rule text to specify that such
updates and patches are not material modifications? Should we provide
additional guidance on what constitutes a material modification, such
as basing it on ``major'' version numbers (e.g., 1.XXX, 2.XXX, 3.XXX,
etc.) vs. ``minor'' version numbers (e.g., X.01, X.02, X.03, etc.)?
Alternatively, are there situations where reference to version numbers
would be inappropriate, such as when a material change for purposes of
this rule would be assigned a minor version number? Should we make any
special accommodation for technologies that are updated on a regular
schedule, regardless of whether such modifications are material? Should
firms be required to consider the cumulative impact of several
modifications, each of which may not be material on its own, when
considering whether a technology has been materially modified? If an
algorithm itself has not been modified, but the data considered has
been materially modified, should this be treated as a ``material
modification'' for purposes of the proposed conflicts rules? If we were
to do so, should we provide additional guidance on how firms should
decide when a dataset has been materially modified?
    32. Is the proposed requirement to test covered technologies
periodically sufficiently clear? Should firms be able to test different
covered technologies on different timeframes depending on the specific
risks of the covered technologies, as proposed? Should we require that
covered technologies at least be tested on an annual basis or other
specified frequency? Should we require some or all covered
technologies, such as technologies whose outcomes may be difficult to
explain or technologies that operate with limited human interaction, to
be tested more frequently, such as every 30, 60, or 90 days?
    33. Should we specify any particular testing methodologies firms
would be required to use, such as A/B testing? If we were to do so,
should we only require such methodologies to be used on certain types
of technologies and, if so, which ones? For example, should we require
only PDA-like technologies (as opposed to all covered technologies) to
be tested using certain methodologies such as A/B testing? Are there
certain testing methodologies that are only applicable to certain types
of technologies? Are there other methods firms may use to test
compliance with the proposed conflicts rules, such as third-party
consultants and, if so, should we explicitly address these other
methods? For example, should we explicitly permit or require a firm to
rely on an analysis prepared by a third party? If we were to explicitly
address third-party analyses, are there particular situations we should
address? For example, should we permit firms to rely on analyses by
developers of covered technologies that are licensed to firms? What
standards would be necessary in order for a firm to reasonably rely on
a third-party analysis? For example, should a third-party analyst be
required to demonstrate a particular level of expertise, possess a
particular credential, certification, or license, or be independent
from the developer of the technology or the firm relying on the
analysis?
    34. Should we provide an exception from the testing requirement?
For example, for urgent changes that are necessary to protect against
immediate investor harm, for regulatory reasons, or to correct
unexpected developments, such as major bugs, security issues, or
conflicts of interest that had not previously been identified (or that
developed between periodic testing intervals). Should we require firms
to create or maintain any documentation in connection with relying on
such an exception? Should reliance on such an exception be subject to
any conditions, such as conducting testing as soon as practicable or
only for a limited, specified period of time (for example, a few days,
a week, or a month)?
    35. Should we provide a temporary exception from the testing
requirement for technologies that are already in use by firms and, if
so, when should that exception expire? If we were to provide a
temporary exception for technologies that are already in use, should
the temporary exception also apply to other aspects of the proposed
conflicts rules, such as the identification and evaluation,
determination, or elimination or neutralization prongs, the policies
and procedures requirement, or the proposed recordkeeping amendments?
c. Conflict of Interest
    Under the proposed conflicts rules, a conflict of interest would
exist when a firm uses a covered technology that takes into
consideration an interest of the firm or its associated persons. The
proposed conflicts rules would cover use of a covered technology by
both a firm and associated persons of the firm

and would address technologies that take into account both interests of
the firm and the interests of its associated persons.\158\ The proposed
conflicts rules would define ``conflict of interest'' broadly and make
clear that, if a covered technology considers any firm-favorable
information in an investor interaction or information favorable to a
firm's associated persons, the firm should evaluate the conflict and
determine whether such consideration involves a conflict of interest
that places the interest of the firm or its associated persons ahead of
investors' interests and, if so, how to eliminate, or neutralize the
effect of, that conflict of interest.
---------------------------------------------------------------------------

    \158\ See paragraph (a) of the proposed conflicts rules. As
discussed previously, while the use of covered technology that takes
into consideration an interest of the firm or an associated person
could present a conflict of interest, the proposed conflicts rules
would provide an exception for situations where the covered
technology is used in investor interactions solely for purposes of
meeting legal or regulatory obligations or providing clerical,
ministerial, or general administrative support. See proposed
conflicts rules at paragraph (a) and discussion supra section
II.A.1.b.
---------------------------------------------------------------------------

    We recognize that the proposed conflicts rules--including the broad
definition of conflict of interest--means that some conflicts will be
identified that do not place the interests of the firm or its
associated persons ahead of those of investors, and thus would not need
to be eliminated or their effect neutralized. However, a covered
technology may consider many factors (e.g., as part of an algorithm or
data input). One factor among three under consideration by the
technology may be highly likely to cause the technology to place the
interests of the firm ahead of investors, and the effect of considering
that factor may be readily apparent. On the other hand, one conflicted
factor among thousands in the algorithm or data set upon which a
technology is based may, or may not, cause the covered technology to
produce a result that places the interests of the firm ahead of the
interests of investors, and the effect of considering that factor may
not be immediately apparent without testing (as discussed above).
Without a broad definition and resulting evaluation, this
differentiation among factors that do, and do not, result in investor
interactions that place the firm's interests ahead of investors'
interests may be impossible.
    There are many ways in which a use of covered technology in
investor interactions can be associated with a conflict of interest.
For example, when covered technology takes into account the profits or
revenues of the firm, that would be a conflict of interest under the
proposal regardless of whether the firm places its interests ahead of
investors' interests. Revenue or profits can be taken into account
directly, such as if a firm populates an asset allocation algorithm on
its website to prioritize investments that it is trying to promote
because it benefits the firm (e.g., by over-weighting funds that make
revenue sharing payments or proprietary funds).\159\ Likewise, if a
firm deploys a covered technology to interact with an investor, such as
by displaying selected or ranked options for retirement accounts that
takes into account the amount of revenue the firm would receive, the
firm's use of the covered technology would involve a conflict of
interest regardless of whether the firm places its interests ahead of
investors' interests.
---------------------------------------------------------------------------

    \159\ A conflict could exist irrespective of whether investment
in such funds is in the best interest of the investor.
---------------------------------------------------------------------------

    Revenue or profits to the firm can also be indirectly taken into
consideration and trigger the proposed conflicts rules, such as through
incentivizing increased trading activity or opening of options or
margin accounts, if increased trading or opening of such accounts would
cause the firm to experience higher profits, such as through increased
commissions or revenue sharing from the wholesaler that executes the
trade or through increased profits for the firm.\160\ For example, if a
firm uses a neural network to provide investment advice or generate
general investment ideas to populate an investment allocation tool, the
network may be caused to ingest vast amounts of historical or real-time
data, then repeatedly be optimized or trained to determine which
outcome(s) to generate.\161\ If one of the pieces of data that the
neural network considers is the effect on the firm's interests, such as
the firm's profitability or revenue, it involves a conflict that should
be examined to determine whether it could produce outcomes, including
changing outcomes over time (e.g., through drift), that place the
interest of the firm ahead of the interest of the investor.
---------------------------------------------------------------------------

    \160\ These conflicts are distinct from the limited exception
for conflicts of interest associated with more generally attracting
investors to open new accounts, discussed in section II.A.2.e,
infra, because generally attracting new investors is essential to
the business of any firm. On the other hand, incentivizing specific
types of activity (such as margin or options trading privileges, as
opposed to opening a general account, or investing in a particular
type of investment, as opposed to just opening an account to invest)
that is particularly profitable to a firm (and is not always in
investors' interest), is intentionally addressed by the proposed
conflicts rules.
    \161\ See, e.g., Alexey Dosovitskiy, Google Research, Optimizing
Multiple Loss Functions with Loss-Conditional Training (Apr. 27,
2020), https://ai.googleblog.com/2020/04/optimizing-multiple-loss-functions-with.html.
---------------------------------------------------------------------------

    The specific interest that is taken into account, and the degree to
which it is weighted in the covered technology, would not affect the
determination of whether a conflict of interest exists, as the presence
of any firm interest in any degree, for the reasons discussed above,
would constitute a conflict of interest. Such considerations would be
relevant, however, when considering whether the conflict of interest
places the interest of the firm ahead of those of investors and
therefore whether it is necessary to eliminate, or neutralize the
effect of, the conflict of interest, as discussed further below, and,
if so, what steps could be taken to do so.\162\
---------------------------------------------------------------------------

    \162\ See infra section II.A.2.e.
---------------------------------------------------------------------------

    We request comment on all aspects of the proposed definition of
conflict of interest, including the following items:
    36. Do commenters agree that a firm would have a conflict of
interest with an investor if the firm takes into consideration its
profits and revenues in its investor interactions using covered
technology? Why or why not? Are there additional circumstances that
should trigger the rule if the firm takes these circumstances into
account in its investor interactions, such as considering any factor
which is not directly in the interest of the investor? Should we narrow
the proposed definition and, if so, are there particular activities
that should be excluded, such as when a technology considers a very
large dataset where the firm has no reason to believe that the data
considers the interests of the firm, like a technology trained on all
books in the English language? Are there other datasets that should be
excluded and, if so, how broad should a dataset be required to be in
order to qualify for the exclusion? If we were to provide an exclusion,
should we do so by excluding particular activities or types of datasets
by name, or through a more principles-based approach?
    37. Is the description of when a conflict of interest exists
sufficiently clear? Would firms be able to identify what would and
would not be a conflict of interest for purposes of the rules? Advisers
already have a fiduciary duty to eliminate, or at least to expose, all
conflicts of interest which might incline them--consciously or
unconsciously--to render advice that is not disinterested, and broker-
dealers already have a duty to identify and at a minimum disclose or
eliminate all conflicts of interest associated with a recommendation
and mitigate certain conflicts of interest under Reg BI. How

do firms currently identify conflicts of interest associated with their
use of what the proposed conflicts rules would define as covered
technologies in order to ensure that such use complies with existing
standards? Will it be confusing to firms that the proposed conflicts
rules also use the term ``conflict of interest'' to describe a
distinct, but related, concept? If so, should we use a different term
other than ``conflict of interest,'' such as a ``technology conflict''
or a ``potential conflict of interest?''
    38. The proposed definition of ``conflict of interest'' would also
include interests of firms' associated persons. What challenges, if
any, would firms face due to this aspect of the proposed conflicts
rules? Should we make any changes as a result? For example, should we
limit the scope of the definition to conflicts of interest of which the
firm is aware or reasonably should be aware? Instead of or in addition
to covering conflicts of interest that arise due to the interests of
firms' associated persons, should we prescribe any additional
requirements, such as additional diligence or policies and procedures,
relating to conflicts of interest of firms' associated persons? In
addition to natural persons, should we explicitly adopt a definition of
``conflict of interest'' that would cover interests of entities
controlling, controlled by, or under common control with firms, or
other affiliates (or modify the rule provisions requiring the
consideration of conflicts of associated persons to remove the
limitations to associated persons that are natural persons)?
    39. If we were to provide an exclusion for technologies that
consider large datasets where firms have no reason to believe the
dataset favors the interests of the firm, should we require such
datasets to meet minimum standards? For example, should we require
firms to conduct diligence regarding how the data was collected in
order to support their determination that the dataset does not
incorporate the firm's interests? Should there be different standards
for data that is itself generated in part by a technology that may meet
the definition of covered technology (and thus may incorporate its own
conflicts of interest), such as subjecting that technology to all or
part of the proposed rules?
    40. Should we incorporate other minimum standards into data
considered by covered technologies that are not directly related to
interests of the firm but may implicate other Commission priorities, or
have public policy implications? For example, should we require firms
to take steps to understand whether the data does or could involve
material nonpublic information? Should firms be required to consider
whether the data is sensitive data that could be subject to
cybersecurity or privacy rules?
    41. Do firms ever provide firm-favorable information to their
covered technologies for the purpose of explicitly instructing the
covered technology not to consider such information? Are there other
circumstances in which covered technologies consider firm-favorable
information that do not raise conflict of interest concerns? If so,
should we make any changes to the definition of conflict of interest as
a result? How could firms determine that no conflict of interest
concerns are associated with their use of a covered technology without
conducting the steps that would be required under the proposed
conflicts rules?
    42. Is it clear that the proposed definition of conflict of
interest includes when the covered technology has the potential to take
into account the firm's (or its associated persons') interests,
including the firm's revenue or profits, directly or indirectly? Are
there steps we could take to clarify, for example by providing
additional examples of factors that, if considered, would constitute a
conflict of interest?
    43. Do commenters agree that, as proposed, a conflict of interest
would exist even if a covered technology factors in a single firm- or
associated person-favorable interest among many other factors that do
not favor the firm or its associated person, regardless of which
interest is favored and the degree to which it is weighted? Should the
specific interest of the firm or associated person that is taken into
account, such as the firm's revenues or profits, or the degree to which
it is weighted in the covered technology, affect the determination of
whether a conflict of interest exists at all? How would this differ in
practice from determining that a conflict of interest does exist but
does not place the firm's interests ahead of investors' interests?
    44. Should we exclude certain categories of conflicts?
d. Determination
    The proposed conflicts rules would require a firm, after evaluating
any use or reasonably foreseeable potential use of a covered technology
by a firm or its associated person in any investor interaction to
identify any conflict of interest associated with that use or potential
use, to determine whether such conflict of interest places or results
in placing the firm's or its associated person's interest ahead of
investors' interests, subject to certain exceptions.\163\ Determining
whether an investor interaction involving such a conflict of interest
would place or results in placing the firm's or its associated person's
interests ahead of investors' interests is a facts and circumstances
analysis, and would depend on a consideration of a variety of factors,
such as the covered technology, its anticipated use, the conflicts of
interest involved, the methodologies used and outcomes generated, and
the interests of the investor. Based on this analysis, a firm must
reasonably believe that the covered technology either does not place
the interests of the firm or its associated persons ahead of investors'
interests, or the firm would need to take additional steps to
eliminate, or neutralize the effect of, the conflict.\164\ Applicable
law already limits firms' use of technologies whose outputs are based
in part on data points favorable to a firm in certain circumstances.
Investment advisers using such technologies to provide investment
advice are already required to consider whether they could cause the
adviser ``consciously or unconsciously to render advice which is not
disinterested.'' \165\ Similarly, broker-dealers that use technology to
make certain recommendations to a retail customer must establish,
maintain, and enforce written policies and procedures reasonably
designed to achieve compliance with Reg BI, including its Conflict of
Interest Obligation.\166\
---------------------------------------------------------------------------

    \163\ Proposed conflicts rules at (b)(2).
    \164\ The proposed conflicts rules do not prescribe strict
numerical weights. Instead, determination of the relative level of
benefits to the firm and to the investor should take into account
all applicable facts and circumstances.
    \165\ See Fiduciary Interpretation, supra note 8.
    \166\ See Exchange Act rule 15-1(a)(2)(iii) and (iv).
---------------------------------------------------------------------------

    In the case of many covered technologies, it may be readily
apparent that, while the technology may take into account an interest
of the firm, it does not result in the firm's interests being placed
ahead of investors' interests. For example, many investment advisers
create financial models of a portfolio company's three financial
statements (i.e., the company's balance sheet, income statement, and
statement of cashflows) to help evaluate whether to advise their
clients to invest in a particular portfolio company. It is not uncommon
for a financial model to show the potential returns of the investment
for the client, along with a potential performance-based fee that would
be received by the adviser, if the portfolio company achieved certain
levels of growth. An adviser's consideration of metrics that are
favorable to it, such as a potential

performance-based fee it could receive, would constitute a conflict of
interest under the proposed conflicts rules. Under the determination
requirement, however, the adviser could, based on the applicable facts
and circumstances, determine that such conflict of interest does not
result in its own interests being placed ahead of investors' interests
if the outcome is equally (or more) favorable to the investor
regardless of whether the factor is considered.\167\
---------------------------------------------------------------------------

    \167\ Even though the proposed conflicts rules would not require
the conflict of interest to be eliminated or its effect to be
neutralized, this would remain a conflict of interest under the
proposed conflicts rules (and under existing law). See Performance-
Based Investment Advisory Fees, Investment Advisers Act Release No.
5904 (Nov. 4, 2021) [86 FR 62473 (Nov. 10, 2021)], at n.3 and
accompanying text (noting the incentive ``to engage in speculative
trading practices while managing client funds in order to realize or
increase [contingent] advisory fees'' such as incentive
allocations). An adviser would still be required to disclose the
conflict with sufficient specificity that a client could provide
informed consent. See Fiduciary Interpretation, supra note 8, at
nn.67-70 and accompanying text.
---------------------------------------------------------------------------

    On the other hand, if the model is designed to screen out an
investment if it would not result in a sufficient performance-based fee
for the adviser despite acceptable returns for investors, this would be
an example of the adviser's interests being placed ahead of investors'
interests because the investors are being deprived of an investment due
to the adviser's consideration of its own interest. Covered
technologies like the model in this example, which explicitly and
intentionally consider a firm's interests as an integral part of its
outputs, are highly likely to result in investor interactions that
place the interests of the firm ahead of investors' interests. Firms
should consider carefully reviewing the outputs of such technologies to
determine whether the firm's or its associated persons' interests are
being placed ahead of the interests of the investor (e.g., by reviewing
how the outputs vary if the firm's or associated persons' interests are
not considered).
    Similarly, a broker-dealer may bring general investment ideas to
the attention of retail investors, using an algorithm for selection,
where some of the investments that may be selected provide revenue to
the firm if the investor places an order to purchase. If the firm
determines that in selecting the investment ideas, the algorithm used
for selecting the investment ideas does not place the firm's interests
ahead of investors' interests--because, for example, it does not give
more prominence to the investments that provide revenue to the firm
than those that do not and no one investment is being recommended--it
could reasonably determine that the conflict of interest created by the
algorithm considering the revenue does not require elimination or
neutralization under the proposed conflicts rules.\168\
---------------------------------------------------------------------------

    \168\ While the proposed conflicts rules may not require
elimination or neutralization, to the extent a broker-dealer uses
such technology to make a recommendation to a retail customer, other
existing regulatory obligations, such as Reg BI and Form CRS, would
apply. See supra section I.B.
---------------------------------------------------------------------------

    If, on the other hand, the firm determined that the algorithm was
more likely to give greater prominence to those investments that are
more profitable for the firm over other options of equal or better
quality, then it could not reasonably determine that the conflict does
not result in investor interactions that place its interests ahead of
investors' interest and thus, would be required to eliminate, or
neutralize the effect of, the conflict by the proposed conflicts rules.
As another example, the covered technology a firm uses to decide when
to communicate with investors may send an automatic message to
investors encouraging them to ``hold steady'' during a period of high
volatility in the market. If the technology is programmed to send out
such a message during a period of high volatility but only after a
certain threshold of fee-earning assets are withdrawn from the firm,
the use of that technology would involve a conflict of interest because
it would consider a proxy for the firm's revenues. However, if the
primary purpose of the automatic message is to keep investors from
over-reacting to short-term market moves, that could be beneficial for
such investors. Even though the firm would be required to identify and
evaluate the conflict of interest in order to comply with the proposed
conflicts rules, the firm could reasonably determine that its interests
were not placed ahead of investors' interests, and thus it did not need
to eliminate, or neutralize the effect of, the conflict of interest.
    A firm generally should tailor the methods by which it determines
whether its use of covered technologies in investor interactions places
its interests ahead of investors based on the circumstances and the
complexity of the underlying covered technology as well as the
complexity of the conflict of interest. To the extent a firm has
difficulty identifying whether a use of a covered technology in an
investor interaction presents a conflict of interest within the meaning
of the proposed conflicts rules, it also would have difficulty
determining whether the technology could place the interests of the
firm ahead of the interests of investors.\169\ In such circumstances,
the firm may need to use additional tools to comply with the proposed
determination requirement. For example, if a firm built
``explainability'' functionality into the covered technology that gives
the model the capacity to explain why it reached a particular outcome,
recommendation, or prediction, this functionality could assist with the
identification and determination elements of the proposed conflicts
rules.\170\ A firm using explainability features could review the
output to determine whether the firm's interests were being placed
ahead of those of investors and, in any circumstance where it was not
clear whether the firm's interests were being placed ahead of
investors, the firm could comply with the proposed conflicts rules for
example, by ceasing to use the technology or by prophylactically
treating such an ambiguity as a conflict of interest that must be
eliminated or its effect neutralized.\171\
---------------------------------------------------------------------------

    \169\ See supra note 151 and surrounding text (discussing
building explainability features into ``black box'' algorithms). We
believe that the ``should have identified'' standard in paragraph
(b)(3) of the proposed conflicts rules addresses situations where a
firm's determination that a conflict of interest does not place its
interests ahead of investors' turns out to be unreasonable because
it would still hold a firm accountable for the unreasonable
determination. See infra section II.A.2.e.
    \170\ See id.
    \171\ See infra section II.A.2.e.
---------------------------------------------------------------------------

    Even when explainability features are built into a covered
technology, a firm might still be unable to determine whether the
covered technology places its own interests ahead of investors'
interests. If a firm cannot determine that its use of a covered
technology in investor interactions does not result in a conflict of
interest that places its interests ahead of those of investors, the
firm generally should consider any conflict of interest associated with
such use as one that must be eliminated or its effect neutralized, and
take steps necessary to do so.\172\ For example, as

explained more fully in the following section, the firm could apply a
``counterweight'' to a conflict (that is, it could give more weight to
certain investor-favorable information in order to make up for the
consideration of firm-favorable information) that would be sufficient
to neutralize the effect of conflicts that the firm reasonably foresees
could result from the use of the covered technology.\173\ We
acknowledge determinations for covered technologies that consider a
multitude of different data points may render it more challenging to
isolate the effect of any particular data point on the outcome and,
thus, to determine whether it causes a conflict of interest that places
the interest of the firm ahead of investors. These cases, in
particular, may benefit from the testing methods outlined above. For
example, A/B testing may reveal that there is no difference in outcomes
in cases where the covered technology includes or excludes certain data
points or groups of data points.
---------------------------------------------------------------------------

    \172\ See infra section II.A.2.e (discussing the ``should have''
identified standard). Firms that are unable to determine whether
their own interests are placed ahead of investors' for purposes of
the proposed conflicts rules should consider whether full and fair
disclosure to facilitate informed consent are feasible in such
circumstances. See, e.g., infra note 316 and accompanying text
(discussing informed consent in the context of highly complex
algorithms). In such circumstances, when informed consent is
impossible, existing law requires an investment adviser to mitigate
the conflict, which could include steps similar to those we outline
in the discussion of elimination and neutralization. Similarly,
where a broker-dealer that makes a recommendation to a retail
customer using covered technology cannot provide ``full and fair''
disclosure of a conflict of interest, the broker-dealer may need to
take additional steps to mitigate or eliminate the conflict under
the existing standard of conduct. See Reg BI Adopting Release, supra
note 8, at section I and text accompanying nn.735-36 (``[B]roker-
dealers are most capable of identifying and addressing the conflicts
that may affect the obligations of their associated persons with
respect to the recommendations they make, and are therefore in the
best position, to affirmatively reduce the potential effect of these
conflicts of interest such that they do not taint the
recommendation.'').
    \173\ This is due to the ``should have identified'' standard.
See infra section II.A.2.e.
---------------------------------------------------------------------------

    We request comment on all aspects of the proposed conflict rules'
determination requirement, including the following items:
    45. Does the proposed conflicts rules' determination requirement
complement, overlap with, or duplicate the existing regulatory
framework for broker-dealers and investment advisers? If so, in what
ways? Specifically, would firms' compliance with those other regulatory
requirements contribute to compliance with the proposed conflicts
rules, and vice versa?
    46. Is the proposed requirement that a firm determine whether any
conflict of interest that it has identified places or results in
placing its or its associated persons' interests ahead of investors'
interests sufficiently clear? Is the requirement sufficiently general
that it would continue to apply to future technologies with features we
may not currently anticipate? If not, why not? Do commenters agree that
a conflict of interest that places a firm's or its associated persons'
interests ahead of investors' interests also results in placing its or
its associated persons' interests ahead of investors' interests? If so,
is the rule clearer by including both phrases or should the proposed
requirement eliminate the phrase ``results in placing''?
    47. How do firms currently determine whether their use of
technology in investor interactions results in a conflict of interest
that places the interests of the firm ahead of investors' interests?
Are there particular processes or strategies that should be required in
the proposed determination requirement? For example, should we
specifically require the use of ``explainability'' features when the
relationship between the outputs of a model and the inputs may be
unclear (and it thus may be difficult to identify whether the interests
of the firm are being placed ahead of investors' interests)? Do firms
use A/B testing to determine the effects of conflicts of interest? What
other types of testing do firms use to determine the effects of
conflicts of interest, if any?
    48. What challenges will firms face when determining whether
conflicts of interest associated with ``black box'' technologies (where
the outputs do not always make clear which inputs were relied on, and
how those inputs were weighted), result in their interests being placed
ahead of those of investors? How prevalent are these situations? How do
firms using ``black box'' technologies to aid in making recommendations
or providing advice determine whether they are complying with existing
conflicts obligations under the investment adviser fiduciary standard
and Reg BI, as applicable? If a firm is not able to determine whether
its use of such a technology results in a conflict of interest that
places its interests ahead of those of investors, what additional steps
will a firm need to take in order to eliminate, or neutralize the
effect of, such conflicts and be able to continue to use the covered
technology?
    49. The determination requirement would also require firms to
determine whether the interests of an associated person of a firm are
placed ahead of investors' interest. What challenges, if any, would
firms face due to this aspect of the proposed conflicts rules? Should
we make any changes as a result? For example, should we limit the scope
of the requirement to conflicts of interest of which the firm is aware
or reasonably should be aware? Instead of or in addition to covering
firms' associated persons' interests, should we prescribe any
additional requirements, such as additional diligence or policies and
procedures, relating to conflicts of interest associated with firms'
associated persons? In addition to natural persons, should the
determination requirement apply in the context of entities that
control, are controlled by, or are under common control with firms?
    50. Should we expand the determination requirement to cover other
situations that would not be a ``conflict of interest'' as defined
under the proposed conflicts rules, but would implicate other Federal
securities laws, or other laws? For example, should firms be required
to identify and evaluate whether their covered technologies use or
consider any information that could be material nonpublic information?
    51. Are there other methods firms may use to determine whether a
conflict of interest results in placing the interest of the firm or an
associated person of the firm ahead of the investor, such as third-
party consultants and, if so, should we explicitly address these other
methods? For example, should we explicitly permit or require a firm to
rely on an analysis prepared by a third party? If we were to explicitly
address third-party analyses, are there particular situations we should
address? For example, should we permit firms to rely on analysis by
developers of covered technologies that are licensed to firms? What
standards would be necessary in order for a firm to reasonably rely on
a third-party analysis? For example, should a third-party analyst be
required to demonstrate a particular level of expertise, possess a
particular certification or license, or be independent from the
developer of the technology or the firm relying on the analysis?
e. Elimination or Neutralization of Effect
    The proposed conflicts rules would require a firm to eliminate, or
neutralize the effect of, any conflict of interest it determines
results in an investor interaction that places the firm's (or its
associated persons') interest ahead of the interests of its
investors.\174\ Consideration of any firm interest would be sufficient
for a conflict of interest to exist under the proposed conflicts rules,
but the consideration of a firm's interest, on its own, would not
necessarily require that the firm eliminate, or neutralize the effect
of, the conflict of interest.\175\ After identifying that a conflict of
interest exists, the firm would then determine whether the conflict of
interest results in the interest of the firm or an associated person
being placed ahead of investors' interests. Only where the firm makes
(or reasonably should make) such a

determination would the firm be required to eliminate, or neutralize
the effect of, the conflict of interest.\176\ The proposed conflicts
rules would require the firm to eliminate, or neutralize the effect of,
any such conflict promptly after the firm determines, or reasonably
should have determined, the conflict placed the interests of the firm
or associated person ahead of the interests of investors. This
requirement is designed to require a firm to take steps that are in
addition to, but not in conflict with, the standard of conduct that
applies when it is providing advice or making recommendations, as
discussed below.\177\
---------------------------------------------------------------------------

    \174\ Proposed conflicts rules at (b)(3).
    \175\ See infra section II.A.2.d.
    \176\ For the avoidance of doubt, the discussion concerns
consideration by a technology of the interests of a firm, including
situations where the firm creates technology that considers the
firm's or an associated person's interests. Firms of course will
consider their own interests (such as whether the cost of the
technology is worth the benefit) when determining whether to deploy
a technology. Such consideration, on its own, would not be within
the scope of the proposed conflicts rules.
    \177\ See infra section III.C.3. (describing the applicable
standards of conduct).
---------------------------------------------------------------------------

    The test for whether a firm has successfully eliminated or
neutralized the effect of a conflict of interest is whether the
interaction no longer places the interests of the firm ahead of the
interests of investors.\178\ Under the proposed conflicts rules, a firm
could ``eliminate'' a conflict of interest, for example, by completely
eliminating the practice (whether through changes to the algorithm,
technology, or otherwise) that results in a conflict of interest or
removing the firm's interest from the information considered by the
covered technology. For example, a firm that determined covered
technology used in investor interactions favored investments where its
receipt of revenue sharing payments placed the firm's interests ahead
of investors' interests could eliminate the conflict, among other
methods, by ending revenue sharing arrangements or by ensuring that its
covered technologies do not consider investments that pay it revenue
sharing payments.
---------------------------------------------------------------------------

    \178\ For the avoidance of doubt, if a firm substitutes one
firm-favorable factor with a different factor that is a proxy for
the firm-favorable factor, the firm has not eliminated, or
neutralized the effect of, the conflict.
---------------------------------------------------------------------------

    However, a firm does not have to eliminate such conflicts. A firm
instead could ``neutralize the effect of'' a conflict of interest by
taking steps to address the conflict. In this regard, whether through
elimination or neutralization, the proposed conflicts rules would
require that any conflicts of interest not place the firm's interest
ahead of investors' interests. In a neutralization scenario, the
covered technology could continue to use the data or algorithm that
includes the firm's or associated person's interest as a factor, but
the firm would be required to take steps to prevent it from biasing the
output towards the interest of the firm or its associated persons. The
measure of whether the effect of the conflict has been neutralized
would be if the investor interaction does not place the firm's or
associated person's interest ahead of the investor. We are including
neutralization as an additional method of addressing conflicts of
interest under the proposed conflicts rules because of the unique ways
that technology can be modified or counterweighted to eliminate the
harmful effects of a conflict, as well as the ways it can be tested to
confirm the modification or counterweighting was successful.
    Neutralization, for example, also could include rendering the
consideration of the firm-favorable information subordinate to
investors' interests, and thus making the conflict harmless, either by
applying a ``counterweight'' (such as considering additional investor-
favorable information that would not have otherwise have been
considered in order to counteract consideration of a firm-favorable
factor) or by changing how the information is analyzed or weighted such
that the technology always holistically weights other factors as more
important so that biased data cannot affect the outcome.
    The proposed conflicts rules do not prescribe a specific way in
which a firm must eliminate, or neutralize the effect of, its conflicts
of interest. For example, if a firm that is a robo-adviser determines
that it uses covered technology to direct or steer investors to invest
in funds the firm itself sponsors and advises when more suitable or
less expensive options for the investor are available through the robo-
adviser, and thereby prioritizes the firm's own profit over investors'
interests, the firm could eliminate this conflict of interest by
removing any data that would allow the robo-adviser to determine which
funds are sponsored or advised by the firm, thus eliminating any bias
in favor of the firm's interest.\179\ The firm, alternatively, may
choose to neutralize the effect of the conflict.\180\ For instance, the
firm could neutralize the effect of the conflict of interest by
sufficiently increasing the weights given to factors, such as cost to
the investor or risk-adjusted returns (including, in each case,
comparisons to funds sponsored or advised by other firms), to provide a
counterweight that prevents any consideration of the firm's own
interests from resulting in an investor interaction that places the
firm's interests ahead of investors' interests. The proposed conflicts
rules permit firms discretion on how to address the conflict--whether
by eliminating it altogether or neutralizing its effect--after
considering the applicable facts and circumstances, provided that the
method used prevents the firm from placing its interests or an
associated person's ahead of investors' interest.
---------------------------------------------------------------------------

    \179\ As discussed supra section II.A.1.b, this includes a
discretionary adviser where the investor does not need to approve
each trade; the investor interaction in this case would be in the
form of engagement through directing trades in the investor's
account.
    \180\ As discussed above, this is also consistent with an
adviser's fiduciary duty. An adviser ``must, at all times, serve the
best interest of its client and not subordinate its client's
interest to its own'' and, unless neutralized, a conflict of
interest would have the effect of subordinating a client's interest
to that of the firm. See Fiduciary Interpretation, supra note 8.
Similarly, under Reg BI, broker-dealers must mitigate (i.e., reduce)
or eliminate conflicts of interest that would otherwise cause the
broker-dealer or its associated person to make a recommendation that
is not in the best interest of the retail customer. See Exchange Act
rule 15l-1(a)(2)(iii); Reg BI Adopting Release, supra note 8, at
section II.C.3.g (``Elimination of Certain Conflicts of Interest'').
---------------------------------------------------------------------------

    The proposed conflicts rules do not prescribe a particular manner
by which a firm must eliminate, or neutralize the effect of, any
conflict of interest because of the breadth and variations of firms'
business models as well as their use of covered technology. Because of
the complexity of many covered technologies, as well as the ways in
which conflicts of interest may be associated with their use, we are
concerned that prescribing particular means to neutralize the effect of
a conflict of interest could be inapplicable or otherwise ineffective
with respect to certain covered technologies (or certain conflicts of
interest, the nature and extent of which may vary substantially across
firms depending on their particular business models and investor
base).\181\ The proposed approach is intended to promote flexibility
and innovation by allowing the firms that use covered technologies the
freedom to determine the appropriate ways to operate them, within the
guardrails provided by the proposed conflicts rules, rather than
requiring the technologies to be designed in a

particular way solely to meet a regulatory requirement.
---------------------------------------------------------------------------

    \181\ This same recognition of the complexity of many covered
technologies is why disclosure alone could be insufficient to
adequately address the conflicts of interest associated with their
use. Cf. infra section III.D.1 (disclosure alone may not necessarily
address negative outcomes when ``the issue lies in human
psychological factors, rather than a lack of information.'').
---------------------------------------------------------------------------

    We recognize that reasonable steps a firm could take to eliminate,
or neutralize the effect of, a conflict of interest that results in an
investor interaction that places the firm's interest ahead of
investors, are likely to vary and would depend on the nature of the
conflict, the nature of the covered technology, the circumstances in
which the covered technology is used, and the potential harm to
investors. For example, if the firm's evaluation of the conflict
indicates that the technology would only result in investor
interactions that place the firm's or an associated person's interests
ahead of investors' interests in certain limited circumstances, a firm
could eliminate the conflict of interest by taking steps to prevent the
technology from being used in such circumstances, or by choosing to
eliminate the business practice that is associated with the conflict in
the first place. Similarly, if a technology only involves a conflict of
interest due to its consideration of certain data or the weights
ascribed to certain data points, the firm could either prevent the
technology from accessing such data (eliminating the conflict), or the
firm could take steps to prevent its consideration of the data from
having an effect on the outcome of the technology (neutralizing the
effect of the conflict), either through consideration of additional,
investor-favorable data designed to provide a countervailing signal to
the technology, or through weighting the data the covered technology
considers so that the firm- or associated person-favorable data would
not be determinative to the outputs.\182\ A firm could also neutralize
the effect of a conflict by requiring that firm personnel who are
trained on the nature of the conflict of interest (e.g., personnel
responsible for supervising the implementation of the firm's compliance
program) operate the technology and only pass along information to
investors after they deem, based on their training, that the
information does not involve a conflict that results in an investor
interaction that places the interests of the firm or an associated
person ahead of investors' interests.\183\
---------------------------------------------------------------------------

    \182\ Whether the firm-favorable data is determinative of the
technology's outputs could be verified through A/B testing. See
supra section II.A.2.b. The specific data or weights that would be
necessary to neutralize a particular conflict would depend on
factors such as the conflict itself as well as the design of the
applicable technology.
    \183\ This example assumes the investor interaction is indirect;
we anticipate that firm personnel would not have the ability to
intervene when a technology directly interacts with investors.
---------------------------------------------------------------------------

    The proposed conflicts rules would require a firm to eliminate, or
neutralize the effect of, a conflict of interest that it determines
results in an investor interaction that places its interests ahead of
investors' interests ``promptly'' after the firm determines, or
reasonably should have determined, that the conflict results in its own
(or an associated person's) interests being placed ahead of investors'
interests.\184\ Determining what constitutes ``promptly'' in any given
situation under the proposed conflicts rules would depend on the facts
and circumstances. If eliminating, or neutralizing, the effect of, the
conflict is straightforward, as would be the case if a firm simply had
to update the settings of an application or restrict access using tools
it already possessed, elimination or neutralization could happen soon
after the identification of the conflict of interest.
---------------------------------------------------------------------------

    \184\ If it is determined before technology is first deployed
that a conflict of interest exists that places the firm's or an
associated person's interests ahead of investors' interests,
``prompt'' elimination or neutralization of the conflict could occur
any time before the technology is initially deployed. That is, we do
not believe it would be consistent with the proposed conflicts rules
for a firm to initially deploy a technology that a firm has already
determined (or should have determined) is subject to conflicts of
interest that place the firm's or an associated person's interests
ahead of its investors' interests, then eliminate, or neutralize the
effect of, those conflicts after the fact.
---------------------------------------------------------------------------

    But if elimination, or neutralization of the effect of, a conflict
of interest would require substantial amounts of new coding by firm
personnel, we recognize that such modifications may take longer to
implement, including because they may constitute material modifications
that would need to be tested to determine whether any modifications
eliminated, or neutralized the effect of, the conflict as expected, as
well as to consider any new conflicts of interest that the
modifications could cause. Though we recognize that modifications would
not happen immediately in all circumstances, an extended period of
implementation may raise questions about whether the firm acted
promptly and may raise questions as to whether they are acting in
accordance with their standard of care. If a firm has determined that
it needs additional time to eliminate, or neutralize the effect of, a
conflict of interest in accordance with the proposed conflicts rules,
it would also need to consider whether continuing to use such covered
technology before the conflict is eliminated or neutralized would
violate any applicable standard of conduct (e.g., fiduciary duty for
investment advisers or Reg BI for broker-dealers). In certain cases, it
may be impossible to comply with the applicable standard of conduct
without stopping use of the covered technology before the conflict of
interest can be adequately addressed. As it develops a schedule for
eliminating, or neutralizing the effect of, the conflict, a firm should
consider the nature of the covered technology, including how it is
being used in investor interactions, and the complexity of any
elimination or neutralization measures. The firm should also consider
and seek to minimize potential risks posed to investors as a result of
the continued use of the covered technology. This might include
implementing heightened review of investor interactions to help ensure
that the harm is relatively limited and weighing the risks of continued
exposure to the conflict of interest during remediation against the
risk of making the covered technology unavailable during remediation.
If a firm has a reasonable basis to believe that pulling a covered
technology out of service due to a conflict of interest would be a
greater risk to investors than the conflict itself, a firm generally
should consider closely surveilling and monitoring the investor
interactions associated with its continued use of the technology to
evaluate whether its expectation is accurate, or whether it should
cease using the covered technology.
    The requirement for a firm to eliminate, or neutralize the effect
of, conflicts of interest that place the firm's or an associated
person's interest ahead of investors' interests covers such conflicts
the firm identifies, as well as those it reasonably should have
identified. That is, in order to comply with the proposed conflicts
rules, a firm would be required to use reasonable care to determine
whether these conflicts could arise as a result of its use of covered
technologies and how they could affect investor interactions, and to
address such conflicts rather than assuming that its covered
technologies do not result in its own (or its associated persons')
interests being placed ahead of investors' interests. The ``reasonably
should have identified'' standard is designed to require firms to
understand the covered technology they are deploying sufficiently well
to consider all the material features of the technology both when
evaluating the technology and identifying conflicts, and later when