Commission did not discuss this issue in the Notice, the Commission now
recognizes that the open/close indicator is currently not captured for
certain options transactions.
---------------------------------------------------------------------------

    \1944\ See Notice, supra note 5, at 30680.
---------------------------------------------------------------------------

(2) Accuracy
    In the Notice, the Commission carefully considered the accuracy of
data currently used by regulators in order to consider whether and to
what degree the CAT NMS Plan would provide more accurate data.\1945\ As
discussed in more detail below, the Commission considered several forms
of data inaccuracy, including data errors, inaccurate event sequencing,
the inability to link data accurately, inconsistent identifiers, and
obfuscating levels of irreversible data aggregation.
---------------------------------------------------------------------------

    \1945\ Id. at 30666-71.
---------------------------------------------------------------------------

A. Data Errors
    With respect to data errors,\1946\ the Commission stated its
preliminary belief that data errors affect most current data used by
regulators and can persist even after corrections. The Commission
specifically noted instances where information was inaccurately
reported by broker-dealers and discussed various errors in data
translated from back-office systems, errors in data from trading
systems, and errors in audit trail data.\1947\ Furthermore, the
Commission noted that the CAT NMS Plan reports that 2.42% of order
events submitted to OATS fail validation checks. Although FINRA sends
these records back to its members to correct, significant error rates
in event linking post-correction are common because OATS limits error
correction requests to records with internal inconsistencies within a
given member's submission and there is no cross-participant error
resolution process. FINRA estimates that 0.5% of

OATS routing reports directed to another FINRA member broker-dealer
cannot currently be linked.\1948\ Also, as stated in the Notice, the
CAT NMS Plan reports that, following the rollouts of three major
updates to OATS, 0.86% of Trade Reporting Facility reported trades
could not be matched to OATS execution reports, 3.12% of OATS route
reports could not be matched to exchange orders, and 2.44% of inter-
firm routes could not be matched to a record of the receiving firm's
receipt of a routed order.\1949\
---------------------------------------------------------------------------

    \1946\ As used herein, the term ``data errors'' refers to
instances where data reflect false information or are missing
information such that they do not reflect order events that occurred
in the market fully and accurately. Under this definition of ``data
errors,'' a trading error or an order entry error would not be a
``data error.'' For example, if a trader submitted an order to an
exchange with an order size of 100,000, an accurate order record
would contain an order size of 100,000. If the trader actually
intended to enter the order size as 1,000, the accurate order record
would still be 100,000 because that would reflect the actual state
of the market at the time. In other words, the 100,000 order size is
not a ``data error.'' If the trader later corrected the order size,
accurate data would reflect the subsequent corrections while still
preserving the accurate state of the market at the time.
    \1947\ Id. at 30666-67.
    \1948\ Id.
    \1949\ Id.
---------------------------------------------------------------------------

    The Commission received several comment letters that discussed the
current state of errors in data used by regulators.\1950\ One commenter
did not believe that OATS data currently achieves ``de minimis''
errors.\1951\ The commenter further stated that there are instances
where errors cannot be corrected in OATS and gave true duplicates and
non-reportable symbols as examples.\1952\ The commenter further
detailed the classification scheme currently used to categorize OATS
errors. According to the commenter, these errors are currently
classified as: Rejects; unmatched executions; unmatched exchange
routes; inter firm received unmatched; inter firm sent unmatched; out
of sequence; and late reports.\1953\
---------------------------------------------------------------------------

    \1950\ Anonymous Letter I at 9-10; Anonymous Letter II at 1-2;
FIF Letter at 55, 60.
    \1951\ FIF Letter at 60.
    \1952\ FIF Letter at 55.
    \1953\ FIF Letter at 54.
---------------------------------------------------------------------------

    Another commenter stated in two separate letters that there are
OATS reporters that are repeatedly non-compliant, both in omitting to
report required data and reporting inaccurate data to FINRA.\1954\ The
commenter contended that the extent of this non-compliance is
significant and is magnified by the lengthy period of time before the
errors are discovered and corrected by FINRA. Also, there is no way to
know the magnitude of noncompliance that is never detected and
therefore never corrected. The non-compliance by reporters may cause
the error rates reported by OATS to be higher than reported.
---------------------------------------------------------------------------

    \1954\ Anonymous Letter I at 9-10; Anonymous Letter II at 1-2.
---------------------------------------------------------------------------

    The Commission has considered the comments received. The Commission
agrees with the commenter that stated there are instances where OATS
data does not fail validation checks, but does contain errors. As
mentioned in the Notice, OATS validation checks are limited to
detecting errors that can be discovered by a concise set of logical
rules and OATS limits error correction requests to records with
internal inconsistencies within a given member's submission.\1955\ The
Commission also recognizes the comment that some OATS reporters fail to
send and/or send inaccurate reports to FINRA and is updating its
analysis to take into account that current data errors in OATS may be
larger than initially considered due to this non-compliance. Finally,
the Commission now considers the error classifications provided by a
commenter in its baseline.
---------------------------------------------------------------------------

    \1955\ See Notice, supra note 5, at 30667.
---------------------------------------------------------------------------

B. Event Sequencing
    With respect to event sequencing, as the Commission addressed in
detail in the Notice, the ability to sequence market events is crucial
to the efficacy of detecting and investigating some types of
manipulation, and the sequencing of order events requires both
sufficient clock synchronization across market participants and
timestamps that are granular enough for accurate sequencing, but the
current clock synchronization standards make this process difficult.
    In the Notice, the Commission discussed that current rules require
most broker-dealers to synchronize their system clocks to within one
second.\1956\ The Commission further noted that ``in practice'' some
broker-dealers currently synchronize their clocks to smaller clock
offset tolerances. The Commission cited the FIF Clock Offset Survey
\1957\ where 29% of respondents report they currently synchronize their
clock to permit a maximum clock offset of one second from NIST, 10% of
respondents permit a maximum offset of 50 milliseconds to one second,
21% of respondents permit a 50 millisecond maximum offset, and 18% of
respondents permit a maximum offset less than 50 milliseconds. The
remaining 22% of respondents report they utilize multiple clock offset
tolerances across their systems ranging from five microseconds to one
second. In addition, the Commission discussed that FINRA had filed a
proposed rule change that would reduce the clock offset tolerance for
members' computer clocks that are used to record events in NMS
securities from within one second of the NIST atomic clock to within 50
milliseconds of the NIST atomic clock.\1958\ Furthermore, the
Commission discussed that if the rule change was approved, more
entities would record timestamps with data at a 50 millisecond clock
offset tolerance regardless of whether the CAT NMS Plan is
approved.\1959\
---------------------------------------------------------------------------

    \1956\ Id. at 30669.
    \1957\ See FIF Clock Offset Survey, supra note 247.
    \1958\ See Notice, supra note 5, at 30668.
    \1959\ Id. at 30683.
---------------------------------------------------------------------------

    For clock synchronization on exchanges, the Commission discussed in
the Notice that exchanges trading NASDAQ securities currently adhere to
clock synchronization standards at or below 100 microseconds, and the
Commission understands that the NYSE, the options exchanges, and the
SIAC SIP have comparable clock synchronization standards. In addition,
the Commission noted that Participants stated ``that absolute clock
offset on exchanges averages 36 microseconds.'' \1960\
---------------------------------------------------------------------------

    \1960\ Id. at 30669.
---------------------------------------------------------------------------

    Also in the Notice, Commission Staff conducted an analysis of the
frequency of order events using MIDAS data which identified whether for
each order event, an event in the same security at another venue
occurred within a given time range. 97.95% of order events for listed
equities and 91% of order events for listed options occurred within one
second of another unrelated order event in the same security. 14.44% of
the unrelated order events for listed equities and 3.12% of the
unrelated order events for listed options in the same security occurred
within 5 microseconds of another order event in the same security. The
Commission noted that the analysis underestimates the true frequency of
unrelated events within the given time frames because it includes only
order events that are included in the MIDAS data, and furthermore
stated that the analysis illustrates how the current frequency of order
events makes sequencing unrelated order events difficult. With respect
to the granularity of timestamps, the Commission discussed in the
Notice that regulators need sufficiently granular timestamps to
sequence events across orders and within order lifecycles, and that the
current lack of uniform and granular timestamps can limit the ability
of regulators to sequence events accurately and link data with
information from other data sources.\1961\ In addition, the Commission
discussed that current data sources have different timestamp
granularity standards, and that many public data sources report time in
seconds or milliseconds, and some, including direct data feeds, report
time in microseconds or nanoseconds. As examples, the Commission stated
that OPRA allows for timestamps in nanoseconds and that the other SIPs
require timestamps in microseconds for equity trades and quotes,
whereas the

short sale transactional data released by exchanges contains timestamps
in seconds.\1962\ In addition, the Commission stated that OATS requires
timestamps in milliseconds for firms that capture time in milliseconds,
but does not require members to capture time in milliseconds.\1963\
---------------------------------------------------------------------------

    \1961\ Id. at 30669-70.
    \1962\ Id.
    \1963\ Id.
---------------------------------------------------------------------------

    One commenter discussed the Commission's analysis of the frequency
of order events in the context of the Commission's baseline assessment
of clock synchronization and timestamp granularity.\1964\ The commenter
pointed out that the Commission's analysis ``used primarily SIP data,
reflecting exchange only recording of events, which is a tightly
controlled, co-located and specialized environment'' and that the
analysis ``does not reflect the broader broker-dealer communities'
recording of events . . . in a distributed environment, a much less
controlled and less precise environment.'' \1965\ That commenter also
stated that ``[w]ithin every order lifecycle, the events leading up to
the execution can be [sequenced] due to daisy chaining.'' \1966\
---------------------------------------------------------------------------

    \1964\ FIF Letter at 118.
    \1965\ FIF Letter at 118.
    \1966\ Id.
---------------------------------------------------------------------------

    As noted above, commenters recognized that lower tolerances were
already mandated by some exchanges as well as ATSs that maintain an
order book.\1967\ One commenter noted that some firms receive direct
feeds from exchanges as precise as 1 microsecond.\1968\ The
Participants and another commenter explained that the marketplace is
segmented such that broker-dealers operate under a different business
model and regulatory environment than ATSs and exchanges.\1969\ While
microsecond tolerances for exchanges and ATSs are already standard
practice, broker-dealers have no standard practice across the industry
and are precluded from using matching engines, which are capable of the
lowest level of granularity.\1970\
---------------------------------------------------------------------------

    \1967\ TR Letter at 7; FIF Letter at 97-99.
    \1968\ Better Markets Letter at 8.
    \1969\ Response Letter II at 4; FIF Letter 97-99, 116.
    \1970\ Id.
---------------------------------------------------------------------------

    One commenter noted the imprecise business process of handling
manual orders.\1971\ Another commenter noted that manual intervention
can take over a second because it involves several steps, which impact
timestamp capture.\1972\
---------------------------------------------------------------------------

    \1971\ FIF Letter at 118.
    \1972\ SIFMA Letter at 35. Specifically, this commenter
explained that manual order taking involves taking an order via
phone, fax, or email and then manually entering the order into an
electronic order management system.
---------------------------------------------------------------------------

    The Participants' response provided new information on the current
clock synchronization standards of Participants.\1973\ Specifically,
the response clarified that all Participants currently operate pursuant
to a clock synchronization standard of 100 microseconds with regard to
their electronic systems.\1974\
---------------------------------------------------------------------------

    \1973\ Response Letter II at 4-5.
    \1974\ Id.
---------------------------------------------------------------------------

    The Commission has considered these comments and, as discussed
below, has updated its analysis of the baseline of clock
synchronization as set out in the Notice.
    In the Notice, the Commission explained that its analysis of the
frequency of order events used MIDAS data, recognized the limitations
that its use of MIDAS data could impose, and explained how the
limitations reflected the Commission's assessment of the
baseline.\1975\ The Commission therefore agrees with the commenter that
its analysis reflects a disproportionate number of exchange events
relative to off-exchange events. But because the commenter did not
explain how the limitations of the Commission's analysis could make the
analysis less useful or what statistical biases could result from these
limitations, the Commission believes that, despite its limitations, the
analysis ``still provides useful insights'' and ``illustrates how the
current frequency of order events makes sequencing unrelated order
events difficult.'' \1976\
---------------------------------------------------------------------------

    \1975\ See Notice, supra note 5, at 30669.
    \1976\ Id.
---------------------------------------------------------------------------

    The Commission generally agrees that events can be sequenced due to
daisy chaining, but notes that for most regulatory activities,\1977\ it
is crucial for the regulators to be able to accurately sequence events
from different orders. Furthermore, the Commission believes that such
sequencing requires both sufficient clock synchronization across market
participants and sufficiently granular timestamps.
---------------------------------------------------------------------------

    \1977\ Id. at 30667.
---------------------------------------------------------------------------

    With respect to comments regarding manual orders, the Commission
believes the new insights provided by commenters are consistent with
the baseline in the Notice.
    The Commission is updating its economic baseline to include the new
information provided by the Participants and also to include the
approval of a FINRA rule amendment. Specifically, the Commission now
believes that all Participants currently operate pursuant to a clock
synchronization standard of 100 microseconds. Also, the Commission
approved the proposed rule change by FINRA that was discussed in the
Notice that reduces the synchronization tolerance for computer clocks
to 50 milliseconds for member firms that record events in NMS
Securities.\1978\ Accordingly, FINRA members that record events in NMS
Securities currently operate, or in the near future will operate,
pursuant to a clock synchronization standard of 50 milliseconds for
their computer clocks.
---------------------------------------------------------------------------

    \1978\ See Securities Exchange Act Release No. 77565 (April 8,
2016), 81 FR 22136 (April 14, 2016).
---------------------------------------------------------------------------

C. Data Linking and Combining
    Regarding data linking, as the Commission addressed in detail in
the Notice, regulators analyzing an event or running a surveillance
pattern often need to link data.\1979\ As examples, the Commission
stated that cross-market examinations require the cumbersome and time-
consuming task of linking many different data sources; that regulators
that are determining whether rule violations have occurred will combine
trading data from sources such as public feeds, SRO audit trails, EBS
data, and trade blotters; and that the analysis and reconstruction of
market events could require linking many different data sources, such
as a dozen SRO audit trails.\1980\
---------------------------------------------------------------------------

    \1979\ Id. at 30670.
    \1980\ Id.
---------------------------------------------------------------------------

    The Commission discussed that merging different data sources often
involves translating the data sources into the same format, which can
be a complex process that is prone to error.\1981\ In addition, the
Commission discussed that linking records within or across data sources
requires the sources to share ``key fields'' that facilitate linkage,
but that regulators may be unable to link some data source combinations
accurately because the data sources do not have key fields in common or
the key fields are not sufficiently granular; also, different data
sources may have key fields in common but the relationship between the
fields is not straightforward so the algorithm to link them may be
necessarily complex and not entirely successful.\1982\ Furthermore, the
Commission discussed that within a single order lifecycle, the order
number may change when a broker-dealer routes the order to another
broker-dealer or exchange or even to another desk at the same broker-
dealer. Finally, the Commission discussed that the inability to link
all records affects

the accuracy of the resulting data and can force an inefficient manual
linkage process that would delay the completion of the data collection
and analysis portion of an examination, investigation, or
reconstruction.\1983\
---------------------------------------------------------------------------

    \1981\ Id.
    \1982\ Id.
    \1983\ Id.
---------------------------------------------------------------------------

D. Customer and Broker-Dealer Identification
    With respect to market participant identifiers (``MPIDs''), the
Commission explained that trade and order data currently available to
the Commission lack consistent customer and broker-dealer identifiers,
which limit regulators' ability to track the activity of one client or
broker-dealer across the market.\1984\ In the case of broker-dealers,
the Commission stated that identifiers are inconsistent and that no
centralized database exists. In addition, although SROs generally
identify their members using MPIDs, those MPIDS are not standardized
across venues.\1985\ The Commission further stated that aggregating a
broker-dealer's activity across venues requires verifying the MPIDs
assigned to a broker-dealer on each venue, usually referencing the
broker-dealer by its Central Registration Depository (``CRD'') number.
Finally, the Commission stated that in the course of manual data
analysis, Commission Staff have experienced challenges in identifying
broker-dealers using CRD numbers, but that the Commission and the SROs
have generally overcome these challenges in the context of automated
regulatory data analysis.
---------------------------------------------------------------------------

    \1984\ Id. at 30670-71.
    \1985\ Id.
---------------------------------------------------------------------------

    In the case of broker-dealer customers, the Commission stated that
identifying customer account owners across multiple broker-dealers is
difficult and prone to error.\1986\ As an example, the Commission
discussed that although the EBS system provides the names associated
with each account traded, these names are drawn from separate records
of each broker-dealer providing data to the EBS system, and the same
party may be identified by a different name across multiple broker-
dealers.
---------------------------------------------------------------------------

    \1986\ Id.
---------------------------------------------------------------------------

    One commenter discussed the difficulty in tracking market
participant activity using MPIDs, stating that ``[w]ith regard to trade
identifiers used by market access providers, some clearing firms have
used one or more MPIDs to conceal the identity of other participants/
clients using these services to manipulate markets.'' \1987\ The
Commission agrees that tracking market participant activity using MPIDs
can be difficult because of sponsored or direct market access
arrangements whereby broker-dealers allow customers to trade
electronically using the broker-dealer's MPID. In cases where the
sponsored or direct market access customer is not a FINRA member, the
EBS system allows regulators to observe the identity of trading parties
that may be concealed by MPIDs, but, as discussed in the Notice, it is
difficult to consistently identify trading parties across multiple
broker-dealers because they may use different names across these
broker-dealers. In addition, as discussed in the Notice, EBS data is
cumbersome to use for broad analysis because of fragmentation of the
data.\1988\ However, in cases where the sponsored or direct market
access customer is a FINRA member, OATS reporting obligations require
both the customer broker-dealer and the sponsoring broker-dealer to
generate reports that, when linked correctly, allow regulators to
observe the identity of the trading party.\1989\
---------------------------------------------------------------------------

    \1987\ Anonymous Letter I at 12.
    \1988\ See Notice, supra note 5, at 30661.
    \1989\ See OATS Compliance FAQ at C84 available at
www.finra.org/industry/faq-oats-compliance-faq.
---------------------------------------------------------------------------

E. Aggregation
    Regarding data aggregation, as addressed in detail in the Notice,
the practice used in some data records of bundling together data from
different orders and trades can make it difficult to distinguish the
different orders and trades in a given bundle. That aggregation reduces
the usefulness of equity and options cleared reports, because the
reports do not have detailed trade information and do not include
activity that does not require clearing.\1990\ In the Notice, the
Commission presented as an example the frequent use of average-price
accounts by brokers to execute and aggregate multiple trades for one or
more customers. The Commission discussed that for these cases, and with
EBS data, the system does not reflect the details of each individual
trade execution.\1991\ Furthermore, the Commission discussed that
information on trade allocations aggregate the trade information to
such an extent that it is difficult for regulators to identify when
particular clients may be afforded preferential treatment because it is
challenging to link subaccount allocations to orders and trades.\1992\
---------------------------------------------------------------------------

    \1990\ See Notice, supra note 5, at 30688-89.
    \1991\ Id. at 30671.
    \1992\ Id.
---------------------------------------------------------------------------

    In addition, as the Commission discussed in the Notice, issuer
repurchase information is aggregated at the monthly and quarterly
level, and this level of aggregation limits the use of such data in
investigations of the timing of issuer repurchases and issuer stock
price manipulation and in analysis of the use of the Rule 10b-18 issuer
repurchase safe harbor.
(3) Accessibility
    As addressed in detail in the Notice, the SROs and the Commission
also lack direct access--i.e., the ability to log into a system in a
manner that would allow them to gather and analyze the data they need--
to many of the data sources described above. SROs generally have direct
access only to their own audit trails and the public data feeds.\1993\
The Commission has direct access only to the public data feeds and the
equity and option cleared data; it lacks direct access to information
provided in EBS or contained in trade blotters, order tickets, order
handling data, SRO audit trails, and OATS data.\1994\
---------------------------------------------------------------------------

    \1993\ FINRA does receive data from certain SROs on a daily
basis and subsequently has direct access to that data. Id. at 30671
n.453.
    \1994\ Id. at 30671-72.
---------------------------------------------------------------------------

    The Commission explained that if a regulator does not have direct
access to data it needs, the regulator would request it, and that this
can result in many burdensome requests to broker-dealers, SROs, and
others. The Commission recognized that data requests could impose
burdens on the entities responding to the requests, in addition to the
burden on the regulators making the requests. In particular, broker-
dealers, investment advisers, and SROs responding to a data request
must incur costs in order to produce, store, and transmit the data for
the Commission or SRO.\1995\
---------------------------------------------------------------------------

    \1995\ Id. at 30672.
---------------------------------------------------------------------------

    The Commission explained that, to complete just one analysis,
regulators may need to request data from many different data providers
because of fragmentation in the data. The Commission discussed the fact
that fragmentation in trade and order data can take many forms. First,
an analysis may require the same type of data from many market
participants. For example, while ATSs and dealers report order events
in equities to OATS, each of the 12 equities exchanges has its own
audit trail. As a result, a market reconstruction for a single security
may involve data requests to multiple exchanges as well as to
FINRA.\1996\
---------------------------------------------------------------------------

    \1996\ Id.
---------------------------------------------------------------------------

    Second, the required data fields for an analysis may be reflected
in different types of data. For example, for investigations that
require tracing a single trade or a set of trades back to an

investor or investors, regulators would first need to request data from
the exchanges or market participants executing trades to find out which
members, subscribers, or broker-dealers sent the orders that led to the
executions. Then, regulators would need to ask the members,
subscribers, and broker-dealers for information on the orders and
repeat that process until they get to the broker-dealer who initiated
the order to see the customer behind the order.\1997\
---------------------------------------------------------------------------

    \1997\ Id.
---------------------------------------------------------------------------

    Third, an analysis may require data on different products covered
in separate data sources. For example, some regulatory activities
require data on both equities and options. And because current data
sources do not contain information regarding both equities and options,
regulators needing data on both types of securities would need to make
several data requests.\1998\
---------------------------------------------------------------------------

    \1998\ Id.
---------------------------------------------------------------------------

    As the Commission discussed in the Notice, data fragmentation also
results in disparate requirements for industry members to record and
report the same information in multiple formats. Because each SRO has
its own data requirements, a market participant that is a member of
multiple SROs may be required to report audit trail data in numerous
formats and interact with multiple regulators in response to normal
data queries.\1999\
---------------------------------------------------------------------------

    \1999\ Id.
---------------------------------------------------------------------------

(4) Timeliness
    As addressed in detail in the Notice, currently, obtaining trade
and order data and converting the data into a form in which they can be
analyzed can involve a significant delay from the time of a particular
event of interest. In some cases the length of time from when an event
occurs until regulators can use relevant data in an investigation or
analysis can be weeks or months. This is especially true for trading
data that includes customer information.\2000\
---------------------------------------------------------------------------

    \2000\ Id. at 30673. The Commission noted that some of the data
sources described above can be accessed by SROs and the Commission
without significant delay. For example, SROs and the Commission have
some real-time direct access to public data and, through MIDAS, the
Commission has next-day direct access to analytics that are based on
public data, such as volumes over various time horizons.
Furthermore, the Commission noted that FINRA receives audit trail
data from exchanges pursuant to Regulatory Services Agreements at
the end of each trading day. Id.
---------------------------------------------------------------------------

    The Commission explained in the Notice that corrected FINRA OATS
data may be available less than two weeks after an event and
uncorrected data on T+1. In particular, FINRA members submit OATS data
on a daily basis, submitting end-of-day files by 8:00 a.m. ET the
following day or they are marked late by FINRA. FINRA acknowledges
receipt of the data an hour after the member submits it, before running
its validation process. FINRA then takes approximately four hours after
acknowledging receipt of OATS data to determine if the data contain any
syntax errors. In addition to the four hours needed to identify errors
within a report, it takes another 24 hours for context checking, which
identifies duplicates or secondary events without an originating event.
Once a context rejection is available, the member has up to five
business days to repair the rejection. Reports for files that contain
internally inconsistent information about processing, linking, and
routing orders may be available within two business days. FINRA
attempts to match the inconsistent information against any additional
data received up to T+2 for linking errors and T+3 for routing errors.
The timing for surveillance programs varies depending on the type of
surveillance being performed; data is assumed to be completely
processed and corrected at T+8.\2001\
---------------------------------------------------------------------------

    \2001\ Id.
---------------------------------------------------------------------------

    The Commission also explained that because market participants
generally do not report or compile datasets immediately after an order
event, there is a delay before regulators may access some data sources.
For example, the compilation of equity and option cleared reports
occurs on T+1 for options and T+3 for equities (i.e., the clearing day)
and the electronic query access for equities is available from the
Securities Information Automation Corporation (``SIAC'') on T+3.
Additionally, when broker-dealers receive a request for EBS, the firm
must first fill in the EBS report and then, if it does not self-clear,
pass the reports on to its clearing firm to compile and send to SIAC.
The EBS submission process can take up to ten business days. More
immediate requests for cleared options data can be submitted to FINRA,
but even this process takes up to two days. Because EBS data do not
contain order entry time and order execution time, regulators must
obtain this information from firms and brokers using either data
requests or subpoenas, and this process generally can take from two to
four weeks depending on the size of the request.\2002\
---------------------------------------------------------------------------

    \2002\ Id. The Commission also noted that it has been the
Commission's experience that trade blotter data requests can take
weeks or in excess of a month depending on the scope of the request
and how accustomed the broker-dealer is with fulfilling such
requests. Id.
---------------------------------------------------------------------------

    In addition, the Commission noted that the lack of direct access to
most data sources may further delay the ability of regulators to use
data in certain cases. When regulators have direct access to a data
source, the time needed to receive data is only the time it takes for a
query to run. On the other hand, when regulators lack direct access,
their data requests can consume significant time, including both the
time required to put the request together and response times from the
SROs, broker-dealers, and others producing the data. For example,
obtaining complete responses from each broker-dealer for an EBS request
can take days or weeks depending on the scope of the request. Likewise,
responses from the Intermarket Surveillance Group (``ISG'') for SRO
audit trail data can take days or weeks.\2003\ As the Commission
discussed in the Notice, once regulators receive the requested data,
the data often have to be processed into a form in which they can be
analyzed. The Commission explained that it can take considerable time
for regulators to combine data from different sources and link records
from within or across data sources. Furthermore, the lack of
consistency in format adds complexity to projects involving data from
multiple data sources, even when the project does not involve linking
of these different data.\2004\
---------------------------------------------------------------------------

    \2003\ Id.
    \2004\ Id.
---------------------------------------------------------------------------

    The Commission further discussed that those who use regulatory data
also typically take time to ensure the accuracy of the data. The
Commission explained that when regulators question the accuracy of
data, they often check several alternative sources until they are
comfortable that their data are accurate. This checking of data
accuracy and augmentation process adds time to an investigation or
analysis.\2005\
---------------------------------------------------------------------------

    \2005\ Id. at 30674.
---------------------------------------------------------------------------

E. Benefits

    In the Notice, the Commission discussed its belief that the
economic benefits of the CAT NMS Plan would come from any expanded or
more efficient regulatory activities facilitated by improvements to the
data regulators use.\2006\ This is because the Plan will create a new
consolidated data source--CAT Data--that should replace the use of some
current data sources for many regulatory activities. Therefore, the
Benefits Section described how CAT Data compares to data regulators
currently use for regulatory activities, how the CAT Data would improve

regulatory activities, and how these improvements would benefit
investors, market participants, and markets in general.\2007\
---------------------------------------------------------------------------

    \2006\ Id.
    \2007\ Id. at 30674-30708.
---------------------------------------------------------------------------

    In the Notice, the Commission discussed its preliminary belief that
the CAT NMS Plan would produce data that would improve on current data
sources because CAT Data would result in regulators having direct
access to consolidated audit trail data, which would in turn improve
many of the regulatory activities discussed in the Baseline
Section.\2008\ As summarized in Table 2, the Commission preliminarily
concluded that the Plan would generate improvements in the quality of
data that regulators would have access to in the areas of completeness,
accuracy, accessibility, and timeliness. The Commission discussed its
preliminary belief that the improvements in the quality of regulatory
data within these categories would significantly improve the ability of
regulators to perform a wide range of regulatory activities, which
would lead to benefits for investors and markets. In addition, the
Commission preliminarily believed that certain provisions in the Plan--
those related to future upgrades of the Central Repository, the
promotion of the accuracy of CAT Data, the promotion of the timeliness
of CAT Data, and the inclusion of specific governance provisions
identified by the Commission in the Adopting Release for Rule 613--
would increase the likelihood that the potential benefits of the CAT
NMS Plan would be realized.
---------------------------------------------------------------------------

    \2008\ Id. at 30674-77.
---------------------------------------------------------------------------

    In the category of completeness, the Commission discussed its
belief that the ability for regulators to access more material data
elements from a consolidated source would enable regulators to more
efficiently carry out investigations, examinations, and analyses
because regulators could acquire data from a single source that they
would otherwise need to compile from many data sources. In the category
of accuracy, the Commission discussed its belief that the Plan would
substantially improve data accuracy by requiring CAT Data to be
collected, compiled, and stored in a uniform, linked format using
consistent identifiers for customers and market participants. In the
category of accessibility, the Commission discussed its belief that the
Plan would substantially improve the access to data for regulators
because the Plan requires regulators to have direct access to CAT Data
and this direct access would dramatically reduce the hundreds of
thousands of requests that regulators must make each year in order to
obtain data, thus reducing the burden on the industry. Finally, in the
category of timeliness, the Commission discussed its belief that the
Plan, if approved as noticed, would significantly improve the
timeliness of data acquisition and use, which could improve the
timeliness of regulatory actions that use data.
    The Commission discussed its expectation that regulatory activities
such as surveillance, investigations, examinations, analysis and
reconstruction of market events, and analysis in support of rulemaking
initiatives would benefit from improved data quality as part of
CAT.\2009\ The Commission explained that data is essential to all of
these regulatory activities, and therefore substantial improvements in
the quality of the regulatory data should result in substantial
improvements in the efficiency and effectiveness of these regulatory
activities, which should translate into benefits to investors and
markets. For example, improved data could lead to more effective and
efficient surveillance that better protects investors and markets from
violative behavior and facilitates more efficient and effective risk-
based investigations and examinations that more effectively protect
investors. The Commission stated that together, these improved
activities could better deter violative behavior of market
participants, which could improve market efficiency. Furthermore, this
increase in directly accessible data should improve regulators'
understanding of the markets, leading to more informed public policy
decisions that better address market deficiencies to the benefit of
investors and markets. The Commission also discussed the fact that the
Plan lacked information regarding the details of certain elements of
the Plan likely to affect the benefits of the Plan, primarily because
many of those details had not yet been determined, which creates some
uncertainty about the expected economic effects.\2010\
---------------------------------------------------------------------------

    \2009\ Id. at 30675-76.
    \2010\ Id. at 30676.
---------------------------------------------------------------------------

    The Commission has considered the comments it received regarding
the likely benefits of the CAT NMS Plan and continues to believe that
the CAT NMS Plan would generate improvements in the quality of data
that regulators would have access to in the areas of completeness,
accuracy, accessibility, and timeliness. The Commission also continues
to believe that improvements in the quality of regulatory data within
these categories would significantly improve the ability of regulators
to perform a wide range of regulatory activities, which would lead to
benefits for investors and markets. In addition, the Commission
continues to believe that certain provisions in the Plan--those related
to future upgrades of the Central Repository, the promotion of the
accuracy of CAT Data, the promotion of the timeliness of CAT Data, and
the inclusion of specific governance provisions identified by the
Commission in the Adopting Release for Rule 613--increase the
likelihood that the potential benefits of the CAT NMS Plan described
below will be realized. As set out in more detail below, the Commission
has taken into account the modifications that have been made to the
Plan where they are relevant to the Commission's analysis of the
benefits of the Plan, and has updated its analysis accordingly.

                                                                                Table 2
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                Routing/
                                  Customer      Broker-dealer      Timestamp        Allocation     Order  display    Buy-to-cover       Special       modification/        Entire       Direct  access   Off- exchange    Timeliness of
                                 identifier       identifier         \2011\        information      information       indicator         handling       cancellation      lifecycle     for  regulators  activity \2012\  data  compiling
                                                                                instructions     information                                                            \2013\
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
OATS........................  No.............  Yes............  Yes (majority    No.............  Yes (for limit   No.............  Yes              Yes............  Yes (before      No (except       Yes............  Raw Data: T+1
                                                                 in                                orders).                          (conditional).                    order reaches    FINRA). Access                    Corrected
                                                                 milliseconds                                                                                exchange).       can take                          Data: T+6.
                                                                 but some in                                                                                No (once order    several weeks.
                                                                 seconds).                                                                                reaches
                                                                                exchange).
COATS.......................  No.............  Yes............  Yes............  No.............  No.............  No.............  No.............  Yes............  No.............  No (except SROs  No.............  Reported same-
                                                                                w/r/t their                       day, but
                                                                                own members).                     separate file
                                                                                transmitted at
                                                                                latest T+1.
SRO Audit Trails............  No.............  Yes............  Yes (majority    No.............  No.............  No.............  No.............  Yes............  No (only once    No (except SROs  No.............  As soon as a
                                                                 in                                                                                order reaches    w/r/t their                       trade is
                                                                 milliseconds                                                                                exchange).       own trails).                      executed.
                                                                 but some in                                                                                Access can
                                                                 seconds).                                                                                take several
                                                                                weeks.
Equity and Option Cleared     No.............  No.............  No.............  No.............  No.............  No.............  No.............  No.............  No.............  Yes............  Yes............  Equity: T+3
 Reports.                                                                                Option: T+1.
Electronic Blue Sheets......  Yes (but not     Yes (but not     Yes............  No.............  No.............  No.............  No.............  No (except for   No.............  No. Access can   Yes............  10 business
                               always           always                                                                                certain                           take several                      days after
                               consistent       consistent                                                                                cancellation                      weeks or                          request is
                               across broker-   across broker-                                                                                information).                     months.                           submitted.
                               dealers)         dealers).
                               \2014\.
Trade Blotters/Order Tickets  Yes (but not     Yes (but not     Yes (can be      No.............  No.............  No.............  No.............  No.............  No.............  No. Access can   Yes............  Same-day.
                               always           always           requested,                                                                                take several
                               consistent       consistent       although not                                                                                days.
                               across broker-   across broker-   always
                               dealers).        dealers).        reliable).
Trading and Order Handling    Depends on the   Yes............  Yes............  No.............  No.............  No.............  No.............  Yes............  Yes (except      No. Regulators   Yes............  Same-day.
 System Data.                  trader.                                                                                allocations).    must request
                                                                                this data (SEC
                                                                                asks for the
                                                                                data within 10
                                                                                days).
Public/Proprietary Data.....  No.............  No.............  Yes (varied      No.............  No.............  No.............  No.............  Yes (except non- No.............  Yes............  Yes............  Same-day.
                                                                 between                                                                              displayed
                                                                 seconds and                                                                          orders).
                                                                 microseconds).
Data from Proposed CAT......  Yes              Yes              Yes              Yes              Yes              Yes              Yes              Yes              Yes (613(j)(9))  Yes (SEC and     Yes (613(c)(2)   Raw Data: T+1
                               (613(c)(7)(i)(   (613(c)(7)(i)(   (milliseconds)   (613(c)(7)(vi)   (613(c)(7)(i)(   (613(c)(7)(i)(   (613(c)(7)(i)(   (613(c)(7)(ii).                   SROs)            and (3)).        Corrected
                               A)).             C)).             (613(d)).        ).               F)).             F)).             F)).                                               (613(e)(2)).                      Data: T+3.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

1. Improvements in Data Qualities
---------------------------------------------------------------------------

    \2011\ As proposed, the CAT NMS Plan also requires CAT Reporters
to synchronize their time clocks to the time maintained by the NIST
with an allowable drift of 50 milliseconds. See CAT NMS Plan, supra
note 5, at Section 6.8. According to a survey conducted by the
Financial Information Forum (FIF), 39% of responding broker-dealers
currently synchronize their clocks with less precision than what is
called for by the CAT NMS Plan. Thus, the CAT NMS Plan would also
increase the accuracy of the timestamps used by certain broker-
dealers. See FIF Clock Offset Survey, supra note 247.
    \2012\ Off-exchange activity includes currently reportable
events that are not handled by a registered securities exchange.
    \2013\ In this instance, ``timeliness'' refers to when the data
are compiled at the source in question (e.g., when OATS receives
data from reporting broker-dealers), not when they become available
to regulators because that timeline can vary depending on the
regulator in question. As shown in the ``Direct Access for
Regulators'' column, it may still take several days, weeks, or
months for regulators to be able to access the data. For example,
while OATS reporters provide the data at T+1, the SEC must request
OATS data in order to access it, which may take several days or
weeks. This narrower definition of timeliness is not used throughout
this economic analysis.
    \2014\ Guidance from FINRA indicates that broker-dealers must
``identify the party to the trade'' through EBS fields such as
``Primary Party Identifier,'' but that party may be another broker-
dealer rather than the ultimate customer. See FINRA, Electronic Blue
Sheet Submissions, FINRA and ISG Extend Effective Date for Certain
Electronic Blue Sheet Data Elements, Regulatory Notice 12-47 (Oct.
2012), available at https://www.finra.org/sites/default/files/NoticeDocument/p194655.pdf. Similarly, under the large trader rule,
persons exercising ``investment discretion'' are reported through
EBS, but in some cases such persons are investment advisers rather
than their customers. See supra note 1912 and accompanying text
(discussing the large trader rule).
---------------------------------------------------------------------------

    Consistent with the Adopting Release, the Commission identified in
the Notice four qualities of trade and order data that impact the
effectiveness of core SRO and Commission regulatory efforts: Accuracy,
completeness, accessibility, and timeliness.\2015\ In assessing the
potential benefits of the CAT NMS Plan, the Commission's economic
analysis compared the data that would be available under the Plan to
the trading and order data currently available to regulators.\2016\ The
Commission preliminarily believed that the Plan would improve data in
terms of all four qualities, but that uncertainty remained as to the
expected degree of improvement in some areas.\2017\ The Commission has
considered the comments received, the Participants' response, and the
modifications to the Plan, and continues to believe that the Plan will
improve accuracy, completeness, accessibility, and timeliness of trade
and order data relative to the Baseline, with some uncertainty as to
the degree of improvement.
---------------------------------------------------------------------------

    \2015\ See Adopting Release, supra note 14, at 45727.
    \2016\ Changes in all four data qualities affect certain data-
driven regulatory activities. The benefits of the Plan derive from
the changes to these regulatory activities.
    \2017\ See Notice, supra note 5, at 30678.
---------------------------------------------------------------------------

a. Completeness
    In the Notice, the Commission discussed how the CAT NMS Plan, if
approved, would result in regulators having direct access to a single
data source that would be more complete than any current data
source.\2018\ The Commission discussed its belief that the CAT Data
\2019\ would be more complete than other data sources because, compared
to existing SRO audit trails and other data sources, the CAT Data would
contain data from a greater number of broker-dealers on more event
types, products, and data fields. While some current data sources
contain many of the elements that would be included in CAT Data, the
Commission explained that CAT Data would consolidate that data into one
source that would be much more complete than any existing source, and
that CAT Data would also include some elements that are not available
from any current data source. In the Commission's view, having this
data consolidated in a single source would provide numerous benefits.
---------------------------------------------------------------------------

    \2018\ Id. at 30678-81.
    \2019\ Id. at 30678.
---------------------------------------------------------------------------

(1) Events and Products
    In the Notice, the Commission discussed the fact that the CAT Data
would include events and products from all current SRO audit trails,
combined into a single data source. In addition, it would include some
off-exchange activity not captured on current SRO audit trails,\2020\
as well as proprietary orders originated by a trading desk in the
ordinary course of a member's market making activities (or ``principal
activity''),\2021\ and information on equities, options and OTC Equity
Securities.\2022\
---------------------------------------------------------------------------

    \2020\ The Commission noted that SRO audit trails currently do
not include the activity of firms that are not members of that SRO.
And, currently only FINRA requires its members to report their off-
exchange activity. While broker-dealers that trade off-exchange must
be members of FINRA unless their activity fits the terms of the
exemption in Rule 15b9-1, firms that qualify for the exemption in
that rule and that are not FINRA members do not report their off-
exchange activity to OATS. This exemption covers a large percentage
of off-exchange activity. Broker-dealers that are not FINRA Members
accounted for 48% of orders sent directly to ATSs in 2014, 40% in
2013, and 32% in 2012. Because all SROs are Participants in the
Plan, under the Plan all broker-dealers with Reportable Events,
including off-exchange, would be required to report the required CAT
Data to the Central Repository. Id. at 30678-79.
    \2021\ Id. at Section IV.D.2.b(1)A.
    \2022\ ``OTC Equity Security'' is defined in the Plan as ``any
equity security, other than an NMS Security, subject to prompt last
sale reporting rules of a registered national securities association
and reported to one of such association's equity trade reporting
facilities.'' Id. at 30679.
---------------------------------------------------------------------------

    Four commenters believed that the CAT NMS Plan would result in a
data source that is not complete enough and argued that CAT should be
significantly expanded in scope to include additional event types, such
as additional short selling information, clearing information, and ETF
creation and redemption data; additional product types, such as stock
index futures and options on index futures; or other types of
regulatory submissions or metrics reports, such as CCAR/DFAST, TLAC,
Volcker, Basel III, or BCBS-283.\2023\
---------------------------------------------------------------------------

    \2023\ Anonymous Letter at 6-9, 12-14, 17; Better Markets Letter
at 7; Data Boiler Letter at 1,10-13, 17-18, 31; CBOE Letter at 1-2.
---------------------------------------------------------------------------

    The Commission recognizes that at least some of these expansions
could potentially make CAT Data more complete and responds to each of
the suggestions above in Section IV.D.4.f. At the same time, the
Commission continues to believe that the CAT NMS Plan will result in
regulators having direct access to a single data source that will be
more complete than any current data source. Furthermore, the Commission
continues to believe that the CAT Data will be more complete than other
data sources because it will contain data from a greater number of
broker-dealers on more event types and products when compared to
existing SRO audit trails and other data sources.
(2) Data Fields
    In the Notice, the Commission also explained that the Plan would
consolidate, in a single source, fields that currently may not be
available from all data sources, including some fields that are
difficult for regulators to compile.\2024\ It discussed its belief
that, in particular, the inclusion of consistent, unique customer
information in the CAT Data represents a significant improvement over
current SRO audit trails in terms of completeness because very few
current data sources contain customer information, and those that do
are limited in terms of the completeness and accuracy of this
information, which significantly limits regulatory efficiency.\2025\ As
proposed in the

Notice, CAT Data would also include other data fields not available
from current SRO audit trails, including allocation information such as
allocation time, open/close information, Quote Sent Time,\2026\ and
information on whether a Customer gave a modification or cancellation
instruction. With respect to the rest of the data fields included in
CAT Data, the Commission discussed the fact that certain of them are
included in some or all current SRO audit trails but that no single
current source contains all of them. For example, the inclusion of
order display information (i.e., whether the size of the order is
displayed or non-displayed) and special handling instructions in CAT
Data improve completeness because they are not always mandatory in SRO
audit trail data and therefore may not be consistently available
without data requests to broker-dealers.
---------------------------------------------------------------------------

    \2024\ See Notice, supra note 5, at 30679-81.
    \2025\ Id. at Section IV.D.2.a(1) and Section IV.D.2.b.(1)b,
supra. As discussed in the Notice, SRO audit trails typically do not
provide customer information, but a recent FINRA rule change would
require its members to report to OATS non-FINRA member customers who
are broker-dealers.
    \2026\ ``Quote Sent Time'' refers to the time that an Options
Market Maker routes its quote, or any modification or cancellation
thereof, to an exchange. Id. at 30755.
---------------------------------------------------------------------------

    The Commission discussed its belief that, while the costs and
benefits of including particular fields can change due to technological
advances and/or changes in the nature of markets, the Plan contains
provisions regarding periodic reviews and upgrades to CAT that could
lead to proposing additional data fields that are deemed
important.\2027\ In addition, the Commission noted that it had reviewed
gap analyses that examine whether the CAT Data would contain all
important data elements in current data sources, and that the
Commission identified some potential data gaps.\2028\ However, the
Commission discussed the fact that the Plan provides that prior to the
retirement of existing systems, CAT Data must contain data elements
sufficient to ensure the same regulatory coverage provided by existing
systems that are anticipated to be retired.\2029\ The Commission
discussed its expectation that, therefore, any missing elements that
are material to regulators would be incorporated into the CAT Data
prior to the retirement of the systems that currently provide those
data elements to regulators.
---------------------------------------------------------------------------

    \2027\ See Notice, supra note 5, at Section IV.E.3.a for a
discussion of adding new data fields and other requirements for
upgrading the CAT Data after approval.
    \2028\ In the Notice, the Commission acknowledged that the
Participants are continuing to study gaps between current regulatory
data sources and the Plan as filed. See Notice, supra note 5, at
30680-81; see also SEC Rule 613--Consolidated Audit Trail (CAT)
OATS-CAT Gap Analysis and SEC Rule 613--Consolidated Audit Trail
(CAT) Revised EBS-CAT GAP Analysis, available at http://www.catnmsplan.com/gapanalyses/index.html.
    \2029\ See Notice, supra note 5, at 30680-81.
---------------------------------------------------------------------------

    Three commenters questioned the benefits of timestamps in the
Allocation Report.\2030\ One of the commenters stated that a
requirement to report allocation time would be ``divorced from the
goals of CAT.'' \2031\ Similarly, another commenter noted that
allocation time would not provide the regulatory completeness benefit
that the Commission is seeking because one likely definition would not
capture what regulators would want.\2032\ This commenter further argued
that if the main regulatory purpose of including allocation timestamps
is to detect cherry-picking, there could be alternate approaches that
achieve the same result using existing data fields.\2033\
---------------------------------------------------------------------------

    \2030\ FSR Letter at 9; SIFMA Letter at 35; FIF Letter at 3-4,
11, 86-89.
    \2031\ SIFMA Letter at 35.
    \2032\ FIF Letter at 11.
    \2033\ FIF Letter at 89.
---------------------------------------------------------------------------

    Three commenters suggested that the open/close indicator for
equities would be a new data field.\2034\ However, these comments did
not address the benefits of the open/close indicator that the
Commission discussed in the Notice.
---------------------------------------------------------------------------

    \2034\ FIF Letter at 84; TR Letter at 9; SIFMA Letter at 25.
---------------------------------------------------------------------------

    One commenter discussed possible data gaps between CAT and current
data sources.\2035\ The commenter indicated that the OATS-CAT Gap
Analysis, published in May 2015, is out of date because it does not
reflect changes that have been incorporated into OATS since 2015
including additional fields to accommodate the Tick Size Pilot and ATS
Order Book Reporting. The commenter also argued that gaps between OATS
and CAT may widen further if changes to OATS continue to be made
without corresponding changes to the CAT Plan for the initial phase.
Furthermore, the commenter noted that other regulatory systems may
indirectly impact CAT reporting requirements; for example, recent NYSE
changes to the Account Type Indicator will require EBS changes, which
in turn impacts CAT.\2036\
---------------------------------------------------------------------------

    \2035\ FIF Letter at 28-29.
    \2036\ FIF Letter at 29.
---------------------------------------------------------------------------

    In their response, the Participants agreed with the Commission's
analysis in the Notice and expressed their belief that there are
benefits associated with including time-stamps in the Allocation
Report, including the detection of allocation fraud.\2037\ With respect
to the open/close indicator, the Participants noted that this data
field is not captured pursuant to current industry practices for
equities or some options transactions.\2038\ The Participants also
responded to the comment regarding the OATS-CAT Gap Analysis, stating
that the gap analysis has been updated by including newly-added data
fields in these duplicative systems, such as the new OATS data fields
related to the Tick Size Pilot and ATS Order Book Reporting
changes.\2039\
---------------------------------------------------------------------------

    \2037\ Response Letter I at 37-38.
    \2038\ Response Letter I at 21-22.
    \2039\ Response Letter II at 21.
---------------------------------------------------------------------------

    The Commission has considered the comments it received and the
Participants' response regarding the potential benefits of the CAT NMS
Plan in terms of data completeness. The Commission disagrees with the
comments that allocation timestamps are outside the goal of CAT and
that they will not provide the Commission with the regulatory benefit
that it is seeking. As discussed in the Notice and below, the
Commission believes that allocation time is an important data field
because it is critical in investigations of violations such as market
manipulation and cherry-picking, and because allocation time is
currently more difficult to acquire than the other information on the
Allocation Report.\2040\ The inclusion of this data field will improve
the efficiency and efficacy of enforcement investigations for
regulators, and this benefit is one of the goals of the CAT NMS Plan.
With respect to the commenter who argued that alternate approaches that
do not rely upon allocation timestamps can be used to detect cherry
picking, the Commission notes that the commenter's example requires an
allocation time.
---------------------------------------------------------------------------

    \2040\ See Notice, supra note 5, at 30679; see also Section
V.E.2.c(3), infra.
---------------------------------------------------------------------------

    Regarding the possibility of data gaps between CAT and current data
sources, the Commission recognizes that there may be other gaps between
current regulatory data sources and the Plan, in addition to those that
the Commission mentioned in the Notice. The Commission also recognizes
that the number and the scope of these gaps can change over time due to
new regulatory developments. However, as discussed above, the
Participants have stated that they have completed the gap
analysis.\2041\ As set out in the Notice (and discussed above), the
Plan specifically provides that, prior to the retirement of existing
systems, CAT Data must contain data elements sufficient to ensure the
same regulatory coverage as the coverage provided by these systems.
Therefore, the Commission continues to believe that any missing
elements that are important to regulators would be incorporated into

the CAT Data prior to the retirement of the systems that currently
provide these data elements.
---------------------------------------------------------------------------

    \2041\ See Section IV.D.9, supra.
---------------------------------------------------------------------------

    The Commission is updating its analysis of these benefits to
recognize two modifications to the Plan. First, modifications to the
Plan to require the reporting of LEIs for Customers and Industry
Members in certain circumstances \2042\ should result in regulators
having access to more complete information identifying Customers and
Industry Members. Second, the Plan has been modified to eliminate the
requirement to report an open/close indicator for equities and Options
Market Markers. The inclusion of this indicator for equities and
Options Market Makers would have assisted regulators in determining
when an investor was buying to cover a short sale in equities or
identifying whether options market makers engage in aggressive risk-
taking trading. Such information would have been useful in detecting
certain market manipulations, violations of rules such as Rule 105,
short sale marking rules, and Rule 204. The Commission now notes that,
due to the elimination of the requirement to report an open/close
indicator for equities and Option Market Makers as part of CAT, these
benefits will no longer be realized. However, the Commission is
approving the Plan with this modification for the reasons discussed in
Section IV.D.4.c, above.
---------------------------------------------------------------------------

    \2042\ See Section IV.D.4.a.(4) and Section IV.D.4.b.(2), supra,
for a description of the LEI reporting requirements in the Plan.
---------------------------------------------------------------------------

b. Accuracy
    In the Notice, the Commission analyzed the expected effect of the
CAT NMS Plan on the accuracy of data available to regulators.\2043\ The
Commission preliminarily believed that the requirements in the CAT NMS
Plan for collecting, consolidating, and storing the CAT Data in a
uniform linked format, the use of consistent identifiers for Customers,
and the focus on sequencing would promote data accuracy. However, in
regard to certain Plan requirements, the Commission preliminary
believed that improvements in data accuracy would be limited. For
example, the Commission discussed its belief that the proposed clock
synchronization requirements in the Plan would only lead to modest
improvements in the percentage of sequenceable order events.\2044\
Also, the Commission noted that the full extent of improvement that
will result from the Plan was uncertain, because the Plan defers many
decisions relevant to accuracy until the Plan Processor publishes the
Technical Specifications and interpretations.\2045\
---------------------------------------------------------------------------

    \2043\ See Notice, supra note 5, at 30681-89.
    \2044\ Id.
    \2045\ See CAT NMS Plan, supra note 5, at Section 6.9.
---------------------------------------------------------------------------

(1) Definitions
    As previously stated, the Plan defers many decisions relevant to
accuracy until the Plan Processor publishes the Technical
Specifications and interpretations. In particular, the CAT NMS Plan
specifies that the ``Technical Specifications shall include a detailed
description of . . . each data element, including permitted values, in
any type of report submitted to the Central Repository'' \2046\ and
``the Plan Processor shall have sole discretion to amend and publish
interpretations regarding the Technical Specifications.'' \2047\ In the
Notice, the Commission explained that this leaves open precise
definitions and parameters for the data fields to be included in CAT
Data.\2048\ Nonetheless, the Commission discussed its preliminary
belief that the Plan provides some procedural protections to mitigate
this uncertainty and help promote accuracy. For example, the Plan
requires that, at a minimum, the Technical Specifications be
``consistent with [considerations and minimum standards discussed in]
Appendices C and D,'' and that the initial Technical Specifications and
any Material Amendments thereto must be provided to the Operating
Committee for approval by Supermajority Vote.\2049\ Further, all non-
Material Amendments and all published interpretations must be provided
to the Operating Committee in writing at least ten days before
publication, and shall be deemed approved unless two or more
unaffiliated Participants call the matter for a vote of the full
Operating Committee.\2050\
---------------------------------------------------------------------------

    \2046\ Id. at Section 6.9(b)(v).
    \2047\ The CAT NMS Plan provides details regarding how the
responsibility for these decisions would be shared between the
Operating Committee and the Plan Processor, with the Plan Processor
having responsibility for data definitions and interpretations. Id.
at Section 6.9(c)(i).
    \2048\ For example, the Completeness section in the Notice noted
that the open/close indicator for equities does not exist in current
data sources. See Notice, supra note 5, at 30681. The accuracy of
the open/close indicator for equities would have been subject to
Plan Processor discretion, because the Plan Processor would have had
responsibility for defining the permitted values and interpreting
when CAT Reporters would use such permitted values and the Plan
Processor would not have had guidance from previous data sources on
how to define or interpret such a field.
    \2049\ See CAT NMS Plan, supra note 5, at Section 6.9(a). The
Commission notes that the standards in Appendices C and D do not
cover all decisions that would affect the accuracy of the data.
    \2050\ Id. at Section 6.9(c)(i).
---------------------------------------------------------------------------

    The Commission received comments about the lack of definitions for
data fields in the Plan such as the open/close indicator,\2051\
allocation time,\2052\ account type,\2053\ and customer type.\2054\
Commenters argued that it is currently uncertain whether the Plan
Processor will select definitions that are the most beneficial to
regulators. For example, one commenter suggested that allocation time
may be challenging to define, stating that ``the industry does not have
a standard business flow which consistently captures time at the same
point in the allocation process.'' \2055\ This commenter further
pointed out that if allocation time is defined as the time the
allocation is booked, ``it will not provide the regulatory benefit
expected by the SEC,'' and provided an example of a way to detect
allocation fraud using the time ``when the allocation was submitted to
move the shares into the intended subaccounts.'' \2056\ The
Participants responded to the comments regarding the definitions of
allocation time, account type, and customer type by saying that the
definitions will be addressed in the Technical Specifications.\2057\
---------------------------------------------------------------------------

    \2051\ FIF Letter at 85.
    \2052\ TR Letter at 9; FIF Letter at 86.
    \2053\ TR Letter at 9.
    \2054\ TR Letter at 9.
    \2055\ FIF Letter at 86.
    \2056\ FIF Letter at 86, 89.
    \2057\ The Participants responded to the comments on open/close
more generally by requesting that the Commission clarify that the
open/close indicator should not apply to equities, and did not
respond regarding the definition. As noted elsewhere, modifications
to the Plan will remove the open/close indicator for equities. See
Section IV.D.4.c, supra.
---------------------------------------------------------------------------

    The Commission has considered the comments and believes they are
consistent with the Commission's assessment in the Notice that leaving
open precise definitions, parameters, and interpretations for the data
fields to be included in CAT Data creates uncertainty about the full
extent of improvements in data accuracy. The Commission is cognizant of
the complexity of certain data fields, such as allocation time. These
complexities mean that the accuracy of the data fields depends on Plan
Processor discretion, because the Plan Processor would have
responsibility for defining the permitted values and interpreting when
CAT Reporters would use such permitted values, and sometimes would not
have guidance from previous data sources on how to define or interpret
such a

field.\2058\ Although the Commission agrees that uncertainty exists in
the selection of data definitions and that definitions ultimately
selected may not promote accuracy as much as certain alternatives, as
discussed in Section V.G.4.a.(2), the Commission continues to believe
that the existing process trades off the need for certainty with the
benefits of an efficient process going forward. Further, for reasons
discussed above in Section IV.B. and below in Section V.E.3.d, the
Commission continues to believe that the Plan provides some procedural
protections to mitigate this uncertainty and help promote accuracy.
---------------------------------------------------------------------------

    \2058\ See Notice, supra note 5, at n.537. While the Commission
would ultimately be able to correct such misinterpretations,
regulators may not detect such a misinterpretation until the
misinterpretation harms an investigation, exam, or other analysis.
---------------------------------------------------------------------------

(2) Data Errors
    In the Notice, the Commission discussed the fact that the CAT NMS
Plan specifies a high-level process for handling errors that includes
target Error Rates for data initially submitted by CAT Reporters and a
correction process and timeline, but explained that it is difficult to
conclude whether the Error Rates and processes in the CAT NMS Plan
would constitute an accuracy improvement as compared to current data
sources. Specifically, because the current OATS error rate is below 1%
and the Plan states that 5% is an appropriate initial Error Rate, the
Commission preliminarily believed that the initial percentage of errors
in CAT would be higher than the current percentage of errors in OATS,
though the OATS error rate may not be directly comparable to the Error
Rate in the Plan.\2059\ As discussed in the Notice, Error Rates for CAT
Data may not be comparable to error rates in OATS because of the
increased scope and level of linkages specified in the Plan and the
new, large, and untested system.\2060\
---------------------------------------------------------------------------

    \2059\ Id. at 30681-82.
    \2060\ Id.
---------------------------------------------------------------------------

    In the Notice, the Commission also discussed that the Plan contains
some uncertainty about the level of the maximum Error Rate because the
initial 5% rate is subject to a quality assurance testing period and
subject to change again before each new batch of CAT Reporters are
brought online. The Commission noted that in time, the rate could be
lowered, but it also could be raised.\2061\ Finally, the Commission
discussed that the Plan specifies an error correction process and
indicates that practically all errors identifiable by the validations
used would be corrected by 8:00 a.m. ET on T+5, but that the Plan does
not provide the level of detail necessary to verify whether the CAT
validation process would run the same validations as OATS, whether
current validations would be relevant, and what validations, if any,
would be added.\2062\
---------------------------------------------------------------------------

    \2061\ Id.
    \2062\ Id.
---------------------------------------------------------------------------

    Although the Commission received several letters regarding data
error rates,\2063\ only a few letters discussed the effect of Error
Rates on the accuracy of CAT Data.\2064\ While supporting the goal of a
``de minimis'' post correction error rate, one commenter suggested that
the errors in CAT Data would not be ``de minimis'' even after the error
correction process because OATS currently does not achieve ``de
minimis'' errors.\2065\ For example, this commenter stated that there
are instances where errors cannot be corrected in OATS and gave true
duplicates and non-reportable symbols as examples.\2066\ The commenter
stated that it is unreasonable to expect CAT Data to be any different
than OATS data, especially because the industry has no experience with
reporting and error correcting the new data types required by the
Plan.\2067\ Another commenter expanded on this concern by questioning
why accuracy problems persist in OATS today and argued that the
improvements to accuracy from the Plan depend on eliminating the
inaccurate/problematic reporting that exists today.\2068\
---------------------------------------------------------------------------

    \2063\ See Section IV.D.11, supra for a complete summary of
comments and the Commission's discussion of those comments. Further,
the Commission responds to comments relevant to alternatives that
would reduce error rates below in Section V.H.2, infra.
    \2064\ FIF Letter at 50-60; Anonymous Letter II at 2; SIFMA
Letter at 6.
    \2065\ FIF Letter at 60.
    \2066\ FIF Letter at 55.
    \2067\ FIF Letter at 60.
    \2068\ Anonymous Letter II at 2.
---------------------------------------------------------------------------

    Other commenters expressed uncertainty regarding whether CAT
Reporters would be able to achieve the initial Error Rate of 5%.\2069\
One commenter indicated that there is not enough information at this
time to assess the Error Rate and that ``Error Rate'' is not
specifically defined.\2070\ Another commenter echoed this sentiment
saying that there is no history of reporting error rates for options,
market making, customer information, or allocations and the Plan
provides ``little or no information . . . regarding the types of errors
that will be identified, and if and how those errors can be
corrected.'' \2071\ The commenter also cited uncertainties related to
the inexperience of some CAT Reporters, unknown interfaces, a lack of
information on test tools and correction tool kits, and an unknown
linkage logic.\2072\
---------------------------------------------------------------------------

    \2069\ FIF Letter at 50; SIFMA Letter at 6.
    \2070\ SIFMA Letter at 6.
    \2071\ FIF Letter at 50.
    \2072\ FIF Letter at 50.
---------------------------------------------------------------------------

    Finally, one commenter agreed with the Commission's analysis that
OATS error rates may not be directly comparable to a CAT Error
Rate.\2073\ In particular, this commenter stated that OATS would be a
sufficient comparison base for equities data only, but not for options,
allocations, Customer Information, or market making reporting.
---------------------------------------------------------------------------

    \2073\ FIF Letter at 55.
---------------------------------------------------------------------------

    In response to the comments on uncertainty in the definition of
Error Rate, the Participants disagreed, pointing to the current
definition in the Plan and in Rule 613(j)(6).\2074\ The Participants
further stated that they intend to keep the definition of Error Rate
the same as in Rule 613 and noted that it is the Compliance Thresholds
\2075\ that relate to the CAT reporting performance of individual CAT
Reporters. In response to commenters expressing uncertainty about the
ability to achieve the Error Rates, the Participants explained that
they performed a detailed analysis that not only considered current and
historical OATS error rates, but also considered the magnitude of the
new reporting requirements and the fact that many CAT Reporters had
never previously been obligated to report data for audit trail
purposes.\2076\ The Participants, however, acknowledged that actual
experience with CAT itself will provide more accurate and applicable
data for determining the appropriate Error Rate and pointed out that
the Plan provides for various opportunities for the Error Rate to be
reevaluated and reset.\2077\
---------------------------------------------------------------------------

    \2074\ Response Letter I at 45.
    \2075\ The Error Rate reports shall include each of the
following--if the Operating Committee deems them necessary or
advisable--``Error Rates by day and by delta over time, and
Compliance Thresholds by CAT Reporter, by Reportable Event, by age
before resolution, by symbol, by symbol type (e.g., ETF and Index)
and by event time (by hour and cumulative on the hour)[.]'' See CAT
NMS Plan, supra note 5, at Section 6.1(o)(v).
    \2076\ Response Letter I at 46.
    \2077\ Response Letter I at 46.
---------------------------------------------------------------------------

    The Commission has considered the comment letters received and the
Participants' response and continues to believe that it is difficult to
determine whether the Error Rates and processes in the Plan would
constitute an accuracy improvement compared to current data. The
Commission recognizes the uncertainty regarding the ability to achieve
a ``de minimis'' post-correction Error Rate discussed by a

commenter and notes that post-correction errors are the ones more
pertinent to the accuracy of data used in regulatory activities. While
the Commission is concerned with the effect of the post-correction
Error Rate on accuracy, it notes that, while uncertain, the Plan does
require the Plan Processor to perform validations within three specific
categories: File Validations (confirmation that the file is received in
the correct format); Validation of CAT Data (checks of format, data
type, consistency, range/logic, data validity, completeness, and
timeliness); and Linkage Validation (checking the ``daisy
chain'').\2078\ Specifically, in regard to Linkage Validation, the Plan
seems to require validations that are more comprehensive than what
FINRA runs on OATS data, where, as stated in the Notice, significant
error rates in event linking are common because there is no cross-
participant error resolution process.\2079\ Further, the OATS error
types described in the Baseline above \2080\ also suggest that the
Plan's validations will be more comprehensive than the validations run
on OATS data.
---------------------------------------------------------------------------

    \2078\ See CAT NMS Plan, supra note 5, at Appendix D, Section
7.2.
    \2079\ See Notice, supra note 5, at 30667.
    \2080\ See Section V.D.2.b(2)A, supra, which lists error types
as rejects, unmatched exchange routes, inter firm received
unmatched, inter-firm sent unmatched, out of sequence, and late
reports.
---------------------------------------------------------------------------

    The Commission agrees with the commenters that expressed
uncertainty about whether CAT would be able to achieve the 5% initial
Error Rate, but also agrees with the Participants' response. In the
Participants' analysis, the Participants considered the magnitude of
the new reporting requirements and the fact that many CAT Reporters had
never previously been obligated to report data for an audit trail when
they set the initial Error Rate. Furthermore, as mentioned in the
Notice, the Plan provides for various opportunities for the Error Rate
to be reevaluated and reset after CAT Reporters have more experience
with CAT.\2081\
---------------------------------------------------------------------------

    \2081\ Id. at 30682.
---------------------------------------------------------------------------

    Finally, the Commission agrees with the comment that OATS error
rates may not be comparable to a CAT Error Rate because there is
currently no reporting regime comparable to OATS for options,
allocations, Customer Information, or market making reporting. In the
Notice, the Commission discussed uncertainty in comparing OATS error
rates to CAT Error Rates due, in part, to the increased scope of the
CAT NMS Plan.\2082\
---------------------------------------------------------------------------

    \2082\ Id.
---------------------------------------------------------------------------

(3) Event Sequencing
A. Timestamp Granularity
    In the Notice, the Commission discussed its preliminary belief that
the minimum timestamp granularity required by the Plan would result in
some improvement in data accuracy, but that the level of improvement
could be limited. The CAT NMS Plan requires timestamps to the
millisecond.\2083\ This is consistent with Rule 613, which requires
timestamps to reflect current industry standards and be at least to the
millisecond.\2084\ Further, pursuant to Rule 613, if a CAT Reporter's
system already utilizes timestamps in increments less than the minimum
required by the Plan, the CAT Reporter must record timestamps in such
finer increments.\2085\
---------------------------------------------------------------------------

    \2083\ See CAT NMS Plan, supra note 5, at Section 6.8(b).
    \2084\ 17 CFR 242.613(d)(3). This requirement does not apply to
certain Manual Order Events, which are exempted from the requirement
and are captured at one-second increments. Timestamp granularity on
Manual Order Events is discussed separately in the Alternatives
section.
    \2085\ Id.
---------------------------------------------------------------------------

    As the Commission discussed in the Notice, many of the systems from
which regulators currently obtain data already capture timestamps in
increments of milliseconds or less, meaning that there would be no
improvement in timestamp granularity as compared to those
systems.\2086\ However, to the extent that some current data sources
report timestamps in increments coarser than a millisecond, which is
the case for 12% of OATS records and all EBS records,\2087\ the
Commission noted that it expected the CAT millisecond timestamp
requirement to improve data granularity, and thereby allow regulators
to more accurately determine the sequence of market events relative to
surrounding events. However, the Commission also explained that the
benefits from the more granular timestamps could be limited by the
level of clock synchronization required by the Plan. In particular, the
Commission explained that timestamp granularity would not be the
limiting factor in sequencing accuracy, because recording events with
timestamps with resolutions of less than one millisecond cannot help to
sequence events occurring on different venues with clocks that may be
100 milliseconds out of sync due to clock synchronization
offsets.\2088\ Therefore, the benefits of timestamping order events at
increments finer than a millisecond would be limited without also
improving the clock synchronization standards of the Plan.
---------------------------------------------------------------------------

    \2086\ For example, OPRA allows for timestamps in nanoseconds,
and the other SIPs require timestamps in microseconds for equity
trades and quotes. See Notice, supra note 5, at Section
IV.D.2.b.(2).
    \2087\ Current OATS rules require timestamps to be expressed to
the nearest second, unless the member's system expresses time in
finer increments. As of September 2014, approximately 12% of OATS
records contain timestamps greater than one millisecond. EBS records
either do not contain times or express timestamps in seconds. Id.
    \2088\ For example, under the requirements in the Plan, an order
event at Broker-Dealer A could have a timestamp that is 1
millisecond sooner than an order event at Broker-Dealer B even if
the event at Broker-Dealer B actually occurred 99 milliseconds
sooner. This could occur if Broker-Dealer A's systems are recording
times 50 milliseconds ahead of NIST while Broker-Dealer B's systems
are recording times 50 milliseconds behind NIST. Both broker-
dealers' systems would be within the Plan's allowable clock
synchronization tolerance.
---------------------------------------------------------------------------

    The Commission discussed the benefits of the one second timestamp
on manual orders and stated that it preliminarily believed that
timestamp granularity of one second would be appropriate for manual
orders, rather than a millisecond granularity, because recording Manual
Order Events at the millisecond level would be ultimately arbitrary or
imprecise due to human interaction.\2089\
---------------------------------------------------------------------------

    \2089\ See Notice, supra note 5, at 30684.
---------------------------------------------------------------------------

    Two commenters thought that a millisecond timestamp would be
sufficient to achieve improvements in event sequencing.\2090\ One of
these commenters suggested that requiring timestamps that are more
granular than one millisecond for CAT Reporters who capture timestamps
more granular than a millisecond would not yield regulatory benefits as
it will result in a false sense of accuracy on event sequencing.\2091\
An additional commenter did not support this requirement, stating that
it would be inequitable and would not serve a regulatory purpose.\2092\
On the other hand, two commenters supported the requirement that CAT
Reporters report sub-millisecond timestamps if they capture them.\2093\
One commenter stated their belief that timestamp granularity ``should
go hand-in-hand with how fast a market participant is allowed to
conduct their HFT activities.'' \2094\ The other commenter stated that
a ``significant portion of today's trades occur at microsecond
intervals,'' and that the Plan's timestamp resolution ``will be
insufficient to show the precise time of the reportable activities.''
\2095\ The commenter further stated that ``[f]or some practices, such
as cancellations,

stuffing, and other ``noisy'' behaviors, the Plan should ``require a
more precise granularity to more comprehensibly and accurately capture
the frequency and scale of such practices.'' \2096\ One commenter
stated their belief that stricter tolerances for the granularity of
timestamps are already in effect at exchanges and ATSs that maintain an
orderbook and did not believe it necessary to mandate timestamp
tolerances for these entities since they already adhere to stricter
tolerances for commercial reasons.\2097\
---------------------------------------------------------------------------

    \2090\ FIF Letter at 112; Data Boiler Letter at 21.
    \2091\ FIF Letter at 12.
    \2092\ SIFMA Letter at 35.
    \2093\ Data Boiler Letter at 21; Better Markets Letter at 8.
    \2094\ Data Boiler Letter at 21.
    \2095\ Better Markets Letter at 8.
    \2096\ Better Markets Letter at 8.
    \2097\ TR Letter at 7
---------------------------------------------------------------------------

    Two commenters indicated that timestamp granularity and clock-
offset tolerance for allocation timestamps should be at one
second.\2098\ One commenter argued that the benefits of allocation time
would not require millisecond precision while three commenters argued
that allocations are not time-critical.\2099\
---------------------------------------------------------------------------

    \2098\ SIFMA Letter at 35; FIF Letter at 87, 89.
    \2099\ FSR Letter at 9; SIFMA Letter at 35; FIF Letter at 89.
---------------------------------------------------------------------------

    One commenter expressed that the irregularity in manual orders made
it difficult to set a tolerance applicable to all manual orders and
suggested that initially a timestamp tolerance of more than one second
be allowed for manual orders.\2100\ However, several other commenters
stated that one second is a reasonable standard for manual
orders.\2101\
---------------------------------------------------------------------------

    \2100\ Data Boiler Letter at 21-22.
    \2101\ FIF Letter at 115; SIFMA Letter at 34; Better Markets
Letter at 8; Response Letter I at 38. However, Better Markets
expressed the concern that gaming of the system could occur by
writing algorithms to make automated orders appear as manual orders.
---------------------------------------------------------------------------

    In their response, the Participants stated their belief that CAT
Reporters should be required ``to report timestamps to the CAT at the
granularity at which they are captured, even if that is more granular
than that required by the Plan.'' They further stated their belief that
capturing such granularity would increase the quality of data reported
to the CAT.\2102\ With respect to the timestamps on Allocation Reports,
the Participants recognized the practical issues raised by requiring
timestamps for Allocation Reports and proposed to amend the Plan to
permit CAT Reporters to report allocation timestamps with a granularity
of one second.\2103\ With respect to manual order timestamps, the
Participants stated that they continued to believe their proposed
approach to Manual Order Events is appropriate.\2104\
---------------------------------------------------------------------------

    \2102\ Response Letter I at 28.
    \2103\ Response Letter I at 37.
    \2104\ Response Letter I at 38.
---------------------------------------------------------------------------

    The Commission has considered the comment letters received and the
Participants' response, and as discussed in more detail above,\2105\
has amended the Plan so that Participants are required to adhere to a
more stringent clock synchronization standard of 100 microseconds and
allocation timestamps need only be reported in seconds instead of
milliseconds. The Commission is updating its economic analysis to
incorporate these modifications to the Plan. The Commission agrees with
the commenter who pointed out that millisecond timestamps are
insufficient to show the precise timestamp of certain activities and
disagrees with commenters who stated that millisecond precision is
sufficient to sequence events. As stated in the Notice, the Commission
believes that a 1 millisecond timestamp granularity offers benefits
over the Baseline, but that a more granular timestamp requirement,
coupled with a more stringent clock synchronization requirement, would
be needed to completely sequence the majority of unrelated market
events. In response to the commenters who questioned the benefits of
reporting the sub-millisecond timestamps if CAT Reporters capture them,
the Commission agrees with the Participants that such a requirement
will increase the quality of data reported to the CAT.
---------------------------------------------------------------------------

    \2105\ See Section IV.D.13, supra.
---------------------------------------------------------------------------

    Modifications to the Plan now require Participants to adhere to a
more stringent clock synchronization standard of 100 microseconds (or
less), and CAT Reporters to record timestamps in finer increments than
1 millisecond if their systems utilize timestamps in such finer
increments. Because, as discussed above,\2106\ Participants already
operate pursuant to a clock synchronization standard of 100
microseconds with regard to their electronic systems, and because many
Participants already report timestamps in microseconds and nanoseconds
in their direct feeds and are currently required to report timestamps
in microseconds for equity trades and quotes, the Commission does not
believe the clock synchronization amendment to the Plan will result in
large accuracy improvements over current standards for timestamp
granularity. However, the Commission is approving the Plan without
further modifications for the reasons discussed in Section IV.D.13,
above.
---------------------------------------------------------------------------

    \2106\ See Section IV.D.13.a(1), supra.
---------------------------------------------------------------------------

    In the Notice, the Commission did not explicitly consider timestamp
granularity or clock synchronization standards for timestamps in
Allocation Reports. However, in response to comments and modifications
to the timestamp on Allocation Reports, the Commission now analyzes
whether the modifications limit the improvements to accuracy. Based on
the experience of its Staff, the Commission understands that
allocations are conducted after a trade and that the allocation time
can aid regulators in ways that do not require millisecond-level
timestamps (or 50 millisecond clock offset tolerance). Further, the
Commission agrees with the commenter's argument that allocations are
not time-sensitive and the benefits from allocation timestamps do not
require millisecond precision. Therefore, the Commission believes that
requiring allocation times to be recorded in milliseconds (with 50
millisecond offset tolerance) compared to seconds (with one second
tolerance) would provide little, if any, additional regulatory benefit.
Therefore, the Commission does not believe that this modification
materially reduces the improvements to accuracy.
B. Clock Synchronization
    In the Notice, the Commission discussed its belief that the clock
synchronization standards in the CAT NMS Plan are reasonably designed
to improve the accuracy of market activity sequencing, but that the
improvements to the percentage of sequenceable order events by Plan
standards are modest and the requirements of the Plan may not be
sufficient to completely sequence the majority of market events
relative to all other events. In particular, the Commission conducted
an analysis using MIDAS data that found that the current FINRA one-
second clock offset tolerance allows only an estimated 1.31% of
unrelated order events \2107\ for listed equities and 6.97% of
unrelated order events for listed options to be sequenced.\2108\ By
comparison, the proposed 50 millisecond clock offset tolerance could
accurately sequence an estimated 7.84% of unrelated order events for
listed equities and 18.83% of unrelated order events for listed
options.\2109\ Also, by comparison, the analysis found that a 100
microsecond clock offset tolerance, if applied to all reporters, could
accurately sequence an estimated 42.47% of unrelated order events for
listed equities and 78.42% of unrelated order events for listed

options.\2110\ In the Notice, the Commission discussed its preliminary
belief that the analysis suggests the standards required by the Plan do
represent an improvement over the current standard but that the
majority of unrelated market events would remain impossible to sequence
based on the Plan's required clock synchronization standards.\2111\
---------------------------------------------------------------------------

    \2107\ See Notice, supra note 5, at 30669 for a definition of
unrelated order events as it relates to this analysis and the
analysis described there.
    \2108\ Id. at 30683. The Commission discussed that these
estimates were upwardly biased.
    \2109\ Id. The Commission discussed that these estimates were
upwardly biased.
    \2110\ A 100 microsecond clock offset tolerance will now be
required of Participants due to an amendment to the Plan.
    \2111\ The Commission noted that the Plan itself states ``[f]or
unrelated events, e.g., multiple unrelated orders from different
broker-dealers, there would be no way to definitively sequence order
events within the allowable clock drift as defined in Article 6.8,''
and that this limitation ``in turn limits the benefits of CAT in
regulatory activities that require event sequencing, such as the
analysis and reconstruction of market events, as well as market
analysis and research in support of policy decisions, in addition to
examinations, enforcement investigations, cross-market surveillance,
and other enforcement functions.'' See Notice, supra note 5, at
30683.
---------------------------------------------------------------------------

    The Commission also discussed in the Notice that, independent of
the potential time clock synchronization benefits, order linking data
captured in CAT should increase the proportion of order events that are
accurately sequenced.\2112\ This is because some records pertaining to
the same order can be sequenced by their placement in an order
lifecycle (e.g., an order submission must have occurred before its
execution) without relying on timestamps.
---------------------------------------------------------------------------

    \2112\ See Notice, supra note 5, at n. 555.
---------------------------------------------------------------------------

    Although the Commission received several comment letters related to
clock synchronization, which are discussed in detail in Section
IV.D.13.a above, only two letters commented on the effects of clock
synchronization standards on event sequencing.\2113\ Both commenters
agreed with the Commission's assessment that provisions in the CAT NMS
Plan related to event sequencing would provide improvements in accuracy
compared to what is currently achievable.\2114\ However, one of these
commenters further stated their belief that unrelated events may not be
sequenceable and stated that it is unclear what the regulatory
relevance is of sequencing unrelated events across market
centers.\2115\ The commenter went on to say that there was no evidence
that lower clock synchronization tolerances would increase the accuracy
of the audit trail; \2116\ however, the commenter also stated that
``more precise timestamps provided by exchanges may be of benefit to
the audit trail as corroborating evidence when sequencing events that
terminate at an exchange.'' \2117\
---------------------------------------------------------------------------

    \2113\ FIF Letter at 97-111; Data Boiler Letter at 31.
    \2114\ FIF Letter at 101; Data Boiler Letter at 31.
    \2115\ FIF Letter at 101.
    \2116\ FIF Letter at 111.
    \2117\ FIF Letter at 98.
---------------------------------------------------------------------------

    The Commission has considered the comment letters received, the
Participants' response, and amendments to the Plan. As explained below,
the Commission continues to believe that requirements in the Plan
related to event sequencing would provide improvements in accuracy
compared to what is currently achievable, but that improvements are
modest and the requirements to the Plan may not be sufficient to
completely sequence the majority of market events relative to all other
events. Orders sent from different broker-dealers to different CAT
Reporters can only be sequenced in CAT Data according to their
timestamp. If the clocks of CAT Reporters are not synchronized with
sufficient precision, it is impossible to definitively sequence these
events. The Plan acknowledges this limitation and states, ``[f]or
unrelated events, e.g., multiple unrelated orders from different
broker-dealers, there would be no way to definitively sequence order
events within the allowable clock drift as defined in Article 6.8.''
\2118\ The Commission disagrees with the comment that sequencing
unrelated market events has no regulatory relevance. As discussed in
the Notice, the ability to sequence market events is crucial to the
efficacy of detecting and investigating some types of manipulation,
particularly those involving high frequency trading, those in liquid
stocks in which many order events can occur within microseconds, and
those involving orders spread across various markets. The Commission
also disagrees with this commenter's assessment that more stringent
clock synchronization standards would not increase the accuracy of the
audit trail. As demonstrated by the Commission's analysis in the
Notice, if clock synchronization standards were made more stringent,
some of the many market events at separate market centers that occur
within small time windows would become sequenceable, which would
increase the accuracy of the audit trail.
---------------------------------------------------------------------------

    \2118\ See CAT NMS Plan, supra note 5, at Appendix C-25.
---------------------------------------------------------------------------

    As discussed in more detail above,\2119\ the Commission has amended
the Plan so that Participants are required to adhere to a more
stringent clock synchronization standard of 100 microseconds with
regard to electronic systems, excluding certain manual systems. In the
Participants' response, they noted that all Participants currently
operate pursuant to a clock synchronization standard of 100
microseconds with regard to their electronic systems, so that the
amended requirement is already met by the Participants.\2120\ In
addition, as discussed in more detail above,\2121\ the Commission has
approved a proposed rule change by FINRA that reduces the
synchronization tolerance for computer clocks of firms that record
events in NMS Securities to 50 milliseconds.\2122\ Because broker-
dealers that are FINRA members are currently required to adhere to a
clock synchronization standard of 50 milliseconds, and because
Participants already adhere to a clock synchronization standard of 100
microseconds, the Commission does not believe the 50 millisecond clock
synchronization requirement of CAT Reporters and the more stringent
clock synchronization requirement of 100 microseconds for Participants,
as specified in the amended Plan, would substantially change the
ability of regulators to accurately sequence unrelated market events
over what is currently achievable using timestamps alone.\2123\
However, the Commission is approving the Plan without further
modifications for the reasons discussed in Section IV.D.13, above.
Further, to the extent CAT captures more events than are currently
captured, such as CAT Reportable Events by broker-

dealers that are not FINRA members (see Section V.E.1.a.(1)),
regulators will be able to accurately sequence a proportion of those
events, which will increase the overall number of sequenced
events.\2124\ In addition, the Commission continues to believe that,
independent of the potential clock synchronization benefits, the order
linking data that would be captured by the CAT should increase the
proportion of events that could be sequenced accurately.\2125\
---------------------------------------------------------------------------

    \2119\ See Section IV.D.13, supra.
    \2120\ Response Letter II at 4-5.
    \2121\ See Section IV.D.13.a(1), supra.
    \2122\ See Securities Exchange Act Release No. 77565 (April 8,
2016), 81 FR 22136 (April 14, 2016).
    \2123\ Although not currently required in the Plan, the
Commission believes there would be additional benefit to event
sequencing if off-exchange execution venues, including alternative
trading systems and broker-dealer internalizers, were required to
adhere to a more stringent clock synchronization standard. As
discussed in Section IV.D.13.a, the Commission understands that
certain Industry Members, such as ATSs and broker-dealers that
internalize off-exchange order flow, today adhere to a finer clock
synchronization standard. As the Participants conduct their annual
reviews, the Commission expects them to consider proposing new clock
synchronization standards whenever they determine the industry
standard for CAT Reporters, or certain categories or systems
thereof, has become more granular than required by the Plan at that
time. In determining the appropriate industry standards for clock
synchronization, the Commission has amended the Plan so that the
SROs should apply industry standards based on the type of CAT
Reporter or system, rather than the industry as a whole. Varied
requirements would segment the broker-dealer community, and one
commenter stated a desire to ``avoid unnecessary market
segmentation'' with regard to clock synchronization. See FSR Letter
at 8. See also Section IV.D.13.a(1), supra. The Commission notes,
however, that these venues are already segmented with respect to
their position within the broker-dealer and also with respect to
other broker-dealers that do not provide these services.
    \2124\ Note that broker-dealers that are not FINRA members are
not subject to FINRA's clock synchronization requirements and do not
submit reports to OATS. Currently, their activity, to the extent it
is captured, is captured and timestamped by exchanges and other
FINRA members that receive their orders.
    \2125\ As discussed in the Notice, this reflects the fact that
some records pertaining to the same order could be sequenced by
their placement in an order without relying on timestamps. This
information may also be used to partially sequence surrounding
events. See Notice, supra note 5, at n.555.
---------------------------------------------------------------------------

(4) Linking and Combining Data
    In the Notice, the Commission discussed its preliminary belief that
the requirements of Rule 613 and the Plan related to data linking would
result in improvements to the accuracy of the data available to
regulators, but the extent of the improvement would depend on the
accuracy of the linking algorithm and the reformatting process that the
Plan Processor would eventually develop. Specifically, the Commission
discussed its belief that the requirement that data be stored in a
uniform format would eliminate the need for regulators to reformat the
data, and that storing data in a linked format removes the need for
regulators to link information from multiple lifecycle events of an
order or orders themselves, which could further reduce errors and
increase the usability of the data. Furthermore, the Commission
discussed its belief that the Plan would significantly improve the
ability of regulators to link order events compared to OATS, and would
link this activity to specific customers, unlike current audit trail
data. However, the Commission also noted that the CAT NMS Plan does not
provide sufficiently detailed information for the Commission to
estimate the likely error rates associated with the linking process
required by the CAT NMS Plan.\2126\ Accordingly, while the centralized
linking should generally promote efficiencies and accuracies, the
Commission stated that these uncertainties make it difficult to gauge
the degree to which the process for linking orders across market
participants and SROs would improve accuracy compared to existing data,
including OATS.\2127\
---------------------------------------------------------------------------

    \2126\ While the 5% Error Rate covers data from CAT Reporters,
the Plan Processor could create errors as well, for example, through
the linking process. Further, the Plan does not include details on
how the Plan Processor would perform the linking process, identify
broken linkages, and seek corrected reports from CAT Reporters to
correct broken linkages. Instead, the Plan defers key decisions
regarding the validation process until the selection of a Plan
Processor and the development of Technical Specifications. The CAT
NMS Plan describes the Plan Processor's responsibility for creating
the Technical Specifications. See CAT NMS Plan, supra note 5, at
Section 6.9.
    \2127\ The Commission notes that the Plan Processor is required
to create a quality assurance testing environment in which, during
industry-wide testing, the Plan Processor provides linkage
processing of data submitted, the results of which are reported back
to Participants and to the Operating Committee for review. Id. at
Appendix D, Section 1.2. This may help identify challenges in the
linking process and allow for their early resolution.
---------------------------------------------------------------------------

    The Commission also explained that uncertainties prevented it from
determining whether the process for converting data into a uniform
format at the Central Repository would improve the accuracy of the data
over existing audit trail accuracy rates.\2128\ The Plan includes two
alternative approaches to data conversion. In the first, called
Approach 1, CAT Reporters would submit data to the Central Repository
in an existing industry standard protocol of their choice such as the
Financial Information eXchange (``FIX'') protocol. In Approach 2, CAT
Reporters would submit data to the Central Repository in single
mandatory specified format, such as an augmented version of the OATS
protocol. Under Approach 1, the data must be converted into a uniform
format at the Central Repository in a second step. Under Approach 2,
the data is already in a uniform format at the time of submission. The
Plan defers the decision regarding which approach to take until the
selection of a Plan Processor and the development of Technical
Specifications. The Commission explained its preliminary belief that
Approach 1 would likely result in a lower Error Rate than Approach 2
because of increased efficiency and accuracy due to specialization by
the Plan Processor.\2129\ However, because of uncertainties regarding
expected Error Rates and error rates in current data, the Commission
was unable to evaluate the degree to which the approach would improve
data accuracy relative to currently available data.\2130\
---------------------------------------------------------------------------

    \2128\ See Notice, supra note 5, at 30686.
    \2129\ Id.
    \2130\ Id.
---------------------------------------------------------------------------

    The Commission also discussed its belief that the Plan's
requirement for standardized Allocation Reports that consistently and
uniquely identify Customers and CAT Reporters should improve the
linkability of allocation information compared to current data, despite
the limitation of direct linkage to order lifecycles, particularly in
scenarios where potentially violative conduct is carried out by market
participants operating through multiple broker-dealers.\2131\ The
Commission stated that this moderate improvement in the linkability of
allocation data should improve regulators' ability to identify market
participants who commit violations related to improper subaccount
allocations.
---------------------------------------------------------------------------

    \2131\ Id.
---------------------------------------------------------------------------

    The Commission received two comment letters that agreed with the
Commission's assessment that Plan provisions related to data linking
would increase the overall accuracy of data available to regulators.
One of these commenters stated that, ``the provisions in the CAT NMS
Plan (linkage requirements, daisy chains, Firm Designated ID) will
result in a more complete and accurate linking of order events across
market participants and SROs.'' \2132\ The other commenter agreed that
data accuracy would improve.\2133\
---------------------------------------------------------------------------

    \2132\ FIF Letter at 96.
    \2133\ Data Boiler Letter at 31.
---------------------------------------------------------------------------

    Commenters also opined on whether data should be stored in a
standardized format and on the relative economic effects of different
approaches to data ingestion formats. One commenter stated that the
Plan's requirement to store data in a standardized format would
increase accuracy within that format, but on the other hand,
transformation by CAT Reporters could introduce errors during the data
submission process.\2134\ The commenter further stated that using
original data reduces the chance of introducing noise.\2135\ Several
commenters indicated that existing and widely used formats or protocols
for data ingestion would promote better data accuracy.\2136\ Some also
noted that without a uniform data ingestion format, data quality would
suffer.\2137\
---------------------------------------------------------------------------

    \2134\ Data Boiler Letter at 31.
    \2135\ Data Boiler Letter at 18.
    \2136\ FIF Letter at 90-91; FIX Letter at 1; ICI Letter at 13;
Better Markets Letter at 7-8.
    \2137\ Better Markets Letter at 7-8; UnaVista Letter at 2-3.
---------------------------------------------------------------------------

    The Commission received one comment related to the ability to link
allocations under the Plan. Specifically, the commenter stated that an
allocation report is ``undeniably useful for analytic[al] purpose[s],''
but noted challenges in linking account and subaccount information to
which an execution is allocated.\2138\
---------------------------------------------------------------------------

    \2138\ Data Boiler Letter at 24-25.

---------------------------------------------------------------------------

    The Commission has considered the comment letters received, and
continues to believe that the requirements of the Plan related to data
linking would result in improvements to the accuracy of the data
available to regulators. The Commission agrees with the commenter who
stated that transforming data into a uniform format can introduce
errors, but the Commission believes such errors will be less common and
severe than those introduced currently by multiple regulators
independently linking together many different data sources with
different formats.\2139\ The Commission agrees with the commenters that
stated requiring existing and widely used formats for data ingestion
would promote the accuracy of data. Because the Plan does not mandate
an ingestion format, uncertainty exists as to what ingestion format (or
formats) will be required and whether the ingestion format(s)
ultimately selected will promote accuracy as much as alternatives. The
Commission acknowledges this uncertainty. In response to the commenters
that stated that data quality would suffer without a uniform data
ingestion format, as specified in Approach 2, the Commission continues
to believe that the benefits to data accuracy are potentially greater
using Approach 1, where data is ingested in an existing industry
standard protocol of the submitter's choice and subsequently converted
to a uniform format at the Central Repository. The Commission believes
this approach is more likely to benefit data accuracy because, as
stated by a commenter, allowing the use of original data eliminates the
introduction of errors and specialization by the Plan Processor should
keep to a minimum the number of errors introduced during the conversion
process.
---------------------------------------------------------------------------

    \2139\ See Section V.D.2.b.(2).C, supra.