o. Maintenance and Distribution of Dynamics Data Requirements and
Reporting Procedures (MOD-013-1)
    1187. MOD-013-1 requires the regional reliability organizations
within an Interconnection to develop comprehensive dynamics data
requirements and reporting procedures needed to model and analyze the
dynamic behavior and response of each Interconnection. More
specifically, the regional reliability organization, in coordination
with its transmission owners, transmission planners, generator owners
and resource planners within an Interconnection, is required to: (1)
Participate in development of documentation for their Interconnection
data requirements and reporting procedures; (2) participate in the
review of those data requirements and reporting procedures at least
every five years and (3) make the data requirements and reporting
procedures available to NERC and other specified entities upon request.
    1188. In the NOPR, the Commission identified MOD-013-1 as a fill-
in-the-blank standard that requires each regional reliability
organization within an Interconnection to develop comprehensive
dynamics data requirements and reporting procedures needed to model and
analyze the dynamic behavior and response for each of the three NERC
Interconnections. The NOPR stated that because the regional
methodologies had not been submitted, the Commission would not propose
to approve or remand MOD-013-1 until the ERO submits additional
information. In addition, in the NOPR we agreed that the Reliability
Standard should apply to the planning authority.
    1189. In the NOPR, the Commission expressed a concern regarding the
1990 cut-off date,\354\ and shared PG&E's concern that the difficulty
in obtaining unit-specific data is not limited to the age, but may also
be due to other factors such as unit configuration. The Commission
requested comment whether it is reasonable to permit entities to
estimate dynamics data if they are unable to obtain unit specific data
for any reason. The Commission believes that to achieve the goal of
this Reliability Standard of having the ability to accurately model and
analyze the dynamic behavior and response of each Interconnection, it
is necessary to have accurate data. Inaccurate data can lead to
unrealistic simulations and inappropriate actions by responsible
entities which may jeopardize the reliability of the Bulk-Power System.
---------------------------------------------------------------------------

    \354\ Requirement R1.1.1 allows for the use of estimated or
typical manufacturer's data on pre-1990 units to model dynamic
behavior when unit-specific data is unavailable.
---------------------------------------------------------------------------

i. Comments
    1190. APPA agrees with the Commission that MOD-013-1 is a fill-in-
the-blank standard, is not sufficient as currently drafted, and should
not be approved as a mandatory Reliability Standard until NERC and the
regional reliability organizations/Regional Entities develop the
necessary regional methodologies and the Commission approves them.
    1191. In response to the Commission's request for comments on
whether it is reasonable to permit entities to estimate dynamics data
if they are unable to obtain unit specific data for any reason, many
commenters responded that it is reasonable to allow estimation of
dynamics data for older units where data is not available.\355\ The
Small Entities Forum expects that the Reliability Standard ultimately
will include requirements that such estimates be based on sound
engineering principles and be subject to technical review and approval
of any estimates at the regional level.
---------------------------------------------------------------------------

    \355\ EEI, LPPC, MidAmerican, Small Entities Forum and TVA.
---------------------------------------------------------------------------

    1192. MidAmerican explains that there may be safety or system
conditions and/or the loss of records that do not permit gathering
unit-specific information, and that in such cases, computations and
engineering reports of estimated capability should be sufficient.
MidAmerican also requests that if there is a farm of similar generation
units (such as wind turbines) or synchronous condensers located in the
same general area, providing unit-specific information for a number of
identical units is not necessary. Instead, MidAmerican proposes that
information about a sample of the identical units (such as two) should
be sufficient to provide enough unit-specific information to be
representative of the farm. MidAmerican also notes that if units are
located in a part of the system that does not typically demonstrate
instability, the value of unit-specific data is reduced, and that there
are a number of such circumstances in which provision of unit-specific
data should not be required.
    1193. International Transmission, stating that the age of the unit
alone may not be the only reason why unit-specific data might be
unavailable, cautions that there should be a requirement in every case
that unit data actually be sought for all generating units before
estimates of dynamics data are used. International Transmission
believes that achieving the most accurate possible picture of the
dynamic behavior of the Interconnection requires the use of actual
data, and that, at a minimum, entities should be required to document
the steps taken to obtain unit-specific data.
    1194. APPA, however, expresses its concern regarding the
difficulties in obtaining accurate unit-specific data to model dynamic
behavior. APPA recommends to NERC that the regional reliability
organizations/Regional Entities and the reliability coordinators review
this type of data on a case-by-case basis to test it for accuracy and
to determine whether estimated data will produce outputs from the
models within acceptable limits. International Transmission confirms
that testing is easily accomplished, and provides up-to-date dynamics
data reflective of the natural degradation of generating units over
their lifetimes. However, International Transmission says that this
effort could be tied to the Generator Model Validation Reliability
Standards (MOD-024-1 and MOD-025-1).
    1195. TANC agrees with the Commission that the standard requirement
is arbitrary in imposing the

1990 cut-off with regard to modeling dynamic behavior. TANC believes
that this requirement allows for the use of estimated or typical
manufacturer's data on pre-1990 units to model dynamic behavior when
unit-specific data is unavailable. TANC notes that difficulty in
obtaining unit specific data is not limited to the age of the unit but
also unit configuration. TANC therefore recommends that the 1990 cut-
off be removed from the proposed Reliability Standard because there is
no justifiable basis for the arbitrary cut-off and that the Reliability
Standard be revised to allow the generally-accepted use of estimated or
typical manufacturer data where unit-specific data is impractical to
obtain. TVA agrees that the 1990 cut-off date is unnecessary.
    1196. In contrast to those who support rejecting the 1990 cut-off
requirement, FirstEnergy states that unit-specific data should be
required for all units installed after 1990. EEI confirms that unit-
specific information should be available for most units placed in
service since 1990.
ii. Commission Determination
    1197. The Commission will not approve or remand MOD-013-1 until the
ERO submits additional information. The Commission directs the ERO to
modify MOD-013-1 through the Reliability Standards development process
as discussed below.
    1198. We agree with many commenters and direct the ERO to modify
the Reliability Standard to permit entities to estimate dynamics data
if they are unable to obtain unit-specific data for any reason, not
just for units constructed prior to 1990. Achieving the most accurate
possible picture of the dynamic behavior of the Interconnection
requires the use of actual data. We disagree with FirstEnergy and EEI
and reject the 1990 cut-off date, because the age of the unit alone may
not be the only reason why unit-specific data is unavailable. We agree
with the Small Entities Forum that the Reliability Standard should
include Requirements that such estimates be based on sound engineering
principles and be subject to technical review and approval of any
estimates at the regional level. That said, the Commission directs that
this Reliability Standard be modified to require that the results of
these dynamics models be compared with actual disturbance data to
verify the accuracy of the models.
    1199. With respect to small units installed in wind farms, we agree
with MidAmerican that data for one unit to represent all identical
units at wind farms is acceptable. The Commission understands that this
is the current approach with any generator that is manufactured in
quantity such as multiple generators used in combined cycle plants.
    1200. We adopt our NOPR proposal and direct the ERO to expand the
applicability section in this Reliability Standard to include planning
authorities because they are the entities responsible for the
coordination and integration of transmission facilities and resource
plans, as well as one of the entities responsible for the integrity and
consistency of the data.
    1201. Accordingly, the Commission neither accepts nor remands MOD-
013-1 until the ERO submits additional information. Because the
regional procedures have not been submitted to the Commission, it is
not possible to determine at this time whether MOD-013-1 satisfies the
statutory requirement that a proposed Reliability Standard be ``just,
reasonable, not unduly discriminatory or preferential, and in the
public interest.'' In the interim, compliance with MOD-013-1 should
continue on a voluntary basis, and the Commission considers compliance
with the Reliability Standard to be a matter of good utility practice.
Although the Commission does not approve or remand MOD-013-1, we direct
the ERO to modify it through the Reliability Standards development
process to: (1) Permit entities to estimate dynamics data if they are
unable to obtain unit specific data for any reason; (2) require
verification of the dynamic models with actual disturbance data and (3)
expand the applicability section to include the planning authority,
transmission operator and transmission planner. As discussed above in
MOD-012-0, we direct the ERO to develop a Work Plan that will
facilitate ongoing collection of the dynamics system modeling and
simulation data specified in MOD-013-1, and submit a compliance filing
containing this Work Plan to the Commission.
p. Development of Steady-State System Models (MOD-014-0)
    1202. MOD-014-0 requires the regional reliability organizations
within each Interconnection to coordinate and jointly develop and
maintain a library of solved Interconnection-specific steady-state
models. These models are to include near- and long-term planning
horizons representing system conditions for various demand levels. The
models are to be updated annually.
    1203. In the NOPR, the Commission identified MOD-014-0 as a fill-
in-the-blank standard that requires the regional reliability
organizations within an Interconnection to develop, coordinate and
maintain a library of solved Interconnection-specific steady-state
models. The NOPR stated that because the regional procedures had not
been submitted, the Commission would not propose to approve or remand
MOD-014-0 until the ERO submits the additional information. In
addition, in the NOPR the Commission stated its belief that the
Reliability Standard should be modified to include a requirement to
verify that steady-state models are accurate.
    1204. In the NOPR, the Commission expressed concern about creating
a duplicate effort if both the transmission owner and the regional
reliability organization separately develop the steady-state base cases
required for the FERC Form 715 filing and for MOD-014-0. The NOPR
suggested that the Reliability Standard contain a requirement
specifying the time period and planning years be identical to those
found in FERC Form 715.\356\ Further, the Commission requested comments
on any incompatibility between requirements under FERC Form 715 and
MOD-014-0.
---------------------------------------------------------------------------

    \356\ FERC Form 715 is available at http://www.ferc.gov/docs-filing/eforms.asp#715
.

---------------------------------------------------------------------------

i. Comments
    1205. APPA agrees with the Commission that MOD-014-0, a fill-in-
the-blank standard, is not sufficient as currently drafted, and should
not be approved as a mandatory Reliability Standard until NERC and the
regional reliability organizations/Regional Entities develop the
necessary regional methodologies and the Commission approves them.
    1206. NRC suggests that a periodic verification against field data
needs to be included in this Reliability Standard.
    1207. Regarding the Commission's request for comments on any
incompatibility between requirements under FERC Form 715 and MOD-014-0,
International Transmission states that the language in MOD-014-0 would
allow the regional reliability organization and the transmission owner
to develop separate base cases. International Transmission notes that
its experience with current practice suggests, however, that this is
not a significant concern. Transmission owners now develop the
information for inclusion in a regional base case, and the regional
base case is rolled up into a FERC Form 715 filing by a regional
entity. International Transmission expects that this process would
continue in the future.

    1208. MISO believes that FERC should revisit the need for
transmission owners to have base case information available for
replication. MISO states that the current Interconnection trend is for
transmission owners to work together more closely in developing large
assessments based on a large model, and that these large assessments
are better guides to the overall capability of the transmission grid to
move power. MISO believes that these assessments should be filed as
part of FERC Form 715.
    1209. Although Northern Indiana does not see any duplication or
incompatibility with FERC Form 715, Northern Indiana is concerned that
the proposed Reliability Standard envisions the use of steady-state
models and benchmarking for long-term planning. Northern Indiana
believes that benchmarking of planning models should be directed
towards validation of line constraints and general comparison of
modeled to actual load levels. Northern Indiana suggests that this
could be accomplished through validation processes that would first
evaluate the data used to model the transformers and the lines and
determine that such data is correct, and then compare the loads in
total against the actual loads, followed by an examination of
individual load points on a system.
ii. Commission Determination
    1210. The Commission will not approve or remand MOD-014-0 until the
ERO submits additional information. Because the regional procedures
have not been submitted to the Commission, it is not possible to
determine at this time whether MOD-014-0 satisfies the statutory
requirement that a proposed Reliability Standard be ``just, reasonable,
not unduly discriminatory or preferential, and in the public
interest.'' The Commission directs the ERO to modify MOD-014-0 as
discussed below.
    1211. We maintain our position set forth in the NOPR that analysis
of the Interconnection system behavior requires the use of accurate
steady-state models. Therefore, we direct the ERO to modify the
Reliability Standard to include a requirement that the models be
validated against actual system responses. We understand that NERC is
incorporating recommendations from the Blackout Report \357\ and
developing models for the Eastern Interconnection.
---------------------------------------------------------------------------

    \357\ Recommendation Number 24 of the Blackout Report at 160.
---------------------------------------------------------------------------

    1212. Further, the maximum discrepancy between the model results
and the actual system response should be specified in the Reliability
Standard. The Commission believes that the maximum discrepancy between
the actual system performance and the model should be small enough that
decisions made by planning entities based on output from the model
would be consistent with the decisions of operating entities based on
actual system response. We direct the ERO to modify MOD-014-0 through
the Reliability Standards development process to require that actual
system events be simulated and if the model output is not within the
accuracy required, the model shall be modified to achieve the necessary
accuracy.
    1213. We believe that steady-state model validation should not be
interrupted while MOD-014-0 is being modified. The lack of accurate
models needed for the simulations would halt regional reliability
assessment processes and hinder planners from accurately predicting
future system conditions, which would be detrimental to system
reliability. We therefore direct the ERO to use its authority pursuant
to Sec.  39.2(d) of our regulations to require users, owners and
operators to provide the validated models to regional reliability
organizations. We direct the ERO to develop a Work Plan that will
facilitate ongoing validation of steady-state models and submit a
compliance filing containing the Work Plan with the Commission.
    1214. Consistent with many commenters' responses, we find changes
to FERC Form 715 are not necessary at this time, because there is no
conflict between data gathering and model construction with the FERC
Form 715 process.
    1215. The Commission neither accepts nor remands MOD-014-0. Because
the regional procedures have not been submitted to the Commission, it
is not possible to determine at this time whether MOD-014-0 satisfies
the statutory requirement that a proposed Reliability Standard be
``just, reasonable, not unduly discriminatory or preferential, and in
the public interest.'' In the interim, compliance with MOD-014-0 should
continue on a voluntary basis, and the Commission considers compliance
with the Reliability Standard to be a matter of good utility practice.
We direct the ERO to: (1) modify the Reliability Standard through the
Reliability Standards development process to require actual system
events be simulated and model output validated against actual system
responses and (2) develop a Work Plan and submit a compliance filing
that will enable validation of the steady-state models while MOD-014-0
is being modified.
q. Development of Dynamics System Models (MOD-015-0)
    1216. MOD-015-0 requires the regional reliability organizations
within each Interconnection to coordinate and jointly develop and
maintain a library of initialized (with no faults and disturbances)
Interconnection-specific dynamics system models. These models represent
near-term years and the years chosen from the longer-term planning
horizon.
    1217. In the NOPR, the Commission identified MOD-015-0 as a fill-
in-the-blank standard that requires the regional reliability
organizations within an Interconnection to develop, coordinate and
maintain a library of initialized Interconnection-specific dynamics
system models. The NOPR stated that because the regional procedures had
not been submitted, the Commission would not propose to approve or
remand MOD-015-0 until the ERO submits the additional information. In
addition, the Commission stated that MOD-015-0 should include a
requirement to verify accuracy of dynamics system models.
i. Comments
    1218. APPA agrees that MOD-015-0 is a fill-in-the-blank standard,
is not sufficient as currently drafted and should not be approved as a
mandatory reliability standard until NERC and the regional reliability
organizations/Regional Entities develop the necessary regional
methodologies and the Commission approves them.
    1219. EEI agrees with the Commission's proposal that a new
requirement for verification of the accuracy of dynamics system models
should be a part of this Reliability Standard. In addition, EEI states
that the validation of models is a valid concern, but that any
requirement in this area should be carefully considered, and that any
requirement should be related to using the models to replicate events
that occur on the system instead of developing separate testing
procedures to verify the models. EEI believes that it would not be
reasonable to subject generation units to artificial disturbances to
validate the models. NRC recommends periodic verification against field
data. APPA notes that if NERC modifies MOD-015-0 as APPA anticipates, a
requirement to verify the accuracy of the dynamics system model would
be included and the Regional Entity would be the compliance monitor.

ii. Commission Determination
    1220. The Commission will not approve or remand MOD-015-0 until the
ERO submits additional information. Because the regional procedures
have not been submitted to the Commission, it is not possible to
determine at this time whether MOD-015-0 satisfies the statutory
requirement that a proposed Reliability Standard be ``just, reasonable,
not unduly discriminatory or preferential, and in the public
interest.'' The Commission directs the ERO to modify MOD-015-0 through
the Reliability Standards development process as discussed below.
    1221. We maintain our position set forth in the NOPR that the
analysis of Interconnection system behavior requires the use of
accurate dynamics system models. Therefore, we direct the ERO to modify
the Reliability Standard to include a requirement that the models be
validated against actual system responses. We agree with EEI and NRC
and confirm our position that a requirement to verify that dynamics
system models are accurate should be a part of this Reliability
Standard. We agree with EEI that this new requirement should be related
to using the models to replicate events that occur on the system
instead of developing separate testing procedures to verify the models.
We direct the ERO to modify the standard to require actual system
events be simulated and dynamics system model output be validated
against actual system responses.
    1222. We believe that dynamics system model validation should not
be interrupted while MOD-015-0 is in the modification process. The lack
of accurate models needed for the simulations would halt regional
reliability assessment processes and hinder planners from accurately
predicting future system conditions, which would be detrimental to
system reliability. We therefore direct the ERO to use its authority
pursuant to Sec.  39.2(d) of our regulations to require users, owners
and operators to provide to the Regional Entity the validated dynamics
system models while MOD-015-0 is being modified. We require the ERO to
develop a Work Plan that will enable continual validation of dynamics
system models and submit a compliance filing with the Commission.
    1223. The Commission neither accepts nor remands MOD-015-0 until
the ERO submits additional information. Because the regional procedures
have not been submitted to the Commission, it is not possible to
determine at this time whether MOD-015-0 satisfies the statutory
requirement that a proposed Reliability Standard be ``just, reasonable,
not unduly discriminatory or preferential, and in the public
interest.'' In the interim, compliance with MOD-015-0 should continue
on a voluntary basis, and the Commission considers compliance with the
Reliability Standard to be a matter of good utility practice. We direct
the ERO to: (1) Modify the Reliability Standard through the Reliability
Standards development process to require verification of the accuracy
of dynamics system models and (2) develop a Work Plan and submit a
compliance filing that will facilitate ongoing verification of the
accuracy of dynamics system models while MOD-015-0 is being modified.
r. Documentation of Data Reporting Requirements for Actual and Forecast
Demands, Net Energy for Load and Controllable Demand-Side Management
(MOD-016-1)
    1224. The purpose of MOD-016-1 is to ensure that past and
forecasted demand data is available for validation of past events and
future system assessments. MOD-016-1 requires the planning authority
and the regional reliability organization to have documentation
identifying the scope and details of the actual and forecast demand and
load data, and controllable DSM data to be reported for system modeling
and reliability analysis.
    1225. In the NOPR, the Commission proposed to approve Reliability
Standard MOD-016-1 as mandatory and enforceable. In addition, the
Commission proposed to direct NERC to submit a modification to MOD-016-
1 that expands the applicability section to include the transmission
planner.
i. Comments
    1226. APPA agrees that MOD-016-1 is sufficient for approval as a
mandatory and enforceable reliability standard.
    1227. In contrast, ISO/RTO Council and ISO-NE do not support
adoption of this standard because it is contingent on standards that
are pending approval by the Commission based on their characterization
as applying only to regional reliability organizations, or because they
have been categorized as fill-in-the-blank standards.\358\ ISO/RTO
Council and ISO-NE agree that as a result, MOD-016-1 cannot be
effectively implemented.
---------------------------------------------------------------------------

    \358\ TPL-005-0, TPL-006-0, MOD-011-0, MOD-013-0, MOD-014-0 and
MOD-015-0.
---------------------------------------------------------------------------

    1228. APPA and FirstEnergy agree with the Commission's proposal to
direct NERC to add the transmission planner function to the
applicability section of the standard, although they argue that NERC,
as the standards-setting entity, should make the decision.
    1229. TAPS does not oppose the proposed applicability of MOD-016-1,
but opposes regional interpretations that apply the standard more
broadly. TAPS criticizes SERC's supplement to MOD-016-1 that makes the
standard applicable to LSEs, even though LSEs do not have the ability
to identify the scope and details of the data required to be reported
for system modeling and reliability analyses. TAPS contends that there
are no physical differences that make SERC LSEs more capable in this
regard than LSEs in other regions. TAPS recommends that the Commission
clarify that it expects standards to be applied in a consistent and
uniform manner as written, and will look closely at regional variations
not justified by physical differences.
    1230. In contrast to APPA, FirstEnergy and TAPS, EEI believes that
the standard assigns appropriate responsibility, and that the
transmission planner should not be added to the applicability section
of this standard. According to EEI, the transmission planner has no
specific responsibilities for ensuring data integrity in day-to-day
practice. EEI understands that data integrity falls within the daily
responsibilities of data management functions, such as metering. EEI
states that the NERC Functional Model does not describe technical
functions at this level of detail. EEI notes, as it also notes in its
comments on the TPL standards, that load-related DSM data of the type
and specificity stated in the NOPR, such as load control of customer-
owned appliances, is related to distribution system and operations
planning, and not to transmission system planning.
ii. Commission Determination
    1231. The Commission approves MOD-016-1 as mandatory and
enforceable. In addition, the Commission directs the ERO to modify MOD-
016-1 as discussed below.
    1232. As an initial matter, we disagree that MOD-016-1 cannot be
implemented until other unapproved standards are modified. As
previously stated, we are requiring the ERO to provide a Work Plan and
compliance filing regarding collection of information specified under
standards that are deferred, and believe there should be no
difficulties complying with this Reliability Standard. We reiterate
that continual collection of data is necessary to maintain system
reliability, and approval of MOD-016-1 will help to achieve this
objective.
    1233. Supported by many commenters, the Commission directs

the ERO to modify MOD-016-1 and expand the applicability section to
include the transmission planner, on the basis that under the NERC
Functional Model the transmission planner is responsible for collecting
system modeling data, including actual and forecast load, to evaluate
transmission expansion plans. We disagree with EEI that this
Reliability Standard should not be applied to the transmission planner
because load-related data for controllable DSM is not only needed for
distribution and transmission operations, but is also necessary for the
transmission planner to take controllable DSM into account in planning
the transmission system. Requirement R1.1 relates to data submittal,
and requires data to be consistent with that supplied for the TPL-005
and TPL-006 standards, which clearly apply to transmission planners. We
approve the ERO's definition in the glossary of DSM as ``all activities
or programs undertaken by a Load-Serving Entity or its customers to
influence the amount or timing of electricity they use.'' Only
activities or programs that meet the ERO definition, with the
modification directed below, may be treated as DSM for purposes of the
Reliability Standards. Recognizing the potential role that industrial
customers who do not take service through an LSE and load aggregators,
for example, may play in meeting the Reliability Standards, we direct
the ERO to modify the definition of DSM. Specifically, we direct the
ERO to add to its definition of DSM ``any other entities'' that
undertake activities or programs to influence the amount or timing of
electricity they use without violating other Reliability Standard
Requirement.
    1234. In response to TAPS's criticism of SERC's desire to expand
its regional standards relative to actual and forecast load to include
LSEs, we clarify that we can only act on the standards before us. We do
not make a decision on SERC's standards in this rule. We therefore
recommend that TAPS raise this issue in the Reliability Standards
development process.
    1235. The Commission approves Reliability Standard MOD-016-1 as
mandatory and enforceable and directs the ERO to develop a modification
to MOD-016-0 through the Reliability Standards development process to
include the transmission planner in the applicability section.
s. Aggregated Actual and Forecast Demands and Net Energy for Load (MOD-
017-0)
    1236. The purpose of MOD-017-0 is to ensure that past and
forecasted demand data is available for past event validation and
future system assessment. MOD-017-0 requires LSEs, planning authorities
and resource planners to annually provide aggregated information on:
(1) Integrated hourly demands; (2) actual monthly and annual peak
demand (MW) and net load energy (GWh) for the prior year; (3) monthly
peak demand forecasts and net load energy for the next two years and
(4) annual peak demand forecasts (summer and winter) and annual net
load energy for at least five and up to ten years into the future.
    1237. In the NOPR, the Commission proposed to approve Reliability
Standard MOD-017-0 as mandatory and enforceable. In addition, the
Commission proposed to direct NERC to submit a modification to MOD-017-
0 that includes new requirements for: (1) Reporting of temperature and
humidity along with peak loads and (2) reporting of the accuracy, error
and bias of load forecasts compared to actual loads while taking
temperature and humidity variations into account.
i. Comments
    1238. APPA agrees that the Commission should approve MOD-017-0 as
mandatory and enforceable.
    1239. In contrast to APPA, ISO-NE does not support approval of this
standard because MOD-017-0 depends on MOD-016-0, which further depends
on various unapproved standards. ISO-NE believes that this makes MOD-
017-0 dependent on unapproved standards, and that consequently, MOD-
017-0 cannot be effectively implemented. Similarly, ISO/RTO Council
states that if the Commission does not approve MOD-016-0, then MOD-017-
0 will refer to an unapproved standard.
    1240. Although MidAmerican does not oppose the Commission's
proposal regarding reporting of temperature and humidity along with
peak loads, it finds it of only limited value. MidAmerican notes that
there are typically other explanatory variables, such as economic
variables, that are needed to understand the relationship between
system load and temperature and humidity. In addition, the relationship
and the importance of temperatures are different for every utility,
which limits the effectiveness of standardization. FirstEnergy suggests
that NERC should allow for a transition period for entities that
currently do not track temperature and humidity along with peak load.
    1241. Xcel states that in many areas of the country, humidity is
not a weather-indicator for peak load. Xcel therefore suggests that
instead of including a reporting requirement for humidity, the standard
be revised to include a more generic term, such as ``peak producing
weather conditions.'' Alcoa requests that the Commission clarify that
these requirements would only apply to load that varies with
temperature and humidity.\359\
---------------------------------------------------------------------------

    \359\ Alcoa states that because its smelting load (the vast
majority of its load) does not vary in accordance with temperature
and humidity, comparing Alcoa's load forecasts to actual loads
taking this information into account would be burdensome without
being useful.
---------------------------------------------------------------------------

    1242. Regarding the Commission's proposal for reporting of the
accuracy, error and bias of load forecasts compared to actual loads
while taking temperature and humidity variations into account, APPA
disagrees that the Commission should direct NERC to modify MOD-017-0 to
include these requirements. APPA argues that requiring the type and
granularity of forecast information and data the Commission proposes
would not necessarily increase the reliability of load forecasts. APPA
believes that it should be up to NERC, as the expert standards-setting
entity, to decide whether such information would yield enough useful
data to make it worth mandating.
    1243. TAPS is concerned that the NOPR's recommendation for
reporting the accuracy, error and bias of load forecasts compared to
actual loads may be interpreted to mean that measuring compliance is a
function of forecast accuracy. TAPS contends that reliance on
percentage-based deviations as a measurement of compliance is
inappropriate when applied to very small entities because an error that
in absolute terms is too small to affect the Bulk-Power System might be
a significant percentage of the entity's load.
    1244. EEI notes that the direction of the NOPR proposal seems to
suggest an expansion of the current reporting processes required under
the Energy Information Administration section 411 process. EEI suggests
that such a proposal should consider whether the section 411 process
itself requires change or provides for an adequate level of reporting,
and the extent to which an explicit NERC process requirement could
distract or confuse industry participants.
    1245. FirstEnergy states that the transmission planner should be
added to the list of applicable entities for this standard. FirstEnergy
also states that it may be reasonable to interpret or apply this
Reliability Standard in a manner to permit an affected entity that is a
subsidiary in a utility holding company corporate structure to satisfy
its

reporting requirements by means of a corporate affiliate. Adopting this
interpretation or application would promote efficiency and decrease
confusion in circumstances where several utility subsidiaries in the
same corporate family are subject to this Reliability Standard.
    1246. MISO recommends that the Commission direct NERC to change the
requirement of this standard so that aggregated actual hourly demand
data (at the balancing authority level) are to be provided within 30
calendar days of a request from NERC. MISO believes that load
aggregated at this level should be sufficient for the modeling
activities associated with system reliability. MISO understands that
hourly data is collected by those utilities that have balancing
authority responsibilities, and that these utilities can report
aggregated hourly loads for their responsibility area within 30 days.
MISO notes that some balancing authority utilities provide energy
services to smaller municipal or distribution cooperative utilities
where the metering system records only the peak demand and total energy
supplied over approximately 30 days. MISO cautions that the balancing
authority will usually have hourly data for demand and energy within a
segment of the network, but may have no hourly metering on a specific
customer served by that segment.
ii. Commission Determination
    1247. The Commission approves MOD-017-0 as mandatory and
enforceable. In addition, the Commission directs the ERO to modify MOD-
017-0 as discussed below.
    1248. As an initial matter, we disagree that MOD-017-0 cannot be
implemented because it is dependent on MOD-016-0, which further depends
on various unapproved standards. As previously stated, we direct the
ERO to provide a Work Plan and compliance filing regarding the
collection of information specified under standards that are deferred,
and believe there should be no difficulty complying with this
Reliability Standard. We reiterate that ongoing collection of data is
necessary to maintain system reliability, and approval of MOD-017-0
will help achieve this goal.
    1249. As a general matter, the Commission is required to insure
that the Reliability Standards are sufficient to adequately protect
Bulk-Power System reliability.\360\ One of the main drivers in
achieving Reliable Operation is to accurately predict the firm
transactions and native load that must be served. Understanding the
accuracy, error and bias of the forecast and taking action to minimize
them would improve the Reliability Standards and achieve the goal.
---------------------------------------------------------------------------

    \360\ Order No. 672 at P 329.
---------------------------------------------------------------------------

    1250. The Commission also directs the ERO to modify the Reliability
Standad to require reporting of temperature and humidity along with
peak load because actual load must be weather normalized for meaningful
comparison with forecasted values.\361\ In response to MidAmerican's
observation that it sees little value in collecting this data, we
believe that collecting it will allow all load data to be weather-
normalized, which will provide greater confidence when comparing data
accuracy, which ultimately will enhance reliability. As a result, we
reject Xcel's proposal that the standard be revised to include only the
generic term ``peak producing weather conditions'' because it is too
generic for a mandatory Reliability Standard.
---------------------------------------------------------------------------

    \361\ See Brattle Group Report on PJM Load Forecast Model,
available at http://www.pjm.com/planning/res-adequacy/load-forecast.html
.

---------------------------------------------------------------------------

    1251. We also reject Alcoa's proposal that the reporting of
temperature and humidity along with peak loads should apply only to
load that varies with temperature and humidity because it essentially
is a request for an exemption from the requirements of the Reliability
Standard and should therefore be directed to the ERO as part of the
Reliability Standards development process. We agree, however, with APPA
that certain types of load are not sensitive to temperature and
humidity. We therefore find that the ERO should address Alcoa's
concerns in its Reliability Standards development process.
    1252. The Commission adopts the NOPR proposal directing the ERO to
modify the Reliability Standard to require reporting of the accuracy,
error and bias of load forecasts compared to actual loads with due
regard to temperature and humidity variations. This requirement will
measure the closeness of the load forecast to the actual value. We
understand that load forecasting is a primary factor in achieving
Reliable Operation. Underestimating load growth can result in
insufficient or inadequate generation and transmission facilities,
causing unreliability in real-time operations. Measuring the accuracy,
error and bias of load forecasts is important information for system
planners to include in their studies, and also improves load forecasts
themselves.
    1253. The Commission agrees with APPA that accuracy, error and bias
of load forecasts alone will not increase the reliability of load
forecasts, and, as a result, will not affect system reliability.
Understanding of the differences without action based on that
understanding would not change anything. Therefore, we direct the ERO
to add a Requirement that addresses correcting forecasts based on prior
inaccuracies, errors and bias.
    1254. Regarding TAPS's concern that accuracy of reporting may be
used as a compliance Measure, we clarify that the compliance Measures
for this Reliability Standard do not measure accuracy as a compliance
Measure. Any change in the Measures would be arrived at in the
Reliability Standards development process.
    1255. The Commission acknowledges EEI's concern that a requirement
for additional information may impose an expansion of existing Energy
Information Administration section 411 reporting requirements.\362\ We
believe, however, that the ERO can ensure that the additional reporting
of temperature and humidity along with peak loads does not conflict
with or jeopardize the Energy Information Administration section 411
reporting process.
---------------------------------------------------------------------------

    \362\ Form EIA-411, ``Coordinated Bulk Power Supply Program
Report'' collects information about regional electric supply and
demand projections for a five-year advance period as well as
information on the transmission system and supporting facilities.
See http://www.eia.doe.gov/cneaf/electricity/page/forms.html.

---------------------------------------------------------------------------

    1256. We agree with FirstEnergy that transmission planners should
be added as reporting entities, and direct the ERO to modify the
standard accordingly. We agree that in the NERC Functional Model, the
transmission planner is responsible for collecting system modeling data
including actual and forecast demands to evaluate transmission
expansion plans.
    1257. The Commission disagrees in general with MISO's
recommendation to allow some exceptions to the requirement to provide
hourly demand data. However, the metering for some customer classes may
not be designed to provide certain types of data. The Commission
therefore directs the ERO to consider MISO's concerns in the
Reliability Standards development process.
    1258. The Commission approves Reliability Standard MOD-017-0 as
mandatory and enforceable. In addition, the Commission directs the ERO
to develop a modification to MOD-017-0 through the Reliability
Standards development process that includes requirements for: (1)
Reporting of temperature and humidity along with the peak loads; (2)
reporting of accuracy,

error and bias of load forecasts compared to actual loads taking
temperature and humidity variations into account; (3) addressing
methods to correct forecasts to minimize prior inaccuracies, errors and
bias and (4) including the transmission planner in the applicability
section.
t. Treatment of Nonmember Demand Data and Uncertainties in the
Forecasts of Demand and Energy for Load (MOD-018-0)
    1259. The purpose of MOD-018-0 is to ensure that past and
forecasted demand data are available for past event validation and
future system assessment. MOD-018-0 requires LSEs, planning
authorities, transmission planners and resource planners to submit load
data reports that: (1) Indicate whether the demand data includes the
regional reliability organization's non-members' demands and (2)
addresses how assumptions, methods and uncertainties are treated.
    1260. In the NOPR, the Commission proposed to approve MOD-018-0 as
mandatory and enforceable.
i. Comments
    1261. APPA agrees that MOD-018-0 is sufficient for approval as a
mandatory and enforceable reliability standard.
    1262. In contrast to APPA, ISO/RTO Council and ISO-NE view MOD-018-
0 as dependent upon fill-in-the-blank NERC standards, and as such,
argue that the Commission should refrain from approving the Reliability
Standard at this time. ISO-NE states that approval of this standard
would create dependency of MOD-018-0 on other unapproved standards.
Consequently, ISO-NE contends that MOD-018-0 cannot be effectively
implemented.
    1263. TAPS reiterates a similar concern it expressed with regard to
MOD-017-0. TAPS notes that uncertainty in a small entity's forecast is
insignificant. TAPS recommends that load forecast uncertainty should be
addressed at an aggregate level on a regional basis (as is often done
in the establishment of reserve obligations).
ii. Commission Determination
    1264. The Commission approves MOD-018-0 as mandatory and
enforceable.
    1265. As an initial matter, we disagree that MOD-018-0 cannot be
implemented because it is dependent on various unapproved standards. As
previously stated, we direct the ERO to provide a Work Plan and
compliance filing regarding the collection of information specified for
standards that are deferred, and believe there should be no
difficulties complying with this Reliability Standard. We reiterate
that ongoing collection of data is necessary to maintain system
reliability, and approval of MOD-018-0 will help to achieve this goal.
    1266. Regarding TAPS's concern that small entities should not be
required to comply with MOD-018-0 because their forecasts are not
significant for system reliability purposes, the Commission directs the
ERO to address this matter in the Reliability Standards development
process.
u. Reporting of Interruptible Demands and Direct Control Load
Management (MOD-019-0)
    1267. The purpose of MOD-019-0 is to ensure that past and
forecasted demand data is available for past event validation and
future system assessment. The Reliability Standard requires that LSEs,
planning authorities, transmission planners and resource planners
annually provide their forecasts of interruptible demands and direct
control load management to NERC, the regional reliability organization
and other entities as specified in MOD-016-1, Requirement R1. The data
should contain the forecasts for at least five years, and up to ten
years.
    1268. In the NOPR, the Commission proposed to approve Reliability
Standard MOD-019-0 as mandatory and enforceable. In addition, the
Commission proposed to direct NERC to submit a modification to MOD-019-
0 that includes new requirements for reporting of the accuracy, error
and bias of controllable load \363\ forecasts.
---------------------------------------------------------------------------

    \363\ While MOD-019-0 and MOD-020-0 use two separate terms,
interruptible load and direct control load management, the NOPR uses
``controllable load'' to refer to both of them.
---------------------------------------------------------------------------

i. Comments
    1269. APPA agrees that MOD-019-0 should be approved as mandatory
and enforceable. However, APPA states that the proper entity to decide
whether the recommended changes to the standards should be made is
NERC, through Reliability Standards development process.
    1270. The ISO/RTO Council and ISO-NE note that MOD-019-0 is
dependent, through MOD-016, on various unapproved standards.
Consequently, they contend that MOD-019-0 cannot be effectively
implemented.
    1271. APPA proposes that NERC consider modifying MOD-019-0 to
include new requirements for reporting on the accuracy, error and bias
of controllable load forecasts. APPA further believes that NERC should
consider adding requirements that would require resource planners to
analyze differences between actual and forecasted demands for the five
years of actual controllable load required in MOD-019-0 and identify
what corrective actions were taken to improve controllable load
forecasting for the 10-year planning horizon.
    1272. EEI and FirstEnergy state that determining the precise
availability and capability of direct load control is a difficult
management and customer relations exercise, and therefore, this
requirement should not be included in the Reliability Standard. EEI
states that, unlike other technical requirements for generation
resources to be tested for various capabilities and limits under
different types of stresses, there are no similar requirements for load
control equipment. Elsewhere in these comments, EEI supports explicit
recognition that load control should be recognized on the same terms as
generation resources for setting reserve requirements. However, EEI
cautions against imposing requirements to verify load control devices
and interruptible loads, because the practical complexities of
conducting such testing and verification, including customer
notification, the need to plan, manage, and coordinate testing with
critical commercial and industrial customer activities, and the need to
conduct such tests at times of peak load, make this an extremely
difficult operational challenge.
    1273. International Transmission notes that many load control
applications are not individually metered, which means impact can only
be estimated within a LSE's service territory. International
Transmission believes that accurate reporting may not be feasible.
    1274. TAPS raises concern that the Commission's recommendation in
the NOPR may be interpreted to make forecast accuracy a component of
Reliability Standards compliance. TAPS cautions that reliance on
percentage-based deviations as a measurement of compliance is
inappropriate when applied to very small entities because an error that
in absolute terms is too small to affect the Bulk-Power System might be
a significant percentage of the entity's load. The percentage deviation
from a forecasted peak of a small (e.g., 10 MW) entity will almost
always be significantly higher than the percentage deviation of a large
(more than 10,000 MW) entity, but the smaller system's deviation will
have little if any impact on the bulk transmission system. In other
contexts, the Commission has recognized that reliance solely on

percentage deviations as compliance measures can produce discriminatory
results, and has applied MW minimums to minimize the discrimination
that would otherwise result.
ii. Commission Determination
    1275. The Commission approves MOD-019-0 as mandatory and
enforceable. In addition, the Commission directs the ERO to modify MOD-
019-0 as discussed below.
    1276. As an initial matter, we disagree that MOD-019-0 cannot be
implemented because it is dependent on MOD-016-0, which further depends
on various unapproved standards. As previously stated, we direct the
ERO to provide a Work Plan and compliance filing regarding the
collection of information specified under related standards that are
deferred, and believe there should be no difficulties complying with
this Reliability Standard. We reiterate that ongoing collection of data
is necessary to maintain system reliability, and approval of MOD-019-0
will help to achieve this goal. We therefore direct the ERO to use its
authority pursuant to Sec.  39.2(d) of our regulations to require
users, owners and operators to provide to the Regional Entity
information related to forecasts of interruptible demands and direct
control load management.
    1277. The Commission adopts the NOPR proposal directing the ERO to
modify this standard to require reporting of the accuracy, error and
bias of controllable load forecasts. This requirement will enable
planners to get a more reliable picture of the amount of controllable
load that is actually available, therefore allowing planners to conduct
more accurate system reliability assessments. The Commission finds that
controllable load can be as reliable as other resources, and therefore
should also be subject to the same reporting requirements. Although we
recognize that verifying load control devices and interruptible loads
may be complex, we do not believe that it is overly so. Further, we
believe that the ERO, through its Reliability Standards development
process can develop innovative solutions to the Commission's concern.
We also note that EEI is concerned about such testing at times of peak
load. We clarify that we are not requiring the testing to be conducted
at peak load conditions. Consequently, we reject the proposals of EEI,
FirstEnergy and International Transmission to discard the requirement
for reporting of the accuracy, error and bias of controllable load
forecasts.
    1278. We direct the ERO to include APPA's proposal in the
Reliability Standards development process to add a new requirement to
MOD-019-0 that would oblige resource planners to analyze differences
between actual and forecasted demands for the five years of actual
controllable load and identify what corrective actions should be taken
to improve controllable load forecasting for the 10-year planning
horizon.
    1279. Regarding TAPS' concern that reporting accuracy could be used
as a compliance Measure, we clarify that compliance Measures for this
Reliability Standard do not include accuracy as a compliance measure.
Any change in this policy would be arrived at in the ERO Reliability
Standards development process.
    1280. Accordingly, the Commission approves MOD-019-0 as mandatory
and enforceable. In addition, the Commission directs the ERO to develop
a modification to MOD-019-0 through the Reliability Standards
development process to require: (1) Reporting of the accuracy, error
and bias of controllable load forecasts and (2) analyzing differences
between actual and forecasted demands for the five years of actual
controllable load and identify what corrective actions should be taken
to improve controllable load forecasting for the 10-year planning
horizon.
v. Providing Interruptible Demand and Direct Control Load Management
Data to System Operators and Reliability Coordinators (MOD-020-0)
    1281. The purpose of MOD-020-0 is to ensure that past and
forecasted demand data are available for validation of past events and
future system assessment. The Reliability Standard requires that each
LSE, planning authority, transmission planner and resource planner
identify its amount of: (1) Interruptible demand and (2) direct control
load management to transmission operators, balancing authorities and
reliability coordinators upon request.
    1282. In the NOPR, the Commission proposed to approve Reliability
Standard MOD-020-0 as mandatory and enforceable. In addition, the
Commission proposed to direct NERC to submit a modification to MOD-020-
0 that includes a new requirement concerning the reporting of the
accuracy, error and bias of controllable load forecasts in its
Reliability Standards development process.
i. Comments
    1283. APPA supports approval of MOD-020-0 as mandatory and
enforceable, as proposed by the Commission. APPA does not oppose NERC's
consideration of possible changes to MOD-020-0 regarding the reporting
of the accuracy, error and bias of controllable load forecasts.
    1284. EEI and FirstEnergy state that for practical reasons,
determining the precise availability and capability of direct load
control is a difficult management and customer relations exercise.
Unlike other technical requirements for generation resources to be
tested for various capabilities and limits under different types of
stresses, there are no similar requirements for load control equipment.
The practical complexities of conducting such testing and verification,
including customer notification, the need to plan, manage and
coordinate testing with critical commercial and industrial customer
activities, and the need to conduct such tests at times of peak load
make this an extremely difficult operational challenge.
    1285. LPPC opposes the Commission's proposal for modification to
report the accuracy of load forecasts. LPPC points out that load
reduction forecasts are imprecise by nature, and, consequently, some
utilities do not undertake them. LPPC also notes that interruptible
loads are often on one-year contracts and, in some regions, instances
of entities actually exercising load reduction are rare; in these
areas, system operators often do not separately forecast interruptible
load reductions, and reporting on the accuracy of forecasts on
interruptible load reductions, even if interruptible load forecasts
were done, is of little value. LPPC states that in other areas, such as
New York, interruptible load reductions are more predictable, because
many large loads have signed interruptible load contracts and have a
history of exercising load reductions. LPPC notes that system operators
in areas similar to New York have sufficient data so that forecasting
for interruptible loads is a useful exercise, and as a result, a
requirement to report on the accuracy of forecasts in these regions
would be of some value, but not elsewhere. Consequently, LPPC
recommends that the requirement should be region-specific and should
only apply to entities that separately forecast interruptible loads.
LPPC further notes that energy efficiency programs are often built into
the larger assumptions in the forecast and are not separately
forecasted.
    1286. TAPS is concerned that the Commission's recommendation in the
NOPR may be interpreted to make forecast accuracy a component of
Reliability Standards compliance.

However, it asserts that reliance on percentage-based deviations as a
measurement of compliance is inappropriate when applied to very small
entities because an error that in absolute terms is too small to affect
the Bulk-Power System might be a significant percentage of the entity's
load. The percentage deviation from a forecasted peak of a small (e.g.,
10 MW) entity will almost always be significantly higher than the
percentage deviation of a large (more than 10,000 MW) entity, but the
smaller system's deviation will have little if any impact on the bulk
transmission system. In other contexts, the Commission has recognized
that reliance solely on percentage deviations as a compliance measure
can produce discriminatory results, and has applied MW minimums to
minimize the discrimination that would otherwise result.
ii. Commission Determination
    1287. The Commission approves MOD-020-0 as mandatory and
enforceable. In addition, the Commission directs the ERO to modify MOD-
020-0 as discussed below.
    1288. We adopt the proposal to direct the addition of a requirement
for reporting of the accuracy, error and bias of controllable load
forecasts because we believe that reporting of this information will
provide applicable entities with advanced knowledge about the exact
amount of available controllable load, which will improve the accuracy
of system reliability assessments. The Commission finds that
controllable load in some cases may be as reliable as other resources
and therefore must also be subject to the same reporting requirements.
We recognize that determining the precise availability and capability
of direct load control is a difficult management and customer relations
exercise, but we do not believe that it will be overly so. Further, we
believe that the ERO, through its Reliability Standards development
process can develop innovative solutions to the Commission's concern.
Regarding LPPC's suggestion that this requirement should be region-
specific and should only apply to entities that separately forecast
interruptible loads, we note that if a region does not forecast
interruptible loads, this Reliability Standard does not apply.
    1289. Regarding TAPS' concern that forecast accuracy may be
interpreted as a component of Reliability Standards compliance, we
clarify that compliance Measures for this Reliability Standard do not
measure accuracy as a compliance measure. Any change in this policy
would be arrived at in the ERO Reliability Standards development
process.
    1290. The Commission approves Reliability Standard MOD-020-0 as
mandatory and enforceable and directs the ERO to develop a modification
to MOD-020-0 through the Reliability Standards development process to
require reporting of the accuracy, error and bias of controllable load
forecasts.
w. Documentation of the Accounting Methodology for the Effects of
Controllable Demand-Side Management in Demand and Energy Forecasts
(MOD-021-0)
    1291. MOD-021-0 requires LSEs, transmission planners and resource
planners to clearly document how each addresses the demand and energy
effects of DSM programs. The standard also requires an applicable
entity to include information detailing how DSM measures are addressed
in the forecasts of its peak demand and annual net energy for load in
the data reporting procedures of MOD-016-0, Requirement R1.
    1292. In the NOPR, the Commission proposed to approve Reliability
Standard MOD-021-0 as mandatory and enforceable. In addition, the
Commission proposed to direct NERC to submit a modification to MOD-021-
0 that: (1) Includes a requirement standardizing principles on
reporting and validation of DSM program information and (2) modifies
the title and purpose statement to remove the word ``controllable.''
i. Comments
    1293. APPA supports the Commission's approval of MOD-021-0 as
mandatory and enforceable.
    1294. In contrast, ISO-NE and ISO/RTO Council oppose adoption of
this standard by the Commission. ISO-NE argues that the LSE,
transmission planner and resource planner should each include
information regarding how DSM measures are addressed in the forecasts
of its peak demand and annual net energy for load in the data reporting
procedures of MOD-016-0 R1. Therefore, they contend that, because MOD-
016-0 is dependent on various unapproved Reliability Standards, MOD-
021-0 is also dependent on unapproved Reliability Standards.
Consequently, ISO-NE contends that MOD-021-0 cannot be effectively
implemented.
    1295. FirstEnergy and SMA support the Commission's proposal to
require consistent and uniform methods for reporting and validating
demand-side information. SMA notes that this will provide more
consistent and uniform evaluation of demand response data to facilitate
system operator confidence in relying on such resources for various
reliability purposes. In addition, APPA believes that NERC should
consider adding requirements to MOD-021-0 that would provide
information to allow resource planners to analyze the causes of
differences between actual and forecasted demands, and to identify any
corrective actions that should be taken to improve forecasted demand
responses for future forecasts. APPA believes that all of these
proposals should be submitted to NERC as the standards-setting body
with technical expertise, and vetted through its Reliability Standards
development process, rather than being imposed by Commission fiat.
    1296. FirstEnergy adds that MOD-019-0, MOD-020-0 and MOD-021-0
should be combined because they all address load forecast inputs, and
that combining these standards will eliminate any inconsistencies and
make compliance easier and more efficient.
ii. Commission Determination
    1297. The Commission approves MOD-021-0 as mandatory and
enforceable. In addition, the Commission directs the ERO to develop a
modification to MOD-021-0 through the Reliability Standards development
process as discussed below.
    1298. As an initial matter, we disagree that MOD-021-0 cannot be
implemented because it is based on MOD-016-0, and through it on various
unapproved standards, which creates an implementation problem. As
previously stated, we direct the ERO to provide a Work Plan and
compliance filing regarding collection of information specified under
related standards that are deferred, and believe there should be no
difficulty complying with this Reliability Standard. We reiterate that
ongoing collection of data is necessary to maintain system reliability,
and approval of MOD-21-0 will help to achieve this goal. Therefore, we
direct the ERO to use its authority pursuant to Sec.  39.2(d) of our
regulations to require users, owners and operators to provide to the
Regional Entity the information required by this Reliability Standard.
    1299. We agree with FirstEnergy and SMA that standardization of
principles on reporting and validating DSM program information will
provide consistent and uniform evaluation of demand response to
facilitate system operator confidence in relying on such resources,
which will further increase accuracy of transmission system reliability
assessment and consequently

enhance overall reliability. We direct the ERO to modify this
Reliability Standard to allow resource planners to analyze the causes
of differences between actual and forecasted demands, and to identify
any corrective actions that should be taken to improve forecasted
demand responses for future forecasts. Therefore, we adopt the NOPR
proposal and direct the ERO to modify MOD-021-0 by adding a requirement
for standardization of principles on reporting and validating DSM
program information.
    1300. With respect to FirstEnergy's suggestion to combine MOD-019-
0, MOD-020-0 and MOD-021-0, we understand that the ERO intends to
consolidate Reliability Standards and encourage FirstEnergy to make its
suggestion in the Reliability Standards development process.
    1301. The Commission directs the ERO to modify the title and
purpose statement to remove the word ``controllable.'' We note that no
commenter disagrees.
    1302. The Commission approves Reliability Standard MOD-021-0 as
mandatory and enforceable. We direct the ERO to develop a modification
to MOD-021-0 through the Reliability Standards development process to
(1) add a Requirement standardizing principles on reporting and
validation of DSM program information; (2) allow resource planners to
analyze the causes of differences between actual and forecasted
demands, and to identify any corrective actions that should be taken to
improve forecasted demand responses for future forecasts and (3) modify
the title and purpose statement to remove the word ``controllable.''
x. Verification of Generator Gross and Net Real Power Capability (MOD-
024-1)
    1303. The purpose of MOD-024-1 is to ensure that accurate
information on generation gross and net real power capability is used
for reliability assessments. The Reliability Standard requires the
regional reliability organization to establish and maintain procedures
to address verification of generator gross and net real power
capability. It also requires a generator owner to follow its regional
reliability organization's procedure for verifying and reporting gross
and net real power generating capability.
    1304. In the NOPR, the Commission identified MOD-024-1 as a fill-
in-the-blank standard that requires the regional reliability
organization to establish and maintain procedures to address
verification of generator gross and net real power capability. The
Commission stated that because the regional procedures had not been
submitted, it would not propose to approve or remand MOD-024-1 until
the ERO submits the additional information. In addition, the Commission
expressed concern that the Reliability Standard is not sufficiently
clear because it does not define test conditions, e.g., ambient
temperature, river water temperature or methodologies for calculating
de-rating factors for conditions such as higher ambient temperatures
than the test temperature. Further, the NOPR stated that Requirement R2
provides that the ``regional reliability organization shall provide
generator gross and net real power capability verification within 30
calendar days of approval'' and noted that it is not clear what
approval is required and when the 30-day period starts.
i. Comments
    1305. APPA agrees that MOD-024-1 is a fill-in-the-blank standard,
is not sufficient as currently drafted, and should not be approved as a
mandatory Reliability Standard until NERC and the regional reliability
organizations/Regional Entities develop the necessary regional
methodologies and the Commission approves them.
    1306. APPA also states that the results of field-testing will
enable NERC to refine this Reliability Standard in an appropriate
manner. APPA further believes that NERC should consider modifying this
Reliability Standard to provide requirements for this information on an
Interconnection-wide basis, in the same manner that IRO-006-2 sets the
requirement for transmission loading relief in each Interconnection.
    1307. Northern Indiana urges the Commission to reconsider the
proposed changes at this time in favor of continuation of the
currently-effective Reliability Standard. Northern Indiana states that
the NOPR's suggestion that there should be greater specificity and
definition of test conditions could potentially create reliability
issues, rather than protect against them. Northern Indiana explains
that certain types of testing, and their preparation, can be
accomplished more quickly than others, with test duration varying from
several minutes to several days.\364\ The problem is compounded if a
test takes some time to complete, and all neighboring generating owners
were required to comply at the same time. The end result would be a
lack of regulating capability in a region.
---------------------------------------------------------------------------

    \364\ Northern Indiana states that the longer the duration, the
more stressed the units--and the system--during these testing
intervals. For example, Commission staff recommends the use of
ambient air temperature and river water temperature as triggering
tests to verify generator gross and net real power capability.
However, temperature-driven test triggers would result in several
neighboring systems in the same region undergoing tests at the same
time in order to meet the test criteria. For example, a temperature
trigger of 90 degrees Fahrenheit for a net demonstrated capacity
test could result in all neighboring generating owners taking their
units off of automatic generator control to reach maximum net
demonstrated capacity for the test. By taking units off automatic
generator control, the generating owners' regulating capabilities
are lost.
---------------------------------------------------------------------------

    1308. Constellation encourages the Commission and NERC to take
extra care in distinguishing between those requirements in each
Reliability Standard that are core requirements as opposed to
supporting information, explanatory statements or administrative
processes. For example, Constellation points out that in MOD-024-1,
NERC proposes that a verification process be made into a Reliability
Standard with full enforceability. Although Constellation agrees that
the verification process spelled out in this Reliability Standard is
important and should be performed by the industry, the Reliability
Standard, alone, exclusively provides for an administrative process
and, therefore, if not strictly complied with, does not necessarily
foreshadow an immediate, real-time reliability problem on the bulk
electric system. Constellation is concerned that the Levels of Non-
Compliance associated with MOD-024-1 and MOD-025-1 are based on
arbitrary percentages that have little to do with the impact a failure
to perform would have on reliability. Constellation believes that these
problems ultimately will reduce the effectiveness of the Reliability
Standards. Consequently, Constellation requests that the Commission
recognize these concerns and direct NERC to take them into
consideration during the Reliability Standards development process.
ii. Commission Determination
    1309. The Commission will not approve or remand MOD-024-1 until the
ERO submits additional information. In order to continue verifying and
reporting gross and net real power generating capability needed for
reliability assessment and future plans, we direct the ERO to develop a
Work Plan and submit a compliance filing.
    1310. The Commission remains concerned that the Reliability
Standard is not sufficiently clear because it does not define the test
conditions and methodologies for calculating de-rating

factors. The Commission does not agree with APPA that NERC should
consider modifying this Reliability Standard to provide requirements
for this information on an Interconnection-wide basis, in the same
manner that IRO-006-3 sets the requirements for transmission loading
relief in each Interconnection. We believe, however, that while the
overall methodology for verification of generator gross and net real
power capability should be the same, test conditions (such as ambient
temperature, river water temperature, etc.) can vary.
    1311. In the NOPR, the Commission stated that the Reliability
Standard could be improved by defining test conditions, e.g., ambient
temperature, river water temperature, and methodologies for calculating
de-rating factors for conditions such as higher ambient temperatures
than the test temperature. With the test information and methodologies,
the generator output that can be expected to be available at forecasted
weather conditions can be determined. The Commission agrees with
Northern Indiana that testing all units at the same time is not
feasible. However, the Commission did not propose simultaneous testing.
Rather, we direct the ERO to develop appropriate requirements to
document test conditions and the relationships between test conditions
and generator output so that the amount of power that can be expected
to be delivered from a generator at different conditions, such as peak
summer conditions, can be determined. Similarly, we respond to
Constellation that any modification of the Levels of Non-Compliance in
this Reliability Standard should be reviewed in the ERO Reliability
Standards development process.
    1312. We repeat our concern that Requirement R2, which specifies
that the ``regional reliability organization shall provide generator
gross and net real power capability verification within 30 calendar
days of approval,'' is not clear. The requirement lacks a definition of
what approval is required and when the 30-day period starts. Therefore,
we direct the ERO to modify this Reliability Standard by adding
information that will clarify this requirement.
    1313. The Commission neither accepts nor remands MOD-024-1 until
the ERO submits additional information. Although the Commission did not
propose any action with regard to MOD-024-1, it addressed above a
number of concerns regarding the Reliability Standard. We therefore
direct the ERO to use its authority pursuant to Sec.  39.2(d) of our
regulations to require users, owners and operators to provide this
information. In the interim, compliance with MOD-024-0 should continue
on a voluntary basis, and the Commission considers compliance with it
to be a matter of good utility practice.
y. Verification of Generator Gross and Net Reactive Power Capability
(MOD-025-1)
    1314. MOD-025-1 requires the regional reliability organization to
establish and maintain procedures to address verification of generator
gross and net reactive power capability. The Reliability Standard also
requires the regional reliability organization to provide its generator
gross and net reactive power capability verification and reporting
procedures, and any changes to those procedures, to the generator
owners, generator operators, transmission operators, planning
authorities and transmission planners affected by the procedure within
30 calendar days of approval of the Reliability Standard.
    1315. In the NOPR, the Commission identified MOD-025-1 as a fill-
in-the-blank standard that requires the regional reliability
organization to establish and maintain procedures to address
verification of generator gross and net reactive power capability. The
NOPR stated that because the regional procedures had not been
submitted, the Commission would not propose to approve or remand MOD-
025-1 until the ERO submits the additional information. In addition,
the Commission suggested that MOD-025-1 could be clearer by requiring a
minimum reactive power (MVAR) capability throughout a unit's real power
operating range. Further, the NOPR stated that requirement R2 provides
that the ``regional reliability organizations shall provide generator
gross and net real power capability verification within 30 calendar
days of approval'' and noted that it is not clear what approval is
required and when the 30-day period starts.
i. Comments
    1316. APPA agrees that the Commission should not approve this
Reliability Standard until NERC and the regional reliability
organizations/Regional Entities develop the necessary regional
methodologies and the Commission approves them.
    1317. MidAmerican notes that the Reliability Standard will be
clearer if minimum reactive power capability is required throughout a
unit's real power operating range. However, making this a Requirement
for existing units would be a hardship for units not built with the
Requirement in mind. Therefore, MidAmerican suggests that any such
requirement should allow existing units to be grandfathered in as they
are currently rated so that a new minimum reactive power standard is
only applicable to new generating units or units that are being
significantly upgraded.
    1318. Northern Indiana cautions the Commission against the
establishment of a minimum capability, because it could diminish a
unit's ability to contribute to Interconnection reliability, and to
maintain its own stability. Northern Indiana points out that all
generators have reactive capability curves from design manufacturers,
and these curves provide operators with a range that is considered by
the manufacturer to be a safe operating limit. Northern Indiana
contends that the continued use of reactive capability curves is
superior to establishment of an MVAR capability, and that operators
effectively use these curves to maintain unit stability, while also
contributing to the reliability of the Interconnection. Northern
Indiana believes that continued reliance on manufacturer reactive
capability curves is a technically sound means to achieve the
Reliability Standard's stated reliability goal in a manner superior to
the establishment of MVAR capability.
    1319. Similarly to Northern Indiana, Wisconsin Electric encourages
the Commission to withdraw this suggested modifications to NERC's
Reliability Standard for several reasons. Wisconsin Electric believes
that a requirement to test and verify the minimum reactive capability
at multiple points over the operating range as part of the additional
minimum MVAR capability requirement would be a significant and
unnecessary burden on utilities. In Wisconsin Electric's experience, a
reactive power test at a single operating point is sufficient and more
practical to achieve.
    1320. SoCal Edison recommends that the Commission specifically
state the effective date for compliance with each Reliability Standard
in its Final Rule. SoCal Edison states that the effective date is
critical and gives the example of MOD-025-1, with effective dates
phased in over several years after they are adopted by the NERC board
of trustees, and well after the date the Final Rule will be issued.
ii. Commission Determination
    1321. The Commission will not approve or remand MOD-025-1 until the
ERO submits additional information. In order to continue verifying and
reporting gross and net reactive power generating capability needed for
reliability assessment and future plans,

we direct the ERO to develop a Work Plan as defined in the Common
Issues section.
    1322. We disagree with commenters that verifying generator reactive
capability is a particularly difficult issue. The capability of
generators to produce reactive power is essential for real-time
analysis and planning. The Reliability Standard addressing this issue
requires a generator to verify reactive capability only at the unit's
full MW loading. However, other than baseload units, most generating
units rarely operate at full MW loading. It is unclear what reactive
capability is available throughout a unit's real power (MW) operating
range. Therefore, we believe a clearer standard would require a
verification of MVAR capability throughout a unit's real power (MW)
operating range. However, we share concern with several commenters that
such a requirement for all generators may not be necessary. Therefore,
we adjust the proposal in the NOPR and direct the ERO to modify MOD-
025-1 to require verification of reactive power capability at multiple
points over a unit's operating range.
    1323. We maintain the concern we expressed in the NOPR that
Requirement R2 provides that the ``regional reliability organization
shall provide generator gross and net reactive power capability
verification within 30 calendar days of approval'' and note that it is
not clear what approval is required and when the 30-day period starts.
We direct the ERO to provide clarification on this requirement.
    1324. The Commission neither accepts nor remands MOD-025-1 until
the ERO submits additional information. Although the Commission did not
propose any action with regard to MOD-025-1, it addresses above a
number of concerns regarding the Reliability Standard. We direct the
ERO to develop a Work Plan to verify and report on generator gross and
net reactive power capability while this Reliability Standard is being
modified and to modify this Reliability Standard through the
Reliability Standards development process to: (1) Require verification
of a reactive power capability at multiple points over a unit's
operating range and (2) clarify Requirement R2 with a definition of
what approval is needed and when the 30-day period starts.
9. PER: Personnel Performance, Training and Qualifications
    1325. The four proposed Personnel Performance, Training and
Qualifications (PER) Reliability Standards are applicable to
transmission operators, reliability coordinators and balancing
authorities with the intention of ensuring the safe and reliable
operation of the interconnected grid through the retention of suitably
trained and qualified personnel in positions that can impact the
reliable operation of the Bulk-Power System. The PER Reliability
Standards address: (1) Operating personnel responsibility and
authority; (2) operating personnel training; (3) operating personnel
credentials and (4) reliability coordination staffing.
a. Operating Personnel Responsibility and Authority (PER-001-0)
    1326. PER-001-0 requires that transmission operator and balancing
authority personnel have the responsibility and authority to direct
actions in real-time. PER-001-0 also requires clear documentation that
operating personnel have the responsibility and authority to implement
real-time action to ensure the stable and reliable operation of the
Bulk-Power System.
    1327. In the NOPR, the Commission proposed to approve PER-001-0 as
mandatory and enforceable.
 i. Comments
    1328. APPA agrees that PER-001-0 is sufficient for approval as a
mandatory and enforceable Reliability Standard.
    1329. ISO-NE supports the adoption of this Reliability Standard
provided that the Commission does not mandate that the tasks performed
by local control centers be included in the definition of transmission
operators. It explains that to do so would suggest that the local
control center has independent autonomy in operating the Bulk-Power
System, which conflicts with the ``one set of hands on the wheel''
philosophy supported by Order No. 2000 and the operating agreements
approved by the Commission to establish ISO-NE as New England's RTO.
ii. Commission Determination
    1330. The Commission agrees with the ``one set of hands on the
wheel'' philosophy described by ISO-NE as it applies to operations of
the Bulk-Power System and has no intention of deviating from it.
Nothing in the Commission's proposed modifications outlined in the NOPR
in regard to the PER Reliability Standards is intended to conflict with
this philosophy. A generic discussion of the local control centers is
included in the Applicability Issues section and specific implications
to operator training are discussed in PER-002-0.\365\
---------------------------------------------------------------------------

    \365\ See Applicability Issues: Use of the NERC Functional
Model, supra section II.C.4.
---------------------------------------------------------------------------

    1331. Accordingly, the Commission approves PER-001-0 as mandatory
and enforceable. We find that the Reliability Standard is just,
reasonable, not unduly discriminatory or preferential and in the public
interest.
b. Operating Personnel Training (PER-002-0)
    1332. PER-002-0 requires that transmission operator and balancing
authority personnel are adequately trained. The Reliability Standard:
(1) Directs each transmission operator and balancing authority to have
a training program for all operating personnel who occupy positions
that either have primary responsibility, directly or indirectly, for
the real-time operation of the Bulk-Power System or who are directly
responsible for complying with the NERC Reliability Standards; (2)
lists criteria that must be met by the training program and (3)
requires that operating personnel receive at least five days of
training in emergency operations each year using realistic simulations.
    1333. In the NOPR, the Commission proposed to approve Reliability
Standard PER-002-0 as mandatory and enforceable. In addition, the
Commission proposed to direct that NERC submit a modification to PER-
002-0 that: (1) Identifies the expectations of the training for each
job function; (2) develops training programs tailored to each job
function with consideration of the individual training needs of the
personnel; (3) expands the applicability to include reliability
coordinators, generator operators, and operations planning and
operations support staff with a direct impact on the reliable operation
of the Bulk-Power System; (4) uses the Systematic Approach to Training
(SAT) methodology in its development of new training programs and (5)
includes performance metrics associated with the effectiveness of the
training program. In addition, the Commission requested comments on the
benefits and appropriateness of required ``hands-on'' training using
simulators in dealing with system emergencies.
i. General Issues
(a) Comments
    1334. EEI supports the Commission's direction for personnel
training and generally agrees with the Commission's proposal for PER-
002-0. EEI states NERC is developing a new Reliability Standard, PER-
005-0, which could be

filed with the Commission as early as July 2007. According to EEI, this
new Reliability Standard will respond to the issues raised in the NOPR
regarding PER-002-0. EEI notes that the ERO plans to retire Reliability
Standards PER-002-0 and PER-004-1 when proposed PER-005-0 is adopted.
It recommends that the Commission consider consolidating all training
requirements into a single Reliability Standard to simplify the
Reliability Standards catalog.
    1335. Additional comments received have been grouped as follows:
Local control center personnel; applicability to generator operators;
applicability to operations planning and operations support staff;
implications to small systems; training performance metrics; use of SAT
methodology; and use of simulators separately, followed by an overall
conclusion and summary.
(b) Commission Determination
    1336. EEI's comments concerning a possible PER-005-0 are beyond the
scope of this proceeding. The Commission will not require the ERO to
consolidate all training requirements into a single Reliability
Standard. We believe that such matters should be left to the discretion
of the ERO through its Reliability Standards development process.
ii. Local Control Center Personnel
    1337. In the NOPR, the Commission noted that decisionmaking and
implementation may be performed by separate groups in an ISO or RTO
context, as well as other organizations that pool resources.\366\ The
Commission proposed that all control centers and organizations that are
necessary for the actual implementation of the decision or are needed
for operation and maintenance made by the ISO, RTO or pooled resource
organization should be part of the transmission or generator operator
function. Although the NOPR discussed this matter in the context of the
Communication (COM) Reliability Standards, the NOPR indicated that the
proposal would apply in the training and certification context, as
well.\367\
---------------------------------------------------------------------------

    \366\ NOPR at P 236-37.
    \367\ Id. at P 237, 779.
---------------------------------------------------------------------------

(a) Comments
    1338. EEI states that the term ``operating personnel'' as used in
the PER group of Reliability Standards needs clarification because it
may be interpreted to mean any person with a capability to take a
unilateral action that can have a potentially significant effect on the
Bulk-Power System. EEI states that the term is open to broad
interpretation in actual practice, subject to various contracts,
operating agreements and ISO/RTO procedures. It states, for example, a
local control center operator may take instructions from and act on
those instructions, whereas the ``transmission operator'' under the
Functional Model may be viewed as a more centralized authority such as
a larger regional system operator. EEI contends that some define local
control center as a transmission operator, while others disagree.
    1339. ISO-NE states the scope of PER-002-0 need not be expanded
because local control center personnel in its footprint implement tasks
delegated to them by ISO-NE for operation of designated transmission
facilities. NPCC argues that expanding PER-002-0 beyond the entities
identified under the NERC Functional Model (i.e., transmission
operators, reliability coordinators and balancing authorities) will
require substantial cost and time but add little value. It states that
there are no certification exams for any entities other than
transmission operators, reliability coordinators and balancing
authorities and to develop and implement such exams and to have the
additional personnel certified would take several years. It also states
that these personnel already function under the authority of NERC-
certified operators and act only at the direction of certified
operators. It concludes that an entity that does not exercise
operational authority should not be subject to the same requirements as
the decisionmaker.
    1340. Northern Indiana states that it is not uncommon in the
industry for employees who perform switching operations to be
supervised by NERC-certified operators and that such employees are
subject to round-the-clock review by, and communication with, their
NERC-certified transmission operators. Similarly, SoCal Edison notes
that large utilities can have operators strategically located
throughout a vast service territory at switching centers with SCADA
capability and that these operators follow the directives of one
control center responsible for Bulk-Power System reliability. SoCal
Edison disagrees that the operators of these switching centers, simply
because the switching center has SCADA capability, must be NERC-
certified.
    1341. LPPC states that the training and certification requirements
should apply only to transmission and generation personnel that are
located in the transmission control center (i.e., responsible for real-
time Bulk-Power System operations). It argues that transmission and
generation operation employees that are located in remote locations
that are not directly involved in the real-time scheduling of
transactions or Bulk-Power System monitoring and control do not need to
be certified for real-time operations because they are not involved in
the type of functions in which regimented training in the Reliability
Standards would be useful. It suggests that a bright line should be
drawn between the training of actual system operators and the training
for operators of generation plants that are not responsible for
scheduling. LPPC also states that the Commission should clarify the
scope of training that the transmission control center real-time
operations personnel should receive.
    1342. Entergy asserts that the training program should be tailored
to the functions local control center operators, generator operators
and operations planning staff perform that impact the reliable
operation of the Bulk-Power System for both normal and emergency
operations.
(b) Commission Determination
    1343. In our discussion above regarding the Functional Model, we
emphasized our concern that there should be no unintentional gaps or
redundancies in responsibility for compliance with the Requirements of
Reliability Standards. This concern arises particularly in the context
of RTOs, ISOs and other pooled resources that may have separate
divisions performing decisionmaking functions and implementing
functions within the transmission operator classification. The topic of
training is one such area of concern. While PER-002-0 applies to
transmission operators, it is important for reliability that personnel
involved in both decisionmaking and implementation receive proper
training.
    1344. Clearly, in a region where an RTO or ISO performs the
transmission operator function, its personnel with primary
responsibility for real-time operations must receive formal training
pursuant to PER-002-0. In addition, personnel who are responsible for
implementing instructions at a local control center also affect the
reliability of the Bulk Power System. These entities may take
independent action under certain circumstances, for example, to protect
assets, personnel safety and during system restorations. Whether the
RTO or the local control center is ultimately responsible for
compliance is a separate issue addressed above, but regardless of which
entity registers for that responsibility, these local control center

employees must receive formal training consistent with their roles,
responsibilities and tasks. Thus, while we direct the ERO to develop
modifications to PER-002-0 to include formal training for local control
center personnel, that training should be tailored to the needs of the
positions.
    1345. As noted by SoCal Edison, there are different operating
structures and therefore there is a need to clarify to which control
centers we direct the Reliability Standard apply. For example, for a
large utility within an RTO or ISO footprint there may be one
centrally-located control center whose function is to supervise several
distributed control centers, each with remote monitoring and control
capability. In this type of structure, the personnel of the centrally-
located control center should receive formal training in accordance
with the Reliability Standard. Personnel at the distributed control
centers also need to be trained, but the responsibility for this
training is outside the scope of the Reliability Standard.\368\
---------------------------------------------------------------------------

    \368\ The Commission expects the entity registered as the
transmission operator to ensure that these personnel are competent
for the tasks that they perform.
---------------------------------------------------------------------------

    1346. Another organizational structure, typically representative of
relatively smaller entities, consists of a single control center that
implements operating instructions from its transmission operator, e.g.,
an RTO, ISO or pooled resource. Similar to the discussion above,
operators at these control centers also may take independent action to
protect assets, safety and system restoration. Such control center
personnel must also receive formal training pursuant to PER-002-0.
    1347. Consistent with the comments of SoCal Edison and Northern
Indiana, the Commission understands that it is common practice to have
traveling operators located in the local control centers who carry out
field switching operations and station inspections at the direction of
the local control center operators. These personnel are not involved
with the transmission operator at the ISO or RTO or at organizations
with pooled resources, and as such, should not be subject to
Reliability Standard PER-002-0.
    1348. The Commission disagrees with those commenters who contend
that, because operators at local control centers take direction from
NERC-certified operators at the ISO or RTO, they do not need to be
addressed by the training requirements of PER-002-0. Rather, as
discussed above, these operators maintain authority to act
independently to carry out tasks that require real-time operation of
the Bulk-Power System including protecting assets, protecting personnel
safety, adhering to regulatory requirements and establishing stable
islands during system restoration.
    1349. Several commenters express concern about requiring local
control center operators to become fully trained to the same extent as
transmission operators, balancing authorities and reliability
coordinators. This is not the Commission's intent. As we stated in the
NOPR, the proposed modifications do not imply a ``one-size-fits-all''
approach but rather ensure the creation of training programs that are
structured and tailored to the different functions and needs of the
personnel involved.\369\ Therefore the Commission agrees with Entergy
that the training program should be tailored to the functions local
control center operators, generator operators and operations planning
staff perform that impact the reliable operation of the Bulk-Power
System for both normal and emergency operations.
---------------------------------------------------------------------------

    \369\ See NOPR at P 773, 775.
---------------------------------------------------------------------------

iii. Applicability to Generator Operators
    1350. The Commission proposed in the NOPR a modification to PER-
002-0 to include real-time operations personnel from reliability
coordinators, generator operators, operations planning and operations
support staff in training programs with a time-phased effective
date.\370\
---------------------------------------------------------------------------

    \370\ Id. at P 772.
---------------------------------------------------------------------------

(a) Comments
    1351. PG&E and FirstEnergy support the Commission's goal of
ensuring appropriate training for generator operators. FirstEnergy,
however, believes that there is some confusion between the Functional
Model and the Reliability Standard requirements concerning the
generator operator classification. FirstEnergy explains that, in some
contexts, ``generator operator'' refers to operations personnel who are
centrally-located at a generation control center (i.e., fleet
operators) while in other contexts it refers to generator operators
located at the generation plant (i.e., unit operator). Further,
according to FirstEnergy, the NERC glossary defines ``generator
operator'' as the entity that operates generating unit(s) and performs
the functions of supplying energy and interconnected operations
services. FirstEnergy requests that the Commission direct NERC to
revise the Reliability Standard to recognize this distinction.
    1352. Other commenters, including Xcel, California PUC and Entergy,
state that the Reliability Standard should not apply to generator
operators. Xcel argues that generator operators take their direction
from transmission operators, balancing authorities and reliability
coordinators, which limits their ability to exercise independent action
impacting the reliability of the Bulk-Power System. Entergy argues that
expanding the applicability to generator operators would provide little
benefit to those personnel in the performance of their own functions,
and could distract them from those functions. It also argues that such
training would be extremely costly and would divert necessary resources
from more important reliability objectives.
    1353. California PUC states that the requirement to include power
plant operators in the applicability of this Reliability Standard
exceeds anything contemplated in the regulation of the Bulk-Power
System under previous NERC guidelines and what is authorized by
statute. It contends that impacts of generator operator actions on the
Bulk-Power System are of a much smaller magnitude and consequence than
those of system operators. Further, it states that other authorities,
such as balancing authorities and state governments, may have acted in
regard to training of power plant operators and, therefore, the
Commission should not act where other authorities have already done so.
In a similar vein, the Nevada Companies state that the activities of
generating station operations personnel are limited to the confines of
the specific generating station. Knowledge of or exposure to
interconnected grid operating principles is simply not applicable to
the tasks normally performed at the generating stations.
    1354. Reliant states that the proposed modification fails to
clarify how generator operators are to satisfy the training program
requirement or the scope of generator operator personnel that must be
trained. It states that the proposed modification could be interpreted
to require generator operators to train the plant operator as well as
the dispatcher in the generator operator's local control center.
Reliant believes, however, that plant operators should not be subject
to the Reliability Standard's training program requirement because
personnel employed in plant operating positions are trained in the
operation of plant equipment and take direction with respect to the
operation of the plant from management personnel as well as from the
local control center. Accordingly, it reasons that, because

these employees take direction with respect to plant operations from
elsewhere, they do not have primary responsibility for the real-time
operation of the Bulk-Power System and should not be responsible for
complying with Reliability Standards. Reliant suggests that PER-002-0
should specifically target generator operator personnel that develop
dispatch instructions and the Reliability Standard should be modified
to accommodate generator operator entities that are members of ISOs and
RTOs with established NERC-approved certification programs. However, it
should exclude those personnel who simply take direction on plant
operations.
    1355. Dynegy, MISO and Wisconsin Electric state that these
Reliability Standards should not be extended to all real-time operation
positions of a generator operator. They state that many real-time
operation positions are staffed by long-tenured union personnel who
routinely operate generating units and take directions from a
centralized generation control center or the local RTO/ISO. They
analogize this type of certification and training requirement with
requiring the outside field force of a transmission operator, including
positions that operate and switch electric transmission lines pursuant
to instructions from a centralized transmission control group, to be
NERC-certified. Dynegy and MISO support a more limited extension of
these Reliability Standards to real-time operation personnel located in
a centralized generation control center that interfaces with the plants
and the local RTO/ISO but not to personnel at the plant level.
    1356. Some commenters address the appropriate scope of training for
generator operators. For example, MidAmerican states that experience
and knowledge necessary for transmission operators may go well beyond
what is needed for generation operations. It contends that a NERC-
approved training course specific to these functions would be an
appropriate alternative. Entergy comments that, if training of
generator operator personnel is required, it should focus on the
functions generator operators must perform, not on the functions that
others perform. SDG&E states that training for generator operators and
others who may directly impact the reliable operations of the Bulk-
Power System need not be identical to or as extensive as that required
of transmission system operators, but should be tailored in scope,
contents and duration so as to be appropriate to the personnel and the
object of promoting system reliability.
    1357. FirstEnergy states that there are no universal certification
or training programs for generator operators; therefore a reasonable
transition period should be established to allow time for generator
operators to comply with this Reliability Standard. It also states that
nuclear units are already subject to NRC training requirements and that
compliance with NRC requirements should satisfy this Reliability
Standard.
    1358. APPA, Process Electricity Committee and TAPS are concerned
that, unless a size limitation is included for the generator operators,
a substantial number of generator operator personnel will have to be
enrolled in training programs. They argue that while a generator plays
an important role in the reliable operations of the bulk electric
system, the generator operator takes commands from the transmission
operator, balancing authority or reliability coordinator. TAPS opposes
the expanded applicability, especially in the case of small systems,
because it believes that the requirement would be costly with no
benefits to reliability.
    1359. Process Electricity Committee is concerned about the effect
of the expanded requirements on end users who have on-site generation.
It argues that the training requirements would present an added cost
for end users with no apparent added benefit and that, in the long
term, end users may be discouraged from developing on-site generation,
which in turn would leave industrial electricity users more vulnerable
to failures elsewhere on the energy grid.
(b) Commission Determination
    1360. The Commission explained in the NOPR that transmission
operators and balancing authorities are not the only entities that have
operating personnel in positions that directly impact the reliable
operation of the Bulk-Power System; and included generator operators
among those that have such an impact.\371\ Xcel and others oppose
extending the applicability of PER-002-0 to generator operators,
because they take directions from balancing authorities and others,
which limits their ability to impact reliability. Although a generator
may be given direction from the balancing authority, it is essential
that generator operator personnel have appropriate training to
understand those instructions, particularly in an emergency situation
in which instructions may be succinct and require immediate action.
Further, if communication is lost, the generator operator personnel
should have had sufficient training to take appropriate action to
ensure reliability of the Bulk-Power System. Thus, we direct the ERO to
develop a modification to make PER-002-0 applicable to generator
operators.
---------------------------------------------------------------------------

    \371\ NOPR at P 771.
---------------------------------------------------------------------------

    1361. We agree with FirstEnergy and others that some clarification
is required regarding which generator operator personnel should be
subject to formal training under the Reliability Standard. As noted
above, a generator operator typically receives instructions from a
balancing authority. Some generator operators are structured in such a
way that they have a centrally-located dispatch center that receives
direction and then develops specific dispatch instructions for plant
operators under their control. For example, a balancing authority may
direct a centrally-located dispatch center to deliver 300 MW to the
grid, and the dispatch center would determine the best way to deliver
that generation from its portfolio of units. In this type of structure,
it is the personnel of the centrally-located dispatch center that must
receive formal training in accordance with the Reliability Standard.
Plant operators located at the generator plant site also need to be
trained but the responsibility for this training is outside the scope
of the Reliability Standard.\372\
---------------------------------------------------------------------------

    \372\ The Commission expects the entity registered as the
generator operator to ensure that plant operators are competent for
the tasks that they perform.
---------------------------------------------------------------------------

    1362. Other generator operators may be structured in such a way
that the dispatch center and the single generation plant are at the
same site. In this structure as well, some personnel will perform
dispatch activities while others are designated as plant operators.
Again, it is the dispatch personnel that must receive formal training
in accordance with the Reliability Standard. Plant operators also need
to be trained but the responsibility for this training is outside the
scope of the Reliability Standard.
    1363. We disagree with Nevada Companies, Xcel and others that
assert that generator operator training will provide limited benefit.
Rather, we conclude that, with the above focused direction regarding
the applicability of the Reliability Standard to generator operator
personnel, the benefits to the Bulk-Power System will be maximized and
the cost of formal training limited. Further, our direction addresses
California PUC's concerns regarding application to plant operators. In
any event, the existence of local training requirements in some regions
does not supplant the need for uniform training requirements for all
generator operators

developed in a Reliability Standard with continent-wide applicability.
    1364. Further, the Commission agrees with MidAmerican, SDG&E and
others that the experience and knowledge required by transmission
operators about Bulk-Power System operations goes well beyond what is
needed by generation operators; therefore, training for generator
operators need not be as extensive as that required for transmission
operators. Accordingly, the training requirements developed by the ERO
should be tailored in their scope, content and duration so as to be
appropriate to generation operations personnel and the objective of
promoting system reliability. Thus, in addition to modifying the
Reliability Standard to identify generator operators as applicable
entities, we direct the ERO to develop specific Requirements addressing
the scope, content and duration appropriate for generator operator
personnel.
    1365. FirstEnergy states that nuclear plant operators are already
subject to NRC training requirements and thus suggests that compliance
with NRC requirements should satisfy this Reliability Standard.
FirstEnergy does not identify the content of the NRC training
requirements, and the Commission is unaware whether the NRC training
requirements adequately address the interaction between a nuclear power
plant and the Bulk-Power System. Accordingly, without drawing any
conclusion on the matter, the Commission directs that the ERO consider
FirstEnergy's comments in the Reliability Standards development
process.
    1366. Commenters' concerns regarding the need for a size limitation
on generator operators should be satisfied by our determination that
the applicability of particular entities should be determined based on
the ERO compliance registry criteria, which APPA and TAPS support. We
believe that limiting the applicability of Reliability Standards to
NERC's definition of bulk electric system will alleviate much of
Process Electricity Committee's concern regarding the effect of the
expanded requirements on end users who have on-site generation. For
larger end users who have on-site generation, the Commission believes
that there is an added benefit to including them in the Reliability
Standards because they sell into the market and should be treated on a
similar basis as any other generator of a similar size.
iv. Applicability to Operations Planning and Operations Support Staff
    1367. As mentioned above, the Commission proposed in the NOPR to
direct the ERO to develop a modification to PER-002-0 to require
training of operations planning and operations support staff of
transmission operators and balancing authorities who have a direct
impact on the reliable operation of the Bulk-Power System.
(a) Comments
    1368. Several commenters, including EEI and APPA, oppose the
proposed applicability of the Reliability Standard to operations
planning and operations support staff. Other commenters contend that
the Commission's proposal is ambiguous and should be clarified.
    1369. EEI states that the extension of the applicability to
``operations support personnel'' could result in a dramatic expansion
of industry training requirements with uncertain benefits to system
reliability. It requests that the Commission reconsider this proposal
or provide some additional clarity on the definition of the term. APPA
also expresses concern about expanding the applicability to operations
planning and operations support staff, especially if the Commission
adopts its proposed interpretation of the bulk electric system because
this would become quite onerous for small utilities. Wisconsin Electric
states that the Commission's proposal does not address how to identify
the operations planning and operations support personnel who would be
subject to the Reliability Standard and how to develop compliance
measures for them. It contends that the proposed modification is
ambiguous and should not be implemented.
    1370. Avista states that individuals who are responsible for
assessing a company's compliance with the Reliability Standards may
simply have an administrative and coordination role, but have no direct
responsibility for reliable operations of the Bulk-Power System. It
argues that such individuals, while operations support staff, should
not be subject to the proposed Reliability Standard. It therefore
requests that the Commission clarify that personnel subject to the
Reliability Standard may include operations planning and operations
support staff.
    1371. Entergy believes it is unnecessary to require all staff
supporting the transmission operator to be trained in the transmission
operator's Reliability Standards responsibilities. It states that as
long as the supporting personnel work under the direction of a NERC-
certified transmission operator, there is no need for duplicative
training for supporting personnel. Entergy comments that, if such
training is required, it should focus on the functions operations
planning and operations support staff must perform, not on the
functions that others perform.
    1372. Northern Indiana states that expanding application of the
Reliability Standard to operations support staff ``with a direct impact
on the reliable operation of the Bulk-Power System'' is ambiguous. It
states that NERC surveyed certified operators for its job function
analysis related to this Reliability Standard with results due at the
end of January 2007. Northern Indiana recommends that the results of
this survey be considered in the development and clarification of this
proposed Reliability Standard. Further, Northern Indiana is concerned
about which specific job functions will be addressed and which will be
exempt, and about what ``direct'' versus ``indirect'' impact means.
(b) Commission Determination
    1373. The Commission directs the ERO to develop a modification to
PER-002-0 that extends applicability to the operations planning and
operations support staff of transmission operators and balancing
authorities, as clarified below. Most commenters express concern about
extending the applicability of the Reliability Standard because they
believe ``operations planning'' and ``operations support'' are not
well-defined and could encompass a significant number of operations
personnel. In the NOPR, the Commission stated that the Reliability
Standard should apply to operations planning and operations support
staff that have a direct impact on the reliable operation of the Bulk-
Power System.\373\ We clarify that these personnel include those who
carry out outage coordination and assessments in accordance with
Reliability Standards IRO-004-1 and TOP-002-2, and those who determine
SOLs and IROLs or operating nomograms in accordance with Reliability
Standards IRO-005-1 and TOP-004-0. The Commission directs the ERO to
include in PER-002-0, personnel who carry out the above functions.
---------------------------------------------------------------------------

    \373\ NOPR at P 780.
---------------------------------------------------------------------------

    1374. In addition, the Commission is aware that the personnel
responsible for ensuring that critical reliability applications of the
EMS, such as state estimator, contingency analysis and alarm processing
packages, are available, up-to-date in terms of system

data and produce useable results can also have an impact on the
Reliable Operation of the Bulk-Power System. Because these employees'
impact on Reliable Operation is not as clear, we direct the ERO to
consider, through the Reliability Standards development process,
whether personnel that perform these additional functions should be
included in mandatory training pursuant to PER-002-0.
    1375. APPA and EEI oppose the proposed extension of the Reliability
Standard to operations planning and operations support staff, claiming
that it could dramatically expand industry training requirements with
uncertain benefits to system reliability. Our clarification above
adequately addresses these concerns because we have identified a
specific set of such personnel that have a direct impact on reliable
operations. With the above clarification, our directive is not as
expansive as EEI and APPA contemplate, and is more clearly connected
with Bulk-Power System reliability. Further, since the Commission is
not adopting the proposed interpretation of the ERO's definition of
bulk electric system, as discussed in the Applicability section above,
the directed modification to PER-002-0 should not be onerous to small
entities as suggested by APPA.
    1376. Several commenters express concern that the operations
planning and operations support staffs will be required to be trained
on the transmission operators' responsibilities. The Commission
clarifies that this is not the case. Training programs for operations
planning and operations support staff must be tailored to the needs of
the function, the tasks performed and personnel involved.
v. Training Performance Metrics
    1377. In the NOPR, we noted the assertion by ISO/RTO Council that
there is no definition for ``adequately trained operating personnel.''
ISO/RTO Council suggested adoption of performance metrics to ensure
that training results in competent operating personnel.\374\ The
Commission agreed and proposed to require that the ERO modify PER-002-0
to include performance metrics to assess the effectiveness of the
training program. The Commission also stated that such performance
metrics are not a substitute for an SAT developed training program.
---------------------------------------------------------------------------

    \374\ Id. at P 776.
---------------------------------------------------------------------------

(a) Comments
    1378. Xcel does not agree that performance metrics should be
included as part of this Reliability Standard. While it believes
performance metrics are generally useful, it states that in this case
it would be difficult to develop the appropriate metrics. MidAmerican
believes that the proposed performance metrics are not essential to
ensuring the appropriateness of training because the Reliability
Standard already requires NERC approval of all training activities, and
specifically requires training in certain areas.
    1379. MISO and Wisconsin Electric state that it is unclear how a
Reliability Standard to measure the effectiveness of a training program
would apply to an organization that contracts for training services,
and that there are many training requirements found in other
Reliability Standards covering the topics and amount of training. They
argue that the proposed modification is overly-prescriptive and
deviates from a fundamental training concept that training should be
tailored to the organization and to the individual.
(b) Commission Conclusion
    1380. Xcel, MISO and MidAmerican state that performance metrics to
assess the effectiveness of training programs are unnecessary. The
Commission believes that, if quantifiable performance metrics can be
developed to gauge the effectiveness of a Reliability Standard, these
performance metrics should be developed, tracked and used to
continually improve an applicable entity's performance and the
Reliability Standard itself. The Commission directs the ERO to explore
the feasibility of developing meaningful performance metrics for
assessing the effectiveness of training programs, and if feasible, to
develop such metrics for the Reliability Standard as part of the
Reliability Standards development process.
vi. Use of Systematic Approach to Training (SAT) Methodology
    1381. In the NOPR, the Commission required the ERO to use the SAT
methodology in identifying the requirements for a training program
because SAT is a proven approach to: identify the tasks and associated
skills and knowledge necessary to accomplish those tasks; determine the
competency levels of each operator to carryout those tasks; determine
the competency gaps; and design, implement and evaluate a training plan
to address each operator's competency.\375\
---------------------------------------------------------------------------

    \375\ Id. at P 775.
---------------------------------------------------------------------------

(a) Comments
    1382. ISO-NE states that the use of SAT methodology should not be
mandated and that responsible entities under this Reliability Standard
should be allowed the flexibility to use the most appropriate training
methodology available. Northern Indiana requests clarification on about
our proposal on the use of SAT methodology.
(b) Commission Determination
    1383. The Commission understands that the new operator training
Reliability Standard PER-005-1-0 currently under development by the ERO
would endorse the use of SAT. In response to ISO-NE, training based on
SAT is a proven approach to identify the skills and knowledge necessary
to accomplish particular tasks, evaluate each operator's competency to
carry out those tasks, determine any competency competency gaps, and
design, implement and evaluate a training plan to address such gaps.
Since SAT is the most appropriate training methodology available, we
believe this addresses ISO-NE's comments. Northern Indiana requests
clarification about the details of our proposal for SAT methodology.
The Commission has not directed how the SAT methodology should be
implemented, but we expect it to be developed through the Reliability
Standards development process. We encourage Northern Indiana to become
involved in the process. Thus, we adopt the NOPR proposal to direct
that the ERO develop a modification to PER-002-2 (or a new Reliability
Standard) that uses the SAT methodology.
vii. Use of Simulators for Training
    1384. The Commission explained in the NOPR that Requirement R4 of
the Reliability Standard requires training in emergency operations
using realistic simulations of system emergencies and noted that there
are various options available for providing operator training simulator
capability, including contracting for this service from others who have
developed the capability. The Commission requested comments on the
benefits and appropriateness of required ``hands-on'' training using
simulators in dealing with system emergencies.\376\
---------------------------------------------------------------------------

    \376\ Id. at P 778.
---------------------------------------------------------------------------

(a) Comments
    1385. While most commenters recognize the benefits of simulator
training, they differ on whether simulator training should be
mandatory.
    1386. NERC comments that there can be significant value gained by
training operating personnel for emergencies under realistic conditions
using training simulators and requests that comments on this matter be
directed to the

Reliability Standards development process for consideration. APPA
believes that significant reliability benefits could result from the
use of simulators by reliability coordinators, transmission operators
and balancing authorities that have operational control over a
significant portion of load and resources. It does not believe,