1 RESOLVE EPA Record Keeping Rule 1/ 31/ 02, Irving, TX Tape 2 [ Side A]

Joe Retzer ( continued from Side B, tape one):   we would handle that in my world is such and such.

Pat McFadden: I'm Pat McFadden with Gustafson. In a paper world, this is easy because, I have my paper. I write down whatever it is I'm changing. The SOPs are in place to tell me how to deal with paper, and it's me and my pen. When you jump into the electronic world, everything changes. So you're asking us to generate records in a form that can't be altered, and you're asking us to maintain these records.

Well, the problem is, a lot of our equipment has software on it that is pre CROMERRR compliant. Now I know that if you buy new software, there is stuff that is compliant with [ the Food and Drug Administration] Part 11, however to upgrade everything would be extremely expensive. So our old software has not computer generated audit trails, and there is no way,

without spending money, that we can put that into place. So that becomes a financial burden, to upgrade all the software 2 into this [ Good Laboratory Practices] compliant software. So we don't have these computer generated audit trails.

Then when we archive things, we archive our paper. It has no meta data, it has no audit trail, and so that, number nine we're also not in compliance with, until we change our software. So when CROMERRR goes into effect, basically you're saying to us, all our software now has to be changed. We have to go out and purchase all new software for all this analytical instrumentation that we have so that we will be in compliance.

Is that my understanding? Is that correct?

Joe Retzer: That would be correct, if CROMERRR were published or promulgated today as it were proposed.

Unidentified Male Speaker: In looking at these, if all these criteria do exist for paper, I'm not sure even all the companies are complying with all the criteria for paper even today.

Joe Retzer: I don't think they are, formally. They're not formally regulatory criteria for paper. I was suggesting that it might be good business practice, is generally what I was saying.

Unidentified Male Speaker: But are they generally going to be requirements for CROMERRR? 3 Joe Retzer: Well that's the question before us today, is whether they should be and are there some that are more difficult than others?

Unidentified Male Speaker: I mean my experience, and I used to run our emission fee billing program for our agency,

and when we needed information from facilities. We'd have to go several levels deep into departments to try and get that information. It wasn't corporate wide. Only a few people knew where it was, and it didn't seem like it would comply with any of these.

Even in our own agency, the refinery people have to submit reports to us telling us how much crude they outload, 1172 reports, and those go to the refinery unit and nowhere else,

and they keep those, and if people need them, they've got to know where that information resides in our agency. It's not corporate wide in a scanned image system and things of that sort, even though we do have an imaging system for a lot of our documents. So there are a lot of documents that are spread out. There are some that are available corporate wide, and it runs the whole gamut.

Nancy Freshour: Nancy Freshour, Dow Chemical again. To answer on question three, if, for example, in our [ Good Laboratory Practices] environment, if we produce an accurate 4 and complete copy of an e record on paper, and we maintain that currently, and do everything, sign, date, this is now adding an extra requirement, it's requiring us now to have an e form retained, so this is an added requirement. So just addressing that aspect of the thing.

Joe Retzer: I'm sorry, so now once you've put it on paper and signed it, then you say I don't need this e record anymore?

Nancy Freshour: Well, if you've established that it's an accurate presentation of that electronic [ unintelligible], the time/ date stamp, the person who generated it, there are multiple levels of security to get into the system, the analyst, for example, signs and dates it and everything and we have it all as an original signed record that way, and you are allowed, currently under [ Good Laboratory Practices], to state that the raw data is the paper copy, now we have to keep an erecord of it, too.

Joe Retzer: Right, so you're saying that under current practices, right now you get rid of the e record, or that you could.

Nancy Freshour: Theoretically yes, you could. But now this requires us to keep it and keep it for all those other criteria  all these criteria. 5 Now going back to your example of oh, we have a forensic signature now, what if somebody, your enemy wants to submit something. What is the risk of that? I mean you're talking about general business practices. Well, anything that we do,

we look at the risks and the benefits of doing it, and also the regulations that are required, of course, but in this rule it's one size fits all. There's no risk evaluation that's been done here about each of these types of records, and there's an immense amount of records under the definition of e record.

And so there's been no risk evaluation. So maybe the signatures require more but there's a lot of personal information that's been required for the signature, but there's been no risk evaluation.

Terry Salem: Terry Salem from the TNRCC. I'd like to revisit a topic that I've touched on in previous meetings in the past, which is the fallacy of the perfect paper system.

And I guess in regard to these criteria, even paper degrades.

Criteria number three cannot even be met by paper forever.

Paper degrades, it becomes human unreadable.

With respect to " cannot be altered without detection" in item one, also, it may not be   there are crude examples where you certain could detect. There are also examples where it would be virtually impossible to detect. 6 I also have some concern about the level to which we will contort ourselves to assure that records have not been altered,

and that goes back to the whole question of the risk analysis that needs to be done with respect to what we are attempting to gain here. And we've chatted about this a lot in previous meetings, and I think that is really the initial analysis that needs to be done by EPA, to determine where you want to gain benefit in areas where we don't have to meet the burden beyond a reasonable doubt standard. I think that's where we need to start.

Trying to do this in one fell swoop and address all of the arenas in 40 CFR where reporting is done by paper and we could have gains. If we started in an area where we don't have to reach that high level of evidentiary burden, we could start to create systems that we all felt comfortable with and that could meet some semblance of these criteria, perhaps not these exact criteria, and we would all gain valuable working knowledge and experience and comfort level, and perhaps start to set the court record on evidentiary burdens, and start to see where courts really have trouble with some of these audit trail issues, the non repudiation issues, and those kinds of things,

and sort of start down this path in a way that is risk analysis 7 based instead of trying to bite off this whole, big, huge piece of the apple at once.

Richard Wiard: Richard Wiard, [ Process Data Control Corp.] again. I guess I'll focus on this paragraph one and go right to that bottom section that says, " cannot be altered without detection." Now if we're talking about electronics,

it's difficult to conceive of something that, with the right amount of technical experience, you cannot move the bits and the bytes and make a change somewhere, at least limited.

It would appear to me more, I guess in harmony with what Terry was saying a second ago, we need to say " cannot be altered except with explicit approval by administrator" or something of that nature that begins to understand the reality of how electronic data is really worked with and who it's accessed by.

For example, a computer software program that would protect and a database that would protect alteration by anybody but a system administrator or an owner or somebody, would appear to me, to meet the spirit of what number one might be aiming toward and be achievable. The words " cannot be altered," I cannot conceive how that can be achieved, in a way.

I think that's been our experience in nearly every electronic media, we used to think a photograph as being evidence in legal 8 matters where now we know they can be doctored where you can almost not detect that a photograph has been changed.

Well, that's bound to have affected the legal process tremendously in regard to photographic evidence. I think this is going to have to address those same realities and get to a point where we say what is a reasonable protection of that record against accidental or, I guess you'd say, someone else trying to change it, or do you make duplicate copies that can be compared some way, but I think we're going to have to come up with some way to interpret that " cannot." And I would just suggest rewording that piece of it to be more reasonable.

Robin Roberts: What of this notion of the standard being too high?

Greg Nutt: I think we have a problem with scope here and we also have a problem with general approach. You don't try and take good practices for paper, as Joe described, and blindly try to apply those to electronic systems, because electronic systems have different limitations.

You start instead with what are good practices for maintaining data integrity in electronic systems? And those practices are well established in the financial world and can be applied to electronic records, however we choose to define that, for environmental records. Standard approaches with 9 respect to separation of authority, with respect to audit trails and how those audit trails are managed, and I think that's where we ought to start.

And that is addressed in CROMERRR to a certain extent, but then these more extreme requirements are put on and I want to note that you left out something on three, if you look at the proposal, three also requires that those records be available for off site review in electronic form, which is something that gives me quite a bit of grief as a web architect and trying to provide that information to anybody off site.

And I also have some real concerns about the enforceability of this rule. TNRCC does not have a legion of systems analysts to go down to Houston and inspect these systems to ensure compliance. And to my knowledge, EPA does not either, so I wonder why we're even making rules that we can never enforce.

Robin Roberts: I'm interested in the extent to which anyone here thinks that these might reflect good business practices, that these are reasonable. I mean you seem to be saying that these nine criteria are stringent even by current good business practices, is that so? Is there anyone who can walk through all nine and say how they do or don't apply now? 10 How do these diverge, at what point in the paper world, even,

would you cover those?

Mel Swoboda: This is Mel Swoboda again, and I guess I can't really get into the details of some of the computer things, but just the observation as a user in a computer system, and Greg might be able. Greg touched on a part of it,

having to manage a website and having to keep firewalls up to keep people out with uncontrolled access.

Many, many of our systems and many of our companies, and probably all the companies, if you have an IS group, and their function is just the security of your systems, making sure that you can get in, that they know who does what from an internal standpoint   I can't get to every record that you'd like me to get to. If you were to ask me, I would say, I don't have access to it, and we've gotten to where we do a lot of those things.

And I think there are   I want to stress think   I believe there are some ISO general standards that a lot of people follow that deal with those specific kinds of issues. Now, do they deal specifically with providing me with detailed audit trails? I don't know, some of those people could probably really tell you that specifically. But I do know they can track a heck of a lot of information around movements, whether 11 it is in a form, necessarily, if I get into one of my systems that you know I'm going to specifically be addressing a specific piece of information, I don't know.

The other thing, I guess, that comes back, I looked at this stuff and I started applying it, it goes back to where do you cut the line off? Where do we draw the line on these trails that makes it appropriate to maybe do some things like this? If it's electronic letters that I'm putting in, maybe it's appropriate. If it's an electronic report that I put in,

maybe they're appropriate.

But when we start backing out of that, it starts to be very, very difficult to figure out where the line is going to be in all this stuff, and Title V is a classic one. I see those guys over there, and I think, thank God I'm in water and I don't have to deal with them. Because you get down to the point, in those data systems and whatnot, that it gets extremely difficult to meet those requirements.

Robin Roberts: Comments on criteria again, these nine criteria, one short of being commandments. Excuse me, I can't talk about religion in a federal government setting.

Jim McFarland: Jim McFarland with IR4 again. What I'm hearing here as far as the electronic systems to meet this particular criteria that CROMERRR has proposed, a lot of the 12 safety features that we're now using are sort of isolating parts so that no one person has access to everything. That's fine for a big company, it's not that practical for the small entities that I deal with, typically, a one or two person field researcher that's doing [ Good Laboratory Practices]

residue trials.

Small laboratories. That type of safeguard system may not be that doable in that kind of situation, regardless of how much money you spend on the software. It's going to be very likely that one person will have access to everything. So whatever safety devices you're trying to put in to satisfy the legal or the lawyer aspects may not even be there if you go the full nine yards.

George Britton: George Britton with Aventis CropScience.

I just attended a meeting of the National Association of Independent Crop Consultants that met in Albuquerque last week.

One of the questions that came up was dealing with electronic notebooks and the [ Good Laboratory Practices] variety. And one fellow got up and asked this specific question. He's a one man operation, his office is in his home. I am assistant manager for the software, because he is the guy, he's the only one there. Because he is the professional investigator doing that,

so he would log in as a different user, and he asked the 13 question, a very simple question, " What happens to me? If I'm out there in the field, I have a heart attack, I die, whatever happens, how is the information brought forward to a sponsor organization?"

And all these kinds of things down there don't apply.

It's basically up to the integrity of the person. All of the fraud cases that have been, at least publicly prosecuted,

there's been IBT, the Craven Labs, and the currently Infratech System that's being prosecuted as we speak. All those fraud issues, where data was manipulated, was on the input side.

It wasn't on the record keeping side of the house where problems were occurring, it was on the input, the calibration curves, it was manipulating of chromatographic peaks before they were recorded. So all the effort to prevent fraud, any evidentiary stake that has been going forward, is not addressed by any of these nine criteria.

Yes, one would like to keep them safe, by the current practice yes, they need to be generally accurate, you're required by law already to be accurate and are prosecuted if you don't. What works for a very small outfit with one or two people will not be the same solution, processes and procedurewise that the larger corporations can do. 14 As an example, Aventis, even before all this came out,

started to look at the possibility of, simply for the ease of information management, looked at a record keeping system,

spent a quarter of a million dollars just to get the hardware,

software, and licenses for a limited installation of this, the idea being that we are a global company. We do have records all over the world. We do make submissions all over the world.

How do we have the final signed reports, not the data that backs them up, the signed reports, available for our regulatory folks to combine and make the necessary submissions to Asia,

Japan, Europe, wherever, so they'll have ready access to the signed, certified documents?

That's just for one little case. I'm not addressing the raw data and the other things. With the records management system, as we begin to do it, it does have the features that you can do audit trails and the fact that you can maintain versions and you know who checked it out and who checked it in and if there were changes made between the two.

Those kinds of things will work at a larger organization,

but they will not work for the low analytical lab, the field contractor, those kinds of solutions won't work. There has to be recognition that I will do things at certain levels, I cannot do them at other levels, simply because of cost. 15 Robin Roberts: I wonder, what's the experience of people having corrupt data input through the system from the beginning as opposed to having it manipulated after it's been entered?

Joe Retzer: Well, they shouldn't speak up if they've manipulated the data.

Robin Roberts: We'll erase your name, then, after input.

Kenna Study: This is Kenna Study with the state of Florida's drinking water program, and we did prosecute labs for dry labbing. Typically, they didn't have data to support it,

and it took a person interceding and reviewing the data to figure that out. It had nothing to do with what was in the database, other than the fact that they got the paper that came in. They looked at it, they became suspicious and did an investigation. It was at the input site, not the transfer of the data or anything like that.

Now you could argue that, at the time they went to the lab and they didn't have the records to support it, that maybe something of a record keeping nature was completely missing and played a part. But I don't see these criteria as being supportive for that kind of an investigation at all. I don't see the difference.

I mean, it would be similar to when we get the data in on paper currently and it's got a signature on it from the lab, us 16 saying, " well, we can't accept that. You need to show up. I need to see I. D., and all this other kind of stuff," is more parallel to what you're proposing here.

Joe Retzer: I think, again, some of what we were doing here clearly reflects the compliance concerns of some EPA compliance folks, and one of the areas that they have had a number of cases over the years in are, for example, in discharge monitoring reports, where there has been some things like stuff whited out on the reports and data manipulated after the initial laboratory work has been done. So there are a number of examples of midstream data manipulation as well as the kind of problems with data entry.

Don Whaley: This is Don Whaley with Navajo Refining again. Not to say that there's not changes under the QA/ QC process, there are times. We have an internal lab. They run GCs. They input data in columns into a database, then they convert that to a spreadsheet and you can obviously see they entered a number in the wrong column. That's part of the QA/ QC review you go through before you generate reports.

I guess   as far as   if fraud were widespread, there wouldn't be so many companies turning in reports showing noncompliance So I think, basically the integrity's out there.

Not that there aren't some fraudulent cases going on out there. 17 So I don't think you should punish the whole industry for the few bad apples that are out there.

Joe Retzer: Well, I guess the point, I'm not an enforcement person myself, but we spent a lot of time talking with them, taking sort of many of your arguments to that group.

And I guess they would say, even if it's 98 percent or so, or whatever, 99 and a half percent, whatever the actual compliance rate is, the problem is that there are. In fact, people who do send in fraudulent reports and, in fact, you need a strong deterrence from doing that and one of the ways of providing that deterrence is to provide a regulatory scheme that enables you to enforce against them if you catch them.

And one of the particular issues that we haven't really talked about a lot here, is the fact that in some of the cases,

the fraud involves not just administrative cases, but criminal ones. And in a criminal case, you really have to be able to track the fraudulent activity back to an individual.

And that's part of the reason for things like an audit trial, to be able to say, I want to know who, what individual at what particular time, may this manipulation, this fraudulent manipulation of the data, whether it's at the front end or somewhere in the middle. 18 And that's a whole sort of different ballgame than other situations. For example, is anyone here involved in TRI reporting? TR reporting, you send your report in. TRI sends you back a statement that says, " Here's the report you sent us,

take another look at it, we think you might have had a problem in this area," or " your piece here translates differently than what you sent last year, are you sure that's right?" And you get sort of a second chance to send the data back in.

So there's a variety of different kinds of reporting circumstances that people operate in their terms of compliance with EPA. But a lot of the reason for the thinking behind these criteria, as well as on the reporting side, the criteria there, is that there are criminal statutes that govern fraud associated with reporting to government agencies, and that means you have to be able to track it back to an individual.

And on the electronic side, that's what's tough. On the paper side, you at least have a signature on paper that people can go look at and see if you can verify the signature. On the electronic signature, you know, how can you be sure it's not a dog, you know, a pet sending you the info? There's this famous New Yorker cartoon.

How do you verify it's the individual, especially if you're talking about a criminal fraudulent case and you have to 19 convince a jury that this person intended to send this information fraudulently, and that it was that person and not somebody else in the office who had access to that computer who sent that data in. And that's kind of a lot what's behind the requirements you see in this rule, both in the record keeping side and the reporting side.

Mel Swoboda: Joe, this is Mel Swoboda of DuPont. I'm going to take off the DuPont hat a minute and put on an old regulator hat. It's been years ago, but we were dealing with paper and you could always, I guess, even if you're in this situation, if you've got a problem, there are things that show up in it that you're going to be able to track that identify you've got a problem.

To think at this time, even if you've got a paper signature, you've got a good signature, I think, with some of the things you've see as far as what you can do with a Xerox machine or a good computer, those days are over, too. I mean the reality is, like my dad used to tell me, locks are to keep honest people honest. If somebody's crooked, they're going to do whatever it takes. Whatever you put that record up there,

they will get through and manipulate it.

Now the question is, what is the true value to what we're trying to protect? Which is environment. And that's the hard 20 question. And I guess I have a hard time seeing where we're doing anything to truly spend a lot of dollars on something that has a lot of value, from an environmental standpoint; I mean straight environmental.

And we're just stacking a lot of things. If I have to do something like this, I'm going to spend a lot of money on it that I could be applying in other places environmentally where I need to do some work.

Robin Roberts: A few more comments and then we should break for lunch, about five minutes early, why don't we make it a one hour and five minute lunch and return at the scheduled time of 1: 15. So 1: 15, we'll see you back here and we'll see you promptly at 1: 15.

What I'd like to do now is find out if over lunch, in your discussions, maybe you had any further thoughts about the general criteria discussion that we concluded before we left that you'd like to say now, that upon further reflection comes to the surface.

And then we'll move on to the 2: 15 to 3: 15 item there on alternative approaches to e record standards, keeping in mind that this is an information exchange kind of an options development conversation we're trying to have and that any 21 alternatives that may be identified by you aren't anything that are being considered as possible implementation items.

I have one comment here and then I'll come back to you,

sir.

George Britton: George Britton of Aventis CropScience.

In thinking about all this, it seems to me there are two levels of risk analysis that need to be done. One is, what does all this have to do with a particular program in terms of the risk of the environment, what is the cost benefit in there?

The other thing is recognizing that each of the current programs require different kinds of records, for example 160 has 60 different kinds of records required to be kept in one form or the other and most of those are not submitted to the Agency under formal reporting purposes.

For example, one of them is the quality assurance master schedule, which is literally a listing of studies and the status ongoing in the facility. It's never submitted to the Agency. It is viewed, for their benefit when they come in, to say what's ongoing, what's the status, what the head counts are and those kinds of issues.

What's the real risk of either fraud or anything else if a particular version of that was misplaced, lost, or damaged in some fashion? I don't think that's a very big risk in the 22 whole scheme of the environment of what's going on. Yes, we'd want to do it, we could regenerate it from the protocols that we are required to keep within the QA/ QC unit.

The other thing, an example of a record that would be in an electronic format that's required under the predicate rule is, for example, that the archive be indexed. Yes, we've chosen to do it electronically, simply because it's convenient to do it electronically. Right now, in our particular case,

I'm tracking over 3,000 studies. If I had 3,000 notecards, if you asked me to find all the pieces of a particular study, it would take me some time to go manually through it.

If I use an electronic database, I can find it much quicker and make it much more available to you when you come in to audit. But even with that, I can still regenerate it, even from the paper, if it were damaged to the point it was irreversible.

But in the whole scheme of things, does that really affect the environment? The answer is no. In all these records,

there doesn't seem to be an acknowledgement that there are records like that that have different policies, that have different risks to the environment or different risks, in terms of good business, if you lost it, fire damage or whatever. 23 There are two issues, first, what it does in terms of protecting what you discharge and put out, protecting the environment and what's the cost of doing it, and the same thing, what does it really do in my normal practice if I lose a particular record? Those are cases that don't seem to be put forward in this and need to be addressed.

Jim McFarland: Jim McFarland from IR4. You'd asked earlier about holes that need to be plugged, additional legislation or rules that need to be made in order to help this process of not only CROMERRR but record keeping in general, it sounds like, and electronic record keeping in particular.

The discussion to me has pointed out that there is not direct correlation between electronic and, maybe some, but there's not a direct correlation between electronic records and paper records, they are different and there are different considerations.

One issue that I touched on earlier and has been touched on by other people in the sense of, the suggestion was made to draw a line and say, from this point forward, CROMERRR applies.

At that point the suggestion was the report that's submitted.

The data and the information previous to that CROMERRR wouldn't apply to. Well, I'm not sure that's workable. 24 But one area that I see, especially in my realm, which is [ the Federal Insecticide, Fungicide, and Rodenticide Act] [ Good Laboratory Practices], there needs to be some sort of line drawn in the sand, so to speak, at the definition of original data. We have an interpretation from Frances Leem at [ the EPA Office of Enforcement and Compliance Assistance]. Hers is very much along the lines of CROMERRR. She believes an electronic signal is the original data.

And this has been a matter of some discussion and a little bit of controversy since I started with the [ Good Laboratory Practices] program in ' 93. The first meeting I actually went to was an EPA meeting in Washington, D. C., where the host of the meeting from the EPA made the statement that the original signal was the raw data. Well it didn't fly very well.

As I know the [ Good Laboratory Practices] right now, it's not specified either in the guidelines, the automated data guidelines, or in the regulation itself what the original raw data is in a situation where the data is being generated electronically. It's pretty cut and dried in some situations,

if you record a number, that's the original raw data. The guidelines say that in the case of a balance, say, that has a printer attached, if you have no direct input into changing or modifying that data, the original data is that printout. 25 If you put it into a spreadsheet, that's a whole different matter. I think this is an area where there needs to be some additional input as far as rules from EPA or some government entity to define what original data is.

Robin Roberts: Other concerns about data, what is original or raw data, or the whole notion of two kinds of risk analyses?

Unidentified Female Speaker: Getting to the contract lab issue, how do we separate the data, how do we get the actual original data, it was mentioned before lunch, there are a lot of proprietary issues involved. To give a very simple example of what happens all the time, a typical environmental contract laboratory will analyze multiple clients' samples in the same analytical batch. And that's standard practice, it's just economically efficient for them to do that.

But then they have to separate that data and give the results to the individual client, and they may print it out.

You get the hard copy from them, but in that whole electronic sequence, they can't give us all that data, because they can't give the other client's data, proprietary reasons of confidentiality. So they have to separate that out. There would be gaps in the electronic sequence. I don't know how those would be handled, how we're going to address those 26 issues, and any time that you separate things out or go through those other processing steps. There's always a chance for error and loss, of course, because it's basically, I guess you would consider it like a transcription of data. You're transcribing data. So those are some of the other issues concerning the contract lab issue.

Robin Roberts: Great, any other comments or thoughts on this, or that came to mind as you were having lunch?

If there aren't, we can come to the next section on our agenda, which is about alternative approaches to electronic record keeping, and to some degree we've spoken to this issue already, but that shouldn't stop us from talking about it some more.

Some of the questions you might want to think about in this area are, are there recognized industry standards for maintained electronic records to assure integrity and reliability? What are good alternatives to audit trails?

Audit trails, that's a complicated thing. We've discussed that a lot of the currently available software that's widely used doesn't have an audit trail component and if this would be a difficult thing to build in.

So basically this is an opportunity to hear from you about alternative technologies that might satisfy record keeping 27 provisions, not of CROMERRR, although that would be great, but also within your own corporate environment or government environment. How are you dealing with these questions of data integrity and data nonrepudiation? Are you going to make me work really hard on this one? Well, are there, then any comments from the panelists on this one?

Joe Retzer: Yes, I'd also just like to hear if anybody has any additional examples of current regulatory ambiguities or problems in the record keeping areas, generally.

Robin Roberts: What about Texas?

Unidentified Male Speaker: Well there's one that's been well documented and been very well known for a long period of time. It gets into the difference in part 160 and 169, books and records for [ Good Laboratory Practices]. The [ Good Laboratory Practices] does allow, that's part 160, allows for verified or certified copies of raw data, it will substitute for the original raw data.

Books and records, 169 Part 2K, specifically says that a producer must keep the records for as long as the product is registered and as long as the producer is in business. The original record.

This has always been a conflict between the [ Good Laboratory Practices] and books and records. Hopefully, with 28 the previous, about 1998, where they were going to revise the [ Good Laboratory Practices] to be in concert with the OLCDs and books and records, some of these things were addressed in that proposal. I don't remember the detail to that particular thing, but that's a particular case where one program will allow, or part of its program will allow, copies when the verbiage in the other says the originals.

Joe Retzer: And do you take that to mean that if the original was electronic, then you'd have to keep the .

Unidentified Male Speaker: Yes, whatever the original is.

And by using this extreme definition of everything that's been maintained, created, dah, dah, dah, I'm in violation, today even, from that. Even before this   books and records, there's some ambiguities within [ the Federal Insecticide, Fungicide,

and Rodenticide Act] itself, where those two items, one allows copies, one doesn't say, it's not clarified what's meant in both of those cases.

That's just one side of the house where electronic records   what we attempted to do, the idea is to maintain number one up here, the generation of accurate records, is we choose to have kept, from the electronic systems, the chromatographs are signed and dated. The original one, the first one off the machine, is kept, regardless. 29 By instrumental analysis the peaks may not be exactly right. Maybe the baseline windows are not exactly right. We may reprocess it four or five times. All the in betweens we've chosen not to keep, but we've taken the one that we will use as the basis of reporting, that's signed and dated, both the very first one that was printed, with all its errors, and the one that was accepted, are both kept. We keep them on paper and the access to that record or records becomes narrower and narrower as you get toward the reporting process.

Once they get to the archive point, basically it's the one or two people in our organization that have access to the archive who have that information. We have chosen to image all the information, all that's kept in the study, file all the correspondence, all invoices . [ end side A]

[ Side B]

  the CDs, and we always can go back and check the electronic image of the CD if you really wanted to go. If this is really what's on there, you could always go back to the original paper and actually check it against it. You can actually go back and, because those CDs, when those scanned images are done, that's the date on the record. So you know when it was done. But once you see the CDRW record, you're not able to go back and add it, or change the record anywhere. 30 Those are some of the kinds of things that are going on to address a lot of this. While there's not an audit trail per se, on the paper you've got the signature side, but as you get from the fellow who generated it to the study director to the QA/ QC unit. To the archive, the access to the records become much more narrow, and basically once it's in the archive, it never sees the light of day again.

That's one reason for imaging it, because you now have the record available, the image of it, that your people can get information from. But the original, which we define as a substitute, verified copy of the paper, is now available for everybody else, it's away, safe, and secure. Policies and procedures as opposed to a rigorous technology force [?]

Robin Roberts: How many of you work with regulations that have a criminal component to them? And how many civil? I'm trying to get at the whole issue of risk analysis. This onesize fits all seems to be geared towards regulations where you're trying to have data that will stand up in court beyond a reasonable doubt, and there is some notion that's too heavy a burden or too heavy a burden for some of this data to carry,

that it's not worthy or up to that kind of status. And I'm just wondering how many of you deal in programs where you don't 31 have that kind of burden and how many of you do. Also trying to figure out a way to get you to talk, but .

Any other ideas about alternative approaches to e record standards and how it's being done?

Nancy Freshour: Sorry, I'm going to digress, Nancy Freshour from Dow Chemical again. We talked about maintaining the data in these systems, if you talk about original data, my experience when, as in the laboratory when we were using, say Access Chrome or Turbo Chrome data systems, you're collecting metagraphic files and at some point, that system is only going to hold so many files. You have system managers, they do backup tapes, they save the tapes, they save all that, but that's not the original, the original is in the data system.

But after a while the system can't hold it, it slows down,

it can't even collect data because it has no place to put it or it's having trouble finding the places to put it, so you have to delete the data from that system, migrate it to something else in order to keep operating your system.

So I don't know how you're going to be able to keep those original, electronic records, unless you fill up a system, then implement another system, it's technically impossible to do that and keep running. 32 Robin Roberts: I even have a question about this original data. Is there an industry standard which says it's okay, it's generally recognized that you're deleting data, because progress, the idea of staying in business?

Nancy Freshour: Well, we print out the chromatograms because it's under [ Good Laboratory Practices]. So we have that and we have the tape backups, but we don't have the original electronic files.

Robin Roberts: At what point is it acceptable, then, or is there a point at which it's no longer, it's generally recognized within the industry that it's unreasonable not to have an electronic record of data? Or can it vary from process to process, industry to industry?

Basically what you're saying is that in industry there's this reality that it goes to paper and what I'm asking is, is there a generally recognized point at which it goes to paper or is this a management call within all the corporations that have to deal with this?

Nancy Freshour: I guess from our experience it's business needs and regulatory, because if you're looking across the business, not just environmental, Dow provides a variety of tools for us, both software and hardware tools, but the businesses have to make a decision and the owners of whatever 33 that is have to make decisions on what they need to meet their business needs and their regulatory needs and implement it that way. So there's no quote, unquote " one size fits all" standard even within our company on how you're going to apply it,

because not everybody is governed by, say, environmental, some people are governed by something else. So I guess that's not really an answer to your question, but that's just the reality.

Bill Yancey: Bill Yancey with BP. We have several means of collecting real time data that we're required to  In the cases of our continuous emission monitors, we take six second,

in other words, every six seconds, a carbon monoxide reading,

and to calculate a rolling average. And as you can imagine,

after a few months the data, we tried, we filled up a few file boxes and filled up a warehouse at the end of five years, but the electronic storage simply was not able to be sustained.

So we kept the paper reports, and that's where this warehoused amounts of rolling averages of carbon monoxide came in. But in the instrumentation that we use for environmental reporting purposes, it goes all the way to things like flow meters, which we use for waste water discharges. And most of our flow meters read every fifteen seconds, and of course continuous pH analyzers, those read about every ten seconds. 34 Again, the electronic storage of all these data bytes is just unpractical so we rely on the printout for that month's worth of data, and that's what we submit, in some cases electronically. But the records that are maintained electronically are very, very short lived. Thanks.

Robin Roberts: Any comments? Well then why don't we just take a look at this next discussion here again. It's something that's come up, but I'm thinking that perhaps you may have more comments about it as well. What special issues are raised by CROMERRR criteria for long term archiving? We've somewhat covered that.

Do you have a question or a comment?

Pat McFadden: Pat McFadden from Gustafson. When you look at long term archiving of electronic data, there are so many issues to consider, that it really is pretty overwhelming.

We currently have several different systems, and recently we tried to pull up some data from 1998. The system crashed when we tried to pull it up. We were going to reinstall the software, the software from 1998, at that point, was still old.

It wouldn't go onto a computer because it had to be a 386 or less, it had to run five and a quarter inch floppies, so there was no way for us to get this data back. The electronic files are lost forever. 35 And that's only three and a half years ago. So you think about, if you're going to have electronic files, how are you going to migrate the data? If you migrate the data, how are you going to migrate the audit trails? How are you going to migrate the meta data? Who's going to be responsible for checking all these?

Our quality assurance guy is not a computer expert. That means that we have to hire somebody new to be in charge of this whole new process of migrating data to new systems. Plus, we have to be able to afford new systems and afford new people,

and it becomes very complex.

So we have old data that we can't look at, because we can't bring it up on a system anymore because we didn't think to migrate it, because we thought it would still be good.

This will be full time jobs for lots of people, we're a small company, but I'm thinking you big guys out there, you have tons of data that's on, hopefully, on CDs or whatever, and you're going to have to migrate them to your new system. Let's say Chem Station totally revamps how it works and Agilent [ Technologies] comes out with brand new software. So you're going to have to take your old software, every single residue study you've ever done, every single sample you've ever analyzed, you're going to have to migrate it to the new system, 36 so that when that EPA auditor comes in to look at your study.

You can bring that file up electronically, because,

theoretically, we're going to have to bring up our electronic files as well as our paper files to show them we've been in compliance with CROMERRR.

We're not sure how that's really going to work, are they going to bring their copy of the software with them on their computers or how that's going to work. We haven't even worried about that problem yet.

But the problem also becomes, then, do you have to save your systems? Are you going to have these legacy systems, or you could have a whole archive full of computers. You know,

here's my little Tandy, I've been saving it just in case you guys come in and look at this. We've gone through lots and lots of different computers, and people keep changing things.

Bill Gates always makes new products so that he can sell more things and he can make more money, which makes us have to upgrade all the time, and every time we upgrade our system,

then we're going to have to validate the system again. So we're not only spending money on migrating data, we're revalidating systems, we're doing verification. All of that is very expensive. 37 If the standard was, make the paper copy, put it in the archive and we'll be happy with that, then our costs would be reduced and I think that's something that needs to be looked at, because saving electronic files for the life of a [ Federal Insecticide, Fungicide, and Rodenticide Act] registration is almost impossible.

Donald Whaley: This is Don Whaley with Navajo Refining.

One point that you brought up as far as the archiving, and the changeover in the technology, even the databases, we are some of the fortunate ones that were able to do a 114 request last year, about this time of year, in fact, and some of the lab data that we went to retrieve, we actually had to go through efforts to load historical databases on, major efforts to get that information.

But EPA requested spreadsheet format submittals, and at that point in time they didn't ask for any kind of validation,

any kind of auditing, yet they were willing to accept that in the context of enforcement issues.

Robin Roberts: So that's past experience with old data being good enough in an enforcement case, and no issues were raised about its integrity or authenticity, and the court held it to be nonrepudiatable, similar criminal cases? 38 Don Whaley: Ultimately a consent decree came out of it although there was never any action taken as a result of the 114. There were some other issues that were settled with the state and then our partnership announcement that came out in December. I guess that would be considered a civil action.

Robin Roberts: Anyone been to court with any data here?

Great, it seems like the notion of having to keep a small museum of hardware in your corporate garages to read old formats that Bill Gates has forced us to move away from. Are there any other concerns about how you would deal with this old data that accumulates in any long term archiving?

Evi Huffer: I just wanted to mention something, we keep coming back to the conversion issue and being allowed to,

particularly in the [ Good Laboratory Practices] world, to keep printed copies of your raw data.

EPA has been doing some additional research in that area.

We have a contractor who is working on a study right now, we'll be getting a copy of it next week. It's looking at converting electronic records to paper records and the feasibility for meeting CROMERRR criteria and what software is out there that's available, etc.

We will be making that report available in the docket, so check the docket in a couple of weeks. We'd like to see the 39 report first, and get the contract and clean it up, if there are typos and things like that in there, but hopefully, even in the next two weeks we'll be able to have it out on the web for you and you'll be able to find it on our website, which is in the presentation package.

Mel Swoboda: Evi, one question. Mel Swoboda, with DuPont. This kind of drives to a piece that really, I know you guys are not involved directly in, but the hard look that needs to happen on what do we really need to keep? I know [ the Resource Conservation and Recovery Act], I understand just this past week came out with a notice about looking at record retention.

The question then comes back, do you really need to know,

the classic one I always use, is do you really need to know in 1979 on January the 31st at two o'clock in the afternoon what the flow rate is on my injection rod? Is that really an important piece of information at this point in time? Or do you need to know what the annual volume that I injected, which you then use in your modeling for another EPA program to model out ten thousand years of where the waste goes?

So it goes back to pushing back to the different departments and saying, let's do a hard, hard look at what am I required to keep and what is the value, and even beyond that, 40 is the data any good? Because as we go through systems, if I look at, and I'm just going to use water quality as an example,

if I go back 15 years and look at the water quality data,

that's garbage. For purposes of doing any compliance today,

it's nowhere near the technology of detections that we have today and it's useless information at this point.

So it's really one of those questions that makes a lot of the things that we're talking about here kind of moot. Now [ the Federal Insecticide, Fungicide, and Rodenticide Act], I understand, is some different things, but I would still venture to guess that there are pieces in that, that the question is still a valid question.

Evi Huffer: Good point.

Nancy Freshour: Well, I agree with the previous comment.

This is Nancy Freshour with Dow Chemical again. I did have a question, Evi, on your comments about the contractor looking at [ Good Laboratory Practices] conversion of electronic records.

Evi Huffer: He wasn't specifically looking at [ Good Laboratory Practices] conversion. The contractor is looking at the feasibility of meeting the CROMERRR criteria, when you take an electronic record not necessarily one under the [ Good Laboratory Practices], and convert it to a paper record, and vice versa. 41 Joe Retzer: You mean the criteria in terms of preserving the meta data.

Evi Huffer: Preserving the meta data, the audit trail,

that type of thing, the date/ time stamp.

Nancy Freshour: Okay, and then what would be available,

and then those would be specified so that we would kind of be limited to what was .

Joe Retzer: No, we're just doing some research on the topic.

Evi Huffer: This is just some research to see what's out there right now, what technologies, what software is available,

what the cost is, etc.

Nancy Freshour: Okay, because it's not just [ Good Laboratory Practices]. [ Good Laboratory Practices] is one of the areas, of course. It's all recognized by everybody here,

that that's where a lot of conversion to paper occurs, [ Good Laboratory Practices], [ the Federal Insecticide, Fungicide, and Rodenticide Act], and [ unintelligible]. And in these continuous monitoring situations, where we have to collect minute to minute data, and are required to do that, because the technology is there and the Agency recognizes that the technology is there. So they want us to collect minute to 42 minute data, whereas years ago we couldn't do that, we don't necessarily keep that on paper.

Evi Huffer: Right.

Nancy Freshour: So even if you have a feasibility study and you can convert that to paper, I don't think we're probably going to want to convert that to paper.

Evi Huffer: And this, I think, goes back to the point about looking at the scope of the rule, and our definition of what an electronic record is. I'm not   when we wrote this rule,

I don't think it was our intention for these volumes of secondby second compliance data to be included in these electronic records.

Joe Retzer: I guess the question is, what does the current regulation on the books require you to keep? And that again, what we're trying to do is say, what does that require you to keep, if the current regulation requires you to keep that data, what we're trying to address is the question of is it easier for you to keep it electronically, if you have to keep it X amount of time, or is it easier for you to keep it electronically but print it out. I mean then print it out after a certain length of time.

And that's the issues we're trying to look at here, is that if, in fact, we don't put in a provision that says once 43 electronic you have to keep it electronic, we're trying to do some research on, basically, do existing approaches to printing out the data preserve what's necessary?

Evi Huffer: Could I just ask a question here? Let's assume that CROMERRR is out on the table and we haven't defined what an electronic record is, that definition is not there.

Would you then, in reading the rule, would you interpret the rule to apply to this type of data, without that definition?

Nancy Freshour: Could you repeat the question again? I don't understand what you're .

Evi Huffer: For your particular program, let's just say we take out the definition of electronic record. It's not in the rule, it's not on the table. And what we've given you is,

we haven't defined what a record is, we're assuming that you know what a record is because it's defined in the predicate regulation. So we've given you this criterion that an electronic record retention system has to meet. Would you assume, in reading the regulation that this data that you're talking about, this monitoring data, would fall under that record retention? So it's more than just the definition.

Nancy Freshour: Nancy Freshour, Dow. Yes, you would have to assume that, because anybody who's enforcing it or inspecting against that would assume the most conservative 44 interpretation of it, and so yes, you'd have to take that route.

Terry Salem: This is Terry Salem with TNRCC, you'd would also have to assume that because the underlying predicate regulation, in most instances, I think, requires that you report that monitoring data in some forms. And so if the predicate regulation is already assuming that that data is, in and of itself, a record.

Evi Huffer: And is being kept?

Terry Salem: And is being kept, Title V is a perfect example, deviation reporting, every bit of that second

bysecond data on that sim is feeding an emission standard that you have to report every little blip, and then you have to go in and certify continuous or intermittent compliance.

And so I think the underlying issue here is more than just how do we keep our data, it's how do we, and I'll say we as environmental regulators, want to structure the picture of environmental compliance, right? What is it we care about,

what is it we're trying to get to, and that goes back to the whole theme earlier of what is it that we really care about?

Data wise, from an environmental compliance perspective, what are you trying to target, what are you trying to attack? Where are the pressure points for you as an environmental regulator 45 to ferret out the information to keep everything in line and whole.

And right now the problem with doing this as a broad swath approach, I think, is that every one of those predicate regulations was never created with this in mind. They were targeted to get the most data possible, to put the most technological constriction out there that was possible, you know, get the most data, get it as timely as possible, get it all, and from that you would see the whole picture of compliance.

And so those underlying regulations are not structured in a way that makes it easy to then back it out and say, well what's really the record of importance from a record keeping perspective?

Sue Chi Shen: This is Sue Chi Shen from Gustafson. I'm in the record [ unintelligible] and I have another situation which I want to share with you. Because from time to time, in the regulatory department, we do interface with EPA before we are ready to set our [ unintelligible] trial, say for field [ unintelligible], say you need twenty locations, but then,

based on the preregistration meeting, based on lots of rationales. Yes, sometimes the EPA says, say Dr. Smith, the head of her division, says yes, okay, based on your rationale, 46 yes, we agree that you are able to reduce your regular locations down from twenty to ten. Okay?

Historically what we do is, yes, we have a communication back and forth by e mail and in the end you will send me a note that says, Sue, we agree, ten locations will be enough. So what we do is, we make a hard copy and I will send that hard copy to Pat, she is in charge of our [ unintelligible] trial   when she finishes her ten location report.

I think it's a bit nice that she actually attaches that communication agreement of Dr. Whoever for the head of her division, the justification why we only do ten instead of twenty. I think that's very important, you know, when you review packages years from now, you'd say that is important.

Okay, this kind of information, again, go back to what the gentleman says, what are required under these electronic records. Do I have to actually, when I negotiate this kind of agreement for the last three months? To come to the final conclusion, is it important, do I have to show all that, my email blah, blah, blah, under this new requirement?

That's my question. This is how do you interface the information communication log between the records people from headquarters. It's important to be included. This is something, I think, that needs to be defined. It has a lot of 47 impact on how our computer lady, over at headquarters. She's going to panic. She'll say, " Oh, God, I've got to worry about your computer instead of worrying about Pat's computer," blah,

blah, blah, this is just another situation.

Robin Roberts: Any comments about this problem? Sure.

Sue Chi Shen: I do have another question, we're talking about the record keeping requirement, what are EPA persons required to do at this point? Dr. Smith gave me the okay two years ago, hey, I send you an e mail, so you can get by with ten locations. Is he required to keep that communication under his file? Is he required to do that, or do you rely on the rest of us to keep every piece of information, say you told me so and so two years ago. Is he required? I can create something from Dr. Smith that shows he told me to do that. Are they required to do it over on their side in case,

unfortunately I lose my paperwork, I can go to him? Are they required to do that, that's my question.

Joe Retzer: I guess that would really depend on the standard operating procedures of that particular group, but emails of the sort you are talking about at EPA are official records, so we have to keep those. We can print them out,

however. 48 Evi Huffer: Actually we have to print them out, we have to print them out at this point.

Robin Roberts: If there are no other questions or comments. Yeah, we have one here.

Janey Shih: Hi, this is Janey Shih from Lockheed Martin.

We are talking about this data from a laboratory, for monitoring data, kind of directly environmental monitoring data. But like for air emission inventory, a lot of the data is coming from our inventory system, another system which, I think, was created maybe fifty years ago, which for sure does not meet the CROMERRR standard. In this case what are we going to do? Are we going to have to redo the whole system for the whole company in order to be able to comply with CROMERRR?

That would be a question, absolutely, that's what we are really worried about.

Robin Roberts: Any response to that?

Joe Retzer: Well I'm not sure I understood the question exactly. If you're saying that part of your system does not meet the CROMERRR requirements, if we were to go ahead with the rule as it stands now, which we're strongly suggesting to you that we are not, but if we were to do that, yes, you would need to come into compliance with those requirements. 49 Janey Shih: So you are talking about all of the systems where our data is coming from has to also meet the CROMERRR standards.

Joe Retzer: You're saying if the data comes from somewhere else, like another company or something like that?

Janey Shih: No, our company's other systems.

Joe Retzer: Oh, your other data systems.

Janey Shih: Yes, we have many, many systems, we don't only have one environmental data management system.

Joe Retzer: Again, it sort of depends on what the underlying regulations define as the record that you have to keep, okay? So if the air, water, FIFRA, whatever regulation defines that as a record, for that program .

Janey Shih: For example, equipment maintenance records,

that would be kept in our maintenance system.

Joe Retzer: Right, if keeping those records is a requirement under one of EPA's regulations at this point, as CROMERRR is now written, it would apply to those records.

Janey Shih: Our IT people will be really busy.

Don Whaley: This is Don Whaley with Navajo Refining. One thing I was curious about, if we go to electronic reporting,

will this lessen the phenomenon of the EPA inspectors showing up and wanting to see copies of all our reports? Because 50 historically, I often wonder why we send in reports at all,

because EPA never seems to have them.

Evi Huffer: That was a comment, right?

Joe Retzer: Well I mean, obviously one of the advantages of electronic reporting is that the data goes into EPA's systems in close to real time, so then they should have access to it as opposed to mailing in a piece of paper, and it may or may not get into the database at some point.

Evi Huffer: And particularly under the system that we're envisioning with CDX, we'll be archiving a copy of the data as it comes in.

Joe Retzer: Right. But also the vast majority, however,

of direct reporting now is actually to the states as opposed to directly to EPA. So if you're talking about a report on the delegated program, but an EPA inspector, you have to realize,

the data goes to the state, gets processed into the state system, and then gets updated into the EPA system on some periodic basis, which may be quarterly or semi annually, and then it gets into the EPA system. So, obviously sometimes there's some lag.

[ Question from audience, not spoken into microphone.]

Joe Retzer: Then they should have it, yes. 51 Don Whaley: These are reports, I was thinking, that either go directly to the regional office and what I would think would be an enhancement from the electronic reporting would be, if EPA would take advantage of it, the accessibility of the data so that an inspector can pull whatever reports he needs prior to going on site and saving us from going through and digging up the data and making additional copies at that time.

Evi Huffer: It should certainly help to improve our own internal business processes to that effect.

George Britton: George Britton, Aventis CropScience. Why do we even need this? If we're submitting this information to you electronically, and you're keeping it and archiving it, why should we even bother to keep it? What is the record for having to do it, other than with any reasonable time for audit?

Evi Huffer: The records that we're most concerned about under Subpart C are those records, those documents, that are not sent to EPA or to the state. They're documents that are maintained solely by the facility, it's information that's never sent to us.

Joe Retzer: But your question is a reasonable one, if you send us a report or an electronic report, why do you have to keep a copy? That's a good question. 52 George Britton: The other side of that is, for the long term, for the information, there are things you can do to collect information, you can convert it into a storable medium that is more stable than magnetic kinds of things. Keeping functionality for a few years, like three years, like the IRS does for normal tax returns, unless you're under investigation,

in which case it goes to seven.

Keeping a system available for a three year period after you quit using it as an active system is a doable, at least in the [ Good Laboratory Practices] environment, it could be a doable thing. But keeping it for the FIFRA, forty years, is a practical impossibility. When was the last time you read Remington, Ray, and UNIVAC programs? When did you run one?

The FIFRA community has had projects registered since 1950.

So therein is the problem with long term archiving. If you moved it to some medium, whether you captured it electronically to an image file, there are several ones out there, then you have a chance of keeping that a little longer.

But for the functionality audit, for the long term retention,

it's a real problem.

It gets back to the same idea with contract labs that supply the sponsors and the environmental monitoring data.

Sometimes the companies that are contracted for the work don't 53 even have, or couldn't even read the information if they had it. The summary report would be the important thing that is kept long term, things that you sign and put your signature on.

That's what you're really, ultimately, certifying, whatever you would normally put a signature on, the equivalent,

electronically, is what you would want to keep. That's what you would fall back to in a litigation scenario.

Joe Retzer: Well I think you're raising kind of another issue here, which is are the current record retention requirements appropriate? What should be kept in FIFRA for what length of time, what should be kept for a particular water regulation, what's the appropriate length of time? Now that's not really an issue that we tried to address in CROMERRR.

Again, we're just trying to build on what the underlying records are, or record retention requirements are, that folks in those programs have come up with over time.

And it may be that if that part were better defined, then the electronic record keeping part would be a lot easier to address.

George Britton: Agreed.

Kenna Study: I was also going to ask a question about when the information is sent to the EPA, if that absolves us from having to keep it. And then I realized it's already 54 happened a number of times. As a state agency, we send up a couple hundred thousand record per quarter, basically from the drinking water systems, and we do get into back and forth over what we did or did not do.

I was going to say, oh, good, we don't have to keep it,

but I realized that we would keep it in order to prove that we did give you something, so it's probably a moot point.

Joe Retzer: We the state, you mean, might conduct an inspection? You're speaking on behalf of the state of Florida?

Kenna Study: Well, for my program, I don't know enough about all the program areas, but we send up a lot of data every quarter to the EPA database that comes to us from the water systems, and it would be nice to say oh well we sent you that,

now you go do something with it and we'll go do something else.

But I think the reality is we'd keep it, because we do have arguments about what we did and didn't do, so that's our evidence, I guess.

We keep it   I mean, once it's in the database, it's possible. I was looking at some of the things about the audit trails, we have a user time stamp associated with records, but I can't tell you what that person did except in instances pertaining precisely to compliance. Like if they deleted a violation, if our user did that, if somebody in the drinking 55 water program did that, we'd have a record. Or if somebody in the drinking water program went into the data system and changed a result for, let's say nitrate has a standard of ten,

and they went in there and that record, which has got maybe five pieces of meta data associated with it, changed it from .1 to .2, I wouldn't know that.

I would know the day they went in there and who went in there, but I wouldn't know precisely which one of those fields had been altered and how. So I wouldn't know, specifically, on your number eight here, how far you intend to take that.

Joe Retzer: Okay, well first of all, it doesn't really apply to states.

Kenna Study: Okay, well now, now. I mean, not yet!

Well, I see this thing filtering up.

Joe Retzer: Right. I guess the distinction, again, is the relationship between states and EPA isn't a regulatory one,

it's one of partnership and agreement on who's going to do what in a broader regulatory scheme.

Kenna Study: Then perhaps as a delegated program for EPA you could let us deal with the industry?

Greg Nutt: This is Greg Nutt from TNRCC again, I just want to address what Joe said before. He talked about the rosy relationship between the states and EPA on data transfer 56 issues, which could be a whole other day of discussion at least.

On the realisticness [ sic] of a records retention schedule in the program regulations, and here's a little experience about what we've gone through in the TNRCC trying to establish a records retention schedule for our internal records. And typically what happens is that the records management people go to the program people and say, " How long do we need to retain this?" And the program people say, " Forever." And the records people go back, and they say, " Okay, here's how much it's going to cost." And then the management in the program areas say,

" You know, we could probably get by with just two or three years."

And so there's a problem here with having the program staff in EPA realistically address   let me say there's a potential to have a problem with the program people in EPA realistically addressing the records retention schedule,

because they're not tied to the costs associated with that record retention.

Joe Retzer: Right. Well, it's probably something that deserves real attention. Typically the way, when EPA's regulations are done, when people think about the compliance costs and they're analyzed, they don't initially think about 57 the record keeping part as being as high as the technology part, for example, of the compliance, and sometimes probably those things like record retention don't get the same amount of attention   except in a rule like this   as some other parts of the rule might.

And many of those rules go back many, many years and are really based on, again, thinking of things in terms of paper retention, so that's really where I rate that potential issue.

Chris Marlia: Chris Marlia from the South Coast again. I know you mentioned that the states don't come under this CROMERRR program in terms of record keeping, however the states do come under the provisions of having to have a recordsreceiving system, which is pretty onerous. We're going to have to duplicate what CDX does before we report to EPA. I just wanted to make that comment.

Joe Retzer: Okay, let's save that to the end and we can talk about that a little bit. For those who are interested in that, they can stay and talk. I'm not sure everyone is, since we sort of billed this as being about records.

Robin Roberts: We might be at that part where we talk about that.

Joe Retzer: Yeah, we might be at the end.

Evi Huffer: Are we going to move on or? 58 Robin Roberts: Oh, we have a comment here.

Unidentified Female Speaker: When it comes down to the final version, I'd like to see the rule say something about who keeps electronic files when something is contracted out. If,

as a sponsor company, I'm required to keep all electronic files for everyone I've contracted with, it could be a lot of people.

I have twenty field cooperators out there growing field corn for me who may be generating electronic weather data or generating freezer data on electronic systems, or whatever other electronic data they may generate. Then I have a contract analytical lab who's generating all sorts of electronic data, none of which I've ever seen. At this point we accept paper from them as our standard.

I would like the rule to define whether or not I will also be responsible for maintaining electronic files for this contract work. If I'm required to maintain electronic files,

then I'll be required to buy the software to look at them with.

If I contract out a C14 study, obviously I don't have that kind of software since I'm not registered to use radio labeled compounds. So I would have to go purchase the software that they use to pull their electronic files up on my computer. The same with weather systems or any other kind of electronic collection systems that we don't have at our facility. 59 If you start thinking about that, there could be a horrendous cost of being able to look at electronic files in our facility for equipment we don't have. I don't want to have to go and buy software for any kind of chromatographic system that we don't have in our building. So I think the rule needs to define how much electronic files we have to keep from contracted people.

Robin Roberts: And that came up earlier also, about the relationship that many of you have with contractors and what your responsibility is to be able to read the data that they collect for you ten years afterwards.

Mel Swoboda: This is Mel Swoboda again. One of the issues, I guess, with long term archiving that makes it partly a problem too is if you look at a lot of the regulations, and we mentioned one where you're monitoring every second, or you have a record data point every second, every unit or something like that.

And I if I go back and look at some of the regulations, if I look at the water regulations, continuous monitoring, well when they wrote continuous monitoring, they were thinking of a chart with a pen on it and that was all they were looking at,

and not the intensity with which a computer can go out and pick up data. 60 One of the pieces that I could see that you could have a lot of value for is looking hard at, from a historical archiving, the filtering capability, or what you do to set some standards around what kind of filtering, so that if I have a data point, or I have a line of information that may move,

maybe a tenth over a long period of time, is that really important? Should I be picking up a data point, every one of those data points? Should I be picking up a data point every hour, if there's no change in those from an archival standpoint? So that you really don't try and get this massive dump of data on you and you get something that really shows what's happening over a period of time.

That, I think, is something that would be of value for CROMERRR to come in with, with some thoughts and some processes from the people who actually do that kind of storing of data,

on what the capabilities are, what's the best way to do it, and then set standards that are reasonable.

Everybody with computers now, numbers are absolute, if it reads out to seven or eight digits, it's an absolute number.

The number may not be good but to one place. But we keep looking at these numbers as if they're the absolute Gospel.

There's something to be said for the old slide rules .

[ End side B, end of transcript.] 61


[Federal Register Volume 77, Number 229 (Wednesday, November 28, 2012)]
[Notices]
[Pages 71022-71023]
[FR Doc No: 2012-28797]

-----------------------------------------------------------------------

SECURITIES AND EXCHANGE COMMISSION

[Release No. 34-68281; File No. SR-NSX-2012-23]

Self-Regulatory Organizations; National Stock Exchange, Inc.;
Notice of Filing and Immediate Effectiveness of Proposed Rule Change To
Clarify the Operation of the Exchange's Depth-of-Book Feed

November 21, 2012.
    Pursuant to Section 19(b)(1) of the Securities Exchange Act of 1934
(the ``Act'') \1\ and Rule 19b-4 thereunder,\2\ notice is hereby given
that on November 15, 2012, National Stock Exchange, Inc.
(``NSX[supreg]'' or the ``Exchange'') filed with the Securities and
Exchange Commission (``Commission'') the proposed rule change, as
described in Items I and II below, which Items have been prepared by
the Exchange. The Commission is publishing this notice to solicit
comment on the proposed rule change from interested persons.
---------------------------------------------------------------------------

    \1\ 15 U.S.C. 78s(b)(1).
    \2\ 17 CFR 240.19b-4.
---------------------------------------------------------------------------

I. Self-Regulatory Organization's Statement of the Terms of the
Substance of the Proposed Rule Change

    The Exchange is proposing to clarify the manner in which it
distributes the NSX depth-of-book feed (``DOB feed'') to authorized
recipients. The text of the proposed rule change is available on the
Exchange's Web site at http://www.nsx.com, at the principal office of
the Exchange, and at the Commission's Public Reference Room.

II. Self-Regulatory Organization's Statement of the Purpose of, and
Statutory Basis for, the Proposed Rule Change

    In its filing with the Commission, the Exchange included statements
concerning the purpose of and basis for the proposed rule change and
discussed any comments it received on the proposed rule change. The
text of these statements may be examined at the places specified in
Item IV below. The Exchange has prepared summaries, set forth in
sections A, B, and C below, of the most significant parts of such
statements.

A. Self-Regulatory Organization's Statement of the Purpose of, and
Statutory Basis for, the Proposed Rule Change

1. Purpose
    On December 15, 2011, the Exchange filed a proposed rule change
with the Commission outlining the manner in which the Exchange
distributes the NSX DOB feed to authorized recipients.\3\ The Exchange
is proposing to clarify a statement contained in SR-NSX-2011-15, which
stated that ``[t]he DOB feed does not disclose the source of any order
or identify any transaction party.'' This statement failed to disclose
that an ETP Holder can request that their quotations be attributed to
them in the NSX DOB feed (``quote attribution''). The Exchange now
submits this proposed rule change to clarify that an ETP Holder may
affirmatively request quote attribution. Absent such a request, the ETP
Holder's quotations are not attributed.
---------------------------------------------------------------------------

    \3\ See Securities Exchange Act Release No. 66007 (December 20,
2011) 76 FR 81000 (December 27, 2011) (SR-NSX-2011-15). Since March
2012, the Exchange has charged authorized recipients for the NSX DOB
Feed. See Securities Exchange Act Release No. 66511 (March 5, 2012)
77 FR 14450 (March 9, 2012) (SR-NSX-2012-04).
---------------------------------------------------------------------------

2. Statutory Basis
    The Exchange believes that the proposed rule change is consistent
with the provisions of Section 6(b) of the Securities and Exchange Act
of 1934 (the ``Act''),\4\ in general, and Section 6(b)(5) of the
Act,\5\ in particular, in that it is designed to promote just and
equitable principles of trade, remove impediments to, and perfect the
mechanism of, a free and open market and a national market system, and,
in general, protect investors and the public interest. Informing an ETP
Holder that it may affirmatively request quote attribution enhances
market transparency and promotes competition. The Exchange also
believes that the proposed change is consistent with Section 6(b)(5) of
the Act,\6\ which also requires, among other things, that the
Exchange's rules are not designed to unfairly discriminate between
customers, issuers, brokers or dealers because all qualified ETP
Holders, and other qualified recipients, are eligible to receive the
DOB Feed and all ETP Holders can request quote attribution.
---------------------------------------------------------------------------

    \4\ 15 U.S.C. 78f(b).
    \5\ 15 U.S.C. 78f(b)(5).
    \6\ 15 U.S.C. 78f(b)(5).
---------------------------------------------------------------------------

B. Self-Regulatory Organization's Statement on Burden on Competition

    The Exchange does not believe that the proposed rule change will
impose any burden on competition that is not necessary or appropriate
in furtherance of the purposes of the Act.

C. Self-Regulatory Organization's Statement on Comments on the Proposed
Rule Change Received From Members, Participants, or Others

    Written comments on the proposed rule change were neither solicited
nor received.

III. Date of Effectiveness of the Proposed Rule Change and Timing for
Commission Action

    Because the foregoing proposed rule change does not: (i)
Significantly affect the protection of investors or the public
interest; (ii) impose any significant burden on competition; and (iii)
become operative for 30 days from the date on which it was filed, or
such shorter time as the Commission may designate, it has

become effective pursuant to Section 19(b)(3)(A) of the Act \7\ and
subparagraph (f)(6) of Rule 19b-4 thereunder.\8\
---------------------------------------------------------------------------

    \7\ 15 U.S.C. 78s(b)(3)(A).
    \8\ 17 CFR 240.19b-4(f)(6). In addition, Rule 19b-4(f)(6)
requires a self-regulatory organization to give the Commission
written notice of its intent to file the proposed rule change at
least five business days prior to the date of the filing of the
proposed rule change, or such shorter time as designated by the
Commission. The Exchange has satisfied this requirement.
---------------------------------------------------------------------------

    A proposed rule change filed under Rule 19b-4(f)(6) \9\ normally
does not become operative prior to 30 days after the date of the
filing. However, pursuant to Rule 19b-4(f)(6)(iii),\10\ the Commission
may designate a shorter time if such action is consistent with the
protection of investors and the public interest. The Exchange has asked
the Commission to waive the 30-day operative delay so that the proposed
rule change may be implemented upon filing with the Commission. The
Commission believes that waiving the 30-day operative delay is
consistent with the protection of investors and the public interest as
it will provide immediate clarification that ETP Holders may
affirmatively request that quotations be attributed to them, and that
absent such a request, the identity of ETP Holders will not be
divulged.\11\ Therefore, the Commission hereby waives the 30-day
operative delay and designates the proposed rule change to be operative
upon filing with the Commission.
---------------------------------------------------------------------------

    \9\ 17 CFR 240.19b-4(f)(6).
    \10\ 17 CFR 240.19b-4(f)(6)(iii).
    \11\ For purposes only of waiving the operative delay for this
proposal, the Commission has considered the proposed rule's impact
on efficiency, competition, and capital formation. See 15 U.S.C.
78c(f).
---------------------------------------------------------------------------

    At any time within 60 days of the filing of the proposed rule
change, the Commission summarily may temporarily suspend such rule
change if it appears to the Commission that such action is necessary or
appropriate in the public interest, for the protection of investors, or
otherwise in furtherance of the purposes of the Act. If the Commission
takes such action, the Commission shall institute proceedings to
determine whether the proposed rule change should be approved or
disapproved.

IV. Solicitation of Comments

    Interested persons are invited to submit written data, views, and
arguments concerning the foregoing, including whether the proposed rule
change is consistent with the Act. Comments may be submitted by any of
the following methods:

Electronic Comments

     Use the Commission's Internet comment form (http://www.sec.gov/rules/sro.shtml); or
     Send an email to<EMAIL_ADDRESS>Please include
File Number SR-NSX-2012-23 on the subject line.

Paper Comments

     Send paper comments in triplicate to Elizabeth M. Murphy,
Secretary, Securities and Exchange Commission, 100 F Street NE.,
Washington, DC 20549-1090.

All submissions should refer to File Number SR-NSX-2012-23. This file
number should be included on the subject line if email is used. To help
the Commission process and review your comments more efficiently,
please use only one method. The Commission will post all comments on
the Commission's Internet Web site (http://www.sec.gov/rules/sro.shtml). Copies of the submission, all subsequent amendments, all
written statements with respect to the proposed rule change that are
filed with the Commission, and all written communications relating to
the proposed rule change between the Commission and any person, other
than those that may be withheld from the public in accordance with the
provisions of 5 U.S.C. 552, will be available for Web site viewing and
printing in the Commission's Public Reference Room, 100 F Street NE.,
Washington, DC 20549, on official business days between the hours of
10:00 a.m. and 3:00 p.m. Copies of such filing also will be available
for inspection and copying at the principal offices of the Exchange.
All comments received will be posted without change; the Commission
does not edit personal identifying information from submissions. You
should submit only information that you wish to make available
publicly. All submissions should refer to File Number SR-NSX-2012-23,
and should be submitted on or before December 19, 2012.

    For the Commission, by the Division of Trading and Markets,
pursuant to delegated authority.\12\
---------------------------------------------------------------------------

    \12\ 17 CFR 200.30-3(a)(12).
---------------------------------------------------------------------------

Kevin M. O'Neill,
Deputy Secretary.
[FR Doc. 2012-28797 Filed 11-27-12; 8:45 am]
BILLING CODE 8011-01-P