Carolyn Greenwood: If you decouple, whether you decouple or not, one thing to keep in mind is there may be some useful 94 corollaries, I guess, to be gleaned from the acid rain program as well as ambient air quality reporting under their air system. They both have, as I understand it, after records are submitted to EPA or to the states for EPA, there are checks that are done on the data. There's a lot of stuff you can do with electronically reported data that you can't do with paper very practically. So if you're concerned about fraud or any kind of inaccuracies in the data, that electronic reporting step will get you part of the way there if EPA also wants to take or the states want to take that extra step of doing some internal checks. A lot of that can turn up errors that are inadvertent or otherwise. So it does accomplish a little bit of the other goal as well as getting at the fraud issue maybe.

Doug Billings: Doug Billings from the U. S. Chamber of Commerce. The Chamber also supports the decoupling proposal,

the idea. I also want to speak strongly in favor of what Mark Duvall proposed in terms of risk assessment. One observation I would make in that regard is that when this rule was originally proposed, because it was presented as voluntary there weren't the various assessments that were required under [ the Small Business Regulatory Enforcement Fairness Act] and the regulatory impact analysis that needs to be done under the executive order, under unfunded mandates.

I would suggest that if this thing is decoupled and the 95 record keeping aspect is looked at separately, that those analyses be done before the rule is re proposed, if it's reproposed I think that's a way to get at the risk assessment and various other elements, in context with working with industry. Certainly the [ Small Business Regulatory Enforcement Fairness Act] review panel would be another way to get industry input on those, but I think all of those collectively, if those analyses are done, should lead to a conclusion on what needs to be done with the fraud issue.

Chris Hornback: Chris Hornback with the Association of Metropolitan Sewage Agencies. I've heard a lot of comments from companies, Monsanto, Dow, Procter & Gamble. I wanted to just represent or at least give you some thoughts from the publicly owned world, the publicly owned treatment works. AMSA represents over 260 of the nation's publicly owned treatment works. Our members treat the majority of the wastewater generated in this country, both residential and industrial.

We present an interesting case study for this particular rule making. Not only are our members permittees under the [ National Pollution Discharge Elimination System] permit program, we collect information, we monitor our discharges, we prepare DMRs or discharge monitoring reports for our permit requirements. But many of our members, actually most of our members, are also running approved pre treatment programs, 96 which may in fact control a lot of you here in terms of your industrial discharges, indirect discharges to publicly owned treatment works. In those instances, we are also operating essentially a regulatory program. We are control authorities and we would in fact be receiving reports from many of you. We may be receiving electronic reports from many of you.

Many of our members are investing thousands if not hundreds of thousands of dollars in information management systems to receive the information from many of you. In that case, we are not only subject to the electronic record keeping requirements but the electronic reporting requirements, and in terms of the data reception, not only are we sending the data out but we're receiving it as well. So our systems are going to have to comply with those portions of the rule as well.

So we've got a lot to deal with with this rule, and so we definitely support the separation or decoupling of these proposals. We would be interested in talking more with the agency about the reporting requirements and the challenges that those present for our members.

Howard Krueger: This is a follow up point on the last point, which I thought was very good. If we were to create a meeting and schedule a meeting to focus on the reporting parts,

we all must remember, or at least as disclosed in the economic impact analysis that the EPA had done for it, the 62 page 97 report by that outside company. It's either 80 but I believe it's 90 percent of all reporting against EPA regulations goes to states and tribes. That's an overwhelming amount of reports going to this, I'll call it a third party, but it's not the EPA in Washington, D. C. It's these other groups. So not to consult them as part of any sort of review on reporting would be to miss the boat, I think, on getting some very important information. That's just kind of a reminder following up on the very valid point made here. The states have to be brought in because they have the programs that receive the data and when they're getting 90 percent of the information they're really the main player.

Unidentified Male Speaker: I think that's a really good point, Howard, and in fact we have sort of a separate or additional process where we're consulting with states. There's one person here from one of those state groups that we've been talking to that we're working on plans to do that.

Robin Roberts: Are there any other questions about the alternatives? There seems to be a   I hate to use the word consensus, but consensus about decoupling.

Unidentified Male Speaker: In terms of addressing reliability of electronic records, if you're starting with a fresh slate, as it were, an appropriate model to think about would be the model that's presented in the E Sign Act. The 98 electronic signatures act has a provision that requires if a statute, regulation, etc., requires that a contract or other record relating to a transaction affecting commerce be retained, that requirement is met by retaining an electronic record of the information, so long as the record accurately reflects the information and remains accessible to all persons who are entitled to access the statute by regulation, etc., in a form capable of being accurately reproduced for later reference, whether by transmission, printing or otherwise.

The e sign legislation with those provisions is interesting for a number of reasons. One is that it allows printing as a storage option. The second is, it has almost no requirements other than a performance requirement that the record accurately reflect the information. If that kind of undetailed performance standard was considered by the Congress of the United States to be good enough for the transactions that are covered by the e sign legislation, then as a matter of public policy it's probably appropriate to consider them to be sufficiently stringent for records required to be kept to meet EPA record keeping requirements.

After all, the e sign legislation addresses transactions that involve thousands, millions, billions of dollars. There's certainly plenty of incentive in terms of money at stake to commit fraud, and yet Congress determined that the interest in 99 having a free flow of electronic commerce was such that the kinds of risks could be managed by a standard that simply says the electronic record must accurately reflect the information.

The e sign legislation goes on to say that agencies can impose appropriate limitations, but the legislative history of that provision indicates that those conditions on acceptability are to be minimal. The interpretive standards that the legislation talks about must be such that they are substantively equivalent to the requirements imposed on records that are not electronic records, so they can't be more restrictive than those applicable to paper records, and they will not impose unreasonable costs on the acceptance and use of electronic records.

So there is a clear public policy expression here of concern about the cost of anti fraud provisions. Again, if these kinds of minimal restrictions on electronic recordkeeping in commerce were articulated by Congress, for all kinds of records other than records required for government recordkeeping requirements, and that one exception was carved out without addressing what was appropriate for government recordkeeping requirements, EPA is certainly in a position to say what's good enough for everything else is also good enough for us, unless there are very specific circumstances that would suggest that something more is needed. That something more 100 would need to be justified in terms of both need and cost.

So the e sign legislation would be a good source of inspiration for designing an alternative to the current proposed regulations.

Lamont Silves: Lamont Silves with Computer Sciences Corporation. It looks to me like what we're essentially trying to do here is to promulgate a top down solution. I would urge you to take a look at this from a different perspective. That would be it's fairly clear that the regulated community is already keeping electronic records. You could work with some of the trade associations that are here today to identify the best practices that they use currently, because that's what you're fundamentally trying to do is find a best practice approach to this. So you could work with these folks to find the best practices that they're using today, validate that with the requirements of the program offices and take that approach to trying to find a workable solution.

Kathy Barrowclough: I'd like to follow up with Mr.

Duvall's recent comment, where he was referring to the e sign law. One of the other things that I read in there is in addition to what he mentioned, he talks about the record   or the e sign law talks about the record after it was first generated in its final form. If you take that piece a little bit further, what you can see is that perhaps you wouldn't need 101 all the audit trails on some of the draft documents that might be created on some of the lab instruments that have integration and reintegration but you get the final form, and that's where you've got your record. Just wanted to bring that point up.

George Britton: George Britton with Aventis CropScience.

One of the things in the area of electronic records and how you do things from [ Federal Insecticide, Fungicide and Rodenticide Act] side of the house and particularly [ Good Laboratory Practices] in particular, historically our laboratory instruments integrate peaks and signals from equipment. Early on it was on thermal paper. The agency, because that is not a permanent record, we have photocopied that information.

Because the early integrators did not save data, they just [ inaudible], processed, spit it out, re erased it and went on.

Over time we've gone to other chromatographic systems. We've taken the same approach, we print the chromatograms and the information data that's with that, sign and date it and save it.

Because our chromatography systems over time do evolve between the same vendor, between different vendors, in mergers,

you get all kinds of things that are totally incompatible and you never can get the pieces together no matter how hard you try to get it readable, much less functional. So what we've done is over time we're now taking the paper for a limited 102 period of time of a period of three to four years, five years,

periods, these systems are operational. You could go back within a reasonable audit period, go back to the systems and verify that the paper is exactly what comes from the electronic record.

You can do that over and over again. But as that system becomes unusable or technology out of date, you've still got the paper record captured image of it.

So at the end of this day, we are in the process of imaging it to an electronic medium. In order to have the information available to whoever needs to see it, if the agency comes in to handle it, let me see the raw data for this thing,

we can actually handle it and manipulate it back. We've still got the paper stored offsite somewhere. The electronic version or image capture of it is, as the gentleman talked earlier about the formatting   long term storage problem, that you can see as a snapshot. You can go back and view, if you're trying to detect fraud, does what was reported match what's in the files, you can go back and compare the two.

With that you have gotten sort of the best of both worlds.

In our particular case is a particular paper system, but it does give you the ability to do electronically, submit electronically and find the stuff in there. It also gives you things like for the archiving or information that is required 103 under [ Good Laboratory Practices], that you have to be able to find the stuff quickly when the agency comes in and wants to view the stuff. Yes, we have it on paper or have it somewhere.

But if we have an information retrieval scenario where you say,

this study is located in this place or on this CD, you're protecting the information, you're trying to do both.

The functionality of the long term storage is the real, at least for the [ Good Laboratory Practice] side of the house, is the real key. You have problems, policies and procedures in place to ensure the security and integrity of the information.

But keeping it functional electronically over time is the big point.

Charles Reese: I just wanted to add a comment to that,

that since we currently take on [ Federal Insecticide, Fungicide and Rodenticide Act] side of things the paper printout to be the acceptable raw data   and I want to make a note here that that is considered an acceptable practice by the agency currently, which is essentially a snapshot of what is going on in that electronic record. We print that paper out; that is saved in the archives. If a company so chooses to image that system for their own benefit because they transfer archives and so forth, that's fine. But the important point here is that taking that snapshot on paper is considered an acceptable practice by the agency in the means of getting an audit of the 104 raw data, what the original raw data is.

As CROMERRR is written right now, the concerns that we really have are you're changing the definition of what original raw data is for us. We say it's the paper printout. CROMERRR says it's the electronic record that's saved now. That creates a whole different, we've already gone over all these problems that creates. But the question of a possible alternative that can be raised is if you can take a snapshot and put it on paper, how is it that you cannot take a snapshot and put it electronically in something that you can maintain for a longer period of time, instead of trying to maintain the original raw data that's going to evolve and change and you're going to have trouble with. But you could possibly take a snapshot electronically and keep that.

The argument that we've heard from the agency over this is that you don't get to keep   you don't have the meta data that generated that file, all the audit trails and so forth and the instrument parameters and everything that's saved in the electronic file that's not on that printout. It wouldn't be on that snapshot of say a TIF image of some sort. But we're also submitting here that it's not on the raw data printout either and that's considered an acceptable form of raw data. So we want to make that point and raise caution that we're changing the definitions of what original raw data is. 105 Tammy White: My name is Tammy White and I am from Rutgers University, New Jersey. I'm part of the IR 4 program, which we have a couple things happening here. We are a United States Department of Agriculture funded program, which means we have limited funds in order to accomplish the work that we do on behalf of the growers of this country. A point that was just made was that there have been precedents for the acceptability of this data that was generated using computer based systems in a printout form. Earlier it was mentioned that when putting CROMERRR together, that Part 11 from the Federal Drug Administration side was utilized as well as the [ inaudible].

But I think there's another precedent that is there, and please if someone can add onto it. Within the [ Organization for Economic Cooperation and Development, Good Laboratory Practice]

scope, the permissibility of using that printout as the original data has been established. So I think that there's precedents already established for that.

On behalf of IR 4, I can't reiterate enough the cost of trying to implement the data retention systems, and to try to revamp the data collection systems over time is an enormous expense on a program that we currently   we have very limited funds. When trying to accomplish a grant funded program and trying to be as efficient in that program and having a requirement such as CROMERRR placed on top of us, the 106 efficiency that we would lose would just be enormous.

Robin Roberts: Could I see a show of hands of those in favor of decoupling? I'm not sure who wouldn't have their hands up. And the whole notion of a work group to work on a narrow issue of doing a time sensitive security or fraud provision? Are there any other comments? There's one over here and then after that we'll probably take a break here.

Paul Toll: I'm Paul Toll with the Bayer Corporation.

Just kind of side with what Charles said earlier and Tammy just said, I think even if we aren't going to decouple the records part of it from the reporting and signatures, I think all these companies or at least [ Federal Insecticide, Fungicide and Rodenticide Act] arena are pretty much global companies. We've got work that's going on over in Europe, even for registrations here in the U. S. To have a different set of rules here in the U. S. than what they have in Europe is very troublesome, and the harmonization of those types of situations would be very beneficial. I would suggest that you also look and see what [ Organization for Economic Cooperation and Development] has already in this arena.

Robin Roberts: There are a couple of options here. We could take a break now or we could go to the next section here,

which is what special interests are raised by CROMERRR criteria for long term archiving. I've got a feeling we've kind of hit 107 on that already.

Before we come back for that, why don't we break now.

It's just been suggested that in addition to just the issues that you'd like to comment on regarding electronic recordkeeping if there are other   the other sections of the rule,

A, B and D, that you'd also like to comment on, we'll have a brief period of time for that later on in the day before we recap everything at the end of the meeting here. So keep that in mind. So if you have something on your mind that you'd like to say that doesn't fall within the scope of this meeting as defined by the agenda, we'll have an opportunity for you to say your piece. Let's take a break for fifteen minutes, that means we come back at ten to three.

[ End of Tape 2]

[ Tape 3 of 3]

EPA, Unidentified Male Speaker: I heard these from a couple of people and I just want to see how other folks feel about these. We had one suggestion and I also received a lengthy written comment about the idea of basically substituting the e sign criteria for record keeping, which I think Mr. Duvall read to you here a while ago. It basically says accuracy and accessibility and describes a little bit, a few sentences, about what that might be about. So I wasn't clear whether your particular suggestion was just substituting 108 those criteria for the ones that we had. I did have a written comment that suggested doing that. I was just wondering, from kind of asking other people in the room what they think,

whether that would be useful, appropriate, helpful, too general, whatever.

Mark Duvall: The e sign legislation I think is an appropriate model to think about. I don't think we've thought about it hard enough yet to say this is the right answer. But it certainly is suggestive that a less detailed, less regulatory approach might well be sufficient.

EPA, Unidentified Male Speaker: One of the reasons perhaps for doing that, if we were to do that, is that because e sign specifically excludes reports specifically designed in response to a federal requirement from those requirements,

right now e sign doesn't apply to that kind of records or those kinds of record keeping. So this would essentially bring those requirements, which are fairly general but important, that would bring those into the record keeping for EPA. Any thoughts from anyone else?

Robin Roberts: I'd like to encourage any other comments from any of you who are familiar with e sign legislation and the difficulty perhaps in extending it to EPA required records.

Unidentified Male Speaker: Just from what I heard about e sign as it was being described, it sounds as if that would 109 apply more logically if you need to apply something on the reporting side. I think it's still a separate issue whether anything at all is needed on the record keeping side. I'd still draw that distinction.

Mark Greenwood: One thought, this really is in the nature of a question. I think we all would be curious to see how other parts of the government are responding to the [ Government Paperwork Elimination Act]. We know about the [ Federal Drug Administration] requirements, I think you've heard a lot of people expressing deep concerns about how those are set up.

But after all, the [ Government Paperwork Elimination Act] is a government wide requirement. I'm curious how other agencies are trying to deal with these kinds of issues. There may be some other models out there that will provide some consistency that would be useful to look at. I would just pose that, if you know, and if not that's a good kind of action item to go look at.

Howard Krueger: I do support e sign as another model,

which I think is a valued alternative. What I think we ought to consider is put that on the table adjacent to the CROMERRR nine criteria and then let EPA in particular identify areas in e sign where they find that it may not measure up to the standards or the need that they have. So e sign may not be a perfect answer, but I think if you were going to try to solve a 110 problem and you had to pick one or the other, I would certainly vote that you pick e sign. But we could work and make it even better, I think. I don't think it's the answer in and of itself.

Mark Duvall: Responding to Mark Greenwood's inquiry about what are other federal agencies doing under the Government Paperwork Elimination Act, the Government[ sic] Accounting Office in September 2001 did an update on what other agencies are doing. It's interesting that with respect to high risk transactions, and this is mainly a reporting issue rather than a record keeping issue, twenty two different agencies had identified high risk transactions, a total of 249 high risk activities. So twenty two other agencies have gone through the kind of risk assessment that [ Office of Management and Budget]

called for. Of those 249 high risk activities that were identified by these twenty two agencies, forty three or 17 percent of them, the agencies planned to use the electronic transactions, electronic reporting option anyway.

So it's clear that outside of EPA, there is a process going on, more to greater or lesser degrees, of considering whether fraud is a concern in particular applications,

identifying those applications where it's a concern and then making a reasoned decision whether or not additional protections are needed to prevent fraud. 111 I might also mention that the National Archives recently,

December 10, 2001, put out a report on current record keeping practices within the federal government. The bottom line there was that government agencies are not prepared themselves to keep records electronically. If I could just read a couple of sentences. " Technology tools for managing electronic records do not exist in most agencies. Despite the growth of electronic media, agency record systems are predominantly in paper format rather than electronic. Virtually every agency visited"   and there are 150 agencies covered in this report   " indicated that the official policy is that their records will be maintained in paper format. The predominant e mail policy is to print out e mails that are considered records and to save the paper copies. The chief paradox of today's federal records management is the disconnect between paper and electronic record keeping."

So if EPA is asking industry to keep records electronically for very extended periods of time, it should do so with the recognition that the federal government itself is having a very difficult time grappling with the same problem,

given all the federal government's resources. When we consider that small businesses, states as well as medium and large sized companies are covered by EPA record keeping requirements, it's worth having considerable pause about imposing long term 112 archiving requirements that the federal government itself is having difficulty meeting.

Unidentified Male Speaker: I just want to cite the [ Securities and Exchange Commission] EDGAR system and the filing that's done, which is a lot of security's required for the securities filings that are done at the [ Securities and Exchange Commission]. They're on to their third generation of technology and in fact, the reporting that's done, you can pull the software off of the web site that facilitates the electronic filing of submissions. That is one model I think that should be considered with respect to certain programs.

The thrust of CROMERRR on some level is to facilitate electronic reporting for many different reasons.

I think a useful model is what the [ Securities and Exchange Commission] has done. In fact, the software that they've developed to address the e sign issues and other issues, some of which get into the audit trail capabilities but do not require record keeping independent of this mechanism for electronic reporting, the software they provide from their own web site.

Unidentified Male Speaker: I do have another question but I just also want to respond to Mark's comment a minute ago. We contributed to that report, or we contributed our [ Government Paperwork Elimination Act] analysis along with other agencies. 113 One of those high risk reports was ours, that we are collecting electronically risk management plan, as an example of that kind of report for those of you that are familiar with that. But that really does refer to electronic reporting, those high risk areas. So it doesn't really refer to the record keeping part of it.

I guess I would also say yes, another part of our office,

not my group but another part of our office of information collection is in charge of record retention for EPA. They're struggling with many of these very same issues.

I guess the thing is that I think maybe either   what you said was that if EPA is requiring people to keep electronic records for long periods of time, then we need   what we're really saying is you can keep paper records if you want. It's just that if you're going to keep electronic records, there's some requirements that should be associated with it. So it's not that we're saying you have to keep electronic records but we don't know how to do it. We're saying if you want to keep electronic records, here's some requirements that should be associated with it.

There's another option that was I think raised by a couple of people earlier, was that rather than focus on something like an audit trail requirement that maybe it would make more sense to do something like have people document what their system is 114 for ensuring integrity and accuracy for their records. I thought that was kind of an interesting idea because you could attach that to a more general kind of performance requirement and then people just sort of   it would allow variations depending on people's different approaches or different software and stuff.

So I thought that was kind of an interesting idea, but I also   I had as an old paperwork reduction person, I was also concerned about what that might mean for companies in terms of costs and in terms of a new sort of document that people would have to create. So I was also just interested in getting some kind of broader thinking about that idea. I think it was John Chellen and somebody else originally suggested that this morning, that some kind of documentation system for how you preserve and protect your records might be a better substitute for some kind of thing, rather than like an audit trail requirement. Any other thoughts about that?

Robin Roberts: So alternatives to the audit trail requirement. Who had raised that point or who would like to speak to that point earlier? We've all noted that audit trail as stated in the rule is pretty cumbersome and this is a way that we could suggest that they be documented and preserved,

protected.

Jim Scialabba: I recommended earlier a record keeping 115 management system be substituted as a performance standard rather than prescriptive. In fact, I believe probably every company in here already has a written records retention policy in their company that would serve as a platform from which a record management system document could be developed. In fact right now I have some of our corporate attorneys who wrote the original record retention policy looking at the implications of CROMERRR on our internal records system. So I don't think it would be all that cost burdensome to companies. Again, I like the fact that it would allow for the inclusion of innovation in technology, where a lot of times we wouldn't even have to change our system. We would just incorporate new technology,

as long as it met the minimum criteria of the performance standard that EPA would outline for us.

David Schwarz: Just a question. A number of people at this meeting and actually a number of people at preceding meetings mentioned the [ Organization for Economic Cooperation and Development] standards. Those standards are I think very interesting and we've looked at them. I guess I'm curious, and this is a question you could answer now or you could give us comments, but I think we'd be very interested in knowing to the extent that you think the [ Organization for Economic Cooperation and Development] consensus document standards represent an approach. Can you sort of identify the universe 116 of records that you think they are best applied to?

Also, I did note I think when I read them that they do in fact involve an audit trail requirement, which is something that a number of people have found objectionable. I guess I'd be curious to know why it is that in the context of the [ Organization for Economic Cooperation and Development]

standards the audit trail requirement is really not objectionable, yet it seems to be objectionable in the context of CROMERRR.

Jamie Conrad: Just reflecting on the sort of management systems approach or the document your system type, I guess I can think of it at sort of two levels. One would be, there actually would be an affirmative requirement to maintain some sort of a records management system which maybe would be part of your record retention policy, that would specify the things you do. The other might be, and that would be sort of performance oriented.

Another, maybe more streamlined approach, which I guess is the e sign approach, would be not to specify that requirement per se but to say that you have to have procedures or mechanisms, systems in place to ensure that your records are maintained accurately and with integrity and so on. But what you specifically had to do in your specific case, given the particular record in case, would be sort of done on a case by 117 case basis.

If someone came to you and challenged you or there was an investigation or enforcement case involving the legitimacy of a record, you would have the burden to substantiate what you did in order to meet that requirement. But it would be a case

bycase thing as opposed to saying here's the specification for your records management system, show me how you meet each of these ten elements. So I think you could do it either way under this framework of a performance oriented approach.

Robin Roberts: Now we'll go on to this question of [ Organization for Economic Cooperation and Development]' s audit trail provisions not being objectionable, but perhaps CROMERRR's being objectionable.

Kathy Barrowclough: I think that one of the points in response to Mr. Retzer, I believe that the [ Organization for Economic Cooperation and Development] principles issue that was raised was as an alternative in some cases to offer another option of printing out, and not to make it so that that would be the answer. That would be my read, anyway. Also,

recognizing that that only applies to one kind or type of record or regulated entity within the EPA community, because indeed those [ Organization for Economic Cooperation and Development] principles relate to the [ Good Laboratory Practices] community and they do not relate to the rest of the 118 environmental monitoring community.

Mark Duvall: As a follow up to that, the [ Organization for Economic Cooperation and Development] principles that we're talking about are [ Organization for Economic Cooperation and Development] good laboratory practice principles, which the [ European Union] has adopted as [ European Union] [ Good Laboratory Practices]. They have two provisions of note. One is, they do have an audit trail provision; they also allow paper copies. Today, EPA's [ Good Laboratory Practices]

regulations have an audit trail requirement and allow paper copies. The concern here is that the parallels that exist between the European/[ Organization for Economic Cooperation and Development]/ U. S. provisions will be disrupted by changing the U. S. requirements to disallow paper copies. The reason that audit trails are not an overwhelming concern under the [ Organization for Economic Cooperation and Development]

principles is that they are a requirement in all those jurisdictions that either endorse the [ Organization for Economic Cooperation and Development] or the U. S. requirements.

The U. S. audit trail requirement in the [ Good Laboratory Practices regulations] appears at 40 CFR, Section 160.30( e).

There's a corresponding provision in the [ Toxic Substance Control Act] [ Good Laboratory Practices regulations].

So there is a provision now for audit trails in the [ Good 119 Laboratory Practices] context. But it's limited to the laboratory context and it's limited not only to laboratories but to laboratories conducting studies under [ Good Laboratory Practices]. So it's quite limited.

Charles Reese: I wanted to clarify a little bit more about the [ Organization for Economic Cooperation and Development]. Obviously when we read the [ Organization for Economic Cooperation and Development]' s, they seem to have a more practical approach. Sometimes Europeans are always, what can you really do, versus what do you think you should do approach. But we have to make sure we're also talking apples to apples here, because I don't think the question was should we not require audit trails whatsoever [ brief noise interruption].

It's not a question of not having an audit trail in some cases, but again we can point out here that this is a [ Good Laboratory Practices] setting, and also we're also talking about in the Part 160 there's fifty seven different types of records that one can have. In an audit trail system in the [ Organization for Economic Cooperation and Development] system and listed in the good automated laboratory practice systems,

GALP, they involve laboratory information management systems.

Therefore data collected by the computer system that's transferred through a computer network and somehow has the 120 opportunity to be manipulated through that computer network to where it finally resides. The GALPs require an audit trail, so anyone doing work for the EPA has to apply the GALPs.

I don't think the question here is we're concerned about   because taking away the audit trail, if you're in that situation where you're following these specific guidelines that you have to have an audit trail, in a [ Good Laboratory Practices] setting I think we're saying we're okay with that.

The [ Organization for Economic Cooperation and Development]

gives you the out here that you can have a different medium if your system becomes obsolete. But that's outside of   there's 95 percent of the rest of the country that's dealing with environmental monitoring under Title XL that's not a [ Good Laboratory Practices] setting. I think those are other concerns when it comes to dealing with audit trails. So from my particular aspect, there's not a problem with audit trails under those specific circumstances, LIMs, laboratory information management systems.

Unidentified Female Speaker: Just one other point on that same issue is that that is related as Charles just mentioned to the laboratory information management systems. What CROMERRR will do is broaden the records that would be managed that way to include indexing systems and SOPs and training records and all of those sorts of things that are not the laboratory 121 information management systems that are collecting data.

Robin Roberts: Great. Were there any other options that you wanted to elaborate further on, Joe, that you heard about alternatives?

Joe Retzer: No.

Robin Roberts: Then we'll turn to the next section of the agenda. Although these issues may seem or we have talked about them, it seems that right now with archiving, current practice among companies is to print the paper or to take an electronic snapshot, somehow image the data and file that. That's what we seem to have heard. This section is intended to kind of delve a little bit more deeply into your current archiving practices.

So the question I guess before us really is, is printing the paper and archiving that and entrusting the paper to an archivist who has a duty of care to maintain that record beyond the life of his employment, is that the way that archiving is done? Or it's snapshot imaging, the data now and then perhaps printing that paper, the way that archiving is done now?

Charles Reese: Sorry if I seem like I'm talking forever here today. Yes, with the archiving issue, the paper record is, the printout to the paper record is the current practice in the [ Good Laboratory Practices] setting. I just wanted to point out, also with what Kathy [ Barrowclough] said, that if you include e records to be everything and you require and 122 we've changed our definition now of what original raw data is,

you have to keep this original raw data for the life of a registration, which can be a very long time, we're talking decades, especially if companies merge, products get sold various places. The kind of conditions that we can run into is virtually having to take that electronic record and migrate it from a platform A to platform B to platform C over time, if we don't have that [ Organization for Economic Cooperation and Development] out that we were given.

I want to point out, and I can probably provide for you later, at the ACS meeting in 1999 in Chicago, there was a team of lawyers there talking about the migration of electronic data and the problems with that. They suggested that for every migration that you do on a data system, that you run the risk of losing between 3 5 percent of that data as you move from platform to platform. They said that virtually no matter how hard you try, no matter how much effort you put into validating and to make sure you catch every little piece as you move from one platform to the next, you will invariably lose between 3 5 percent of that data.

If you've got fifty years to go, by the time you've migrated to that fifth platform you have a possibility of losing 25 percent of the original raw data. There's a significant issue there of compliance, if something as you 123 migrate becomes completely irretrievable. So far we don't have solutions or answers to these questions yet that we can feel comfortable with, outside of printing out.

Robin Roberts: Migration, loss of data over migration through different generations has been something that's come up in our other meetings.

Unidentified Female Speaker: I'd like to first add a couple additional issues before I address these. Just basically the maturity of the technology today does not allow businesses to comply with CROMERRR as it is written. There is the possibility to be able to archive and manage in all kinds of realms, long term, bringing it back, all that kind of stuff.

Translation from machine language or databases or whatever to human readable forms or the ability or some form of archive and the migration of this information from legacy systems to newer systems, now introduces the concept of copies of data. They are no longer the original raw data files, they are copies. So what are the conditions that CROMERRR will address for that?

Another one is media degradation. Media doesn't live for thirty years. You have to continually retrieve it. Now you're making another copy of it, you're bringing it back to something, you put it on new media and you're shipping it out for storage again. Again you're making another copy of it.

This has nothing to do with retrieving it, trying to read it or 124 anything, this is just maintaining it long term.

One of the ones that was mentioned before, I'll just bring it up again, is setting the retention periods for electronic records rather than imposing existing paper requirements on the electronic records. Again, one size does not fit all for the definition of archival. We need definitions for archival requirements for the different types of systems and the different electronic record keeping systems that are out there.

This is kind of in the realm of the data collection versus data manipulation and all the way down to, for example, a two year study. A two year study could produce 500 800 megabytes, but one analytical method development can generate 4 gigabytes of data. So there's different quantities, there's different conditions that are needed depending on where the data is being collected. So this might take it down to a program office level to be able to determine that.

The last one is study reconstruction in the original environment. If a product goes to market for thirty years, in thirty years I'm supposed to reconstruct my original environment to bring that data back? That's almost impossible.

So we see those as big issues for long term archiving.

One of the additional comments against an issue that you have listed is that I just wanted to concur with what was said earlier about the loss of data between whenever migration 125 occurs or going from one medium to the next. This doesn't have to do with losing data because of some type of inadequate conversion, but because of the cause of technology changing.

So it's not that the systems weren't validated and the actual data when it was migrated there was just pieces that were dropped off the end of the earth, it's the fact that physically it's not possible.

Paul Toll: I just wanted to reiterate something that was said earlier and bring it into the current discussion. At least with Bayer Corporation, we do a lot of contracting out of work. To assure that we have the raw data, at least have the data in our house in case one of these contract [ inaudible] go out of business, we always ask that all the data be sent to us instead of having it archived at the contract facility. Like I think as Charles [ Reese] said earlier, we may not even have the same software or same hardware that this data was generated on.

So we get paper printouts of the raw data and we archive those.

So if we're going to have to try to archive completely in electronic form, it may be impossible to do.

Bill Barta: I just wanted to share an experience with migration. I'm not an IT person, I'm quality assurance, but just to give you a qualitative description. We migrated some images, 100,000 images from a VAX to an NT server. The problems we had was corruption of files, there were old images. 126 Secondly, another thing that happens is we went to a different platform, the software we were using, which was old and no longer supported by the vendor, developed a glitch in the search mechanism. The vendor does not want to help us fix it because they no longer support the product.

Another area you need to be aware of is that even though manufacturers like Sony say that their optical disks last fifty years, that's a projection they made. If you bought optical disks in 1992 and the head that reads the disk broke, it was probably proprietary and you can't find another one to read that information. So you would have a problem moving it to somewhere else to read that information.

The issue of degradation of media and software compatibility is a very large one. There's probably no way you can avoid some of these issues. So the requirement that you maintain functionality becomes highly problematical, because if the software changes, the vendor changes it to improve it, and you've lost something that you consider critical, there is really nothing you can do about it. In theory, the agency will come in and say you've done something really bad and punish you for it. So that's the scenario I see happening.

Tammy White: My name is Tammy White, I'm with the IR 4 program with the USDA ARS [ US Department of Agriculture,

Agricultural Research Service]. One of the problems in 127 addition to the things that have been reiterated here about data degradation, the media degradation and things, in our lab in particular doing the chromatographic analyses, just in the short period of time that I've been doing this work instruments go out of   the vendors don't support them anymore and you end up with a media that can only be run on a particular instrument. You can't open the data from the chem station without having the instrument attached to it. So you're talking that we'd have to archive chromatographic equipment as well as computer equipment and software.

Then you run into the problem of if the chromatograph doesn't function because a board goes on it or something like that, the vendor isn't supporting it anymore, you can't get the part, you can't turn on the instrument. Therefore you can't even get to the software and get something that's functional.

So being able to make that paper copy under the [ Good Laboratory Practices] is very important to our laboratory in particular, just for being able to archive beyond the timeframes of an instrument surviving. Thank you.

Frank Miller: Just a couple of feedbacks on this longterm archiving, mentioned before but just quickly go over them again. The storage media and dealing with its obsolescence and so on is something that is reasonably achievable. It's not the easiest thing to do, but you can do it and you can do a 128 validation and you can work that. I guess the issues you run into are eventually getting to a point where the first recording of data is really not available anymore and you have to live with the subsequent generations of that.

Around the area of meta data and e signatures and connecting that up to the actual electronic record, that's really the technology and so on is fairly new, the whole concept of meta data and so on. I don't think there's really good evidence of tools and applications that have really adopted that in the way that you're intending to move towards.

It's too emerging, too new technology. Certainly it's conceivable that we could work in that direction, but nothing off the shelf really that's at that grade of that today.

Then the other area that's really an issue is just the whole concept of application obsolescence and migration, and the ability to reconstruct the application that you originally had and be able to preserve that. Unless we can work towards vendors that survive over the long haul and they eventually take over the marketplace in these areas, it's going to be difficult if not impossible to keep those applications and that business logic and all the things that surround how the electronic record was created for long retention periods.

Mark Greenwood: I want to reinforce a comment I made earlier, because I think this archiving issue really does 129 highlight the fact that we need to look carefully at the underlying retention periods and record keeping requirements themselves. I think everybody views this as inevitability that we're going to be more and more going to electronic records.

There are these archiving problems. We are often looking at reporting and record keeping requirements that are a quarter century old, that were not carefully thought of   having been involved in some of those myself   at the time. It is a time to really go back and start looking at some of those and ask the question, how long do we really need to know this?

What do we need to know about something that happened twenty years ago? What is the value of that for us today in terms of managing the environment? I think that's a fundamental issue that I know you can't do as [ Office of Environmental Information] by yourself, but it's an issue that's inevitably going to have to be part of this discussion.

Mary Catherine Fish: Mary Catherine Fish with MCF Consulting. I think that this archiving criteria may also go to a point that's come up in different forms, which is the difference between possibly the intent that the record keeping requirements would not create additional record keeping requirements, and the perception that it in fact creates a whole slew of new record keeping requirements. Which is that the archiving criteria as it's written requires that the 130 archiving records include the context, the meta data, the audit trail. So for every   where you previously had one requirement to preserve a required record, you now would have many. So there's an exponential increase of the requirement, because of this part of the wording of the archiving requirement.

So that's one mechanism that leads to the problems that people are encountering, as part of what makes this so big.

It's an actual   we can go back to the basic argument over whether it's voluntary or mandatory, I won't get into that.

But this archiving criteria, I think is really one of the concrete ways that makes this proposal a great expansion of what's currently required. I would think that that would have ramifications all down the line in terms of [ Information Collection Rules] and any other kind of analysis that EPA is going to do about how onerous or burdensome its record keeping requirements are.

Mike Penders: This discussion has illustrated to me just that the record keeping requirements are not, should not logically function as a condition of electronic reporting per se, but pertain to the precise programs and technologies and monitoring that are relevant to keeping records under record retention requirements, statute by statute, program by program.

Really I think it has brought home to me the importance of decoupling these notions and letting record retention evolve 131 with technologies, program by program, that meet the fraud concerns and other concerns with reference to the technologies,

audit trails, that are particular to the testing, monitoring,

equipment and forms of auditing that may be required under those programs, but do not relate back as a condition of electronic reporting per se.

Jim VanHorn: One of the things I don't understand what information technology means and I don't speak IT, but I have been through three major mainframe computer changes in the positions I've been in, production, quality and environmental.

I have been through one complete merger of two companies and they had completely different systems that they were dealing with and their process on it. Each of those forced things on it required a year or a year and a half of pre planning, then there was the implementation of crossover on it to do. As a general rule it was a year before they were stable. In every instance on that, we still had things in that, with all of the planning and as much input as was put into it, that would not transfer.

So this is just an example of some of the things that I have been seeing that go through that point out the problems in why you're not going to get 100 percent transfer when you go from one system to the other. I just know for a fact it has happened that way. 132 I still have the question that I have not understood the [ inaudible] of it as to what the magnitude of this is going to be for Smurfit Stone, is just what records and data and all are we still including in this? Is it everything in our facility that has   I can even back up to this far point on environmental related? Or is it really the final reports that we submit that must be maintained in this record system? If it is, then it creates a much less of a concern with us than if it does have to be all the way back to the individual data that's collected throughout all of the processes.

Henry Bartholomew: Just an observation on archiving in general. I work a lot with folks in our accounting area of our companies. We've run into the same discussion, for example with [ Federal Energy Regulatory Commission] and the [ Securities an d Exchange Commission], on long term archiving and migration and so forth. Whether people need to keep records on paper or electronically, it comes as you're hearing at a significant cost. It's cost of space, memory if it's computer. It's cost of the software, it's cost of personnel to do that.

I think putting this as a third issue and having focus on the individual program areas, the key question starts with what information do companies need and does EPA need and for what amount of time. If the [ Internal Revenue Service] for tax purposes and bring in the money that runs government can get 133 along with limited timeframes like five to seven years worth of data, maybe that's where we ought to be thinking. Again,

distinguish between report information that's provided EPA and [ inaudible] of reliability for that versus internal information that may lie behind that but be necessary to company operations. What EPA should be focusing on is the former, the reporting side, but not all the other information that goes in the company.

So several things come out of that. Look for short timeframes, do it program by program, look for minimal archiving and record keeping long term requirements,

distinguish between what you need to substantiate on the reporting side from what companies need to do their internal operations and stay the heck away from that. Keep it as short as possible, recognizing cost. I think having it as a separate exercise makes a lot of sense.

Robin Roberts: We have at least one more comment. I would just like to say that the war stories have been great,

they're all edifying. But I would like to turn at some point towards a more solution oriented sharing of experience. Where have you made the change? We'll just segue into the last part of our agenda, if we can make that transition.

Carolyn Greenwood: Most of what I was going to say was just said, except that I just want to add that you need to 134 remember that when people   we've had some examples of companies that have to keep a lot of data one way or another.

They've decided, when they come to us they say   [ End of Tape 3, Side A; Side B begins]

 they have to manage all this stuff on their hard drive.

Their system tends to get slower, they get more frustrated with the system itself. It makes it more tricky for the software folks to design a system that will be more efficient, because it's dragging all this data along with it. So to kind of confirm or second one of the things that was just said, I think program by program is the way you want to do this. Some people need to keep lots of data and some people don't. It would be real unwieldy for some folks to have to keep a lot of this data, only because you wanted to have it consistent across all the programs.

Kathy Barrowclough: I guess in response to what Mr.

Roberts just asked, get away from war stories and what would be potential solutions. What I believe I'm hearing is we don't have solutions to the archiving issues, to the long term archiving issues. There are issues and we recognize that, but we don't see solutions. One of the potential solutions is really to look at the archiving requirements, the record retention requirements, and maybe reevaluate those.

Charles Reese: But I like war stories, and what better 135 way to give you an example. One particular example just springs to mind. It was a brilliant idea to create a document management system so that people could share reports throughout the company without having to physically go find where the report was filed. So a scanning mechanism was put in place to scan 300,000, 600,000, 1.3 million pages of documents it suddenly turned into, as they thought this was going to be a great idea.

Halfway into doing this system, the company that we had had the proprietary software, set up our system, was purchased by a larger company. That company said, that's okay, we will continue to support the older system. Which lasted a year, and the older system went away in favor of the   it was a competitor buying a competitor, now they favor their system.

It literally left our document management system in the lurch.

To purchase new system software and reconfigure, it actually took a manual transfer along the lines. Right when we got finished with that, another merger happened and you see where this is going.

The potential for mergers in the software industry and the drop off of support are another part of this angle that you worry about when you're trying to migrate data and move it forward and keep it for the life of the record retention period that you have to keep it. You may lose support over a period 136 of time that forces you into a different scenario, that raises your costs and changes your platform and creates frustration across the board. You may actually reach a point where it simply can't be done anymore.

Robin Roberts: I'd just be curious, Carol [ sic]

[ Greenwood], not to put you on the spot, but are there any success stories you've run across? You just mentioned that you have successfully migrated information from databases, old ones, and they've become unwieldy and slow. But is there some really sleek operation that's   finding your financial statement as an example of why someone would hire your corporation?

Carolyn Greenwood: We've done lots of good things. I don't want to say this is always   I could come to a meeting like this and say, make it as hard as possible, because it just means more business for us. But obviously what we want out of this is something   we've got lots of stuff on our plate already, so I'm not begging for business. What we really want out of this is a clear rule that's reasonable for everybody to work with. We don't want frantic customers coming to us with emergency needs anymore than they want to be a frantic customer.

We have lots of success stories. We're confident we can do whatever comes out because we know there's enough checks on 137 the system that you guys won't go too crazy with what you've proposed. But I think the bottom line of it all is yes, the things you're talking about even now could be done. But do you really want to do that to these folks? We can do that kind of stuff, we can technologically speaking do what you're talking about, all the different things you've talked about doing. But it means a big burden on a lot of folks in this room.

Frankly, we don't even have the manpower to do some of what it sounds like you're talking about. If all our customers came to us and said, all the utilities and state agencies and local agencies came to us and said, make us CROMERRR compliant,

in its worst scenario of what I've heard today, we would just scream and throw up our hands and go, slow down, wait, how long have we got to do this. It would be a lot of work, so I know it would be a lot of cost for them.

That's not to say we can't. We have done a lot of successful migrations. We've updated a lot of Part 75 systems,

acid rain systems, where the way we migrated the data was through EPA's format. If it was another vendor and maybe they went out of business or whatever, we would take EPA's format as the common ground and bring that up into our system. There's ways that you can do this stuff, but I don't know that that's the question here. Maybe I'm misunderstanding your question.

Robin Roberts: So the glass is definitely half empty? No 138 success stories at all, no hope on new technology delivering us from this paper conundrum.

Frank Miller: A comment on the new products or emerging technologies. Some thoughts around the idea of where we might be able to get some opportunity. I think the current environment for environmental health and safety information systems and so on is somewhat fragmented and there's really no Goliath in the marketplace. Probably the same appears for the services that one might buy around information services, again,

specialties and niche driven, maybe no large vendor that can own it all.

There appears to be some good movement in the enterprise business systems area where potentially that may be able to help with these kinds of requirements. Software vendors,

there's a couple, reasonable amount of fairly big software vendors that operate in this area. Many of them appear to be implementing or have already implemented environmental health and safety modules that integrate across the enterprise and take advantage of information.

Almost all of them are working in the area of e business type technologies to make sure that their systems can play in that area and can offer some technologies to help meet some of these requirements. Many of them either strive for, have methods for factory floor integration or other kinds of lower 139 level instrumentation or systems integration.

Ideally for us as the regulated would be to try to get to more of a single system approach, to come into compliance with these kinds of requirements. To have sort of a one data source approach to this. It would be better for us to really work with a software vendor to come into compliance with this, like our Year 2000 compliance, coming and making sure we were prepared for that. For example, when the euro currency was brought upon us we counted on our software vendors to help us get there and really didn't have to do too much with internal software to have to come into compliance.

That's sort of what we would keep our eye on if this rule passed and we had to come into compliance with these kinds of requirements, to see if we could move in that direction.

However, I don't think anybody has a lock on this, from what I can tell.

Mark Duvall: The [ Federal Drug Administration] has a web site relating to Part 11. In that web site are presentations to [ Federal Drug Administration] by a variety of vendors.

These vendors promise much. They have lots of products for sale. They are all addressing the Part 11 requirements, which as we know are very similar to the CROMERRR record keeping requirements. These solutions by and large are untested,

unproven. They would need to be validated. They would need to 140 be validated not only as a piece of software available for sale, but more significantly perhaps they would need to be validated within each user's computing environment. For adaptability, compatibility with existing systems; for integration with other kinds of systems, so that information can flow freely throughout the enterprise.

The first thing to be said is, are there emerging technologies? Yes, and there are a variety of companies that would be glad to sell them to us. But one, they are unproven;

two, they're very expensive. When you consider the wide variety of electronic records being kept that would come under CROMERRR, any one solution will be inadequate to address the totality of the problem.

For example, there is a vendor who has made a presentation to [ Federal Drug Administration] claiming that they can attach an audit trail to Microsoft Excel. Maybe they can. Up until just a few weeks ago, their product was listed as being under development. They claim it's ready or almost ready. Maybe.

Maybe it is, and at some point it will be, presumably. But a solution that works for Microsoft Excel doesn't do a darn thing for software other than Excel. When Excel files are integrated with other information, electronic information, the fact that the Excel piece has an audit trail on it doesn't do a thing for all the other pieces of the complete package of information 141 that doesn't, for the rest of it doesn't have an audit trail.

So system wide solutions are what's needed. These are tremendously expensive. Dow has been looking at these systemwide approaches to things, has been pricing them. We consider that it would cost easily between $ 50 100 million for Dow to buy a system wide approach, assuming it would work. Those costs include not only the costs of the vendor, but significantly greater costs within Dow for the cost of validating and integrating them into the Dow system. Training the people in how to use them. In addition, storing the information.

If you're talking about audit trails, you're talking about storing meta data, potentially for many years. The whole idea of memory is you store as little as possible as efficiently as possible. If we have to store electronic records longer than we would normally choose to do so, and we have to store meta data along with the original information, then our storage requirements increase by a large degree.

So I think the bottom line is, if we can put a man on the moon, if we can solve the Y2K problem, then yes, we can figure out as a society a technical solution to archiving electronic data. The question is, is it appropriate in the context of a record keeping statute, a record keeping regulation,

promulgated under an amendment to the Paperwork Reduction Act, 142 to impose on private industry the immense technical problems and costs associated with implementing emerging, possible,

untested solutions. My response is no, that's not an appropriate allocation of responsibility.

We see the federal government itself tackling this problem. The National Archives is very concerned about government agencies archiving electronic records for extended periods under governmental archiving requirements. The government agencies have not yet figured out a satisfactory solution. They have much better resources, the United States government has better resources than any single company does.

Yet each company would individually be responsible for finding an acceptable solution.

The magnitude of the technical and financial challenge for industry is immense. Is the problem technically capable of being solved? Of course. But should it be solved by industry in the context of a Paperwork Reduction Act regulation? No.

That's the wrong place for this problem to be resolved.

Possibly in the future, as new technologies have been developed and established and proven and found to be efficient and effective, then it might well be a good idea to revisit the question of whether long term archival electronically is an appropriate requirement to impose on industry. But not until then. 143 Howard Krueger: I'd just like to confirm from a costimpact standpoint that I did a rough assessment for Procter &

Gamble. Again, we're a worldwide company, we have over 100,000 employees with about half and half international and U. S. We have thirty seven plants in twenty four states. Just using the EPA numbers of $ 40,000 per site per system to acquire a compliance system, and then $ 17,000 per facility per year to keep things going, and that was just manufacturing facilities.

Then added to that all the records and compliance documents that we keep for [ Federal Insecticide, Fungicide and Rodenticide Act] and [ Toxic Substance Control Act] products that we make, my estimate   and I did a fairly careful analysis and I had a number of people contributing to this. My assessment came out between $ 80 100 million for our company.

Just setting that aside, the bigger challenge is that those are real live dollars you've got to come up with and there is no one budget that can accommodate that kind of an impact in even a three to five year period. It's kind of overwhelming in today's environment. So given the current requirements, that's for a large multinational company, that's what you're talking, which is validated by the comments that Mark made. I hadn't known what his assessment was nor did he know what mine was before we were here today.

Robin Rorick: Robin Rorick with [ the American Petroleum 144 Institute] API. It's not just the system cost but it's also sort of having to take inventory of the current systems, both assessing each application and identifying the gaps between the functionality and each of the new requirements. Sort of closing these gaps with custom system code, either new software and/ or add ons to the existing software. Updating each of the integration routines for applications integrated into other information. Redeveloping the reports and the records in accordance with changes made to the underlying applications.

Updating the system documentation and user manuals. Then conducting all the end user training. So it's not just the system costs for the archiving and everything else, it's all the other associated steps and the costs associated with that.

API hired a contractor to take a look and do a technical analysis of this, and came out with a study that we will append to our written comments when they're submitted. But the average cost per facility came out to be about $ 2 million.

That was for a facility with just five computer system applications that would have to be updated. In the end, when they looked at the overall cost impact on the oil and gas industry, they compared it to what had to be done for Y2K compliance. The estimate of that was $ 2 billion for the oil and gas industry.

Robin Roberts: Great. I would just recap some of the 145 bigger issues I've heard here. It would seem to be there are a lot of concerns about the audit trail and the gap between the current technology and what CROMERRR is asking us to do. The issues raised by archiving, which are many but which seem to be characterized mostly by rapid obsolescence of the technology,

the cost of it, the efficiency of it if it were to be installed.

Some concern about the scope of the rule and the definition of terms like electronic records and what exactly do we mean by electronic record keeping systems, and whether the time periods for which records should be kept, although they may have been fine in the paper world, do those apply in an electronic world where the media inherently degrades over time.

That goes to questions of migration and loss of data integrity over different generations, how do you migrate legacy systems.

There also seems to be some concerns about anti fraud,

that if we should have a workgroup to go of industry and EPA to look at these programs that are at highest risk for anti fraud.

Maybe go after those first, especially since that particular aspect of the rule wouldn't be time sensitive as would be compliance with the [ Government Paperwork Elimination Act]

purpose sections of it.

As for alternatives, there seems to some concern about risk analysis of the records, that we do a risk analysis to 146 find out where an appropriate solution may lie, the notions that the signature legislation with a few adaptations could apply also to EPA required recommendations. Then again, there seems to be strong support for this notion of a workgroup with EPA to come up with this record keeping issue if it's decoupled from the electronic reporting and gone at in a more deliberative and risk analysis type of environment, where the enforcement or the data security is commensurate with the risk of the program.

I would like to turn to the panelists and if possible have them recap just the major things that they heard or respond to specific questions that you heard or comments that were heard.

Joe Retzer: I think you did a good job of listing a lot of the things we heard today, so I don't need to repeat those.

I guess I just want to thank you for coming and providing us these comments. I want to let you know that I think they've been very thoughtful. We haven't had a lot of [ inaudible]

maybe inwardly you've been shaking your fists, but outwardly you didn't. At least the tone I think was one where we could really hear what you were saying today and pay attention to it.

Our comment period continues until February 27, so please get your written comments in as well. We also have another public meeting in two weeks in Dallas. Obviously we're going to give very serious consideration to decoupling the record 147 keeping part from the other, or looking at possibly some other simpler or more straightforward kind of criteria, as well as looking at modification of the ones that we have here. So we've got a lot of options in front of us, a lot of good ideas.

I think we heard a new idea that I heard today was trying to do something in terms of looking at the broader recordkeeping requirements themselves. That's really not a function of our office and information office, as opposed to the programs who set up those individual record keeping and reporting requirements for their programs. But it may well make sense to try and take a look at these broader recordkeeping requirements in terms of things like the longevity of a record keeping requirement that might have been set some time ago. For example, before people were keeping records primarily in electronic form. So that may be something we would want to take a look at as well.

Charles Reese: My only comment was, we've talked about these workgroups here. What are your plans as far as doing some actual field research to   because you've asked questions about what types of electronic files and what types of records that we here use. It seems to me that there's probably some recognition to the fact that you didn't know of some of the ways that we conducted our business. What would be your approach to actually going out in the field and observing what 148 actually is taking place on a ground level, to see how people really are using these records and to verify the things that you've heard here today? What approaches you may take prior to the end of the comment period or after that comment period.

Joe Retzer: I guess there's two things. One is we do have some contractor work going on to do some research into some of these areas. That report will be available as part of the record, part of the docket on the rule. The other thing is I guess you could invite us. If there's something locally,

some idea you have about us coming to see how you keep records in your company, please invite us. We have limited amount of time during the rest of the comment period, but if you've got an idea invite us and if we can arrange something we'll do it.

EPA, Unidentified Male Speaker: Let me add a couple thoughts to that. We have also been meeting with small groups of industry people. We have plans in fact to hold at least one more meeting. As Joe was saying, time is relatively short between now and the end of February, but we are trying to have smaller group discussions to the extent that we can, to help us sort of make a decision about what the next step should be. If we do decide to decouple the two parts of the rule, then I think we need to step back and do sort of a longer thinking about our process.

Something that you might want to be aware of, the agency 149 has from time to time used what's sometimes referred to as negotiated rule making under the Federal Advisory Committee Act. If we actually constructed a process where we worked with some of you, your colleagues, as a workgroup to continue the rule making, we'd have to be constituted under [ Federal Advisory Committee Act]. There are certain requirements and limitations. So that will   if we think about that, we'll have to think about whether or not we could cope with [ Federal Advisory Committee Act] requirements. So that's just a constraint.

If that responds to your question, I did want to also add something about electronic reporting, which sort of came up a little bit from time to time. We did of course talk about electronic reporting at the two public meetings that we had in late October and early November. It's just because of time and resource constraints, it's unlikely that we'll be able to schedule another public meeting to talk specifically about   it's [ inaudible] we'll be able to schedule another public meeting before the end of the comment period, period. At some point there's so much we can do.

However, I guess I would suggest a couple things and I really urge those of you who have electronic reporting issues to follow up with us one way or another. Certainly, the most avenue right now I think for your expression of comments and 150 concerns is to put them into your written comments that you file, as part of the comment period. As with all the issues that you raised today, electronic record keeping [ inaudible]

electronic reporting. If you have issues, please do put them in writing and get them to us by February 27.

I guess the second thing is that, again, within the constraints of time, I am certainly prepared to try to meet with at least one or two small groups of people if they want to constitute themselves, to talk about issues on the electronic reporting side. That's certainly something that we could consider.

We may also be able to have some sort of ongoing discussions beyond the closed public comment period, although I'll have to check with the lawyers and make sure that we don't violate anything under the administrative procedures act. But there probably are ways to continue the dialogue at some point or another. We do want to do that because we do want to make electronic reporting, we do want to try to push that part of the rule forward even if we do end up decoupling the recordkeeping side of it.

I guess the only other thing is I don't have my slides here for the electronic reporting part, but as you think about what's in the proposal and that notice of proposed rule making,

I guess I would urge you to try and bear in mind that there's 151 sort of three distinct areas in a way that deal with electronic reporting. As you comment, try to make clear to us which area you're really concerned with. There is the provisions, I think they're Subpart B, that deal with electronic, which are the regulatory requirements, proposed regulatory requirements, that deal with electronic reporting directly to EPA. They're fairly short and straightforward. That's Subpart B.

Subpart D, I think I have that right, in terms of what's in the proposed rule making provisions, is the much more robust treatment of requirements on electronic report receiving system. These are requirements that are primarily imposed, to the extent they are imposed, on states and local governments who want to implement electronic reporting. They do have some implications, obviously because some of you read between the lines and see that, they have some implications for what your electronic reporting experience will be like, reporting systems that meet those criteria, particularly the criteria that address the signature method, the requirement for some sort of registration process and the signature certification scenario.

You can kind of look through those criteria. Although they don't directly apply to companies reporting, they will have an impact.

Where you will see the greatest example of that impact,

where I suspect probably there will be substantial comments, is 152 in the preamble discussion of the way that EPA has been proposing to implement electronic reporting through the central exchange system. That lays out the conceptual design of that system as we were thinking about it about a year and a half ago. Mostly that's still true, but we're still thinking about it.

One of the things that will influence our thinking, of course, are the comments that we get on what's in the preamble.

Bear in mind that the way that that system is laid out in the preamble is one but only one of many possible ways, we think,

of satisfying the criteria for systems in Subpart D.

It may very well be that the comments, which we will be very interested to see on the system concept may or may not have much effect on the criteria themselves. They may or may not. But the criteria are actually, if you look at them,

considerably more flexible than a reading of the central data exchange concept, because that's going to be a concrete thing whereas the criteria are meant to be somewhat neutral.

So I guess I would encourage you to kind of keep that distinction in mind as you express your concerns, because I think that will help us understand how best to interpret what your concerns are.

Robin Roberts: Are there any concerns about electronic reporting that maybe quickly someone wants to mention? 153 Unidentified Female Speaker: I have sort of an unrelated question, I'm thinking now as a wrapping up. At the beginning of the day, it was mentioned that all of the remarks made here today would be made available. I wonder if those will be in sort of transcript or summary form, when they might be made available and will be provided with a list and affiliation of the attendees here today.

Robin Roberts: The transcript will be available, it's usually a two week turnaround time on that. Also a list of those that attended, I can have that available soon. Releasing it to you is an EPA call. The formal way that we proceed is that I present the transcripts and the attendance list to EPA,

they take a look at it and pass it on.

Evi Huffer: We plan to make that information available to the public in the docket. We'll put in the summary of the meeting as well as the actual transcript, as well as the participants. You'll be able to find that in the docket in about three or four weeks.

Unidentified Speaker: [ inaudible]

Unidentified Male Speaker: It's electronic.

Evi Huffer: It's electronic and you can find it actually on our web site at epa. gov/ cdx.

Robin Roberts: If you have the sheet that has the FRN notice on it, the web address of the docket is on it. 154 Unidentified Male Speaker: I was looking for the docket,

I found the electronic docket for the number of comments that have been submitted by SQA and Dow and Mobil, but I have not seen the transcript from either one of the October or November meetings. Are they out there? I haven't found them.

Evi Huffer: They evidently haven't been put on our web site yet. They should be and hopefully they will be up next week.

Unidentified Male Speaker: We'll check.

Unidentified Male Speaker: That's one part of the reason for the question is, from the earlier meetings we haven't seen them yet.

Unidentified Female Speaker: Just one comment on the record on the reporting requirements. Would you please, in whatever comes out finally, make it more clear what people who currently report to EPA need to do, or if this applies how it applies, and what they need to do. As a vendor, you know,

we're just trying to gear up. It was not clear this go around what all these things meant and what we should start doing, if anything.

Unidentified Male Speaker: [ inaudible] particular programs that you're aware of that   Evi Huffer: The ones we deal with most would be the acid rain, Part 75, and NOX budget. Also Part 60, CEM systems, some 155 of whom report to states and most report of course just federally. Then ambient air quality requirements, we do a lot of that, ozone mapping kind of stuff. Any of those I'd be most interested in, mostly it's all air that we deal with.

Robin Roberts: Last thoughts, parting thoughts? One in the back here.

Unidentified Male Speaker: Just again, from work I've been doing with FERC, Federal Energy Regulatory Commission,

over the past two years or so, they've been really moving aggressively toward electronic reporting, both of standardized forms that utilities have to submit but also pleadings in individual cases. They do a lot of judicial determination type work. They're moving very quickly in trying to accommodate that.

The message we've given them, they've heard it and it's worked quite well, is for reporting allow companies to keep information they have to submit to EPA or states and so forth and the states pass along to EPA, in whatever form they typically keep their internal information. Whether it's Word,

WordPerfect, Adobe, a variety of formats, and allow that to be sent over and keep it as simple as possible.

FERC's got it now so you can do pleadings through their web site and you can use any of those various [ Microsoft] Word formats. They capture it, they turn it into an Adobe document 156 which preserves the pagination and so forth so it becomes a fixed document for the public record. You don't have to go through a lot of elaborate signature tracking and so forth. If you want to submit something, you have to have registered in the sense of signing on through the web site as who am I, where am I and contact information. I'm able to submit stuff for my boss, general counsel of our association, and he's also registered in there. And that's sort of the end of it, it's a five minute process once you've done that initial background work and it's very easy.

So that would be my observation. Electronic reporting can have lots of benefits, both in terms of us as reporting entities, the states and maybe more directly dealing with EPA,

but also for EPA in terms of managing this data. So just simple, simple, simple and let people use their existing software systems, is my suggestion.

Robin Roberts: Thank you. Any statements from the panelists and then I think we'll just wrap it up. Are there any outstanding questions? I guess I'd like to thank you all for coming. This is a road show, we'll be appearing in Dallas January 31 as well.

[ End of transcript]


[Federal Register Volume 86, Number 211 (Thursday, November 4, 2021)]
[Notices]
[Pages 60835-60837]
[FR Doc No: 2021-24067]

-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2019-E-5390]

Determination of Regulatory Review Period for Purposes of Patent
Extension; DENGVAXIA

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Food and Drug Administration (FDA or the Agency) has
determined the regulatory review period for DENGVAXIA and is publishing
this notice of that determination as required by law. FDA has made the
determination because of the submission of an application to the
Director of the U.S. Patent and Trademark Office (USPTO), Department of
Commerce, for the extension of a patent which claims that human
biological product.

DATES: Anyone with knowledge that any of the dates as published (see
SUPPLEMENTARY INFORMATION) are incorrect may submit either electronic
or written comments and ask for a redetermination by January 3, 2022.
Furthermore, any interested person may petition FDA for a determination
regarding whether the applicant for extension acted with due diligence
during the regulatory review period by May 3, 2022. See ``Petitions''
in the SUPPLEMENTARY INFORMATION section for more information.

ADDRESSES: You may submit comments as follows. Please note that late,
untimely filed comments will not be considered. Electronic comments
must be submitted on or before January 3, 2022. The https://www.regulations.gov electronic filing system will accept comments until
11:59 p.m. Eastern Time at the end of January 3, 2022. Comments
received by mail/hand delivery/courier (for written/paper submissions)
will be considered timely if they are postmarked or the delivery
service acceptance receipt is on or before that date.

Electronic Submissions

    Submit electronic comments in the following way:
     Federal eRulemaking Portal: https://www.regulations.gov.
Follow the

instructions for submitting comments. Comments submitted
electronically, including attachments, to https://www.regulations.gov
will be posted to the docket unchanged. Because your comment will be
made public, you are solely responsible for ensuring that your comment
does not include any confidential information that you or a third party
may not wish to be posted, such as medical information, your or anyone
else's Social Security number, or confidential business information,
such as a manufacturing process. Please note that if you include your
name, contact information, or other information that identifies you in
the body of your comments, that information will be posted on https://www.regulations.gov.
     If you want to submit a comment with confidential
information that you do not wish to be made available to the public,
submit the comment as a written/paper submission and in the manner
detailed (see ``Written/Paper Submissions'' and ``Instructions'').

Written/Paper Submissions

    Submit written/paper submissions as follows:
     Mail/Hand Delivery/Courier (for written/paper
submissions): Dockets Management Staff (HFA-305), Food and Drug
Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852.
     For written/paper comments submitted to the Dockets
Management Staff, FDA will post your comment, as well as any
attachments, except for information submitted, marked and identified,
as confidential, if submitted as detailed in ``Instructions.''
    Instructions: All submissions received must include the Docket No.
FDA-2019-E-5390 for ``Determination of Regulatory Review Period for
Purposes of Patent Extension; DENGVAXIA.'' Received comments, those
filed in a timely manner (see ADDRESSES), will be placed in the docket
and, except for those submitted as ``Confidential Submissions,''
publicly viewable at https://www.regulations.gov or at the Dockets
Management Staff between 9 a.m. and 4 p.m., Monday through Friday, 240-
402-7500.
     Confidential Submissions--To submit a comment with
confidential information that you do not wish to be made publicly
available, submit your comments only as a written/paper submission. You
should submit two copies total. One copy will include the information
you claim to be confidential with a heading or cover note that states
``THIS DOCUMENT CONTAINS CONFIDENTIAL INFORMATION.'' The Agency will
review this copy, including the claimed confidential information, in
its consideration of comments. The second copy, which will have the
claimed confidential information redacted/blacked out, will be
available for public viewing and posted on https://www.regulations.gov.
Submit both copies to the Dockets Management Staff. If you do not wish
your name and contact information to be made publicly available, you
can provide this information on the cover sheet and not in the body of
your comments and you must identify this information as
``confidential.'' Any information marked as ``confidential'' will not
be disclosed except in accordance with Sec.  10.20 (21 CFR 10.20) and
other applicable disclosure law. For more information about FDA's
posting of comments to public dockets, see 80 FR 56469, September 18,
2015, or access the information at: https://www.govinfo.gov/content/pkg/FR-2015-09-18/pdf/2015-23389.pdf.
    Docket: For access to the docket to read background documents or
the electronic and written/paper comments received, go to https://www.regulations.gov and insert the docket number, found in brackets in
the heading of this document, into the ``Search'' box and follow the
prompts and/or go to the Dockets Management Staff, 5630 Fishers Lane,
Rm. 1061, Rockville, MD 20852,<PHONE_NUMBER>.

FOR FURTHER INFORMATION CONTACT: Beverly Friedman, Office of Regulatory
Policy, Food and Drug Administration, 10903 New Hampshire Ave., Bldg.
51, Rm. 6250, Silver Spring, MD 20993,<PHONE_NUMBER>.

SUPPLEMENTARY INFORMATION:

I. Background

    The Drug Price Competition and Patent Term Restoration Act of 1984
(Pub. L. 98-417) and the Generic Animal Drug and Patent Term
Restoration Act (Pub. L. 100-670) generally provide that a patent may
be extended for a period of up to 5 years so long as the patented item
(human drug or biologic product, animal drug product, medical device,
food additive, or color additive) was subject to regulatory review by
FDA before the item was marketed. Under these acts, a product's
regulatory review period forms the basis for determining the amount of
extension an applicant may receive.
    A regulatory review period consists of two periods of time: A
testing phase and an approval phase. For human biological products, the
testing phase begins when the exemption to permit the clinical
investigations of the biological product becomes effective and runs
until the approval phase begins. The approval phase starts with the
initial submission of an application to market the human biological
product and continues until FDA grants permission to market the
biological product. Although only a portion of a regulatory review
period may count toward the actual amount of extension that the
Director of USPTO may award (for example, half the testing phase must
be subtracted as well as any time that may have occurred before the
patent was issued), FDA's determination of the length of a regulatory
review period for a human biological product will include all of the
testing phase and approval phase as specified in 35 U.S.C.
156(g)(1)(B).
    FDA has approved for marketing the human biologic product DENGVAXIA
(dengue tetravalent vaccine, Live). DENGVAXIA is indicated for the
prevention of dengue disease caused by dengue virus serotypes 1, 2, 3
and 4. DENGVAXIA is approved for use in individuals 9 through 16 years
of age with laboratory-confirmed previous dengue infection and living
in endemic areas. Subsequent to this approval, the USPTO received a
patent term restoration application for DENGVAXIA (U.S. Patent No.
8,142,795) from Sanofi Pasteur, and the USPTO requested FDA's
assistance in determining this patent's eligibility for patent term
restoration. In a letter dated December 26, 2019, FDA advised the USPTO
that this human biological product had undergone a regulatory review
period and that the approval of DENGVAXIA represented the first
permitted commercial marketing or use of the product. Thereafter, the
USPTO requested that FDA determine the product's regulatory review
period.

II. Determination of Regulatory Review Period

    FDA has determined that the applicable regulatory review period for
DENGVAXIA is 5,710 days. Of this time, 5,466 days occurred during the
testing phase of the regulatory review period, while 244 days occurred
during the approval phase. These periods of time were derived from the
following dates:
    1. The date an exemption under section 505(i) of the Federal Food,
Drug, and Cosmetic Act (21 U.S.C. 355(i)) became effective: September
14, 2003. FDA has verified the applicant's claim that the date the
investigational new drug application became effective was on September
14, 2003.
    2. The date the application was initially submitted with respect to
the human biological product under section 351 of the Public Health
Service Act (42

U.S.C. 262): August 31, 2018. The applicant claims August 30, 2018, as
the date the biologics license application (BLA) for DENGVAXIA (BLA
125682) was initially submitted. However, FDA records indicate that BLA
125682 was submitted on August 31, 2018.
    3. The date the application was approved: May 1, 2019. FDA has
verified the applicant's claim that BLA 125682 was approved on May 1,
2019.
    This determination of the regulatory review period establishes the
maximum potential length of a patent extension. However, the USPTO
applies several statutory limitations in its calculations of the actual
period for patent extension. In its application for patent extension,
this applicant seeks 1,248 days of patent term extension.

III. Petitions

    Anyone with knowledge that any of the dates as published are
incorrect may submit either electronic or written comments and, under
21 CFR 60.24, ask for a redetermination (see DATES). Furthermore, as
specified in Sec.  60.30 (21 CFR 60.30), any interested person may
petition FDA for a determination regarding whether the applicant for
extension acted with due diligence during the regulatory review period.
To meet its burden, the petition must comply with all the requirements
of Sec.  60.30, including but not limited to: Must be timely (see
DATES), must be filed in accordance with Sec.  10.20, must contain
sufficient facts to merit an FDA investigation, and must certify that a
true and complete copy of the petition has been served upon the patent
applicant. (See H. Rept. 857, part 1, 98th Cong., 2d sess., pp. 41-42,
1984.) Petitions should be in the format specified in 21 CFR 10.30.
    Submit petitions electronically to https://www.regulations.gov at
Docket No. FDA-2013-S-0610. Submit written petitions (two copies are
required) to the Dockets Management Staff (HFA-305), Food and Drug
Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852.

    Dated: October 28, 2021.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2021-24067 Filed 11-3-21; 8:45 am]
BILLING CODE 4164-01-P