Managing Electronic Public Records
Recognizing Perils and Avoiding Pitfalls
Welcome to the Managing Electronic Public Records tutorial. This online tutorial takes approximately 45 minutes to complete and must be completed in one session. Throughout the tutorial, a list of resources or links may appear in the sidebar on the right. These include other publications and guides created by the Government Records Branch of the Department of Cultural Resources, as well as resources created by other states and federal agencies.
When you are ready to start the tutorial, move through the pages using the page numbers at the top of each page, or by clicking on "Previous Page" and "Next Page" at the bottom of the page. Next to the page numbers at the top of the page, move your mouse over TOC to see the Table of Contents as a drop-down box with page numbers and the section titles that can be used to jump to a specific section.
Throughout the tutorial you will find Quiz Me and Self Check icons that you should click to answer short matching or true/false questions. Information about the quizzes you complete will be tracked but will not be reported; at the end of the tutorial you will have the option to print a certificate or your score results, but this is not required. Also notice in the top right-hand corner, below the title, is a print all link that allows you to print the tutorial for reference.
The information covered here is taken from the Managing Electronic Public Records workshop offered by the Government Records Branch of the State Archives of North Carolina. If you have questions or would like more information, please visit the Government Records Branch Web site to locate the appropriate contact information.
In this tutorial we will:
Electronic records are public records and subject to the same laws as paper records. Two general statutes address the government's legal obligation to take care of its records. The first is the Archives and History Act, G.S. 121, which assigns records management authority to the Department of Cultural Resources (DCR), regulates the destruction of public records, and instructs DCR to provide assistace to public officers in managing records. Read G.S. 121 in its entirety via the link in the sidebar.
G.S. 132 is the Public Records Act, which identifies public records as the property of the people. This means that we, as state employees, are required to maintain public records and provide access to them upon request. G.S. 132 also defines the term public record.
G.S. § 132-1(a)
"Public record" or "public records" shall mean all documents, papers, letters, maps, books, photographs, films, sound recordings, magnetic or other tapes, electronic data-processing records, artifacts, or other documentary material, regardless of physical form or characteristics, made or received pursuant to law or ordinance in connection with the transaction of public business by any agency of North Carolina government or its subdivisions.
Public records include anything created in the course of public business regardless of format. This includes any type of electronic record, and could be an Access database, e-mail, Word documents, images, or Web sites. Click the link in the sidebar to learn more about the Public Records Act.
All material you create and handle in your role as a state employee should be managed according to your agency's records retention and disposition schedule, which is an agreement between DCR and your agency.
A records retention and disposition schedule lists records as items. The general schedule covers the items that are generally applicable to all state agencies, like budget records and personnel files. The program schedule is specific to each agency and the records that agency creates; for example, the program schedule for DENR will be different from ITS. If there are any differences in how the general and program schedules instruct you to handle a record, always follow your agency-specific program schedule. The program schedule is developed by the Government Records Branch in consultation with your agency's staff. Any differences reflect the needs of your agency.
In reviewing your agency's schedule, if you notice anything that seems outdated, or missing information, contact your chief records officer (CRO), who is the point person for your agency when making changes to the schedule.
Now that we've reviewed the laws that affect how we manage any public record, let's talk about what makes electronic records special.
So what, exactly, is an electronic record?
An electronic record is a series of 1s and 0s that require computer hardware and software to interpret and present. Unlike information written on paper, or available on microfilm, electronic records are not human-readable. They require something – hardware and software – to translate those 1s and 0s into a language we can understand and see.
Because electronic records rely on technology to be accessible, they require active management if you plan on being able to read them more than five years from now. We'll talk about why technology is so "high maintenance" in a moment.
Just like public records in any other format, electronic records must be managed according to a records retention schedule.
If you have questions about how you should be managing any records, review your retention schedule to make sure you are following the guidelines your agency has agreed upon. Some records might need to be printed out, while others might be okay remaining in their electronic form.
Format matters! The correct formats must be noted in the retention schedule. If you are managing records differently from your retention schedule, talk to your CRO about possible changes and why you recommend them. Then the CRO can work with a records analyst in the Government Records Branch to update the retention schedule if needed.
Protecting confidential material can be even more complicated when electronic records are involved. When you get rid of your computer, for example, then you must make sure that all confidential material is removed from your hard drive. Just deleting it will not do it. Deleted does not mean it is erased. Computer forensics can recover "deleted" files from a computer. Now, ideally, this isn't something you will have to deal with, but do keep in mind that when deleting information you should make every effort to completely remove that information, even doing something as simple as emptying the Recycle Bin. More advanced tools are available, including software that "shreds" the hard drive to completely remove any information.
Perhaps it makes sense to keep your records with confidential material in a separate location, with stricter access restrictions – off the network, or just on one computer.
Electronic records with confidential information are also more tricky to share, because you must make sure you completely redact that confidential information. As mentioned when reviewing G.S. 132, the public must be able to access the records upon request. If you show someone records containing confidential material, then you must remove the confidential material before you give it to them.
Examples of failures in this area include one county in Virginia that posted public documents containing social security numbers and signatures on the internet. The county is now rescanning the documents without the SSN, but only for those who call and complain.
Accessibility in general is becoming more complicated. Marriage licenses online have dates of birth and mothers' maiden names (often used by credit card companies to verify a person's identity.) This information has always been available at the court house, but with the increase in identity thefts, people do not want their information so easily accessible.
One of the simplest mistakes to make when handling confidential information is to improperly redact information. In a case from several years ago involving a sniper in Washington, D.C., the Washington Post put a PDF version of the sniper's letter on their Web site that detailed the sniper's demand for money to be deposited into a stolen credit card account. Certain personally identifying details were blacked out in the PDF file. Anyone with the full commercial version of Adobe Acrobat (not the free Reader version) could easily remove the blacked-out part to reveal the text underneath. The best way to redact information like that is to black out the information in a PDF, Word document, or image file, and then save it again as a new file, in an image format like TIFF or JPEG. We'll talk more about these formats later.
Of course, if you're going to provide hard copies of the files, then you can black out the information on the electronic file and then print it out.
Because of the variety of concerns that accompany managing electronic records, threats to digital files encompass a range of issues, including user error, problems with storage media, and obsolescence of software and hardware.
Sometimes, data is lost because of technology; sometimes, it is lost because of human error. It seems as if every day brings a new story of missing records caused simply by negligence or mistakes. In 2007, Iron Mountain lost a back-up drive with Social Security numbers and bank account information for Louisiana students who had applied for federal student aid over nine years. The data was lost while being transferred between storage facilities. The truck driver was fired for failing to follow company security procedures and the Louisiana Office of Student Financial Assistance took steps to protect thousands of affected students from identify theft and fraud.
It is important to make sure everyone knows their responsibilities in managing and protecting public records, regardless of format.
Another type of human error is loss of records due to disorganization. While search functions are getting better and better, proper organization from the start is the best way to make sure you, and others in your office, can find records when needed.
Naming records correctly is a standard held over from managing paper records that is equally applicable, perhaps even more so, for electronic records.
electronic_records_KE_2007_10_29.ppt or
ElectronicRecordsWorkshopKE20071029.ppt
This is an example of the naming convention used in the Electronic Records Unit. This is the name of the electronic records workshop as saved on the department's server. It is also filed correctly in folders, in
Government Records Branch/Workshops/Current
If someone else needs to access it in the future, it will be clear what the file contains without opening it. Another important thing to note is that the name doesn't include any special characters like dashes or question marks. In places in which there is normally a space there is an underscore. This helps the computer read the file and follow the correct links.
Organization and file-naming are particularly challenging when working with databases and Web sites. If you are responsible for these or other types of records in your office, make sure that you are creating them or maintaining them according to accepted best practices. Does your office have a file-naming convention, or rules for saving files in a certain location? DCR offers a document listing best practices for file-naming, which you can read by following the link in the sidebar.
As technology advances, storage media changes. We have gone from punch cards, 5 1/4 inch floppies, and 3 1/2 inch disks, to flash drives, solid state drives (SSD), and optical media like DVDs. If you have data on these older media, will you be able to access it?
DVDs are the most common type of optical discs used today. CDs are still used, but DVDs can hold significantly more information and are more popular. DVDs were originally used for video storage, hence the name "digital video disc." However, because of their popularity now, the name has changed to "digital versatile disc."
Increased storage is one of the primary reasons all other forms of storage media have become obsolete. Even now, as we move to exclusively using DVDs, we have begun to see a rising popularity of high-density, or high-definition discs. Only recently, in February of 2008, did Blu-ray win the battle of high definition optical discs against HD-DVD (similar to the VHS vs. Betamax battle), when Toshiba, the main producer of HD-DVD disc players and burners, ceased production of this hardware.
A DVD can store 4.7 GB of data; a Blu-ray DVD can store 50 GB. (A CD can store 700 MB.)
To ensure longevity, just like when handling any other media such as paper, the quality is important, as are storage conditions. Inexpensive burnable discs have the shortest lifespan, while higher quality discs are available with greater longevity.
"Many of the cheap burnable CDs available at discount stores have a life span of around two years," Gerecke said. "Some of the better-quality discs offer a longer life span, of a maximum of five years."
The National Institute of Standards and Technology has developed a set of standards for handling optical media. The most important rules are to:
Learn more by following the link to the standards in the sidebar.
As storage capacities increase, the size of storage devices decrease. How long before we are all carrying a personal storage device for electronic information? Many of us already do - at the State Archives all employees have a flash drive to use when moving and sharing files.
When storing data for work, try to keep your storage options to a minimum – when you or someone else needs to find that information, you don't want to have to search. Also, using flash drives for storage is also not a good idea because it means important files could be offsite when needed. Use flash drives only for moving files, or for carrying a copy to an offsite event. The "original" should stay in the office.
This list of threats to digital storage media can for the most part be applied to any records – especially the environmental issues. It's important to protect your records, no matter how or where they are stored.
It is difficult to overestimate the importance of backups. Restoring a back-up is much easier than recreating a file, even a back-up that is a week old. That being said, it isn't exactly easy to retrieve data from back-ups, and you should treat back-ups as what they are – a way to protect against accidents and errors.
Be consistent with back-ups - create and follow a back-up schedule. If your department has a server, store important documents there, as it is most likely backed up daily. When you back up your computer, remove the disk or hard drive from the computer, and store it somewhere else. Make sure through periodic checks that the back-ups actually contain the content they are supposed to have, and that they are labeled properly.
Ideally you will have more than one back-up. For example, if you back up your files on DVD, consider also storing them on an external hard drive, or make another set of DVDs to store at another location.
You can find a link to more information about back-up procedures in the sidebar on the right.
Another effect of constantly evolving technology is the rapidity with which hardware and software become obsolete. This means not only that offices must work to maintain the most efficient software and hardware, but also that we work to ensure access to older records that have been created with these out-of-date tools. When software or hardware changes, will you still be able to access those older files? Do you still have Betamax tapes, or even VHS tapes? How about a Betamax deck or a VCR?
When talking about software, there are a few common phrases that might be thrown around that make a big difference:
File formats are often associated with the software that creates them. For example, .doc files are created by Microsoft Word. Some file formats are non-proprietary, which means they can be created, used, and modified by a variety of software and are not created exclusively for use with one program. An example of a non-proprietary file format is .txt, which can be read in Microsoft Word, but also in free programs like Notepad, or Open Office.
|
Best |
Medium |
Worst |
|
XML (.xml) |
CSS (.css) |
|
|
Plain Text (.txt) |
Rich Text (.rtf) |
Word (.doc/.docx) *** |
|
XHTML/XSL |
HTML 4.x |
|
|
PDF/A-1 |
PDF with embedded fonts |
PDF with external fonts |
|
TIFF (.tif) |
JPEG (.jpg) |
GIF (.gif) |
This chart lists common file formats and sorts them based on their appropriateness for storage.
Generally the file formats to the left have a better chance of being able to last into the future than the formats on the right, primarily because they are open source—that is, the source code and specifications have been made openly available. It is difficult to preserve something if you don't know how it has been made, and open-source formats generally have more community involvement in their development and maintenance.
Word has been placed to the right because it is a closed, proprietary system, but there are some caveats. Microsoft Office has been adopted as the official software of state government and is, of course, widely used throughout the rest of the world, so at the Government Records Branch we have an obligation to figure out how to preserve Microsoft Office products even though they use proprietary software.
Notice that there are several versions of PDF on the chart. PDF/A - a is for archival - is a recent development by Adobe that, based on the provision for more metadata and the release of some of its source code, is a more appropriate preservation format than the original PDF format. The International Standards Organization (ISO) has approved pdf/a as an accepted preservation standard for digital files. Adobe has also embedded the fonts—made them an unalterable part of the document, so that the font in which the document was written remains a part of the electronic record, ensuring that preserved records retain the "look and feel" of the original.
If you have a choice of file format for an archival electronic records series, try to use one from the left. This is particularly important if you are in charge of developing new systems. And remember DCR can always advise you on these matters.
Talking about choosing what file formats to use is important for managing electronic public records not just as an individual but as an office. While taking steps to manage your own records as a state employee is important, developing and following office policies and standards will ensure consistent, active management of public records throughout the office.
The first type of policy an office should have applies to all types of records, not just electronic. A disaster recovery plan is vital to make sure everyone in the office knows what to do in preparation for, and if necessary, after any type of disaster. The plan should include contact information for the appropriate people in your office – generally those who will be in charge of any disaster recovery activities, or those who will be the point person for a specific area of the office. For example, the Government Records Branch's disaster recovery plan lists people who have access to our records storage facility and who will have the most information about what records are where.
You need names and phone numbers of data recovery vendors – whom are you going to contact if you lose information?
Information about your back-ups of all data needs to be included. How can you access the data, where is it stored, and whom do you need to contact?
Test the plan. Have a mock disaster and find out what your plan's shortcomings are. Just as you might hold fire drills, consider having a disaster drill every six months to see if your plan will work.
Always remember that DCR can serve as a resource to answer any questions.
Your office should have an electronic records policy. The Electronic Records Unit provides guidelines on their Web site that can be used to develop policies specific to your office's needs.
The basics that should be included are:
Everyone needs to have a password on their computer. There need to be guidelines available to help employees develop good, safe passwords for their computer, so that everyone knows to use long, complicated, frequently changed passwords.
Should employees use the server or network to store files for back-up, or are they responsible for backing up any files on their desktop? Routine back-ups should take place, and be stored in secure off-site storage. Current recommendations for hurricane prone areas indicate that back-ups should be stored over 80 miles from the original site, in a region less affected by hurricanes.
Protect the hardware – consider access to computers, disks, external hard drives - but also protect the software and files as well – who can modify files, or move them?
Apply the best practices identified by DCR in the file-naming document that's available on the DCR Web site (follow the link in the sidebar) to implement file-naming standards for your office. What abbreviations will you use? What kind of file structure?
All policies should comply with state and federal laws.
Once you have a policy make sure that everyone knows what that policy is. Be sure and train new employees, too. Making training a regular part of your records management will ensure compliance.
A policy that is definitely already in place is your agency's records retention and disposition schedule. It is important that the agency make sure the retention schedule is updated as needed. This is especially important to remember when your office is changing what formats records are saved as. If your agency is moving from printing and filing records, to storing them digitally – in a database, or even just Word documents – that change needs to be noted in your schedule. For example, if your office is considering a scanning project to replace your paper records, please consult DCR first.
When you're working with your records analyst to make these changes, he or she can also be a resource to answer questions and provide advice concerning the best formats. Remember that electronic formats are not always the best choices for long-term storage of records. Microfilm is a very stable, human-readable format that should be considered for archival records.
The following factors should be considered when your office chooses the formats that are the best choice for both the office and for long-term access. Once these decisions are made, it's a good idea to incorporate these standards into your office policy, listing recommended formats for documents, spreadsheets, images, and any other file type, including databases. Does your office support Office 2007, or is Office 2003 still the standard? Do you want all final documents stored as PDFs or TIFFs?
"Digital information lasts forever – or five years, whichever comes first." - Jeff Rothenberg
Realizing how easily digital files can be altered can be overwhelming. Active security and management are critical.
It is essential that any planning you do in your office concerning the retention of electronic records take into account the financial and human resources needed to maintain electronic records and the software, hardware, and storage needed to preserve them.
Never trust claims that you won't have to actively manage records over time. Recognize that hardware and software are going to change. Make sure that you have a way to get your records out of proprietary or vendor-based systems.
No matter what changes you make to ensure access, it's important to keep the original bit-stream – the original file, in its original format – to be able to prove the record's history and provenance.
And active management also means checking on these files periodically to make sure that they are still accessible – that all of the technology required to read that file is still working properly, and the media on which the files are stored are still readable.
Digital archives follow the LOCKSS principle: Lots Of Copies Keeps Stuff Safe. Make sure you have back-ups of your data. Keep multiple copies in multiple locations. Consider storing files in more than one file format simultaneously; for example, as a rich text format file and PDF.
We've already reviewed the limitations of proprietary software. Make sure that the policies and standards your office adopts are supported by several vendors, so that if one company goes out of business, you can still access your data.
Once you have committed to preserving your electronic records, there are some steps to take to prevent problems with storage media and file formats.
Refreshing is replacing the media on which digital files are stored. For example, considering the lifespan of some CDs and DVDs, you might want to refresh data on older discs every two or three years.
Migration is converting your files to a new format or a new version of a format. You could migrate your .doc files to .rtf format, for example. This is useful when collaborating with colleagues who might be using different software; migrating files to a less proprietary format will ensure accessibility.
Emulation is backwards programming that enables modern computers to read old programming languages. One of the most popular examples of this is emulating 1980s video games to play them on your computer today. You can now play Pacman and Oregon Trail, for example, and they will look just as they did twenty years ago. Emulation is not really something your office wants to incorporate into policies and planning. It's useful to know about, but given the technical knowledge needed, you probably are not going to consider it unless there's a situation involving permanent records with no other alternatives.
These rules to live by are a summary of the ideas we've talked about in this tutorial.
Recognize that managing electronic records requires active, sustainable management and that the challenges of preserving records with long-term retention can apply to electronic records with retention periods of as little as 10 years. Digital preservation will always involve risks – that is why it is important to plan for them and mitigate them as much as possible by avoiding single points of failure and unnecessary software dependencies, defining policies and responsibilities, and planning ahead to allocate resources for sustainability.
The two most important take-aways from this tuturial are:
Government Records Branch: www.records.ncdcr.gov
DCR Archives & History: www.history.ncdcr.gov
Published Guidelines: www.records.ncdcr.gov/erecords
Back-up Procedures: www.records.ncdcr.gov/erecords/BackupsProcedsfinal020822.pdf
Workshops at the Government Records Branch: www.records.ncdcr.gov/workshops.htm
Managing Your Inbox: E-mail as a Public Record (online tutorial): www.records.ncdcr.gov/tutorial_email_20080801/index.html
North Carolina State Government Web Site Archives
www.archive-it.org
www.archives.ncdcr.gov/webarchives/index.html
EMCAP: E-Mail Collection And Preservation
www.records.ncdcr.gov/EmailPreservation
GeoMAPP: Geospatial Multistate Archive and Preservation Partnership
www.geomapp.com
For more information, please contact us or check out additional resources available online through our Web site.
Government Records Branch | Department of Cultural Resources
Physical Address: 215 N. Blount Street | Raleigh, NC 27601
Mailing Address: 4615 Mail Service Center | Raleigh, NC 27699-4615
Phone: 919-807-7350 | Fax: 919-715-3627
Email Address: records@ncdcr.gov
A list of each agency's chief records officer, as well as each agency's records analyst, can be found here.