For legal admissibility, it does depend if you can show that your scanned copy is a 'true copy' of the original - and Mark's 'sensible precautions' outline the way to do this.
But I assume these aren't 'ordinary office documents' - there would be no point in scanning say Purchase Invoices in colour.
And if you are scanning maps or plans or drawings or artistic designs or historic documents, using a lossy compression is harder to justify - there tends to be more variation in this type of document and you may find yourself doing the 'sensible precautions' test repeatedly for very small batches of documents.
I notice ISO 19005-1, whilst allowing lossy JPEG compression, states:
"PDF/A writers should not use lossy compression, subsampling, downsampling, or any other process that either alters the content or degrades the quality of source data in the PDF/A document.
These best practice recommendations are designed to assure the authenticity of records produced according to specific image resolution, compression or prohibited processes that either alter or dispose of approved data."
This National Archives document
http://www.nationalarchives.gov.uk/documents/information-management/digitisation-image-specification.pdf
shows their requirements for digitisation projects, all options of which specify lossless compression. Though I'm not sure I agree with the requirement that only either JPEG2000 (lossless option) or TIFF can be used - JPEG2000 isn't read by all imaging software yet - and has PDF/A been considered ?.
Some organisations recommend compressing with JPEG using 'highest quality' (or 'Quality = 100', depending what your software offers). I suspect this may have originated in the mistaken belief that 'highest quality' means lossless - but in the case of JPEG (or TIF or PDF files compressed with JPEG) this isn't true, there will still be some data loss.
In practice, this issue comes down to what type of documents you have and can you show your scanned copy is a 'true copy'. Also consider the volume of documents - if there are relatively few disk storage is cheap nowadays.
As Mark says, it is difficult to imagine a scenario where there would be a legal issue - but you know your own documents better than we do.
Geoff mentioned "NXpowerLite" software to reduce storage. I'm not familiar with this software but before using it I would want to be sure it was lossless compression (that is it's compression could be 'undone' to return any file back to it's original state).
ISO 19005 states:
"If external compression is to be applied to a PDF/A-1 file, either for storage or transmission, a lossless compression method must be used. The use of lossless compression will ensure that the integrity of such a file is preserved, since lossy compression methods on PDF documents will damage the document, usually beyond repair."
This refers specifically to PDF/A files but I consider it applies to all (archival) files.
Best regards,
Steve
-----Original Message-----
From: The Information and Records Management Society mailing list
[mailto:[log in to unmask]]On Behalf Of Marc Fresko
Sent: 25 October 2012 20:06
To: [log in to unmask]
Subject: Re: Legal admissibility of compressed images in EDRMS
I do not fully agree with Steve. Louise, you should go ahead and (with sensible precautions) use whatever form of compression you want, lossy or not.
The 'sensible precautions' consist of testing the scanning and compression, and keeping the results. No big deal, a small handful of scans will do the trick. Scan standard test pages and/or pages with fonts at a variety of small weights and sizes, with varying resolutions and compression ratios. You might find that some extreme settings can result in the loss of very small decimal points - which would make those settings unacceptable (I assume a decimal point is the smallest thing you might want to preserve when a page is scanned). You probably will find that 'sensible' settings mean that even small decimal points are preserved. Choose a suitable set of settings. Keep the test results, complete with the original test documents; and make sure that your chosen settings are clearly stated in your scanning procedures, and that the procedures are observed. To be really tough, check the settings every few weeks and repeat the tests similarly, plus certainly every time you change a scanner. That's it. It is so simple that it won't take you much longer than it took me to type this (OK, I'm exaggerating a bit...)
The circumstances under which lossy compression with the above can cause an evidential issue are difficult to imagine. Some legal opponent would have to (a) imagine the challenge - unlikely; (b) mount the challenge successfully - even more unlikely; (c) show what the document should have looked like and present evidence to support that assertion (well...) In other words, the courts will presume in your favour unless a challenge is successful, but in practice you won't be challenged unless you choose some inappropriate combination of parameters.
One final proviso: there might be a dependency on what kind of document you are scanning. I wrote the above assuming you were talking about 'ordinary' office documents - letters, invoices and the like. These rarely need colour, though you may face the demand for colour. But if you are talking scanning inactive documents for your archive, or photos, it is even more difficult to imagine a risk.
Good luck.
Marc Fresko
-----Original Message-----
From: The Information and Records Management Society mailing list [mailto:[log in to unmask]] On Behalf Of Steve Norris
Sent: 25 October 2012 10:50
To: [log in to unmask]
Subject: Re: Legal admissibility of compressed images in EDRMS
Hi Louise,
Certainly there is no need to store raw TIF files. LZW compressed TIF is a good format - it is "lossless" compression so all data is retained. You can use JPEG compression with TIF files, but JPEG is a "lossy" compression so some data is lost - though often this is not noticeable (particularly with photographs) to the naked eye. JPEG compressed TIF files will be smaller than LZW compressed TIF. There used to be legal issues (patents) preventing the use of LZW but this isn't a problem now. Some more information here: http://www.alliancegroup.co.uk/tiff.htm
PNG isn't widely used for archiving. You may also consider PDF/A (the archiving format of PDF), though you have the same issues with compression here - and LZW compression isn't available in PDF/A.
It is this question of "lossy" or "lossless" compression that is the key to the issue. Lossless compression (such as LZW) shouldn't be an issue for legal admissibility. Lossy compression is more questionable. The Code of Practice for Legal Admissibility shows that scanned documents will be accepted by a court, but they can be challenged. Can you therefore show that your scanned copy is a 'true copy' of the original ? With lossless compression this is more straightforward, with lossy compression it may be more difficult to prove. Lossy compression reduces the evidential weight of the image.
Best regards,
Steve
-----Original Message-----
From: The Information and Records Management Society mailing list [mailto:[log in to unmask]]On Behalf Of Louise Pichel
Sent: 25 October 2012 09:28
To: [log in to unmask]
Subject: Legal admissibility of compressed images in EDRMS
Dear all,
We are currently investigating long term storage of colour images in our EDRMS and the potential storage implications.
Storing the raw TIFF files for the required 7 year retention period has been rejected due to the large amount of storage space required, and we have been pointed towards both the LZW compression algorithm and/or storage as PNG files instead.
I would be very grateful for any comments/suggestions on both the use of LZW compression and PNG's for long term storage in terms of legal admissibility. I have looked at the Code of Practice for Legal Admissibility but it doesn't go into much detail about compression specifically.
I would be more than happy to summarise responses to the list.
Many thanks.
Louise
Assistant Archivist
Library and Museum of Freemasonry
To view the list archives go to: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=RECORDS-MANAGEMENT-UK
To unsubscribe from this list, send an email to [log in to unmask] with the words UNSUBSCRIBE RECORDS-MANAGEMENT-UK
For any technical queries re JISC please email [log in to unmask] For any content based queries, please email [log in to unmask]
This message has been scanned by MailController - portal1.mailcontroller.co.uk=
To view the list archives go to: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=RECORDS-MANAGEMENT-UK
To unsubscribe from this list, send an email to [log in to unmask] with the words UNSUBSCRIBE RECORDS-MANAGEMENT-UK
For any technical queries re JISC please email [log in to unmask] For any content based queries, please email [log in to unmask]
To view the list archives go to: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=RECORDS-MANAGEMENT-UK
To unsubscribe from this list, send an email to [log in to unmask] with the words UNSUBSCRIBE RECORDS-MANAGEMENT-UK
For any technical queries re JISC please email [log in to unmask]
For any content based queries, please email [log in to unmask]
This message has been scanned by MailController - portal1.mailcontroller.co.uk=
To view the list archives go to: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=RECORDS-MANAGEMENT-UK
To unsubscribe from this list, send an email to [log in to unmask] with the words UNSUBSCRIBE RECORDS-MANAGEMENT-UK
For any technical queries re JISC please email [log in to unmask]
For any content based queries, please email [log in to unmask]
|
