Print

Print


My thoughts on these discussions so far:

 

         Do we need Benchmarks?

I'd suggest that benchmarks are very important to help us protect the service that we (as an HER) provide and (where we don't meet a benchmark) to provide evidence to our parent organisation to gain support for HER developments. Other organisations may also want to see that the HER is of a certain standard; and the benchmarks help with this. For example, archaeological contractors or academic researchers or district councils may be able to use the results of the Audits and Benchmarks to understand the quality of the data they are receiving, and/or to lobby for improvements to the HER (and the HER data) from the HER's parent organisation (where this is lacking). 

         Do you currently use the benchmarks?

As part of the audit process; and if we need to demonstrate to our parent organisation (the County Council) that the HER is lacking in certain areas. 

         How do you see development of the benchmarks going forward?

The benchmarks definitely need updating. The benchmarks as they currently stand (from 2002) need updating to incorporate changes in local government, changes in the planning system and agri-environment systems, and how HERs interact with other professionals / the public /etc. 

         How do you see the inter-relationship between benchmarks and the HER audit process?

I personally think that the Benchmarks and the Audit system should be very closely aligned. The Audit process acts as an external, funded, independent verification of the benchmarks - and as such allow HERs to help protect or develop their service (see Do we need Benchmarks?, above). 

         How do we measure ourselves against the Benchmarks?

HERs need some external moderation if we are to have an independent, robust and verifiable measurement against the Benchmarks. I think the English Heritage Audit process is the best way to do this. I think Rob's idea of getting the regional ALGAO HER groups comment is interesting - but in these times of dwindling resource, resources to do anything effective (over-and-above 'rubber-stamping' the EH Audit process) will be lacking. 

         How do audits and Benchmarks address specific issues like data quality?

I think the earlier discussion about data quality (from an archaeological contractors point-of-view) is interesting. I'd suggest that to ensure we have 'fit-for-purpose' datasets (particularly GIS datasets); we need better measurements for data quality, derived from better standards. This then needs to be incorporated within the Benchmarking and Auditing process. 

 

However, as Sarah eludes:

What do we mean when we say benchmark?

The current Benchmarks include sections on:

         User Services and Access

         Information Coverage and Content

         Information Management

         Organisation Management

I'd suggest that this is still broadly appropriate - but there are areas where we may need refinement. Also, I would argue that the Audit process could be more streamlined (although steps to do this have already taken place, there is still some-way to go). 

Also - making the results of the Audit and Benchmarks available publicly should be encouraged - this would enable greater openness and transparency - and allow us (and others) to lobby for improved HER services. Could (or should) - for example - English Heritage hold on their website a summary of each HER's Audit and how that HER has (or hasn't) met its benchmarks? 

 

Anyone else got any thoughts on this? I imagine that this is one of the main chances that we as the HER community can influence future Benchmarks and Audits.

 

Best regards,

 

Graham

 

-- 
Graham Tait 
Archaeologist 
Historic Environment Team 
Devon County Council 
Lucombe House 
County Hall 
Exeter 
EX2 4QD 
Telephone:  01392 382214 
Email: [log in to unmask] 
Web:  http://www.devon.gov.uk/historicenvironment <http://www.devon.gov.uk/historicenvironment>   
Disclaimer: http://www.devon.gov.uk/email <http://www.devon.gov.uk/email>  

 

 

From: MacLean, Sarah [mailto:[log in to unmask]] 
Sent: 30 October 2013 14:48
To: [log in to unmask]
Subject: Re: [HERFORUM] B&A2013 - Welcome to the HER Benchmark and Audit Discussion

 

Dear all,

 

I don't know if it is helpful but there is a brief summary of the audit process in Informing the Future of the Past at http://archaeologydataservice.ac.uk/ifp/Wiki.jsp?page=SectionB.2#section-SectionB.2-HERAudits . This briefly explains how they work.

 

The audit reports that English Heritage receive are treated in confidence and are not shared with third parties. However the HER may choose to share all or part of their reports, although we would recommend HERs work within Data Protection Act and any guidance their host organisation provides. If anyone has any further queries about the audit process I'm happy to answer these on or off list.

 

The Benchmarks are available in the HER Forum Files are at https://www.jiscmail.ac.uk/cgi-bin/filearea.cgi?LMGT1=HERFORUM&f=/Benchmark_and_Audit_Discussion (see pages 5 to 9 of the Benchmark document in the folder).

 

Thank you to everyone who is participating in the discussion, especially those who don't often post to the list. The contributions so far have been interesting and provoked useful discussion. Yesterday Rob asked 'what do we, as a community, want the HER benchmarks for?'. Perhaps considering this and the recent posts should we be asking what do we mean when we say benchmark?

 

all the best

 

Sarah

 

________________________________

From: Issues related to Historic Environment Records [mailto:[log in to unmask]] On Behalf Of Jenny Hall
Sent: 30 October 2013 13:48
To: [log in to unmask]
Subject: Re: B&A2013 - Welcome to the HER Benchmark and Audit Discussion

Just to pick up , our clients pay for every HER enquiry we make at a very high hourly rate ( I have just worked out we have paid over 1000 for HER data requests in the last year). 

 

I spend a lot of time explaining the history of the HERs to clients, why the data is as it is etc but there is a point beyond which it can't be defended and we are just left red-faced on behalf the whole archaeological community. 

 

We, as contractors, feed data back to the HER in the form of reports, comments and amended datasets amongst others.  Sometimes we would like to feel appreciated and included as part of the solution, not part of the problem.  We all know about the problems and issues surrounding HER data, but contractors are actually very quiet about the quality of the data they receive.  That is presumably perceived as them being happy with the data they get, but that may not be the case.

 

I do feel passionately about supporting HERs but have been finding it harder and harder to say the content is worthwhile when there are so many mistakes.  I fully understand staff involved in HER content management being defensive, been there, done that, but that doesn't make the problems go away.

 

As this dicussion is about benchmaking and auditing, how accessible are the resulting documents to users of the HERs?  The more information that is available about any HER, maybe makes it easier for the rest of the profession and the wider world to engage and work together to solve the problems.  

 

----- Original Message ----- 

	From: Robinson, Jonathan C <mailto:[log in to unmask]>  

	To: [log in to unmask] 

	Sent: Wednesday, October 30, 2013 1:12 PM

	Subject: Re: B&A2013 - Welcome to the HER Benchmark and Audit Discussion

	 

	Perhaps a leaf could be taken out of the Open Source software communities book here (in terms of data quality that is). In their world if someone has a specific requirement for an improvement to a  software feature they can either wait until the joint effort of contributors gets around to making the change based on perceived demand and the availability/capability of volunteer code contributors OR they can invest in getting their need met by 'incentivising' the development (i.e. paying for it to be speeded up by making a contribution to developers) which is then available for all other users but they get what they need (more or less) when they need it.

	 

	In a similar way if a requestor of data from an HER has a specific data requirement but it does not currently meet their needs they could wait for it to happen or if they are time limited take what the HER has, commission the improvement and feed it back into the HER for the benefit of others.

	 

	The HER is not, after all, free to produce and I am sure that does sometimes happen.

	 

	I appreciate that there is not for the most part a community of volunteer 'improvers' who could be 'incentivised' but I presume that contractors producing work for clients based on HER outputs are requested to feed back the results of any 'cleaning up' they have to do of the data or additional information gathered as a condition of the supply of data in a format that can easily be absorbed. How much that actually happens I have no idea. If contractors do not highlight the challenges HERs face to clients and encourage them to pay a contribution towards the work required to feed it back perhaps they are missing a trick. I appreciate they have to make their money from the contract but at the same time they should not be ashamed at the quality of HER data they present if they do not make it clear to clients why it is as it is.

	 

	I do not want to set off a fire storm here and I also appreciate this is slightly off topic but I am not often moved to comment. Aspiring to these 'higher standards' is all most local authority funded HERs can manage under the current regime of cutbacks and threats to their existence. The methodologies to address shortcomings I would have thought can only realistically consist of getting others to fund improvement (charity/third sector/'clients').

	 

	Jon 

	 

	From: Issues related to Historic Environment Records [mailto:[log in to unmask]] On Behalf Of Jenny Hall
	Sent: 30 October 2013 10:54
	To: [log in to unmask]
	Subject: Re: B&A2013 - Welcome to the HER Benchmark and Audit Discussion

	 

	In rely to Chris Wardle's comments I accept that I am looking for better quality standards than most HERs can afford but that doesn't stop them being something to aspire to and to develop methodologies to address.  The problems I am talking about are to do with actual errors in the NGRs, not issues to do with linears, extensive sites/complexes, ill-defined records from antiquarian sources etc etc

	 

	If we want the whole archaeological community and well beyond to support HERS they have to be worth having and fighting for.  I have hung my head in shame on a couple of occasions recently when our client had looked at the raw data from an HER before we had cleaned it up and I realised what they had seen. 

	 

	Obviously not all HERs are the same and some will be shining examples but in order for global support they all have to be reaching certain standards.  

	 

	 

	 

	 

	 

	----- Original Message ----- 

		From: Chris Wardle <mailto:[log in to unmask]>  

		To: [log in to unmask] 

		Sent: Wednesday, October 30, 2013 10:36 AM

		Subject: Re: B&A2013 - Welcome to the HER Benchmark and Audit Discussion

		 

		As it stands benchmarking is there to try to ensure that all HERs meet certain minimum standards across.

		-the bare minimum of the data on each record (not even requiring a description!)

		-a minimum level of access

		-a minimum level of staffing

		-that they be digital records linked to a GIS.

		 

		It was and is a worthwhile exercise. For instance without it the Heritage Gateway would not have been possible, but the principal aim, to make HERs a statutory service has never been achieved The closed we have some is the statement in the NPPF that every LPA should have or have access to an HER.

		 

		They were largely the work of Stewart Bryant in the early 2000s, based on the work David Baker carried out in the 1990s.  There were more 'aspirational' level 2 benchmarks, but in these days of salami sliced budgets few HERs have aspirations rising much above survival .

		 

		In this context Jenny's comments are not very helpful. Accuracy of things like NGRs is very difficult to measure and hence near impossible to include specific requirements in a benchmark: The level of accuracy that would be vital for a market cross would be unachievable for a reported antiquarian find.  If she has specific complaints about the accuracy of data from specific HER/s she ought to take them up with that/those HER.

		 

		For my part the glaring omission in the current benchmarks is a standard level of data for a GIS. 

		 

		 

		From: Issues related to Historic Environment Records [mailto:[log in to unmask]] On Behalf Of Jenny Hall
		Sent: 29 October 2013 12:57
		To: [log in to unmask]
		Subject: Re: B&A2013 - Welcome to the HER Benchmark and Audit Discussion

		 

		One question is how does the auditing and benchmarking actually ensure that a dataset is fit for purpose?  

		 

		This may sound a daft question but as an example we are currently finding that any one HER enquiry (typically for a 2km radius search area) is giving an error rate in the NGR of records of anywhere from1% up to 20-25% .  That includes records that are completely in the wrong place, ie many kilometres distant and ones that are "only" a few tens or hundreds of metres wrong.  Much of this has come from pushing a paper based, more intuitive record into the unforgiving digital environment but it is very hard to identify without going through record by record and checking.  Some errors might be caught by checking and changing 6 figure NGRs to 8 or 10 figure, but others would be very hard to detect.

		How does benchmarking and auditing address this particular issue?  It is one that is of vital importance if we are to argue, that hand on heart, our HERs are worth having.

		 

			----- Original Message ----- 

			From: EDWARDS, Robert (Environment) <mailto:[log in to unmask]>  

			To: [log in to unmask] 

			Sent: Tuesday, October 29, 2013 12:17 PM

			Subject: Re: B&A2013 - Welcome to the HER Benchmark and Audit Discussion

			 

			Hi,

			From my perspective I think there are three fundamental questions that the HER community needs to address first.

			 

			Do we need HER benchmarks? 

			I think this is a good question to ask and personally I think the answer is a resounding yes, but it's fundamentally tied up with the answer to the second question. 

			 

			What do we, as a community, want the HER benchmarks for? 

			I think that the HER benchmarks can be used to define what a HER or HER service is. Thanks to the NPPF access to a HER is now a requirement for a planning authority, but although we have a lot of guidance (IFP2 etc.), we seem to lack a full and proper definition. Personally I think this is important in helping us to protect our services at a time when discretionary spending in local authorities is under such pressure. I also feel that reporting our progress against the benchmarks helps inform those authorities supporting the HER as to the range and quality of the service which they are paying for. Furthermore, failure to reach a national benchmark or standard is valuable evidence when trying to garner support for HER development.

			 

			How do we measure ourselves against the benchmarks? 

			Personally I think the HER Audits should be an important part. The audit should provide the baseline data from which we can measure ourselves against the benchmarks, BUT we will still need some form of external moderation if we are to achieve a robust measurement. Should this be an integral part of the English Heritage audit or perhaps we could submit our assessment to our regional ALGAO HER groups for comment? With the latter, it does provide an opportunity to provide an ALGAO 'stamp of approval', the former English Heritage's.

			 

			Best wishes

			Rob

			 

			 

			Rob Edwards 
			Historic Environment Records Officer
			Cheshire Archaeology Planning Advisory Service

			 

			Tel: 01244 973667

			Email: [log in to unmask] <mailto:[log in to unmask]> 

			Location: The Forum, Chester, Cheshire, CH1 2HS. 

			-----Original Message-----
			From: Issues related to Historic Environment Records [mailto:[log in to unmask]] On Behalf Of MacLean, Sarah
			Sent: 28 October 2013 08:05
			To: [log in to unmask]
			Subject: B&A2013 - Welcome to the HER Benchmark and Audit Discussion

			 

			Dear all,

			 

			Today we are launching the HERs Benchmarks and Audits discussion here on HER Forum.  The discussion will remain open until 5pm on 15th November to ensure as many list members are able to participate as possible. For the background to this discussion please refer to my e-mail of last week - https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=herforum;620e7b93.1310 <https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=herforum;620e7b93.1310>  . Before we start I thought it would be useful to outline how we intend for this to work.

			 

			Last week we provided some broad themes we hope to cover in the course of the discussion. To help structure and focus the discussion, and to start things off, we have set out a number of specific questions at the end of this e-mail (although discussion is not limited to these).

			 

			As mentioned in my previous e-mail we have put a copy of the 2002 Benchmark report and the current Audit specification in the file area for reference. The files are in a folder called 'Benchmark and Audit Discussion' (see https://www.jiscmail.ac.uk/cgi-bin/filearea.cgi?LMGT1=HERFORUM&f=/Benchmark_and_Audit_Discussion <https://www.jiscmail.ac.uk/cgi-bin/filearea.cgi?LMGT1=HERFORUM&f=/Benchmark_and_Audit_Discussion> )

			 

			The HER Forum list will still be open for business as usual so do feel free to post on other topics. To make it easier to follow the discussion please include B&A2013 in the subject line of your e-mail.

			 

			The HER Forum e-mail list is open to anyone with an interest in HERs and all list members are welcome to participate in the discussion. Please can you ensure that when you post, even as a reply, you sign off with your name and organisation (if applicable). Can I also take this opportunity to remind list members that the HER Forum archives are publically available online and can be read by non list members.

			 

			If you want to change your e-mail settings please use the HER Forum FAQ (available here https://www.jiscmail.ac.uk/cgi-bin/filearea.cgi?LMGT1=HERFORUM <https://www.jiscmail.ac.uk/cgi-bin/filearea.cgi?LMGT1=HERFORUM> ) for details on how to do this.

			 

			We always welcome feedback on the format of these discussions, what worked and what didn't. Please get in touch with me off list if you have any comments (including ideas for future topics) we can use to improve future discussions.

			 

			If you have any queries about this specific discussion, please get in touch with me off list.

			The discussion is now open. To start off the discussion the questions we would like you to consider are:-

			 

			*   Do you currently use the benchmarks?

			    *   If so, for what purpose?

			    *   If you do not use the benchmarks, why?

			*   How do you see development of the benchmarks going forward?

			*   How do you see the inter-relationship between benchmarks and the HER audit process?

			In your response it would be helpful if you could indicate if you have undertaken an audit, and if so when.

			 

			Over to you!

			 

			We hope you will all enjoy the discussion.

			 

			best wishes

			 

			Sarah

			 

			Sarah MacLean MA MIfA

			Heritage Information Partnerships Supervisor English Heritage Designations Department Engine House Firefly Avenue Swindon

			SN2 2EH

			E-mail: [log in to unmask]<mailto:[log in to unmask] <mailto:[log in to unmask]:[log in to unmask]> >

			Telephone: 01793 414880

			 

			This e-mail (and any attachments) is confidential and may contain personal views which are not the views of English Heritage unless specifically stated. If you have received it in error, please delete it from your system and notify the sender immediately. Do not use, copy or disclose the information in any way nor act in reliance on it. Any information sent to English Heritage may become publicly available.

			 

			Portico: your gateway to information on sites in the National Heritage Collection; have a look and tell us what you think. 

			http://www.english-heritage.org.uk/professional/archives-and-collections/portico/ <http://www.english-heritage.org.uk/professional/archives-and-collections/portico/> 

			************************************************************************
			Disclaimer:
			
			If you are not the intended recipient of this email (and any attachment), please inform the sender by return email and destroy all copies. Unauthorised access, use, disclosure, storage or copying is not permitted.
			The views expressed by the author do not necessarily reflect the views or policies of Cheshire West and Chester Borough Council. The Council cannot guarantee that this message or any attachment is virus free or has not been intercepted and amended. You should perform your own virus checks.
			Cheshire West and Chester Borough Council may monitor emails and as a public sector organisation; the Council may disclose this email (or any response to it) under the Freedom of Information Act 2000.
			Contracts cannot be concluded with the Council nor service effected by email, unless otherwise expressly agreed. The contents of this e-mail may be subject to privilege.
			************************************************************************

	
	_____________________________________________________________________________
	Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www-935.ibm.com/services/us/index.wss/offerfamily/iss/a1026954
	_____________________________________________________________________________

	
	_______________________________________________________________________
	This e-mail contains confidential information (which may also be legally privileged) and is intended solely for the use of the intended named recipient.
	
	If you are not the intended recipient you may not disclose, 
	copy, distribute or retain any part of this message or its 
	attachments. If you have received this message in error please notify the originator immediately by using the reply facility in your e-mail software.
	
	Incoming and outgoing emails may be monitored in line with 
	current legislation.
	
	All copies of the message received in error should be destroyed. 
	Any views or opinions expressed are solely those of the original author. This email message has been scanned for viruses, and declared to be virus free at the point of exit from Cumbria County Council's network.
	
	http://www.cumbria.gov.uk/_
	____________________________________________________________________
	This email has been scanned by the Symantec Email Security.cloud service.
	For more information please visit http://www.symanteccloud.com
	______________________________________________________________________


This e-mail (and any attachments) is confidential and may contain personal views which are not the views of English Heritage unless specifically stated. If you have received it in error, please delete it from your system and notify the sender immediately. Do not use, copy or disclose the information in any way nor act in reliance on it. Any information sent to English Heritage may become publicly available.

Portico: your gateway to information on sites in the National Heritage Collection; have a look and tell us what you think. 
http://www.english-heritage.org.uk/professional/archives-and-collections/portico/