The MINERvA data implementation requires a lot of attention as previous effort stalled. I am actually in the middle or rewriting (or writing for the first time) the spec/requirements for this development, following discussions with Marco, based on recent developments in Comparisons fmwk and the understanding from implementing several other comparisons.
I think Julia’s skills might be more suited for expanding the unit testing. I think she could be helpful with MINERvA comparisons too, as she understands the Comparisons code. 
But, someone who is involved at some level with MINERvA and understands the measurements might be more efficient.
In any case, I would also like Marco to have an oversight on this since, at this point, he understands the details of the Comparisons framework better than anybody else.
C.

On 4 Dec 2018, at 18:38, Dytman, Steven A <[log in to unmask]> wrote:

Going forward, we are asking Julia to work on expanding unit tests (about 3
examples in there now) and adding recent data to Comparisons framework,
e.g. Minerva data is only up to 2015.  Finding right balance between these
2 tasks is hard.  Any thoughts?

I add Walter because he is her supervisor now and I don't think he's on this list.

Steve

On 12/4/2018 12:09 PM, Constantinos Andreopoulos wrote:
[log in to unmask]" class="">Yes. Interestingly, CCQE sections / Ma reweight is one of the few things for which there is a test, by JuliaY, within the UnitTesting framework.
I hope this is how we caught this error. Would be rather depressing if it was not.
C

On 4 Dec 2018, at 18:03, Gabriel Nathan Perdue <[log in to unmask]> wrote:

Won’t help with this problem now, but in general this also argues for investments in unit tests and even more rigorous automated validation.

pax
Gabe

On Dec 4, 2018, at 11:16, Constantinos Andreopoulos <[log in to unmask]> wrote:



On 4 Dec 2018, at 16:16, Gabriel Nathan Perdue <[log in to unmask]> wrote:


On Dec 4, 2018, at 12:16 AM, Constantinos Andreopoulos <[log in to unmask]> wrote:

I hope we are learning lessons with some of the codes that were developed without planning / designing / documenting, without thinking all aspects through before starting to type lines in the editor. They are codes that we have very hard time understanding, evaluating, improving now. Again, a detailed document is a pre-requisite for any new development - We want to see that there is thought put into the design of the code, the integration with and potential ramifications to other parts of GENIE etc.
This project *should* have been caught much earlier, during the development of the new QE generator!!

Creating an incubator project does not amount to just adding a name in the GENIE web page. It amounts to getting that document written, reviewed and agreed.
We have 22 projects (that I know of, and I should have known them all) in incubation, all of which have a GENIE docDB number (see web page) with a skeleton for a detailed planning document (scope / deliverables / milestones / requirements / outline of development plan / validation plan / review points / interdependencies with other projects / references etc etc)
This needs to be done properly (and not just mock it up by adding a couple of lines) for all projects we are working on. 


We can and should do project documentation, of course, but I think the take-away from this should be that we really need to partner more firmly with the theorists who wrote the original model. Or possibly provide only interfaces and ask them to contribute code that targets those interfaces (and take responsibility for the model functioning). 

As long as we have complete responsibility for everything, we're only going to run into more problems like this and it will only get worse as models get more complicated. Our collaboration doesn't have the bandwidth to manage _all_ of the complexity.

This is true, but in this case the problem is not with a theoretical calculation. 

The problem is how we use an input theoretical calculation to generate 4-vectors. Kinematical transformations / Jacobians and pulling numbers out a p.d.fs is all that is needed here.
So this is firmly on our side, and this is the thing we will always have the responsibility of doing, in order integrate any theoretical calculation into our broader simulation.

Here, the problem is purely a wrong and non-sustainable attitude towards GENIE devel (or softw devel in general): Do the absolute minimum and compress initial development to 1 day, instead of 1 week, tick a box and claim a success, and then spend weeks on end trying to figure out problems and refactorize from within a straightjacket. Especially this specific QE code hit us twice to date! Back when Marco was starting to characterise all comprehensive models and writing a detailed tech note, he hit a problem there (QE generator was not updating the maximum cross-section used for the acceptance/rejection method as, apparently, all testing was done at a fixed energy and didn’t consider we generate events over a range…). It took Marco/Steve weeks to sort that out, because  fixing this brought into sharp focus speed issues etc. It threw us completely off track trying to finish and document the model characterisation.
Just look at this code - It probably took (Andy or Rik or someone else who worked on updating QE modelling) a couple of hrs to put together, starting from a template I provided. It never went beyond being a test code, yet we included it in a public version and then it took us weeks and weeks discussing and correcting it. We are still not done and, it seems, a right longer-term course of action is to just rewrite the whole thing anyway! With inefficiencies like this, yes, we don’t have bandwidth to handle anything. Interestingly, all people who worked on updating the QE modelling wrote papers on the GENIE implementations and took the credits. Are they still around to help us fix the issues? Which reinforces my belief we should veto any publication or public code releases till the work is done to a satisfactory standard.

cheers
C



To unsubscribe from the NEUTRINO-MC-CORE list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=NEUTRINO-MC-CORE&A=1




To unsubscribe from the NEUTRINO-MC-CORE list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=NEUTRINO-MC-CORE&A=1