Project

General

Profile

Actions

Task #11561

open

Small-scale MC production for testing the effect of LST-MST stereo trigger

Added by Moralejo Olaizola Abelardo almost 6 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assigned To:
Start date:
12/01/2015
Due date:
% Done:

0%

Estimated time:

Description

Produce MC gammas & protons with the Prod-3 telescope layouts (starting with Paranal) for two different trigger settings for LSTs:

1. the standard ones in Prod-3

2. new settings (higher thresholds) for which the rate of stereo accidental triggers in LSTs (at 2*darkNSB) would be the same as with the settings in (1), once stereo triggering of each LST with any of its surrounding MSTs is also allowed (besides "any 2 LSTs"). Finding out the values for these new settings is the goal of another task within the "LST stereo trigger study" subproject.

The goal is to check whether, for a fixed rate of recorded accidentals, we can improve the array performance by allowing LST+MST stereo triggers.

For the MC production it is convenient to save all events with >= 1 triggered telescopes of any type. The stereo condition can be implemented later at the analysis level.


Files

LST_MST_trigtest_energy.pdf (15 KB) LST_MST_trigtest_energy.pdf Moralejo Olaizola Abelardo, 02/10/2016 04:06 PM
LST_MST_roughtest.tiff (203 KB) LST_MST_roughtest.tiff Moralejo Olaizola Abelardo, 02/10/2016 04:14 PM
trigTest_LaPalma_partial_2.png (12 KB) trigTest_LaPalma_partial_2.png Terzić Tomislav, 03/07/2016 05:41 PM
trigTest_Paranal_set1_2.png (11.8 KB) trigTest_Paranal_set1_2.png Terzić Tomislav, 03/07/2016 05:41 PM
AngRes.png (21.4 KB) AngRes.png Angular resolution Terzić Tomislav, 04/12/2016 10:23 AM
EffectiveAreaEtrue.png (17.7 KB) EffectiveAreaEtrue.png Effective area Vs. true energy Terzić Tomislav, 04/12/2016 10:23 AM
ERes.png (23.9 KB) ERes.png Energy resolution Terzić Tomislav, 04/12/2016 10:23 AM
ERes_log.png (21.8 KB) ERes_log.png Energy resolution with log scale Terzić Tomislav, 04/12/2016 10:23 AM
Actions #1

Updated by Moralejo Olaizola Abelardo over 5 years ago

Moralejo Olaizola Abelardo wrote:

Produce MC gammas & protons with the Prod-3 telescope layouts (starting with Paranal) for two different trigger settings for LSTs:

1. the standard ones in Prod-3

2. new settings (higher thresholds) for which the rate of stereo accidental triggers in LSTs (at 2*darkNSB) would be the same as with the settings in (1), once stereo triggering of each LST with any of its surrounding MSTs is also allowed (besides "any 2 LSTs").

A correction: actually the trigger thresholds (in the standard Prod-3 configuration) were set in 1*darkNSB conditions (see issue #11560). And the number of accidental triggers will change little with the extra alowed stereo triggers (because of the lower single-telescope rates of MSTs), so we expect a small change of thresholds.

I have just submitted to the development version of MARS some changes in the stereo reconstruction routines which allow to implement mixed LST-MST stereo triggers. For now the telescope combinations have to be set inside CTAstereo.C (an example is inside the current CVS version of the macro).

I attach an example of the results, obtained with 10 gamma files (20 deg zenith angle) from Prod-3 @ La Palma. The 2D plot shows a scatter plot of core positions of triggered events with the normal trigger (i.e. requiring >=2 LSTs or >=2 MSTs) and with the "enhanced trigger" in which each LST can also trigger with its three closest MSTs ("outwards", i.e. the central MST is not included in any combination). The enhanced trigger provides, at trigger level, 2.5% more events in the E<300 GeV range, and the core distributions look reasonable. There is a larger concentration of "new" events south of LST-1, because it is the LST closest to the edge of the MST array footprint, so the one which "recovers" more events which are triggering just the LST and one additional MST.

Actions #2

Updated by Moralejo Olaizola Abelardo over 5 years ago

One more comment: in the previous entry I am checking only "new events" (completely absent in the normal trigger sample), which are recorded thanks to the combined MST-LST trigger.

There will also be events which were already triggered and recorded by >=2 MSTs with the normal trigger, and which now, with the enhanced trigger, will have an extra LST available for the shower reconstruction (formerly ignored because there was no companion LST to trigger the LST sub-system). A quick check shows that these may be as many as 20% of the events in the shown La Palma example. The additional LST will improve the reconstruction of those events, so the effect of the proposed LST+MST has to be tested post-analysis as well, at the level of sensitivity and angular&energy resolution.

Actions #3

Updated by Terzić Tomislav over 5 years ago

Moralejo Olaizola Abelardo wrote:

Moralejo Olaizola Abelardo wrote:

Produce MC gammas & protons with the Prod-3 telescope layouts (starting with Paranal) for two different trigger settings for LSTs:

1. the standard ones in Prod-3

2. new settings (higher thresholds) for which the rate of stereo accidental triggers in LSTs (at 2*darkNSB) would be the same as with the settings in (1), once stereo triggering of each LST with any of its surrounding MSTs is also allowed (besides "any 2 LSTs").

A correction: actually the trigger thresholds (in the standard Prod-3 configuration) were set in 1*darkNSB conditions (see issue #11560). And the number of accidental triggers will change little with the extra alowed stereo triggers (because of the lower single-telescope rates of MSTs), so we expect a small change of thresholds.

But we should still be able to use the standard Prod-3 settings, at least to start with, right?

A few questions considering the settings:
1. We should use trigger_telescopes = 1; stereo triggers are only implemented at the analysis level, right?
1a. What about mono events? Are they also considered, or does the choice inside CTAstereo.C discard mono events?
2. Do we need to simulate the whole array, or we can use only inner circle of MSTs? Using only part of an array would certainly save time and disk space, but I'm not sure if only partially filled gamma_20deg_0deg_run274___cta-prod3-merged_desert-2150m-Paranal-subarray-?.simtel.gz files will be usable. We could try to use Data/tmp/ files gamma_20deg_0deg_run274___cta-prod3-lst_desert-2150m-Paranal-subarray-0-lst.simtel.gz + gamma_20deg_0deg_run274___cta-prod3-mst-nc_desert-2150m-Paranal-subarray-?-mst-nc.simtel.gz. The results that Dijana showed in Munich were obtained from gamma_20deg_0deg_run274___cta-prod3-lst_desert-2150m-Paranal-subarray-0-lst.simtel.gz files only, but they all had the same name format, so we didn't have any troubles. I don't know if the same trick would work with '*_cta-prod3-lst_*' + '*_cta-prod3-mst-nc_*' combination of files.

Actions #4

Updated by Moralejo Olaizola Abelardo over 5 years ago

But we should still be able to use the standard Prod-3 settings, at least to start with, right?

Yes, absolutely.

A few questions considering the settings:
1. We should use trigger_telescopes = 1; stereo triggers are only implemented at the analysis level, right?

Correct, this is done inside the stereo reconstruction task.

1a. What about mono events? Are they also considered, or does the choice inside CTAstereo.C discard mono events?

No, they are not. Shower reconstruction would be very poor, and it is not even possible with the current MARS-CTA analysis to do it. Mono events will not be written to disk in CTA.

2. Do we need to simulate the whole array, or we can use only inner circle of MSTs? Using only part of an array would certainly save time and disk space, but I'm not sure if only partially filled gamma_20deg_0deg_run274___cta-prod3-merged_desert-2150m-Paranal-subarray-?.simtel.gz files will be usable.

From what I posted yesterday you can see that most of the "new" events are outside the MST array footprint, in the area where its edge is closest to MSTs. Therefore it seems very dangerous to do the test with partial arrays: the advantage of having mixed LST-MST triggers would appear exaggerated.

[…] but they all had the same name format, so we didn't have any troubles. I don't know if the same trick would work with '*_cta-prod3-lst_*' + '*_cta-prod3-mst-nc_*' combination of files.

Not sure what the different file sets are… Do you have access to the CTA computing grid? If not, please get it. Then you can download standard Prod-3 Paranal files, at least to do some preliminary tests with good statistics, which will help you in determining what needs to be produced, and how. The La Palma site standard Prod-3 files can be obtained at Konrad's site in Heidelberg, even already in chimp-output image-parameters version: [[https://www.mpi-hd.mpg.de/personalhomes/bernlohr/cta-chimp/Prod-3/LaPalma/]]. There is also a smaller Paranal production (in simtel output format) under [[https://www.mpi-hd.mpg.de/personalhomes/bernlohr/cta-raw/Prod-3/]]

Actions #5

Updated by Moralejo Olaizola Abelardo over 5 years ago

A quick check shows that these may be as many as 20% of the events in the shown La Palma example.

Erratum: actually it is just about 7% of the events which, with the new trigger, contain an LST which had been dropped with the normal trigger (due to lack of another LST in the event). This has anyway to be checked better, and after analysis as well.

Actions #6

Updated by Terzić Tomislav over 5 years ago

I produced plots like Abelardo's LST_MST_trigtest_energy.pdf. There are 2 plots, one for La Palma using Konrad's simulations, and one for Paranal using our own data.

For La Palma I used 17 gamma_20deg_0deg_run*___cta-prod3-lapalma-2147m-LaPalma-NectarCam_I.root files with 163525 gamma showers. 92950 of those caused LST+LST stereo triggers, and 95295 caused LST+MST stereo triggers. The difference for the whole energy range (0 < log10(E/GeV) < 6) is 2345 triggers, which sums up to 2.5%. Taking only energy range in which the trigger number difference is > 1 (0.78 < log10(E/GeV) < 3.06, i.e. ~6 < E/GeV < 1150), the difference is ~2.8%. For E < 300 GeV, the difference is 2.4% - 2.5%.

The star files used for Paranal contained 84679 gamma showers. 32716 caused LST+LST stereo triggers, while 33008 caused LST+MST stereo triggers. The difference is 292 events, which is 0.9% of LST+LST triggers, for the full energy range (0 < log10(E/GeV) < 6). Considering only bins in which the difference between the number of LST+LST and LST+MST triggers is > 1, i.e. 1.2 < log10(E/GeV) < 1.9 (~15 < E/GeV < ~85) the difference is 285. In that energy range, the total number of LST+LST triggers is 14990, and 15275 LST+MST triggers. The difference is 1.9% LST+LST triggers (for that energy range).

Actions #7

Updated by Terzić Tomislav over 5 years ago

I forgot to mention in the previous post:
The analysis for La Palma was done for configuration 3AL4M15-2, so it can be compared to Abelardo's result. The analysis for Paranal was done for subarray_1 and configuration 3HB1-NG.

Please let me know, if there is some other parameter I forgot to mention.

Actions #8

Updated by Terzić Tomislav over 5 years ago

Just another explanation (sorry for not writing it all at once): when quoting a number of events, I always read it from the Energy container from the same branch for all three cases (star file, superstar file with LST stereo trigger only, and superstar file with LST+MST stereo trigger included). For La Palma, it is MMcEvt_1.fEnergy, and for Paranal MMcEvt_4.fEnergy (only LSTs 4, 5, 6 and 7 in that particular configuration). This is probably obvious, but just to be on the safe side.

Actions #9

Updated by Terzić Tomislav over 5 years ago

Dear all,

here are comparisons of energy resolution, angular resolution and effective area for PureLST Vs. LST+MST stereo triggers. It was done for Paranal, subarray_1, configuration 3HB1-NG.

The plots are results of CTAsensi.C, for 0 hours. At first I ran CTAsensi.C for 50 hours, but those processes are still running. However, none of these results should depend on time, so I ran the same thing for 0 hours, which was done quite quickly. CTAsensi is still running for 50 hours. I will compare the results when it is done.

I used RFs that Abelardo gave us, gamma files from our own production to run CTAflux.C (63 files with a total of 1E5 events in star files). I also used a single electron and proton files from our own production to run CTAflux.C (I needed these to run CTAsensi.C). Therefore statistics for protons and electrons is rather low, but again, this should not affect the resolution.

I see no significant difference between results obtained with LST+MST stereo trigger compared to Pure LST stereo trigger.

Actions #10

Updated by Moralejo Olaizola Abelardo over 5 years ago

Hi,

Terzić Tomislav wrote:

The plots are results of CTAsensi.C, for 0 hours. At first I ran CTAsensi.C for 50 hours, but those processes are still running. However, none of these results should depend on time, so I ran the same thing for 0 hours, which was done quite quickly. CTAsensi is still running for 50 hours. I will compare the results when it is done.

did you mean 0.5 hours ? (instead of 0). Note that, contrary to what you wrote, results may depend on time. When we calculate sensitivity for different observation times it is not just the time itself that changes, but also the cuts (hadronness, theta2, multiplicity) are optimized to achieve the best sensitivity (according to our standard definition) in the given time. This is relevant because in shorter time obviously the minimum detectable flux is larger, hence the relative background rate smaller, and the limit "excess at least 5% of residual background" is easier to fulfill. This ends up in looser cuts resulting from optimization. And of course, different cuts will affect things like angular and energy resolution.

I used RFs that Abelardo gave us, gamma files from our own production to run CTAflux.C (63 files with a total of 1E5 events in star files). I also used a single electron and proton files from our own production to run CTAflux.C (I needed these to run CTAsensi.C).

That approach will have an unpredictable result. The background files (electrons & protons) will be used in the cut optimization. If background statistics is very poor, no optimization will be possible, so I do not know what kind of cuts may have come out... Instead of using CTAsensi as a black box, you should rather try to look at the data at a "lower level", from the CTAflux outputs:

  • apply some typical cuts in each range of estimated energy you want to explore, like 50 - 70% gamma efficiency (from hadronness cut), ~<0.01 deg^2 in theta2, and see what is the effect of the MST-LST trigger in energy and angular resolution (playing also with different telescope multiplicities).
  • to do the above you will have to develop some custom code, a simple root macro to apply the cuts and calculate ang. and energy resolution.
  • since it looks like the differences are going to be small (the ones you show in your plots actually show worse performance (??) when you have the LST-MST trigger that allows to keep some LST images that would otherwise be "lost"), you should also show the differences in reconstruction evaluated just for the subsample of events which actually "gain" an LST. That is, look for the events which actually benefit from having an additional LST available for reconstruction, and check how much closer do the reconstructed quantities (direction, energy) get to the real quantities. This is a basic consistency check to see if there is the possibility of having an actual gain in performance. Of course, we aim at improving the final performance, so in the end one has to evaluate the impact in the total sample, but this "restricted test" is needed to make sure things are working.
  • About significance of differences, note that you are testing accuracy of reconstruction parameters in two event samples which are very correlated (same events, reconstructed in different ways), so even differences within the error bars might not just be fluctuations.
Actions

Also available in: Atom PDF