ISIS preparation and use, Master dark, bias and flat frames?

Post Reply
Hamish Barker
Posts: 226
Joined: Thu Jun 20, 2019 12:11 am

ISIS preparation and use, Master dark, bias and flat frames?

Post by Hamish Barker »

HI All,

I have been using ISIS to prepare master dark and bias and flat frames but would like to confirm what the program does and what should be used when reducing spectra.

Bias frames:
- ISIS master bias preparation expects a series of bias frames and i assume averages them. Ok, seems straightforward.
- in ISIS general tab, I use the master bias fits file. all good.

Dark frames:
- ISIS master dark preparation asks for master bias and a series of darks.
Can anyone confirm, that as I assume, it averages the darks, then subtracts the master bias?
So the resulting master dark would be dark current only, since bias has been subtracted.

Then in the isis general tab, if I use both a master bias and a master dark, isis appears to scale the master dark frame according to the actual target spectrum frame duration (if different to the master dark. ) and then subtracts that master dark and also subtracts the master bias from the target spectrum frames?

OR does ISIS assume that the master dark may be has NOT had the bias frame subtracted, and therefore first subtracts the nominated master bias frame first before time scaling?

Also, what happens if I do not provide a master bias frame in the general tab, and only a master dark (for example because I might have a series of dark frames of the same duration and temperature as the target spectrum frames). Will ISIS take the lack of provided bias frame as meaning no scaling of dark frame need be applied?

The latter case might be able to provide a slightly lower noise, since when calculating a master bias frame, there is still a little residual noise (since the bias frame is based on average of a finite number of bias frames), and subtraction of the master bias from the stack of master darks means the (stacking reduced) read noise gets doubled.

Does anyone have any information / details on the algorithm which ISIS applies in use of bias and dark frames for the general tab?

Similarly, master flat preparation in the masters tab asks for master bias and dark frames. I struggle to get sufficient brightness at the blue end of my flat frames, and as a result, low signal means bias frame treatment isn't negligible. Sometimes ISIS completely fails to produce a useful spectrum if I apply flat in the general tab, so usually I don't use them at all. I have seen some discussion of whether flats are even mandatory elsewhere, since problems with them can mess up the whole data reduction.

Cheers,
hamish
Hamish Barker
Posts: 226
Joined: Thu Jun 20, 2019 12:11 am

Re: ISIS preparation and use, Master dark, bias and flat fra

Post by Hamish Barker »

103 views, but no responses.

In case anyone who knows the internal programming of ISIS (Christian does I guess - perhaps I should contact him directly?), I'd really appreciate (and perhaps others would also) any responses.

Cheers,
Hamish
Robin Leadbeater
Posts: 1926
Joined: Mon Sep 26, 2011 4:41 pm
Contact:

Re: ISIS preparation and use, Master dark, bias and flat fra

Post by Robin Leadbeater »

Can anyone confirm, that as I assume, it averages the darks, then subtracts the master bias?
So the resulting master dark would be dark current only, since bias has been subtracted.


Correct! (actually it probably uses median rather than average to remove outliers)

if I use both a master bias and a master dark, isis appears to scale the master dark frame according to the actual target spectrum frame duration (if different to the master dark. ) and then subtracts that master dark and also subtracts the master bias from the target spectrum frames?

Correct!

Also, what happens if I do not provide a master bias frame in the general tab, and only a master dark (for example because I might have a series of dark frames of the same duration and temperature as the target spectrum frames). Will ISIS take the lack of provided bias frame as meaning no scaling of dark frame need be applied?

No!

If you want to do this you must produce a "long exposure master bias", enter this as the bias and leave the dark entry blank

With some cameras the thermal signal can be very low and it takes a long time to take enough darks to produce a good low noise master dark so to reduce noise some people do not use a dark and instead use a master bias together with a cosmetic calculated from the dark to heal the warm cool pixels.

Similarly, master flat preparation in the masters tab asks for master bias and dark frames. I struggle to get sufficient brightness at the blue end of my flat frames, and as a result, low signal means bias frame treatment isn't negligible. Sometimes ISIS completely fails to produce a useful spectrum if I apply flat in the general tab, so usually I don't use them at all. I have seen some discussion of whether flats are even mandatory elsewhere, since problems with them can mess up the whole data reduction.


I always use a flat with slit spectrographs and I have never had ISIS fail to produce a useable spectrum with a flat applied.

1. it removes ripples in the camera response which are otherwise difficult to remove from the instrument response using a standard star

2. If I see an interesting feature in the spectrum I do not know it is an artifact if I dont have a flat.

The application of a halogen flat exaggerates the appearance of noise in the flat corrected image at the blue end because it makes the image artificially brighter at that end. I recommend comparing the result with and without a flat. The difference in noise may not be as large as you think. Having said that it can be difficult to get enough signal in the UV with a halogen lamp but you should not have any problem down to say 3800A and flats are normally short exposures so it does not take long to make a big enough stack to reduce the noise (I typically average 30) The system for making a flat has to be good though, for example the internal flat in the LHIRES does not work well at the blue end because there is too much undispersed scattered light.

Cheers
Robin
LHIRES III #29 ATIK314 ALPY 600/200 ATIK428 Star Analyser 100/200 C11 EQ6
http://www.threehillsobservatory.co.uk
Robin Leadbeater
Posts: 1926
Joined: Mon Sep 26, 2011 4:41 pm
Contact:

Re: ISIS preparation and use, Master dark, bias and flat fra

Post by Robin Leadbeater »

Robin Leadbeater wrote:
I always use a flat with slit spectrographs and I have never had ISIS fail to produce a useable spectrum with a flat applied.
Here is an example using the ALPY with a flat on a bright star at very high SNR where any deterioration in SNR due to the flat would be obvious. The noise is still not excessive even at 3650A
gamoph_20210919_854_Leadbeater.png
gamoph_20210919_854_Leadbeater.png (24.14 KiB) Viewed 3692 times
Cheers
Robin
LHIRES III #29 ATIK314 ALPY 600/200 ATIK428 Star Analyser 100/200 C11 EQ6
http://www.threehillsobservatory.co.uk
Robin Leadbeater
Posts: 1926
Joined: Mon Sep 26, 2011 4:41 pm
Contact:

Re: ISIS preparation and use, Master dark, bias and flat fra

Post by Robin Leadbeater »

In my view, people (amateurs but also some professionals) get obsessive about improving SNR when the real "elephant in the room" are the systematic errors which are much more important and are harder to detect and control. (I will generally be happy to trade some SNR to reduce systematics)

For example getting an SNR of 300 say on a bright star which means the 3 sigma error due to noise is <1% is straightforward but who can honestly say their response correction is accurate to 1% ?

You see the same thing in variable star photometry. If you look in the AAVSO data you will see claims of uncertainties of <0.01 mag brightness, calculated by the software they use but you then look at the scatter in the actual data and it is much higher.

Cheers
Robin
LHIRES III #29 ATIK314 ALPY 600/200 ATIK428 Star Analyser 100/200 C11 EQ6
http://www.threehillsobservatory.co.uk
Hamish Barker
Posts: 226
Joined: Thu Jun 20, 2019 12:11 am

Re: ISIS preparation and use, Master dark, bias and flat fra

Post by Hamish Barker »

Thanks very much robin.

Particularly the comment about long exposure master bias. That makes sense.

I will persevere with the flats. I now have a couple of different halogen lamps ( one is a dichroic reflector, which is not good I guess since who knows what fancy ripples can be present in the reflector spectrum) and the other is one of those building site work lights with a long thin halogen bulb and a plain aluminium reflector.

Now I have managed to servo automate my main flip mirror with an Arduino, next plan is also to automate switching of flat and calibration lamps and add a servo to the L200's little flip mirror for the calibration lamp. Although all of those operations happen less often than the main flip mirror switching from platesolve camera to spectrograph input, so the productivity gains are less.
Hamish Barker
Posts: 226
Joined: Thu Jun 20, 2019 12:11 am

Re: ISIS preparation and use, Master dark, bias and flat fra

Post by Hamish Barker »

So, when the flat division happens, at the blue end due to the low adu counts, it ends up increasing the pre-response signal, right? Then the reference star response division operation should bring it back down to the actual calibrated value.

When I had problems it was with a scaled dark so perhaps that was the issue, not the flat.
Post Reply