Spectra Problems

Design, construction, tuning of spectroscopes
Information and discussion about softwares (telescope remote, autoguiding, acquisition, spectral processing ...)
Keith Graham
Posts: 126
Joined: Thu Sep 29, 2011 6:20 pm

Spectra Problems

Post by Keith Graham »

Hi All,

i am sorry if this is a repeat post. thought I posted this earlier today but it did not appear in the forum. So I am making a second attempt.

I have recently noticed a big change in my BeSS spectra that I am hoping someone can help me solve. A few weeks ago my spectra started having an undulating continuum along with some recurring big bumps at around 6546A and 6626A. After many hours of attempting to narrow down the cause, I have concluded that it has something to do with my flat image and/or with the processing routine.

My equipment consists of a 12” SCT, Lhires III with 2400 grating, and SBIG ST8mxe camera. Image acquisition is done with Maxim DL and image processing is done with ISIS v8 (and I have recently upgraded to v 9.2). I have always reduced my images using darks, flats, bias, and instrument response images. My issue is described as follows:
Up until a couple of weeks ago, my processing methodology described above has produced spectra that have been validated by the BeSS staff. When I recently sent in about 20 spectra, they were not validated due continuum undulation and bumps as described above. Previously, my specs used to have smooth continuum and no bumps. Important: all of those previous images were flat fielded and validated. Now something has caused a reversal in my results. When I flat fielded my recent spectra, they appear as in my archived specs when no flat was used (undulating continuum with bumps). However, if I now do not use a flat, the continuum is not undulating and the bumps are gone, just like the earlier specs that were flat field corrected. I tried without success to include these images as attachments, but ARAS apparently does not accept fits files or other extensions. I even tried including them in the body of this email with no success.

In attempt to narrow down the exact cause, I pulled up some spectral images taken a couple of years ago. Those specs were perfect and were processed with ISIS using my flat image, IR, bias, and dark. The continuum was smooth and there were no bumps. However, when I ran them through again using the same flats, darks, and IR curve used when I originally processed them, the results showed undulating continuum and bumps. Without the flat field the continuum was level and there were no bumps.

These results tend to make me believe something has changed in my processing routine since the archived specs that were originally submitted showed no bumps and undulating continuum when flat fielded. Those imperfections now disappear when the flat field is left out of the processing routine, just like my newer specs. So next I tried processing with different versions of ISIS but the results are the same.

After many hours of trying to figure this out, I now have two questions:
1) Does anyone have any idea how such a reversal in my flat fielded results could have occurred and what a remedy might be?
2) I have seen different opinions on the value of using flats. Since I am now getting much better results when not using flats, would not using them leave a spectrum that would be imperfect in any way that would render them unusable? I would prefer to use flats as I always had success using them. But this recent bizarre occurrence has me so perplexed I cannot figure out what is happening and think not using them is the only way around this situation.
Thank you in advance for any suggestions you might have that would rectify this. BTW, I would deeply appreciate it if anyone has a suggestion as to how I can get my spectra attached. Pictures would illustrate what I am experiencing much better than only my description.

Cheers,

Keith Graham
Andrew Smith
Posts: 321
Joined: Fri Sep 30, 2011 12:23 pm

Re: Spectra Problems

Post by Andrew Smith »

Hi Keith. I am not an ISIS expert but as no one else has responded I thought I would have a go.

Firstly, are you using the same calibration frames as you always have or are they new?

If you process your reference hot (A, B) star (used for the instrument calibration) is it ok?

Try processing the old and new Flats as stars (turn off background subtraction General tab -> sky not removed).

In MaxIm I would inspect all the frames with the Line profile tool to see if any frame have the lumps and bumps.

Similarly in ISIS keep all the intermediate images and profiles (switch in settings) and inspect them in MaxIm (images) and ISIS for profiles.

Do the processing steps you can manually in ISIS and inspect the results.

The aim is to localise where the issue is coming from.

Try processing in BASS or other software if you can't pin it down with MaxIm and ISIS.

You probably tried all these ideas but you never know.

I think you would have to put the files in something like DropBox and give us access to share them.

Hope you sort it out quickly.

Regards Andrew
Robin Leadbeater
Posts: 1926
Joined: Mon Sep 26, 2011 4:41 pm
Contact:

Re: Spectra Problems

Post by Robin Leadbeater »

Hi Keith
Keith Graham wrote: In attempt to narrow down the exact cause, I pulled up some spectral images taken a couple of years ago. Those specs were perfect and were processed with ISIS using my flat image, IR, bias, and dark. The continuum was smooth and there were no bumps. However, when I ran them through again using the same flats, darks, and IR curve used when I originally processed them, the results showed undulating continuum and bumps. Without the flat field the continuum was level and there were no bumps.
This does appear to narrow it down to something in the processing rather than anything in the equipment. Do you still have the log and xml files from the original run that ISIS generates? These should help identify any differences in the program configuration. (For example you could run using the original xml file to exactly duplicate the original settings and see if the problem is still there)

Robin
LHIRES III #29 ATIK314 ALPY 600/200 ATIK428 Star Analyser 100/200 C11 EQ6
http://www.threehillsobservatory.co.uk
Robin Leadbeater
Posts: 1926
Joined: Mon Sep 26, 2011 4:41 pm
Contact:

Re: Spectra Problems

Post by Robin Leadbeater »

a quick test to see if the forum allows zipped fits files.

It does ! :-)
Attachments
_sn2017eaw_20171125_921_Leadbeater.zip
(4.13 KiB) Downloaded 281 times
LHIRES III #29 ATIK314 ALPY 600/200 ATIK428 Star Analyser 100/200 C11 EQ6
http://www.threehillsobservatory.co.uk
Keith Graham
Posts: 126
Joined: Thu Sep 29, 2011 6:20 pm

Re: Spectra Problems

Post by Keith Graham »

Thank you Andrew and Robin for your replies. I agree that I have pretty much narrowed it down to processing. I have attached a zipped file (thanks Robin) containing the 2 images I could not originally upload. Image 1 is the result when the flat is applied. Image 2 is the image when no flat is used. Previous to this problem the flat fielded images looked like image 2.

Andrew, I did try many of your suggestions. I am printing out your list and will go through it one at a time to see if I can locate the problem step by step. I am just puzzled as to what could have caused this strange reversal in my flat vs no flat processing.

BTW-can someone tell me just what, if any, negative effects might affect a spec that has not been flat fielded. I do understand the effects in full frame images (like photometry), but what about images taken through such a narrow slit as in spectroscopy?

Cheers,

Keith
Attachments
_hd217543 1.zip
(21.47 KiB) Downloaded 269 times
Keith Graham
Posts: 126
Joined: Thu Sep 29, 2011 6:20 pm

Re: Spectra Problems

Post by Keith Graham »

I may be getting closer to the source of the issue. Andrew, I went through your suggestions and found one I had not tried – the one to retain all images in the processing routine and put them through their paces. Here are the results.

No Flat Used:
Individual @pro1.fit, @pro2.fit, etc files all good with smooth continuum and no bumps.
Individual (@1.fit, @2,fit, etc files all good with smooth continuum and no bumps.

Flat Used
Individual @pro1.fit, @pro2.fit files all have undulating continuum and bumps.
Individual (@1.fit, @2,fit, etc files all good with smooth continuum and no bumps

So the individual images in Maxim show perfect profiles whether or not a flat is used. The difference is in the individual profile images in ISIS. I am not sure about this, but I am assuming that the individual spectral profiles seen in ISIS are derived from those images seen in Maxim. So if this is true, something must be happening in the process when ISIS derives the spectral profiles from those images. If this is true, I can only assume that there may be something in the settings I am missing, but I am at a loss as to what that might be.

Cheers
Andrew Smith
Posts: 321
Joined: Fri Sep 30, 2011 12:23 pm

Re: Spectra Problems

Post by Andrew Smith »

Glad you are making progress Keith, you probably need an ISIS expert now.

Are you sure you are using the Flat you think you are? (I know but I have made the mistake before.)

I would try another Flat. I might make one in MaxIm. Copy your existing one subtract it from the original adding a large constant in Pixel Maths in MaxIm .

Any shift in your cameras bias level, firmware updates?

Good luck Andrew
Keith Graham
Posts: 126
Joined: Thu Sep 29, 2011 6:20 pm

Re: Spectra Problems

Post by Keith Graham »

Hi Andrew,

Yes, I have actually used different flats including the ones I used for each session all with the same results. It does appear as if I am knocking on the door to rectifying this strange behavior, but that door just refuses to open up - at least for now. Your suggestion helped to bring me one step closer. On the plus side, I originally had thought this might be a camera issue, but delving deeper into possible causes I have saved big money by not jumping too quickly into that.
Robin Leadbeater
Posts: 1926
Joined: Mon Sep 26, 2011 4:41 pm
Contact:

Re: Spectra Problems

Post by Robin Leadbeater »

Hi Keith,

The flat is useful to get rid of any small scale variations due to defects in the image (eg dust etc) and small scale variations in the spectral response (fringes for example). These can also be included in the instrument response and removed that way but it is easier to use a flat to deal with these, leaving just large scale features to be removed by the instrument response. If you are flat correcting and using ISIS for data reduction, the instrument response using the LHIRES at high resolution at Halpha is typically very close to being a straight horizontal line with no significant features. What does your IR look like ?

I wonder if you are getting some sort of interaction between the corrections made by instrument response and the flat correction. (It is important that both ref star and target are processed exactly the same way ie both using the flat, or both without a flat. (Note also that you cannot mix IR measured using ISIS with ones produced by other programs either as ISIS corrects for the nominal black body spectrum of the flat lamp)

Robin
LHIRES III #29 ATIK314 ALPY 600/200 ATIK428 Star Analyser 100/200 C11 EQ6
http://www.threehillsobservatory.co.uk
Keith Graham
Posts: 126
Joined: Thu Sep 29, 2011 6:20 pm

Re: Spectra Problems

Post by Keith Graham »

Hi Robin,

Yes, the IR curve is essentially a straight line. The irregularities occur with or without using the IR curve so apparently they are not caused by any interaction between the IR curve and the flat.

I would like to try your suggestion of running ISIS using the XML file from a previous run. BUT, having never done that before, can you please tell me how to do that or perhaps send me a link that has the instructions for doing this? I have seen those XML files but never knew their purpose.

Keith
Post Reply