I have been struggling recently with processing of data collected in the UV and suspect that my issues stem from my flat frames.
The spectroscope is a UVEX with a 600 lines/mm grating. The camera is an AS533MM with images unbinned. The grating is positioned to replicate the bandpass of the Johnson B filter covering 3635 - 5518 A.
Recently I have been looking at a target that has interesting emission lines at around 3726 A. However, when processed in the normal way, the emissions look much too strong for what I would expect.
My initial thought was that the IR curve I was using was not adequate. The IR curve was developed from an A1V star featured in the Miles database and my reasoning was that the drop off in signal above 3750 A was depressing the IR curve in this region and so artificially increasing the intensity of this part of the spectrum.
So I took a spectrum of LAWD74, a white dwarf with a very smooth spectrum that increases in the UV. As it is featured in the Calspec database (accessed via ISIS), it would provide a reliable IR curve.
Sadly it doesn't. I attach the spectrum compared to the Calspec spectrum. The divergence in the UV is very clear.
Looking at my flat frames, I can see that the signal from the tungsten lamp drops to near zero right where the spectrum departs from the Calspec spectrum. The bulb is the standard tungsten lamp included in the Alpy Calibration unit. I cannot increase the exposure time for my flats as the images will be saturated in the visual range. My conclusion is that this part of the flats is adding noise.
I recall that Demetra has a method to develop a synthetic flat where a constant low level signal is added to a flat at shorter wavelengths that is specified by the user. I can't see how to do this in SpecInti.
Is there a way to deal with this in SpecInti or ISIS? What do other people do? There must be a way to address this given how many UVEX users there are that collect valuable data in the UV.
*********************************************************
J'ai eu récemment des difficultés avec le traitement des données collectées dans l'UV et je soupçonne que mes problèmes proviennent de mes cadres plats.
Le spectroscope est un UVEX avec un réseau de 600 lignes/mm. La caméra est un AS533MM avec des images non binnées. Le réseau est positionné pour reproduire la bande passante du filtre Johnson B couvrant 3635 - 5518 A.
J'ai récemment examiné une cible qui présente des lignes d'émission intéressantes à environ 3726 A. Cependant, lorsqu'elles sont traitées de manière normale, les émissions semblent beaucoup trop fortes par rapport à ce à quoi je m'attendais.
J'ai d'abord pensé que la courbe IR que j'utilisais n'était pas adéquate. La courbe IR a été développée à partir d'une étoile A1V figurant dans la base de données Miles et mon raisonnement était que la chute du signal au-dessus de 3750 A abaissait la courbe IR dans cette région et augmentait donc artificiellement l'intensité de cette partie du spectre.
J'ai donc pris un spectre de LAWD74, une naine blanche avec un spectre très lisse qui augmente dans l'UV. Comme elle est présentée dans la base de données Calspec (accessible via ISIS), elle fournirait une courbe IR fiable.
Malheureusement, ce n'est pas le cas. Je joins le spectre comparé au spectre Calspec. La divergence dans l'UV est très claire.
En regardant mes cadres plats, je peux voir que le signal de la lampe au tungstène tombe à près de zéro là où le spectre s'écarte du spectre Calspec. L'ampoule est la lampe au tungstène standard incluse dans l'unité d'étalonnage Alpy. Je ne peux pas augmenter le temps d'exposition de mes flats car les images seront saturées dans la plage visuelle. Ma conclusion est que cette partie des flats ajoute du bruit.
Je me souviens que Demetra a une méthode pour développer un flat synthétique où un signal de faible niveau constant est ajouté à un flat à des longueurs d'onde plus courtes spécifiées par l'utilisateur. Je ne vois pas comment faire cela dans SpecInti.
Existe-t-il un moyen de gérer cela dans SpecInti o ISIS? Que font les autres ? Il doit y avoir un moyen de résoudre ce problème étant donné le nombre d'utilisateurs d'UVEX qui collectent des données précieuses dans l'UV.
Pete
UV Processing
-
- Posts: 153
- Joined: Tue May 12, 2020 7:36 am
Re: UV Processing
Researching online, I encountered (again!) the excellent. presentation that Christian Buil and Valerie Desnoux gave at ASW2020. It refers to the challenges of obtaining a suitable flat frame in the UV.
It refers to a command tool - I assume in ISIS - to reduce noise in the flat. However, I can't find any other reference to the command FLAT_OPT. Can anyone shed any light on this? And my version of ISIS does not recognise the command
I attach a screenshot of the relevant page.
Pete
It refers to a command tool - I assume in ISIS - to reduce noise in the flat. However, I can't find any other reference to the command FLAT_OPT. Can anyone shed any light on this? And my version of ISIS does not recognise the command
I attach a screenshot of the relevant page.
Pete
-
- Posts: 153
- Joined: Tue May 12, 2020 7:36 am
Re: UV Processing
While I have not received a reply to this post I do see that it has been viewed 88 times so I'll keep updates coming in case this is of interest to anyone else.
In ISIS in the Masters tab there is a tool to combine flat frames to produce a PRNU. The command line FILL_LEFT [image in] [image out] [x pixel] [intensity] then sets the signal level for all pixels to the left of the nominated x pixel to the intensity specified. This should be the average intensity of the PRNU file.
I applied this modified PRNU file as if it were a flat master in SpecInti and achieved a better result as it reduced the noise. What is clear though is that the problem is with the IR curve I have been using. Processing my images without an IR curve but using the PRNU produced a nice spectrum without all the noise in the UV. But of course it doesn't reflect the true signal in UV. Add the IR curve I have a very noisy UV.
Perhaps I don't have enough signal in my IR curve - though I am not sure why that would be the case. The reference star was imaged near the meridian and it was well positioned on the slit. Maybe something else is causing the issue>>
The region being processed is nice and wide in the Y axis so its not that.
My scope is a PW CDK 12.5 operating at its native f8 so it's unlikely to be the cause. This leads me to conclude that the ZWO ASI533MM camera I am using is the culprit as it attenuates the signal in the UV. If so I have only 3 options:
1. Buy a new camera with better QE in the UV - not an attractive proposition
2. Replace the cover glass to increase the throughput in the UV - I have no idea how to do this or what impact it would have on other wavelengths.
3. Reconcile myself to the fact that the UV Explorer spectrograph simply can't generate useful data above 3800 A. Definitely not a happy outcome and one that is hard to accept given that others seems to be obtaining fine results down to 3600 A.
If I am missing something obvious, please let me know.
*******************************************************************
Bien que je n'aie pas reçu de réponse à ce message, je vois qu'il a été consulté 88 fois, je continuerai donc à vous tenir au courant au cas où cela intéresserait quelqu'un d'autre.
Dans ISIS, dans l'onglet Masters, il existe un outil permettant de combiner des images plates pour produire un PRNU. La ligne de commande FILL_LEFT [image in] [image out] [x pixel] [intensity] définit ensuite le niveau de signal pour tous les pixels à gauche du pixel x désigné à l'intensité spécifiée. Cela devrait être l'intensité moyenne du fichier PRNU.
J'ai appliqué ce fichier PRNU modifié comme s'il s'agissait d'un master plat dans SpecInti et j'ai obtenu un meilleur résultat car il a réduit le bruit. Ce qui est clair, c'est que le problème vient de la courbe IR que j'ai utilisée. Le traitement de mes images sans courbe IR mais en utilisant le PRNU a produit un joli spectre sans tout le bruit dans l'UV. Mais bien sûr, il ne reflète pas le vrai signal dans l'UV. Ajoutez la courbe IR et j'ai un UV très bruyant.
Peut-être que je n'ai pas assez de signal dans ma courbe IR - bien que je ne sache pas pourquoi ce serait le cas. L'étoile de référence a été photographiée près du méridien et elle était bien positionnée sur la fente. Peut-être qu'autre chose est à l'origine du problème >>
La région en cours de traitement est belle et large sur l'axe Y, donc ce n'est pas ça.
Mon oscilloscope est un PW CDK 12.5 fonctionnant à son f8 natif, il est donc peu probable que ce soit la cause. Cela m'amène à conclure que la caméra ZWO ASI533MM que j'utilise est la coupable car elle atténue le signal dans l'UV. Si c'est le cas, je n'ai que 3 options :
1. Acheter une nouvelle caméra avec un meilleur QE dans l'UV - ce n'est pas une proposition attrayante
2. Remplacer le verre de protection pour augmenter le débit dans l'UV - je n'ai aucune idée de comment faire cela ou de l'impact que cela aurait sur d'autres longueurs d'onde.
3. Je dois accepter le fait que le spectrographe UV Explorer ne peut tout simplement pas générer de données utiles au-dessus de 3 800 A. Ce n'est certainement pas un résultat heureux et il est difficile à accepter étant donné que d'autres semblent obtenir de bons résultats jusqu'à 3 600 A.
Si quelque chose d'évident m'échappe, n'hésitez pas à me le faire savoir.
Pete
In ISIS in the Masters tab there is a tool to combine flat frames to produce a PRNU. The command line FILL_LEFT [image in] [image out] [x pixel] [intensity] then sets the signal level for all pixels to the left of the nominated x pixel to the intensity specified. This should be the average intensity of the PRNU file.
I applied this modified PRNU file as if it were a flat master in SpecInti and achieved a better result as it reduced the noise. What is clear though is that the problem is with the IR curve I have been using. Processing my images without an IR curve but using the PRNU produced a nice spectrum without all the noise in the UV. But of course it doesn't reflect the true signal in UV. Add the IR curve I have a very noisy UV.
Perhaps I don't have enough signal in my IR curve - though I am not sure why that would be the case. The reference star was imaged near the meridian and it was well positioned on the slit. Maybe something else is causing the issue>>
The region being processed is nice and wide in the Y axis so its not that.
My scope is a PW CDK 12.5 operating at its native f8 so it's unlikely to be the cause. This leads me to conclude that the ZWO ASI533MM camera I am using is the culprit as it attenuates the signal in the UV. If so I have only 3 options:
1. Buy a new camera with better QE in the UV - not an attractive proposition
2. Replace the cover glass to increase the throughput in the UV - I have no idea how to do this or what impact it would have on other wavelengths.
3. Reconcile myself to the fact that the UV Explorer spectrograph simply can't generate useful data above 3800 A. Definitely not a happy outcome and one that is hard to accept given that others seems to be obtaining fine results down to 3600 A.
If I am missing something obvious, please let me know.
*******************************************************************
Bien que je n'aie pas reçu de réponse à ce message, je vois qu'il a été consulté 88 fois, je continuerai donc à vous tenir au courant au cas où cela intéresserait quelqu'un d'autre.
Dans ISIS, dans l'onglet Masters, il existe un outil permettant de combiner des images plates pour produire un PRNU. La ligne de commande FILL_LEFT [image in] [image out] [x pixel] [intensity] définit ensuite le niveau de signal pour tous les pixels à gauche du pixel x désigné à l'intensité spécifiée. Cela devrait être l'intensité moyenne du fichier PRNU.
J'ai appliqué ce fichier PRNU modifié comme s'il s'agissait d'un master plat dans SpecInti et j'ai obtenu un meilleur résultat car il a réduit le bruit. Ce qui est clair, c'est que le problème vient de la courbe IR que j'ai utilisée. Le traitement de mes images sans courbe IR mais en utilisant le PRNU a produit un joli spectre sans tout le bruit dans l'UV. Mais bien sûr, il ne reflète pas le vrai signal dans l'UV. Ajoutez la courbe IR et j'ai un UV très bruyant.
Peut-être que je n'ai pas assez de signal dans ma courbe IR - bien que je ne sache pas pourquoi ce serait le cas. L'étoile de référence a été photographiée près du méridien et elle était bien positionnée sur la fente. Peut-être qu'autre chose est à l'origine du problème >>
La région en cours de traitement est belle et large sur l'axe Y, donc ce n'est pas ça.
Mon oscilloscope est un PW CDK 12.5 fonctionnant à son f8 natif, il est donc peu probable que ce soit la cause. Cela m'amène à conclure que la caméra ZWO ASI533MM que j'utilise est la coupable car elle atténue le signal dans l'UV. Si c'est le cas, je n'ai que 3 options :
1. Acheter une nouvelle caméra avec un meilleur QE dans l'UV - ce n'est pas une proposition attrayante
2. Remplacer le verre de protection pour augmenter le débit dans l'UV - je n'ai aucune idée de comment faire cela ou de l'impact que cela aurait sur d'autres longueurs d'onde.
3. Je dois accepter le fait que le spectrographe UV Explorer ne peut tout simplement pas générer de données utiles au-dessus de 3 800 A. Ce n'est certainement pas un résultat heureux et il est difficile à accepter étant donné que d'autres semblent obtenir de bons résultats jusqu'à 3 600 A.
Si quelque chose d'évident m'échappe, n'hésitez pas à me le faire savoir.
Pete
-
- Posts: 1253
- Joined: Thu Sep 29, 2011 6:35 am
- Location: Rhône Alpes FRANCE
- Contact:
Re: UV Processing
Hi Peter,
I don't have much experience with UVEX, which I've only used a few times on my RC12, especially in the UV section,
A few things to check:
- With regard to the telescope, it seems to me that a CDK has lenses and that this can therefore affect transmission in the near-UV range below 400nm (to be verified?).
- For flats, it's clear that this is a problem, partly because of the cameras we use (not very sensitive below 400nm) and partly because the temperature of the light source in the 2900K range doesn't help to get a good flat in the blue part of the spectrum. What I do is try to make flats with long exposure times so as to get around 50,000 ADU on a flat and make a lot of them (say thirty or more).
Here's what I came up with with a 600 gr/mm grating and my UVEX on a bright star.
You can see the Hugins bands around 340nm and the Balmer jump, but you need a long exposure and many exposures to get a good SNR. But it's a bright star and I don't know what it's possible for fainter targets...
I don't have much experience with UVEX, which I've only used a few times on my RC12, especially in the UV section,
A few things to check:
- With regard to the telescope, it seems to me that a CDK has lenses and that this can therefore affect transmission in the near-UV range below 400nm (to be verified?).
- For flats, it's clear that this is a problem, partly because of the cameras we use (not very sensitive below 400nm) and partly because the temperature of the light source in the 2900K range doesn't help to get a good flat in the blue part of the spectrum. What I do is try to make flats with long exposure times so as to get around 50,000 ADU on a flat and make a lot of them (say thirty or more).
Here's what I came up with with a 600 gr/mm grating and my UVEX on a bright star.
You can see the Hugins bands around 340nm and the Balmer jump, but you need a long exposure and many exposures to get a good SNR. But it's a bright star and I don't know what it's possible for fainter targets...
LHIRES III #5, LISA, e-Shel, C14, RC400 Astrosib, AP1600
http://o.garde.free.fr/astro/Spectro1/Bienvenue.html
http://o.garde.free.fr/astro/Spectro1/Bienvenue.html
-
- Posts: 153
- Joined: Tue May 12, 2020 7:36 am
Re: UV Processing
Thanks Olivier
Responding to your comments:
1. CDK design - yes, the scope includes a lens set which I expect impacts on sensitivity in the UV. I don’t expect it’s the principal issue but it’s worth keeping in mind. There’s not much I can do about it though.
2. Flats - I agree that the flats are part of the problem. I did some tests this morning, comparing results using my standard flats and a PRNU made up of flats that has had the low signal area filled with the average intensity of the rest of the PRNU. The standard flats introduced quite a bit of noise in the UV. In contrast, the PRNU provided a clean signal in the UV (hooray!) but produced a worse result at the visual end. This surprised me. I don’t think it’s simply noise - rather the bumps that are minor using a normal flat are of greater intensity with the adjusted PRNU. I’ll need to do some more work on this if I am to keep using the PRNU method but is promising.
Like you, I collect plenty of flats - I take 20 immediately before collecting my ArNe calibration and spectral images. The region I am working on is roughly 3500 - 5500 A. The exposure time is limited by saturation in the visual range. So a 10s flat exposure will generate a flat with around 55 000 ADU at one end of the spectrum but around 3 000 ADU in the UV. I experimented with breaking up my spectrum into 2 parts and processing them separately with different flats - long ones for UV and short ones for the visual end. This didn’t really improve things much.
That said, I do think you are right about the need for long exposures in the UV. To get a good, smooth continuum for my IR curve, I was imaging LAWD74, a white dwarf. It generates a nice curve but it’s faint at around 12.4 magnitude in B. I took 3 x 1200s exposures thinking that this would be enough to generate a decent IR curve. It may well be. But because the target is faint, when I apply the IR curve to the spectrum, the spectrum is very noisy in the UV above 3800 A. In other words - garbage in, garbage out.
So I’ll look for a nice and bright O or early B star, collect some data and run some more tests with flats vs PRNU files and see how I go.
Thanks again
Pete
Responding to your comments:
1. CDK design - yes, the scope includes a lens set which I expect impacts on sensitivity in the UV. I don’t expect it’s the principal issue but it’s worth keeping in mind. There’s not much I can do about it though.
2. Flats - I agree that the flats are part of the problem. I did some tests this morning, comparing results using my standard flats and a PRNU made up of flats that has had the low signal area filled with the average intensity of the rest of the PRNU. The standard flats introduced quite a bit of noise in the UV. In contrast, the PRNU provided a clean signal in the UV (hooray!) but produced a worse result at the visual end. This surprised me. I don’t think it’s simply noise - rather the bumps that are minor using a normal flat are of greater intensity with the adjusted PRNU. I’ll need to do some more work on this if I am to keep using the PRNU method but is promising.
Like you, I collect plenty of flats - I take 20 immediately before collecting my ArNe calibration and spectral images. The region I am working on is roughly 3500 - 5500 A. The exposure time is limited by saturation in the visual range. So a 10s flat exposure will generate a flat with around 55 000 ADU at one end of the spectrum but around 3 000 ADU in the UV. I experimented with breaking up my spectrum into 2 parts and processing them separately with different flats - long ones for UV and short ones for the visual end. This didn’t really improve things much.
That said, I do think you are right about the need for long exposures in the UV. To get a good, smooth continuum for my IR curve, I was imaging LAWD74, a white dwarf. It generates a nice curve but it’s faint at around 12.4 magnitude in B. I took 3 x 1200s exposures thinking that this would be enough to generate a decent IR curve. It may well be. But because the target is faint, when I apply the IR curve to the spectrum, the spectrum is very noisy in the UV above 3800 A. In other words - garbage in, garbage out.
So I’ll look for a nice and bright O or early B star, collect some data and run some more tests with flats vs PRNU files and see how I go.
Thanks again
Pete