Release 1.0.3 | 22 February 2007 | Associated with Catalogue version 1.0 |
Prepared by the XMM-Newton Survey Science Centre Consortium (http://xmmssc-www.star.le.ac.uk/) |
This User Guide refers directly to the full FITS and plain-text formats of the pre-release catalogue. It provides a detailed account of the production and contents of the catalogue. Users interested in the main properties of the catalogue will find the summary and sections 1 & 4.3 of most immediate interest.
2XMMp is the pre-release second catalogue of serendipitous X-ray sources from the European Space Agency's (ESA) XMM-Newton observatory, and has been constructed by the XMM-Newton Survey Science Centre (SSC) on behalf of ESA. This 2XMMp catalogue is essentially a subset of the full 2XMM catalogue due to be released later this year. To make the 2XMMp catalogue we have carried out an initial visual screening of each observation and excluded from the catalogue those observations which exhibit deficiencies in the automatic processing which will require full visual screening to flag source detections affected by these deficiencies (~17% of observations). In this way the overall quality of the 2XMMp catalogue is maintained at the expense of excluding these problematic observations. These observations will, of course, be included in the full 2XMM catalogue once the detailed (and time-consuming) visual screening is completed.
The pre-release catalogue contains source detections drawn from 2400 XMM-Newton EPIC observations made between 2000 February 4 and 2006 April 20; all datasets included were publicly available by 2006 May 31 but note that not all public observations are included in this catalogue. The total area of the catalogue fields is ~ 400 deg2, but taking account of the substantial overlaps between observations, the net sky area covered independently is ~ 285 deg2.
The processing to generate the catalogue is largely based on the pipeline developed for the re-processing of all observations to date and includes a number of significant improvements over the previous data processing system (as used by the SSC in routine processing of XMM-Newton data on behalf of ESA). These improvements include a more sensitive source detection scheme using exposures of all cameras, the detection and parameterization of extended sources and the extraction of spectra and time series for the brightest sources. The catalogue contains 153105 X-ray source detections above the processing likelihood threshold of 6. The 153105 X-ray source detections relate to 123170 unique X-ray sources, that is, a significant fraction of sources, namely 46796, have more than one detection in the catalogue.
The median flux (in the total photon-energy band 0.2 - 12 keV) of the catalogue detections is ~ 2.4*10-14 erg/cm2/s; ~ 20% have fluxes below 1*10-14 erg/cm2/s. The positional accuracy of the catalogue detections is generally < 2 arcsec (68% confidence radius). The flux values from the three EPIC cameras are overall in agreement to ~ 2% for on-axis sources, and ~ 6% off-axis.
Known problems and open issues are discussed in Sec. 4.2
Pointed observations with the XMM-Newton Observatory detect significant numbers of previously unknown 'serendipitous' X-ray sources in addition to the original target. Combining the data from many observations thus yields a serendipitous source catalogue which, by virtue of the large field of view of XMM-Newton and its high sensitivity, represents a significant resource. The serendipitous source catalogue enhances our knowledge of the X-ray sky and has the potential for advancing our understanding of the nature of various Galactic and extragalactic source populations.
While the first XMM-Newton catalogue contains X-ray source detections from 585 XMM-Newton observations made between launch in 2000 March 1 and 2002 May 5, the present 2XMMp catalogue contains sources from over 4 times as many observations (2400) made between 2000 February 4 and 2006 April 20 which are in the public domain since 2006 May 31. The production of this catalogue has been undertaken by the XMM-Newton Survey Science Centre (SSC) consortium in fulfillment of one of its major responsibilities within the XMM-Newton project. The catalogue production process has been designed to exploit fully the capabilities of the XMM-Newton EPIC cameras and to ensure the integrity and quality of the resultant catalogue through rigorous screening of the data.
The selection of XMM-Newton observations for processing in the 2XMM catalogue pipeline is based on the desire to re-process all available obervations with the latest data processing system and calibration data available. All observation that are in the public domain prior to the release of the 2XMM catalogue will be included. For the purpose of the 2XMM pre-release catalogue a sub-selection of these (made available to the public by 2006 May 31) is used, which have been selected through visual screening of the sources detected in the field. Only 'good' observations, that is, those that have no or very few obvious spurious detections (usually found in image artefacts), have been included (cf. Sec. 3.3.5 ). They comprise about ~ 83% of all public observation that have EPIC data. In contrast, the full 2XMM catalogue (to be released later in the year) will contain all observations, and spurious detections will be manually flagged.
Table 2.1 gives the list of the final 2400 observations which are included in the catalogue.
Most, but not all, XMM-Newton observations comprise a single exposure with each of the three EPIC cameras. (A significant number of observations have multiple exposures and/or do not include exposures with one or more of the three cameras.) For each observation we selected exposures for each of the three EPIC cameras for processing using the following criteria:
(i) | An exposure must have > 1000 seconds duration. |
(ii) | The exposure must have been taken through a scientifically useful filter. In practice this rejected all exposures for which the filter position was closed, calibration or undefined. The possible filters used in the observations selected for the catalogue are Medium , Thick, Thin1, Thin2 (PN only), and Open. For a detailed description of the filters see in the XMM user hand book 3.3.6 EPIC filters and effective area. |
(iii) | The exposure must have been taken in a mode which could usefully be processed by the detection stage. PN small window modes were rejected since the effective field of view in these modes is small making the background fitting stage of the source detection problematic. For the MOS nearly all modes, including those modes in which the area of the central CCD was windowed or missing (e.g., timing modes, here 'Fast Uncompressed') or modified ('Refreshed Frame Store' mode), were included. The possible observing modes used in the observations selected for the catalogue are given in Tab. 2.2. For a detailed description of the modes see in the XMM user hand book 3.3.2 Science modes of the EPIC cameras. |
(iv) | Background filtering (see Sec. 3.1.1 c)) must have been successfully applied. This was not the case when the sum of all Good Time Intervals (hereafter GTIs) was less than 1000 seconds. Without background filtering the source detection is typically of limited value due to the much higher net background. |
(v) | After background filtering has been applied, each of the five images of an exposure (bands 1 - 5) must have sufficient range (as tested with fimgstat), i.e., at least one pixel per image must have more than one event detected. |
Where more than one exposure with a particular camera met the above selection conditions, all exposures with the same filter and data mode were merged and then the exposure with maximum duration was chosen for the source detection. The zoom-in flow chart below visualizes the selection procedure. The source-specific products, in contrast, were extracted for all (unmerged) exposures that met the above first four criteria (see Sec. 3.1.4).
![]() |
The processing of the observations was facilitated through an improved pipeline configuration over the one currently used for the routine production processing of observations. This new processing pipeline is used to re-process all available observations up-to-date and will become the new routine pipeline after the re-processing has been concluded.
After creation of the pipeline data products (Sec. 3.1), some of the output was modified based on post-processing analysis (Sec. 3.2). The catalogue file was constructed to contain key columns from the source lists plus additional columns which include observation-level meta-data for each source as well as further analyses and evaluation (Sec. 3.3). Additional products for each source were created to facilitate data access (Sec. 3.4).
Throughout the documentation references to catalogue columns are marked as links to their description. The prefix 'ca_' in a column name indicates a wildcard for any of the three EPIC cameras, i.e. 'ca' is to be replaced by 'PN,' 'M1' or 'M2'; it can also stand for 'EP' where applicable. Some of the column names also include an energy band identifier ('_b_', where b = 1,2,3,4,5,6,7,8,9) which is typically not explicitly indicated. The four hardness ratios are identified by the designator 'n' (n = 1,2,3,4).
The zoom-in flow chart below gives an overview of the processing steps from the XMM-Newton observation to the final source list as described in this section. The 'exposure group selection' is described in Sec. 2.2.
![]() |
The following sections describe the individual steps taken within the processing chain leading to images on which source detection could be performed.
1. A first pass constructs a flare lightcurve selecting events with the (XMMEA_22 = REJECT_BY_GATTI & XMMEA_EM = GOOD_MOS_EVENTS) flags set
2. GTIs are made by filtering the flare lightcurve using the MOS flare threshold. These GTIs are used to define the time regions in which bad pixel searching occurs.
3. All GTIs with a duration of less than 100 seconds are excluded.
4. The SAS task embadpixfind is used to locate dark pixels in each MOS CCD (using events which have not been filtered through the flare GTIs).
5. If no flare GTIs were made the SAS task embadpixfind is used to locate bright pixels
6. If flare GTIs do exist the events are filtered through the flare GTIs and then embadpixfind is used to locate bright pixels.
7. The SAS task badpix is run on each CCD event file in order to add a bad pixel extension.
8. The intervals in the global GTI file are aligned with the event list and merged with the CCD GTIs.
9. Attitude correction is applied to the individual events to convert raw CCD pixel coordinates, through camera coordinates, to celestial coordinates.
10. Raw event pulse height values are converted to rectified event energies.
11. Unwanted events are filtered out before lists are merged.
12. The per-CCD event lists are merged into one per camera.
13. Filter the good imaging events into final event lists.
14. Copy the CIF into a separate extension in the event list.
15. Make a second pass flare lightcurve selecting events with the (XMMEA_22 = REJECT_BY_GATTI & XMMEA_EM = GOOD_MOS_EVENTS) flags set and bad pixels excluded.
16. Create flare GTIs using the MOS flare threshold.
17. Filter the event files through GTIs into final event files.
1. The SAS task badpixfind is run to create a mask of non-source pixels to be used in generating a background flare lightcurve
2. Badpixfind is run on each CCD to locate bright and dead pixels
3. Attitude correction is applied to the individual events to convert raw CCD pixel coordinates, through camera coordinates, to celestial coordinates.
4. Raw event pulse height values are converted to rectified event energies.
5. Filter events by selecting events with the (XMMEA_EP = PN_GOOD_EVENTS) flags set
6. Filter the CCD event files on the HK GTIs and merge into one
7. Copy the CIF into a separate extension in the event list.
8. Make a background flare lightcurve using events with energies between 7 keV & 15 keV events and excluding bad pixels by using the previously created pixel mask.
9. Create flare GTIs using the PN flare threshold for use in later processing stages.
The MOS flare lightcurves were produced from GATTI-flagged (essentially events with energies above 14 keV), single-pixel events from the outer CCDs. After binning the lightcurve, the flare GTIs were selected by imposing a rate threshold of 2 cnts/arcmin2/ksec.
The PN flare lightcurves were produced in the 7.0 - 15 keV energy range. After binning the lightcurve, the flare GTIs were selected by imposing a rate threshold of 10 cnts/arcmin2/ksec.
1. Exclude from the flare GTIs all intervals with duration less than 100 seconds.
2. For each energy band make a counts image. The images are 600 x 600 pixels with 4-arcsecond pixel sides. The images are tangent plane projections of celestial coordinates. Note that the energy bands have changed slightly with respect to previous processing: the old band 2 is split into two bands now (0.5 keV - 1.0 keV and 1.0 keV - 2.0 keV), while the old bands 4 and 5 have been merged into a single band. The definitions of all the new energy bands are given in Tab. 3.1 below.
Basic energy bands: | 1 | = | 0.2 - 0.5 keV | ||
2 | = | 0.5 - 1.0 keV | (formerly part of band 2) | ||
3 | = | 1.0 - 2.0 keV | (formerly part of band 2) | ||
4 | = | 2.0 - 4.5 keV | (formerly band 3) | ||
5 | = | 4.5 - 12.0 keV | (formerly bands 4 and 5) | ||
Broad energy bands: | 6 | = | 0.2 - 2.0 keV | soft band, no images made | |
7 | = | 2.0 - 12.0 keV | hard band, no images made | ||
8 | = | 0.2 - 12.0 keV | total band | ||
9 | = | 0.5 - 4.5 keV | XID band |
3. Event selection for PN images is PATTERN <= 4 and RAWY > 12 with events ON_OFFSET_COLUMN and excluded. Band 1 images have the additional stricter requirement PATTERN = 0, while band 8 images have PATTERN = 0 below 0.5 keV. Band 1 - 5 images have also events OUT_OF_FOV excluded.
4. For MOS band 1 - 5 and 8 images no PATTERN selection is made beyond the 0 - 25 selection made in the event lists. Events OUT_OF_FOV are excluded for band 1 - 5 images.
5. Make exposure images corresponding to bands 1 - 5 count images
Source detection is performed simultaneously on images in the energy bands 1 - 5 and from the three EPIC cameras. For observations having multiple exposures from the same camera, exposures are merged by filter and observing mode, and for each camera the merged ('added') exposure with the longest integration time is used for source detection (cf. Sec. 2.2).
In the fitting routine source parameters are determined for the three cameras [PN,M1,M2] in the energy bands 1 - 5 as well as the XID band (cf. Tab. 3.1). These parameters are then combined to obtain camera dependent parameters in band 8 as well as all-EPIC parameters.
Exposure maps hold the effective exposure time for each
detector point (see Sec. 3.1.1).
They are created by the SAS task eexpmap for each
EPIC camera and energy bands 1 - 5 using the latest
calibration information on the mirror vignetting, quantum efficiency
and filter transmission. The exposure maps (see ca_EXP
)
are corrected for bad pixels, bad columns and CCD gaps as well as being
multiplied by an out-of-time factor (oot_factor):
oot_factor = | 0.9411 | for PN PrimeFullFrame modes, |
0.97815 | for PN ExtendedFullFrame modes, | |
1.0 | for all other PN and M1/M2 modes. |
The SAS task emask is used to create a detector mask for each camera. Detector masks define the area of the detectors which is suitable for source detection. Only the areas of the detector where the exposure is at least 50% of the maximum exposure have been used for source detection.
The SAS task eboxdetect is used to create a preliminary source list. Eboxdetect has two operation modes: local and map detection. At this stage eboxdetect is run in local mode and performs a sliding box cell detection (box size 5 x 5 pixels) on the detector areas defined by the detection masks. Eboxdetect in local mode uses a local background that is determined in a frame around the search box. All sources with a 0.2 - 12 keV EPIC detection likelihood above 5 are included in the output source list.
The SAS task esplinemap is used to
create background maps for each camera and energy bands
1 - 5. Using a cut-out radius dependent on source
brightness, esplinemap blanks out the areas of
the images where sources where detected by eboxdetect
in local mode. Then esplinemap performs
12 x 12 nodes spline fits on the resulting
source-free images to calculate a smoothed background map for the
entire images (ca_BG
).
A second pass of eboxdetect is carried out in map mode. It creates a new source list using this time the background maps generated by esplinemap, increasing thereby the source detection sensitivity as compared to the local detection step. The box size is again set to 5 x 5 pixels. All sources with a 0.2 - 12 keV EPIC detection likelihood above 5 are included in the eboxdetect map mode source list.
The sources detected by eboxdetect in
map mode are passed on to the SAS task emldetect.
Emldetect does not perform source detection,
instead it calculates PN/M1/M2 source parameters in the bands
1 - 5 by fitting the instrumental point spread
function (PSF) convolved with a source extent model to the distribution
of counts of the sources detected by eboxdetect
(in map mode) simultaneously in the bands 1 - 5 and
the three cameras. The extent model used is a beta-model profile. Free
parameters of the fits are source positions, extent (ep_EXTENT
),
and count rates (ca_RATE
).
Positions and extent are constrained to be the same in all energy bands
and for all cameras while count rates are obtained from the best fit
value for each camera and energy band. Detection likelihoods (ca_DET_ML
)
and extent likelihoods (ca_EXTENT_ML
)
are derived as well.
In a second loop emldetect attempts to fit two PSFs to sources detected as extended, and for those detections where the split was successful the (point) source parameters were re-calculated.
Emldetect uses the multiband exposure maps to correct the count rates for vignetting and losses due to inter-chip gaps and bad pixels/columns as well as for losses in the PN due to events arriving during readout times (out of time events):
count_rate = source_counts / exp_map .
Emldetect derives four camera dependent X-ray colours known as hardness ratios (HR), which are obtained combining corrected count rates from different energy bands. Each hardness ratio, HRn, is obtained as
HRn = (RATE_b - RATE_a) / (RATE_b + RATE_a)
where RATE_a and RATE_b
are the corrected count rates in energy bands a
and b (see ca_b_RATE
).
Energy bands 1 & 2 are used to
obtain ca_HR1
,
2 & 3 for ca_HR2
,
3 & 4 for ca_HR3
,
and 4 & 5 for ca_HR4
.
Count rates and therefore hardness ratios are camera dependent. In addition they depend on the blocking filter used for the observation, specially the HR1. This needs to be taken into account when comparing hardness ratios for different sources. Note, that a large fraction of the hardness ratios are calculated from marginal or non-detections in at least one of the energy bands. Consequently, individual hardness ratios should only be deemed reliable if the source was above the detection likelihood threshold in both energy bands, else they have to be treated as upper limits.
Emldetect calculates source fluxes (ca_FLUX
)
in bands 1 - 5 in units of [erg/s/cm2],
using the count rates (ca_RATE
)
in those bands using the following expression:
Flux = Rate / ECF ,
where the ECF is an energy conversion factor (to 'convert' count rates to fluxes). ECFs have been calculated using the most recent calibration matrices. They have been calculated assuming an spectral model of an absorbed power-law with absorbing column density Nh = 3.0 * 1020 cm2 and continuum spectral slope Gamma 1.7 (see CAL-TN-0023-v2.0.ps).
ECFs for each camera, energy band and observation blocking filter in units of [1011 cts cm2/erg] are given in Tab. 3.2. EPIC-PN Thin1 and Thin2 filters have been assumed to have the same transmission.
Camera | Band | Open | Thin | Medium | Thick |
PN | 1 | 16.1784 | 8.95403 | 7.82028 | 4.71096 |
2 | 10.0418 | 8.09027 | 7.83782 | 6.02015 | |
3 | 6.17030 | 5.88255 | 5.78272 | 5.00419 | |
4 | 1.95859 | 1.92805 | 1.90529 | 1.80647 | |
5 | 0.555924 | 0.555226 | 0.554529 | 0.547205 | |
9 | 5.07412 | 4.53836 | 4.43953 | 3.74772 | |
MOS-1 | 1 | 3.15223 | 1.80399 | 1.60150 | 1.06500 |
2 | 2.27921 | 1.88017 | 1.82853 | 1.48465 | |
3 | 2.14933 | 2.05034 | 2.01594 | 1.79446 | |
4 | 0.757786 | 0.746128 | 0.737800 | 0.707822 | |
5 | 0.143619 | 0.143340 | 0.143131 | 0.141213 | |
9 | 1.54600 | 1.42040 | 1.39361 | 1.23264 | |
MOS-2 | 1 | 3.17622 | 1.81179 | 1.60670 | 1.06620 |
2 | 2.28390 | 1.88369 | 1.83088 | 1.48818 | |
3 | 2.15017 | 2.05117 | 2.01594 | 1.79530 | |
4 | 0.761672 | 0.750569 | 0.741687 | 0.711708 | |
5 | 0.151083 | 0.150769 | 0.150560 | 0.148537 | |
9 | 1.54912 | 1.42326 | 1.39647 | 1.23524 |
Note that all count rates (ca_RATE
)
and fluxes (ca_FLUX
)
correspond to the flux in the entire PSF and do not need any further
corrections for PSF losses.
Band 8 source parameters are derived from the
combination of parameters from bands 1 - 5. For
details on how each parameter was obtained see the column descriptions
for the source
parameters
.
Emldetect detection likelihood values (ca_DET_ML
)
are based on the likelihood ratio described by Cash (1979).
To allow comparisons of source detection runs with different source
parameters, the detection likelihoods in emldetect
are given in the form of 'equivalent' detection likelihoods, i.e., they
are corrected for the number of free fit parameters. All sources (as
detected by eboxdetect map mode) with
0.2 - 12 keV EPIC detection likelihoods
greater than 6 as determined by emldetect
are included in the output source list.
Band 9 source parameters are for the XID band (0.5 - 4.5 keV). Instead of combining parameters from bands 2, 3 and 4, which will produce overall larger source parameter errors, the SAS task emldetect is run a second time using merged images, exposure maps and background maps from bands 2 - 4. Source positions are kept fixed and a likelihood threshold of zero is used to ensure that band 9 parameters are obtained for all sources detected in the first run of emldetect. The output source list contains only band 9 parameters with errors determined directly from the merged images.
One of the improvements over the previous processing pipelines
is the setting of automatic flags by the SAS task dpssflag;
based on the available information in the emldetect
source list it writes a string of nine different flags back into the
source list (ca_FLAG
)
to indicate various conditions. Because the decision tree had to be
simple these flags should be understood mainly as a warning. In
particular, sources with a low coverage on the detector, sources in
problematic areas (near a bright source or within an extended source)
as well as sources near artefacts like the known bright MOS-1 corner or
the occasionally bright low gain columns of the PN are flagged. In
addition, an attempt was made to identify spurious extended sources
which can often be found near bright sources, within complicated
extended emission, or generally in areas where the background changes
considerably on a small spatial scale and the spline maps can not adapt
well enough. The nine flag positions have been assigned the meanings
given in Tab. 3.3
(note that flags 1, 8, and 9 are camera dependent):
1 | Low detector coverage | ca_MASKFRAC < 0.5 |
2 | Near other source | R ≤ 65 * SQRT (EP_RATE );
R(min) = 10", R(max) = 400" |
3 | Within extended emission | R ≤ 3 * EP_EXTENT ;
R(max) = 200" |
4 | Possible spurious extended source near bright source | Flag 2 is set and EP_CTS (min)
= 1000 for the causing source |
5 | Possible spurious extended source within extended emission | R ≤ 160" and fraction of rate wrt causing source is 0.4 |
6 | Possible spurious extended source due to unusal large single-band DET_ML | Fraction of ca_b_DET_ML
wrt the sum of all ≥ 0.9 |
7 | Possible spurious extended source | At least one of the flags 4, 5, 6 is set |
8 | On bright MOS-1 corner or bright low gain PN column | |
9 | Near bright MOS-1 corner | R ≤ CUTRAD = 60" of a bright pixel the corner |
The default value of every flag is F for False. When a flag was set it means it has been changed to T for True.
Dpssflag sets all flags except the camera-specific flags (i.e., flags 2,3,4,5,6,7) on the summary row (EPIC band 8) which are then propageted backwards to the individual cameras and bands (but see the caveat described in Sec. 4.2).
The emldetect source list for the bands 1 - 5 is
merged with the emldetect XID source list into a common list by
the SAS task srcmatch. The output table consists of a single row
per detection with parameters from both input lists in different
columns. Srcmatch also calculates band 1 - 5 EPIC fluxes
(EP_b_FLUX
), EPIC
hardness ratios (EP_HRn
),
and their respective errors.
Srcmatch introduces flag columns which
are later populated by the pipeline, e.g., for sources where
source-specific products (Sec. 3.1.4)
have been made (TSERIES
and SPECTRA
).
References:
Cash, W., 1979,
Parameter estimation in Astronomy through application of the likelihood
ratio,
ApJ, 228, p. 939
The SAS task eposcorr correlates the X-ray positions from an observation (as determined by the fitting routine of emldetect) with catalogued optical positions and minimizes the positional offsets by applying a translation and rotation to the X-ray positions. For the catalogue pipeline the srcmatch source lists were correlated with the USNO B1.0 optical catalogue. The correlation allows offsets in RA/DEC of up to 10 arcseconds while all optical sources more than 15 arcseconds from an X-ray source are removed prior to correlation.
The SAS task evalcorr evaluates the quality of the position rectification of eposcorr. For 2XMMp the following empirically determined condition was used to accept the refined astrometric solution:
POSCOROK is set to True by evalcorr if
LIK_HOOD > 5.0 + ( 2.0 * LIK_NULL ) ,
where LIK_HOOD and LIK_NULL are determined by the SAS task eposcorr. LIK_NULL is the likelihood calculated for purely coincidental X-ray/optical matches in a given observation, i.e., if there were no true counterparts.
If POSCOROK is set to True the columns RA
and DEC
give the corrected X-ray positions
calculated by eposcorr. If the refined astrometric solution was
not accepted the columns RA
and DEC
are the same as the
uncorrected values RA_UNC
and DEC_UNC
.
The intrinsic systematic 1-sigma error for XMM-Newton fields
(i.e., before any correction is applied) was estimated from the width
of the distributions of position shifts found in eposcorr
runs. A value of SYSERRCC
=
1.5 arcseconds is assigned to detections in all fields for
which no acceptable astrometric correction using eposcorr
was determined (that is, POSCOROK is False).
The residual systematic 1-sigma error for XMM-Newton fields
where an acceptable astrometric correction was determined using eposcorr
is SYSERRCC
=
0.5 arcseconds (that is, POSCOROK is True).
This value was estimated from a comparison of the width of the
distribution of accepted XMM position shifts with expectations based on
the statistical errors alone.
The new pipeline automatically extracts time series and spectra for the
brighter detections (EPIC counts ≥ 500; where the detection was
only observed with one or two cameras the equivalent EPIC counts were
calculated using the PN to MOS count ratio 3.5 : 1). All
exposures that passed the filtering (i) - (iv) in Sec. 2.2 were used for extraction. To extract
a detection for a particular camera two conditions had to be met: (i)
ca_MASKFRAC
≥ 0.5, and (ii) ca_DET_ML
≥ 15.
The source counts were extracted from a circular aperture with radius 28", and the background counts were extracted from an annulus around the detection position with r(min) = 60" and r(max) = 180". PATTERN selection is the same as for image creation (PN: PATTERN <= 4, MOS: PATTERN <= 12). FLAG selection was done according to the recommandations: FLAG = 0 for the PN, XMMEA_EM for MOS time series, and XMMEA_SM for MOS spectra. The energy range for the extraction of all products is 0.2 - 0.12 keV.
While time series are filtered only by instrumental GTIs (see Sec. 3.1.1 a) and b)), spectra are also filtered for the flare background (see Sec. 3.1.1 c)). The variability tests, however, exclude times where the flare background is high.
The bin size for the time series was chosen is such a way that the PN bins contain at least 18 counts and the MOS bins at least 5 counts (as derived from the source lists; note that these are background subtracted according to the background maps determined in the source detection process, see Sec. 3.1.2 d). The minimum bin size is 10 seconds, and all other bin sizes are rounded up to an integral multiple of 10.
To test for variability we have used a Chi2-test (suitable
for binned data) with the Pearson's approximation for Poissonian
data. Times with high background flaring were excluded from the test. The
SAS task ekstest writes four keywords into the header of the time
series file, namely CHI2PROB
for the probability, CHISQUAR for the Chi2-statistic, N_POINTS
for the number of bins used in the test, and AVRATE for the mean rate in
the number of bins used for the test.
The available spectral products for each selected detection in 2XMMp are (i) a grouped source spectrum (20 counts/bin; energies below 0.35 keV as well as energies in the PN around the copper line at 8.05 keV are set to 'bad'), (ii) a background spectrum, (iii) a source ARF (auxillary response file), and (iv) a spectral plot made with XSPEC. A keyword in the header of the source spectrum file indicates the name of the canned RMF (response matrix file) that can be used for this detection.
The available time series products for each selected detection in 2XMMp are (i) a time series file containing the source minus background and background arrays (corrected for exposure, cosmic rays, and dead time) as well as the keywords regarding the variability, and (ii) a plot of the time series and the background made by the SAS task elcplot.
The available products are identified by their observation ID (OBSID
), exposure ID and the
observation-specific source number SRC_NUM
in the hexadecimal
system. Further details and a discussion of the limitations of an automatic
extraction can be found in SSC-LUX-RE-0155.ps.
The ca_ONTIME
values in the source lists are copied from the chip-dependent ONTIME
keywords in the header of the images by the SAS task emldetect.
For 2XMMp these keywords were not up-dated when exposures were merged for
the source detection, and as a consequence the ONTIME value in the
individal source lists refers to the first exposure in the merged set (only
in ~ 2% of all observations there are one or more merged
exposures). This has been rectified in the catalogue, that is, ca_ONTIME
refers now to the
merged (and GTI filtered) exposure time of the chip where the
detection is located. Note, though, that the ONTIME values are undefined
when the position of a detection fell into a chip gap or outside the
detector.
Due to a problem in the SAS task srcmatch
there are errors of hardness ratios given in the individual source
lists while the hardness ratio itself is NULL. This affected only the
columns EP_HR3_ERR
and EP_HR4_ERR
.
These have been set to NULL in the catalogue.
In some cases, the individual source lists give band-dependent
likelihoods ca_b_DET_ML
that are defined for one band but undefined (i.e., NULL) for another
band for a given detection and camera. This is not correct since either
the detection was observed (all values are defined) or not observed
(all values are undefined). These 'wrong' NULL values have been set to
zero in the catalogue if at least one other band likelihood was given
as defined and > 0.
Most of the catalogue columns are derived from information in the output lists by the srcmatch task (see Sec. 3.1.2 i)), but it was also necessary to extract information from the emldetect source lists (see Sec. 3.1.2 f)). Some modifications to a few columns were necessary, see Sec. 3.2. Additional columns derived from other products and from further processing are explained below.
The catalogue includes meta data derived from extisting products to help
characterize the detections. These are the observation ID (OBSID
); revolution number (REVOLUT
); the beginning and end
of the observation in Modified Julian Date format (MJDSTART
and MJDSTOP
); filter (ca_FILTER
) and submode (ca_SUBMODE
); note that the
latter two apply to all exposures in a merged set, see Sec. 2.2 .
Every row in the catalogue is a detection and has received a running
number (DETID
). Several
detections can refer to the same physical source in the sky (observed at
different times), these are identified with a unique source ID (SRCID
, see the description in
subsection a) below). Every detection is also identified by their
observation-specific (decimal) source number SRC_NUM
which, in the hexadecimal
system, is used together with the observation and exposure ID to identify
source-specific products via their file name.
Many parts of the sky were observed more than once, either because an
interesting object was a target more than once, or because two or more
fields happened to overlap. It was therefore desirable to identify all
cases in which the same source was responsible for two or more detections,
i.e., separate rows in the catalogue. All detections for which this appears
to be true have been given the same SRCID
number.
The matching to find unique srouces was performed on the basis of
coincidence of celestial coordinates within certain limits. An estimated
postitional error, POSERR, was computed by taking the larger of the
RADEC_ERR
value from the
source fitting and the estimated systematic error, SYSERRCC
, for the
observation. Because in a few cases RADEC_ERR values were unreasonably
large (up to 20 arcseconds) an upper limit to matching distance of
7 arcseconds was also applied.
All possible pairs of detections from different observations are considered and the great-circle distance between them, GCDIST, computed. Two detections a and b are are considered to be matched if (using SQL notation):
GCDIST < LEAST (0.9 * a.DIST_NN, 0.9 * b.DIST_NN, 7.0, 3.0 * (a.POSERR + b.POSERR)) .
The DIST_NN
value for each detection records the distance to its nearest neighbour
in that observation, which in a few cases was less than
7 arcseconds, generally because a detection which initially
appeared to be an extended object was split into two. The
0.9 * DIST_NN part of the formula was therefore used
to ensure that close pairs of detections did not cross-match
incorrectly.
The matching was performed efficiently within a Postgres database using R-tree indexing.
Since the actual X-ray sky remains unknown, it is likely that a few cases exist in which two distinct objects have been assigned the same SRCID number, or vice-versa. Further refinement of the matching algorithm will be attempted for the final 2XMM catalogue.
An IAU identification, IAUNAME
, has been assigned to each
unique source (SRCID
) based
upon the IAU registered classification 2XMMp. The form of these names is
"2XMMp Jhhmmss.sSddmmss" where hhmmss.s is taken from the eposcorr
corrected right ascension coordinate given in the column RA
and Sddmmss is the sign and
eposcorr corrected declination taken from the column DEC
. A detection (entry in the
catalogue) will be uniquely identified by their IAUNAME
and DETID
(with six digits): "2XMMp
Jhhmmss.sSddmmss:detid".
Several source parameters were averaged or otherwise combined to
characterize a unique source in the catalogue (N_DETECTIONS
indictaes the
number of detections found for the unique source). All columns referring to
the parameters of a unique source have the prefix AV.
Weighted means (inversely with the estimated variance) and the error of
it are given for coordinates (AV_RA
, AV_DEC
, AV_POSERR
) as well as the flux
(AV_FLUX
, AV_FLUX_ERR
) and hardness ratios
(AV_HRn
, AV_HRn_ERR
). Note that the error
on a weighted mean is calculated as
mean_err = SQRT( 1.0 / SUM( 1 / err_i2 ) ).
Maximum likelihoods (AV_DET_ML
and AV_EXT_ML
) are 'equivalent'
detection likelihoods, i.e., they are summed over all detections of this
unique source and corrected for the number of free parameters (cf. Sec. 3.1.2 f)). Finally, the maximum of
all summary flags was determined (AV_SUM_FLAG
).
Of the 2400 observations used for the present catalogue 458 observations
are in common with the 1XMM
Serendipitous Source Catalogue (cf. the selection of observations in Sec. 2.1). The most likely counterparts in the
1XMM catalogue were found by cross-matching the 1XMM detections with the
2XMMp unique source positions (using AV_RA
and AV_DEC
) with a simple limit of 10
arcseconds in distance; only the closest match is given. Their names
(MATCH_1XMM
) and the
distance between the two detections SEP_1XMM
) are given in the
catalogue.
Visual screening of the processed data was performed in a very simple
manner: Each observation was visually inspected with the sources overlaid
and the automatic flags indicated. It was assumed that the automatic
flags 7 and 8 indicate detections that are probably spurious and
should be excluded from any clean sample, that is, what we call 'spurious'
in the following definitions is assumed to not have
flags 7 or 8 set. With that in mind, four classes of observations
were indentified (see OBSFLAG
):
0 (52%): | no obvious spurious detections (may include one or two non-obvious ones); |
1 (30%): | one or more spurious detections, but usually less than 8 (or a smaller number when the total number of detections was exceptionally small); |
2 (0.9%): | should be part of class 1 but with the addition that there is a group of very densely positioned sources where many sources were missed and the quality of the source parameters may be questionable; |
3 (16%): | not accepted for inclusion in the catalogue; |
In addition, observations with known problems that affect the source detection and/or parameterization were excluded (0.7%).
The column SUM_FLAG
provides on overall quality indication of a detection, as a single integer
value, based on the automatically set flags (Sec. 3.1.2 h)).
The summary flag is defined as follows, with the individual flags being set to True for:
0 : | Summary flag 0 is given if none of the flags for the three cameras [PN,M1,M2] are set to True, i.e., there are no negative indications for this detection. |
1 : | Summary flag 1 is given if any of the warning flags [1,2,3,9] for any of the cameras [PN,M1,M2] is True, i.e., the source parameters are considered to possibly have some problems. |
2 : | Summary flag 2 is given if any of the 'spurious detection' flags [7,8] for any of the cameras [PN,M1,M2] is True (note that flag 7 is set to True if any of the flags for possible spurious extended detection [4,5,6] is set to True). i.e., the detection is likely to be spurious. |
A variability flag VAR_FLAG
was set to
True for a detection if at least one of the time series
for this detection (derived from all appropriate exposures) has a
Chi2-probability ≤ 1E-5 as determined by the SAS
task ekstest (see CHI2PROB
and Sec. 3.1.4). If the flag was set the camera and
exposure ID with the lowest Chi2-probability is given as well
(VAR_EXP_ID
).
Thumbnail images have been made of every detection in the catalogue. A maximum of 12 small and one large thumbnail image (called location image) is available per source. The small thumbnail images were made in the bands 6, 7, and 8 (see Tab. 3.1) for each of the (sometimes merged) exposures used in the source detection from the set M1, M2 and PN as well as the all-EPIC mosaiced image. The thumbnails are stored as GIF files.
The small and large thumbnail images are 8 and 24 arcmin across in size, respectively. The images are not smoothed. The band 1 - 5 image data are taken from the fits images which form part of the catalogue product set, and thus embody the same X-ray event selections as these images. The merged images are all derived by adding the respective individual band images together. The brightness scaling of the thumbnail images is linear, but pixel brightness is truncated at a given saturation value. The 'heat' colour map is used, and the images are scaled so that the pixel range from 0 to the saturation limit spans the colour map. The value of the saturation is calculated for optimum display of the source at the centre of the field.
Green cross-hairs are overlaid over the centre of the image to display the source position of interest.
The legend at the head of each image gives the following:
IAUNAME
and DETID
in the form "2XMMp
Jhhmmss.sSddmmss:detid".OBSID
). This is followed by the
two-letter camera identification code (M1, M2, PN, and EP; EP means the
summed image of all cameras, where the exposures are not corrected) and the
energy band (cf. Tab. 3.1). Summary html pages have been made for every detection in the catalogue. These pages give a selection of (somtimes simplified) parameters from the catalogue together with zoom-in images of the thumbnails, time series, and spectra.
There are 275 columns in the catalogue; they are grouped together and explained in the links below.
For each observation there are up to three cameras with one or more exposures which were merged when the filter and submodes were the same (Sec. 2.2). Each exposure is divided up into several energy bands (Tab. 3.1). Consequently, the source parameters can refer to some or all of these levels: on the observation level there are the final mean parameters of the source (prefix EP), on the camera level the data for each of the up to three cameras are given (prefix PN, M1, or M2), and on the energy band level we give the energy-dependent details of the source parameters (indicated by a _b_ in the column name where b = 1,2,3,4,5,8,9). Finally, on a meta-level we have combined some parameters of sources that were detected more than once (prefix AV), see Sec. 3.3.3.
The column name is given in capital letters, the FITS data format in brackets and the unit in square brackets. If the column originates from a SAS task, the name of the task is given to the right hand side and a link is set to the online SAS 6.5.0 package documentation on http://xmm2.esac.esa.int/sas/6.5.0/doc/packages.All.html (note that this is a slightly older version compared to the SAS used in the processing of 2XMMp which is not publicly available, see A.3 for more details). A description of the column and possible cross-references follow.
Entries with NULL are given when the detection was not
observed with the respective camera (that is, ca_MASKFRAC
< 0.15).
Part 1: | 6 columns: Identification of the source |
---|---|
This includes cross matches with the 1XMM catalogue | |
Part 2: | 11 columns: Details of the observation and exposures |
Part 3: | 8 columns: Coordinates |
The external equatorial and Galactic coordinates and the internal equatorial coordinates are given together with the error estimates. | |
Part 4: | 221 columns: Source parameters |
The parameters of the source detection as derived from the SAS tasks emldetect and srcmatch are given here. | |
Part 5: | 7 columns: Detection flags |
This part lists the flags to qualify the detections. The summary flag, which gives an overall assessment for the detection, is followed by particular flags for each camera. A flag each is given if there exists at least one time series or one spectrum for this source. | |
Part 6: | 5 columns: Source variability |
This part gives variability information for those detections for which time series were extracted. | |
Part 7: | 17 columns: Unique source parameters |
This part lists the source parameters for the unique sources across all observations (using the prefix AV); these are coordinates, fluxes, hardness ratios, likelihoods, and a summary flag. The number of detection is given also. |
The SAS task dpssflag sets the non-camera-specific flags
(flags 2,3,4,5,6,7, see Tab. 3.3) on the
summary row (EPIC band 8) and then propagates them backwards to the
individual cameras and energy bands. Due to a bug in the code it failed to
do so for all subsequent cameras when the source was not detected
in one camera (the order of cameras is PN, M1, M2). As a consequence, in a
small number of cases the flags 2 - 7 in PN_FLAG
, M1_FLAG
, and M2_FLAG
are not set to
True when they should have been. It is therefore
recommended to use the always correct EP_FLAG
when referring to
flags 2 - 7, and use the reliable camera-specific flags 1,8,9
when a discrimination into cameras is needed.
There is evidence for a small offset in the reported Chi2 values, not accounted for in the statistical test. This amounts to the measured value being too high by up to 0.05 in the Chi2 per degree of freedom ( = CHISQUAR / (N_POINTS - 1)), thus yielding too low a value for the probability (CHI2PROB) that the source is constant. At the chosen threshold of CHI2PROB <= 1E-5, the corrected probability may be up to ~5E-5 and has a median value of ~2.5E-5. On this basis, one may still expect no more than a few constant sources to be flagged as variable, given the ~60,000 total, individual timeseries.
In cases where there are two or more exposures per detection the catalogue columns PN_CHI2PROB, M1_CHI2PROB, and M2_CHI2PROB contain not the value of the smallest chi-squared probability for the available exposures but the largest. The value of VAR_FLAG is however correctly calculated. A table giving the original and correct values for the 32 detections where there was significant variability and there were multiple exposures is provided in Appendix A.4.
The XID band count rates are 3 times too low. This is due to the (known) problem in the 2XMMp pipeline that the band 2-4 exposure maps were added instead of averaged.
The band 4 EPIC fluxes were not calculated properly in the catalogue. The correct formulae to calculate the band 4 EPIC flux and its error are:
EP_4_FLUX = SUM ( ca_4_FLUX / ca_4_FLUX_ERR2 ) / SUM ( 1 / ca_4_FLUX_ERR2 ),
EP_4_FLUX_ERR = SQRT ( 1.0 / SUM ( 1 / ca_4_FLUX_ERR2 )),
where ca = PN, M1, M2.
In a small number of cases there are no bins in the spectrum file with good quality; in these cases xspec does not make a plot. During processing this was not recognized and a previous plot was used instead. A list of these spectrum plots which should all be empty can be found in Tab 4.1.
The catalogue contains source detections drawn from 2400 XMM-Newton EPIC observations made between 2000 February 4 and 2006 April 20, selected according to the criteria described in Sec. 2. Net exposure times in these observations range from < 1000 up to ~ 90000 seconds.
The total sky area of the 2400 XMM-Newton observations is ~ 400 deg2 which is reduced to ~ 285 deg2 when corrected for field overlaps. For more detail see Fig. 4.1 which shows the sky area as a function of net exposure time.
The catalogue contains 153105 X-ray detections with total-band [0.2 -12 keV] likelihood values ≥ 6. Of these 123170 are unique X-ray sources (Sec. 3.3.2 a)), that is, 46796 X-ray sources were observed more than once and up to 20 times in total. Of the 153105 X-ray detections 6271 are classified as extended, and 5717 of these are unique extended X-ray sources.
The total-band [0.2 -12 keV] median flux of the catalogue detections is ~ 2.4*10-14 erg/cm2/s, while ~ 20% of the detections have fluxes below 1*10-14 erg/cm2/s. Hence, the new source detection used for 2XMMp is more sensitive than the source detection used in 1XMM where the median flux was ~ 3*10-14 erg/cm2/s and only ~ 12% had fluxes below 1*10-14 erg/cm2/s.
The overall astrometric accuracy of the catalogue source positions has been checked statistically by cross-correlating with various optical catalogues and comparing the distribution of position offsets with expectations. These checks confirm that the 2XMMp positions have no systematic shift and that the quoted errors are realistic.
Release No. | Release Date | Comments |
1.0 | 24 July 2006 | First release |
1.0.1 | 04 August 2006 | Correct column names EP_EXTENT_ML and EP_EXTENT_ERR |
1.0.2 | 31 August 2006 | Added notes in section 4.2 on CHI2PROB problem and XID band count rates |
1.0.3 | 22 February 2007 | - Added notes in section 4.2 on problems with EPIC
band 4 fluxes and a list of incorrect spectrum plots. - Corrected an erroneous in the observation list in Table 2.1 (for observation 0200430701, camera M2). - Corrected equation for extent likelihoods. |
A small number of associated data products are released together with the catalogue and are available from various Web-based user interfaces (see here). (Note that the final 2XMM catalogue will come with all EPIC products produced in the pipeline). These selected products are:
- | EPIC images in fits format (bands 1 - 5 and 8 per exposure and EPIC band 8 image) and png format (all band 8 images); |
- | The EPIC exposure maps in fits format (bands 1 - 5 and 8 per exposure and EPIC band 8 image) and png format (all band 8 images); |
- | The source time series files in fits format; |
- | The source time series plots in pdf format; |
- | The source spectra fits files (source as well as background); |
- | The source arf fits files; |
- | The source spectra plots as pdf files. |
- | The optical finding charts as pdf files. |
These products follow the standard specification, as described in the Data Files Handbook and the SSC products Specification available at http://xmm.vilspa.esa.es/external/xmm_user_support/documentation/index.shtml.
Outside of the pipeline processing (Sec. 3.4) two extra set of products were made: the thumbnail and source location images (Sec. 3.4.1), and the source summary html pages (Sec. 3.4.2):
- | The thumbnail images are graphical products and were made from the fits (merged) images. They are named C<obsid><exp>SRCIMG<band><srcnum>.GIF. |
- | The location image is a graphical product made from the fits EPIC (merged) images; it is named C<obsid>EPX000SRCIMW8<srcnum>.GIF. |
- | The source summary page is an html page which includes several graphic products and some extracts from the catalogue; it is named C<obsid>EPX000SRCSUM0<srcnum>.HTM. |
These files were not made using the PCMS, and for this reason they have filenames starting with C instead of P (the other parts of the filename follow the pipeline product standard).
List of observations ('fields').
The processing was conducted during April and May 2006 using
the calibration files (CCFs) that were available on 11 April 2006. The
pipeline manifest can be found here
where the SAS task versions used for making the catalogue are listed
together with the task version numbers which are publicly available
(SAS 6.0, 6.5, and 7.0). The references to the SAS tasks given in the
user guide refer to the SAS 6.5 version which is slightly older than
the version used for processing and which is not publicly available.
detid | srcid | oldpn_chi2prob | pn_chi2prob | oldm1_chi2prob | m1_chi2prob | oldm2_chi2prob | m2_chi2prob |
---|---|---|---|---|---|---|---|
9557 | 7650 | 1.17377e-05 | 2.10999e-08 | 1.93683e-07 | 1.93683e-07 | 2.67107e-10 | 2.67107e-10 |
9559 | 7650 | 1.19683e-06 | 1.31235e-07 | 0.00324579 | 0.00324579 | 0.00188667 | 0.00188667 |
10446 | 8288 | 0.00660075 | 0 | 4.89135e-06 | 1.75755e-15 | 3.57594e-05 | 1.54045e-12 |
29787 | 23932 | 0 | 0 | 2.88454e-32 | 0 | 0 | 0 |
31605 | 24731 | 0.691837 | 1.93202e-16 | 0.898659 | 0.608448 | 0.882586 | 0.412855 |
31606 | 24731 | 0.115039 | 0.00187067 | 0.278877 | 1.84597e-06 | ||
32499 | 25582 | 0.639053 | 0.000613184 | 0.617727 | 4.06858e-06 | ||
32507 | 25590 | 1.74005e-20 | 1.74005e-20 | 0.639053 | 1.36321e-12 | 0.617727 | 9.53159e-08 |
32543 | 25624 | 3.15928e-07 | 3.15928e-07 | 0.639053 | 0.00165751 | 0.617727 | 0.0148653 |
32550 | 25631 | 4.38522e-24 | 4.38522e-24 | 0.771211 | 1.36499e-07 | 0.00221883 | 1.19183e-05 |
32802 | 25828 | 0.183871 | 2.0335e-09 | 4.32752e-06 | 4.32752e-06 | 0.0587464 | 0.0587464 |
32805 | 25830 | 0.360471 | 7.39745e-19 | 0.000125234 | 0.000125234 | 0.0092283 | 0.0092283 |
32818 | 25842 | 0.897808 | 7.72032e-24 | 0.00017236 | 0.00017236 | 2.52377e-07 | 2.52377e-07 |
33016 | 25993 | 0.109717 | 0.0381989 | 0.391389 | 0.391389 | 3.0614e-06 | 3.0614e-06 |
35106 | 27727 | 8.35872e-20 | 8.35872e-20 | 0.343035 | 1.04537e-06 | 0.83493 | 1.49114e-09 |
46915 | 36611 | 0.0191414 | 2.51477e-11 | 0.409126 | 0.000968114 | 0.622424 | 0.000166273 |
48243 | 37721 | 0.000812056 | 0 | 0.98771 | 0.98771 | ||
50481 | 39529 | 0.000225779 | 0 | 0.738336 | 5.58982e-14 | 0.833166 | 2.70628e-17 |
50811 | 39827 | 0.396345 | 7.174e-07 | 0.381365 | 1.99242e-17 | ||
57188 | 44295 | 0.390173 | 7.17607e-23 | 0.630363 | 0.0470923 | ||
57193 | 44297 | 0.0411725 | 1.52168e-20 | 0.939411 | 0.0786263 | ||
57526 | 44439 | 1.11496e-20 | 1.11496e-20 | 0.163991 | 1.35788e-09 | 0.0768756 | 1.26636e-09 |
57744 | 44563 | 5.83865e-20 | 5.83865e-20 | 0.697517 | 2.1888e-07 | 0.154472 | 1.70633e-05 |
61927 | 48076 | 0.704294 | 3.40813e-06 | ||||
64780 | 50589 | 0.474691 | 0 | 0.953092 | 0 | ||
64807 | 50609 | 0.474691 | 2.17314e-07 | 0.953092 | 1.56783e-08 | ||
64816 | 50618 | 0.457168 | 8.71879e-16 | 0.953092 | 1.29783e-18 | ||
95914 | 76436 | 0.211565 | 1.81129e-08 | 0.0775838 | 1.31466e-06 | ||
117246 | 94297 | 0.388802 | 2.21633e-11 | ||||
131566 | 105332 | 0.000964788 | 3.27169e-22 | 0.000789271 | 0 | 1.96626e-14 | 0 |
132697 | 106206 | 0.317102 | 0.317102 | 5.97427e-27 | 5.97427e-27 | 0.00981824 | 0.00186613 |
142328 | 113584 | 0 | 0 | 1.39407e-11 | 1.54977e-14 | 6.33997e-14 | 1.11164e-14 |