1. 24 Jul, 2018 1 commit
  2. 01 Feb, 2018 4 commits
    • Mohammad Akhlaghi's avatar
      Added link to describe reproducible science · 151b2f38
      Mohammad Akhlaghi authored
      A link to the page where I have described reproducible science has
      been added to the README to help people unfamiliar with the concept
      understand the purpose of this reproduction pipeline.
    • Mohammad Akhlaghi's avatar
      Used direct Zenodo link for Gnuastro download · 160f6eea
      Mohammad Akhlaghi authored
      The link describing where to get the necessary version of Gnuastro was
      changed to point directly to the file in Zenodo. Also, the README file
      was slightly edited.
    • Mohammad Akhlaghi's avatar
      Unified and updated copyright information · aafdb049
      Mohammad Akhlaghi authored
      The copyright information on the files were not in a unified
      format. They are now all in a single format and mention the years that
      work has been done on them (2016-2018).
    • Mohammad Akhlaghi's avatar
      Updates for hosting on Zenodo · 38026113
      Mohammad Akhlaghi authored
      The main input dataset (the MUSE pseudo-broad-band images) and the
      whole pipeline in general (along with the HST images and throughputs,
      and also the necessary version of Gnuastro) will be uploaded to Zenodo
      for easy access in the future.
      With this commit, if the MUSE images are not in the given directory,
      they will also be downloaded and unpacked for use (similar to the HST
      images). The README file is also updated to account for this
      change. The contents of this commit will also be uploaded to Zenodo.
  3. 31 Jan, 2018 1 commit
    • Mohammad Akhlaghi's avatar
      Finalized reproduction pipeline · 408ebae5
      Mohammad Akhlaghi authored
      This commit does not affect the processing.
      Some final touches were made to the pipeline to make it easier and
      more stable for reproduction. The most important changes are:
       - The configuration files now have the specific version of Gnuastro
         in them so the pipeline doesn't run with any other version.
       - A README file was added to explain how to run the pipeline and also
         the dependencies.
       - Since the `flock' program might not be available on some systems, a
         Perl script was put in `reproduce/scripts/flock' so people can use
         it instead.
       - Some corrections were made in the downloaded file names.
       - Until now, the pipeline used a separate version of Gnuastro for its
         Table program which could write output into a FITS table (version
         0.2.51-bc56 which is used in the pipeline can't do that). But the
         format of the output is irrelevant, for the purpose here, the
         contents are important. So this step was removed and the final
         tarball output of the pipeline is now fully reproducible with the
         single 0.2.51-bc56 version of Gnuastro.
       - The final tarball is placed in the `reproduce/output' directory,
         because it may be necessary later (when the intermediate files are
  4. 30 Jun, 2017 1 commit
    • Mohammad Akhlaghi's avatar
      Minor edit in data-product target place (no effective change) · f6768ca6
      Mohammad Akhlaghi authored
      The `data-products' target was defined under the comments for the
      BibLaTeX target. So to make it more clear, some comments were
      specifically written for it and it was placed separtely in that file.
      This change has no effect on the results, only helps in readability.
  5. 05 Jan, 2017 3 commits
    • Mohammad Akhlaghi's avatar
      Outputs in FITS table format · b48e7024
      Mohammad Akhlaghi authored
      The comments on the final tables were corrected to fit the new
      Gnuastro plain text table format. So using the Table program from new
      Gnuastro (currently under development) datastruct branch, the plain
      text tables were converted to FITS tables. They are then packed into a
      `.tar.gz' file to be easily communicated.
    • Mohammad Akhlaghi's avatar
      Plots for difference in RA and Dec created · fd222a3a
      Mohammad Akhlaghi authored
      The plots for the difference in RA and Dec as a function of magnitude
      for the mosaic and the UDF10 have been created in the final PDF.
    • Mohammad Akhlaghi's avatar
      NoiseChisel on MUSE images for astronometry · 21afff81
      Mohammad Akhlaghi authored
      Until now NoiseChisel was ran on the HST images and the segmentation
      map was used for the comparisons of both magitude and
      astrometry. While this is accurate for the former, it is not so good
      for the latter, especially as the targets become fainter. The shape of
      the segmentation map (from HST) will bias the shallower MUSE
      astrometry. So for astrometry, we done the opposite: doing detection
      on MUSE and using its segmentation map on both MUSE and HST. The final
      comparison catalogs are now made with this commit and we can go onto
      the statistics.
  6. 03 Jan, 2017 2 commits
    • Mohammad Akhlaghi's avatar
      Included the difference in distance plots for each filter · a5d7f860
      Mohammad Akhlaghi authored
      The difference in distance plots are now also plotted in a similar way
      to the differences in magnitude. To do this generically and easily,
      the $(two-d-hists) rule in `statistics.mk' was slightly modified, so
      all the two-d histograms are generated with separate invokations of a
      single AWK command. This greatly helps later corrections/additions.
    • Mohammad Akhlaghi's avatar
      Reorganized files, working on statistics · 506e81d5
      Mohammad Akhlaghi authored
      The source files have been separated: with the Make (`.mk') files now
      in `reproduce/make' and the scripts (for the time being only AWK) in
      `reproduce/scripts'. Also work has started on deriving the statistics
      of each field, but is not yet complete (it isn't a prerequisite of the
      final PDF for now in this commit).
  7. 27 Dec, 2016 1 commit
    • Mohammad Akhlaghi's avatar
      Degraded HST images are now sky subtracted · d5828815
      Mohammad Akhlaghi authored
      Until now, NoiseChisel was run on the degraded HST images only for the
      segmentation map. However, because of the bad Sky subtraction in the
      HST images (as shown in the previous commit), the F814W result was
      strongly affected. So we now run NoiseChisel on the degraded HST
      images first, save the segments, and then subtract the Sky value from
      the degraded image. The result in the F814W image is now reasonable.
  8. 26 Dec, 2016 3 commits
    • Mohammad Akhlaghi's avatar
      A description is written for the process and results · 526d8e92
      Mohammad Akhlaghi authored
      A first draft of a description has been written to explain the
      process.  Since the F814W results are so significantly
      offset/different, a cutout was also included as PDF in the paper to
      display the possible cause.
    • Mohammad Akhlaghi's avatar
      Segmentation maps from degraded HST image · 3ccbc979
      Mohammad Akhlaghi authored
      Until now, the segmentation maps were created from the original
      resolution HST image, then warped to the grid of MUSE. However, the
      mixed pixels that were created would bias the position measurements of
      the objects, so now, we are using NoiseChisel on the degraded HST
      image. But since there is practically no noise left in those images,
      we use very lax NoiseChisel parameters. Also, the Sky value of the HST
      images is subtracted before convolution.
    • Mohammad Akhlaghi's avatar
      Corrected ranges of 2D histograms · 7b6f75d4
      Mohammad Akhlaghi authored
      The ranges of the 2D histograms were corrected to more
      reasonable/descriptive values for better comparison.
  9. 25 Dec, 2016 2 commits
    • Mohammad Akhlaghi's avatar
      Plots are built in description.pdf · 1f5b0616
      Mohammad Akhlaghi authored
      PDF plots of the magnitude differences are now made and included in
      the description. Currently, the description doesn't have any text, but
      tha will be added when everything is thoroughly checked.
    • Mohammad Akhlaghi's avatar
      Catalogs ready for plotting are created · 69a635ba
      Mohammad Akhlaghi authored
      Catalogs with HST and MUSE magnitudes and positions are now created
      for the full UDF Mosaic and the UDF-10 field. The comments above the
      rules are descriptive enough.
  10. 24 Dec, 2016 2 commits
    • Mohammad Akhlaghi's avatar
      One pixel rows or columns adjusted in MUSE cutouts · 2135a3ba
      Mohammad Akhlaghi authored
      Due to very small WCS differences, we might be having one column or
      row difference between the warped HST images and the MUSE images. To
      generate the catalog, the two images need to be the same size, so this
      was a problem.
      To solve it, the MUSE images are now cutout from the input image in a
      new directory (`muse-cutouts'). Then, after the HST images are warped,
      the MUSE and HST-warped image sizes are compared and if they are
      different, the MUSE image corrected (by one row or column of pixels
      along the respective dimension). Then it is put into the old `cutouts'
      directory. If the sizes are the same, then a copy is just put there.
    • Mohammad Akhlaghi's avatar
      Catalogs generated for similar sized cutouts · 0654f265
      Mohammad Akhlaghi authored
      Magnitude and position catalogs are now generated for both the HST
      images and the MUSE images. Using GNU Make's Secondary expansion, we
      don't need the `-h' or `-m' suffixes for the segmentation maps any
      more. Since the instrument name will be removed from the output
      filename during secondary expansion.
      The Sky and Sky standard deviation images are now also created. Note
      that since the purpose of this study is calibration, they are set to
      all 0 and all 1 valued images. Finally, using the filters and
      instrument the zeropoint magnitudes are calculated and used for
      Currently the catalogs are only used for similarly sized original
      cutouts. But the cutouts can be differently sized (due to sub-pixel
      differences in the WCS). So when the images have a different size, it
      currently stops.
  11. 23 Dec, 2016 1 commit
    • Mohammad Akhlaghi's avatar
      Segmentation maps built, directories reorganized · e4e7a6c5
      Mohammad Akhlaghi authored
      NoiseChisel is now run on the original HST images and the object
      labeled image is then warped to the MUSE resolution and prepared to
      derive the segmentation maps. To make the prerequisite management of
      the final catalog much more easier, the final cutouts of both MUSE and
      HST are now in the same directory, only differing with a `-m.fits' or
      `-h.fits' suffix. The segmentation maps also follow the same naming
      convention, however, since the segmentation map is identical, the
      `-m.fits' file is actually a symbolic link to the `-h.fits' file.
  12. 22 Dec, 2016 2 commits
    • Mohammad Akhlaghi's avatar
      Correct estimation of MUSE PSFs and deconvolution · 2c31a594
      Mohammad Akhlaghi authored
      The model MUSE PSFs were incorrectly calculated until now! I was
      mistakenly assuming that the `FSF01FWA' was the FWHM. However, the
      FWHM must be calculated from `b*lambda+a' where `b' is the value to
      `FSF01FWB' keyword and `a' is the value of `FSF01FWA'. The effective
      `lambda' (or wavelength) to use for each broad-band filter is the
      throughput-weighted average wavelength of the filter.
      So, the pipeline now uses the HST filter throughputs to find `lambda'
      and then uses the values of these keywords to generate the proper FWHM
      for the Moffat function. The throughputs will be downloaded if not
      already available. The FWHMs decreased from around 14 pixels to around
      10 pixels.
      After implementing this correction, a test run of the deconvolution
      gave a better result. The reason was that the image crop to feed into
      the deconvolution algorithm was larger and so lower frequencies could
      be sampled. After seeing this trend, I increased the truncation radius
      of the model MUSE PSFs and also the deconvolution image size and saw
      that the deconvolution output kernel is becoming better and better. I
      finally stopped at 6 times the FWHM since it was apparently not making
      much more change.
      So, now the proper kernel to convolve with the HST images is created
      and we can directly compare the results with the MUSE images without
      worrying about a larger PSF. After convolution, the HST images are now
      also sampled to the same pixel size as MUSE.
    • Mohammad Akhlaghi's avatar
      Top level dirs in build made by one rule · 42f5ffb5
      Mohammad Akhlaghi authored
      The directories directly under the build directory are now built by
      one rule in the top level `Makefile' to avoid the confusion and Make
      errors that can be caused by doing it every time. This can also
      encourage building more of these directories for easy checking.
      The size of the kernel was also increased because in the MUSE PSF, 21
      pixels was too small. The problem with the deconvolution is certainly
      more fundamental than that and must be solved separately. So for the
      time being we are ignoring the steps in generating the HST PSF.
  13. 21 Dec, 2016 2 commits
    • Mohammad Akhlaghi's avatar
      Moved the aligned UDF images to a separate directory · 487d752d
      Mohammad Akhlaghi authored
      The aligned UDF (not UDF10) images were in the same directory as the
      cutouts of each subfield. They are now saved in a separate directory
      of the top build directory to make the cutouts directory more easily
      understandable/managable. Especially since the HST images will be
      coming in too soon.
    • Mohammad Akhlaghi's avatar
      MUSE and HST PSFs created, but images not matched · bdca120b
      Mohammad Akhlaghi authored
      The MUSE PSFs were given as parameters to a Moffat function fit. So
      they were created using Gnuastro's MakeProfiles. The HST PSF was also
      generated from one of the bright stars in the UDF. It isn't the
      brightest, but so far we haven't checked if it was saturated or
      not. However, apparently, Gnuastro's convolve had problems in finding
      the kernel to convolve with the HST image to get to the MUSE
      So for now, we have kept the rules (to work on them later), but they
      are currently ignored in the pipeline (they aren't a prerequisite to
      anything). Since the MUSE PSF is much wider than the HST, for now, we
      will just convolve the HST image with the MUSE PSF. Later, when the
      problem above is solved, we will find the proper kernel more
  14. 18 Dec, 2016 2 commits