COMMANDLINES TO CHECK AND MAKE UP MISSING FILES 0. Locations for L1B, L2 and L3 links /data/data055/DATA_LINK/ 1. Script: /data/data055/operational/routine/script/checkfile Function: List all VIIRS-SNPP RDR/SDR files for a certain date Usage: checkfile YYYYMMDD (check VIIRS-SNPP) checkfile YYYYMMDD j01 (check VIIRS-NOAA20) 2. Script: /data/data055/operational/routine/script/scdrdown.sh Function: Download RDR/SDR files from SCDR to the correct location Usage: scdrdown.sh YYYYMMDD DATA_TYPE [SENSOR_NAME] DATA_TYPE includes: SVM01, SVM02, ... , RVIRS-RNSCA, IVOBC, etc for VIIRS, and S3A_OL_1_EFR, S3B_OL_1_EFR, S3A_OL_2_WFR, etc for OLCI, can be combined like "SVM01 SVM02" SENSOR_NAME includes: npp and jpss1 Example: scdrdown.sh 20190120 "S3A_OL_1_ERR S3A_OL_1_EFR S3A_OL_2_WRR S3A_OL_2_WFR S3B_OL_1_ERR S3B_OL_1_EFR S3B_OL_2_WRR S3B_OL_2_WFR" 3. Script: /data/data055/operational/routine/script/daily_gain_state Function: Extract gain status info from SNPP VIIRS L1B for use in MSL12 (must be run on rhw1075 or rhw1183) Usage: daily_gain_state [YYYYMMDD] if YYYYMMDD is omitted, will update the gain status for the last 8 days 4. Script: /data/data055/operational/routine/script/daily_nrt_msl12 Function: Produce destriped VIIRS-SNPP NRT L2 & L2bin files (must run on rhw1075 or rhw1183) Usage: daily_nrt_msl12 YYYYMMDD [overwrite] YYYYMMDD can take the form of e.g. 20181016_t19 to subset the processing time range 5. Script: /data/data055/operational/routine/script/daily_repr_msl12 Function: Produce VIIRS-SNPP SCI L2 files before destriping (must run on rhw1075 or rhw1183) Usage: daily_repr_msl12 YYYYMMDD STREAM_NAME [overwrite] YYYYMMDD can take the form of e.g. 20181016_t19 to subset the processing time range STREAM_NAME includes: bmw, swir, nsw, test, etc., can be combined like "bmw swir" 6. Script: /data/data055/operational/routine/script/daily_nrt_j1 Function: Produce destriped VIIRS-NOAA20 NRT L2 & L2bin files (must run on rhw1075 or rhw1183) Usage: daily_nrt_j1 YYYYMMDD [overwrite] YYYYMMDD can take the form of e.g. 20181016_t19 to subset the processing time range 7. Script: /data/data055/operational/routine/script/daily_repr_j1 Function: Produce VIIRS-NOAA20 SCI L2 files before destriping (must run on rhw1075 or rhw1183) Usage: daily_repr_j1 YYYYMMDD STREAM_NAME [overwrite] STREAM_NAME includes: bmw, swir, nsw, etc., can be combined like "bmw swir" YYYYMMDD can take the form of e.g. 20181016_t19 to subset the processing time range 8. Script: /data/data055/operational/routine/script/daily_olci_s3a (daily_olci_s3b for OLCI-S3B) Function: Produce OLCI-S3A/S3B data (must run on rhw1075 or rhw1183) Usage: daily_olci_s3a YYYYMMDD [overwrite] 9. Script: /data/data055/operational/routine/script/daily_destripe Function: Produce destriped SCI L2 and L2bin files (must run on rhw1075 or rhw1183, need to have striped data ready under 'orig' sub-folder) Usage: daily_destripe YYYYMMDD STREAM_NAME [overwrite|""] [j01] STREAM_NAME includes: bmw, swir, nsw, etc., can be combined like "bmw swir" YYYYMMDD can take the form of e.g. 20181016_t19 to subset the processing time range To disable overwriting use "" as the third argument Add j01 as the fourth parameter to destripe VIIRS-NOAA20 instead of VIIRS-SNPP 10. Script: /data/data055/operational/routine/script/map_tc_update Function: Produce true color imagery for various sensors/projectins; copy it to web server for OCView Usage: map_tc_update config.txt [YYYYMMDD] See ls /data/data055/operational/routine/lut/config_*_tc.txt for sample config files. Please do not change these config files unless you really know what you are doing. If YYYYMMDD is specified, will only run for that day; Otherwise, will run for recent days as specified by ndays_update variable in config.txt Uses significant amount of temporary storage; be sure the host running this script has enough directly acessible storage For recent days, will reuse and update the temporary files if run on the same host 11. Script: /data/data055/operational/routine/script/binmap_daily Function: Produce daily L3 bin files and product imagery; copy to web server for OCView Usage: binmap_daily [-f] [-g] [-p] data_stream YYYYMMDD [infile] Options: -f force update of bin files, even if they appear up-to-date -g generate imagery in geographic (lon-lat) projection -p generate imagery in polar projection Available data_streams: IDPS-NIR, OC-NIR, OC-SWIR, OC-NIRSWIR, OC-TEST, NOAA20-NRT, NOAA20-NIR, NOAA20-SWIR, NOAA20-NIRSWIR, OLCI-NRT, OLCI-OTC, OLCIB-OTC, SGLI-OTC infile=[infile_par|infile_nnp], if omitted, will use [infile] Determines if to run for regular products (including iop products), or par, or nnp for ESA OLCI 12. Script: /data/data055/operational/routine/script/binmap_timeave Function: Produce time averaged L3 bin files and product imagery; copy to web server for OCView Usage: binmap_timeave [-f] [-g] [-p] sensor/datastream timeave YYYYMMDD [infile] Options: -f force update of bin files, even if they appear up-to-date -g generate imagery in geographic (lon-lat) projection -p generate imagery in polar projection sensor/datastream - for example, VIIRS/noaa_msl12_swir see /data/data320/L3bin_global for all sensor/datastream combinations timeave=8_day|monthly|61day|climatology|monhly_climatology YYYMMDD - reference_date from which to start the time average infile=[infile|infile_iop|infile_par|infile_nnp] Determines if to run for regular products, iop products, or par, or nnp for ESA OLCI 13. Script: /data/data055/operational/routine/script/binmap_mixed Function: Produce two or more sensor/datastream mixed L3 bin files and product imagery; copy to web server for OCView Usage: binmap_mixed [-f] [-g] [-p] "sensor1/algo1 sensor2/algo2 ... sensorN/algoN" sensorMIX/algoMIX "prod1 prod2 ... prodM" "time1 time2 ... timeK" [infile] Options: -f force update of bin files, even if they appear up-to-date -g generate imagery in geographic (lon-lat) projection -p generate imagery in polar projection sensor1/algo1, sensor2/algo2 - input sensor/datastreams, for example, VIIRS/noaa_msl12_swir etc sensorMIX/algoMIX - output sensor/datastream, for example, VIIRSMIX/noaa_msl12_nrt see /data/data320/L3bin_global for all sensor/datastream combinations prod1, prod2, ... - product names (as specified in bin files) time1, time2, ... - time for which to run, for example daily/2018/20180814, or monthly/2019/201901 see /data/data320/openlayers/{sensor}/l2products/{datastream}/{product}/ for time averages Examples: binmap_mixed "VIIRS/noaa_msl12_nrt VIIRSJ1/noaa_msl12_nrt" VIIRSMIX/noaa_msl12_nrt "chlor_a kd_490 kd_par" daily/2018/20180814 binmap_mixed "OLCI/msl12_olci_nrt VIIRS/noaa_msl12_nsw VIIRSJ1/noaa_msl12_nsw" MIXED/noaa_msl12_nsw "chl_oci kd_490" daily/2018/20180814 infile=[infile|infile_iop|infile_par|infile_nnp] Determines template for L3 bin files (regular products, iop products, or par, or nnp for ESA OLCI) 14. Script: /data/data055/operational/routine/script/binmap_anom Function: Produce L3 bin files for product anomaly, and make anomaly imagery; copy to web server for OCView Usage: binmap_anom [-f] sensor/datastream timeave YYYYMMDD [infile] Options: -f force update of bin files, even if they appear up-to-date -g generate imagery in geographic (lon-lat) projection -p generate imagery in polar projection timeave=[8_day|monthly|61day|climatology] (time average to use for anomaly calculation) YYYYMMDD - date for which to run infile=[infile|infile_iop|infile_par] Determines if to run for regular products, iop products, or par 15. Script: /data/data055/operational/routine/script/binmap_update /data/data055/operational/routine/script/binmap_update_threads (same script, uses parallel processing with threads) Function: Update all recent L3 bin files and the associated imagery This script can be configured to call binmap_daily, binmap_timeave, binmap_anom, binmap_mixed, and chl_dailypic as needed for each sensor/datastream Usage: binmap_update config_script Look for config scripts under /data/data055/operational/routine/lut/binmap_config/ Please do not change these scripts unless you really know what you are doing The ndaysmax variable set in config_script determines how far back in time the script will search and look for data to update This script is running as a cron job, and will identify and fill the recent data gaps automatically so should not normally be run manually 16. Script: /data/data055/operational/routine/script/chl_dailypic Function: Produce daily chl global image with NOAA logo for Ocean Color web front page Usage: chl_dailypic [sensor] [YYYYMMDD] sensor = [VIIRS|VIIRSJ1|VIIRSMIX|OLCI] optional, default=VIIRS YYYYMMDD - date, optional, default = 28 hours ago 17. Script: /data/data055/operational/routine/script/grib2down.sh Function: Download original GFS and GDAS grib files from SCDR Usage: grib2down.sh [YYYYMMDD] 18. Script: /data/data055/operational/routine/script/getdata_nrt.sh Function: Convert SCDR-downloaded GFS files to hdf format Usage: getdata_nrt.sh YYYYMMDD 19. Script: /data/data055/operational/routine/script/getdata_sci.sh Function: Convert SCDR-downloaded GDAS files to hdf format Usage: getdata_nrt.sh YYYYMMDD 20. Script: /data/data055/operational/routine/script/anc_update Function: Download ancillary files from NASA as well as linking NCEP hdf ancillary files Usage: anc_update [YYYYMMDD]