HDF-EOS Subsetting: HEW and other tools HDF and

HDF-EOS Subsetting: HEW and other tools HDF and

HDF-EOS Subsetting: HEW and other tools HDF and HDF-EOS Workshop VI San Francisco, CA December 4-5, 2002 Dr. Sara Graves, Bruce Beaumont, Helen Conover, Matt Smith Presented by Steve Tanner University of Alabama in Huntsville, Information Technology and Systems Center SUBSET.ORG Agenda Subsetting HDF-EOS Web-based subsetter

(HEW) HEW Subsetting Appliance: Integration with ECS HSE: A new Subsetting Engine Software availability options HEW Demonstration December 4-5, 2002 HDF-EOS Workshop VI Subsetting Goal: to provide a science data user with only the data they request as quickly as possible. Benefits science data users and data centers: - reduces analysis time by reducing amount of data

- reduces time for data delivery - reduces resources (network, personnel, media, etc.) Steps: - locate spatial / temporal / spectral area of interest - extract - re-assemble for distribution/use December 4-5, 2002 HDF-EOS Workshop VI Currently Available/Planned Subsetting Applications HDF-EOS Subsetting - HEW

Complete System (available) Subsetting Back-end Only (available) SPOT - Subsettability Checker (available) Subsetting Center at UAH (available) HEW Integration with ECS (in work) HEW Subsetting Engine (in work) Subsetting as a Web Service (planned) Customized Subsetting

MODIS tools (available) Coarse-grain SSM/I Subsetter (available) General Purpose Customizable Subsetting Based on ADaM Data Mining Engine (available) Subsetting Tool using ESML (in work) December 4-5, 2002 HDF-EOS Workshop VI

HEW HDF-EOS Web-based Subsetter Dataset-independent for HDF-EOS files Optional Front-end/GUI Uses HTML forms and JavaScript Back-end Needs subset criteria and HDF-EOS data Performs subsetting as a batch job http://subset.org December 4-5, 2002

HDF-EOS Workshop VI HEW Back End Uses HDF-EOS (and HDF) library Instructed via a subset criteria file (ODL) Handles multiple similar files Handles Swath and/or Grid objects Unix (SGI & Sun) executables available Subsetted output files contain: StructMetadata (HDF-EOS)

ArchiveMetadata* ProductMetadata (added by HEW ODL file) CoreMetadata* (w/ modified bounding box & time info) optionally placed in .met file * if present in parent file December 4-5, 2002 HDF-EOS Workshop VI HEW Complete System The User Interface checks the HDF-EOS file and presents

the attributes to the user. The user interacts with the browser to specify the subsetting criteria. The User Interface creates the subsetting criteria file. The HEW Back-end uses the subset criteria file with the input HDF-EOS file to create the subset HDF-EOS file. Users Browser (HTML) User Interface (CGI) Input file

Subsetting API (ODL) HEW Back-end December 4-5, 2002 HDF-EOS Workshop VI Output file HEW Back-end Only User-supplied

front-end Subsetting API (ODL) Input file HEW Back-end Output file A user-supplied program or other application creates the subsetting criteria file.

The HEW Back-end uses the subset criteria file with the input HDF-EOS file to create the subsetted HDF-EOS file. December 4-5, 2002 HDF-EOS Workshop VI Example Subset Criteria File GROUP = SUBSET PARENT_FILE =(/AQUA/AMSR/AE_L2A.hdfeos) LATITUDE_RANGE = (35.000000, 40.000000) LONGITUDE_RANGE = (-77.000000, -72.000000) EMAIL = [email protected] MET_FILE = YES GROUP = SPOG

NAME = swath_1 TYPE = SWATH PARAMETERS = (89.0V_Res.1_TB, 89.0V_Res.2_TB) SUBSAMPLING = (GeoTrack, 2, GeoXtrack, 1) END_GROUP = SPOG END_GROUP = SUBSET END December 4-5, 2002 HDF-EOS Workshop VI HSE: HEW Subsetting Engine Similar functionality to HEW back end, but contained

within a callable function User Application Code Subsetting criteria Return code HEW Subsetting Engine function Input file(s) HSE_LogMsg callback

Output file(s) HSE_StatusMsg callback User application code builds subsetting criteria structure Subsetting engine function calls users functions for status and log messages No e-mail is sent

Beta version and documentation available now Supports only grid subsetting at this time BCEA and SOM grids cannot be subsetted yet Swath subsetting available first quarter 2003 December 4-5, 2002 HDF-EOS Workshop VI HEW Subsettable data EOS DATASETS Terra MODIS MOPITT

ASTER Aqua AMSR-E AIRS Aura HIRDLS December 4-5, 2002 OTHERS TRMM TMI NOAA-15,16 AMSU-A any other HDF-EOS data written

with HDF-EOS library subsetting calls in mind HDF-EOS Workshop VI SPOT Subsettability checker Displays content/structure of HDF-EOS files Examines files for subsettability by HEW Simple command-line interface Stand-alone operation v1.4 now available for SGI and Sun Available at subset.org

December 4-5, 2002 HDF-EOS Workshop VI Subsetting Center Remote user pushes file (or subsetting center could pull file) to subsetting center Remote user interacts with HEW to perform subsetting Remote user pulls file (or subsetting center could push file) to remote site Remote User ftp Subsetting Center

HEW Input disk December 4-5, 2002 HDF-EOS Workshop VI Output disk Subsetting as a Web Service (Planned) Subsetting Center Subset request

Subsetted data Science User URL to data Data Store December 4-5, 2002 HDF-EOS Workshop VI HSA: HEW Subsetting Appliance UAH/ITSC has been working with ESDIS, ECS (EOSDIS Core System), and DAACs to incorporate subsetting appliance into the

ECS/EOS Data Gateway (EDG) system. Provides for subsetting as part of ECS automated order filling process. Infrastructure written by ITSC/UAH to allow other appliances (subsetting, reprojecting, reformatting, etc.) to be added later by other data centers. December 4-5, 2002 HDF-EOS Workshop VI ECS integration plans EDG v3.4 has basic subsetting options Testing at NSIDC, operational in Jan 03

Testing at LPDAAC (EDC) Testing will begin at GDAAC in 03 Further enhancements as requested by DAACs December 4-5, 2002 HDF-EOS Workshop VI Tools developed for MODIS Scientists MODIS Land, Quality Assessment modland subsetter for MODIS gridded data stitcher pieces together 2 or 4 contiguous MODIS tiles

MODIS Atmosphere modair specialized subsetter for MODIS swaths December 4-5, 2002 HDF-EOS Workshop VI Subsetting Tool using ESML (in work) HDF-EOS ESML file Binary/ ASCII

ESML file Network For HDF-EOS data not formatted for subsetting: ESML file can be used to correct the semantic tag required to subset HDF-EOS data without the need to recreate the data file December 4-5, 2002

ESML Library Subsetting Algorithm Subsetted Data HDF-EOS Workshop VI Other Formats ESML file Subsetting web-site subset.org The subsetting portal is being

created for everyone involved in subsetting Advertising Forums Data Software Glossary Tutorials Links to specialized subsetters December 4-5, 2002 HDF-EOS Workshop VI Subsetting Demonstration

December 4-5, 2002 HDF-EOS Workshop VI Select TMI files for subsetting Display combined geographicc overage Select subset

parameter Combined geographicc overage of input files Geographic Select geographic subset criteria subset criteria

Output directory for this job Display combined geographicc coverage of subset results Combined geographicc overage of

subset results

Recently Viewed Presentations

  • The Cold War - Henry County School District

    The Cold War - Henry County School District

    1950 Red Scare. McCarthy made a public accusation that more than two hundred "card-carrying" communists had infiltrated the United States government. Though eventually his accusations were proven to be untrue, and he was censured by the Senate for unbecoming conduct....
  • Poetic Devices - Loudoun County Public Schools

    Poetic Devices - Loudoun County Public Schools

    Auditory Imagery: From "Dover Beach" - Matthew Arnold Listen! You hear the grating roar Of pebbles which the waves draw back, and fling, At their return, up the high strand, Begin, and cease, and then again begin, With tremulous cadence...
  • How Much Can Policy Affect How Much People Drive?

    How Much Can Policy Affect How Much People Drive?

    Adding in a recent RFF estimate of "externalities" would add a few cents per mile to the cost of autos Adding in the most extreme and implausible automobile costs proposed by smart growth activists would add about $0.23 per passenger-mile...
  • C

    C

    Rejecting configuration: q = q. r, or u = e and transition says move leftHalting configuration: Accepting/rejecting configuration. One-step execution semantics: for non-halting configuration C, Next(C) is the configuration resulting from executing one transition of M.
  • Chapter 1: Matter and Measurement

    Chapter 1: Matter and Measurement

    Chemistry 140 Fall 2002 * This equation shows CO32- acting as a base instead of OH-. Our definition must be expanded from simple Arrhenius theory (Bronsted-Lowry and Lewis) Chemistry 140 Fall 2002 * Fe(2) is oxidized to Fe(3). Therefore peroxide...
  • CTC Mod 2 - Pennsylvania Child Welfare Resource Center

    CTC Mod 2 - Pennsylvania Child Welfare Resource Center

    Please click on the button below to access CWIS. ... Your notes should reflect what you learned in CTC Mod 2. Click below to review: Categories of Abuse. Key Legal Definitions. ... Please listen as Kanesha engages Ms. McAbee about...
  • Machine Architecture - University of Maryland, Baltimore County

    Machine Architecture - University of Maryland, Baltimore County

    Bytes. Discussed briefly earlier. Consists of 8 bits. Historically, this was machine-dependent, however the de-facto standard became 8 bits. Smallest addressable unit of memory in a modern computer. You can't request the value of a single bit in memory. The...
  • Chapter 3 ENTREPRENEURS IN A MARKET ECONOMY

    Chapter 3 ENTREPRENEURS IN A MARKET ECONOMY

    Variable costs are costs that go up and down depending on the quantity of the good or service produced. MARGINAL BENEFIT AND MARGINAL COST Marginal benefit measures the advantages of producing one additional unit of a good or service. ......