#
Minutes of the Radiative Transfer Workshop

Bremen, April 26-29, 1999

*** List of Participants: (name, affiliation, email, initials) **** (in alphabetical order) Stefan Buehler IFE sbuehler@uni-bremen.de SB Patrick Eriksson IFE patrick@rss.chalmers.se PE Johannes Kaiser IFE johannes@uni-bremen.de JK Yasuko Kasai CRL ykasai@crl.go.jp YK Miriam von Koenig IFE miriam@schalk.physik.uni-bremen.de MK Klaus Kuenzi IFE kunzi@physik.uni-bremen.de KK Thomas Kuhn-Sander IFE tkuhn@uni-bremen.de TK Nicolas Lautie OBX lautie@observ.u-bordeaux.fr NL Frank Merino MISU frank@misu.su.se FM Jungang Miao IFE jungang@thor.physik.uni-bremen.de JM Satoshi Ochiai CRL ochiai@crl.go.jp SO Joachim Urban OBX urban@observ.u-bordeaux.fr JU Carmen Verdes IFE cverdes@smiles4.physik.uni-bremen.de CV *** Addresses: ****************************************** CRL: Communications Research Laboratory Global Environment Division 184-8795 Japan IFE: Institute of Remote Sensing University of Bremen 28353 Bremen, Germany MISU: Meteorological Institute Stockholm University Sweden OBX: Observatoire de Bordeaux Radio-Aeronomie B.P. 89 33270 Floirac, France ********************************************************* ===== April 26, Afternoon ====== 1. Welcome by Prof. KK. 2. Introduction by SB o Goals of the workshop: - Need of a new forward model in Bremen - No discussions about the implementation, we want to - focus on physics and algorithms. Gather alternatives. o Proceedings: - Contributions of the participants will be collected after the workshop and published as a report or something similar. o Finished model - Will be public available, similarly to GNU programs - Strong emphasis on modularity o The agenda 3. Presentation of existing models GOMETRAN (JK) ------------------ o UV/Vis o measures scattered sunlight o absorption is included, but no thermal emission o pseudo-spherical geometry SCIATRAN (JK) ------------------- o full spherical geometry o UV/VIS/NIR - data bases - includes thermal emission o everything else similar to GOMETRAN o there already was a single scattering prototype Q: KK: Mie Scattering PE: Quasi analytical weighting functions, what means answer: exactly what you mean by "analytical w.f." JU: Is spherical standard? Skuld (FM) ------------------------- o Nov 96 Workshop in Gothenburg, organised by PE o two parallel codes, skuld and moliere o emphasis: - general - user friendly - accurate and fast (weighting functions!) o controlled by control file o user manual available (plus report) o geometries: limb/ground o abs: - line by line - Liebe 93 for O2, H2O, N2 - Verdandi catalogue - line shape specified by the user o integration: - optional refraction (Snell's law) - tangent altitudes specified as viewing angles - integration along the line of sight with arbitrary step length (fixed) - sum up homogenous layers o instrumental effects - antenna pattern + Odin simulated + Gauss + static or moving beam - side bands + primary plus image band + frequency dependent sideband ratio stored in file - spectrometer + shape of each channel described by response curve o other: - Doppler shift - data binning - thermal noise - weighting functions Skuld part 2 (PE) ---------------------- o data binning - different binning pattern for each viewing angle (but same number!) - spectra can be removed (e.g., if opacity is too high) - binning data stored in file o species weighting functions - treated as part of the forward model - analytical - incl. sensor characteristics - normalised to a priori, VMR, number density - arbitrary grid, but same for all species - retrieval grid consists of layers o other weighting functions - T - spectro parameters - pointing offset - interfering species - frequency shift (not useful, too non-linear) o outlook - 3 geometries - no hard coded frequency limit - sensor model by matrix approach - weighting functions: + for as many variables as possible + analytical for T, no hydrostatic eq. + both offset and polynomial representation for pointing and absorption fits Q: JU: Scattering in IR? --- No MOLIERE (JU and NL) ------------------------ o Philippe Baron's o different models o combined forward/inversion model o abs line by line using Verdandi - Liebe 93 - refraction Q: KK: Cross check with Skuld? --- Yes (difference only due to different Voigt algorithm) MAES at CRL (SO, YK) ---------------------- o catalogue JPL or HITRAN o LS: Lorenz or Voigt, dep. on pressure o MPM93 o limb or ground-based o integration: - total abs coef. for each layer - calculate tau - integrate by trapezoidal method o antenna pattern included o C/UNIX program o config file, ASCI o problems: - pressure broadening parameter database - p-shift, Doppler shift missing - far wing line shape (is this really a problem?) - speed, necessary: + line selection + different frequency grid between emission and tau calculation (for weighting functions) - how to quickly calculate the average emission in the passband of a filter Q: KK: Who includes p-shift? --- just IFE IFE - forward model (SB) ----------------------------------------------- o Long history, presently model started early 90īs o Implementation well validated by several intercomparison o Mix of Pascal, Fortran and C o Standard structure o Absorption coefficients: - JPL and MYTRAN - All species of JPL/HITRAN - Pressure shift - Special treatment of 60 GHz oxygen cluster - Several continuum models (Liebe, CKD and Rosenkranz) - Twin-Voigt lineshape based on Drayson Voigt-algorithm - P-shift included where data available o RTE - Geometries: zenith, limb and nadir - Tangent altitudes are used (not viewing angles) - Adaptive layer breakup, depending on opacity - Linear or exponential interpolation of abs. coefficients - Ground: two emissivities (horizontal and vertical polarisation) - Refraction (needs very fine layering to be accurate) o Instrumental parameters - Antenna, sideband and spectrometer o Meta features - Line selection (only for JPL) - Frequency grid selection (only JPL and narrow bands) o Missing features - Scattering - Analytical limb WF - Automatic selection of angular and frequency grids o Good - General, validated and accurate o Bad - Chaotic code - Slow - Not portable, runs only on IBM RS 6000 - No documentation - No dynamical memory handling o Scattering so far has been handled by separate code 4. Discussion of desired features for the new model. Results: o 1 = essential, 2 = ----, 3 = optional, 4 = not at all o Refraction, 1 o Zeeman, 3 (but leave this possibility, not to be discussed here) o Scattering, 2 o 2-D, 3 o Pressure shift: 1 o Geometries: up, down and limb o Only emission measurements o Continuum: 1 o The range around 60 GHz, 2 o Only LTE o Weighting functions: 1 o Data binning: 1 o Sensor: 1 - Antenna: 1 - Sideband: 1 - Spectrometer: 1 - Baseline ripple: ? -- 4 - Doppler: 4 5. Discussion of line catalogue contents and format: o Line format, changes to the MYTRAN/Verdandi format - Add reference temperature for pressure broadening (already in MYTRAN) - Add reference temperature for intensity - Add overlap parameters - Add Zeeman splitting constant(s) - Not include degree of freedom for the molecule - The source of the spectral data should be included o Labelling of lines - Two schemes exist, JPL and HITRAN - Symbolic representation, e.g. H2O-18 - Difficult to find a symbolic name scheme. Is it possible? - AI: IFE looks into this problem and sends out a specific suggestion after the workshop o Partition functions - The total partition function shall be given - Different interpolation schemes can give differences of some percent - Give the the partition function as polynomial coefficient - Interesting temperature range 150 - 300 K - Frequency range as HITRAN ===== April 27, Morning ====== 1. Introduction of PE: Again about the purpose of the workshop: o Participants should state what he/she wants to have integrated in the FM. 2. Topic of discussion: Radiative transfer equation (RTE) o theoretical introduction of this problem by PE (geometrical considerations sensor <-> path of view and refraction). o short presentations of the different ways to tackle the problem for the different existing FMs: MAES: * Equally spaced horizontal grid points (129 points between tangent altitude point and end of 120 km thick radial atmospheric layer). * Logarithmic interpolation between horizontally successive points of calculated absorption coefficients to calculate the opacity (tau). * Calculating absorption coefficients at each tangent altitude. * Linear interpolation in the vertical direction. Skuld: * Calculating the tangent altitude first. Then divide the horizontal line of view into equidistant parts. These grid points are different from the points where the absorption coefficients are calculated. To get the information of the abs. coeff. at the horizontal grid points of interest, a linear interpolation is used. => SB: Far from line center an exponential interpolation is more appropriate. But the accuracy is also strongly dependent on the grid spacing. * The absorption coeff. is assumed to be constant in each horizontal grid interval respectively. * The atmosphere along the line of view is assumed symmetric to the tangent point (space <-> tangent point and tangent point <-> sensor). * The effect of refraction bends the "line of view" to the earth due to Snellīs law for spherical layers (see PEīs thesis p. 246). Moliere: * code is very similar to that of Skuld. The way how the refraction is implemented is different to the Swedish one. The expression for the refraction is linearized in a Taylor series around the known (but false) geometrical tangent height. But a problem can be, that the true tangent height can differ considerably from the geometrical tangent height and this can lead to not reliable results of the Taylor series. => SB: how can the refraction be treated in an experiment with limb sounding from a balloon or aircraft. => PE: occultation sounding is even more difficult to implement into a FM. => PE: refraction index for dry air and water vapor depends differently on frequency. Dry air is more or less constant below 1 THz, where the refraction index for water vapor shows some resonance structures at strong water vapor lines (picture from his thesis). BREMEN-FM: * Calculating the contribution to the intensity from each vertical layer separately. * A normal sum over all vertical intervals is not very accurate near the tangent point. If the opacity is too high (above a threshold value), then the interval will be split in further smaller intervals. But this is a danger of the FM, because it is not really transparent what actually is done by the FM (-> frequency dependence?). => JUR: this threshold vale is not optimized for speed. Therefore this way of doing is in fact not so obscure. But anyway it should be removed. o Question of interpolation: Temperature is constant throughout an interval and the abs. coeff. is supposed to be linear increasing in the BREMEN-FM. PE ran into the error function when he assumed both Temperature and abs. coeff. as linear increasing. -> what should be assumed inside a layer for temp. and abs. coeff. constant or linear? o Integration in equal opacity levels? --> Theoretically better, but not very efficient (opacity has to be calculated on fine grid first anyway) o ->Problems: how to interpolate VMR and number density (pressure). VMR is assumed linear, and number density (or pressure) exponential. This point is connected with the question which coordinate system, pressure or tangent altitude, is used. Temperature is also assumed linear. The absorption coefficient is proportional to pressure in the far wing of the line but not near the line center. Therefore one should implement linear as well as exponential interpolation. The selection of an interpolation method for the abs. coeff. in a layer depends strongly on the width of this layer. o approaches for interpolations? Skuld interpolates linearly between vertical VMR profile points. => SB: what is meant with effective abs. coeff. and effective temp. in an vertical layer? PE: if the temp. is constant then eff. abs. coeff. is the mean value. Conclusion: implementation of both strategies: linear and constant for the abs. coeff and constant for temperature for vertical layers. But there is no general rule because it also depends strongly on the vertical layers you implement in the FM. ===== April 27, Afternoon ====== o Definition of observation geometries? => PE: viewing angle is the natural coordinate for limb sounding. o RTE in pressure steps? Pressure, temperature and altitude are the three quantities of concern. but with hydrostatic equilibrium assumption and ideal gas law these three are correlated. Spherical geometry has to be considered for this question (air mass function). Conclusions: do not assume hydrostatic equilibrium in advance. One has to consider all three variables as input into the FM. Hydrostatic balance can be enforced before, if desired. o Because of the future need, the new FM should be usable for limb sounding from the aircraft and the balloon. o 2 D - Horizontal structure. - Particularly useful if observation in the orbit plane. - Approach: ray tracing (needs alpha and T on 2 D grid). - One looses the symmetry around the tangent point. - No principal problems, just higher computation time. - Usual approach to antenna convolution not possible (because sensor movement). Needs explicit set of pencil beam calculations for each tangent altitude. - There are smart ways to avoid calculating so many absorption coefficients (e.g., pre-calculate for mean and some other states and interpolate). - Just practical (computation time), no principal difficulties. o Single scattering (Intro by JK) - need to do pencil beam computation in all directions for radiation scattered into the beam. - SB: Most dramatic difference comes from the change in EXTINCTION. (For limb geometry, according to MASTER study, conducted at IFE) - different sources: + thermal emission in place + direct solar light + thermal emission around (from all directions) - MK: What about polarization. - JM: Calculation of phase function is the real problem. - For inclusion in extinction only you don't need phase function, just scattering coefficient. - JM: How to calculate weighting functions? - JM: Quantitative results only with polarization - The hierarchy of increasing effort: 1. Extinction only o need scattering coefficients (spherical particles) o absorption == extinction no longe holds, but this is just a small change in the RTE 2. Also include radiation scattered into LOS from other directions o Different possible sources (sun, thermal) o RTE integration in all directions (no azimuthal dependency, for this `correction' maybe assume flat earth.) 3. Same thing polarized (has been done for nadir, by Evans) 4. Above thing with multiple scattering, but not polarized (This has already been done, e.g., SHDOM) 5. Both multiple-scattering and polarized. (This has never been done so far.) 3. Calculation of absorption coefficient revisited o Spectral line catalogue --> yesterday o Format/units --> skipped o Merging/conversion tools - AI Bordeaux to look for Philippes program which appears to be lost. - Reliable parameters: Quantum numbers BUT: difficult - YK: JPL quantum number system is better - AI YK: Write document how to convert JPL QN to HITRAN QN o Line shape function - Use single Voigt function for all altitudes for all species except H2O and O2 - (nu/nu_0)^2 term? --> yes - Problem: oxygen, h2o o What is the best Voigt implementation? --> Kuntz o Speed issues: - maybe tabulated line shape functions - JU: neglecting something only way to really improve speed o Refraction - Liebe o Should refractive index be calculated for all gases, or just for oxygen and water vapor (continua)? --> PEs data indicates that effect of lines is small o P-shift: --> Simple, so include it. o Overlap correction: - Species: + O2, HNO3?, HCl? - Additional parameter delta (needs three additional line parameter) o Continua - PE: Define outside the model (don't hardcode) - Which? + Rosenkranz or NEW CKD (look at new CKD). TASK LIST for Proceedings (starting yesterday) o Summary of existing models - MAES - Skuld - MOLIERE - Bremen - GOMETRAN/SCIATRAN o Absorption (Line catalogue) - YK: JPL and HITRAN quantum numbers - NL to look for Philippes program which appears to be lost - Verdandi description (PE) - Draft a format (IFE) o Absorption (general) - Absorption intro (SB) - Continuum Models (TK) - Planned lab measurements (YK) - How to calculate p-shift (SB) - Simple parameterizations for refraction (NL) ( - ask Agnes Perrin where p-shift and line mixing effects have to be expected (SB) ) - Sources for extinction coefficients (Spherical cloud particles) (JM) o RTE - Detailed descriptions for Skuld (as intro) (PE) - 2 D (IFE) - Need of altitude profiles (think over, possibly write something, or discuss later during the workshop) (all) - Scattering (JK) ===== April 28, Morning ====== o Weighting Functions: which variables should be included ? Species, Temperature, Absorption offsets etc. - SB gave an introduction about weighting functions. - PE shows the analytical derivation of WFs inside Skuld. In this program the FM is split up into two sub-models: one for the atmospheric physics (FMP) and one for the sensor (FMS), FM = FMP & FMS. FMP is an input of FMS. Another question is the units one uses inside the FM, e.g. for the concentrations of the species. => MK: If the weighting functions are calculated at slightly different points from the retrieved VMR, then an interpolation has to be done. She interpolate linearly, where PE uses a step function for Skuld. The grid for the weighting functions should be finer than that for the retrieved VMR. The retrieval grid is not as fine as the calculation grid for the weighting function (normally). Therefore the question is how the weighting function value for the coarser retrieval grid can be calculated from the weighting functions for the finer grid. (Two issues: 1. Conversion from grid equally spaced along line of sight to grid equally spaced in the vertical; 2. How to assign weights, so that the weighting functions on the vertical grid are consistent with the idea that the profile is represented by point values with linear interpolation between points.) => JK: The weighting functions already have a internal weight proportional to the layer thickness they cover. Therefore a weighted sum of the weighting functions is not necessary, instead an ordinary sum is sufficient. => JM: Explained the meaning of total and differential weighting functions. The differential weighting functions have already the information about their covered layer thickness. - SO showed how MAES tackles the problem of weighting function calculations. He said that MAES is not in a final state and has to be refined, checked and verified in future. This FM uses a completely different concept/approach for weighting functions as, e.g., Skuld or IFE-FM. - NL: if the weighting function is divided by the vertical layer thickness then this new term is free of thickness weighting and one can use this expression ("normalized weighting function") for the contribution along the line of sight in limb sounding. - SB: for which quantities should the weighting functions be calculated? - Trace gas species (analytically) - Temperature + without hydrostatic ( (semi)-analytically ) + with hydrostatic (by perturbation) - Absorption offsets (analytically) o Work for the proceedings: description (e.g. which quantities ?) of the concept/approach of the weighting function algorithms will be written by the following participants: PE / FM (Skuld) MK (IFE-FM) JU / NL (Moliere) SO / YK (MAES) JK (GOMETRAN) MK should think and write (for the proceedings!) about the topic of a proper normalizing of the weighting functions. o Weighting Functions: how to calculate - numerically or analytically? - SB: for the species it should be possible to calculate the weighting functions analytically. For the temperature, it should be analytically in the case of non hydrostatic equilibrium and numerically (perturbed) in the case of hydrostatic equilibrium. - Analytical calculation is much faster than numerical in the case of weighting functions. - PE: one can analytically express the derivative of the abs. coeff. to temperature if one makes some approximations. - SB: description of the absorption offset used in the MASTER study. Which assumption about the abs. offset is taken depends on the width of the frequency band considered. For wide bands, a quadratic offset gives good results, but for narrow bands, a linear offset is more appropriate, because one can badly determine the quadratic shape in this case. In the FM, weighting functions for a quadratic/linear abs. offset should be implemented. - SB: The pointing can be described by a polynomial. - Perturbation calculations for other parameters: Should these really be implemented in the FM or should they be done by the retrieval model in the sense, that in the retrieval model one can call the FM with slightly different parameters. ====== April 28, Afternoon ====== - Weighting functions by perturbation (continued) List (not necessary complete) of parameters which could be considered: # pointing # spectroscopic parameters # instrumental parameters antenna far wing, sideband, spectrometer o Modeling of instrumental parameters is the new topic. - Inventory of instrument characteristics # sideband integration # filterbank integration # antenna integration # Doppler shift (?) - Does it have an impact in which order the steps are taken? SB: Gives the formulas for sideband and antenna integration, how the IFE-FM is doing this. FM: it makes a difference in which order the antenna and sideband integration is done, because the antenna pattern is frequency dependant. The correct order of the instrumental effects is: 1. antenna pattern 2. sideband convolution 3. filterbank (it has then to be mirrored). SB: is it allowed from the physical point of view to interchange point 2. and 3. ? --> Probably yes. Anyway, keeping the right order is no problem. - The question was after discussed, in which way one should implement these three points. Each effect can be expressed as a matrix vector multiplication. Instead of doing repeated matrix vector multiplications, one can multiply the matrices associated with the different effects directly. - SB: Shows the formulas for the calculation of the side band ratio. There are two different formulations in the literature (see SBs thesis). PE: Writes down his formula, which is again different from those that SB showed. But one can easily transform each formula into one of the others. - How should two continuous functions be integrated when only discrete values at different grid spacings are given. This leads to the question how the interpolation is treated. It is stated, that standard methods are given in textbooks. - Data binning is the next point. This means reduction of data in the sense to group data in appropriate sets (e.g. reduction of frequencies). PE: data binning is included in Skuld. There are binning patterns defined. These patterns should depend on the altitude because in the spectrum at low altitude the lines are very wide (more flat and only few data points are needed) and at higher altitudes the line width is small and a lot of data points around the peak are needed, therefore a pattern which is valid for all altitudes is not very helpful. The main question is how to group the data in a good way. SB suggests a relation with the weighting functions. PE think orthogonal functions should also be appropriate and standard methods should be available form other fields (e.g. image processing). The further question is how the grouped data points should be treated, just the mean of them? JK: he says that he has a paper about it from peoples from Oxford. He will distribute this paper to the participants. SB: question if a reduction is also practicable for the internal frequency grid. A rough method is already used in the MASTER/Soprano studies. o The question of modules and their interfaces depends strongly on the implementation. The figure of the schematic description of the FM of PE (see thesis) was used to discuss this topic. ===== April 28, after Dinner ====== Working out detailed descriptions of the three modules `absorption', `RTE', and `weighting functions' in two parallel groups. ===== April 29, Morning ====== o Results of the group work last evening: The Absorption group and the RTE/Weighting function group presented their results. (See transparencies) o Presentation of JM: Atmospheric Parameters in Polar Regions. (From SSM/I and SSM/T2 data) - Water vapor - Clouds? (LWP from SSM/I with improved algorithm) o Discussion of meta features / higher level features - Spectral line selection MAES: take all lines Skuld: full calculation for each line (pencil beam) - calc. abs (f=f0, or closest end of band) - RTE for ztan = max(abs) - check if Tb > Tb_lim - if Tb > Tb_lim/factor: check a few lower ztanDo some combination. - Viewing angle grid: + Antenna pattern can depend on viewing angle (if you model continuous scan by boxcar convolution of antenna). + Angular grid should be specified explicitly (analogously to frequency and altitude) (potential for optimization) - Integration grid along LOS: + This can be equidistant SO: Exchange Antenna convolution and RTE? PE+SB: It seems that this only works if you assume the optically thin case. AI SO: Write about his approach to weighting functions / antenna. - Splining + Splining in frequency should work well for uplooking, but probably not for limb sounding. --> We don't do this. o User interface - Interactive use: PE wants it + JU also + SB: Must be handled in a simple and generic way. - Controlfile structure: + Parameters specified on the command line can override parameters in controlfile (So you can use the controlfile as a template) o General program structure - Strongly modular --> Much smaller and simpler blocks than in current programs JK: Modules `knowing' what they need. (Recursive approach) So the `user' just calls the `last' module (the one that generates the measurement vector). This then call the module that generates its input. o Input for proceedings: - Send LaTeX - Figures in Postscript - Deadline June 5. - Ask ESTEC if they want to print it. o Raw notes will be sent by Email o Goal: First Beta version by autumn. o Thanks to all participants and goodbye by SB. -------------------------------------------------------------- List of action items for the proceedings Deadline: June 5, 1999 Text: LaTeX Figures: Postscript -------------------------------------------------------------- JU: o Short description of Moliere. o Weighting function calculation in Moliere. o Frequency optimization. NL: o Look for Philippes Baron's Catalogue merging program, which appears to be lost. o Write about the simple parameterization for the refractive index. SO: o Short description of MAES. o Weighting function calculation in MAES, in particular concerning the antenna integration. YK: o Write documentation how to convert JPL QN to HITRAN QN and vice versa. o Write about planed laboratory measurements IFE: o Suggestion for line catalogue format (send out draft to the others to comment on). o A scheme for tag groups / species naming, that allows arbitrary grouping of lines o A few words about 2D TK: o Write about continuum models MK: o Weighting function calculation for uplooking o How to properly normalize the weighting functions - What is the best definition of a normalized weighting function (independent of layer thickness)? - How to compute it practically? SB: o Short description of the old IFE/forward program. o Description of MYTRAN o Absorption intro o Describe explicitly how to calculate p-shift PE: o Short description of Skuld o Description of Verdandi o RTE intro o Line selection FM: o Describe weighting function calculation in Skuld JK: o Short description of GOMETRAN/SCIATRAN o Write something about scattering (just very basic intro) o Weighting function calculation in GOMETRAN. o Distribute the Oxford paper on data binning JM: o Write about sources for extinction coefficients (for spherical cloud particles).