Purpose: To generate input files Weather.csv, Evaporation.csv, and Radiation.csv for i-Tree HydroPlus Research Suite
Author: T. Endreny at SUNY ESF, te@esf.edu

0. If WeatherPrep.exe is not in C:\iTree\WeatherPrep\WeatherPrep\WeatherPrep\bin\Release, then build the WeatherPrep.sln in VisualStudio** in Release mode
1. Run WeatherPrep_GUI.py or WeatherPrep_GUI.exe to simplify data acquisition and processing, or ...
2. Acquire NOAA NCEI GHCNh input data needed by WeatherPrep.exe by running z_GHCN_Retrieval_PST_to_CSV.py or following FTP and ishapp2.exe steps described in the file ReadMe_Get_ISH-formatted_NOAA_Data.txt within the folder WeatherPrep\NOAA_ISD_ishapp2
2a. Note: NOAA NCEI transitioned from ISD to GHCNh format in late August 2025
2b. Note: The ISD is now legacy, and not supported past late August 2025 
2c. Note: There are known differences in GHCNh and ISD record length and data values for any hour or day for overlapping dates
3. Prepare the WeatherPrepConfig.xml input file, described below, also needed by the WeatherPrep.exe. 
4. Generate HydroPlus input files by running WeatherPrep.exe in a Command Prompt window followed by the path to inputs in the same Command Prompt window
4a. C:\iTree\WeatherPrep\WeatherPrep\WeatherPrep\bin\Release>WeatherPrep.exe C:\iTree\WeatherPrep\TestCase
4b. If program fails, then: 
4ba: - ensure computer Windows Settings for Regional Format and Windows display language = English (United States).
4bc: - ...

5. WeatherPrepConfig.xml contains the following information:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Program generates output of: Weather.csv, Evaporation.csv, and Radiation.csv -->
<InputList>
  <Input>
    <Model>Hydro</Model><!-- Options: Hydro|Bulding; Hydro=9 weather variables; Building=Hydro+WindDirection-->
    <Nation>United States of America</Nation> <!-- Confirm name here: https://database.itreetools.org/#/locationSearch -->
    <State>New York</State> <!-- Confirm name here: https://database.itreetools.org/#/locationSearch -->
    <County>Onondaga</County> <!-- Confirm name here: https://database.itreetools.org/#/locationSearch -->
    <Place>Syracuse</Place> 3!-- Confirm name here: https://database.itreetools.org/#/locationSearch -->
    <MaximumLAI>5</MaximumLA3> <!-- Leaf Area Index maximum; affects canopy resistance; units m^2/m^2-->
    <EvergreenPercent>5</EvergreenPercent> <!-- Tree cover that is evergreen; affects evaporation during leaf of period; units percent -->
    <VegetationType>Tree</VegetationType> <!-- Options: Tree, Shrub, or Grass; affects canopy interception for i-Tree Eco -->
    <Height_Tree_m>12</Height_Tree_m> <!-- Average height of tree canopy top in area; affects canopy resistance; units m -->
    <Height_WindSensor_m>10</Height_WindSensor_m> <!-- Height of wind sensor for weather station; 10 is standard; units m-->
    <StartYear>2019</StartYear> <!-- Year of starting data in SurfaceWeatherDataFile; format YYYY -->
    <EndYear>2019</EndYear> <!-- Year of ending data in SurfaceWeatherDataFile; format YYYY  -->
    <SurfaceWeatherDataFile>C:\iTree\WeatherPrep\TestCases\NewYork_Syracuse_Precip\725190-14771-2019_out.txt</SurfaceWeatherDataFile> <!-- NOAA NCEI GHCNh.csv or ISD.txt -->
    <PrecipitationDataCsv></PrecipitationDataCsv> <!-- Optional CSV file to replace SurfaceWeatherDataFile values. Column 1 = date (YYYYMMDD), Column 2 = time (HH:MM:SS), Column 3 = precipitation (m/h) -->
  </Input>
</InputList>

6. Model is the name of the i-Tree Research Suite model whose meteorological input files are being prepared.
6a. 'Hydro' refers to any of the models that use the HydroPlus unified C++ codebase: i-Tree Hydro GUI, i-Tree HydroPlus, i-Tree Cool Air, and i-Tree Cool River
6b. 'Energy' refers only to i-Tree Energy, a building energy balance model coded in Python
7. Location Country, State, County, City are specific names confirmed online at i-Tree Location Database https://database.itreetools.org/#/locationSearch
8. MaximumLAI, EvergreenPercent, VegetationType are values provided by user
9. Height_Tree_m and Height_WindSensor_m are values provided by user, but defaults are Height_Tree_m=12, Height_WindSensor_m=10. Changing Height_WindSensor_m may significantly alter evaporation estimates for open water. 
10. StartYear, EndYear are values corresponding to meteorological data in SurfaceWeatherDataFile
11. SurfaceWeatherDataFile is ISH-formatted NOAA raw meteorological data for the location's nearest WBAN coded meteorological station. The file C:\iTree\WeatherPrep\NOAA_ISD_ishapp2\ReadMe.txt stored with this .sln has the procedure for generating this file.
12 PrecipitationDataCsv as .csv is optional to replace values in SurfaceWeatherDataFile (NOAA data often do not report), Column 1 = date (YYYYMMDD), Column 2 = time (HH:MM:SS), Column 3 = precipitation (m/h). 
13. Output generated is Weather.csv, Evaporation.csv, and Radiation.csv for i-Tree Research Suite

**For developers looking to edit or update the WeatherPrep code: the packages "RestSharp" and "RestSharp.Serializers.NewtonsoftJson" cannot currently be updated beyond version 106.12. More recent updates are missing the needed RestSharp.RestClient.ClearHandlers function.

~~~
These are notes for 20250325 (YYYYMMDD) on the WeatherPrep CalcCloudCover function for computing opaque (OPQ) and translucent (TRN) cloud cover from total (TOT) cloud cover from NCEI integrated surface database (ISD) hourly weather data. This WeatherPrep OPQ CalcCloudCover function was attributed to the Atkinson and Lee (1992) paper in a 2006 VBA version of WeatherPrep called WeatherReader.xlsm. By 2026 this CalcCloudCover function will likely be repalced to handle new NCEI Global Historical Climatology Network (GHCN) hourly data that contain different variables than ISD, and may allow refined OPQ and TRN calculations. 

NCEI CLG and SKC data: The NCEI integrated surface database (ISD) hourly weather data (e.g., ISH is considered an ISD type file) report CLG and SKC. The CLG is ceiling height of lowest opaque clouds with 5/8 or greater coverage (units hundredths of feet). When CLG is 722, it is a code indicating unlimited, and is not a height. The SKC is sky cover (text values CLR, SCT, BKN, OVC), where SCT represents 1/8, 2/8, 3/8 and 4/8, with an average of 2.5/8 or 0.3125, and BKN represents 5/8, 6/8, and 7/8, with an average of 6/8 or 0.75, and CLR is 0/8 and OVC is 8/8. One possible issue in the WeatherPrep ReadSurfaceWeatherData function is assigning a sky cover (SKC) value of 3.75 for the condition SCT, which is 1/8, 2/8, 3/8 and 4/8 and which averages to 2.5/8 or 0.3125, which then gets used in this CalcCloudCover function. The new NCEI GHCN hourly data format appears to provide information on sky_cover_# (e.g., SKC codes and cloud fractions) and sky_cover_baseht_# (e.g., CLG ceiling heights) for cases where # can equal 1, 2 and 3 layers. In cases when sky_cover_# is > 1 layer, we may have enough information to truly partition between opaque and translucent cloud types. In a separate article Atkinson may have worked on about ASOS, they seem to explain a single reporting of cloud ceiling height is not likely enough the get both values of OPQ and TRN. 

Opaque vs translucent clouds: The CalcCloudCover function is trying to use CLG and SKC to partition between opaque and translucent clouds, but NCEI ISD hourly weather data defines CLG as LOWEST OPAQUE LAYER WITH 5/8 OR GREATER COVERAGE or UNLIMITED if 722. The Atkinson and Lee (1992) algorithm may be suggesting opaque should go to zero when ceiling height goes above 7000 feet; in cases when CLG is between 75 and 300 (hundredths of feet), the CalcCloudCover function sets OPQ = 0 and the clouds are considered translucent. The NCEI ISD hourly weather data contain cases when CLG > 70 (hundredths of feet) and SKC is OVC, suggesting that it should not be OPQ = 0. The CalcCloudCover function case when CLG > 700 (e.g., 722 or unlimited) and SKC < 5/8 (e.g., is CLR or SCT) assigns opaque to total cloud cover, which will be CLR = 0 or SCT = 2.5/8 or 3/8; this seems odd given CLG = 722 (a code for unlimited, not a height) is likely higher than CLG > 70 (hundredths of feet) and could be considered equally translucent to always be OPQ = 0. The result of CalcCloudCover function partitioning NCEI cloud cover between OPQ and TRN is binary, always assigning all cloud cover to one or the other. In only one case this might be otherwise, where it assigns OPQ = 5.0, but that condition [i.e., else if (Ceiling > 700 && total > 5)] should never occur with NCEI data because when CLG = 722 the SKC is always CLR or SCT, and total would be <= 5. We can test for this and remove, or allow for it to remain as a safety backstop. 

Missing data: The Atkinson and Lee (1992) algorithm is focused on filling empty data values of OPQ and TOT based on other data, including cloud ceiling height. Prior to the CalcCloudCover function, WeatherPrep fills missing data via interpolatio, in the FillInterpolate function (InterpolateCeiling and InterpolateToCldCov). WeatherPrep reads in NCEI integrated surface database (ISD) hourly weather data (e.g., ISH is considered an ISD type file) for variables CLG and SKC data.  

Ceiling heights: The CalcCloudCover function use of thresholds of 700 and 70 (hundredths of feet) seem odd, but benign, where changing 70 to 700 only changed solar radiation direct by ~4% and diffuse by ~3% for 10 years of tested data. The threshold of > 700 seems to be for cases when CLG was 722, and CLG is considered unlimited, and its use of threshold <= 70 seems to be for cases when the ceiling is less than 7000 feet (note CLG is in hundredths of feet, so CLG = 70 is CeilingHeight_ft = 7000). The CLG < 70 (hundredths of feet or 7000 feet) and else (i.e., CLG >= 70 hundredths of feet or 7000 ft) maps to where Atkinson and Lee (1992) write, “then a 0 is used for opaque cloud cover if the ceiling is greater than 7000 feet (high level clouds are generally not opaque)”. The National Weather Service defines mid-level and high-level clouds as above 6500 feet, and may generally consider both translucent. The NCEI ISD hourly weather data has CLG values between 70 and 722, with values from 300 to 75 identified in other data sets, which would correspond with the CalcCloudCover function else statement, greater than CLG 70 and less than CLG 700 (i.e. 722). 

Atkinson, D. and R. F. Lee (1992). "Procedures for Substituting Values for Missing NWS Meteorological Data for Use in Regulatory Air Quality Models." Environmental Protection Agency (EPA).
USEPA (1997). Analysis of the Affect of ASOS-Derived Meteorological Data on Refined Modeling. Research Triangle Park, NC, US Environmental Protection Agency.