Timeseries Gap Identification
In this example we will use `nadi_csv` plugin to load CSV files. The plugin is external, refer to the installation page to install it into the NADI System.
Different from other chapters, the code in this chapters are run in the order they are given, meaning each task code blocks are not independent.
Load the network
network load_file("data/scioto/scioto.network")
network cairo.table("<Name => {NAME}", "output/scioto-net.svg")
Results:
Load timeseries data from CSV, and convert any timeseries without gaps into a complete one.
We can see the number of valid data and number of total data to see that the timeseries have gaps on them.
network csv.load_timeseries("data/scioto/scioto.csv", "date", "streamflow");
# in future version csv.load_timeseries should do this while loading
node ts_complete("streamflow")
node array(ts_len("streamflow", valid=true), ts_len("streamflow"))
Results:
{
03229610 = [7032, 38078],
03229500 = [36616, 38078],
03228689 = [1119, 38078],
03228500 = [31502, 38078],
03228300 = [13237, 38078],
03228805 = [22495, 38078],
03228750 = [11747, 38078],
03227500 = [38067, 38078],
03227107 = [3639, 38078],
03226800 = [21306, 38078],
03225500 = [35155, 38078],
03223425 = [10319, 38078],
03221646 = [3744, 38078],
03219781 = [1234, 38078],
03220000 = [29678, 38078],
03221000 = [37896, 38078],
03219500 = [33695, 38078],
03217500 = [13604, 38078],
03217424 = [1285, 38078]
}
None of the timeseries are complete. We can visualize the gaps using the
node.good = (ts_len("streamflow", valid=true) / ts_len("streamflow")) > 0.75
node.visual.nodeshape = "circle";
node(good).visual.nodecolor = "darkgreen";
node(good).visual.textcolor = "darkgreen";
network svg_ts_blocks("output/scioto-ts-gap-id.svg", "{NAME}", "streamflow", 620.0, 820.0, arr_width=500.0, bgcolor="#ffffff33")
Results:
